View
225
Download
0
Category
Preview:
Citation preview
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
1/27
QuizIReviewProbabilisticSystemsAnalysis6.041/6.431
Massachusetts InstituteofTechnology
October7,2010
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 1/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
2/27
QuizInformationClosed-bookwithonedouble-sided8.5x11formulasheetallowed
Content:
Chapters1-2,
Lecture
1-7,
Recitations
1-7,
Psets1-4,Tutorials1-3
Showyourreasoningwhenpossible!
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 2/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
3/27
AProbabilisticModel: SampleSpace: Thesetofallpossibleoutcomesof
anexperiment. ProbabilityLaw: Anassignmentofanonnegative
numberP(E)toeacheventE.
Event A
Event B
EventsA B
Sample Space Probability Law
ExperimentP(A)
P(B)
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 3/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
4/27
ProbabilityAxiomsGivenasamplespace:
1. Nonnegativity: P(A)0foreacheventA2. Additivity: IfAandB aredisjointevents,then
P(AB) =P(A) +P(B)IfA1,A2, . . . , isasequenceofdisjointevents,
P(A1A2 ) =P(A1) +P(A2) + 3. NormalizationP()=1
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 4/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
5/27
PropertiesofProbabilityLawsGiveneventsA,B andC:
1. IfAB,thenP(A)P(B)2. P(AB) =P(A) +P(B)P(AB)3. P(AB)P(A) +P(B)4. P(ABC) =P(A) +P(AcB) +P(AcBcC)
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 5/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
6/27
DiscreteModels DiscreteProbabilityLaw: Ifisfinite,theneach
eventAcanbeexpressedasA={s1,s2, . . . ,sn} si
Thereforethe
probability
of
the
event
A
is
given
as
P(A) =P(s1) +P(s2) + +P(sn)
DiscreteUniformProbabilityLaw: Ifalloutcomesareequally likely,
P(A) = |A|||
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 6/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
7/27
ConditionalProbability GivenaneventB withP(B)>0,theconditional
probabilityofaneventA isgivenasP(A|B) =P(
PA
(B)
B)
P(A B) isavalidprobability lawon,satisfying1.|P(A|B)02. P(B) = 13.
P(
A1
|
A
2
|B
) =P(
A1|
B) +
P(
A2|
B) +
,
where{Ai}i isasetofdisjointevents P(A|B)canalsobeviewedasaprobability lawon
therestricteduniverseB.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 7/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
8/27
MultiplicationRule LetA1, . . . ,An beasetofeventssuchthat
n1P Ai >0.
=1i
Thenthejointprobabilityofallevents isn n1
Pi=1Ai =P(A1)P(A2|A1)P(A3|A1A2) P(An|i Ai)=1
P(A1)A1
A3
A1 A2 A3A
2P(A2|A1) P(A3|A1 A2)
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 8/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
9/27
TotalProbabilityTheoremLetA1, . . . ,An bedisjointeventsthatpartition. IfP(Ai
)>
0for
each
i,
then
for
any
event
B,
n n
P(B) = P(BAi) = P(B|Ai)P(Ai)i=1 i=1
B
A1 A2
A3
A1
A2
A3
B
B
Bc
Bc
Bc
B
A1 B
A2B
A3 B
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 9/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
10/27
BayesRuleGivenafinitepartitionA1, . . . ,An ofwithP(Ai)>0,then
for
each
event
B
with
P(B)
>
0
P(Ai|B) =P(B|Ai)P(Ai) =
niP=1
(B|Ai|)P
i(Ai)
Ai)P(B) P(B A)P(
B
A1 A2
A3
A1
A2
A3
B
B
Bc
Bc
Bc
B
A1 B
A2 B
A3 B
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 10/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
11/27
IndependenceofEvents EventsAandB areindependent ifandonly if
P(AB) =P(A)P(B)or
P(A B) =P(A) ifP(B)>0| EventsAandB areconditionally independent
givenaneventC ifandonly ifP(AB|C) =P(A|C)P(B|C)
orP(A|BC) =P(A|C) ifP(BC)>0
Independence
ConditionalIndependence.
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 11/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
12/27
IndependenceofaSetofEvents TheeventsA1, . . . ,An arepairwise independent if
foreach i =jP(Ai Aj) =P(Ai)P(Aj)
TheeventsA1, . . . ,An areindependent ifP Ai = P(Ai) S {1,2, . . . ,n}
iS iS Pairwise independence independence,but
independence pairwise independence.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 12/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
13/27
CountingTechniques
BasicCounting
Principle:
Foranm-stageprocesswithni choicesatstage i,
#Choices=n1n2 nm Permutations: k-lengthsequencesdrawnfromn
distinct itemswithoutreplacement(order isimportant):
#Sequences=n(n1) (nk+ 1)= (nn!k)!Combinations: Setswithk elementsdrawnfromn distinct items(orderwithinsets isnot important):
#Sets= n = n!k k!(nk)!
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 13/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
14/27
CountingTechniques-contd Partitions: Thenumberofwaystopartitionann-element
set intordisjointsubsets,withnk elements inthekth
subset: n
= n nn1 nn1 nr 1n1,n2, . . . ,nr n1 n2 nr
n!
= n1!n2!, ,nr! where
n n!=k k!(nk)!
rni =n
i=1(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 14/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
15/27
DiscreteRandomVariablesArandomvariable isareal-valuedfunctiondefinedonthesamplespace:
X : R Thenotation{X =x}denotesanevent:
{X =x}={|X() =x} Theprobabilitymass function(PMF)forthe
randomvariable
Xassigns
aprobability
to
each
event{X =x}:
pX(x) =P({X =x}) =P {|X() =x}(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 15/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
16/27
PMFPropertiesLetX bearandomvariableandS acountablesubsetofthereal line Theaxiomsofprobabilityhold:
1. pX(x)02. P(X S) = pX(x) xS3. x pX(x) = 1
Ifg isareal-valuedfunction,thenY =g(X) isarandomvariable:
X X() g g(X())=Y() withPMF
pY(y) = PX(x)x g(x)=y|
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 16/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
17/27
ExpectationGiven
arandom
variable
X
with
PMF
pX(x):
E[X] = x xpX(x) GivenaderivedrandomvariableY =g(X):
E[g(X)]= g(x)pX(x) = ypY(y) =E[Y]x y
E[Xn] = xnpX(x)x
LinearityofExpectation: E[aX+b] =aE[X] +b.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 17/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
18/27
VarianceThe
expected
value
of
aderived
random
variable
g(X
)is
E[g(X)]= g(x)pX(x)
xThevarianceofX iscalculatedas
var(X) =E[(XE[X])2] = (xE[X])2pX(x)xvar(X) =E[X2]E[X]2
var(aX+b) =a2var(X)Notethatvar(x)0
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 18/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
19/27
MultipleRandomVariablesLetX andY denoterandomvariablesdefinedonasamplespace.
ThejointPMFofX andY isdenotedby
pX,Y(x,y) =P {X =x} {Y =y} ThemarginalPMFsofX andY aregiven
respectivelyasp
X(
x) =
pX
,
Y(
x,y)
ypY(y) = pX,Y(x,y)
x(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 19/26
F f M l l R d V bl
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
20/27
FunctionsofMultipleRandomVariablesLetZ =g(X,Y)beafunctionoftworandomvariables
PMF:pZ(z) = pX,Y(x,y)
(x,y)g(x,y)=z| Expectation:
E[Z] = g(x,y)pX,Y(x,y)x,y
Linearity: Supposeg(X,Y) =aX+bY +c.E[g(X,Y)]=aE[X] +bE[Y] +c
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 20/26
C di i d R d V i bl
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
21/27
ConditionedRandomVariables
ConditioningX
on
an
event
A
with
P(A)
>0results
inthePMF:
pX|A(x) =P{X =x}|A
=P {X
P
=
(A
x)
} A
ConditioningXontheeventY =y withPY(y)>0results inthePMF: pX Y(x y) =P {X =x} {Y=y} =pX,Y(x,y)| |
P {Y =y} pY(y)(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 21/26
C di i d RV d
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
22/27
ConditionedRV- contd
MultiplicationRule:
pX,Y(x,y) =pX|Y(x|y)pY(y)
Total
Probability
Theorem:
n
pX(x) = P(Ai)pX Ai(x)|i=1
pX(x) = pX|Y(x|y)pY(y)y
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 22/26
C di i l E i
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
23/27
ConditionalExpectationLetX andY berandomvariablesonasamplespace.
GivenaneventAwithP(A)>0E[X A] = xpX A(x)|
x|
IfPY(y)>0,thenE[X|{Y =y}] = xpX|Y(x|y)
x TotalExpectationTheorem: LetA1, . . . ,An bea
partitionof. IfP(Ai)>0i,thenn
E[X] = P(Ai)E[X Ai]|i=1
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 23/26
I d d
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
24/27
IndependenceLetX andY berandomvariablesdefinedonandletAbe
an
event
with
P(A)
>
0.
X isindependentofAifeitherofthefollowinghold:
pX|A(x) =pX(x)xpX,A(x) =pX(x)P(A)x
X andY are independent ifeitherofthefollowinghold:
pX|Y(x|y) =pX(x)xypX,Y(x,y) =pX(x)pY(y)xy
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 24/26
I d d
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
25/27
IndependenceIfX andY are independent,thenthefollowinghold:
Ifg andharereal-valuedfunctions,theng(X)andh(Y)are independent.
E
[XY] =E
[X]E
[Y](inverse isnottrue) var(X+Y) =var(X) +var(Y)
Given independentrandomvariablesX1, . . . ,Xn,var(X1+X2+ +Xn) =var(X1)+var(X2)+ +var(Xn)
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 25/26
S Di t Di t ib ti
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
26/27
SomeDiscreteDistributionsX pX(k) E[X] var(X)
Bernoulli 1 success0 failure p k =11p k=0 p p(1p)
Binomial NumberofsuccessesinnBernoullitrials
nk pk(1p)nkk=0,1,...,n np np(1-p)
Geometric Numberoftrialsuntilfirstsuccess
(1p)k1pk=1,2,...
1p
1pp
2
Uniform An integer inthe interval[a,b]
1ba+1 k=a,...,b0 otherwise a+b2 (ba)(ba+2)12
(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 26/26
8/2/2019 MIT6 041F10 Quiz01 Review - Copy
27/27
MIT OpenCourseWarehttp://ocw.mit.edu
6.041 / 6.431 Probabilistic Systems Analysis and Applied Probability
Fall 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
http://ocw.mit.edu/http://ocw.mit.edu/termshttp://ocw.mit.edu/termshttp://ocw.mit.edu/Recommended