MIT6 041F10 Quiz01 Review - Copy

8/2/2019 MIT6 041F10 Quiz01 Review - Copy

1/27

QuizIReviewProbabilisticSystemsAnalysis6.041/6.431

Massachusetts InstituteofTechnology

October7,2010

(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 1/26


2/27

QuizInformationClosed-bookwithonedouble-sided8.5x11formulasheetallowed

Content:

Chapters1-2,

Lecture

1-7,

Recitations

1-7,

Psets1-4,Tutorials1-3

Showyourreasoningwhenpossible!



3/27

AProbabilisticModel: SampleSpace: Thesetofallpossibleoutcomesof

anexperiment. ProbabilityLaw: Anassignmentofanonnegative

numberP(E)toeacheventE.

Event A

Event B

EventsA B

Sample Space Probability Law

ExperimentP(A)

P(B)



4/27

ProbabilityAxiomsGivenasamplespace:

1. Nonnegativity: P(A)0foreacheventA2. Additivity: IfAandB aredisjointevents,then

P(AB) =P(A) +P(B)IfA1,A2, . . . , isasequenceofdisjointevents,

P(A1A2 ) =P(A1) +P(A2) + 3. NormalizationP()=1



5/27

PropertiesofProbabilityLawsGiveneventsA,B andC:

1. IfAB,thenP(A)P(B)2. P(AB) =P(A) +P(B)P(AB)3. P(AB)P(A) +P(B)4. P(ABC) =P(A) +P(AcB) +P(AcBcC)



6/27

DiscreteModels DiscreteProbabilityLaw: Ifisfinite,theneach

eventAcanbeexpressedasA={s1,s2, . . . ,sn} si

Thereforethe

probability

of

the

event

A

is

given

as

P(A) =P(s1) +P(s2) + +P(sn)

DiscreteUniformProbabilityLaw: Ifalloutcomesareequally likely,

P(A) = |A|||



7/27

ConditionalProbability GivenaneventB withP(B)>0,theconditional

probabilityofaneventA isgivenasP(A|B) =P(

PA

(B)

B)

P(A B) isavalidprobability lawon,satisfying1.|P(A|B)02. P(B) = 13.

P(

A1

|

A

2

|B

) =P(

A1|

B) +

P(

A2|

B) +

,

where{Ai}i isasetofdisjointevents P(A|B)canalsobeviewedasaprobability lawon

therestricteduniverseB.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 7/26


8/27

MultiplicationRule LetA1, . . . ,An beasetofeventssuchthat

n1P Ai >0.

=1i

Thenthejointprobabilityofallevents isn n1

Pi=1Ai =P(A1)P(A2|A1)P(A3|A1A2) P(An|i Ai)=1

P(A1)A1

A3

A1 A2 A3A

2P(A2|A1) P(A3|A1 A2)



9/27

TotalProbabilityTheoremLetA1, . . . ,An bedisjointeventsthatpartition. IfP(Ai

)>

0for

each

i,

then

for

any

event

B,

n n

P(B) = P(BAi) = P(B|Ai)P(Ai)i=1 i=1

B

A1 A2

A3

A1

A2

A3

B

B

Bc

Bc

Bc

B

A1 B

A2B

A3 B



10/27

BayesRuleGivenafinitepartitionA1, . . . ,An ofwithP(Ai)>0,then

for

each

event

B

with

P(B)

>

0

P(Ai|B) =P(B|Ai)P(Ai) =

niP=1

(B|Ai|)P

i(Ai)

Ai)P(B) P(B A)P(

B

A1 A2

A3

A1

A2

A3

B

B

Bc

Bc

Bc

B

A1 B

A2 B

A3 B



12/27

IndependenceofaSetofEvents TheeventsA1, . . . ,An arepairwise independent if

foreach i =jP(Ai Aj) =P(Ai)P(Aj)

TheeventsA1, . . . ,An areindependent ifP Ai = P(Ai) S {1,2, . . . ,n}

iS iS Pairwise independence independence,but

independence pairwise independence.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 12/26


13/27

CountingTechniques

BasicCounting

Principle:

Foranm-stageprocesswithni choicesatstage i,

#Choices=n1n2 nm Permutations: k-lengthsequencesdrawnfromn

distinct itemswithoutreplacement(order isimportant):

#Sequences=n(n1) (nk+ 1)= (nn!k)!Combinations: Setswithk elementsdrawnfromn distinct items(orderwithinsets isnot important):

#Sets= n = n!k k!(nk)!



14/27

CountingTechniques-contd Partitions: Thenumberofwaystopartitionann-element

set intordisjointsubsets,withnk elements inthekth

subset: n

= n nn1 nn1 nr 1n1,n2, . . . ,nr n1 n2 nr

n!

= n1!n2!, ,nr! where

n n!=k k!(nk)!

rni =n

i=1(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 14/26


15/27

DiscreteRandomVariablesArandomvariable isareal-valuedfunctiondefinedonthesamplespace:

X : R Thenotation{X =x}denotesanevent:

{X =x}={|X() =x} Theprobabilitymass function(PMF)forthe

randomvariable

Xassigns

aprobability

to

each

event{X =x}:

pX(x) =P({X =x}) =P {|X() =x}(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 15/26


16/27

PMFPropertiesLetX bearandomvariableandS acountablesubsetofthereal line Theaxiomsofprobabilityhold:

1. pX(x)02. P(X S) = pX(x) xS3. x pX(x) = 1

Ifg isareal-valuedfunction,thenY =g(X) isarandomvariable:

X X() g g(X())=Y() withPMF

pY(y) = PX(x)x g(x)=y|



17/27

ExpectationGiven

arandom

variable

X

with

PMF

pX(x):

E[X] = x xpX(x) GivenaderivedrandomvariableY =g(X):

E[g(X)]= g(x)pX(x) = ypY(y) =E[Y]x y

E[Xn] = xnpX(x)x

LinearityofExpectation: E[aX+b] =aE[X] +b.(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 17/26


18/27

VarianceThe

expected

value

of

aderived

random

variable

g(X

)is

E[g(X)]= g(x)pX(x)

xThevarianceofX iscalculatedas

var(X) =E[(XE[X])2] = (xE[X])2pX(x)xvar(X) =E[X2]E[X]2

var(aX+b) =a2var(X)Notethatvar(x)0



19/27

MultipleRandomVariablesLetX andY denoterandomvariablesdefinedonasamplespace.

ThejointPMFofX andY isdenotedby

pX,Y(x,y) =P {X =x} {Y =y} ThemarginalPMFsofX andY aregiven

respectivelyasp

X(

x) =

pX

,

Y(

x,y)

ypY(y) = pX,Y(x,y)

x(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 19/26

F f M l l R d V bl


20/27

FunctionsofMultipleRandomVariablesLetZ =g(X,Y)beafunctionoftworandomvariables

PMF:pZ(z) = pX,Y(x,y)

(x,y)g(x,y)=z| Expectation:

E[Z] = g(x,y)pX,Y(x,y)x,y

Linearity: Supposeg(X,Y) =aX+bY +c.E[g(X,Y)]=aE[X] +bE[Y] +c


C di i d R d V i bl


21/27

ConditionedRandomVariables

ConditioningX

on

an

event

A

with

P(A)

>0results

inthePMF:

pX|A(x) =P{X =x}|A

=P {X

P

=

(A

x)

} A

ConditioningXontheeventY =y withPY(y)>0results inthePMF: pX Y(x y) =P {X =x} {Y=y} =pX,Y(x,y)| |

P {Y =y} pY(y)(Massachusetts InstituteofTechnology) Quiz IReview October7,2010 21/26

C di i d RV d


23/27

ConditionalExpectationLetX andY berandomvariablesonasamplespace.

GivenaneventAwithP(A)>0E[X A] = xpX A(x)|

x|

IfPY(y)>0,thenE[X|{Y =y}] = xpX|Y(x|y)

x TotalExpectationTheorem: LetA1, . . . ,An bea

partitionof. IfP(Ai)>0i,thenn

E[X] = P(Ai)E[X Ai]|i=1


I d d


24/27

IndependenceLetX andY berandomvariablesdefinedonandletAbe

an

event

with

P(A)

>

0.

X isindependentofAifeitherofthefollowinghold:

pX|A(x) =pX(x)xpX,A(x) =pX(x)P(A)x

X andY are independent ifeitherofthefollowinghold:

pX|Y(x|y) =pX(x)xypX,Y(x,y) =pX(x)pY(y)xy


I d d


25/27

IndependenceIfX andY are independent,thenthefollowinghold:

Ifg andharereal-valuedfunctions,theng(X)andh(Y)are independent.

E

[XY] =E

[X]E

[Y](inverse isnottrue) var(X+Y) =var(X) +var(Y)

Given independentrandomvariablesX1, . . . ,Xn,var(X1+X2+ +Xn) =var(X1)+var(X2)+ +var(Xn)


S Di t Di t ib ti


26/27

SomeDiscreteDistributionsX pX(k) E[X] var(X)

Bernoulli 1 success0 failure p k =11p k=0 p p(1p)

Binomial NumberofsuccessesinnBernoullitrials

nk pk(1p)nkk=0,1,...,n np np(1-p)

Geometric Numberoftrialsuntilfirstsuccess

(1p)k1pk=1,2,...

1p

1pp

2

Uniform An integer inthe interval[a,b]

1ba+1 k=a,...,b0 otherwise a+b2 (ba)(ba+2)12



27/27

MIT OpenCourseWarehttp://ocw.mit.edu

6.041 / 6.431 Probabilistic Systems Analysis and Applied Probability

Fall 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
http://ocw.mit.edu/http://ocw.mit.edu/termshttp://ocw.mit.edu/termshttp://ocw.mit.edu/

Documents

MIT6 041F10 Quiz01 Review - Copy