Probability and Random Processes€¦ · Asst. Prof. Dr. Prapun Suksompong [email protected] 8 Discrete Random Variable 1 Probability and Random Processes ECS 315 Office Hours:

Asst. Prof. Dr. Prapun [email protected]

8 Discrete Random Variable

1

Probability and Random ProcessesECS 315

Office Hours: BKD, 6th floor of Sirindhralai building

Wednesday 14:00-15:30Friday 14:00-15:30

Example 8.15: pdf and probabilities

2

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x

probability mass function (pmf)

1/2

1 2 3 4x

1/41/8

stem plot:

2 ?

1 ?

P X

P X

Consider a random variable (RV) X.

=

Example 8.15: pdf and probabilities

3

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1/2

1 2 3 4x

1/41/8

stem plot:

12 24

1 2 3 41 1 1 14 8 8 2

X

X X X

P X p

P X p p p


Example: pdf and its interpretation

4

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1/2

1 2 3 4x

1/41/8

stem plot:


?


5

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4x

probability “mass”



6

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4x

probability “mass” of size 1/4


Example: Support of a RV

7

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4x What about the support of

this RV X?



8

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4

The set {1,2,3,4} is a support of X.



9

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4

The set {1,2,2.5,3,4,5} is also a support of this RV X.

2.5 5



10

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4

The set {1,2,4} is not a support of this RV X.



11

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1 2 3 4

The set {1,2,3,4} is the “minimal” support of X.

For discrete RV, we take the collection of x values at which to be our “default” support.



12

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x


1/2

1 2 3 4x

1/41/8

stem plot: The “default” support for this RV is the set SX = {1,2,3,4}.


=


9 Expectation and Variance

1




Expectation and Variance

2

The expectation (or mean or expected value) of a discrete random variable X is given by

The expected value of a function g of a RV X is given by

The variance of a RV X is given by

The standard deviation of a RV X is given by

Xx

X xp x

Xx

g X g x p x

2 22Var X X X X X

VarX X

Example

3

1,2,3,4X

1 , 1,21 , 2,41 , 3,480, otherwise

X

x

xp x

x

Approximately 50% are number ‘1’s

2 1 1 2 1 4 1 1 1 11 1 4 1 1 2 4 2 2 13 1 1 2 3 2 4 1 2 42 1 1 2 1 1 3 3 1 11 3 4 1 4 1 1 2 4 14 1 4 1 2 2 1 4 2 14 1 1 1 1 2 1 4 2 42 1 1 1 2 1 2 1 3 22 1 1 1 1 1 1 2 3 22 1 1 2 1 4 2 1 2 1

[GenRV_Discrete_finite_support.m]

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

Rel. freq. from sim.pmf pX(x)

Example

4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

x

Rel. freq. from sim.pmf pX(x)

n = 100 n = 106

[GenRV_Discrete_finite_support_average.m]

average 1.8739average 1.7400

15 1.8758

X As → ,the average will converge to

Christiaan Huygens (1629-1695)

5

Dutch astronomer In 1657, wrote the first treatise (textbook) on

probability theory: “On Reasoning in Games of Chance” Van Rekeningh in Spelen van Geluck De ratiociniis in ludo aleae http://www.york.ac.uk/depts/maths/histstat/h

uygens.htm

Interest sparked partly by the work of Pascal and Fermat.

Originally introduced the concept of expected value.

http

://e

n.w

ikip

edia

.org

/wik

i/C

hrist

iaan

_Huy

gens

#m

edia

view

er/F

ile:C

hrist

iaan

_Huy

gens

.jpg

http

://b

c.ub

.leid

enun

iv.n

l/bc

/ten

toon

stel

ling/

Huy

gens

/Im

ages

/htm

l/03

_2.h

tml

Christiaan Huygens (1629-1695)

6

Also famous for the “Huygens’ Principle”

All points on a wavefront serve as point sources of spherical secondary wavelets. After a time t, the new position of the wavefront will be that of a surface tangent to these secondary wavelets.

Calculations of Expected Values

7

Poisson()

Binomial(n,p)

X

X np

Government Lottery (สลากกนิแบ่งรฐับาล)

8

ตั้งแต่งวดวันที่ 1 ก.ย. 2560 เป็นต้นไป สาํนักงานสลากกนิแบ่งรัฐบาล ปรับปรุงรูปแบบสลากฯใหม่ จากเดิมฉบบัคู่ 80 บาท (ฉบบัละ 40 บาท) เป็นรูปแบบใบเดียวฉบบัละ 80 บาท

เงินรางวัลยังเท่าเดิม เปลี่ยนแค่ขนาดที่กระชับเลก็ลงเท่านั้น

หวย (Huay)[http://www.glo.or.th]

http

s://

ww

w.p

ptvh

d36.

com

/new

s/ประเดน็

ร้อน/

6258

3


9

“คอหวย” ปลง นาํหวยมาทาํ “วอลเปเปอร์บา้น”

10

นายพิภพ ปานแย้ม รองนายกเทศมนตรีเทศบาลเมืองคลองหลวง นาํลอ็ตเตอรี่จาํนวนมากติดฝาผนังบ้าน

ของตนที่ อ.คลองหลวง จ.ปทุมธานี

[http://money.sanook.com/204469/][http://www.nationtv.tv/main/content/lifestyle/378418937/]


11

Expected Profit 32

Before Sep 1, 2015

12

Expected Profit 16

Assumption

Sep 2015 to Sep 2017

13

http://www.nationtv.tv/main/content/social/378469538/

Expected Profit 16

Can only press once

14

“Similar” Example

15

ฉันเหมือนคนที่มีเสื้อใส่ แต่ยังไม่พอใจกบัที่ฉันมี

เพราะแค่เพียงได้เจอเสื้อใหม่ อย่างที่ฉันพอใจอยากจะรีบควา้

ใครกเ็ตอืนว่าไม่คุ้มกบัสิ่งที่ฉันทิ้งไป เพื่อสิ่งที่ฉันยังไม่ได้มา

ใครกเ็ตอืนอย่ารีบร้อนจะเสีย่งทาํไมนะ แต่มันกย็ังถลาํไปหมดทั้งใจ

ฉันอตุส่าห์ไม่รกัเขาเพือ่ที่จะรกัเธอ

ยอมทุม่เทหมดแล้วให้เธอ แล้วเธอกท็ิ้งไป

เสียเคา้แลว้ยงัตอ้งเสียใจเธอสอนฉันให้เข้าใจ

การลงทุนเสีย่งเหลือเกิน

From the SET’s website,…

16

[Stock Exchange of Thailand]

[www.set.or.th]


10 Continuous Random Variables

1




Ex. rand function

2

Generate an array of uniformly distributed pseudorandom numbers. The pseudorandom values are drawn

from the standard uniform distribution on the open interval (0,1).

rand returns a scalar. rand(m,n) or rand([m,n])

returns an m-by-n matrix. rand(n) returns an n-by-n matrix

Ex. Muscle Activity

3

Look at electrical activity of skeletal muscle by recording a human electromyogram (EMG).

[http://www.adinstruments.com/solutions/education/ltexp/electromyography-emg-german]

Three Important Continuous RVs

4

0 50 100-2

0

2

4

-4 -2 0 2 4 60

5

10

0 50 100-5

0

5

-4 -2 0 2 4 60

5

10

15

0 50 1000

2

4

6

-4 -2 0 2 4 60

10

20

30

Mean = 1Std = 1N = 100

[IntroThreeContinuousRV.m]


5

Mean = 1Std = 1N = 1,000

0 500 1000-2

0

2

4

-4 -2 0 2 4 60

50

100

0 500 1000-5

0

5

-4 -2 0 2 4 60

50

100

150

0 500 10000

5

10

-4 -2 0 2 4 60

200

400


6

0 5000 10000-2

0

2

4

-4 -2 0 2 4 60

200

400

600

0 5000 10000-5

0

5

-4 -2 0 2 4 60

1000

2000

0 5000 100000

5

10

15

-4 -2 0 2 4 60

2000

4000

6000

Mean = 1Std = 1N = 10,000

Review: P[some condition(s) on X]

7

For discrete random variable,

Sum over all the x values that satisfy the condition(s)

somecondition s on

Discrete RV

P[some condition(s) on X]

8


For continuous random variable,

Sum over all the x values that satisfy the condition(s)

somecondition s on

Discrete RV

Integrate over all the x values that satisfy the condition(s)

somecondition s on

Continuous RV


probability density function (pdf)

pmf → pdf

→

Support of a RV

9

In general, the support of a RV is any set such that

In this class, we try to find the smallest (minimal) set that works as a support.


For continuous random variable,

World Map of Population Density

10 [http://i.imgur.com/gBYMfWO.jpg]

Thailand’s Population Density

11

https://www.researchgate.net/publication/260378246_Climate-Related_Hazards_A_Method_for_Global_Assessment_of_Urban_and_Rural_Population_Exposure_to_Cyclones_Droughts_and_Floods/figures?lo=1


12


13 http://globe.chromeexperiments.com/

“Density”

14

Density = quantity per unit of measure.

Population Density = number of people per unit area Location with high density value means there are a lot of people

around that location. Given a region, we integrate the density over that region to get

the number of people residing in that region.

Probability Density = probability per unit “length”. Given an interval, we integrate the density over that interval to

get the probability that the RV will be in that interval.

pdf and cdf for continuous RV

15

“ ”

Sections 10.1-10.2

16

Continuous RV

0 pdf ∶

Two characterizing properties: 0

1

: 0 somecondition s on

allthe valuesthatsatisfythecondition s

cdf is a continuous function.

Discrete RV

pmf: ≡ Two characterizing properties:

0 ∑ 1

: 0 somecondition s on

allthe valuesthatsatisfythecondition s

cdf is a staircase function with jumps whose size at gives .

1/2

3/47/81

1 2 3 4

1

Chapter 9 vs. Section 10.3

17

Continuous RVDiscrete RV

2 2

X

X

X

X xf x dx

g X g x f x dx

X x f x dx

Xx

X xp x

Xx

g X g x p x

2 2 Xx

X x p x

2 22Var

VarX

X X X X X

X

Johann Carl Friedrich Gauss

18

1777 –1855

A German mathematician

German 10-Deutsche Mark Banknote (1993; discontinued)

Ex. Muscle Activity

19

Look at electrical activity of skeletal muscle by recording a human electromyogram (EMG).

[http://www.adinstruments.com/solutions/education/ltexp/electromyography-emg-german]

Ex. Measuring the speed of light

20

100 measurements of the speed of light (1,000 km/second), conducted by Albert Abraham Michelson in 1879.

Expected Value and Variance

21

>> syms x>> syms m real>> syms sigma positive

>> int(1/(sqrt(sym(2)*pi)*sigma)*exp(-(x-m)^2/(2*sigma^2)),x,-inf,inf)ans =1>> EX = int(x/(sqrt(sym(2)*pi)*sigma)*exp(-(x-m)^2/(2*sigma^2)),x,-inf,inf)EX =m>> EX2 = int(x^2/(sqrt(sym(2)*pi)*sigma)*exp(-(x-m)^2/(2*sigma^2)),x,-inf,inf)EX2 =-(2^(1/2)*(limit(- x*sigma^2*exp((x*m)/sigma^2 - m^2/(2*sigma^2) - x^2/(2*sigma^2)) - m*sigma^2*exp((x*m)/sigma^2 - m^2/(2*sigma^2) - x^2/(2*sigma^2)) -(2^(1/2)*pi^(1/2)*sigma*erfi((2^(1/2)*(x - m)*i)/(2*sigma))*(m^2 + sigma^2)*i)/2, x == -Inf) - limit(- x*sigma^2*exp((x*m)/sigma^2 - m^2/(2*sigma^2) -x^2/(2*sigma^2)) - m*sigma^2*exp((x*m)/sigma^2 - m^2/(2*sigma^2) - x^2/(2*sigma^2)) - (2^(1/2)*pi^(1/2)*sigma*erfi((2^(1/2)*(x - m)*i)/(2*sigma))*(m^2 + sigma^2)*i)/2, x == Inf)))/(2*pi^(1/2)*sigma)

>> EX2 = simplify(EX2)EX2 =m^2 + sigma^2>> VarX = EX2 - (EX)^2VarX =sigma^2

“Proof ” by MATLAB’s symbolic calculation

Gaussian Random Variable

22 [Wikipedia.org]

mmmmm m m m m


23

Standard scores 1

[Wikipedia.org]

mmmmm m m m m


24

10

[Wikipedia.org]

mmmmm m m m m

SIIT Grading Scheme (Option 3)

25 [Wikipedia.org]

F D D+ C C+ B B+ A

7% 9% 15%19%19% 15% 9% 7%

Class GPA 2.25

mmmmm m m m m

From the News

26

4 July 2012

They claimed that by combining two data sets, they had attained a confidence level just at the "five-sigma" point -about a one-in-3.5 million chancethat the signal they see would appear if there were no Higgs particle.

However, a full combination of the CMS data brings that number just back to 4.9 sigma - a one-in-two million chance.

Particle physics has an accepted definition for a discovery: a “five-sigma” (or five standard-deviation) level of certaintyThe number of sigmas measures how unlikely it is to get a certain experimental result as a matter of chance rather than due to a real effect

6

6

1 3.5 101

15

4.92 10

1

Six Sigma

27

Six Sigma

28

If you manufacture something that has a normal distribution and get an observation outside six of , you have either seen something extremely unlikely or there is something wrong with your manufacturing process. You’d better look it over.

This approach is an example of statistical quality control, which has been used extensively and saved companies a lot of money in the last couple of decades.

The term Six Sigma, a registered trademark of Motorola, has evolved to denote a methodology to monitor, control, and improve products and processes.

There are Six Sigma societies, institutes, and conferences. Whatever Six Sigma has grown into, it all started with

considerations regarding the normal distribution.

[Olofsson, 2006, p. 168]

Six Sigma

29 [Bass, 2007, p. 20]


11 Multiple Random Variables

1




Chapter 6 vs. Chapter 11

11

P A B , ( , ) ,X Yp x y P X x Y y

P A BP B

P B A

P A

P B

B

A P

,|

|

( , )|

( | ) ( )

X YX Y

Y

Y X X

Y

p x yp x y

p yp y x p x

p y

A X x

B Y y

Conditional pmf

Joint pmf

P A B P A P B Events A and B are independent: RVs X and Y are independent:

, ( , ) ( )X Y X Yp x y p x p y for any x and y

Example: small joint pmf matrix

12

close all; clear all;x = [1 3];y = [2 4];PXY = [3/20 5/20; 5/20 7/20];

[X Y] = meshgrid(x,y); X = X.'; Y = Y.';

stem3(X,Y,PXY,'filled')xlim([0,4])ylim([0,5])xlabel('x')ylabel('y')

01

23

4

01

23

450

0.1

0.2

0.3

0.4

xy

Ex. 11.7


13




01

23

45

01

23

40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

xy

(More)

Ex. 11.26

Example: large joint pmf matrix

14

close all; clear all;n = 10; p = 3/5;x = 0:n;y = 0:n;

pX = binopdf(x,n,p);pY = binopdf(y,n,p);

PXY = pX.'*pY;


stem3(X,Y,PXY, 'filled')%mesh(X,Y,PXY)%surf(X,Y,PXY)

xlabel('x')ylabel('y')

02

46

810

0

5

100

0.01

0.02

0.03

0.04

0.05

0.06

0.07

xy

Evaluation of Probability

15

Consider two random variables X and Y.

Suppose their joint pmf matrix is

Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

,X YP


16



Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

3 4 5 6 75 6 7 8 96 7 8 9 108 9 10 11 12

2 3 4 5 6xy

1346

Step 1: Find the pairs (x,y) that satisfy the condition“x+y < 7”

One way to do this is to first construct the matrix of x+y.

,X YP

x y


17



Find

0.1 0.1 0 0 0 0.1 0 0 0.1 00 0.1 0.2 0 00 0 0 0 0.3

2 3 4 5 6xy

1346

3 4 5 6 75 6 7 8 96 7 8 9 108 9 10 11 12

2 3 4 5 6xy

1346

Step 2: Add the corresponding probabilities from the joint pmf (matrix)

,X YP

x y7 0.1 0.1 0.1

0.3


18




01

23

45

01

23

40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

xy

, 115

415

215

815

x3

4

y 1 3Ex. 11.29

Joint pmf matrix for independent RVs

19

>> pX = [1/3 2/3]pX =

0.3333 0.6667>> pY = [1/5 4/5]pY =

0.2000 0.8000>> sym(pX'*pY)ans =[ 1/15, 4/15][ 2/15, 8/15]>>

Command Window

Joint pmf for two i.i.d. RVs

20

close all; clear all;n = 10; p = 3/5;x = 0:n;y = 0:n;

pX = binopdf(x,n,p);pY = binopdf(y,n,p);

PXY = pX.'*pY;


%stem3(X,Y,PXY, 'filled')mesh(X,Y,PXY)%surf(X,Y,PXY)

xlabel('x')ylabel('y')

02

46

810

0

5

100

0.01

0.02

0.03

0.04

0.05

0.06

0.07

xy

i.i.d. 3, 10,5

X Y

Note how the pmfsare multiplied because of the independence.

Correlation

21

Correlation measures a specific kind of dependency. Dependence = statistical relationship between two random

variables (or two sets of data). Correlation measures “linear” relationship between two random

variables.

Correlation and causality. “Correlation does not imply causation” Correlation cannot be used to infer a causal relationship

between the variables.

Two “Unrelated” Events

22Correlation: 0.666004 http://www.tylervigen.com/

Two “Unrelated” Events

23

85 90 95 100 105 110 115 120 1251

1.5

2

2.5

3

3.5

4

Number people who drowned by falling into a swimming-pool

Num

ber o

f film

s N

icol

as C

age

appe

ared

in

Correlation: 0.666004 http://www.tylervigen.com/

Spurious Correlation

24http://www.tylervigen.com/Correlation: 0.992082


25

1.8 2 2.2 2.4 2.6 2.8 3

x 104

5000

5500

6000

6500

7000

7500

8000

8500

9000

US spending on science, space, and technology [Millions of todays dollars]

Sui

cide

s by

han

ging

, stra

ngul

atio

n an

d su

ffoca

tion

[Dea

ths]

Correlation: 0.992082 http://www.tylervigen.com/


26http://www.tylervigen.com/


27http://www.tylervigen.com/


28

(gross number of murders)

[http://www.geek.com/microsoft/does-internet-explorers-falling-market-share-mirror-the-drop-in-us-homicides-1537095/]


29

ECS315 - 8 - Discrete RVECS315 - 9 - Expectation and VarianceECS315 - 10 - Continuous Random Variables - u1ECS315 - 11 - Multiple Random Variables - u1

Documents

Probability and Random Processes€¦ · Asst. Prof. Dr. Prapun Suksompong [email protected] 8 Discrete Random Variable 1 Probability and Random Processes ECS 315 Office Hours: