Review of Probability Theory. © Tallal Elshabrawy 2 Review of Probability Theory Experiments, Sample Spaces and Events Axioms of Probability Conditional

Review of Probability Theory

2© Tallal Elshabrawy

Review of Probability Theory

Experiments, Sample Spaces and Events Axioms of Probability Conditional Probability Bayes’s Rule Independence Discrete & Continuous Random Variables


Random Experiment

It is an experiment whose outcome cannot be predicted with certainty

Examples:

Tossing a Coin Rolling a Die


Random Experiment in Communications

Why is this a random experiment? We do not know

The amount of noise that will affect the transmitted bit Whether the bit will be received in error or not

Transmission of Bits across a Communication Channel

Waveform

Generator

Waveform

Detection

Channel

v rx y

0 T

0 T

+A V.

-A V.

vivi=1

vi=0

xi

0yi>0

yi<0

ri=1

ri=0

ri+

zi ]-∞, ∞[

yi


Random Experiment in Networks

Why is this a random experiment? We do not know

Whether the packet will reach the destination or not If the packet reaches the destination, how long would it take to get

there?

Transferring a Packet across a Communication Network

Packet

Packet


Sample Space

The set of all possible outcomes Tossing a coin

S = {H,T}

Rolling a die S = {1,2,3,4,5,6}

The AWGN in a Communication Channel S = ] -∞, ∞ [

Heads Tails

xi +

zi

yi


Event

An event is a subset of the sample space S

Examples Let A be the event of observing one head in a coin

flipped two times A = {HT,TH}

Let B be the event of observing two heads in a coin flipped twice B = {HH}


Axioms of Probability

Probability of an event is a measure of how often an event might occur

no. of sample pts in P( )

no. of sample pts in

AA

S

Axioms of Probability

1. 0 P 1

2. P 0,P 1

3. P P +P -P ,

A

S

A B A B A B


Example

Let Event A characterize that the outcome of rolling the die once is smaller than 3 A = {1,2} P(A) = 2/6 = 1/3

Let Event B characterize that the outcome of rolling the die once is an even number B = {2,4,6} P(B) = 3/6 = 1/2

12

4

6

P , 1/ 6

P 1/3 1/ 2 1/ 6

A B

A B

A B

S

35


Conditional Probability

Probability of event B given A has occurred

P ,P

P

A BB A

A

P ,P

P

A BA B

B

Probability of event A given B has occurred


Example

Two cards are drawn in succession without replacement from an ordinary (52 cards) deck. Find the probability that both cards are aces

Let A be the event that the first card is an ace Let B be the event that the second card is an

ace

P , =P P

4 3 1P , =

52 51 16 17

A B A B A

A B


Conditional Probability in Communications

Conditioned on v=1, what is the probability of making an error?

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

r=0Decision

Zone

0 T

0 T

+A V.

-A V.

vivi=1

vi=0

xi

0yi>0

yi<0

ri=1

ri=0

ri+

zi ]-∞, ∞[

yi

[ ] [ ][ ] [ ][ ] [ ]101

101

101

x=x+z<Pr=v=errorPr

x=y<Pr=v=errorPr

v=r=Pr=v=errorPr


Bayes’s Rule

P ,P

P

A BB A

A

P ,

PP

A BA B

B

( )( ) ( )

( )BAAB

BAP

PP=P


Theorem of Total Probability

Let B1, B2, …, Bn be a set of mutually exclusive and exhaustive events.

( ) ( ) ( )∑ n

1=i PP=P ii BBAA


Bayes’s Theorem

Let B1, B2, …, Bn be a set of mutually exclusive and exhaustive events.

( ) ( ) ( )( ) ( )∑ n

1=i PP

PP=P

ii

ii

iBBA

BBAAB


Independent Events

A and B are independent if P(B|A) = P(B)P(A|B) = P(A)P(A,B) = P(A)P(B)


Example

Let A be the event that the grades will be out on Thursday P(A)

Let B be the even that I will get A+ in Random Signals and Noise P(B)

So What is the probability that I get A+ if the grades are out on Thursday P(B|A) = P(B)


Random Variable

Characterizes the experiment in terms of real numbers

Example X is the variable for the number of heads for a coin tossed three

times X = 0,1,2,3

Discrete Random Variables The random variable can only take a finite number of values

Continuous Random Variables The random variable can take a continuum of values


Bernoulli Discrete Random Variable Represents experiments that have two possible outcomes.

These experiments are called Bernoulli Trials

Associates values {0, 1} with the two outcomes such that P[X = 0] = 1-p P[X = 1] = p

Examples Coin tossing experiment maps a ‘Heads’ to X = 1 and a ‘Tails’ to

X = 0 (or vice versa) such that p=0.5 for a fair coin

Digital communication system where X = 1 represents a bit received in error and X = 0 corresponds to a bit received correctly. In such system p represents the channel bit error probability


Binomial Discrete Random Variable

A random variable that represents the number of occurrences of ‘1’ or ‘0’ in n Bernoulli trials

The corresponding random variable X may take and values from {0, 1, 2, …, n}

The probability mass function PMF for having k ‘1’ in n Bernoulli trials isP[X = k] = nCk pk(1-p)n-k

Examples In a digital communication system, the number of bits in error in a

packet depicts a Binomial discrete random variable


Geometric Discrete Random Variable Geometric distribution describes the number of Bernoulli

trials in succession are conducted until some particular outcome is observed (lets say ‘1’)

The corresponding random variable X may take and values from {1, 2, 3, …, ∞}

The probability mass function PMF for having k Bernoulli trials in succession until an outcome of ‘1’ is observedP[X = k] = (1-p)k-1p

Examples: In a communication network, the number of transmissions until a

packet is received correctly follows a Geometric distribution

Documents

Review of Probability Theory. © Tallal Elshabrawy 2 Review of Probability Theory Experiments, Sample Spaces and Events Axioms of Probability Conditional