CS6825: ProbabilityCS6825: Probability

An IntroductionAn Introduction

DefinitionsDefinitions

An An experimentexperiment is the process of observing is the process of observing a phenomenon with multiple possible a phenomenon with multiple possible outcomesoutcomes

The The sample spacesample space of an experiment is of an experiment is allall possible outcomespossible outcomes• The sample space may be The sample space may be discretediscrete or or

continuouscontinuous An An eventevent is a set (collection) of one or is a set (collection) of one or

more outcomes in the sample spacemore outcomes in the sample space

Presenting dataPresenting data Pie and bar chartsPie and bar charts

Frequency diagramFrequency diagram

Scatter diagramScatter diagram

Taken fromTaken from““Multidimensional Multidimensional Representation of Concepts Representation of Concepts as Cognitive Engrams in the as Cognitive Engrams in the Human Brain “Human Brain “

Body Pixels

Background

Face Pixels

Other

Pie chartPie chart A Pie Chart is useful for presenting nominal A Pie Chart is useful for presenting nominal

data.data.• For each category we calculate the relative For each category we calculate the relative

frequency of its occurrence.frequency of its occurrence.• Then we take a circle and divide (slice) it Then we take a circle and divide (slice) it

proportionally to the relative frequency and portions proportionally to the relative frequency and portions of the circle are allocated for the different groupsof the circle are allocated for the different groups

Body Pixels

Background

Face Pixels

Other

ExampleExample A manager of Athletics store has to decide, A manager of Athletics store has to decide,

which brands to keep in the new season. 200 which brands to keep in the new season. 200 runners were asked to indicate their favorite runners were asked to indicate their favorite type of running shoe.type of running shoe.

Type of shoe # of runners % of total

Nike 92 46.0

Adidas 49 24.5

Reebok 37 18.5

Asics 13 6.5

Other 9 4.5

Example: Pie chart for running Example: Pie chart for running shoesshoes

46%

24.50%

18.50%6.50%

4.50% Nike

Adidas

ReebokAsics

Other

We can express this in words by saying the probability of Nike is 46% and the probability of Reebok is 18.5%

The The probabilityprobability of an of an event is the proportion of event is the proportion of

times the event is times the event is expected to occur in expected to occur in

repeated experimentsrepeated experiments

Probability PropertiesProbability Properties The probability of an event, say event A, is denoted P(A).The probability of an event, say event A, is denoted P(A). All probabilities are between 0 and 1.All probabilities are between 0 and 1.

(i.e. 0 < P(A) < 1)(i.e. 0 < P(A) < 1) Sample Space – set of all possible events. In previous Sample Space – set of all possible events. In previous

example Set = {Nike, Adidas, Reebok, Asic, Other} example Set = {Nike, Adidas, Reebok, Asic, Other}

The sum of the probabilities of all possible outcomes The sum of the probabilities of all possible outcomes (sample space) must be 1.(sample space) must be 1.

NOTE: it is possible to us a scale of 100% instead of 1 but, NOTE: it is possible to us a scale of 100% instead of 1 but, in statistics we use the scale of 1.in statistics we use the scale of 1.

What are the ProbabilitiesWhat are the Probabilities

46%

24.50%

18.50%6.50%

4.50% Nike

Adidas

ReebokAsics

Other

P(Nike) = 46/100 = .46P(Adidas) = 24.5/100 = .245P(Reebok) = 18.5/100 = .185P(Asics) = 6.5/100 = .065P(Other) = 4.5/100 = .045

Assigning ProbabilitiesAssigning Probabilities

Guess based on prior knowledge Guess based on prior knowledge alonealone

Guess based on knowledge of Guess based on knowledge of probability distribution (to be probability distribution (to be discussed later)discussed later)

Assume equally likely outcomesAssume equally likely outcomes Use relative frequenciesUse relative frequencies

Guess based on prior Guess based on prior knowledge aloneknowledge alone

Event B = {It rains Tomorrow}Event B = {It rains Tomorrow}

Weth R. Guy says “There is a Weth R. Guy says “There is a 30% chance of rain 30% chance of rain

tomorrow.”tomorrow.”

P(B) = .30P(B) = .30

a priori Knowledge

What do to when no prior What do to when no prior knowledge and no training knowledge and no training

data …..Assume equally likely data …..Assume equally likely outcomesoutcomes

Use Relative FrequenciesUse Relative Frequencies

Gather Gather training datatraining data to estimate to estimate probabilities….Flip a coin how probabilities….Flip a coin how many times get head versus many times get head versus tails.tails.

i.e. Take a bunch of images of the i.e. Take a bunch of images of the data and see what it means to data and see what it means to be yellow for a banana?be yellow for a banana?

Additional material….Additional material….

Beyond the very beginningBeyond the very beginning

Complement*Complement*

The The complementcomplement of an event A, of an event A, denoted by A, is the set of denoted by A, is the set of outcomes that are not in Aoutcomes that are not in A

A A meansmeans A A does not occurdoes not occur

* Some texts use Ac to denote the complement of A

Law of ComplementLaw of Complement

P(A) = P(A) = Probability of anyProbability of any event except A occurringevent except A occurring

= P(all Events) - P(A)= P(all Events) - P(A) = = Sum(all events i P(i))Sum(all events i P(i)) – P(A)– P(A) = 1 – P(A)= 1 – P(A)

UnionUnion

The The unionunion of two events A and B, of two events A and B, denoted by A denoted by A UU B, is the set of B, is the set of outcomes that are in A, or B, or outcomes that are in A, or B, or

bothboth

IfIf A A UU B B occurs, then either occurs, then either AA or or BB or or both occurboth occur

Intersection

The intersection of two events A and B, denoted by AB, is the set of outcomes that

are in both A and B.

If AB occurs, then both A and B occur

Addition LawAddition Law

P(A U B) = P(A) + P(B) - P(AB)P(A U B) = P(A) + P(B) - P(AB)

(The probability of the union of A (The probability of the union of A and B is the probability of A plus and B is the probability of A plus

the probability of B minus the the probability of B minus the probability of the intersection of A probability of the intersection of A

and B) and B)

Mutually Exclusive Mutually Exclusive Events*Events*

Two events are Two events are mutually mutually exclusiveexclusive if their if their

intersection is empty.intersection is empty.

Two events, A and B, are Two events, A and B, are mutually exclusive if and mutually exclusive if and

only if P(AB) = 0only if P(AB) = 0

Addition Law for Addition Law for Mutually Exclusive Mutually Exclusive

EventsEventsP(A U B) = P(A) + P(B)P(A U B) = P(A) + P(B)

Conditional ProbabilityConditional Probability

The probability of event A occurring, The probability of event A occurring, given that event B has occurred, is given that event B has occurred, is called the called the conditional probability of conditional probability of event A given event Bevent A given event B, denoted , denoted P(A|P(A|

B)B)

Conditional ProbabilityConditional Probability

P(AB)P(AB)P(A|B) = --------P(A|B) = -------- P(B)P(B)

oror

P(AB) = P(B)P(A|B)P(AB) = P(B)P(A|B)

IndependenceIndependence

IfIf

P(A|B) = P(A)P(A|B) = P(A)

oror

P(B|A) = P(B)P(B|A) = P(B)

oror

P(AB) = P(A)P(B)P(AB) = P(A)P(B)

then A and B are then A and B are independentindependent..

IndependenceIndependenceTwo events A and B are Two events A and B are

independent independent ifif

P(A|B) = P(A)P(A|B) = P(A)

oror

P(B|A) = P(B)P(B|A) = P(B)

oror

P(AB) = P(A)P(B)P(AB) = P(A)P(B)NOTE: this is an assumption sometimes researchers make about theirsystems when they have no a priori knowledge to tell them differently.They do it as it makes math simpler. BE CAREFUL, it may be a WRONGAssumption!!!i.e. in motion tracking – person 1 leaves means nothing about person 2leaving. They are independent….. But, is this true in practice?