Introduction to Probabilities

Fall 2010Dept. Electrical Engineering

National Tsing Hua University

劉奕汶

2

What is probability?

• Literally, how probable an event is to occur.– We live in a random world– Relative-frequency interpretation

• 機率／概率／或然率– This interpretation is problematic

• Involved law of large number• Not all experiments could be repeated • Not all repeating processes have convergent frequency

– Axiomatic approach

3

A bit of History• 3500 B.C., Egyptians used bones to gamble

– Since then, dice, playing cards, mahjong, etc.

• 15-16th centuries: Italy (Galilei et al.)

• 17-18th centuries: Western-central Europe– Pascal, Fermat, Laplace, Poisson, Gauss– Huygens (1629-1695) On Calculations in Games of Chance

• 19-20th : Russia– 1900: Hilbert’s 23 problems– 1933: Kolmogorov: probability theory axiomatized

4

Probability in EE/CS• Signal processing

– “Signal” = Random Process– Random because of noise and uncertainty

• Machine learning– Natural language processing– Pattern recognition

• Communication– Source coding– Channel coding– Modulation and estimation

5

Probability in Finance/Economics

• Investment / Gambling– Portfolio theory

• Advertisement / Pricing

6

7

Probability in Physics (i)

• Statistical mechanics– Equilibrium– Entropy and 2nd law of thermodynamics– Definition of temperature

Probability in Physics (ii)

• Quantum mechanics– Schrödinger’s wave function– “Measurement makes reality”

• The paradox of Schrödinger’s cat• Einstein’s famous comment

8

9

Probability in Biomedicine

• Genomics• Proteomics• Neuroscience• Ecology• Epidemiology

10

Probability and Statistics

• Law of Large Number

• Central Limit Theorem– Why Gaussian distribution is “Normal”

• Counter-example: stock market

11

Syllabus• Textbook: S. Ghahramani, Fundamentals of Probability: with

stochastic processes, 3rd Edition– Chapters 1-3: probability space – Chapters 4-5: discrete random variables– Chapter 6: Continuous random variables– Midterm exam (35%)– Chapters 7: continuous random variables II– Chapters 8: bivariate distributions– Chapter 10-11: advanced topics (Correlations, LLN, CLT, etc)– * Measure theory and axioms of probability– Final exam (35%)

• A4 double-side cheat sheet permitted for both exams – 6 homework assignments (30%)

• Office hours: Monday 5-6 pm, Rm 704B• Website: http://www.ee.nthu.edu.tw/ywliu/ee3060/

Statistics of last semester’s grades (N = 37)

• 期中考 : M = 51.4, SD = 7.9• 期末考 : M = 49.3, SD = 10.8• 總成績 : M = 78, SD = 11

– 36 passed, 1 failed. – 4 scored 90 or above (A+)

12