primer clase de estadística bayesiana

  • View
    230

  • Download
    0

Embed Size (px)

Text of primer clase de estadística bayesiana

  • 8/9/2019 primer clase de estadística bayesiana

    1/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian StatisticsLecture 1

    Motivation

    Alvaro Mauricio Montenegro D́ıaz

    August, 2011

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    2/19

    Bayesian Statistics Lecture 1 Motivation

    Table of contents

    1 Brief History2 Bayesian-frequentist controversy

    3 Motivating Examples.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    3/19

    Bayesian Statistics Lecture 1 Motivation

    Brief History

    Table of contents

    1 Brief History

    2 Bayesian-frequentist controversy

    3 Some basic Bayesian models

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    4/19

    Bayesian Statistics Lecture 1 Motivation

    Brief History

    Classical Statistics

    Classical Age

    1 Probability is old. Several hundreds of years ago, most mathematicianssupported by rich noblemen advise them on how to maximize theirwinnings in games of chance.

    2 Statistics is young. Linear regression rst appeared in the work of Francis Galton in the late 1800s. Karl Pearson added correlation andgoodness-of-t measures around the turn of the last century.

    3 Statistics blossomed . Between 1920 and 1930 R.A. Fisher developed thenotion of likelihood of general estimation. Jerry Neyman and EgonPearson developed the basis for classical hypothesis testing.

    4 The war was an engine. A urry of research activity was energized bythe World War II.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    5/19

    Bayesian Statistics Lecture 1 Motivation

    Brief History

    Bayesian Statistics

    Bayesian Age

    1 Bayesian methods are much older. The original paper dating in 1763by the Rev. Thomas Bayes, a minister and amateur mathematician.

    2 Laplace and Gauss and others were interest ed in the 19th century.3 Bayesian approach was ignored or actively opposed in the early of 20th

    century.

    4 Several non-statisticians as Jeffreys (a physicist) and Bolye (aneconometrician), continued to lobby on behalf of the Bayesian ideas,

    (which they referred as inverse probability .5 Bayesian statistics is here. Beginning around 1950 statisticians as

    Savage, Finetti, Lindley, and many others began advocating Bayesianmethods as remedies for certain deciencies in the classical approach.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    6/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Table of contents

    1 Brief History

    2 Bayesian-frequentist controversy

    3 Some basic Bayesian models

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    7/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Example 1. Classical point of view

    Example 1. Frequentist interval estimation

    Suppose X i iid

    ∼N (θ, σ 2), i = 1 , · · · , n. We desire a 95% interval estimate forthe population mean θ.

    Provided n sufficiently large (say bigger than 30), a classical approach woulduse the condence interval .

    δ (x ) = x̄ ±1.96s / √ n

    Interpretation. Before any data are collected the probability that the interval

    contains the true value is 0.95, for any value of θ and σ2

    .

    After collecting the data and computing δ (x ), the interval contains the true θor it does not; its coverage probability is not 0.95, but either 0 or 1.

    It is not correct to say that the true θ has a 95% chance of falling in δ (x ).

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    B i St ti ti L t 1 M ti ti

    http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    8/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Example1. Bayesian point of view

    Bayesian interval estimation

    Bayesian condence intervals are called credibility sets . They are free of theawkward frequentist interpretation.

    In the Bayesian, approach, the data are rst observed. Then, conditional on theobserved data the posterior distribution p (θ|x ) is obtained.A 95% credibility interval can be obtained by taking the 2 .5 and 97.5percentiles of the posterior distribution.

    Interpretation. The actual value of θ has a 95% chance of falling incondence interval. However, there is a price to pay. It is necessary to specify a(possible vague) prior distribution for θ.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://goforward/http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    9/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Example 2. Test of hypothesis

    Frequentist Test of hypothesis

    Suggested by Lindley and Phillips(1976).

    Suppose in 12 independent toses of a coin, we observe 9 heads and 3 tails.

    We wish to tests the hypothesis H 0 : θ = 1 / 2 versus the alternative H a θ > 1/ 2,where θ is the true probability of heads.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    10/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Example 2. Test of hypothesis

    Possible modelsGiven only this information, two choices for the sampling distribution emerge:

    1 Binomial: The number n = 12 tosses was xed beforehand, and therandom quantity X was the number of heads observed in the n toses.

    Then the likelihood function is given by

    L1(θ) =nx

    θx (1 −θ)1− x =

    129

    θ9(1 −θ)3

    2 Negative binomial: Data collection involve ipping the coin until the third

    tail appeared. Here X is the number of heads required to complete theexperiment, so that X ∼NegBin (r = 3 , θ), with likelihood

    L2(θ) =r + x −1

    x θx (1 −θ)1

    − x =119

    θ9(1 −θ)3

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    11/19

    Bayesian Statistics Lecture 1 Motivation

    Bayesian-frequentist controversy

    Example 2. Test of hypothesis

    ComputationsThe p -value to the rejection region Reject H 0 if X ≥c are computed as: Usingthe binomial likelihood we have

    α 1 = P θ = 12

    (X

    ≥9) =

    12

    j =9

    j

    j −12θ j (1

    −θ)12− j = 0 .075,

    Using the negative binomial likelihood we have

    α 2 = P θ = 12 (X ≥9) =∞

    j =9

    2 + j j

    θ j (1 −θ)3 = 0 .0325.

    Uppss!!

    Using the usual type I error level α = 0 .05, the two model assumptions lead totwo different decisions. The problem is that there is not sufficient informationin the problem setting to help us in the selection of the correct model.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    12/19

    Bayesian Statistics Lecture 1 Motivation

    Some basic Bayesian models

    Table of contents

    1 Brief History

    2 Bayesian-frequentist controversy

    3 Some basic Bayesian models

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    13/19

    y

    Some basic Bayesian models

    Bayes theorem

    The most basic Bayesian model has three stages:

    1 Likelihood specication Y ∼θf (y |θ).2 Prior specication θ∼p (θ), where Y or θ can be vectors. p (θ) can becompleted specied (the simplest case) or not.

    3 Computation of the posterior distribution

    The posterior distribution of θ is given by

    π(θ|y ) = f (y

    |θ)p (θ)

    m(y ) ,where

    m(y ) = f (y |θ)p (θ)d θis the marginal distribution of Y . This is the Bayes’ Theorem.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    14/19

    y

    Some basic Bayesian models

    Distributions in the Bayesian perspective.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    15/19

    Some basic Bayesian models

    A Gaussian/Gaussian (normal/normal) model

    Y ∼N (θ, σ 2), σ2 known.In this case both the prior and the likelihood (the observational model) areGaussian distributions, namely,

    θ

    N (µ, τ 2)

    Y |θ∼N (θ, σ 2),thus, the marginal distribution of Y is N (µ, σ 2 + τ 2), and the posteriordistribution of θ is Gaussian with mean and variance given by

    E [θ|Y ] = B µ + (1 −B )Y Var [θ|Y ] = (1 −B )σ2,

    whereB = σ2/ (σ2 + τ 2).

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    16/19

    Some basic Bayesian models

    A beta/binomial model

    Y ∼Bin(n, θ)This is a model which could be used in the problem of test of hypothesispresented above. After observe Y , the number of success in n independenttrials, the sampling distribution (observational model) is given by

    P (Y = y |θ) = f (y |θ) = ny θy (1 −θ)1− y

    To obtain a closed form for the marginal distribution, we use the Beta (a , b )prior distribution. That is

    P (θ) = 1

    B (a , b ) θa − 1

    (1 −θ)b − 1

    ,where B (a , b ) is the beta function.

    For convenience we use the reparametrization ( µ, M ) where µ = a/ (a + b ), isthe prior mean and M = a + b , a measure of the prior precision.

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    17/19

    Some basic Bayesian models

    A beta/binomial model II

    Y ∼Bin(n, θ)The prior variance given by µ(1 −µ)/ (M + 1) is a decreasing function of of M .The marginal distribution of Y is called beta-binomial , and can be shown that

    E

    Y

    n = µ

    Var Y n

    = µ(1 −µ)

    n1 +

    n −1M + 1

    .

    The posterior distribution of θ is again a Beta distribution with

    θ̂ = E [θ|Y ] = M M + n

    µ + n

    M + nY n

    Var [θ|Y ] = θ̂(1 −θ̂)/ (M + n) .

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/

  • 8/9/2019 primer clase de estadística bayesiana

    18/19

    Some basic Bayesian models

    Poisson distribution

    Y i ∼Po (λ ), i = 1 . . . , nIt usual to assume a gamma distribution for the prior of λ, say G (α, β ). Forthis example, we assume the parameterization

    p (x ; α, β ) = x α − 1e − β x

    Γ(α )β − α ,

    so, E [X ] = α/β and Var [X ] = α/β 2. The posterior distribution is

    p (λ |y) ∝ n

    i =1

    e − λ λ y i λ α − 1e − βλ = e − (n+ β )λ λ i y i + α − 1

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    Bayesian Statistics Lecture 1 Motivation

    http://find/http://goback/

  • 8/9/2019 primer clase de estadística bayesiana

    19/19

    Some basic Bayesian models

    Example 1. Bayesian estimate of λ

    Y i ∼Po (λ ), i = 1 . . . , nThen,

    p (λ |y) = G n

    i =1

    y i + α, n + β

    Hence, the EAP estimate of λ is

    λ =ni =1 Y i + αn + β =

    nn + β

    ni =1 y i n +

    β n + β

    αβ .

    Alvaro Mauricio Montenegro D́ıaz Bayesian Statistics Lecture 1 Motivation

    http://find/