Embed Size (px)
Citation preview
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist methods
Geir Storvik
STK4020 24 November 2008
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Outline
1 Background
2 “Objections” by Gelman
3 Pragmatic approaches
4 Other issues
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Outline
1 Background
2 “Objections” by Gelman
3 Pragmatic approaches
4 Other issues
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Outline
1 Background
2 “Objections” by Gelman
3 Pragmatic approaches
4 Other issues
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Outline
1 Background
2 “Objections” by Gelman
3 Pragmatic approaches
4 Other issues
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist
Dempster:
A person cannot be Bayesian or frequentist. Rather aparticular analysis can be Bayesian or frequentist.
When should we use Bayesian methods?Depending on contextDepending on available toolsDepending on knowledge of tools
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist
Dempster:
A person cannot be Bayesian or frequentist. Rather aparticular analysis can be Bayesian or frequentist.
When should we use Bayesian methods?Depending on contextDepending on available toolsDepending on knowledge of tools
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist
Dempster:
A person cannot be Bayesian or frequentist. Rather aparticular analysis can be Bayesian or frequentist.
When should we use Bayesian methods?Depending on contextDepending on available toolsDepending on knowledge of tools
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist
Dempster:
A person cannot be Bayesian or frequentist. Rather aparticular analysis can be Bayesian or frequentist.
When should we use Bayesian methods?Depending on contextDepending on available toolsDepending on knowledge of tools
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian versus frequentist
Dempster:
A person cannot be Bayesian or frequentist. Rather aparticular analysis can be Bayesian or frequentist.
When should we use Bayesian methods?Depending on contextDepending on available toolsDepending on knowledge of tools
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Increased use of Bayesian methods
Bayesian methods more and more usedwithin the statistical communityin other areas based on empirical analysis
Choice of Bayesian methods based onexistence of WinBUGSincreased use of complex/hierarchical models and a belief that onlyBayesian methods can handle such models.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Increased use of Bayesian methods
Bayesian methods more and more usedwithin the statistical communityin other areas based on empirical analysis
Choice of Bayesian methods based onexistence of WinBUGSincreased use of complex/hierarchical models and a belief that onlyBayesian methods can handle such models.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
“Objections” to Bayesian methods
Last issue of Bayesian Analysis:
Discussion paper by Andrew Gelman (Gelman, 2008) onObjections to Bayesian statistics
Discussants: José Bernardo, Joseph Kadane, Stephen Senn,Larry Wasserman
Written in the voice of a hypothetical anti-Bayesian statistician,include Bayesian interpretation of frequentist statistics
Discusses much of the criticism of Bayesian statistics
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Definitions by Gelman
Bayesian inference is a method for summarizing uncertainty andmaking estimates and predictions using probability statementsconditional on observed data and an assumed model.
Include prior on parameterIntegrate uncertainty in parametersMake statements conditioned on observed data.
Frequentist statistics is an approach for evaluating statisticalprocedures conditional on some family of posited probabilitymodels.
Impute estimate of parameterDerive properties based on many possible outcomes.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Definitions by Gelman
Bayesian inference is a method for summarizing uncertainty andmaking estimates and predictions using probability statementsconditional on observed data and an assumed model.
Include prior on parameterIntegrate uncertainty in parametersMake statements conditioned on observed data.
Frequentist statistics is an approach for evaluating statisticalprocedures conditional on some family of posited probabilitymodels.
Impute estimate of parameterDerive properties based on many possible outcomes.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Definitions by Gelman
Bayesian inference is a method for summarizing uncertainty andmaking estimates and predictions using probability statementsconditional on observed data and an assumed model.
Include prior on parameterIntegrate uncertainty in parametersMake statements conditioned on observed data.
Frequentist statistics is an approach for evaluating statisticalprocedures conditional on some family of posited probabilitymodels.
Impute estimate of parameterDerive properties based on many possible outcomes.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Definitions by Gelman
Bayesian inference is a method for summarizing uncertainty andmaking estimates and predictions using probability statementsconditional on observed data and an assumed model.
Include prior on parameterIntegrate uncertainty in parametersMake statements conditioned on observed data.
Frequentist statistics is an approach for evaluating statisticalprocedures conditional on some family of posited probabilitymodels.
Impute estimate of parameterDerive properties based on many possible outcomes.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity
ObjectionsSubjectivity in choosing priors.No good objective principle for choosing a non-informative prior.Interpretation of subjective probability.The pure subjective Bayesian approach is impossible to apply.
Vague/non-informative/objective priors
Objections to objectionsBoth priors and likelihoods are subjective.Bayesian methods are objective in the sense that the final resultonly depends on the model assumed and the data obtained.All statistical methods make assumptions,sensitivity analysis wrt assumptions should be included.Both likelihood and prior only need to capture the main importantfeatures.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity
ObjectionsSubjectivity in choosing priors.No good objective principle for choosing a non-informative prior.Interpretation of subjective probability.The pure subjective Bayesian approach is impossible to apply.
Vague/non-informative/objective priors
Objections to objectionsBoth priors and likelihoods are subjective.Bayesian methods are objective in the sense that the final resultonly depends on the model assumed and the data obtained.All statistical methods make assumptions,sensitivity analysis wrt assumptions should be included.Both likelihood and prior only need to capture the main importantfeatures.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity
ObjectionsSubjectivity in choosing priors.No good objective principle for choosing a non-informative prior.Interpretation of subjective probability.The pure subjective Bayesian approach is impossible to apply.
Vague/non-informative/objective priors
Objections to objectionsBoth priors and likelihoods are subjective.Bayesian methods are objective in the sense that the final resultonly depends on the model assumed and the data obtained.All statistical methods make assumptions,sensitivity analysis wrt assumptions should be included.Both likelihood and prior only need to capture the main importantfeatures.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity
ObjectionsSubjectivity in choosing priors.No good objective principle for choosing a non-informative prior.Interpretation of subjective probability.The pure subjective Bayesian approach is impossible to apply.
Vague/non-informative/objective priors
Objections to objectionsBoth priors and likelihoods are subjective.Bayesian methods are objective in the sense that the final resultonly depends on the model assumed and the data obtained.All statistical methods make assumptions,sensitivity analysis wrt assumptions should be included.Both likelihood and prior only need to capture the main importantfeatures.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity
ObjectionsSubjectivity in choosing priors.No good objective principle for choosing a non-informative prior.Interpretation of subjective probability.The pure subjective Bayesian approach is impossible to apply.
Vague/non-informative/objective priors
Objections to objectionsBoth priors and likelihoods are subjective.Bayesian methods are objective in the sense that the final resultonly depends on the model assumed and the data obtained.All statistical methods make assumptions,sensitivity analysis wrt assumptions should be included.Both likelihood and prior only need to capture the main importantfeatures.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity (cont)
Model selection/validation: Prior much more influential.Learning versus verifying.
Subjectivity reasonable in learningObjectivity more desirable when verifying
Much progress in objective Bayes methods. Objective posterior!
Problems with vague priors ↔ frequentist difficulties.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity (cont)
Model selection/validation: Prior much more influential.Learning versus verifying.
Subjectivity reasonable in learningObjectivity more desirable when verifying
Much progress in objective Bayes methods. Objective posterior!
Problems with vague priors ↔ frequentist difficulties.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity (cont)
Model selection/validation: Prior much more influential.Learning versus verifying.
Subjectivity reasonable in learningObjectivity more desirable when verifying
Much progress in objective Bayes methods. Objective posterior!
Problems with vague priors ↔ frequentist difficulties.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Subjectivity (cont)
Model selection/validation: Prior much more influential.Learning versus verifying.
Subjectivity reasonable in learningObjectivity more desirable when verifying
Much progress in objective Bayes methods. Objective posterior!
Problems with vague priors ↔ frequentist difficulties.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Problems with/focus on computational methods
Many Bayesian applications are based on MCMC algorithms.Objections
How do we known that an MCMC-method has converged?Focus in research on computational efficiency rather than statisticalideas on experimental design etc.
Objections to objectionsComputational difficulties in all complex problems.How do we know that a frequentist method has “converged” to itsasymptotic properties?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Problems with/focus on computational methods
Many Bayesian applications are based on MCMC algorithms.Objections
How do we known that an MCMC-method has converged?Focus in research on computational efficiency rather than statisticalideas on experimental design etc.
Objections to objectionsComputational difficulties in all complex problems.How do we know that a frequentist method has “converged” to itsasymptotic properties?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Problems with/focus on computational methods
Many Bayesian applications are based on MCMC algorithms.Objections
How do we known that an MCMC-method has converged?Focus in research on computational efficiency rather than statisticalideas on experimental design etc.
Objections to objectionsComputational difficulties in all complex problems.How do we know that a frequentist method has “converged” to itsasymptotic properties?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Pragmatic approaches
Bayesian: Evaluate frequency properties.Frequentists: Bayesian approaches for deriving statisticalprocedures.
Kernel density estimationRidge/lasso regressionPenalized likelihood
Bayesian approach: Useful way to come up with an estimator incomplicated problems with structured data.
“Bayesians” use frequentist model selection/validation methods.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Pragmatic approaches
Bayesian: Evaluate frequency properties.Frequentists: Bayesian approaches for deriving statisticalprocedures.
Kernel density estimationRidge/lasso regressionPenalized likelihood
Bayesian approach: Useful way to come up with an estimator incomplicated problems with structured data.
“Bayesians” use frequentist model selection/validation methods.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Pragmatic approaches
Bayesian: Evaluate frequency properties.Frequentists: Bayesian approaches for deriving statisticalprocedures.
Kernel density estimationRidge/lasso regressionPenalized likelihood
Bayesian approach: Useful way to come up with an estimator incomplicated problems with structured data.
“Bayesians” use frequentist model selection/validation methods.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Pragmatic approaches
Bayesian: Evaluate frequency properties.Frequentists: Bayesian approaches for deriving statisticalprocedures.
Kernel density estimationRidge/lasso regressionPenalized likelihood
Bayesian approach: Useful way to come up with an estimator incomplicated problems with structured data.
“Bayesians” use frequentist model selection/validation methods.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Pragmatic approaches
Bayesian: Evaluate frequency properties.Frequentists: Bayesian approaches for deriving statisticalprocedures.
Kernel density estimationRidge/lasso regressionPenalized likelihood
Bayesian approach: Useful way to come up with an estimator incomplicated problems with structured data.
“Bayesians” use frequentist model selection/validation methods.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Including parameter uncertainty and conditioning inprediction
Aim: Predict yn+1 based on y1:n
Model: p(yn+1|θ, y1:n)
Bayesian: Integrate out uncertainty in θ
Many possibilities for frequentists:Plug in: p(yn+1|θ̂, y1:n)Parametric bootstrappingNonparametric bootstrappingWhat about conditioning?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Including parameter uncertainty and conditioning inprediction
Aim: Predict yn+1 based on y1:n
Model: p(yn+1|θ, y1:n)
Bayesian: Integrate out uncertainty in θ
Many possibilities for frequentists:Plug in: p(yn+1|θ̂, y1:n)Parametric bootstrappingNonparametric bootstrappingWhat about conditioning?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Including parameter uncertainty and conditioning inprediction
Aim: Predict yn+1 based on y1:n
Model: p(yn+1|θ, y1:n)
Bayesian: Integrate out uncertainty in θ
Many possibilities for frequentists:Plug in: p(yn+1|θ̂, y1:n)Parametric bootstrappingNonparametric bootstrappingWhat about conditioning?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Including parameter uncertainty and conditioning inprediction
Aim: Predict yn+1 based on y1:n
Model: p(yn+1|θ, y1:n)
Bayesian: Integrate out uncertainty in θ
Many possibilities for frequentists:Plug in: p(yn+1|θ̂, y1:n)Parametric bootstrappingNonparametric bootstrappingWhat about conditioning?
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Example: Time series data
Consider timeseries data y1 → y2 → ...
L(θ; y1:n) =n∏
i=1
f (yi |y1:i−1, θ)
Aim: Predict yn+1 based on y1:n.
Bayesian straightforward:
p(yn+1|y1:n) =
∫θ
p(yn+1|θ, y1:n)p(θ|y1:n)dθ
Not obvious how to do for frequentist
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Example: Time series data
Consider timeseries data y1 → y2 → ...
L(θ; y1:n) =n∏
i=1
f (yi |y1:i−1, θ)
Aim: Predict yn+1 based on y1:n.
Bayesian straightforward:
p(yn+1|y1:n) =
∫θ
p(yn+1|θ, y1:n)p(θ|y1:n)dθ
Not obvious how to do for frequentist
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Example: Time series data
Consider timeseries data y1 → y2 → ...
L(θ; y1:n) =n∏
i=1
f (yi |y1:i−1, θ)
Aim: Predict yn+1 based on y1:n.
Bayesian straightforward:
p(yn+1|y1:n) =
∫θ
p(yn+1|θ, y1:n)p(θ|y1:n)dθ
Not obvious how to do for frequentist
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Example
πjk =Pr(xt = k |xt−1 = j), j, k ∈ {1, 2}
yt =µxt + σεt
Time
mu[
x]
0 20 40 60 80 100
−2
−1
01
2
Parametric bootstrap: θ̂ → x∗
1:n → y∗
1:n → θ∗
Simulate y∗
n+1|θ̂, x∗1:n, y∗
1:n
Simulate y∗
n+1|θ∗, y1:n
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Automatic inference engine
ObjectionsDifferent methods work well in different settingsMultiplicity of parameters can be handled via hierarchical models inan automatic way.Implausible that this could really work automatically.
Objections to objectionsDifferent methods allow for subjectivity.Not automatic, three stages: Formulating model, fitting, checking
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Automatic inference engine
ObjectionsDifferent methods work well in different settingsMultiplicity of parameters can be handled via hierarchical models inan automatic way.Implausible that this could really work automatically.
Objections to objectionsDifferent methods allow for subjectivity.Not automatic, three stages: Formulating model, fitting, checking
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian interpretation of frequentist methods
Frequentist methods can not handle hierarchical models.
Analysis made using only the first two moments of the dataimplicitly assume multinormality, for otherwise important relevantinformation would be lost.
The mathematics of subjective probability works well incombining information from multiple sources.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian interpretation of frequentist methods
Frequentist methods can not handle hierarchical models.
Analysis made using only the first two moments of the dataimplicitly assume multinormality, for otherwise important relevantinformation would be lost.
The mathematics of subjective probability works well incombining information from multiple sources.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Bayesian interpretation of frequentist methods
Frequentist methods can not handle hierarchical models.
Analysis made using only the first two moments of the dataimplicitly assume multinormality, for otherwise important relevantinformation would be lost.
The mathematics of subjective probability works well incombining information from multiple sources.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
Other issues
Oversold as an all-purpose statistical solution.
Appliers use too complex models.
Do scientists want Bayesian or frequentist uncertainty intervals?
Empirical Bayes.
Conditioning on data.
Geir Storvik Bayesian versus frequentist methods
Background“Objections” by GelmanPragmatic approaches
Other issuesReferences
References I
Gelman, A. (2008). Objections to Bayesian statistics. BayesianAnalysis 3(3), 445–450.
Geir Storvik Bayesian versus frequentist methods
