Upload
girellibruni
View
224
Download
0
Embed Size (px)
Citation preview
7/27/2019 Berger the Likelihho Principle 1988 Book
1/252
Institute of Mathematical StatisticsLECTURE NOTES-MONOGRAPH SERIES
The Likelihood PrincipleSecond Edition)James O. Bergerurdue niversityRobert L. Wolpertuke niversity
7/27/2019 Berger the Likelihho Principle 1988 Book
2/252
7/27/2019 Berger the Likelihho Principle 1988 Book
3/252
T H E L I K E L I H O O D P R I N C I P L E :A R E V I E W , G E N E R A L I Z A T I O N S , A N D S T A T I S T I C A L I M P L I C A T I O N S
S E C O N D E D I T I O N
7/27/2019 Berger the Likelihho Principle 1988 Book
4/252
7/27/2019 Berger the Likelihho Principle 1988 Book
5/252
InstituteofMathematical StatisticsLECTURE NOTES-MONOGRAPH SERIESShantiS Gupta, Series EditorVolume6
The Likelihood PrincipleSecond Edition)JamesO BergerPurdue University
RobertL Wolpert uke University
InstituteofMathematical StatisticsHayward,C alifornia
7/27/2019 Berger the Likelihho Principle 1988 Book
6/252
Institute of Mathem atical S tatisticsLecture Notes-Monograph Series
Series Editor Shanti S. Gupta Purdue University
The production of the IMS Lecture Notes-Monograph Series isman aged by the IM S Business Office: Nicholas P. Jew ell IMSTreasurer and Jose L. Gonzalez IMS Business Manager.
Library of Congress Catalog Card Number: 88-81456International Standard Book Number 0-940600-13-7Copyright 1988 Institute of Mathem atical Statistics
All rights reservedPrinted in the United States of Am erica
7/27/2019 Berger the Likelihho Principle 1988 Book
7/252
T o m y p a r e n t s , O r v i s a n d T h e l m a
J a m e s B e r g e r
T o m y w i f e , R u t a
R o b e r t W o l p e r t
7/27/2019 Berger the Likelihho Principle 1988 Book
8/252
7/27/2019 Berger the Likelihho Principle 1988 Book
9/252
P R E F E
T h i s m o n o g r a p h b e g a n w i t h r e s e a r c h d e s i g n e d t o p r o v i d e a g e n e r a l i -z a t i o n o f t h e L i k e l i h o o d P r i n c i p l e L P ) t o q u i t e a r b i t r a r y s t a t i s t i c a ls i t u a t i o n s . T h e p u r p o s e o f s e e k i n g s u c h a g e n e r a l i z a t i o n w a s t o p a r t i a l l ya n s w e r c e r t a i n c r i t i c i s m s t h a t h a d b e e n l e v i e d a g a i n s t t h e L P , c r i t i c i s m s w h i c hs e e m e d t o p r e v e n t m a n y s t a t i s t i c i a n s f r o m s e r i o u s l y c o n s i d e r i n g t h e L P a n d i t si m p l i c a t i o n s . T h e r e s e a r c h e f f o r t s e e m e d w o r t h w h i l e b e c a u s e o f t h e s i m p l i c i t y ,c e n t r a l i m p o r t a n c e , a n d f a r r e a c h i n g i m p l i c a t i o n s o f t h e L P .
B a c k g r o u n d r e a d i n g f o r t h e r e s e a r c h r e v e a l e d a w i d e r t h a ne x p e c t e d r a n g e o f p u b l i s h e d c r i t i c i s m s o f t h e L P . I n a n a t t e m p t t o b e c o m p l e t ea n d a d d r e s s a l l s u c h c r i t i c i s m s , t h e r e s e a r c h p a p e r e x p a n d e d c o n s i d e r a b l y .E v e n t u a l l y i t s e e m e d s e n s i b l e t o e n l a r g e t h e p a p e r t o a m o n o g r a p h . T h i s a l s oa l l o w e d f o r d i s c u s s i o n o f c o n d i t i o n i n g i d e a s i n g e n e r a l a n d f o r a r e v i e w o f t h ei m p l i c a t i o n s o f t h e L P . I t w a s d e c i d e d , h o w e v e r , t o s t o p s h o r t o f a g e n e r a lr e v i e w o f c o n d i t i o n a l m e t h o s i n s t a t i s t i c s . I n p a r t i c u l a r , t h e m o n o g r a p h d o e sn o t d i s c u s s t h e m a n y l i k e l i h o o d b a s e d s t a t i s t i c a l m e t h o d o l o g i e s t h a t h a v e b e e nd e v e l o p e d , a l t h o u g h r e f e r e n c e s t o t h e s e m e t h o d o l o g i e s w i l l b e g i v e n . T h i sl i m i t a t i o n w a s , i n p a r t , b e c a u s e s u c h a n e n d e a v o r w o u l d b e f a r t o o a m b i t i o u s ,a n d , i n p a r t , b e c a u s e w e f e e l a n d i n d e e d a r g u e i n C h a p t e r 5 ) t h a t B a y e s i a ni m p l e m e n t a t i o n o f t h e L P i s t h e c o r r e c t c o n d i t i o n a l m e t h o d o l o g y .
T h e m a t h e m a t i c a l l e v e l o f t h e m o n o g r a p h i s , f o r t h e m o s t p a r t ,k e p t a t a n o n t e c h n i c a l l e v e l . T h e m a i n e x c e p t i o n i s t h e g e n e r a l i z a t i o n o f t h eL P i n S e c t i o n 3 . 4 , w h i c h i s n e c e s s a r i l y ) p r e s e n t e d a t a m e a s u r e - t h e o r e t i cl e v e l , b u t c a n b e s k i p p e d w i t h n o l o s s i n c o n t i n u i t y . A l s o , t h e m o n o g r a p h
vii
7/27/2019 Berger the Likelihho Principle 1988 Book
10/252
v i i i P R E F A C E
p r e s u p p o s e s n o f a m i l i a r i t y w i t h c o n d i t i o n i n g c o n c e p t s . I n d e e d C h a p t e r 2p r o v i d e s a n e l e m e n t a r y r e v i e w o f c o n d i t i o n i n g , w i t h m a n y e x a m p l e s .
T h i s s e c o n d e d i t i o n w a s p r o d u c e d u n d e r t h e r a t h e r s e v e r e c o n s t r a i n tt h a t t h e o r i g i n a l m a n u s c r i p t , u s e d f o r p h o t o - o f f s e t p r i n t i n g , w a s i n a d v e r t a n t l yd e s t r o y e d ; o n l y t h e p h o t o s w e r e k e p t . T h u s c h a n g e s c o u l d o n l y b e m a d e b yr e t y p i n g e n t i r e p a g e s o r i n s e r t i n g n e w p a g e s . A l i s t o f c o r r e c t i o n s t h a t w e r et o o m i n o r t o j u s t i f y t h e r e t y p i n g o f a n e n t i r e p a g e i s g i v e n a t t h e e n d o f t h em o n o g r a p h . I n s e r t e d p a g e s r e c e i v e d d e c i m a l p a g e n u m b e r s : e . g . 7 4 . 1 , 7 4 . 2 . Al i s t o f a d d i t i o n a l r e f e r e n c e s w a s a d d e d , a n d n e w d i s c u s s i o n s w e r e k i n d l y c o n t r i -b u t e d b y M . J . B a y a r r i a n d M . H . D e G r o o t , B r u c e H i l l , a n d L u c i e n L e C a m .
S u b s t a n t i a l c h a n g e s o r a d d i t i o n s w e r e m a d e i n S e c t i o n s 3 . 1 , 3 . 5 ,4 . 2 . 1 , 4 . 4 , a n d 4 . 5 . T h e c h a n g e s i n S e c t i o n 4 . 4 c o r r e c t a g l a r i n g o v e r s i g h t i nt h e f i r s t e d i t i o n : t h e f a i l u r e t o e m p h a s i z e t h e m i s l e a d i n g c o n c l u s i o n s t h a tc a n r e s u l t f r o m v i o l a t i o n o f t h e L i k e l i h o o d P r i n c i p l e i n s i g n i f i c a n c e t e s t i n go f a p r e c i s e h y p o t h e s i s . A n o t h e r v e r y w e a k p a r t o f t h e f i r s t e d i t i o n w a sS e c t i o n 3 . 5 , w h i c h d i s c u s s e d p r e d i c t i o n , d e s i g n , a n d n u i s a n c e p a r a m e t e r s . T h en e w m a t e r i a l i n c o r p o r a t e s r e c e n t s u b s t a n t i v e i n s i g h t s f r o m t h e l i t e r a t u r e .
N u m e r o u s o t h e r m i n o r c h a n g e s a n d l i t e r a t u r e u p d a t i n g s w e r e m a d et h r o u g h o u t t h e m o n o g r a p h . W e d i d n o t a t t e m p t c o m p l e t e c o v e r a g e o f r e c e n t l i t -e r a t u r e , h o w e v e r .
W e a r e g r a t e f u l t o a n u m b e r o f p e o p l e f o r v a l u a b l e d i s c u s s i o n s o nt h i s s u b j e c t a n d / o r f o r c o m m e n t s a n d s u g g e s t i o n s o n o r i g i n a l d r a f t s o r t h ef i r s t e d i t i o n o f t h e m o n o g r a p h . I n p a r t i c u l a r , w e w o u l d l i k e t o t h a n k G e o r g eB a r n a r d , M . J . B a y a r r i , M a r k B e r l i n e r , L a w r e n c e B r o w n , G e o r g e C a s e l l a , M o r r i sD e G r o o t , J . L . F o u l l e y , L e o n G l e s e r , P r e m G o e l , C l y d e H a r d i n , B r u c e H i l l ,J i u n n H w a n g , R a j e e v K a r a n d i k a r , L u c i e n L e C a m , K e r - C h a u L i , D e n n i s L i n d l e y ,G e o r g e M c C a b e , G e o r g e s M o n e t t e , J o h n P r a t t , D o n R u b i n , H e r m a n R u b i n , M y r aS a m u e l s , S t e v e S a m u e l s , a n d T o m S e l l k e . W e a r e e s p e c i a l l y g r a t e f u l t o M . J .B a y a r r i a n d M . H . D e G r o o t f o r a n e x c e p t i o n a l l y c o m p l e t e a n d i n s i g h t f u l s e t o fc o r r e c t i o n s a n d c o m m e n t s o n t h e f i r s t e d i t i o n . W e a r e a l s o g r a t e f u l t oS h a n t i G u p t a f o r t h e e n c o u r a g e m e n t t o t u r n t h e m a t e r i a l i n t o a m o n o g r a p h .
7/27/2019 Berger the Likelihho Principle 1988 Book
11/252
7/27/2019 Berger the Likelihho Principle 1988 Book
12/252
T A B L E OF C O N T E N T S
CHAPTER 1. INTRODUCTION 1CHAPTER 2. CONDITIONING 5
2.1. Simple Examples 52 2 Relevant Subsets 112.3. Ancillarity 132.4. Conditional Frequentist Procedures 14
2.4.1. Conditional Confidence 142 . 4 . 2 . Estimated Confidence 16
2.5. Criticisms of Partial Conditioning 16CHAPTER 3. THE LIKELIHOOD PRINCIPLE AND GENERALIZATION S . 19
3.1. Introduction 193 2 History of the Likelihood Principle 223.3. Birnbaum s Development the Discrete Case 24
3.3.1. Evidence Conditionally and Sufficiency 243.3.2. Axiomatic Development 26
3.4. Generalizations Beyond the Discrete Case 283.4.1. Difficulties in the Nondiscrete Case 293.4.2. Evidencej Conditionality> and Sufficiency 313.4.3. The Relative Likelihood Principle 32
3.5. Prediction , Design, Nuisance Paramete rs, and the LP 363.5.1. Introduction 363.5.2. Unobserved Variables: Prediction and Design 373.5.3. Nuisance Variables and Parameters 41
x
7/27/2019 Berger the Likelihho Principle 1988 Book
13/252
TABLE OF CONTENTS xi
3.6. Criticisms of Birnbaum s Axiomatic Development 423.6.1. The Model Assumption 433.6.2. The Evidence Assumption 453.6.3. The Weak Conditionality Principle 453.6.4. The Sufficiency Principle 46
3 7 Violation of the Likelihood Principl e: Inadmissibility andIncoherency 503.7.1. Introduction 503.7.2. Decision Theoretic Evaluation 523.7.3. Betting Evaluation 59
CHAPTER 4. CONSEQUENCE S AND CRITICISMS OF THE LIKELIHOOD PRINCIPLE AND THERELATIVE LIKELIHOOD PRINCIPLE 65
4.1. Incompatibility with Frequentist Concepts 654.1.1. Introduction 654.1.2. Objectivity 674.1.3. Procedures for Nonspecialists 684.1.4. Repeatability 704.1.5. The Confidence Principle 71
4 The Irrelevance of Stopping Rules 744.2.1. Introduction 744.2.2. The Discrete) Stopping Rule Principle 754.3.2. Positive Implications 774.2.4. Criticisms 804.2.5. Stopping Rules and Inadmissibili ty 834.2.6. The General Stopping Rule Principle 864.2.7. Informative Stopping Rules 88
4.3. The Irrelevance of Censoring Mechanisms4.3.1. Introduction 904.3.2. F i x e d e n s o r i n g and E q u i v a l e n t e n s o r i n g M e c h a n i s m s . . . 9 24 3 3 Random Censoring 954.3.4. Informative Censoring 103
7/27/2019 Berger the Likelihho Principle 1988 Book
14/252
x TABLE OF CONTENTS
4.4. Significance Testing 1044.4.1. onflictwith theLikelihoodPrinciple 1044.4.2. Averaging Over More Extreme Observations 1054.4.3. Testing a Single Null Model 1094.4.4. Conclusions 109.3
4.5. Randomization Analysis 1104.5.1. Introduction 1104.5.2. FinitePopulationSampling 1124.5.3. RandomizationTests 117
CHAPTER 5. IMPLEMENTATION OF THE LIKELIHOOD PRINCIPLE 1215.1. Introduction 1215.2. Non Bayesian Likelihood Methods 1225.3. Arguments for Bayesian Implementation 124
5.3.1. Generalonsiderations 1245.3.2. TheFraser Monette Ng Stone, and Stein Examples. . . 127
5.4. Robust Bayesian Analysis 1365.5. Conclusions 141
R E F E R E N C E S 1 4 3A D D I T IO N A L O R U P D A T E D R E F E R EN C E S 1 60DISCUSSION BY M. J. BAYARRI AND M. H. DEGROOT 160.3DISCUSSION BY BRUCE HILL 161DISCUSSION BY DAVID LANE 175DISCUSSION BY LUCIEN LE CAM 182REPLY TO THE DISCUSSION 186ADDITIONAL REFERENCES IN THE DISCUSSION 198INDEX OF EXAMPLE S 200AUTH OR INDEX 201SUBJECT INDEX 204
ERRATA AND CLARIFICATIONS 207
7/27/2019 Berger the Likelihho Principle 1988 Book
15/252
C H A P T E R 1 . I N T R O D U C T I O N
A m o n g a l l p r e s c r i p t i o n s f o r s t a t i s t i c a l b e h a v i o r , t h e L i k e l i h o o dP r i n c i p l e ( L P ) s t a n d s o u t a s t h e s i m p l e s t a n d y e t m o s t f a r r e a c h i n g . I t e s s e n t i a l l y s t a t e s t h a t a l l e v i d e n c e , w h i c h i s o b t a i n e d f r o m a n e x p e r i m e n t , a b o u t a nu n k n o w n q u a n t i t y , is c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n o f f o r t h e g i v e nd a t a . T h e i m p l i c a t i o n s o f t h i s a r e p r o f o u n d , s i n c e m o s t n o n B a y e s i a n a p p r o a c h e st o s t a t i s t i c s a n d i n d e e d m o s t s t a n d a r d s t a t i s t i c a l m e a s u r e s o f e v i d e n c e ( s u c ha s c o v e r a g e p r o b a b i l i t y , e r r o r p r o b a b i l i t i e s , s i g n i f i c a n c e l e v e l , f r e q u e n t i s tr i s k , e t c . ) a r e t h e n c o n t r a i n d i c a t e d .
T h e L P w a s a l w a y s i m p l i c i t i n t h e B a y e s i a n a p p r o a c h t o s t a t i s t i c s ,b u t i t s d e v e l o p m e n t a s a s e p a r a t e s t a t i s t i c a l p r i n c i p l e w a s d u e i n l a r g e p a r tt o i d e a s o f R . A . F i s h e r a n d G . B a r n a r d ( s e e S e c t i o n 3 .2 f o r r e f e r e n c e s ) . I tr e c e i v e d m a j o r n o t i c e w h e n B i r n b a u m ( 1 9 6 2 a ) s h o w e d i t t o b e a c o n s e q u e n c e o ft h e m o r e c o m m o n l y t r u s t e d S u f f i c i e n c y P r i n c i p l e ( t h a t a s u f f i c i e n t s t a t i s t i cs u m m a r i z e s t h e e v i d e n c e f r o m a n e x p e r i m e n t ) a n d C o n d i t i o n a l i t y P r i n c i p l e ( t ha te x p e r i m e n t s n o t a c t u a l l y p e r f o r m e d s h o u l d b e i r r e l e v a n t t o c o n c l u s i o n s ) . S i n c et h e n t h e L P h a s b e e n e x t e n s i v e l y d e b a t e d b y s t a t i s t i c i a n s i n t e r e s t e d i n f o u n d a t i o n s , b u t h a s b e e n i g n o r e d b y m o s t s t a t i s t i c i a n s . T h e r e a r e p e r h a p s s e v e r a lr e a s o n s f o r t h i s . F i r s t , t h e c o n s e q u e n c e s o f t h e L P s e e m s o a b s u r d t o m a n yc l a s s i c a l s t a t i s t i c i a n s t h a t t h e y f e el i t a w a s t e o f t i m e t o e v e n s t u d y t h ei s s u e . S e c o n d , a c u r s o r y i n v e s t i g a t i o n o f t h e L P r e v e a l s c e r t a i n o f t s t a t e do b j e c t i o n s , f o r e m o s t o f w h i c h i s t h e a p p a r e n t d e p e n d e n c e o f t h e p r i n c i p l e o na s s u m i n g e x a c t k n o w l e d g e o f t h e ( p a r a m e t r i c ) m o d e l f o r t h e e x p e r i m e n t ( so t h a ta n e x a c t l i k e l i h o o d f u n c t i o n e x i s t s ) . S i n c e t h e m o d e l i s r a r e l y t r u e , ( h a s t y )
1
7/27/2019 Berger the Likelihho Principle 1988 Book
16/252
2 T H E L I K E L I H O O D P R I N C I P L E
r e j e c t i o n o f t h e L P m a y r e s u l t . T h i r d , t h e L P d o e s n o t s a y h o w o n e i s t o p e r -f o r m a s t a t i s t i c a l a n a l y s i s ; i t m e r e l y g i v e s a p r i n c i p l e t o w h i c h a n y m e t h o d o fa n a l y s i s s h o u l d a d h e r e . I n d e e d B a y e s i a n a n a l y s i s i s o f t e n p r e s e n t e d a s t h e w a yt o i m p l e m e n t t h e L P ( w i t h w h i c h w e e s s e n t i a l l y a g r e e ) , a v e r y u n a p p e a l i n gp r o s p e c t t o m a n y c l a s s i c a l s t a t i s t i c i a n s .
T h e m a j o r p u r p o s e o f t h i s ( m o s t l y r e v i e w ) m o n o g r a p h i s t o a d d r e s st h e s e c o n c e r n s . A s e r i o u s e f f o r t w i l l b e m a d e , t h r o u g h e x a m p l e s a n d a p p e a l s t oc o m m o n s e n s e , t o a r g u e t h a t t h e L P i s i n t u i t i v e l y s e n s i b l e , m o r e s o t h a n t h ec l a s s i c a l m e a s u r e s w h i c h i t i m p u n e s . A l s o , a g e n e r a l i z e d v e r s i o n o f t h e L Pw i l l b e i n t r o d u c e d , a v e r s i o n w h i c h r e m o v e s t h e r e s t r i c t i o n o f a n e x a c t l y k n o w nl i k e l i h o o d f u n c t i o n , a n d y e t h a s e s s e n t i a l l y t h e s a m e i m p l i c a t i o n s . ( O t h e rc r i t i c i s m s o f t h e L P w i l l a l s o b e d i s c u s s e d . ) F i n a l l y , t h e q u e s t i o n o f i m p l e -m e n t a t i o n o f t h e L P w i l l b e c o n s i d e r e d , a n d i t w i l l b e a r g u e d t h a t B a y e s i a na n a l y s i s ( m o r e p r e c i s e l y r o b u s t B a y e s i a n a n a l y s i s ) i s t h e m o s t s e n s i b l e a n dr e a l i s t i c m e t h o d o f i m p l e m e n t a t i o n . A t h o r o u g h d i s c u s s i o n o f t h i s i s s u e i s ,h o w e v e r , o u t s i d e t h e s c o p e o f t h e m o n o g r a p h , s o t h e m a i n t h e s i s w i l l s i m p l y b et h a t t h e L P i s b e l i e v a b l e a n d t h a t b e h a v i o r i n v i o l a t i o n o f i t s h o u l d b ea v o i d e d t o t h e e x t e n t p o s s i b l e .
A c c e p t a n c e o f s u c h a t h e s i s r a d i c a l l y a l t e r s t h e w a y o n e v i e w ss t a t i s t i c s . I n d e e d , t o m a n y B a y e s i a n s , b e l i e f i n t h e L P i s t h e b i g d i f f e r e n c eb e t w e e n B a y e s i a n s a n d f r e q u e n t i s t s , n o t t h e d e s i r e t o i n v o l v e p r i o r i n f o r m a t i o n .T h u s S a v a g e s a i d ( i n t h e D i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) )
I m y s e l f , c a m e t o t a k e . . . B a y e s i a n s t a t i s t i c s . . .s e r i o u s l y o n l y t h r o u g h r e c o g n i t i o n o f t h e l i k e l i -h o o d p r i n c i p l e .
M a n y B a y e s i a n s b e c a m e B a y e s i a n s o n l y b e c a u s e t h e L P l e f t t h e m l i t t l e c h o i c e .S u f f i c i e n t t i m e h a s p a s s e d s i n c e t h e a x i o m a t i c d e v e l o p m e n t o f
B i r n b a u m t o h o p e t h a t a n y v a l i d o b j e c t i o n s t o t h e L P w o u l d b y n o w h a v e b e e nf o u n d . I n d e e d , t h e r e a r e n u m e r o u s a r t i c l e s i n t h e l i t e r a t u r e p r e s e n t i n ge x a m p l e s , c o u n t e r e x a m p l e s , a r g u m e n t s , a n d c o u n t e r a r g u m e n t s f o r t h e L P . W e w i l l
7/27/2019 Berger the Likelihho Principle 1988 Book
17/252
7/27/2019 Berger the Likelihho Principle 1988 Book
18/252
4 T H E L I K E L I H O O D P R I N C I P L E
b e g i v e n i n s i m p l e a r t i f i c i a l s e t t i n g s , r a t h e r t h a n r e a l i s t i c a l l y c o m p l i c a t e ds t a t i s t i c a l s i t u a t i o n s , a g a i n f o r e a s e o f r e a d i n g a n d b e c a u s e c o m p l i c a t e d s i t
u a t i o n s a r e o f t e n t o o i n v o l v e d t o c l e a r l y r e v e a l k e y i s s u e s . A d v a n c e m e n t o f as u b j e c t u s u a l l y p r o c e e d s by a p p l y i n g t o c o m p l i c a t e d s i t u a t i o n s t r u t h s d i s c o v e r e d i n s i m p l e s e t t i n g s .
T h r o u g h o u t t h e m o n o g r a p h , X wi l l d e n o t e t h e r a n d o m q u a n t i t y t o b eo b s e r v e d , X t h e s a m p l e s p a c e , x ( t h e o b s e r v e d d a t a ) a r e a l i z a t i o n o f X , a n dP () t h e p r o b a b i l i t y d i s t r i b u t i o n o f X o n 2 , w h e r e G i s u n k n o w n . A l t h o u g h w i l l b e c a l l e d t h e p a r a m e t e r a n d t h e p a r a m e t e r s p a c e t h e f a m i l y { P ( )> n e e d n o t b e a t y p i c a l p a r a m e t r i c f a m i l y ; c o u l d j u s t d e n o t e s o m e ( p o s s i b l y n o n p a r a m e t r i c ) i n d e x . A l s o , w i l l b e u n d e r s t o o d t o c o n s i s t o f a l l u n k n o w n f e a t u r e s o f t h e p r o b a b i l i t y d i s t r i b u t i o n . O f t e n , t h e r e f o r e , o n l y p a r t o f w i l l b e o f i n t e r e s t , t h e r e m a i n d e r b e i n g a n u i s a n c e p a r a m e t e r . I n d i s c u s s i n g s e q u e n t i a l a n d p r e d i c t i o n p r o b l e m s i t w i l l s o m e t i m e s b e c o n v e n i e n t t o c o n s i d e r u n o b s e r v e d r a n d o m v a r i a b l e s Z , a s w e l l a s t h e u n k n o w n ; z w i l l t h e n d e n o t e a p o s s i b l e v a l u e o f Z . T o s i m p l i f y t h e e x p o s i t i o n i n t h e m o n o g r a p h , h o w e v e r , w e w i ll u s u a l l y o n l y c o n s i d e r t h e s i m p l e r c a s e i n w h i c h Z i s a b s e n t . N o t et h a t f o r s o m e s t a t i s t i c a l p r o b l e m s i t i s i m p o s s i b l e t o s e p a r a t e Z a n d { P _ ( ) }
S e e S e c t i o n 3 .5 f o r d i s c u s s i o n o f s u c h p r o b l e m s .W h e n n e c e s s a r y , ^ w i l l d e n o t e t h e f i e l d o f m e a s u r a b l e e v e n t s i n X .
I f a d e n s i t y f o r X e x i s t s i t w i l l b e d e n o t e d f Q ( x ) , a n d w e w i l l p r e s u m e t h ee x i s t e n c e o f a s i n g l e d o m i n a t i n g f i n i t e m e a s u r e v ( ) f o r { P ( )> E ) s u c ht h a t P Q ( B ) = J f Q ( x ) v ( d x ) f o r e a c h B e 3 . I n a ll t h e e x a m p l e s v w i l l b e t a k e nt o b e c o u n t i n g m e a s u r e i n t h e d i s c r e t e c a s e a n d L e b e s g u e m e a s u r e i n t h e c o n t i n u o u s c a s e , w h e n X i s a s u b s e t o f E u c l i d e a n s p a c e . U s u a l l y w e w i l l w r i t e t h er e f e r e n c e m e a s u r e s i m p l y a s d x ( i m p l i c i t l y t a k i n g L e b e s g u e m e a s u r e f o r v ) ;t h e f o r m u l a s w i l l r e q u i r e m i n o r c h a n g e s f o r c a s e s ( i n c l u d i n g t h o s e i n v o l v i n gd i s c r e t e d i s t r i b u t i o n s ) i n w h i c h o t h e r r e f e r e n c e m e a s u r e s a r e m o r e c o n v e n i e n t .
7/27/2019 Berger the Likelihho Principle 1988 Book
19/252
C H A P T E R 2 . C O N D I T I O N I N G
The most commonly used measures o f accura cy o f evidenc e ins t a t i s t i c s a r e p re exp er im en t a l . A p a r t i c u l a r p r oc e dure i s de c ide d upon fo rus e , a nd the a c c ur a c y o f the e v ide nc e f rom a n e x pe r ime nt i s i d e n t i f i e d w i th thelong r un be ha v ior o f the p r oc e d ur e , we re the e x pe r ime nt r e pe a te d l y pe r fo r me d .This long run be hav ior is eva luat ed by averag ing the per formance of the proce dur e ove r the sa mple spac e Z . I n c o nt r a s t , the LP s ta t e s t ha t p o s t exp er im en t a lreasoning should be used, where in only the ac tua l observat ion x (and not theo the r obs e r v a t ions i n % t ha t c ou ld have oc c ure d) i s r e l e v a nt . The re a r e av a r i e t y o f i n t e r m e d i a t e p o s i t io n s w hic h c a l l f o r p a r t i a l c o n d i t io n i n g o n x andp a r t i a l lo ng r un f re q u e nc y i n t e r p r e t a t i o n s . P a r t l y f o r h i s t o r i c a l p u rp o se s ,a nd pa r t l y to i nd i c a te tha t the ca se fo r a t l e a s t some s or t o f c ond i t i on i ng i scom pe l ling, we d iscuss in th is cha pte r var ious co ndi t ion ing v iew points .2 1 SIM PLE EXAMPLES
The fo l lowing s imple examples revea l the necess i ty of a t least somet imest h i n k i n g c o n d i t i o n a l l y , a nd w i l l be im p o r t a n t l a t e r .EXAMPLE 1 . Suppose X, and X 2 are independent and
P ( X = 1 ) = PQ ( X . = + 1 ) = , i = 1 , 2 .H e r e > < < is an u n k n o w n p a r a m e t e r to be e s t i m a t e d f r o m X, and X 2 It ise a s y to see h a t a 7 5 % c o n f i d e n c e set f s m a l l e s t s i z e for is
c x v x 2 =t h e p o i n t y X j + X g ) if X 1f
t h e p o i n t X ^ l if X 1=5
7/27/2019 Berger the Likelihho Principle 1988 Book
20/252
6 T H E L I K E L I H O O D P R I N C I P L E
T h u s , i f r e p e a t e d l y u s e d i n t h i s p r o b l e m , C ( X , , X 2 ) w o u l d c o n t a i n w i t hp r o b a b i l i t y . 7 5 .
N o t i c e , h o w e v e r , t h a t w h e n x , f x i t i s a b s o l u t e l y c e r t a i n t h a t = X n + X p ) , w h i l e w h e n x ^ = x ~ i t i s e q u a l l y u n c e r t a i n w h e t h e r = xj1 o r = x , + l ( a s s u m i n g n o p r i o r k n o w l e d g e a b o u t ) . T h u s , f r o m a p o s t e x p e r i m e n t a lv i e w p o i n t , o n e w o u l d s a y t h a t C ( x 1 9 x 2 ) c o n t a i n s w i t h " c o n f i d e n c e " 1 0 0 % w h e nx , f x 2 , b u t o n l y w i t h " c o n f i d e n c e " 5 0 % w h e n x , = x 2 C o m m o n s e n s e c e r t a i n l ys u p p o r t s t h e p o s t e x p e r i m e n t a l v i e w h e r e . I t i s t e c h n i c a l l y c o r r e c t t o c a l lC U p X o ) a 7 5 % c o n f i d e n c e s e t , b u t , i f a f t e r s e e i n g t h e d a t a w e k n o w w h e t h e r i ti s r e a l l y a 1 0 0 % o r 5 0 % s e t , r e p o r t i n g 7 5 % s e e m s r a t h e r s i l l y .
T h e a b o v e e x a m p l e f o c u s e s t h e i s s u e s o m e w h a t : d o e s i t m a k e s e n s et o r e p o r t a p r e e x p e r i m e n t a l m e a s u r e w h e n i t i s k n o w n t o b e m i s l e a d i n g a f t e rs e e i n g t h e d a t a ? T h e n e x t e x a m p l e a l s o s e e m s i n t u i t i v e l y c l e a r , y e t i s t h e k e yt o a l l t h a t f o l l o w s .E X A M P L E 2 . S u p p o s e a s u b s t a n c e t o b e a n a l y z e d c a n b e s e n t e i t h e r t o al a b o r a t o r y i n N e w Y o r k o r a l a b o r a t o r y i n C a l i f o r n i a . T h e t w o l a b s s e e me q u a l l y g o o d , s o a f a i r c o i n i s f l i p p e d t o c h o o s e b e t w e e n t h e m , w i t h " h e a d s "d e n o t i n g t h a t t h e l a b i n N e w Y o r k w i l l b e c h o s e n . T h e c o i n i s f l i p p e d a n dc o m e s u p t a i l s , s o t h e C a l i f o r n i a l a b i s u s e d . A f t e r a w h i l e , t h e e x p e r i m e n t a lr e s u l t s c o m e b a c k a n d a c o n c l u s i o n m u s t b e r e a c h e d . S h o u l d t h i s c o n c l u s i o nt a k e i n t o a c c o u n t t h e f a c t t h a t t h e c o i n c o u l d h a v e b e e n h e a d s , a n d h e n c e t h a tt h e e x p e r i m e n t i n N e w Y o r k m i g h t h a v e b e e n p e r f o r m e d i n s t e a d ?
T h i s , o f c o u r s e , i s a v a r i a n t o f t h e f a m o u s C o x e x a m p l e ( C o x ( 1 9 5 8 ) s e e a l s o C o r n f i e l d ( 1 9 6 9 ) ) , w h i c h c o n c e r n s b e i n g g i v e n ( a t r a n d o m ) e i t h e r a na c c u r a t e o r a n i n a c c u r a t e m e a s u r i n g i n s t r u m e n t ( a n d k n o w i n g w h i c h w a s g i v e n ) .S h o u l d t h e c o n c l u s i o n r e a c h e d b y e x p e r i m e n t a t i o n d e p e n d o n l y o n t h e i n s t r u m e n ta c t u a l l y u s e d , o r s h o u l d i t t a k e i n t o a c c o u n t t h a t t h e o t h e r i n s t r u m e n t m i g h th a v e b e e n o b t a i n e d ?
I n s y m b o l i c f o r m , w e c a n p h r a s e t h i s e x a m p l e a s a " m i x e d e x p e r i m e n t "
7/27/2019 Berger the Likelihho Principle 1988 Book
21/252
7/27/2019 Berger the Likelihho Principle 1988 Book
22/252
8 T H E L I K E L I H O O D P R I N C I P L E
N o t e t h a t t h e r e i s n o t h i n g l o g i c a l l y w r o n g w i t h r e p o r t i n g e r r o rp r o b a b i l i t i e s i n E x a m p l e 3 ; i t j u s t s e e m s t o b e a n i n a d e q u a t e r e f l e c t i o n o f t h ee v i d e n c e c o n v e y e d b y t h e d a t a t o r e p o r t = . 0 2 2 8 f o r b o t h x = 0 a n d x = 1 .P r a t t ( 1 9 7 7 ) ( p e r h a p s s o m e w h a t t o n g u e i n c h e e k ) t h u s c o i n s
T H E P R I N C I P L E O F A D E Q U A C Y . A c o n c e p t o f s t a t i s t i c a l e v i d e n c e i s ( v e r y )i n a d e q u a t e i f i t d o e s n o t d i s t i n g u i s h e v i d e n c e o f ( v e r y ) d i f f e r e n t s t r e n g t h s .E X A M P L E 4 a . S u p p o s e X i s 1 , 2 , o r 3 a n d is 1 o r 2 , w i t h P ( x ) g i v e n i n t h ef o l l o w i n g t a b l e :
poP l
1 9
1
X2
1
9 8 9
3 9 9
1
T h e t e s t , w h i c h a c c e p t s P Q w h e n x = 3 a n d a c c e p t s P , o t h e r w i s e , i s a m o s tp o w e r f u l t e s t w i t h b o t h e r r o r p r o b a b i l i t i e s e q u a l t o . 0 1 . H e n c e , i t w o u l d b ev a l i d t o m a k e t h e f r e q u e n t i s t s t a t e m e n t , u p o n o b s e r v i n g x = 1 , " M y t e s t h a sr e j e c t e d P Q a n d t h e e r r o r p r o b a b i l i t y i s . 0 1 . " T h i s s e e m s v er y m i s l e a d i n g ,s i n c e t h e l i k e l i h o o d r a t i o i s a c t u a l l y 9 t o 1 in f a v o r o f P Q , w h i c h i s b e i n gr e j e c t e d .E X A M P L E 4 b . O n e c o u l d o b j e c t i n E x a m p l e 4 a , t h a t t h e . 01 l e v e l t e s t i si n a p p r o p r i a t e , a n d t h a t o n e s h o u l d u s e t h e . 0 01 l e v e l t e s t , w h i c h r e j e c t s o n l yw h e n x = 2 . C o n s i d e r , h o w e v e r , t h e f o l l o w i n g s l i g h t l y c h a n g e d v e r s i o n :
poP l
1 5
5 1
X2
5
. 9 8 4 9
3 9 9 1
A g a i n t h e t e s t w h i c h r e j e c t s P Q w h e n x = 1 o r 2 a n d a c c e p t s o t h e r w i s e h a s e r r o rp r o b a b i l i t i e s e q u a l t o . 0 1 , a n d n o w i t i n d e e d s e e m s s e n s i b l e t o t a k e t h ei n d i c a t e d a c t i o n s ( i f w e s u p p o s e a n a c t i o n m u s t b e t a k e n ) . I t s t i l l s e e m s
7/27/2019 Berger the Likelihho Principle 1988 Book
23/252
O N I T I O N I N G 9
unreasonable, however, to r e p o r t ane r r o r p r o b a b i li t y of .01 upon re jec t ingPQwhenx = 1, s ince the data provides yery l i t t l e evid en ce in favor of P,.EXAMPLE5. For a dec is ion t he o re t ic example, cons ider the i n t e r e s t i n g S t e inphenomenon, concerned w ith es tim at ion of a p v a r i a t e normal mean (p >_ 3) basedon X 7? ( , I) andunder sum of squares e r r o r lo ss. The usual pre exper imenta lmeasure of the performance of an es t ima to r is the r i s k f u n c t i o n (or expectedloss)
R ) = E0 j . X ) ) 2 .
T h e c l a s s i c a l e s t i m a t o r h e r e is 6 (x) = x, but James and St e in (1960) showedthat
J S x ) = ( 1 _ B z |)
has R( , ) < R( , ) = p f or al l . One can t h us r e p o r t 6 as a lwaysbei n g b e t t e r t h a n f r om a p r e e x p e r i m e n t a l v ie w p o i n t . However, i f p = 3 andx = ( 0 , . 0 1 , . 0 1 ) is o b s e r v e d , t h e n
6JS(x) = (0, 49.99, 49.99),w h i c his ana b s u r d e s t i m a t eof . H e n c e canbet e r r i b l e for c e r t a i nx.O f c o u r s e the p o s i t i v e p a r t v e r s i o nof 6 ,
6J s + ( x , = ( i . f i= | ,+Xi
c o r r ec t s t h i s g l a r i n g p r o b l e m ,but the p o i n t is t h a t a procedure which looksgrea t p re exper imen ta l l y cou ld be t e r r i b l e for p a r t i c u l ar x, and i t may notalways be so obvious when this is the case.
Confidence sets fo r canalso bedeveloped (seeCasel laandHwang (1982)) which have l a r g e r p r o b a b i l i t i e s of coverage than the c l a s s i c a lc o n f i d e n c e e l l i p s o i d s , are never l a r g e r in s i z e , and for smal l | x|c o n s i s t of the s i n g l e p o i n t {0}. Indeed , these sets are of the s imple form
7/27/2019 Berger the Likelihho Principle 1988 Book
24/252
1 0 THE L I K E L I H O O D P R I N C I P L E J ~ S + ( x ) | 2 p ( l ) } if | x | >
C ( x ) ={ 0 } if | x | < ,
2where x (1 ) i s t he 1 t/z p e r c e n t i l e o f th e c h i s q u a re d i s t r i b u t i o n w i t h pdegrees o f f reedom, and i s s u i t ab l y sm al l . A lt hough t h i s con f idence procedure looks grea t p re exp er im ent a l l y , one wou ld l ook ra the r f o o l i sh to concludewhen p = 3 and x = (0 , .01 , .0 1) t h a t i s t he p o in t {0 } w i th con f idence 95%.
The above examples , though s imple, ind icate most of the in tu i t i vereasons fo r co n d i t io n in g . There are a wide var iet y of oth er such examples.The Un if o rm ( , + ) d i s t r i b u t i o n ( , 3 known) p r ov ides a host o f exam p leswhere co nd i t i ona l reason ing d i f f e r s cons iderab ly f rom pre exper im enta l r easoni n g ( c . f . Welch (1939) and Pr at t (19 61 ) ) . The Ste in 2s tage procedure fo r
2o b t a i n i n ga c o n f i d e n c e i n t e r v a lof f i x e d w i d t h f o r t h e m e a n of a ? ? ( , ) d i s 2t r i b u t i o nis a n o t h e r e x a m p l e . A p r e l i m i n a r y s a m p l e a l l o w s e s t i m a t i o n of ,f r o m w h i c h it is p o s s i b l e to d e t e r m i n e the s a m p l e s i z e n e e d e dfor a s e c o n ds a m p l e in o r d e r to g u a r a n t e e an o v e r a l l p r o b a b i l i t y of c o v e r a g e f o r a f i x e dw i d t h i n t e r v a l . B u t w h a t if t h e s e c o n d s a m p l e i n d i c a t e s t h a t the p r e l i m i n a r y
2e s t i m a t e of was w o e f u l l y l o w ? T h e n o n e w o u l d r e a l l y h a v e m u c h l e s s r e a lc o n f i d e n c ein the p r o p o s e d i n t e r v a l ( c . f . L i n d l e y ( 1 9 5 8 ) a n d S a v a g e e t . l.( 1 9 6 2 ) ) . A n o t h e r e x a m p l e is r e g r e s s i o n on r a n d o m c o v a r i a t e s . It isc o m m o np r a c t i c e to p e r f o r m t h e a n a l y s i s c o n d i t i o n a l l yon t h e o b s e r v e d v a l u e sof thec o v a r i a t e s , r a t h e r t h a n g i v i n g c o n f i d e n c e s t a t e m e n t s , e t c . , v a l i din ana v e r a g e s e n s e o v e r a l l c o v a r i a t e s t h a t c o u l d h a v e b e e n o b s e r v e d . R o b i n s o n( 1 9 7 5 ) a l s o g i v e s e x t r e m e l y c o m p e l l i n g ( t h o u g h a r t i f i c i a l ) e x a m p l e sof then e e d to c o n d i t i o n . P i c c i n a t o ( 1 9 8 1 ) g i v e s s o m e i n t e r e s t i n g d e c i s i o n t h e o r e t i ce x a m p l e s .
A f i n a l i m p o r t a n t e x a m p l e is t h a t of r o b u s t e s t i m a t i o n . A c o n v i n c i n g c a s e can e m a d e t h a t i n f e r e n c e s t a t e m e n t s s h o u l d be m a d e c o n d i t i o n a l l yo n the r e s i d u a l s ; if t h e d a t a l o o k s c o m p l e t e l y l i k e n o r m a l d a t a , use n o r m a lt h e o r y . B a r n a r d ( 1 9 8 1 ) s a y s
7/27/2019 Berger the Likelihho Principle 1988 Book
25/252
C O N D I T I O N I N G " W e s h o u l d r e c o g n i s e t h a t ' r o b u s t n e s s 1ofi n f e r e n c e is a c o n d i t i o n a l p r o p e r t y s o m ei n f e r e n c e s f r o m s o m e s a m p l e s are r o b u s t .B u t o t h e r i n f e r e n c e s , or the s a m e i n f e r e n c e sf r o m o t h e r s a m p l e s , may d e p e n d s t r o n g l y ond i s t r i b u t i o n a l a s s u m p t i o n s . "
D e m p s t e r ( 1 9 7 5 ) c o n t a i n s w e r y c o n v i n c i n g d i s c u s s i o n and a h o s t of i n t e r e s t i n ge x a m p l e s c o n c e r n i n g t h i s i s s u e . R e l a t e d to c o n d i t i o n a l r o b u s t n e s s is l a r g es a m p l e i n f e r e n c e , w h i c h s h o u l d o f t e nbe d o n e c o n d i t i o n a l l y on s h a p e f e a t u r e so f t h e l i k e l i h o o d f u n c t i o n . T h u s , in u s i n g a s y m p t o t i c n o r m a l t h e o r y for them a x i m u m l i k e l i h o o d e s t i m a t o r , , one h o u l d g e n e r a l l y use I ( ) ~ , the i n v e r s eo f o b s e r v e d F i s h e r i n f o r m a t i o n , as the c o v a r i a n c e m a t r i x , r a t h e r t h a n I ( ) ~ ,t h e i n v e r s e of e x p e c t e d F i s h e r i n f o r m a t i o n . F o r e x t e n s i v e d i s c u s s i o n oft h e s eand r e l a t e d i s s u e s s e e J e f f r e y s ( 1 9 6 1 ) , P r a t t ( 1 9 6 5 ) , A n d e r s e n ( 1 9 7 0 ) ,E f r o nand H i n k l e y ( 1 9 7 8 ) , B a r n d o r f f N i e l s e n ( 1 9 8 0 ) , Cox ( 1 9 8 0 ) , a n d H i n k l e y( 1 9 8 0 a , 1 9 8 2 ) .2 . 2 R E L E V A N T S U B S E T S
Fisher ( c . f . F isher (1956a)) long advocated co nd i t ion ing on what hec a l l e d relevant subsets of (a l so ca l l ed recog n izab le sub sets , r e fe rencese ts , o r co nd i t i ona l exper im enta l frames o f re fe re nc e ) . There i s a cons iderab le l i t e ra tu re on the sub jec t , which tends to be more fo rmal t han thein t u i t i ve t ype o f reason ing presented i n t he examples o f Sec t ion 2 .1 . Thebas ic idea is to f ind subsets of ( o f t e n d e te rm in ed by s t a t i s t i c s ) w h ic h ,when co nd i t ion ed upon, change the pre expe r ime ntal m easure. In Example 1 , f o ri ns t anc e ,
= {x : x^ = x^} U {x : x 1 t x 2 ha n d the c o v e r a g e p r o b a b i l i t i e s of C U ^ ^ ) c o n d i t i o n e d on o b s e r v i n g X in the" r e l e v a n t " s u b s e t s { x : x = x 2 ) or { x : x ]fx 2> are 1 and . 5 , r e s p e c t i v e l y .I n E x a m p l e 2, the two u t c o m e s of t h e c o i n f l i p d e t e r m i n e two r e l e v a n t s u b s e t s .I n E x a m p l e s 3, 4, and 5 it is not c l e a r w h a t s u b s e t s s h o u l d be c o n s i d e r e d
7/27/2019 Berger the Likelihho Principle 1988 Book
26/252
1 2 THE L I K E L I H O O D P R I N C I P L Er e l e v a n t ,but m a n y r e a s o n a b l e c h o i c e s g i v e c o n d i t i o n a l r e s u l t s q u i t e d i f f e r e n tf r o m thep r e e x p e r i m e n t a l r e s u l t s .
F o r m a l t h e o r i e sofr e l e v a n t s u b s e t s ( c . f . B u e h l e r ( 1 9 5 9 ) ) p r o c e e di naf a s h i o n a n a l o g o u stothe f o l l o w i n g . S u p p o s e C ( x ) is ac o n f i d e n c e p r o c e d u r ew i t h c o n f i d e n c e c o e f f i c i e n t 1 for ll, i.e.,( 2 . 2 . 1 ) P ( C ( X ) c o n t a i n s)= 1 forall.T h e nB isc a l l e dar e l e v a n t s u b s e t of if,for s o m e >0, e i t h e r( 2 . 2 . 2 ) P Q ( C ( X ) c o n t a i n s|X B _ ( 1 ) forallo r( 2 . 2 . 3 ) P ( C ( X ) c o n t a i n s | X B > 1 ) + f o r a l l .When ( 2 . 2 . 2 ) or ( 2 . 2 . 3 ) h o l d s and x G B is o b s e r v e d , i t is q u e s t i o n a b l e w h e t h e r( 2 . 2 . 1 ) s h o u l d be the measure of e v i d e n c e r e p o r t e d . T h i s f o r m e d the b a s i s ofF i sh e r ' s o b j e c t i o n ( F i s h e r ( 1 9 5 6 b ) ) to the A s p in W e l c h ( 1 9 4 9 ) s o l u t i o n to theB e h r e n s F i s h e r p r o b l e m se e a l s o Y a t e s ( 1 9 6 4 ) and C o r n f i e l d ( 1 9 6 9 ) ) . A n o t h e re x a m p l e f o l l o w s . For mo re e x a m p l e s , see C o r n f i e l d ( 1 9 6 9 ) , O lshen ( 1 9 7 7 ) , andF r a s e r ( 1 9 7 7 ) . )EXAMPLE 6 . (B row n ( 1 9 6 7 ) , w i t h e a r l i e r r e l a t e d e x a m p l e s by S t e i n ( 1 9 6 1 ) and
pB u e h l e r and F e d d e r s o n ( 1 9 6 3 ) ) . I f X , , . . . , X n is a sa m p l e f r o m a 7 ?( , )d i s t r i b u t i o n , b o t h and unknown, the u s u a l 1 0 0 ( l ) % c o n f i d e n c e i n t e r v a l f or is
C x.s) = x t / 2 ^ *+ t
/ 2 ^where x and s are the sample mean and standard de viation, re spect ive ly, andt i is the appropriate cr it ic a l value for the t dis tr ibut ion with n1 degreesof freedom. For n = 2 and = . 5 we thus have
P 2 C X,S) contains ) = .5 for all , 2 , 9b u t B r o w n ( 1 9 6 7 ) s h o w e d t h a t
P 2 ( C ( X , S ) c o n t a i n s | X | / S _f for all , 2,
7/27/2019 Berger the Likelihho Principle 1988 Book
27/252
7/27/2019 Berger the Likelihho Principle 1988 Book
28/252
1 4 THE L I K E L I H O O D P R I N C I P L Ea n c i l l a r y s t a t i s t i c ( h a v i n ga d i s t r i b u t i o n c l e a r l y i n d e p e n d e n t o f ) . Thec o n d i t i o n a l d i s t r i b u t i o no f T g i v e n S = s is u n i f o r m on the set
X = { ( u , v ) : vu s and 2 < u < + 2 " s } .I n f e r e n c e w i t h re s p e c t t o t h i s c o n d i t io n a l d i s t r i b u t i o n i s s t r a i g h t f o r w a r d .For i ns ta nc e , a 1 0 0 ( l ) % ( c o n d i t i o n a l ) c o nf id e nc e in t e r v a l f o r i s
C(U, V) = 1 (U+ V) l ( l ) O s ) ,one of the solu tion s proposed by Welch ( 1 9 3 9 ) . Th is c ond i t iona l i n t e r v a l i scons idera bly more app ea l ing than var ious op t ima l noncond i t iona l in t e rv a ls ,as d iscussed in Pra t t ( 1 9 6 1 ) .
T he r e a r e a num ber o f d i f f i c u l t i e s i n t h e d e f i n i t i o n and use o fa n c i l l a r y s t a t i s t i c s ( c . f . Basu ( 1 9 6 4 ) and Cox ( 1 9 7 1 ) ) . N e v e r t h e l e s s, c o n d it i on i ng on a nc i l l a r i e s goe s a long way towa rds pr ov id i ng be t t e r c ond i t i ona lpr oc e dure s . A fe w r e fe r e nc e s , fr om whic h o the r s c an be ob t a i ne d , a r e F is he r( 1 9 5 6 a ) , Anderson ( 1 9 7 3 ) , Barnard ( 1 9 7 4 ) , Cox and Hin kle y ( 1 9 7 4 ) , Cox ( 1 9 7 5 ) ,Dawid ( 1 9 7 5 , 1 9 8 1 ) , Efron and Hink ley ( 1 9 7 8 ) , B a r n d o r f f N i e l s e n ( 1 9 7 8 , 1 9 8 0 ) ,Hink ley ( 1 9 7 8 , 1 9 8 0 ) , S e i d e n f e l d ( 1 9 7 9 ) , Grambsch ( 1 9 8 0 ) , Amari ( 1 9 8 2 ) ,B a r n e t t ( 1 9 8 2 ) , and Buehler ( 1 9 8 2 ) .2 . 4 CONDITIONAL FREQUENTIST PROCEDURES
An a mbi t ious a t te m pt to fo r m a l i z e c ond i t i on i ng w i t h i n a f r e qu e nt i s tframework was undertaken by Kiefer ( 1 9 7 7 ) . ( Se e a l so K i e f e r (1 9 7 5 , 1 9 7 6 ) ,Brown ( 1 9 7 7 ) , Brownie and Kie fer ( 1 9 7 7 ) , and Berger ( 1 9 8 4 c , 1 9 8 4 d ). ) Thef o r m a l i z a t i o n was i n tw o d i s t i n c t d i r e c t i o n s , w hic h K i e f e r c a l l e d c o n d i t i o n a lc onf i de nc e a nd e s t i ma te d c onf i de nc e .2 . 4 . 1 Con di t iona l Conf idence
The ba s i c ide a o f c ond i t i ona l c onf ide nc e i s to de f i ne a p a r t i t i o nW $ : s and then associate with each set in the partition th e appropriate conditionalfrequency measure for the procedure considered. In xample 1, the partitionwould be into th e sets X = {x : x^ = x} and X = {x : x-j t x 2 ) I n
7/27/2019 Berger the Likelihho Principle 1988 Book
29/252
CONDITIONING 15Example 2, the p a rt i t io n would be into the sets where heads and t a i l s areo b s e r v e d , r e s p e c t i v e l y .
When dea l ing wi t h a conf idence procedure {C X )} , the con di t ion alfrequency measure th at would be rep orte d, i f x X were obse rved, is
r ( ) = P_( C( X ) co n t a i n s | X
7/27/2019 Berger the Likelihho Principle 1988 Book
30/252
1 6 T H E L I K E L I H O O D P R I N C I P L E
2 . 4 . 2 E s t i m a t e d C o n f i d e n c eA n a l t e r n a t i v e a p p r o a c h t o c o n d i t i o n i n g , w h i c h c a n b e j u s t i f i e d
f r o m a f r e q u e n t i s t p e r s p e c t i v e ( c . f . K i e f e r ( 1 9 7 7 ) o r B e r g e r ( 1 9 8 4 c ) ) , i s t op r e s e n t a d a t a d e p e n d e n t c o n f i d e n c e f u n c t i o n . I f a c o n f i d e n c e s e t p r o c e d u r eC ( x ) i s t o b e u s e d , f o r i n s t a n c e , o n e c o u l d r e p o r t l ( x ) a s t h e " c o n f i d e n c e "i n C ( x ) w h e n x i s o b s e r v e d . P r o v i d i n g( 2 . 4 . 2 ) E ( X ) ) < P ( C ( X ) c o n t a i n s ) f o r a ll ,
t h i s " r e p o r t " h a s t h e u s u a l f r e q u e n t i s t v a l i d i t y t h a t , i n r e p e a t e d u s e , C ( X )w i l l c o n t a i n w i t h a t l e a s t t h e a v e r a g e o f t h e r e p o r t e d c o n f i d e n c e s . T h u s ,i n E x a m p l e 2 , o n e c o u l d r e p o r t l ( x ) = 1 o r j a s x , ^ x 2 o r x = x 2 >r e s p e c t i v e l y . E s t i m a t e d c o n f i d e n c e ( o r , m o r e g e n e r a l l y , e s t i m a t e d r i s k ) c a nb e v e r y u s e f u l i n a n u m b e r o f s i t u a t i o n s w h e r e c o n d i t i o n a l c o n f i d e n c e f a i l s( s e e K i e f e r ( 1 9 7 7 ) o r B e r g e r ( 1 9 8 4 c ) ) .2 .5 C R I T I C I S M S O F P A R T I A L C O N D I T I O N I N G
T h e n e e d t o a t l e a s t s o m e t i m e s c o n d i t i o n s e e m s t o b e w e l lr e c o g n i z e d , a s t h e b r i e f r e v i e w i n t h i s c h a p t e r h a s i n d i c a t e d . T h e a p p r o a c h e sd i s c u s s e d i n S e c t i o n s 2 . 2 , 2 . 3 , a n d 2 .4 . 1 c o n s i d e r o n l y p a r t i a l c o n d i t i o n i n g ,h o w e v e r ; o n e s t i l l d o e s a f r e q u e n c y a n a l y s i s , b u t w i t h t h e c o n d i t i o n a l d i s t r i b u t i o n o f X o n a s u b s e t . T h e r e a r e s e v e r a l m a j o r c r i t i c i s m s o f s u c h p a r t i a lc o n d i t i o n i n g . ( T h e e s t i m a t e d c o n f i d e n c e a p p r o a c h i n S e c t i o n 2 . 4 . 2 h a s a q u i t ed i f f e r e n t b a s i s ; c r i t i c i s m o f i t w i l l b e g i v e n a t t h e e n d o f t h i s s e c t i o n . )
F i r s t , t h e c h o i c e o f a r e l e v a n t s u b s e t o r a n a n c i l l a r y s t a t i s t i co r a p a r t i t i o n { z $ : s e S ) c a n b e y e r y u n c e r t a i n . I n d e e d , i t s e e m s f a i r l yc l e a r t h a t i t is h a r d t o a r g u e p h i l o s o p h i c a l l y t h a t o n e s h o u l d c o n d i t i o n o n ac e r t a i n s e t o r p a r t i t i o n , b u t n o t o n a s u b s e t o r s u b p a r t i t i o n . ( A f t e r a l l , i ts e e m s s o m e w h a t s t r a n g e t o o b s e r v e x , n o t e t h a t i t i s i n , s a y , , a n d t h e n f o r g e t a b o u t x a n d p r e t e n d o n l y t h a t i s k n o w n t o h a v e o b t a i n e d . ) R e s e a r c h e r sw o r k i n g w i t h a n c i l l a r i t y a t t e m p t t o d e f i n e " g o o d " a n c i l l a r y s t a t i s t i c s t o c o n d i t i o n u p o n , b u t , a s m e n t i o n e d e a r l i e r , t h e r e a p p e a r t o b e n o c o m p l e t e l ys a t i s f a c t o r y d e f i n i t i o n s . A l s o , a n c i l l a r y s t a t i s t i c s d o n o t e x i s t i n m a n y
7/27/2019 Berger the Likelihho Principle 1988 Book
31/252
C O N D I T I O N I N G 1 7
s i t u a t i o n s w h e r e i t s e e m s i m p o r t a n t t o c o n d i t i o n , a s t h e f o l l o w i n g s i m p l ee x a m p l e s h o w s .
E X A M P L E 8 . S u p p o s e = [ 0 , ] ) , a n d' w i t h p r o b a b i l i t y 1
X =0 w i t h p r o b a b i l i t y .
(An i n s t r u m e n t m e a s u r e s e x a c t l y , b u t w i l l e r r o n e o u s l y g i v e a z e r o r e a d i n gw i t h p r o b a b i l i t y e q u a l t o . ) C o n s i d e r t h e c o n f i d e n c e p r o c e d u r e C ( x ) = { x }( t h e p o i n t x ) . H e r e P Q ( C ( X ) c o n t a i n s ) = 1 . I t i s c l e a r , h o w e v e r , t h a t o n ew a n t s t o c o n d i t i o n o n { x : x > 0 } , s i n c e C ( x ) = { } f o r s u r e i f x > 0 . B u tt h e r e i s n o a n c i l l a r y s t a t i s t i c w h i c h p r o v i d e s s u c h a c o n d i t i o n i n g .
I n s i t u a t i o n s s u c h a s E x a m p l e s 3 , 4 , 5 , a n d 6 , t h e s e l e c t i o n o f ap a r t i t i o n f o r a c o n d i t i o n a l c o n f i d e n c e a n a l y s i s s e e m s q u i t e a r b i t r a r y . K i e f e r( 1 9 7 7 ) s i m p l y s a y s t h a t t h e c h o i c e o f a p a r t i t i o n m u s t u l t i m a t e l y b e l e f t t ot h e u s e r , a l t h o u g h h e d o e s g i v e c e r t a i n g u i d e l i n e s . T h e d e v e l o p m e n t o fi n t u i t i o n o r t h e o r y f o r t h e c h o i c e o f a p a r t i t i o n s e e m s v e r y h a r d , h o w e v e r( s e e a l s o K i e f e r ( 1 9 7 6 ) , B r o w n ( 1 9 7 7 ) , a n d B e r g e r ( 1 9 8 4 c ) ) .
E v e n m o r e d i s t u r b i n g a r e e x a m p l e s , s u c h a s E x a m p l e 4 ( b ) , w h e r e i ts e e m s i m p o s s i b l e t o p e r f o r m t h e i n d i c a t e d s e n s i b l e t e s t a n d r e p o r t c o n d i t i o n a le r r o r p r o b a b i l i t i e s r e f l e c t i n g t h e t r u e u n c e r t a i n t y w h e n x = 1 i s o b s e r v e d .( A t h r e e p o i n t s c a n n o t b e p a r t i t i o n e d i n t o t w o n o n d e g e n e r a t e s e t s , a n d o n ad e g e n e r a t e s e t t h e c o n d i t i o n a l e r r o r p r o b a b i l i t y m u s t b e z e r o o r o n e . ) A n yt h e o r y w h i c h c a n n o t h a n d l e s u c h a s i m p l e e x a m p l e i s c e r t a i n l y s u s p e c t .
T h e s i t u a t i o n f o r e s t i m a t e d c o n f i d e n c e t h e o r y i s m o r e a m b i g u o u s ,b e c a u s e i t h a s n o t b e e n v e r y e x t e n s i v e l y s t u d i e d . I n p a r t i c u l a r , t h e c h o i c eo f a p a r t i c u l a r e s t i m a t e d c o n f i d e n c e o r r i s k i s v e r y d i f f i c u l t , i n a ll b u t t h es i m p l e s t s i t u a t i o n s . A n d , i n s i t u a t i o n s s u c h a s E x a m p l e s 3 a n d 4 ( b ) ,e s t i m a t e d c o n f i d e n c e f u n c t i o n s w i l l h a v e c e r t a i n u n d e s i r a b l e p r o p e r t i e s . I nE x a m p l e 3 , f o r i n s t a n c e , a n y e s t i m a t e d e r r o r p r o b a b i l i t y , ( x ) , w h i c h i s
7/27/2019 Berger the Likelihho Principle 1988 Book
32/252
7/27/2019 Berger the Likelihho Principle 1988 Book
33/252
7/27/2019 Berger the Likelihho Principle 1988 Book
34/252
2 0 T H E L I K E L I H O O D P R I N C I P L E
E X A M P L E 9. S u p p o s e Y , Y 2 , . . .are . i . d . B e r n o u l l i ( ) r a n d o m v a r i a b l e s . Ine x p e r i m e n t E . . ,af i x e d s a m p l e s i z e of 12o b s e r v a t i o n s is d e c i d e d u p o n , and the
1 2s u f f i c i e n t s t a t i s t i cX.= JY.t u r n s o u t to be x,=9. Ine x p e r i m e n t E ~ , it1 i=l i s d e c i d e dto t a k e o b s e r v a t i o n s u n t i lat o t a l of3z e r o e s h a s b e e n o b s e r v e d ,a t w h i c h p o i n t t h e s u f f i c i e n t s t a t i s t i cX2= Y . t u r n s o u t to 9. T h e d i s t r i b u t i o nof X^in. is b i n o m i a l w i t h d e n s i t y
w h i c h f o r x ,= 9y i e l d s the l i k e l i h o o d f u n c t i o n
* g ( Q ) = ( g 2 ) 9 ( l ) 3 T h e d i s t r i b u t i o nofX2in 2is n e g a t i v e b i n o m i a l w i t h d e n s i t y
f 2 ( 2 )=( X f ) X 2 ( I ) 3 ,which for x2 = 9y i e l d s the l i k e l i h o o d f u n c t i o n
= 191) 9 l )3.
I n t h i s s i t u a t i o n , t h eLP a y s t h a t ( i ) f o r e x p e r i m e n t E.a l o n e ,t h e i n f o r m a t i o n a b o u tis c o n t a i n e d s o l e l y in & Q ( ) ;and( i i ) s i n c e & i ( ) and2& ( ) a r e p r o p o r t i o n a las f u n c t i o n sof, the i n f o r m a t i o n a b o u tin e x p e r i
m e n t sE1andE 2is i d e n t i c a l .T h e s e c o n c l u s i o n s a r e , ofc o u r s e , at o d d s w i t h f r e q u e n t i s t
r e a s o n i n g . T h e b i n o m i a l and e g a t i v e b i n o m i a l d i s t r i b u t i o n s w i l l t e n dtog i v ed i f f e r e n t f r e q u e n t i s t m e a s u r e s . For i n s t a n c e ,ao n e t a i l e d s i g n i f i c a n c e t e s to f H Q : =Yw i l l g i v e s i g n i f i c a n c e l e v e l s of =. 0 7 3 0 a n d =. 0 3 3 8 in the
7/27/2019 Berger the Likelihho Principle 1988 Book
35/252
7/27/2019 Berger the Likelihho Principle 1988 Book
36/252
7/27/2019 Berger the Likelihho Principle 1988 Book
37/252
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 1 . 2
m o s t f a m i l i a r s t a t i s t i c a l s i t u a t i o n . O f t e n , h o w e v e r , t h e r e a r e u n k n o w n s w h i c ha r e r e l e v a n t t o a s t a t i s t i c a l p r o b l e m b u t w h i c h d o n o t d i r e c t l y a f f e c t t h e d i s t r i b u t i o n o f X . O n e e x a m p l e i s p r e d i c t i o n , i n w h i c h i t i s d e s i r e d t o p r e d i c ta n u n k n o w n r a n d o m v a r i a b l e Z , a f t e r o b s e r v i n g X . O t h e r e x a m p l e s a r i s e i nd e s i g n a n d s e q u e n t i a l a n a l y s i s p r o b l e m s , w h e r e a s y e t u n o b s e r v e d d a t a ca na f f e c t t h e d e c i s i o n t o b e m a d e . E x a m p l e s a r e g i v e n i n S e c t i o n 3 . 5 .
I n g e n e r a l , t h e r e f o r e , t h e L P s h o u l d b e f o r m u l a t e d i n s u c h a w a yt h a t c o n s i s t s o f a l l u n k n o w n v a r i a b l e s a n d p a r a m e t e r s t h a t a r e r e l e v a n t t ot h e s t a t i s t i c a l p r o b l e m . ( A n y a t t e m p t t o p r e c i s e l y d e f i n e " r e l e v a n t t o t h es t a t i s t i c a l p r o b l e m " w o u l d i n v o l v e b o t h d e c i s i o n t h e o r y a n d m o d e l f o r m u l a t i o n ,a n d l e a d u s t o o f a r a s t r a y . ) T h e m a j o r d i f f i c u l t y w i t h w o r k i n g i n s u c h g e n e r a l i t y i s t h a t o f d e f i n i n g w h a t i s t h e n m e a n t b y a l i k e l i h o o d f u n c t i o n f o r ( c f . B a y a r r i , D e G r o o t , a n d K a d a n e ( 1 9 8 7 ) ) . W e h a v e o p t e d f o r d i s c u s s i n g t h i sg e n e r a l s i t u a t i o n o n l y i n S e c t i o n 3 . 5 , t h o u g h w e b e l i e v e t h a t v i r t u a l l y a lli s s u e s r a i s e d f o r t h e s p e c i a l c a s e o f b e i n g t h e m o d e l p a r a m e t e r a l s o a p p l yt o a p p r o p r i a t e f o r m u l a t i o n s o f t h e g e n e r a l s i t u a t i o n . I n a n y c a s e , i t i si m p o r t a n t t o k e e p i n m i n d t h e q u a l i f i c a t i o n t h a t m u s t c o n t a i n a l l u n k n o w n sr e l e v a n t t o t h e p r o b l e m f o r t h e L P t o b e v a l i d i n i ts s i m p l e f o r m .
A s e c o n d q u a l i f i c a t i o n f o r t h e L P i s t h a t i t o n l y a p p l i e s f o r af u l l y s p e c i f i e d m o d e l { f f l} I f t h e r e is u n c e r t a i n t y i n t h e m o d e l , a n d if o n ed e s i r e s t o g a i n i n f o r m a t i o n a b o u t w h i c h m o d e l i s c o r r e c t , t h a t u n c e r t a i n t y m u s tb e i n c o r p o r a t e d i n t o t h e d e f i n i t i o n o f .
A t h i r d q u a l i f i c a t i o n i s t h a t , i n a p p l y i n g t h e L P t o t w o d i f f e r e n te x p e r i m e n t s , i t i s i m p e r a t i v e t h a t b e t h e s a m e u n k n o w n q u a n t i t y i n e a c h .T h u s , i n E x a m p l e 9 , w e a s s u m e d t h a t r e p r e s e n t e d t h e s a m e s u c c e s s p r o b a b i l i t yi n e i t h e r t h e b i n o m i a l o r n e g a t i v e b i n o m i a l e x p e r i m e n t . I n a p p l y i n g t h e L P t ot w o d i f f e r e n t e x p e r i m e n t s , w e a l s o r e q u i r e t h a t t h e c h o i c e o f a n e x p e r i m e n t b en o n i n f o r m a t i v e ( e . g . i m p l e m e n t e d b y a c h a n c e m e c h a n i s m n o t i n v o l v i n g ) ;
7/27/2019 Berger the Likelihho Principle 1988 Book
38/252
7/27/2019 Berger the Likelihho Principle 1988 Book
39/252
7/27/2019 Berger the Likelihho Principle 1988 Book
40/252
7/27/2019 Berger the Likelihho Principle 1988 Book
41/252
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 5
e x p e r i m e n t . ( T h e " s t r u c t u r a l t h e o r y " o f F r a s e r a n d t h e " p i v o t a l t h e o r y " o fB a r n a r d d e e m a d d i t i o n a l i n f o r m a t i o n r e l a t i n g X , , a n d t h e r a n d o m n e s s t o b ei m p o r t a n t , h o w e v e r . T h i s i s s u e w i l l b e d i s c u s s e d in S e c t i o n s 3 . 6 . 4 a n d 3 . 7 . )
T h e o u t c o m e o f t h e e x p e r i m e n t i s t h e d a t a X = x , a n d f r o m E a n d xw e a r e t o i n f e r o r c o n c l u d e s o m e t h i n g a b o u t ( o r a b o u t s o m e t h i n g r e l a t e d t o ) . F o l l o w i n g B i r n b a u m ( 1 9 6 2 a ) , w e w i l l c a l l t h i s i n f e r e n c e , c o n c l u s i o n , o rr e p o r t t h e e v i d e n c e a b o u t a r i s i n g f r o m E a n d x , a n d w i l l d e n o t e t h i s b yE v ( E , x ) . W e p r e s u p p o s e n o t h i n g a b o u t w h a t t h i s e v i d e n c e i s ; it c o u l d ( a t t h i ss t a g e ) b e a n y s t a n d a r d m e a s u r e o f e v i d e n c e , o r s o m e t h i n g e n t i r e l y n e w . ( S i n c eE i s a n a r g u m e n t , i t c o u l d c e r t a i n l y b e a f r e q u e n t i s t m e a s u r e . ) A l s o , w e d on o t p r e c l u d e t h e p o s s i b i l i t y t h a t E v ( E , x ) d e p e n d s o n " o t h e r i n f o r m a t i o n , " s u c ha s p r i o r i n f o r m a t i o n a b o u t , o r a l o ss f u n c t i o n i n a d e c i s i o n p r o b l e m . T h ef o c u s w i l l b e o n t h e m a n n e r i n w h i c h t h e " r e p o r t " E v ( E , x ) s h o u l d d e p e n d o n E a n dx . ( D a w i d ( 1 9 7 7 ) p r e f e r s t o t a l k a b o u t m e t h o d s o f i n f e r e n c e b a s e d o n E a n d x ,a n d p r i n c i p l e s w h i c h t h e s e m e t h o d s s h o u l d s a t i s f y . I n a s e n s e , b y l e t t i n gE v ( E , x ) d e n o t e w h a t e v e r c o n c l u s i o n o n e i s g o i n g t o r e p o r t , w e a r e a l s o t a k i n gt h i s v i e w , w h i l e k e e p i n g B i r n b a u m ' s n o t a t i o n . ) A s o n e f i n a l p o i n t , E v ( E , x )c o u l d b e a c o l l e c t i o n o f " e v i d e n c e s " a b o u t , o b v i a t i n g t h e c r i t i c i s m t h a t t h eL P i s b a s e d o n t h e a s s u m p t i o n t h a t a s i n g l e m e a s u r e o f e v i d e n c e e x i s t s .
T h e C o n d i t i o n a l i t y P r i n c i p l e e s s e n t i a l l y s a y s t h a t , i f an e x p e r i m e n t i s s e l e c t e d b y s o m e r a n d o m m e c h a n i s m i n d e p e n d e n t o f , t h e n o n l y t h ee x p e r i m e n t a c t u a l l y p e r f o r m e d is r e l e v a n t . ( T h e s e l e c t i o n m e c h a n i s m i sa n c i l l a r y , s o t h i s i s a v e r s i o n o f c o n d i t i o n i n g o n a n a n c i l l a r y s t a t i s t i c . )T h e g e n e r a l c o n d i t i o n a l i t y p r i n c i p l e i s n o t n e e d e d h e r e . I n d e e d w e n e e d o n l yt h e f o l l o w i n g c o n s i d e r a b l y w e a k e r p r i n c i p l e , n a m e d b y B a s u ( 1 9 7 5 ) .
W E A K C O N D I T I O N A L I T Y P R I N C I P L E ( W C P J . S u p p o s e t h e r e a r e t w o e x p e r i m e n t s1 2
E = ( X , , { f } ) a n d E 9 = ( X 9 , , ( f Q } ) w h e r e o n l y t h e u n k n o w n p a r a m e t e r n e e d b e c o m m o n t o t h e t w o e x p e r i m e n t s . C o n s i d e r t h e m i x e d e x p e r i m e n t E * ,w h e r e b y J = 1 o r 2 i s o b s e r v e d , e a c h h a v i n g p r o b a b i l i t y ^ ( i n d e p e n d e n t o f ,X , , o r X p j j a n d e x p e r i m e n t E , i s t h e n p e r f o r m e d . F o r m a l l y , E * = ( X * , , f } ) >
7/27/2019 Berger the Likelihho Principle 1988 Book
42/252
2 6 T H E L I K E L I H O O D P R I N C I P L E
w h e r e X * = ( J , X j ) a n d f * ( ( j , X j ) ) = \ f ( j ) hen>E v ( E * , ( j , . ) ) = E v ( E . , . ) ,j J J
i . e . , t h e e v i d e n c e a b o u t f r o m E * i s j u s t t h e e v i d e n c e f r o m t h e e x p e r i m e n ta c t u a l l y p e r f o r m e d .
T h e W C P i s n o t h i n g b u t a f o r m a l i z a t i o n o f E x a m p l e 2 , a n d h e n c e i se s s e n t i a l l y d u e t o C o x ( 1 9 5 8 ) . I t is h a r d to d i s b e l i e v e t h e W C P , y e t , a sm e n t i o n e d a f t e r E x a m p l e 2 , e v e n t h e W C P a l o n e h a s s e r i o u s c o n s e q u e n c e s .
T u r n i n g f i n a l l y t o t h e f a m i l i a r c o n c e p t o f s u f f i c i e n c y , w e s t a t et h e f o l l o w i n g w e a k v e r s i o n ( n a m e d b y D a w i d ( 1 9 7 7 ) ) .W E A K S U F F I C I E N C Y P R I N C I P L E ( W S P ) . C o n s i d e r a n e x p e r i m e n t E = ( X , , { f ) ) , a n ds u p p o s e T ( X ) i s a s u f f i c i e n t s t a t i s t i c f o r . T h e n , i f T ( x ^ ) = T ( x 2 ) ,E v t E ^ ) = E v ( E , x 2 ) .
T h e L P w il l b e s e e n t o f o l l o w d i r e c t l y f r o m t h e W C P a n d W S P . Av a r i e t y o f a l t e r n a t e p r i n c i p l e s a l s o l e a d t o t he L P ( c f . B a s u ( 1 9 7 5 ) , D a w i d( 1 9 7 7 ) , B a r n d o f f N i e l s e n ( 1 9 7 8 ) , B e r g e r ( 1 9 8 4 a ) , B h a v e ( 1 9 8 4 ) , a n d E v a n s ,F r a s e r , a n d M o n e t t e ( 1 9 8 5 c , 1 9 8 6 ) ) . T h e W C P a nd W S P a r e t h e m o s t f a m i l i a r ,h o w e v e r . A n o t h e r p r o m i n e n t p r i n c i p l e is " M a t h e m a t i c a l E q u i v a l e n c e , " g i v e n i nB i r n b a u m ( 1 9 7 2 ) . T h i s p r i n c i p l e i s a w e a k v e r s i o n o f t h e s u f f i c i e n c y p r i n c i p l e ,s t a t i n g t h a t i f , i n a g i v e n e x p e r i m e n t E , f ( x , ) = f ( x 0 ) f o r a l l , t h e nE v ( E , X j ) = E v ( E , x 2 ) . O n e c o u l d b a s e t h e L P o n m a t h e m a t i c a l e q u i v a l e n c e , p l u sa m i n o r g e n e r a l i z a t i o n o f t h e W C P . T h e w e a k e n i n g o f s u f f i c i e n c y i s c a r r i e d tot h e u l t i m a t e i n E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) , w h i c h d e r i v e s t h e L P s o l e l yf r o m a g e n e r a l i z e d v e r s i o n o f t h e c o n d i t i o n a l i t y p r i n c i p l e .3 . 3 .2 A x i o m a t i c D e v e l o p m e n t
T h e f o r m a l s t a t e m e n t o f t h e L P i s a s f o l l o w s .F O R M A L L I K E L I H O O D P R I N C I P L E . C o n s i d e r t w o e x p e r i m e n t s E , = { X , , , { f 1 } ) a n dE = ( X o s { f g } ) > w h e r e i s t h e s a m e q u a n t i t y i n e a c h e x p e r i m e n t . S u p p o s et h a t f o r t h e p a r t i c u l a r r e a l i z a t i o n s x a n d x | f r o m E , a n d E 2 r e s p e c t i v e l y ,
7/27/2019 Berger the Likelihho Principle 1988 Book
43/252
T H E L I K E L I H O O D P R I N C I P L E AND G E N E R A L I Z A T I O N S 2 7* x * ( ) = * )x 1 x2
cons tan t c ( i . e . , f ( x ) = c f .( x ) / or ) . T/zercE v ( E x * ) = E v ( E2 , x * ) .
LIKELIHOOD PRIN IPLECOROLLARY.If E = (X, , {fQ}) is an experiment, thenEv E,x) should depend on Eand x only through ) .
THEOREM 1 Birnbaum 1962 a) ). The Formal Likelihood Principle follows from theWCPand the SP. The converse is also true.Proof I f E1 and E,, are the two experiments about , consider the mixede x p e r i m e n t E* as defined in the WCP. From the WCP weknow tha t( 3 . 3 . 1 ) Ev E*, j ,x . . ) ) = Ev Ej f Xj ) .
N e x t , t h i n k i n g s o l e l yof i t h r a n d o m o u t c o m e ( J , X j ) , c o n s i d e rt h e s t a t i s t i c
f ( l . x f ) ifJ =2,X2= xT ( J . X j )=
( J , X j ) o t h e r w i s e .( T h u sthe twoo u t c o m e s ( l , x ^ )and( 2 , x ^ ) r e s u l tinthes a m e v a l u eofT.)T isa s u f f i c i e n t s t a t i s t i c for . T h i sisc l e a r , s i n c e
P ( X *=( j , X j ) | T = tf( l , x | ) )=1 if(j.xJ= t
0 o t h e r w i s e ,and
P ( X *=( l , x f ) | T= 1 P ( X *=( 2 , x | ) | T=( l . x f ) )
f ( x ) +\a l lof h i c h arei n d e p e n d e n tof. The WSP t h u s i m p l i e s t h a t( 3 . 3 . 2 ) E v ( E * , ( l , x f ) )=E v ( E * , ( 2 , x * ) ) .
7/27/2019 Berger the Likelihho Principle 1988 Book
44/252
2 8 T H E L I K E L I H O O D P R I N C I P L E
C o m b i n i n g ( 3 . 3 . 1 ) a n d ( 3 . 3 . 2 ) e s t a b l i s h e s t h e r e s u l t .T o p r o v e t h a t t h e L P i m p l i e s t h e W C P , o b s e r v e t h a t , f o r E * ,
T h i s i s c l e a r l y p r o p o r t i o n a l t o f ? ( x , ) , t h e l i k e l i h o o d f u n c t i o n i n E . w h e n x . j j j
i s o b s e r v e d , s o t h e L P i m p l i e s t h a tE v ( E * , ( j , X j ) ) = E v E j . X j ) .
T o p r o v e t h a t t h e L P i m p l i e s t h e W S P , i t s u f f i c e s t o n o t e t h a t , i fT (x j ) = T ( x 2 ) i n a n e x p e r i m e n t f o r w h i c h T i s s u f f i c i e n t , t h e n x 1 a n d x ^ h a v ep r o p o r t i o n a l l i k e l i h o o d f u n c t i o n s . ||P r o o f o f t h e L P C o r o l l a r y . F o r g i v e n x * X , d e f i n e
1 i f X = x *Y =
0 i f X f x * ,a n d n o t e t h a t Y h a s d i s t r i b u t i o n g i v e n b y( 3 . 3 . 3 ) f j ( l ) = f ( x * ) = l f j ( ) .F o r t h e e x p e r i m e n t E * o f o b s e r v i n g Y , i t f o l l o w s f r o m t h e L P t h a t
E v ( E , x * ) = E v ( E * , l ) .B u t E * , a n d h e n c e E v ( E * , l ) , d e p e n d o n l y o n f . ( x * ) = v * ( ) ( u s i n g ( 3 . 3 . 3 ) ) . ||
T h e a b o v e r e s u l t s a r e w o r t h d w e l l i n g u p o n f o r a m o m e n t . T h e L P ise x t r e m e l y r a d i c a l f r o m t h e v i e w p o i n t o f c l a s s i c a l s t a t i s t i c s , a s w i l l b e s e e ni n C h a p t e r 4 . Y e t t o r e j e c t t h e L P , o n e m u s t l o g i c a l l y r e j e c t e i t h e r t h e W C Po r t h e W S P . B u t t h e W S P i s , i t s e l f , a c o r n e r s t o n e o f c l a s s i c a l s t a t i s t i c s , a n dt h e r e i s n o t h i n g i n s t a t i s t i c s a s " o b v i o u s " as t h e W C P ( o r E x a m p l e 2 ) .
3 .4 G E N E R A L I Z A T I O N S B E Y O N D T H E D I S C R E T E C A S EB a s u ( 1 9 7 5 ) a n d o t h e r s h a v e a r g u e d t h a t t h e s a m p l e s p a c e x i n a n y
p h y s i c a l l y r e a l i z a b l e e x p e r i m e n t m u s t b e f i n i t e , d u e t o o u r i n a b i l i t y t om e a s u r e w i t h i n f i n i t e p r e c i s i o n . T h i s s u g g e s t s t h a t t h e L i k e l i h o o d P r i n c i p l ef o r d i s c r e t e e x p e r i m e n t s ( as i n S e c t i o n 3 . 3 ) is a l l t h a t o n e n e e d s . W e a r e
7/27/2019 Berger the Likelihho Principle 1988 Book
45/252
THE LIKELIHOOD PRINCIPLE AND GENERALIZATIONS 2 9
p h i l o s o p h i c a l l y i n a g re e m e n t w i t h t h i s .On th e o th e r han d , co n t in u o us and o th e r more g en e r a l p r o b a b i l i ty
d i s t r i b u t i o n s a r e e n or m ou sly u s e f u l in s i m p l i f y i n g s t a t i s t i c a l c o m p ut a t io nsa nd i n p r o v id i n g n u m e r i c a l a p p r o x im a t i o n s w hic h a r e o f t e n q u i t e a c c u r a t e . I ti s p o s s i b le f o r t h e l i k e l i h o o d f u n c t i o n f o r a c o nt in uo u s m ode l t o d i f f e rs t r i k i n g l y f ro m t h a t o f t h e d i s c r e t e m ode l i t i s i n t e n d e d t o a p p r o x im a t e , soit is not obvious t h a t t h e v a l i d i t y o f t h e LP in d i s c r e t e p ro ble m s e x t e nd s t oi t s v a l i d i t y i n t h e a p p r o x im a t in g c o nt in uo us p r o ble m s . I n a ny c a s e , e x t e n s io no f th e LP to more g en e r a l s i tu a t i o n s can o n ly s t r e n g th e n i t s case . Such ane x t e n s io n i s o u r t a s k i n t h e p r e s e n t s e c t i o n .
As i n S e c t io n 3 . 3 , an e x pe r im e n t E = ( X , , { P J ) w i l l be u nd e rs to odto inv olv e t h e o b s e r v a t io n o f t h e random v a r i a b l e X , h a vin g p r o b a b i l i t y d i s t r i b u t i o n P on X9 . ( I t w i l l n o t be n ecessar y to assume th e ex i ste n ce o fa d e n s i t y . ) T he r e i s , u n a vo i d a b ly , m e a s u r e t h e o r e t i c m a t he m a t ic s i n t h i ss e c t i o n , b ut t h e s e c t i o n ca n be s k ip p e d , i f d e s i r e d , w i t h o u t a ny e s s e n t i a ll o s s o f c o n t i n u i t y .
The sample space w i l l b e assumed to b e a l o cal l y co m p a ctH au sd o r f f space whose to p o l o g y ad mi ts a co u n tab l e b ase ( LCCB sp ace , f o r sh o r t ) ,and the P w i l l b e assumed to b e B o r e l m easur es . Of co u r se, X o f te n ar i se s asa n % v a l u e d r a n d om v a r i a b l e o n a p r o b a b i l i t y s p a c e ( , 3 , { } ) e q u i p p e d w i t hua f a m i l yofp r o b a b i l i t y m e a s u r e s i n d e x e dby 6 . S u c h u n d e r l y i n g s t r u c t u r ew i l lnot ber e l e v a n t in oura n a l y s i s , h o w e v e r .3 . 4 . 1 D i f f i c u l t i e s in theN o n d i s c r e t e C a s e
I nane x p e r i m e n t E = (X, ,{ P Q} )for w h i c h t h e r e is an x Xs a t i s f y i n g P {x}) = 0 for e v e r y , it isd i f f i c u l t toa s s i g n any p a r t i c u
l a r m e a n i n g to n E v ( E , x ) " . Fore x a m p l e , B a s u ( 1 9 7 5 ) and J o s h i ( 1 9 7 6 ) h a v eo b s e r v e d t h a t an a i v e a p p l i c a t i o n ofB i r n b a u m ' s ( 1 9 6 2 a ) s u f f i c i e n c y p r i n c i p l ew o u l d s u g g e s t fors u c h an xt h a t E v ( E , x ) =E v ( E , y ) fore v e r yy X> s i n c e them a p T: x- w h i c h t a k e s x o n t o y and l e a v e s all o t h e r p o i n t s ( i n c l u d i n g y )f i x e d is s u f f i c i e n t for . T h i s isp a r t i c u l a r l y d i s t u r b i n g forc o n t i n o u s
7/27/2019 Berger the Likelihho Principle 1988 Book
46/252
30 THE LIKELIHOOD PRINCIPLE
d i s t r i b u t i o n s , s i n c e t h e n PQ ( { x } ) = 0 f or every x e andeve r y e ; B i r n b a u m ' ss u f f i c i e n c y p r i n c i p l e t h en s ug g e st s t h a t a l l p o s s i b l e o b s e r v a t i o n s l e n dp r e c i s e l y the same evidence and t h e r e f o r e n on e) a b o u t .
T h e u n i q u e s p e c i f i c a t i o n of a l i k e l i h o o d f u n c t i o n c auses s i m i l a rp r o b l e m s . I f t h e r e is no s i n g l e f i n i t e m easure v onz whose n u l l se t sc o i n c i d e w i t h t h o s e B o r e l s e t s N fo r wh i c h P. N) = 0 fo r al l 6 , t h e n nol i k e l i h o o d f u n c t i o n e x i s t s . T h i s is the u s u al s t a t e of a f f a i r s in nonp a r a m e t r i c p ro b le m s ( r e c a l l t h a t c o u l d be an a r b i t r a r y i nd e x set ) and can evena r i s e i n s i m p l e p a r a m e t r i c e x a m p l e s ; f o r e x a m p l e , PQA) =j //\dx+ j U ) > =X = [ 0 , 1 ] , d e s c r i b e s an e x p e r i m e n t in wh i c h X= w i t h p r o b a b i l i t y j andi s o t h e r w i se u n i f o r m l y d i s t r i b u t e d o ve r the u n i t i n t e r v a l ; no f i n i t e m easurev domina tes {P } , and no l i k e l i h o o d f u n c t i o n e x i s t s . ( I n c i d e n t a l l y , t h i s seemst o be a s o u r c e of c o n f u s i o n in c e r t a i n c o u n t e r ex a m p l e s to the LP such as thesecond examp le in S e c t i o n 2.5 of B ir n b au m ( 1 9 6 9 ) . )
Even in p r ob l ems whe r e t he r e is a measure v w i t h the i n d i c a t e dp r o p e r t i e s , the Ra don N ik odym d e r i va t i v es
U ) = f f l x) = P fldx)/v dx)are determined only up to sets of vmeasure zero; these functions of could bespecif ied in an entir ely arbit rary manner for al l x in any set N c z withv N) = 0. One way to salvage a likelihood princ iple in the face of suchambiguity is to specify a part icular version of P dx)/ v dx) for each ; for
example, in case a valmost everywhere) continuous density exists we couldset = {open neighborhoods of x }and put
U ) = in f sup P_ U)/v U))ve U
UcVf o r x in the support of v, i ) = 0 otherwise.
By res tr ic t ing our attent ion to valmost everywhere) continuousdensit ies, continuous suff ic ient s tat ist ics, etc. we could develop versions ofthe conditionality, sufficiency, and likelihood principles very similar tothose in the discrete setting.
7/27/2019 Berger the Likelihho Principle 1988 Book
47/252
7/27/2019 Berger the Likelihho Principle 1988 Book
48/252
3 2 THELIKELIHOOD PRINC IPLE
a n o t h e r LCCB s p a c e r . Thes t a t i s t i cT de t e rm ine s a f a m i l y {P } ofB o r e lmeasures on by
P I A = P A ) ) ,a n d h e n c eane x p e r i m e n tE =( T , j, { P } ) U n l e s sTis 11 we e x p e c t (ing e n e r a l ) t h a tE w i l l t e l lusl e s s a b o u tt h a n E,s i n c e d i f f e r e n t o u t c o m e sx w i t h p o s s i b l y d i f f e r e n t e v i d e n t i a l i m p o r t can be m a p p e d o n t o the s a m eT ( x ) e x . The e x c e p t i o n a l c a s e is t h a t in w h i c hTis s u f f i c i e n t .D E F I N I T I O N . For the x p e r i m e n tE s u p p o s e t h e r e e x i s t saf a m i l y { g . :t J }o f B o r e l p r o b a b i l i t y m e a s u r e s on s a t i s f y i n g
P ( A ) = / g t ( A ) p J ( d t ) = / g ( ) ( A ) P ( d x )for all Borel sets A c t ThenTis called sufficient11 (or sometimes11sufficient for ).
N o t e t h a t g. is notp e r m i t t e d to depend upon ;o t h e r w i se g. = PQwould a l wa y s work . Anyo n e t o o n e m e a s u r a b le m a p p in gT is s u f f i c i e n t ; j u s tl e t g tbe a poi nt ma s s at T t )X
The Su f f ic i en cy P r in c i p l e makes prec ise the n o t io n t h a t T(x) in jt e l l s as much about as x in E;
SUFFICIENCY PRINCIPLE (SP). If T: X + J is sufficient, thenE v E , x ) =E v E T , T x ) ) fo r {P } a. e. x X
Again we mayd e l e t e the im po ss ible outcomes whenx is countable toremove the {PQ} - a.e. q u a l i f i c a t i o n andconclude tha t Ev E,x) = E v E , y )w h e n e v e ras u f f i c i e n t s t a t i s t i cTs a t i s f i e s T ( x )=T ( y ) , and sor e c o v e r thed i s c r e t eWSP ofS e c t i o n 3 . 3 . 1 .3 . 4 . 3 . TheR e l a t i v e L i k e l i h o o d P r i n c i p l e
L e t E ]=(X.|, ,{ P ' } ) and Ep=( X 2 , ,{P })be twoe x p e r i m e n t sa n d s u p p o s e ( f o r m o t i v a t i o n a l p u r p o s e s ) t h a t e a c h a d m i t sal i k e l i h o o d f u n c t i o n ,
7/27/2019 Berger the Likelihho Principle 1988 Book
49/252
THE LIKELIHOOD PRINCIPLE ANDGENERALIZATIONS 33
i . e . a f ini t e m e a s u r e v. on the s a m p l e s p a c e Z and a f a m i l y { f l ) } of1 I i n t e g r a b l e f u n c t i o n s s a t i s f y i n gP Q ( A )=/ f j ( x ) v . ( d x ) , A c z . .
T h e L i k e l i h o o d P r i n c i p l e ( w e r eit toh o l d h e r e ) w o u l d a s s e r t t h a tE v ( E 1 , x 1 )=E v ( E 2 , x 2 )
w h e n e v e r f n ( x )=c f Q ( x o )fora l lands o m e c o n s t a n t c =c ( x , , x 0 ) not1 2d e p e n d i n g on ,i . e . w h e n e v e r t h e r e l a t i v e l i k e l i h o o dc =f Q ( x ) / f Q ( x o ) d o e s I D Cnot dependon . Ourfreedom to s p e c i f y ^ Q X ^ ) a r b i t r a r i l y w he ne ve rv ^ x ^ } ) = 0 makes i t c l e a r t h a t t h i s p r i n c i p l e n e e d s r e f o r m u l a t i o n b e f o r e it iss u i t a b l e for experiments with uncountable sample spaces. (However, at p o in t sx, and x which areatoms of v, and v> r e s p e c t i v e l y , the LP is r e a s o n a b l e ,andca n beshown to fol low from theWCPand SP as in S e c t i o n3 . 3 . )
To develop a s u i t a b l e g e ne r a l p r i n c i p l e , weg e n e r a l iz e theconceptt h a t ther e l a t i v e l ik e l ih o o d of x j and x2 is independent of . B a s i c a l l y , i fa mapping exists between twosubsets of andX for which the RadonNikodymd e r i v a t i v e of the induced measure w ith resp ec t to thee xist in g measure on, say,
Z , ) is independent of , then we cane s t a b l is h an equ ivalen ce of evidencebetween thecorresponding obse rvat ions in thesu b set s . Thereasons forg e n e r a l i z i n g the LP in t h i s d i r e c t i o n are : i ) I t can be s t a t e d in g r e a tg e n e r a l i t y , w i t h o u t r e q u i r i n g m o d e l s or d e n s i t i e s ; i i ) I t w i l l beshown tofol low from theWCPand SP, as did the LP; and i i i ) I t , in t u r n , can beshownt o imply in s u b s ta n t ia l g e n e r a l it y ) theS t o p p in g Ru le P r in cip le and CensoringP r i n c i p l e , b e s i d e s h a v i n g d i r e c t l y i m p o r t a n t i m p l i c a t i o n s of i t s own. Them a j o r l i m i t a t i o n of the RLP(comparedto the LP) is t h a t i t does not provideany such convenient summarization of evidence as the l i k e l i h o o d f u n c t i o n ( w h ic hneednote x i st in theg e n e r a l c a s e ) .RELATIVE LIKELIHOOD PRINCIPLE RLP) . Let : Uj + l be a Borel bimeasurableone toone mapping from U, c %..onto IL
7/27/2019 Berger the Likelihho Principle 1988 Book
50/252
7/27/2019 Berger the Likelihho Principle 1988 Book
51/252
T H E L I K E L I H O O D P R I N C I P L E A ND G E N E R A L I Z A T I O N S 35f ( x , ) v a n i s h e s for all, sothe" { P } a . e . " q u a l i f i c a t i o n is
U I u n n e c e s s a r y ) . ||T H E O R E M 3.TheCPan dt h e SP t o g e t h e r i m p l y theRLP.P r o o f . Let E,andEp betwoe x p e r i m e n t s , ab i m e a s u r a b l e m a p p i n g f r o ma B o r e ls e t U^e %1 o n t o U ^ ^ ,and : U^ >( 0,) a m e a s u r a b l e f u n c t i o n s a t i s f y i n g
P < A )= / . ] C V c ( x ) ] p J ( d x ) (A)f o r a l l Bore l A c I L , a l l . Le t E* be t he mi x ture of E, and E2 , andd e f i n e a mapping T: X * X * by
) i f i = 1 a nd x]T ( i , x . ) =
]
e l s e .T h i s d e t e r m i n e sanewe x p e r i m e n t E* T= ( T , X*, { P } ) , w h e r e P I ( A )=
F i r s twe s h o w t h a t T is s u f f i c i e n t . Fore a c h t =(i,x 6X*
d e f i n ea m e a s u r e g.onz * by ( A . ) = e t ( A ) if i = 1 or x.
g t ( A )=( c ( A , ) + ( A 2 ) ) / ( l + c ) if 1 = 2, ,andx., = " ] ( x 2 ) .
H e r e c = c ( x ) ande , , . d e n o t ethe u n i t p o i n t m a s s e s at xj Z- , x 2 Z o >t X*r e s p e c t i v e l y ; A. d e n o t e s { x . X ^ : ( l ^ x ^ ) A } . It is s t r a i g h t f o r w a r dt o v e r i f y t h a t
P * ( A ) = / g t( A ) p J ( d t )f o r e a c h B o r e l A c ^ * , so T is s u f f i c i e n t .
B y the SP we can c o n c l u d e t h a tE v ( E * , ( l , X l ) ) = E v ( E * T , ( 2 , ( . , ) ) ) andE v ( E * , ( 2 , x 2 ) ) = E v ( E *T , ( 2 , x 2 ) )
^ 6 X and { P } a . e . x2 X r In pa rt icu lar , foror - a.e.
7/27/2019 Berger the Likelihho Principle 1988 Book
52/252
3 6 HE LIKELIHOOD PRINCIPLE
{ P * } a . e . ) x G U and X2 = ^ x ^ we ha ve
E v ( E * , ( l , x 1 ) ) = E v ( E * T , ( 2 , x 2 ) ) = Ev ( E * , ( 2 , x 2 ) ) .
B y t h e WCP we hav e
E v U M l . X j ) ) = EvE^Xj) and E v ( E * , ( 2 , x 2 ) ) = E v ( E 2 > x 2 ) ,
s o we ca n c o n c l u d e t h a t
EvEj.Xj) =Ev E2, f o r { ?h a . e . x , E U | |
T h eRLP w i l l be u s e d inC h a p t e r 4 to e s t a b l i s h g e n e r a l v e r s i o n s ofi m p o r t a n t c o n s e q u e n c e s oft h e L P . T h e o r e m 3 d e m o n s t r a t e s t h a t r e j e c t i o n oft h e s e c o n s e q u e n c e s ( a nd s e v e r a lare u i t e u n p a l a t a b l e f r o m the f r e q u e n t i s tv i e w p o i n t ) i m p l i e s r e j e c t i o nof the W CP r the SP.3 . 5 P R E D I C T I O N , D E S I G N , N U I S A N C E P A R A M E T E R S , A N D T H E LP3 . 5 . 1 I n t r o d u c t i o n
T h e LP ass t a t e d a b o v e has he w e r y i m p o r t a n t q u a l i f i c a t i o n t h a t itd o e s not a p p l y if d o e s not i n c l u d e a ll u n k n o w n q u a n t i t i e s g e r m a n e to the exp e r i m e n torp r o b l e m . Fori n s t a n c e , ind e s i g n orp r e d i c t i o n p r o b l e m s the unk n o w n f u t u r e o b s e r v a t i o n is o b v i o u s l y r e l e v a n t , a n d y e t is notn e c e s s a r i l y ap a r tof thep a r a m e t e r d e f i n i n gthe i s t r i b u t i o n oft h e o b s e r v a b l e X. Ar e l a t e d d i f f i c u l t yis t h a t , o f t e n , o n l y a p a r t of isr e a l l y ofi n t e r e s t , ther e m a i n d e r b e i n ga " n u i s a n c e " p a r a m e t e r . T h e s e i s s u e s are e x p l o r e d in t h i ss e c t i o n .
W e b e g i n bye x p a n d i n g the e f i n i t i o n of toi n c l u d e u n o b s e r v e d andn u i s a n c e v a r i a b l e s . D e f i n e
= (z ) = ( y , w ; , ) ,w h e r e z = ( y , w ) is the v a l u e of anu n o b s e r v e d v a r i a b l e Z,w i t h y b e i n g of int e r e s t and b e i n g a n u i s a n c e v a r i a b l e , and w h e r e = ( , n ) is thep a r a m e t e r
7/27/2019 Berger the Likelihho Principle 1988 Book
53/252
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 3 7
t h a t d e t e r m i n e s t h e d i s t r i b u t i o n s o f b o t h X a n d Z , w i t h b e i n g o f i n t e r e s t a n d b e i n g a n u i s a n c e p a r a m e t e r . ( W e w i l l p u r p o s e f u l l y r e m a i n v a g u e o n t h e d e f i n i t i o n o f " n u i s a n c e v a r i a b l e " a n d " n u i s a n c e p a r a m e t e r " ; f o r m a l d e f i n i t i o n s c o u l db e a t t e m p t e d a l o n g d e c i s i o n t h e o r e t i c l i n e s , b u t w o u l d t a k e u s t o o f a r a f i e l d . )T o i n d i c a t e t h a t e v i d e n c e a b o u t a n d y i s d e s i r e d f r o m E w e w i l l w r i t e
E V ? (E' X)f o r t h e e v i d e n c e a b o u t a n d y f r o m t h e o b s e r v a t i o n o f x i n an e x p e r i m e n t E .
T w o d i f f i c u l t i e s a r i s e i n a t t e m p t i n g t o a p p l y t h e L P i n t h i s m o r eg e n e r a l c o n t e x t . T h e f i r s t i s t h a t t h i s g e n e r a l i z e d i s n o l o n g e r j u s t t h ep a r a m e t e r d e f i n i n g t h e d i s t r i b u t i o n o f X . T h u s t h e d e f i n i t i o n i n ( 3 . 1 . 1 ) o fi ( ) a s t h e d e n s i t y o f X g i v e n m a y n o l o n g e r b e a s u i t a b l e d e f i n i t i o n . I n d e e d , i f Z i s c o n d i t i o n a l l y i n d e p e n d e n t o f X g i v e n , t h e n ( by t h e d e f i n i t i o no f c o n d i t i o n a l i n d e p e n d e n c e ) i t c a n b e s h o w n t h a t ( 3 . 1 . 1 ) b e c o m e s
I ( ) f ( x ) = f ( x ) ,x z , 01 w h i c h d o e s n o t e v e n i n v o l v e z . T h e s e c o n d d i f f i c u l t y i s t h a t t h e n u i s a n c ep a r a m e t e r , , w i l l a p p e a r i n t h i s l i k e l i h o o d f u n c t i o n e v e n t h o u g h i t i s n o to f i n t e r e s t .
T o r e s o l v e t h e