8/9/2019 Hamermesh 1997 Some Thoughts on Replications and Reviews

1/3

E L S E V I E R Labour Econom ics 4 (1997)107 -109

L B O U RECONOMICS

Some thoughts on rep l ica t ions and rev iews

D a n i e l S H a m e r m e s h

Department of Economics University of Texas Austin TX 78712 USA

JEL classification C1; C8 0; B4; J00

Keywords: Replication; Hypothesis testing

R e p l i c a t i o n i s a t h o r o u g h l y r e s p e c t a b l e a n d a l s o u s e f u l i n t e ll e c t u a l a c t i v i t y if i ti s d o n e c a r e f u l l y. R e p l i c a t i o n i s n o t , h o w e v e r, m e r e l y r e p e t i ti o n : M i n d l e s s l y t a k i n gt h e e x a c t s a m e d a t a a n d c h e c k i n g t o s ee i f a n a u t h o r ' m a d e a m i s t a k e ' i s n o t au s e f u l a c t i v i t y in th e s o c i a l s c i e n c e s . I f w e c o n d u c t e d p h y s i c a l o r b i o l o g i c a le x p e r i m e n t s , d u p l i c a ti o n w o u l d h a v e t h e b e n e f i t o f e n s u r i n g t h a t p r o p e r l a b o r a t o r yc o n d i t i o n s a r e m a i n t a in e d . D e s p i t e s o m e o f t h e h u b ri s o f o u r m o r e p o m p o u sc o - c o n s p i r a to r s , e c o n o m i c s i s n o t n o w a n d c a n n e v e r b e a fi e ld w h e r e m e r ed u p l i c a t io n c o u l d b e o f a n y i n te r es t. O f c o u r s e p e o p l e m a k e m i s t a k e s ; b u tp u b l i s h i n g s t u d i e s th a t p o i n t th i s a n d n o t h i n g e l s e o u t a d d n o t h in g . M i s t a k e s w i l la b o u n d , a s th eJournal of Money Credit and Banking s t u d y s h o w s . W h e n t h e ya r i s e a n d a r e m a j o r , o t h e r s i m i l a r s t u d i e s w i l l s h o w t h e r e s u l t t o b e s u c h a n o u t l i e rt h a t it s i m p o r t a n c e i n o u r a p p r a i s a ls o f h o w t h e w o r l d w o r k s w i l l q u i c k l y a p p r o a c h

ze ro .T h e b e s t r e p l i c a t i o n s t u d i e s , a n d t h e o n e s t h a t t h i s j o u r n a l s h o u l d b e p u b l i s h i n g ,w i l l a t t e m p t d u p l i c a t i o n a s t h e i r s ta r t i n g p o in t , b u t g o f a r b e y o n d t h a t. T h e y m i g h t ,f o r e x a m p l e , u s e t h e s a m e l a r g e c r o s s - s e c ti o n d a t a s e t t o t ry t o e s ti m a t e t h e s a m ee q u a t i o n s a s t h e o r i g i n a l s t u d y u s i n g i t s m e t h o d s , b u t t h e n g o o n t o t r y a l t e r n a t i v em e t h o d s a n d o t h e r s p e c i f i c a t i o n s . T h e y m i g h t t a k e t h e s a m e t i m e - s e r i e s d a t a a n dc o n s i d e r h o w w e l l t h e o r i g i n a l s t u d y ' s e q u a t i o n s f i t o u t s i d e t h e o r i g i n a l s a m p l ep e r i o d .

0927-5371/97/ 17.00 Copyright © 1997 Elsevier Science B.V. All rights reserved.Pll S 0 9 2 7 - 5 3 7 I ( 9 7 ) 0 0 0 1 5 -8

8/9/2019 Hamermesh 1997 Some Thoughts on Replications and Reviews

2/3

1 8 D.S. Hamermesh / Labour Economics 4 1997) 107-109

Th e r e i s a we l l - known p roc l i v i t y f o r j ou rna l s t o pub l i sh on ly s i gn i f i c an tcoe ff ic ien t s , and a resu l t ing incen t ive fo r empi r ica l resea rchers to e nga ge inda ta -min ing and op t imal s topping in the i r s ta t i s t i ca l ana lyses (Leamer, 1984;D eL on g and Lang , 1992). R epl ica t ion can prov ide a usefu l an t ido te to th i sunfor tuna te t endency. I t can a l so go too fa r in the o ther d i rec t ion : Given thel ike l ihood tha t ma ny o f the rep l ica t ion s tud ies wi l l genera te resu l t s d i ffe ren t f romthose of the or ig ina l s tud ies (whose resu l t s re jec ted under ly ing hypotheses ) , theproper k ind of rep l ica t ion tha t I ou t l ine above i s l ike ly to l ead to nega t iveconc lus ions , i .e . , i t wi l l r e jec t the re jec t ions of the hyp otheses . T his i s unfor tuna te :As the rea l bus iness cyc le l i t e ra ture makes eminent ly c lea r, the re i s a l ready toomuch o f a t e ndency i n o u r p ro f e s s ion t o pooh -pooh emp i r i c a l f i nd ings and t or ew a r d youn g economi s t s f o r a rmcha i r t heo r i z ing and mind l e s s s imu la t i on s .Labour conomics

sho u ld be ca r e fu l t ha t i ts Rep l i c a t i ons and Rev i e ws s ec t iondoes no t become a l aunching pad for an t i - in te l lec tua l miss i les .

An o the r po t en t i a l d a nge r i s t ha t young r e sea r che r s and s t uden t s may come tobe l ieve tha t the rou te such a sec t ion offe rs (and i t s publ ica t ion guaran tee) i s a goodway to s ta r t the i r ca reers . Severa l researchers I know of , inc lud ing one whose on lyp ub l i c a t i o ns I e v e r s aw w ere commen t s ( and whom I t hus a lways men ta l l yr e f e r red to a s Th e Co m m en ta to r ) , s e em to have m ade ( f a ir l y med ioc r e ) r epu t a -t ions by wr i t ing shor t empi r ica l papers tha t a re very t igh t ly based on ear l i e rs tud ies . Tha t rou te might l ead to a t enured pos i t ion a t a th i rd- leve l ins t i tu t ion ; bu ti t wi l l no t l ead any fur ther, and i t su re ly genera tes on ly marg ina l advances in

k now ledge . So m e r e p l ic a t io n i s u sefu l , and mo re needs t o be done ; bu t t he j ou rna ls h ou l d no t b e c ome a dum pin g g r ound fo r r epea t ed o f f e r i ngs by a f ew peop l e whoa re t r y i n g to m a k e c a r ee r s by pu b l i sh ing r epea t ed exe rc i s e s o f t h is va r i e t y. A l i m i to f o ne r ep l i ca t ion s t ud y pe r au t ho r ove r a f i ve -yea r pe r i od m igh t r em ove an yincen t ive for people to t ry s t ruc tur ing the i r ca reers a round rep l ica t ions .

W h i l e t he R e p l i c a t i o n s a n d Rev i ew s s ec t ion v i ews me ta - ana ly se s a s a r e a son -ab le top ic fo r inc lus ion , mos t such ana lyses s imply average es t imates tha t thea u th o r be l i eve s r ep r e sen t som e i mpor t an t pa r am e te r o r e f f ec t. Th i s app roac h s eem ss t ra igh t forward , bu t i t isn t : A s I show in my a t temp t to unders tand wh at theem p i r i c a l l it e ra t u re t e aches u s ab ou t e l a s ti c it ie s o f demand fo r l abou r (Ham erm e sh ,1993 , ch . 3 ) , seemingly s imi la r es t imates o f ten represen t d i ffe ren t under ly ingparameters . A survey paper tha t cons iders the mer i t s o f d i ffe ren t s tud ies and tha tsor t s ou t what each s tudy i s rea l ly es t imat ing , why, and to what ex ten t i t s resu l t sa re va luab le i s very usefu l . Mindless averag ing i s no t .

Gran t ing a l l these p i t fa l l s and cavea ts , a p roper ly ed i ted sec t ion i s a usefu laddi t ion to the journa l l i t e ra ture in l abour econ om ics . So lo ng as i t do esn t tu rnin to a shoo t i ng ga l l e ry t ha t e n ds up cava l i e r l y and un fa i r l y d i smi s s ing much o fwh at we in fac t know, i t wi l l avo id adding to the n ih i l i sm tha t i s too wide spread inou r p r o f e s s ion . I f done p r op e r l y i t c an kee p em p i r i c a l re s ea r ch a b i t m ore hon e s t

wh i l e he lp i n g p r ac ti t io ne r s t o ga rne r a b i t be t t e r fe e l f o r wha t w e do n o t know - -an d wha t w e d o .

8/9/2019 Hamermesh 1997 Some Thoughts on Replications and Reviews

3/3

D.S. Hamermesh / Labour Economics 4 1997) 107 109 109

e f e r e n c e s

DeL ong , B . and Lan g , K .A re a l l e con om ic hypo th ese s f a l s e? . Jou rna l o f Po l i ti c a l Econo my. , (Vo l . 1 (l(1)

( 1 9 9 2 ) 1 2 5 7 - 1 2 7 2 .Ha me nn esh , D . , 1993. Lab or Dema nd . P r ince ton Un ive r s i t y P re s s , P r ince ton , NJ .L e a m e r, E . L e t s t a k e t he c o n o u t o f e c o n o m e t r i c s . A m e r i c a n E c o n o m i c R e v i e w., ( Vo l . 7 3) (1 9 8 41

3 1 - 4 3 .