Upload
alex-holcombe
View
275
Download
0
Embed Size (px)
Citation preview
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
A new era of large-scale replication and collaboration?
@ceptional
http://www.slideshare.net/holcombea/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
False positives zone Look for areas wherein victory declared whenever null hypothesis is rejected, where size of effect, plausibility considered unimportant.
Where science is not self-correcting
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
"This one's Psych Science. 1998. Put your money on ‘will replicate’?”
"What, you think I was born yesterday?”
"Fine," muttered the bookie.
#AcademicNoir
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Well, Mr. P. Hacker, your luck is about to run out… down at the journals, they’re running a new game. They call it ‘preregistration’.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
5
https://osf.io/tvyxz/wiki/home/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
6
https://osf.io/tvyxz/wiki/home/
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
7
https://osf.io/tvyxz/wiki/home/
100% P-hacking FreeHere, check our numbers.
Here’s how you can replicate our result.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
"This one's Psych Science. 1998. Put your money on ‘will replicate’?”
"What, you think I was born yesterday?”
"Fine," muttered the bookie.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 9
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 10
Failed to replicate my result?
Have you considered
the unknown moderators?
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php 11
Any one of us alone, he’ll say we messed up. But if we all run the replication together, we
can show the big guy we’re right.
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Meta-analysis forest plot
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Meta-analysis forest plot
Failed to replicate my result?
Have you considered the unknown moderators?
You messed up!
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
PreregistrationLarge-scale replicationOpen dataOpen protocol and code
A new era of large-scale replication
and collaboration?
Can we do this bottom-up?
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php
Reproducibility workshop 26 Nov http://bizlab.unsw.edu.au/website/conference-2015.php