Embed Size (px)
Citation preview
Associate Professor Genevieve McArthurDirector of the Macquarie University Cognition Clinic
for ReadingActing Head of the Department of Cognitive Science
ARC Centre for Cognition and its DisordersMacquarie University
How to make evidence based decisions about literacy interventions
Outline
The difference between the truth (fact) and what we are told (fiction)
Treatments for “dyslexia”
“causal” and “reading” treatments
good examples of evidence-based thinking
a step-by-step guide to the evidence-based treatment of dyslexia
practice
The difference between truth (fact) and what we are told (fiction)
Why am I banging on about flossing?
This represents the size of the gap (i.e., Grand Canyon)
between what science and the media tell us about the
treatment of dyslexia
Steps to close the gap …
Dorothy Bishop
dystalk.com/talks/60-evaluating-alternative-solutions-for-dyslexia
Good examples of evidence-based thinking
Ben Goldacre
http://www.youtube.com/watch?v=O1Q3jZw4FGshttp://www.youtube.com/watch?=_UrhJ8P8iZ8&feature=related
Commercial treatments for dyslexia
Reading treatments“Causal” treatments
“treatment x” review“treatment x” meta analysis
Yes
Go to step 2
No
Found
Use conclusion
Step 1: Find a systematic review (a review of all properly controlled trials of a
“treatment”)
Reading or spelling? Independent? Peer reviewed?
Go to step 2
Not found
“treatment x” systematic review
Yes
Go to step 3
Found
Reading or spelling? Independent? Peer reviewed?
Not found
Control group
Significantly bigger reading gains in
treated group than control group
No difference in reading gains in
treated and control groups
Go to step 3
Go to step 3
No
No control group
Step 2: DIY systematic review
“treatment x” randomised controlled trial “treatment x” study “treatment x” controlled trial
Step 3: Use indirect evidence
a treatment
reading
is k
now
n to
imp
rove
reading
is k
now
n to
imp
rove
trained skill
?? ✗✓
Step 3a: Find out what skill a treatment trains
What does “treatment x” train?
Step 3b: Find out if training that skill improves reading
Step 3: Use indirect evidence
a treatment
reading
is k
now
n to
imp
rove
reading
is k
now
n to
imp
rove
trained skill
?? ✓✓✗ ✗
Step 4: Strike out
No systematic review
No studies for DIY systematic review
No indirect evidence
Conclusion: no scientific evidence
Do not try treatment
Evidence-based decision Stab in dark
Try treatment
Beware of the brain
Brain data cannot tell us if an educational treatment works or not
If a program did not improve reading, but did change the brain, would you buy it?
If a program did improve reading, but did not change the brain, would you not buy it?
Focus on the behaviour (i.e., reading behaviour) and not the brain
Beware of the anecdote
Biased
Subjective
Confounded by other factors (e.g., treatments)
Cognitive dissonance
Examples
“Causal” treatments Reading treatments
Example 1
Yes
Use conclusion
Step 1: Find a systematic review
Reading or spelling? Independent? Peer reviewed?
Found
fast forword systematic review fast forword meta analysis fast forword review
little or no effect
Example 2
Yes
Use conclusion
Step 1: Find a systematic review
Reading or spelling? Independent? Peer reviewed?
Found
Irlen lenses systematic review Irlen lenses meta analysis Irlen lenses review
little or no effect
Example 3
arrowsmith reviewarrowsmith meta analysis
Go to step 2
Step 1: Find a systematic review
Not found
arrowsmith systematic review
Step 2: DIY systematic review
arrowsmith randomised controlled trial arrowsmith study arrowsmith controlled trial
Go to step 3
No studies
Step 3: Use indirect evidence
reading
is k
now
n to
imp
rove
trained skill
✗✓
reading
is k
now
n to
imp
rove
??
Step 3a: Find out what skill a treatment trains
What does arrowsmith train?
The Arrowsmith Program identifies, intervenes and strengthens the weak cognitive capacities that affect learning. Students are able to capitalize on their increased learning capacities and after a three or four year program can function without special education assistance or program accommodations. Upon completion of the program some students may require one to two years to gain experience using their newly strengthened cognitive capacities and some students may need tutoring initially to bring academic skills to grade level given the limited amount of time within the program to address academic skill deficits.
Step 3b: Find out if training that skill improves reading
Step 3: Use indirect evidence
reading
is k
now
n to
imp
rove
trained skill
??
cognitive capacity ?
reading
is k
now
n to
imp
rove
??
Step 4: Strike out
No systematic review
No studies for DIY systematic review
No indirect evidence
Do not try treatment
Conclusion: no scientific evidence
Evidence-based decision Stab in dark
Try treatment
✓✓✓
✓✓✓
If you want a backup or its all a bit too much ...
Outline
Treatments for dyslexia
“causal” and “reading” treatments
good examples of evidence-based thinking
a step-by-step guide to the evidence-based treatment of dyslexia
practice
Good luck
Example 4
reading eggs reviewreading eggs meta analysis
Go to step 2
Step 1: Find a systematic review
Not found
reading eggs systematic review
Step 2: DIY systematic review
reading eggs randomised controlled trial reading eggs study reading eggs controlled trial
Go to step 3
No studies
Step 3: Use indirect evidence
reading
is k
now
n to
imp
rove
trained skill
✗✓
a treatment
reading
is k
now
n to
imp
rove
??
Step 3a: Find out what skill a treatment trains
What does reading eggs train?
phonics
Step 3b: Find out if training that skill improves reading
phonics reviewphonics meta analysis
Yes
Use conclusion
Step 3b: Find a systematic review of that skill
Reading or spelling? Independent? Peer reviewed?
Found
phonics systematic review
moderate to large effects in dyslexia
Step 3: Use indirect evidence
a treatment
reading
is k
now
n to
imp
rove
reading
is k
now
n to
imp
rove
trained skill
✓✓
phonics
