Upload
timothy-duckett
View
32
Download
1
Embed Size (px)
Citation preview
The Well- EQUIPped Classroom: Using the Electronic Quality of Inquiry Protocol to evaluate effects of science inquiry professional
development on PreK-3 classroom practice Presented by
Gale A. Mentzer, PhDT. Ryan Duckett, MA
Toledo, OH
Research and Evaluation, LLC1811 N. Reynolds Road, Suite 204Toledo, OH 43615
“Accuracy of observation is the equivalent of accuracy of thinking.”--Wallace Stevens
American Evaluation Association National ConferenceOctober 29, 2016Atlanta, GA
Validity Issues
Surveys Tests Artifacts
EQUIP
• Time
Usage
• Instructio
n
• Discourse
• Assessme
nt
• Curriculu
m
Marshall, J. C., Horton, B., & White, C. (2009). EQUIPping teachers: A protocol to guide and improve inquiry-based instruction. The Science Teacher, 76(4), 46-53.
Example of EQUIP Instructional Factor
EQUIP as Evaluation Tool
Developing (2)
Proficient (3)
Pre-Inquiry (1)
Some More Much More
Exemplary (4)
Rasch at a Glanceexp(Bp - Di)
[1 + exp(Bp - Di)]P{xpi=1|Bp,Di}=
Higher levels of the trait
Level of Ability
(Person Measures)
Lower levels of the trait
Difficult to Endorse
Items that target the latent trait
(Item Measures)
Easy to Endorse
3.0
2.0
1.0
0.0
-1.0
-2.0
-3.0
Rasch Process in Four StepsStep OneAre the items
functioning properly?
Step Two
Step Three Step Four
Are the teachers participating reasonably?
Are you measuring a single latent trait?
How do we use what we know to understand?
Measuring Item Fit
Item FitHarder to Affirm
Teacher provides depth of content that connects big
picture
Lesson allowed for student designed investigation
Lesson integrated content and student
investigation
Student organized information in effective
ways to communicate their learning
Overall assessment of curriculum factor
Easier to Affirm
Difficulty Fit
2.46
1.21
-0.91
-1.26
-1.50
Reliability = .95
0.63
0.93
0.30
1.58
1.07
Rating Scale
Example of a poorly functioning rating scaleRating scale for Spring 2015 Curriculum factor
Building a Staircase
Pre-Inquiry (1)
Developing (2)
Proficient (3)
Exemplary (4)
Person FitSpring 2015 (Pre-intervention)
Fall 2015 (Post-intervention)
Rasch SeparationSeparation statistic of 5.77 = 8 distinct groups!
Dimensionality
Assessing Contrast
Evaluating the Intervention
Scale t α
Instruction 4.623 <.0001
Discourse 3.701 <.0001
Assessment 6.067 <.0001
Curriculum 3.877 <.0001
Assessing long term effects: ANOVA
Instruction 1 < 2 (α = .002) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)
Discourse 1 < 2 (α = .013) 1 < 3 (α < .003) 2 = 3 (α = 1.0)
Assessment 1 < 2 (α < .0001) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)
Curriculum 1 < 2 (α = .004) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)
Any Questions?
This presentation will be uploaded to our website:www.acumenresearcheval
.com