How to Improve Diagnostic Assessment Reporting Practices: What Information Do Teachers Need and How Can We Best Communicate It?

Communicating Results of Learning Progression-Based Assessments

Emily LaiJennifer Kobrin

Insight Learning System

Professional Development

Game Classroom Activity

Performance Assessment

Learning Progression

Student Profile

Insight Learning System

Professional Development

Game Classroom Activity

Performance Assessment

Learning Progression

Student Profile

Professional Development: Goals

✓ Familiarize teachers with the learning progression and increase their content knowledge

✓ Help teachers elicit and identify evidence of students’ mastery of the stages of the learning progression

✓ Help teachers use the learning progression to make relevant instructional decisions to support student progress

Student Profile Development

➢ Designed to help teachers interpret information to support instructional decisions, by responding to key questions:

– Who is struggling? With what?– How could I group my students to improve

learning?– What should I reteach or focus on in class?– How can I personalize learning for my students?

➢ Quick, iterative design process3 rounds of testing with 8-12 teachers in each round

–Think alouds with follow-up on designs

1st Round Feedback: Interpreting Probability

Prompt: probability of mastery is .70Response: So they got 70% of the questions right?

This will be a major challenge for diagnostic models and communication

Decided to set cut points for probability and display as mastery, non-mastery, or undetermined.

For class aggregation, what percent of class should have mastered for the picture to indicate the class has mastered? Response: Consensus right around 80%

Third Round Design Solutions

Third Round Design Solutions

Sort by Column (group)

“Columnize” SOWOs

“What now”

Individual Observation Solution:- Emphasize

behavioral observations

- Provide suggestions for next steps

Third Round Solutions

Pilot Testing Results with Six Third Grade Teachers

Professional Development

Six third grade teachers from an elementary school in the Midwest participated in approximately 20 hours of professional development over 5 months:

❖ Introductory Webinar: What are Learning Progressions? How can they be used for Formative Assessment?

❖ Face-to-Face Sessions:I. The Learning ProgressionII. Performance AssessmentsIII. Classroom ActivitiesIV. Student Profile

❖ Virtual Forum: Questions of the Week

Professional Development

Six third grade teachers from an elementary school in the Midwest participated in approximately 20 hours of professional development:

❖ Introductory Webinar: What are Learning Progressions? How can they be used for Formative Assessment?

❖ Face-to-Face Sessions:I. The Learning ProgressionII. Performance AssessmentsIII. Classroom ActivitiesIV. Student Profile

❖ Virtual Forum: Questions of the Week

Professional Development Session on the Student Profile

• Overview of the Student Profile

• Instructional Decisions Based on Mock Report

• Think Aloud with Their Own Class Results

• Feedback on the Usability of the Student Profile

Individual Student Mastery Report

What can you infer about Harry?

“The area where you have a half-circle is where you would be working.”

So it looks like you need to start in the area progression, and make sure that he can properly calculate area, and iterate an area, and then challenge him with maybe some different shapes to see if he can figure out about area is additive before you even go into the length progression for perimeter because if he can't distinguish the difference then you want to start on that area progression and make sure that he's mastered all of that before you continue on.

Usability Feedback

• Teachers felt there were too many stages and several times mentioned that it would be more usable if “similar stages were lumped together” and it was more like a rubric.

• “The part I struggle with is that I don't know if it's necessary for them to go through every level of the progression, and they're gonna bounce back and forth based on what their previous knowledge is, how they view it, how they perceive [the task].”

• “I think while this is a nice tool, if I think about doing this, though, for every math topic, it’s kind of, um, seems overwhelming…”

Following Indiana Tryouts

Following Indiana Tryouts

Questions for Future Research & Development

• Can teachers interpret the Student Profile without participating in professional development to familiarize them with the learning progression?

• What is the right “grain size” for reporting?

• If we build it, will they use it? What prevents teachers from personalizing instruction?