Facilitator Notes

Facilitator Notes

• All of the slides with Red backgrounds are copies of handouts that are available for the participants – they are hidden because they are there for your reference only

• All of the slide hidden are slides that fit with the conversations but are not necessary – you may keep them hidden or reveal them dependent upon your audience.

Integrating Multiple Sources of Data for a Evaluation and Design of Instruction for Students With Specific Learning Disabilities

SLD Manual Trainers Community of Practice and

Special Education Policy DivisionMinnesota Department of Education

[email protected]

Where in the Training Sequence are we?

What are the Big Ideas that are being presented?

You Will Need More Training and Support

Training Components

Knowledge Skill Demonstration

Use in the Classroom

Theory and Discussion

10% 5% 0%

…+Demonstration in Training

30% 20% 0%

…+Practice and Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce & Showers 2002

Don’t Get Stuck: Focus on What is Within Our Control

Circle of Concern

Circle of Influence

http://www.youtube.com/watch?v=CwOi4D9lPig

http://www.youtube.com/watch?v=oRBchZLkQR0

http://www.youtube.com/watch?v=oRBchZLkQR0

Turn and Talk

• Brainstorm common list of concerns regarding a student who is being discussed

• Review the list and star those that are within their circle of influence

The Manual Documents Changes in SLD Practices

Rationale for change:• End circular arguments

in the identification process

• Focus on “What data?" for “Which purpose?” to accelerate skill acquisition to make progress towards grade-level standards

Making the General Curriculum Accessible to Students with

Disabilities

These are the 5 Big Ideas We are Including in Training This Year

1. All Students Need to Make Progress Towards Proficiency

Job #1 =Make progress towards grade level content standards . . .

Eligibility Determination

Choral Read

A Student With a Learning Profile that Suggests a Disability Must Have Access to the

General Curriculum

Adapted from Sugai 2009

Phonemic Awareness

Math Fact Retrieval

Emotional Regulation Vocabulary

Listening comprehension

Concepts

2. We Improve Achievement with Problem Solving

2. Identify the instructional variables to be included in plan to address the gap.

3. Carry out the intervention as intended.

4. Evaluate the data and determine if the plan is working.

1. Identify gaps between performance and grade-level expectations.

Chapter 4-15, 6, 8, 9, 10

Even Comprehensive Evaluation is Problem Solving

Evaluation report reflects all that is known from previous cycles

Comprehensive evaluation is next formal cycle of problem solving Chapter 9 and 10

• Problem Solving Prior to Suspicion of Disability

• Present Levels of Academic and Functional Performance

Formal Evaluation Process

We Are Always Evolving a Picture of the Learner to Improve Response to Instruction

3. Sequence the Data Collection to Find Patterns

• Observe• Test

• Interview

• Record Review

1st 2nd

3rd4th

1. Review records

2. Conduct Interviews

3. Make Observations

4. Corroborate Suspicions with Testing

Coaches Corner

• Remember to use ICEL and RIOT in your conversations.– Your questions when talking to teachers should

include ICEL and RIOT components• As a coach you will need to guide teachers

into gathering data from different sources (or consider the sources of data they have access to)

4. All Data is Important for Creating a Picture of the Learner

Quantitative Data What’s working or not

Qualitative DataWhy is it working

or not

Expert ReviewWhy is it working

or not

What does learner

need

Both SLD Criteria Options Require Integrating Data from Multiple Sources

Full page in handouts

5. We Will Find Patterns in the Data

What do you notice?

See full page in handouts

Chapter 9-13

The 5 Big Ideas In Training

1. A___ students make progress towards p_________2. We improve a_______ with p______ s_______3. We use a s________ to integrate and prioritize data

feeding it f_______ at each step of problem solving4. All d____ is important for creating a picture of the

learner5. We will find p______ that link a______ and b_____

p_______ p_______

Guiding Questions Are Useful for:

A. Applying to quality practices that help us make the next right instructional step.

B. Ensuring we are in compliance.

Guiding Questions Are at the End of Each Chapter :

Chapter 8-11

Guiding Questions

Core Instruction Supplemental Intervention Specialized Instruction (IEP)

Access to high quality scientific-research-based instruction

How has the team determined the student has had sufficient access to high quality instruction and opportunity to perform within grade level standards?

Rule 3525.1341. Subp.1B

What supplemental efforts, aligned with grade level standards, were implemented to accelerate the student’s rate of learning and level of performance?

Statute 125A.56

Rule 3525.1341. Subp.2D

What has and has not worked to increase access and participation in the regular classroom environment?

What additional supports, accommodations or modifications are necessary to provide access to the grade level standards?

Limitations in adequate achievement or performance

(ELL, lack of appropriate instruction in reading or math).

What areas of educational performance/ achievement continue to be below grade level expectations?

34 C.F.R. § 300.8(a)(2)

34 C.F.R. § 300.304(b)(3)

34 C.F.R. § 300.304(c)(2) Rule 3525.1341. Subp.2A

What factors limit performance? What supplemental efforts have been successful in mediating the impact?

34 C.F.R. § 300.304(c)(1)(ii)

What about the student’s profile leads the team to suspect a disability and the need for special education and related service supports?

34 C.F.R. § 300.306(b)

What special education supports would be sufficiently rigorous to accelerate performance towards grade level achievement standards?

OR

Given previous efforts, what additional supports are required to help the student gain control over academic, non-academic, and transition goals?

Impairment/ Disability

(Sensory, cognitive delay, emotional or behavioral)

How is the student functionally limited from making progress towards grade level standards?

34 C.F.R. § 300.304-.306

How is the student limited from participating in the 5 areas of transition namely work, recreation and leisure, home living, community participation, postsecondary training and learning opportunities

Minnesota Rule 3525.2900, subp. 4(A)

What evidence is there that indicates the student needs protections afforded through Reauthorized Federal IDEA 2004 for specific learning disability to make progress towards grade level standards?

34 C.F.R. § 300.8(a)(1)

34 C.F.R. § 300.304(c)(2)-(7)

34 C.F.R. § 300.8(b)

What are all the needs that must be addressed and the evidence based instruction that will accelerate achievement towards grade-level standards?

34 C.F.R. § 300.305(a)(2)

34 C.F.R. § 300.304(b)(1)

Minnesota Rule 3525.2710, subp. 4(D)-(E)

Participant Handout

Chapter 8-11

Guiding Questions

Core Instruction Supplemental Intervention Specialized Instruction (IEP)

Access to high quality scientific-research-based instruction

How has the team determined the student has had sufficient access to high quality instruction and opportunity to perform within grade level standards?

Rule 3525.1341. Subp.1B

What supplemental efforts, aligned with grade level standards, were implemented to accelerate the student’s rate of learning and level of performance?

Statute 125A.56

Rule 3525.1341. Subp.2D

What has and has not worked to increase access and participation in the regular classroom environment?

What additional supports, accommodations or modifications are necessary to provide access to the grade level standards?

Limitations in adequate achievement or performance

(ELL, lack of appropriate instruction in reading or math).

What areas of educational performance/ achievement continue to be below grade level expectations?

34 C.F.R. § 300.8(a)(2)

34 C.F.R. § 300.304(b)(3)

34 C.F.R. § 300.304(c)(2) Rule 3525.1341. Subp.2A

What factors limit performance? What supplemental efforts have been successful in mediating the impact?

34 C.F.R. § 300.304(c)(1)(ii)

What about the student’s profile leads the team to suspect a disability and the need for special education and related service supports?

34 C.F.R. § 300.306(b)

What special education supports would be sufficiently rigorous to accelerate performance towards grade level achievement standards?

OR

Given previous efforts, what additional supports are required to help the student gain control over academic, non-academic, and transition goals?

Impairment/ Disability

(Sensory, cognitive delay, emotional or behavioral)

How is the student functionally limited from making progress towards grade level standards?

34 C.F.R. § 300.304-.306

How is the student limited from participating in the 5 areas of transition namely work, recreation and leisure, home living, community participation, postsecondary training and learning opportunities

Minnesota Rule 3525.2900, subp. 4(A)

What evidence is there that indicates the student needs protections afforded through Reauthorized Federal IDEA 2004 for specific learning disability to make progress towards grade level standards?

34 C.F.R. § 300.8(a)(1)

34 C.F.R. § 300.304(c)(2)-(7)

34 C.F.R. § 300.8(b)

What are all the needs that must be addressed and the evidence based instruction that will accelerate achievement towards grade-level standards?

34 C.F.R. § 300.305(a)(2)

34 C.F.R. § 300.304(b)(1)

Minnesota Rule 3525.2710, subp. 4(D)-(E)

Coaches Corner:

• Review the handout• Consider who might you need to work with to

ensure the guiding questions are answered• Identify people who currently may be able to

answer the guiding questions. Is this an effective and efficient use of resources?

Identifying the LEARNING PROBLEM

What is the Gap Between Grade Level Content Standards and

Student’s Performance?

What is the Learning Problem?• What is grade-level

content standard or peer group expectation?

• Do we understand what grade-level performance looks like?

• Do we see the where the learning is breaking down?

• Do we have data that targets what we thought was constraining growth?

Grade Level Expectation

Current level of functional performance

Chapter 4-28 through 31

Spring of 3rd Grade

Expectations

Identifying Gaps in Performance

•Reads 60 words correct per minute•Acquiring intermediate word

analysis skills•Inconsistent generalization of self-

monitoring skills

•Reads 110 words correct per minute with expression

•Generalizes advanced word analysis skills

•Generalizes self-monitoring of reading and self-corrects when comprehension breaks down.

Grade Level Expectations Compared to Present Level of Performance

Back

What is the Learning Problem?• What is grade-level








What is the Standard and What Does Grade Level Performance Look Like?

FLUENCY

Phonemic

awareness

Language developmen

t

PhonicsWord

analysis

Alphabetics

Comprehension

Metacognition

Vocabulary elaboration

and morphology

Application of strategic

thinking

Mea

ning

Mak

ing

Reading Development Maps to the Standards

3rd Grade Standards: Cross-referenced with Development

The student will use a variety of strategies to expand reading, listening and speaking vocabularies.

Acquire, understand and use new vocabulary through explicit and indirect vocabulary instruction. . .

Identify and correctly use antonyms, synonyms, homonyms and multiple-meaning words.

The student will apply word recognition strategies to decode unfamiliar multi-syllabic words and will read grade-appropriate text with accuracy and fluency.

Read unfamiliar complex and multi-syllabic words using advanced phonetic and structural analysis.

Read narrative and expository text with fluency, accuracy and comprehension. . .

The student will understand the meaning of texts using a variety of comprehension strategies and will demonstrate literal, interpretive, and evaluative comprehension.Recall and use prior learning . . .

Monitor comprehension and use strategies to self-correct when needed.

8th Grade Standards: Cross-referenced with Development


Acquire, understand and use new vocabulary through explicit and indirect vocabulary instruction. . .

Recognize and interpret words with multiple meanings.

The student will read with accuracy and fluency.

Read unfamiliar complex and multi-syllabic words . . .

Read narrative and expository text with fluency, accuracy and comprehension. . .

The student will understand the meaning of texts using a variety of strategies and will demonstrate literal, inferential, interpretive, and evaluative comprehension.

Recall and use prior learning . . .Make inferences and draw conclusions based on explicit and implied information from texts.

Do We Know What the Needs are?• What is grade-level








Turn and Talk

• What is the Key Message in Slides 35-42• How does the "stairs" graphic relate to the key

message• Take turns with partner explaining these slides

Coaches Corner:

Good questions to ask teachers when coaching include:– How do you know that? (watch your tone – point is to learn

what sources of data they have, not be accusatory)

– Tell me more about that.– Do you see this deficit across areas?

If teachers are not able to provide data to support what they are saying, your role is to guide them to gather that data

Adapted from Teaching the Learning Disabled (p. 68) bgy D. D. Smith, 1981, Englewood Cliffs, NJ: Prentice Hall.

AIMAccuracy(90-100%)

Fluency(desired rate)

Retention Expansion

Acquisition Initial Advanced

Proficiency Maintenance Generalization

0 to

25%

65%to

80%

Highrate and

accuracy High

rate and accuracy

Transferto new

settings

EntryLevel

No orlow

frequency

Adaption

Capitalize on

knowledge

Extension

Stages of Learning the Breakdown

Adaptation

Illustrative Example of Clarified Learning Problem

Jim is currently reading at 60 (WCPM) on 3rd grade passages. To reach grade level expectations he needs to acquire and reach maintenance with strategies for decoding multi-syllabic and approaching irregular words he encounters in 3rd grad texts. He needs to consistently apply self-monitoring and fix-up strategies to improve comprehension to grade-level text (fiction and non-fiction).

Coaches Corner:

• These last few slides usually elicits murmurs and/or “ah-ha” moments in training. Since you won’t be training, how can you get this information to teachers/teams? When might you bring this information into a discussion/dialogue with teachers/teams?– Grade level team meetings, Problem-solving team

meetings, graph analysis meetings, etc.

GATHERING MULTIPLE SOURCES OF DATAThe framework helps us organize,

prioritize, integrate, and efficiently in collecting data to identify the

problem

RIOT - ICELSequence• Record Review

• Interview

• Observation

• Test

Within our control

• Instruction

• Curriculum

• Environment

• Learner

Leve

ls of

Ana

lysis

Sources of Evidence that Can be Mined for Decision Making

Data feeds forward

SRBI 1 SRBI 2Evaluation for Special Education

Standards-based IEP

School-wide

Grade or classroom

Group

Individual

Leve

ls of

Ana

lysis


Data feeds forward


Standards-based IEP

School-wide

Grade or classroom

Group

Individual

Leve

ls of

Ana

lysis


Data feeds forward


Standards-based IEP

School-wide

Grade or classroom

Group

Individual

I dentify Data Source

Excerpt of 8th Grade Reading Standards

The student will read with accuracy and fluency.

MCA I I


1. Acquire, understand and use new vocabulary through explicit and indirect vocabulary instruction and independent reading. 2. Determine the meaning of unknown words by using a dictionary or context clues. 3. Recognize and interpret words with multiple meanings.

Content Limit: Vocabulary items require context clues from a passage to determine word meanings (i.e., word structure may provide support but not the sole basis for the item). 6–8 Word structure may only assist in determining meaning and may include prefixes, suffixes, syllabication or root words. Dictionary clues are provided in the item when required. No more than one dictionary item exists on an operational test. Multiple-meaning words are the same as synonyms for this benchmark.

4. Describe the influences of other languages on the English language. 5. Apply knowledge of Greek and Latin roots, prefixes and suffixes to understand content area vocabulary. 6. Determine word meanings by using definition, restatement, example, comparison or contrast. 7. Identify and explain analogies, similes and metaphors. 8. Apply correct word pronunciation and inflection.

MCA I I

MCA I I

The student will understand the meaning of texts using a variety of strategies and will demonstrate literal, inferential, interpretive, and evaluative comprehension.

1. Summarize and paraphrase main idea and supporting details. Content Limit: Items will require a summary or paraphrase solely from the text given. Items will require the identification of the explicit or inferred main idea and/or its relevant details solely from the text given.

2. Recall and use prior learning and preview text to prepare for reading. 3. Comprehend, interpret and evaluate information in a variety of texts using a combination of strategies before, during and after reading. 4. Make inferences and draw conclusions based on explicit and implied information from texts. 5. Trace the development of an author’s argument, point of view or perspective. 6. Evaluate the adequacy, accuracy, and appropriateness of the author’s evidence in a persuasive text.

Content Limit: Adequacy and appropriateness are synonymous in these items. Items do not require verification of accuracy outside the text given. Persuasive text can be an element of persuasive text found in any genre.

7. Use knowledge of narrative and expository text structures in a variety of content areas to summarize information.

Coaches Corner

• What activities can you use in your coaching to help teams understand what data they have and where the gaps are in their data?

• When would these activities fit into the schedule?

• You may need to guide teams in discussions about data they have. They may not realize some sources (eg. ODRs)

Integrate the Analysis At Each Phase

Screen

• Benchmark below grade level expectations (20-30% ile)• ELL progress data corroborates• Teacher test data and noted behaviors(QRI, IRI, error analysis)

Intervene

• Problem statement—(synthesis of prior data+ parent input)• Progress monitoring data• Interviews and or observations for fidelity check

Intervene

• Revisit problem statement with additional data (incorporates all prior data)

• Continue to progress monitor and check fidelity• If evidence indicates lack of response, consider referring

student• Observe student to further target what to assess in

comprehensive evaluation

Chapter 4, 5, and 6

Different Representations of Integrating Problem Solving Over Time

Student participates in first cycle of intervention

Student participates in second cycle of intervention

Screen

Intervene

Intervene

Chapter 4, 5, and 6

Are We Confident that the Data Gathered Reflects the Students True Abilities?

Chapter 4, 5, and 6

Fill in the BlanksSequence• R• I• O• T

Within our control

• I• C• E• L

Record Reviews: Inform how the Learner Functions within. . .

Instruction

Curriculum

Environment

• Record Review

• Record Review

• Record Review

Chapter 4-11, 4-21 & 23, 6-7

Record ReviewPrioritize Your Analysis of Existing Data

– Standardized test results• Level of performance and area of concern• Trends across time• Fidelity

– Attendance– Student work

• Uncorrected performance and response to feedback• Mis-understandings• Productivity

Chapter 4-21 and 30

•Data dashboards

•MCA-II and NWEA data compared with Survey of Enacted Curriculum

•Attendance•Office discipline referrals•Oral reading fluency, etc.•Observations academic engaged time (group, individual)

Record Review: Trend Data

Record Review Scaffolded in the ManualDomain Source Data Outcome

Instruction Permanent Products Nature of instructional demands reflected in paper-pencil tasks (e.g., style demands of the task, difficulty levels, skill requirements). Teacher records of: o How expectations are communicated and the criteria for success. o How content delivery is structured. o Specificity of feedback on performance. o Student response to directions. o Teacher response to students request for clarification or assistance. o Opportunities and methods of practice.

Curriculum Permanent Products (ex. Books, worksheets, curricular guides)

Nature of instructional demands reflected in: o Stated outcomes, standards and benchmarks. o Scope and sequence of instruction. o Arrangement and timing of curriculum sequence. o In curriculum and instructional materials. o Instructional approaches. o Learning tasks and pre-requisite skills.

Pacing for stages of learning (acquisition, accuracy, fluency, generalization/application).

Environment School and Classroom Procedures

o Discipline policies and procedures that define what is deemed as “situationally appropriate.”

o Positive behavioral supports, e.g., explicit instruction in expectations (task, classroom, school) and routines.

o Relational influences (peer to peer, student to instructor, student to family).

o Physical arrangement of the classroom (noise, position relative to focus of instruction, etc.).

Chapter 6-7 through 6-9

Critical Questions

• What does the data say about how the learner functions?

• What data are missing or under represented (Instruction, Curriculum, Environment)?

• Are there areas where you have only qualitative data? quantitative data? No data?

Practice on Tara

Coaches Corner

• As you are building consensus through coaching, you will have some teachers ready to start learning/doing the process before others. Discuss with your partner how you could go about working through this process with a teacher. Think about the Tara/Jim case study.

Interviews: Inform How the Learner Functions Within. . .

Instruction

Curriculum

Environment

• Record Review• Interview



Chapter 4-11, 4-22 & 23, 6-7 to 9

Interviews: Further Clarify Learning Problem(s)

Use existing data to generate interviews • Parents

– Concerns and input on interventions– Relevant medical, homework, tutoring, observations

• Teachers and Student– Nature of concern – Context and expectations– Positive and negative influences on performance– Compensatory strategies

Chapter 4-24, 6-9, 7-24

Interviews: Further Clarify Learning Problem(s)

Use ________ data to generate interviews • Parents

– Concerns and ________ on interventions– Relevant medical, _________, tutoring, observations

• Teachers and Student– ________ of concern – Context and __________– _________ and _________ influences on performance– _____________ strategies

Chapter 4-24, 6-9, 7-24

Interviews Further Define the Problem(s)Concern happens… (when, where, with whom?)

What about performance is concerning?

What is expectation (e.g. grade level, rate of progress, peer comparison)

When he is doing independent s reading

He last read 15 words per minute with 20 errors...

…. his 2nd grade peers in the group read on average 70 words per minute with 3 errors.

When given directions by teachers to complete work…regardless of teacher or group size

…she responds with a loud voice and feet stomping, throwing materials. . .and takes up to 5 minutes to get to work

…while her kindergarten peers respond by getting to work within 5-10 seconds of direction.

Chapter 4-23, 6-10, and 7

Critical Questions

• Does the data say about how the learner functions?



Practice on Tara

Observations Inform how the Learner Functions within. . .

Instruction

Curriculum

Environment

• Record Review• Interview• Observation



Chapter 4-11, 4-22 & 23, 6-7 and sample form in handouts

Observations: Targeted to Hypothesis and Suspected Basic Psychological Processes

1. Observe student while learning– Goodness of fit (better and worse)– Tailor later instructional and accommodations

2. Observe suspected basic psychological processes– Nature of concern—targets testing in comprehensive

evaluation– Teacher’s natural differentiation or accommodation and

students response – Identify compensatory strategies and their effectiveness

Chapter 6-11 through 16, and sample draft in materials

Use a Variety of Evidence Because it Provides ContextQualitative Quantitative– Observation– Interviews/

conference notes– Record reviews– Notable behaviors

– Tests, GOM/CBM– Checklists/rating scales– Duration, intensity,

frequency measures

Provides context for numbers

Provides standard for comparison

Chapter 8-15 Professional Judgment

Turn and Talk

• The Principal/Psychologist/Leadership Committee member says that the general ed staff in your district/building doesn’t know how to systematically collect RIO data. Or, they say they don’t think the general ed staff in their district/building will follow this procedure. Turn to the person next to you and practice addressing these comments.

Critical Questions

• What does the data say about how the learner functions?



Practice on Tara/Jim

Using RIOT to Write A Problem Statement

Now what do we know about how the student is performing? What

makes it better/worse?

Writing a Problem Identification Statement From the Data



•Define it in observable and measurable terms

•Target specific parts of performance

•Focus on alterable variables

Effective Problem Statement: Pete 3rd

• Reads passages 60 WCPM--slow but accurate reader

• Reads word lists 110 WCPM--has difficulty with connected text

• Decoding errors are with irregular and multi-syllabic words

• Uses a letter by letter decoding strategy

• Adjusts his reading rate according to prior knowledge

• Vision screening test 20/20

Critical Features addressing questions

• Defined in observable and measurable terms

• Targets specific parts of performance

• Focuses on alterable variables

Problem Statement: Pete

Pete is currently reading at 60 (WCPM) on 3rd grade passages. Peers are reading at 120 (WCPM). To reach grade level expectations he needs to acquire and reach maintenance with strategies for decoding multi-syllabic and approaching irregular words he encounters in 3rd grade texts. He needs to consistently apply self-monitoring and fix-up strategies to improve comprehension to grade-level text (fiction and non-fiction).

Trainer Tip

• Participants may say this is more specific data than they have ever received from general ed staff. Acknowledge that we are all getting better and there are places in the state where teams are receiving this type of data. If teams were to receive this type of data, how would it impact their planning and problem solving? What would they be prepared to contribute?

Common Challenges with Problem Analysis?

• Maintaining a focus on learning vs. learner• Having relevant data from a variety of sources• Focusing on right mix of variables

– Instruction– Curriculum– Environment

• Successfully discriminating “can’t do” from “won’t do” problems using data

See Checklist for Improving Problem Statements in Handouts

Unacceptable Descriptors• Focus on unimportant or factors beyond the control

of instructor • Have unclear measures—does it pass the “stranger

test” • Indicate conflicting views between grade level

expectations and student’s current performance• Lack baseline data • Lack standard of comparison • Provide a standard that is not related to the behavior

With the Problem Statement You have to Match Needs with Instruction

Chapter 4-15

Activity: Create a Hypothesis for Tara

Check your understanding

Problem Identification Statement : Grade Level Expectations for Tara

Accurate and Fluent reading: The expected rate for 8th grade benchmark assessments is 133 wcpmComprehension Standard: The student will understand the meaning of texts using a variety of strategies and will demonstrate literal, inferential, interpretive, and evaluative comprehension.

Recall and use prior learning and preview text to prepare for reading.Comprehend, interpret and evaluate information in a variety of texts using a combination of strategiesMake inferences and draw conclusions based on explicit and implied information from texts.

Vocabulary: Acquire, understand and use new vocabulary through explicit and indirect vocabulary instruction and independent reading.

Determine the meaning of unknown words by using a dictionary or context clues.

Effective Problem Statement: Tara 8th

• Tara an 8th grader who reads at 110 wcpm

• Not able to summarize what she reads

• Misses 90% of inferential and vocabulary questions

• Is a second generation immigrant• Spends extensive amounts of time

with extended family who speak but are not literate in English

• Spends 1 month in Mexico during school year

• Defined in observable and measurable terms

• Targets specific parts of performance

• Focuses on alterable variables

Data Relevant for Assessing Culturally & Linguistically Diverse Learners

Chapter 7-15

When do English Learners really “catch up?”

60

55

50

45

40

35

30

25

20

15

10

5

Formal instruction

begins

After 5 years of

instruction 47,450 hrs.

CALP

Cumulative Hours

of Language Exposure

in Thousands

-18,000

B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 K 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th 9 th

Age and Grade Level

21,900 hrs.

3,650 hrs.

23, 725 hrs.

-24,000Native EnglishSpeaker (L1)

ENGLISH LANGUAGE EXPOSURE

Awake AsleepAge 0 to 5: 12 12

365days x 12hrs. x 5yrs.= 21,900 hrs

Age 5 to 10+: 14 10

365days x 14hrs. x 5yrs.= 25,550 +21,900 47,450

Limited English Speaker (L2)

ENGLISH LANGUAGE EXPOSURE

Native (L1) English(L2)

Age 0 to 5: 10 2

365days x 2hrs. x 5yrs. = 3,650 hrs.

Age 5 to 10+ 3 11

365days x 11hrs. x 5yrs.= 20,075 +3,650 23,725

Ortiz, S. 2010

Activity: Create a Hypothesis for Tara

Check your understanding

Problem Statement cont: Current Performance

• She needs to support decoding skills in connected text at grade level.

• As a 8th grader Tara misses 90% of inferential and vocabulary questions in 6th grade level passages.

• She must develop strategies for acquiring vocabulary with application to selecting the appropriate meaning from multiple meanings.

• She must acquire a larger body of prior knowledge, figurative language, and conceptual knowledge to make inferences, find the main idea when implied, and understand author’s intent when reading 8th grade text.


• She needs to develop strategies to independently activate prior knowledge when using vocabulary, solving math problems, applying to new situations.

• She needs strategies to organize her ideas, sequence and connect ideas to support a point.

• She shows difficulty in writing paragraphs that draw comparisons and contrasts, persuasion, and supporting opinions with facts.


• She needs to develop better problem solving, conceptual understanding of fractions, decimals, solving equations, etc.

• She needs to be able to translate word problems into mathematical sentences.

• Tara is most successful when she works with a partner or in small groups on tasks.

• She responds to information presented visually.

Coaches Corner:

• Tara is an example of an EL student, and her case is complicated. What are the biggest learner challenges in your building(s)? Finding ways to help teachers make progress in working with these students may be a good way to build consensus.

• As a coach you need to make it ok for teachers to seek help in writing problem statements.

You Are Ready To Implement and Monitor

Chapter 6

Cycle Repeats for Second Pre-referral Intervention or For Tier 3

Intervention

Brain Break

Suspecting a Disability

Integrating What is Known into the Prior Written Notice

When Interventions Are Ineffective You Suspect a Disability

Chapter 6

Indicators of a Disability

Inadequate Instruction Suspicion of Disability

Progress is Unexpected Given other Strengths

Size of Gap + Lack of Progress

SRBI effective and consistently provided

Progress is Evident

No SRBI

Chapter 7

Partner Work

• Use the script to explain the previous slide to your partner. Include any data that you may collect in your school.

• Switch roles• List any questions you and your partner may

still have.

Suspicion of Disability: Integrate What You Know into the Prior Written Notice

• The District proposes to conduct an evaluation/reevaluation for the following reason(s);

• Describe the other options considered and the reasons why those options were rejected;

• Description of each evaluation procedure, . . . ;• Statement of adaptations needed to conduct this evaluation;• Other factors affecting this proposal

INDIVIDUALIZING THE COMPREHENSIVE

EVALUATIONFeeding Data Forward

Individualize the Comprehensive Evaluation: Don’t Reinvent the Wheel

• Problem statement and

• Corroborating data (records, parent input, observation)

Existing Data

• Complete guiding questions with existing data

• Identify gaps in data

Gap Analysis • Fills in gaps• Corroborates data

with less technical validity

Gather Additional Data

Planning of evaluations are not dictated by test-kits

Chapter 8

Activity – Guiding Questions

Prepare for Comprehensive Evaluation

Don’t Lose What you learned in Problem Solving

Comprehensive evaluation is next formal cycle of problem solving

Chapter 8

Comprehensive Evaluation is More Intensive Problem Solving with Different Experts Is there a disability?

Data with high technical adequacy, focused on variables intrinsic to learner

Data focused on alterable variables outside learner available

Data focused on performance compared with expectation available

Chapter 8

Gather Additional Evidence For ContextQualitative Data• Attitude toward task• Comprehension and follow

through• Attempts to change task

approach mid-stream• Self-awareness/monitoring• Thinking aloud• Flexibility in problem

solving and persistence• Response to success or

failure

Quantitative DataWork sample analysis

-before and after feedbackAdditional areas of concern Amount of instructionError AnalysisFidelity

Chapter 6 and 8-16 Notable Behaviors

Turn and Talk

• Share with your partner:– Difference between Quantitative and Qualitative

Data– Examples of each available within your district – How that information is incorporated into the

problem solving process

RIOTest: Corroborate Findings and Plan for Instruction

Administer tests of processing

Apply added value procedures

Corroborate Data

Suspected Disability from RIOT

Provide Access

to general curriculum

Accelerate skill

acquisition

Facilitate Progress towards

proficiency

Comprehensive Evaluation Yields One of Three Results

Problem IdentificationPresent Levels of Academic and Functional Performance

Formal Evaluation Process

What is the Difference Between a Problem Statement and a Present Levels Statement?

Standards-based PLAAFP

• How does the data come together to identify the disability and how it impacts student performance in grade level content standards

• See PLAAFP guided discussion • Illustrative example

Instructional Level

Content

Enrolled Grade Level

Content

Results of Evaluation Should Help us Balance Access with Intensive

Instruction•Analysis of grade-level content standards•Differentiated instruction

•Content•Process•Product

•Accommodations to reduce barriers•Mapping of teaching sequence with what it takes to increase student understanding

Coaches Corner:

• Know your audience• Validate points of views of the various staff

members during your coaching sessions.• Validate people that are at different levels of

understanding.

Does it Feel Like This

Or

http://www.youtube.com/watch?v=4wp3m1vg06Q

http://www.youtube.com/watch?v=fYTJ9v2vsaE

Addressing Exclusionary Criteria While Solving the Learning

Problem

Applying the Framework of Integrating Data to Systemic Problem

Solving

How do we determine “access to appropriate instruction in reading

and math. . .?”

What data should we triangulate? How early should this start?

Address the Problem at the Right Level

Adap

ted

from

Chr

ist, T

. (20

08),

Best

Pra

ctice

s in

Prob

lem

Ana

lysis

.Be

st P

racti

ces i

n Sc

hool

Psy

chol

ogy.

NAS

P.

Chapter 4-10

Leve

ls of

Ana

lysis


Data feeds forward


Standards-based IEP

School-wide

Grade or classroom

Group

Individual

Leve

ls of

Ana

lysis


Data feeds forward

This should be familiar to you now. Fill in each of the levels and the progression

• MCA-II—school-wide trends and annual results

• Survey of Enacted Curriculum results or curriculum maps

• Measures of Academic Progress benchmarks

• General Outcome Measure benchmarks

• Attendance

• Office discipline referrals

• Principal walk through observations

Leve

ls of

Ana

lysis

School-wide

Grade or classroom

Small Group

Individual

School Improvement– School-wide Evidence Indicating Progress Towards Achievement Goals

Effectiveness of instruction • Data suggests how well

instructional practices and materials are meeting needs of all students

• Data is used proactively to formulate core instructional match to student needs

All Native Amer

ELL0%

20%

40%

60%

80%

100%

School-wide Evidence Can be Used for Judging Effectiveness

Chapter 4-12 and 13

Low SES

• MCA-II grade level results -disaggregated

• Survey of Enacted Curriculum results or curriculum maps

• Measures of Academic Progress benchmarks

• General Outcome Measures Benchmarks change from Fall to Winter to Spring

• PLC work—analysis of student work samples

• Formative assessment data

Leve

ls of

Ana

lysis

School-wide

Grade or classroom

Small Group

Individual

Grade or Class Level Data Indicates Progress Towards Achievement Goals

Goal 70%Target: 43 72 90

Total Enrollment: 68 67

Fall Winter Spring

5 students

Intensive <26

17 students25%

Intensive10 students

15%

Intensive

Strategic

4 Students

Benchmark45 students05-06 66%04-05 61%03-04 56%

Benchmark47 students05-06 70%04-05 69%03-04 61%

Benchmark05-06 04-05 68%03-04 54%

Strategic 6 students

9%

Strategic10 students

15%2nd

grad

e

42

Peggy N 43-71Tom T 65-70

2

10

Slide Originally Presented at the Midwest Leadership Conference, 2007 By Kim Gibbons.

Class or Grade Level Data Can be Used to Judge Effectiveness

Grade Level Progress Over Time

• Measures of Academic Progress change (F, W, S)

• General Outcome Measures Benchmark (F, W, S)

• Progress Monitoring Data –group comparison

• Minutes of Instruction Delivered

• Observation or fidelity checks

Leve

ls of

Ana

lysis

School-wide

Grade or classroom

Small Group

Individual

Group Level Evidence Indicates Progress Towards Achievement Goals

• Near 50%ile Benchmark

AIMS-web Data Indicating Effectiveness

Above 50%ile Benchmark

Below 50%ile Benchmark

Below 25%ile Benchmark20 19.429.4

Leve

ls of

Ana

lysis

School-wide

Grade or classroom

Small Group

Individual

•MCA-II trend and Learning Locator (across grades)•Measures of Academic Progress (across grades)•General Outcome Measures benchmark (F, W, S)•Progress monitoring data –individual and compared to peer group•Minutes of instruction delivered •Interview of teacher and parent •Observation of student engaged and responding to instruction

Individual Response– Documented Progress Towards Achievement Goals

How do we determine if it is Poor Attendance ?


Likely scenarios:• Minutes received is less

than scheduled

• Inconsistent data or instruction

• Critical features of intervention missing

Was intervention implemented as

intended?

Re-examine Results of Intervention

Chapter 5-15

How do we determine if it is EBD or SLD?



Questions to Tease Apart SLD/EBD

• How well does the student respond to academic instruction once individual positive behavioral supports are in place?

• What happens to academic performance when behavioral or social/emotional skills are taught?

• What happens to behavior when instruction is provided at the student’s instructional level?

• Is student performance different across classrooms, teachers, and content areas? In which combination of circumstances is behavior better or worse? Is there a teacher that the student performs better for than others?

• What happens to behavior as achievement improves? Expectations rise?


Leve

ls of

Ana

lysis

Ignore the Technical Adequacy column.

Activity: Brainstorm school and grade-level improvement data that could be used could to answer the guiding questions in your materials.

Follow up questions:•How often would you need to •access it?•Who has access?

Activity – Guiding Questions

Illustrative Example

For ideas see handouts

Illustrative Example

Additional Handout Resource