Slide 1

Utilizing Data Informed Decision-Making to Improve the PSSA Performance of CTE StudentsPACTA PIL October 18, 2010Agenda11:15 12:45

12:45 1:301:30 3:003:00 3:153:15 5:005:00 5:455:45 6:00Overview of the School Improvement ProcessData, Data, EverywhereLUNCHData AnalysisBreakRoot Cause Analysis ProceduresSharing & Reporting OutWrap-up and EvaluationWhat impacts (has an effect on) heart health?Family HistoryDietExerciseSmoking

What are indicators of heart health?Family HistoryDiet - WeightExercise Resting heart rateCholesterol Triglycerides Blood pressure

What are the impacts and indicators of heart disease?If we are to improve the health of our hearts, we need to be aware of both the impacts of heart health as well as the indicators of heart health.

Another example:STEELERS FOOTBALL!

What impacts Student Achievement? What are indicators of Student Achievement? Overview of the School Improvement ProcessSchool Improvement CycleData Informed Decision Making Cycle for __________ IMPROVEMENT

Did it work?Data analysisStrategic PlanningResourcesDataDataDataDataDataDataDataDataRemember:Numbers areour friendsAugust 12, 200410Actually data is needed throughout the improvement cycle. What about the data analysis results?What about the implementation timeline data?What about the data thats collected to ensure that the plan is implemented?

Remember, Numbers are our friends.Using Data to Improve Learning for All: A Collaborative Inquiry Approach by Nancy Love et al.

Added by ShulaPDEs Getting Results

Focused and Un-forcused Improvement Cycles a la Bernhardt

Impacts and Indicators of Student Achievement

Data, Data Everywhere

Types of Data a la Bernhardt Indicators & ImpactsDemographicsSchool ProcessesStudent LearningPerceptionsNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action16What are your Impacts and Indicators?Identify your Impact and Indicators by their type

Blue Dots Student Learning

Yellow Dots Demographics

Red Dots School Processes and Programs

Green Dots Perceptions

Multiple Measures of Student LearningSummative AssessmentsPSSANOCTINAEPFormative AssessmentsInformal teacher observationsInterim Assessments4SightGradesDiagnostic AssessmentsCDTIndicatorsNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action18Multiple Measures of Student Learning

4SightPSSANOCTINew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action19Multiple Measures of Student Learning over timeLongitudinal DataAnalysis of annual performanceAnalysis of across the yearsAnalysis of cohort groups across the years (8th grade vs. 11th grade)PVAASNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action20What are YOUR measures of Student Learning Data?(summative, formative, interim, diagnostic)PSSANOCTI Grades

New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action21Multiple Measures of DemographicsTypical Data:

EthnicityIEPEconomically DisadvantagedGenderMobilityEnrollmentAttendanceTeacher Demographics?

ImpactsNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action22Demographics to DisaggregateDisaggregation is not a problem solving strategy..Its a problem finding strategy.New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action23Student Learning AND DemographicsAre all students performing at the same level? IEP students? LEP students? Economically disadvantaged students?Is the achievement gap (between high and low poverty students) decreasing or increasing?Do students who attend school every day get better grades?Are achievement levels higher for those students who stay in a school building for two or more years?New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action24What are your demographic measures?StudentsCommunity

Teachers New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action25Multiple Measures of School Processes (Programs)Typical Data:

Description of school programs and processes.How are students identified for programs and services?

Impacts and IndicatorsNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action26Student Learning AND Demographics AND School ProcessesAre there differences in achievement scores (or in rates of progress) for 11th grade females and males by the type of career program in which they are enrolled?New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action27What are your programs or procedures/processes? TutoringTitle I Grading policy Enrollment into a CTCPart time CTC transportation issue New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action28Multiple Measures of PerceptionsTypical Data:

Perceptions of Learning EnvironmentSchool ClimateValues and BeliefsObservations

ImpactsNew Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action29Student Learning AND Demographics AND PerceptionsDo students of different ethnicities perceive the learning environment differently, and do they score differently on standardized achievement tests consistent with these perceptions?

New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action30What are your measures of perception? Teachers Students ParentsSending Districts New Castle Area SD Administrator's RetreatAugust 12, 2004Shula Nedley, Ph.D. Data In Action31Identify your Impact and Indicators by their typeBlue Dots Student Learning

Yellow Dots Demographics

Red Dots School Processes and Programs

Green Dots Perceptions

Whats missing?

What additional data should be collected? Examined? Considered?

Data AnalysisNow that the data is gathered (or it is on the to be gathered list), its time to analyze the data.Remember..Numbers are our friendsData AnalysisGathering your PSSA data using the Feeder Report from eMetricWhat percent of 11th graders (in 2010) scored Below Basic, Basic, Proficient, and Advanced in Reading? Math? (These are this years 12th graders)What about the class of 2010 (PSSA grade 11 in 2009)? The class of 2009 (PSSA grade 11 in 2008)?Examine the three year trend of 11th grade performance in reading and math.Observations just the facts!Are more students reaching proficiency?Are fewer students below basic?Repeat the above looking at:Current 9th graders (8th graders in 2010)Current 10th graders 8th grade PSSA scores (from 2009)Current 11th graders 8th grade PSSA scores (from 2008)Data AnalysisDisaggregation is a problem finding strategy!11th GradeBy ProgramBy GenderBy Sending DistrictBy Reporting Categories8th GradeBy ProgramBy GenderBy Sending DistrictBy Reporting CategoriesWhy all this data?Root Cause AnalysisRule #1 no blaming othersThe Blame Poem

Observation and ReflectionWhat are you seeing?JUST THE FACTS!What are you thinking about the results? Whats causing these results?More females are proficient than males.Over the past three years, the percent of students reaching proficiency has increased.The percent of students below basic has remained constant over three years.

Students dont arrive at my grade level as prepared as they should be.Support programs are lacking.

Root Cause Analysis (Paul Preuss)Definition the deepest underlying cause, or causes, of positive or negative symptoms within any process that if dissolved would result in elimination, or substantial reduction, of the symptom.Root cause analysis eliminates patching and wasted effort.Root cause analysis conserves scarce resources.Root cause analysis induces discussion and reflection.How do you know youve found the root cause?You run into a dead end asking what caused the proposed root cause.Everyone agrees that this is a root cause.The cause is logical, makes sense, and provides clarify to the problem.The cause is something that you can influence and control.If the cause is dissolved, there is realistic hope that the problem can be reduced or prevented in the future.School improvement teams and others using root cause analysis often wonder when to stop seeking cause and make the decision that sufficient data and effort have been used to arrive at a reasonable root. This is often a judgment call that will improve with experience. Often, the lack of data and the pressures of time frustrate the effort and force it to halt at a level below the surface symptom, but perhaps not as deep as it must ultimately go. (Preuss 2003)Root Cause Analysis prerequisitesKey Indicators of Student SuccessMeasures of each indicatorDesired Ideal Condition of the indicator (e.g., 56% proficient or better)Gap between the desired ideal condition and the present conditionIs this gap a priority issue?Goal statementSearch for Root CausePossible strategies for improvement

Root Cause ProcessesQuestioning the DataThe Diagnostic TreeThe Five WhysForce Field Analysis

Throughout each process, reflect back on to your list of impacts and indicators

Questioning DataWhat do you see?What questions do you have about what you see?

Questioning the Data a la Dr. Shula:What do you see? JUST THE FACTSWhat are you thinking/feeling/believing about what you see?What other data or data analysis might shed more light on the issue?The Diagnostic TreeThe Red Flag event or priority issueLocation LevelHypotheses Level

PSSA Math scores are below AYP targetLocation - incoming 9th graders from X Middle SchoolsHypotheses Is this related to Student Demographics? Curriculum? Instruction? System Processes? Organizational Culture? External Factors:The Five WhysWhy?Why?Why?Why?Why?Team: Why do we have so many class tardies?Students: Because we do not have enough time.Team: Why dont you have enough time to get from one class to another?Students: Because 4 minutes isnt enough time to get from one end of the building to the next and go to locker or rest room.Team: Why only 4 minutes?Principal: Because we wanted to reduce the time that students were in the halls.Team: Why did we want to reduce the hall time?Principal: Because we wanted to reduce disciplinary problems.Team: Why did we want to reduce disciplinary problems?Principal: We wanted to improve school safety and climate.Force Field Analysis Driving Forces and Restraining ForcesDriving Forces apply pressure to move in a direction of changeRestraining Forces apply pressure to remain in placeEither the driving forces have to be increased or the restraining forces have to be decreased.Pulling it together with the your Final ReportThe Final Report PACTA PIL ProgramBrief introductionThree year analysis of reading and math scoresStrengthsDeficienciesRoot Causes for each CTE ProgramAction Plans to address the Root CauseTimeline for Implementing & MonitoringSharing & Reporting OutNew insights?Additional data/information to be gathered and examined?New theories?Next steps