Data Culture Survey
Analysis
August 2014
1
• Understand why we are conducting data culture surveys
• Analyze district and school results of the data culture survey and plan for implications at your school
• Provide feedback to the data culture team on the goals and direction moving forward
2
Objectives
• The 2020 Denver Plan calls for dramatic acceleration of student learning and growth with specific goals around: – College and Career Readiness– School Readiness – Eliminating the opportunity gaps – Support for the whole child
• The district believes that a focus on data driven instruction is a key lever for meeting the goals of the Denver Plan
3
Purpose
• In Leverage Leadership and Driven by Data Paul Bambrick-Santoyo states that a key principle of data driven instruction is that “Effective instruction is not about whether we taught it. It’s about whether the students learned it.” DDI includes creating a culture of using assessment and data to drive action and instruction.
• He presents numerous case studies of the dramatic student achievement gains (often higher than 15%) by focusing on implementing data-driven instructional systems and culture.
4
Why Data-Driven Instruction?
Informing our Improvement Strategy:Focus on Data Driven Instruction
• Hypothesis: Deep implementation of data driven instruction will accelerate student achievement in all categories
• Implementation:– RELAY– DDI district initiatives (SCAN, etc)– Data Culture Framework– Student Learning Objectives (SLOs)– Professional Learning Days (green/blue days)
• Initial findings:– Promising results in schools that have shown exemplary data driven
practice.
5
Initial Results: Growth Among Schools Engaged in Data Driven Instruction
DSST: Green Val-ley Ranch HS
DSST: Stapleton MS
Force Elementary School
Grant Beacon Middle School
High-Tech Early College
Skinner Middle School
STRIVE Prep - Federal
0
50
100
150
200
250
65 6957 60 57 50 49
9378.5
52
71
4964
49
68 7465
5749
57 60
226 221.5
174188
155171
158
TCAP MGP for Data-Driven SchoolsReading Math Writing Sum
6
DPS: 163
7
Monitoring our Progress from 2012
• Create a vision for data inquiry at the district, school, and classroom level•Align tools, resources, and
systems to support effective data use practices• Build capacity of school
leaders in utilizing data inquiry
What were we trying to learn by conducting the Data Culture Survey?
Questions we were trying to answer:
• In what areas—if any—has the district (teachers, schools, and the central office) made progress in the implementation of effective data use practices?• How—if at all—have understandings of effective data use
practices shifted since the prior survey administration?• In what areas is there room for improving data use practices
at the classroom, school, and central office levels?• How can the district invest its resources to address areas for
improvement and to build upon areas of success?
Objective: To develop a reliable picture of growth and gaps in the district’s data culture in order to prioritize professional development and other supports to promote adoption of effective data use practices
8
•Teachers 59%•School Leaders: 90%•Central Office Personnel: 72%
9
Participation Rates
10
Analyze: Celebrations
Identify Student Strengths ...
Provide Timely Descriptive F...
Make Instructional Decisions
Collaberation with Other Tea...
Form
ative
Ass
essm
ents
Inte
rim A
sses
smen
ts
87%
68%
85%
29%
91%
74%
90%
37%
Assessment Use
2014 2012
11
Analyze: Celebrations
Very Familiar Not Familiar at All
92%
79%73%
58%
Impact of Data Inquiry Cycle Knowledge on Implementa-tion
Formative AssessmentInterim Assessment
12
Analyze: Celebrations
31%
21%
16%
15%
14%
4%
Access to Data Inquiry Cycle
School LeadersDistrict TrainingDistrict Resources (Standards Toolkit)TeachersCoachesCentral Office Per-sonnel and Iss
13
Analyze: Celebrations
1 2
69%
82%
School Leader Awareness of DPS Data Inquiry Cycle
2012 2014
14
Analyze: Celebrations
School-Level PD District-Level PD
91%
85%
99%
94%
Professional Development for Effective Data UseTeachersSchool Leaders
15
Analyze: Identified Gaps
“How often do you examine formative data?”
“How often do you examine interim data?”
2012 2014
Rarely 5% 3%
Monthly 19% 16%
Weekly 44% 46%
Daily 33% 35%
2012 2014
Rarely 8% 9%
After some units
22% 21%
After most units
29% 28%
After all units 42% 42%
While teachers report increases in engagement in most data use practices, formative, interim and summative data frequency remains unchanged.
16
Analyze: Identified Gaps
1 2
56% 56%
Familiarity with DPS Data Inquiry Cycle
2012 2014
School leader and teacher familiarity with the inquiry cycle and access of the data inquiry toolkit varies across networks.
Familiarity with the inquiry cycle varies greatly from one network to the next:
School LeaderVery or Moderately Familiar: 74% to 100%
TeacherVery or Moderately Familiar: 28% to 49%
Use of the Data Inquiry Toolkit to access resources to support data use varies across networks.
Among school leaders, this percentage
varies from 14% to 79% depending on the network.
Among teachers, this percentage varies
from 7% to 24% depending on the network.
17
Analyze: Identified Gaps
18
Analyze: Identified Gaps
Teachers School Leaders Central Office Personnel
20%
43%
25%
Accessing Data Inquiry Toolkit
19
Analyze: Identified Gaps
Central Office Personnel School Leaders Teachers
50%
44%
31%
Common, Clear, and Consistent Vision for Data Decision-Driven
Series1
Analyze: Summary of findings from the survey
Teachers are slightly more engaged in formative, interim, and summative data use practices. Frequency of data use has not increased since 2012.
Awareness and knowledge of the DPS inquiry cycle and tools varies across the district.
Despite district articulation of an inquiry cycle, there remains a lack of clarity about how data will be used at the school level.
A significant majority of teachers report that existing school- and district-level PD has contributed to their ability to effectively use data.
1
2
3
4
20
2012 Area of Focus Rating Evidence
Create a vision for data inquiry at the district, school, and classroom level
While we have created the DPS Data Inquiry Cycle, it alone is not a vision for Data Culture. We will need to further clarify Data Culture Vision and expectations for data use practices.
Align tools, resources, and systems to support effective data use practices
The knowledge of the cycle and available resources varies across DPS. We will need to expand the availability of resources and continue to focus on aligning systems and tools to support effective practices.
Build capacity of school leaders in utilizing data inquiry
While school leaders have reported an increase in data use practices, teachers have not reported an increase. We will need to expand the audience and the availability of professional learning to continue to build capacity.
21
Analyze: Are we on track?
22
Plan
2012 Area of Focus Rating 2014-2015 Action Steps
Create a vision for data inquiry at the district, school, and classroom level
The development and use of the Data Culture Framework will communicate clear expectations for teacher and school leaders to engage in effective data use practices as well as provide a roadmap for implementation and building/enhancing data culture and assessment practices.
23
Plan
2012 Area of Focus Rating 2014-2015 Action Steps
Further promote awareness of data inquiry resources, tools, and expectations.
The Standards Toolkit can be better leveraged by leadership in order to support its use by teachers.
Expanding the School Data Culture Team creates an opportunity to provide differentiated support to network teams and TECs, build capacity for implementing data practices, and the use of available resources.
24
Plan
2012 Area of Focus Rating 2014-2015 Action Steps
Enhance professional learning that targets a variety of school skill sets across the district and includes opportunities for collaborative data analysis, instructionally-relevant content, and time for participants to apply skills.
The School Data Culture Team provides differentiated support and opportunities for professional learning, including the tier 1 and tier 2 schools in the Foundations of School Data Culture Standards Pathway.
Professional learning opportunities will move from optional sessions for school leaders only, to a more robust menu of options including network teams, coaches, teacher leaders, teachers through online and blended learning.
• Implement– Data Culture Framework – Blue/Green Day Guidance – Professional Learning: affinity groups, teacher leader cohorts, TEC’s,
school leaders’ course catalogue, online modules, networks and partners– Foundations of School Data Culture Standards Pathway
• Evaluate– Professional Learning Surveys – Blue/Green Day Surveys – Data Culture Framework Results – Data Culture Survey (Spring of 2015) – Observations, interviews, and focus groups
• Adjust – After fall and winter interims – After spring break – Summer 2015 25
Implement, Evaluate, Adjust: Plan for Monitoring Progress
• Analyze– Identify celebrations and gaps – Determine/brainstorm root cause
• Plan – To address gaps – Utilize tools and resources found on the standards
toolkit • Implement, Evaluate, and Adjust – How will you monitor progress? (consider systems in
place like your UIP and the data culture rubric)
26
Practice: Analyze your network or school’s results from the data culture survey
• Script what you’ll say using the template • Role play with a partner • Use cheat sheet to provide feedback • Redo implementing the feedback you’ve
received
27
Practice: Role play presenting findings (either district or school) to your staff
Appendix
28
Formative Data Use• To identify strengths and weaknesses in student learning (in both specific content areas as well as English language
development for English language leaners)• To provide timely descriptive feedback to students• To make instructional decisions
Interim Data Use• To evaluate student performance• To modify instruction for my whole class, small groups, and individual students• To monitor student progress towards content standards mastery• To identify student-specific strengths and weaknesses (in both specific content areas as well as English language
development for English language leaners) against a defined set of content standards• To set goals for student learning• In comparison with other teachers’ students to identify successful strategies and make changes to instruction to
improve student learning
Summative Data Use• To evaluate student performance (in both specific content areas as well as English language development for
English language learners) against a defined set of content standards• Of outgoing students to monitor and assess student progress towards goals• Of outgoing students to inform how I can improve my teaching for the upcoming year• Of incoming students to determine how to pace classroom instruction• Of incoming students to determine how to group students to maximize their learning
APPENDIX: Teacher Data Use Practices by Category
29
Includes using data to:► Establish a baseline of student
performance► Identify strengths and weaknesses in
student learning► Determine instructional needs► Inform student placements and
interventions► Allocate resources for reteaching content► Monitor alignment of the written
curriculum, classroom instruction, and student assessment
► Develop goals and relevant action items to be documented in the UIP
► Monitor the success of school improvement efforts
APPENDIX: School Leader Data Use Practices
30