Upload
north-carolina-association-for-middle-level-education
View
865
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Building Data Literacy Among Middle School Administrators and Teachers Data literacy is an essential trait for middle school administrators and teachers to possess. In this session, the Research and Accountability Team from Durham Public Schools will discuss how it has expanded its focus on Data-to-Action to building data literacy amongst its middle school administrators and teachers during 2013-14. J. Brent Cooper, Terri Mozingo & Karin Beckett Durham Public Schools - Durham, NC
Citation preview
Data-to-Action: Building Middle School Administrators’ and Teachers’ Data Literacy Capacity
in Durham Public Schools
04/12/2023 1
2014 North Carolina Association of Middle Level Educators Conference (NCAMLE)
March 17, 2014
Dr. J. Brent Cooper Dr. Terri Mozingo
Desired Outcomes• Describe actions, strategies, and trainings within DPS that support the
development of a Data-to-Action, data literate culture amongst central office/school administrators and teachers.
• Share ways school administrators’ data literacy skills have been developed through work with the Deputy Superintendent of Academic Services, Dr. Stacey Wilson-Norman, and Research and Accountability during 2013-14.
• Present the Area Network Model as applied to the work of the Middle School Area Network through the leadership of Dr. Julie Spencer, Middle School Area Superintendent.
• Summarize lessons learned.• Project the course of Data Literacy trainings for the rest of 2013-14 and for the
2014-15 school year.
2
3
Responses to Data
Significance of Data Literacy Training
• Increase emphasis on developing high-quality, data literate school administrators to lead schools.
• Empower principals and central office leaders to develop data literate assistant principals and teachers.
4
What Does it Mean to Be Data Literate?
A data literate person possesses the knowledge to gather, analyze, synthesize, and convey information graphically and in writing to support decision making.
04/12/2023 5
Skills of a Data Literate Person
Data Location
Data Comprehension
Data Interpretation
Instructional Decision Making Question Posing
04/12/2023 6U.S. Department of Education report Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011)
Find Data Evaluate Data
Use Data
Data Location• The ability to find relevant data• The ability to manipulate data from a complex
table or graph to support reasoning
04/12/2023 7In the U.S. Department of Education report Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011).
Data Interpretation
• Examining score distributions• Understanding the effects of outliers• Appreciating limits of generalizability• Understanding measurement error
04/12/2023 8U.S. Department of Education report Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011).
Data Use for Instructional Decision Making• Understanding the value of scale scores and item
level data• Using student data to plan differentiated
instruction based on student needs• Synthesizing multiple data sources to inform
instructional practices
04/12/2023 9
U.S. Department of Education report Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011),.
Question Posing• Aligning questions with purpose and data• Forming queries that lead to actionable data• Appreciating the value of multiple measures
04/12/2023 10U.S. Department of Education report Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011), the research team identified five skills
Four Types of Data
04/12/2023 11
Type Definition Example1) Achievement or Assessment Data
Data used to determine the level of student achievement in a particular content area
Performance-based assessments, written exams, or quizzes
2) Demographic Data Data that provides descriptive information about the school community and the students served in the community
Enrollment, gender, ethnicity, economic status, student attendance, grade levels, school suspensions, limited English proficiency status, and special education status
3) Program Data Data that defines the programs, instructional strategies, and classroom practices of the teachers
Assessment/Achievement Data, Attendance, Perception Data, and Implementation Records
4) Perception Data Data that tells us what students, parents, staff, and others think about the learning environment
Questionnaires, interviews, surveys, and observations
Source: Guide to Using Data in School Improvement Efforts: A Compilation of Knowledge From Data Retreats and Data Use at Learning Points Associates (December 2004)
Assessment and Achievement Data Tiers
04/12/2023 12
Assessment Purpose
Rate of Feedback
Type of Feedback
Example Primary Target of Feedback
Tier IV(Federal)
Annual large scale testing for Federal
Accountability
Infrequent General and broad
Condition of Education
Federal and State Policymakers
Tax payersAdministrators
Tier III(State)
Annual large scale testing for
accountability
Infrequent General and broad
Summative
End-of-Grade ExamsEnd of Course
Exams
PolicymakersBoard Members
Community Administrators
Tier II(District)
Periodic grade level and subject area
Infrequent/Frequent
Specific and Formative
Local Assessments AdministratorsTeachers
Tier I(Classroom)
On-going classroom
Frequent Specific and narrow
Formative
Unit examsClassroom projects
Homework Class work
TeachersStudents
Questions to Guide Achievement Data Collections
• What evidence can we collect about our students’ learning? • What evidence do we have that shows the knowledge, skills,
and understandings our students have achieved? • Which data indicate the degree to which our students show the
conceptual understandings and generalizations in our standards?
• What evidence shows which students are meeting or exceeding our achievement expectations and which are not?
• What do we know about how each individual student learns?
04/12/2023 13
Locating Achievement or Assessment Data•PAPA Datamining Reports•Achievement Series • State Testing Portal•mClass
Local(Tier II)
•NCDPI – Data and Statistics Website•University of North Carolina General Administration•Education Value-Added Assessment System (EVAAS)
State (Tier III)
• National Assessment of Educational Progress • Federal Data Warehouse
National(Tier IV)
04/12/2023 14
Types of Demographic Data
Student Demographics
• Gender• Ethnicity• Limited English Proficiency• Exceptional Children• Parent Characteristics
School Demographics
• Mobility Patterns• School Transportation Needs• LEP and EC Participation Rates• Suspension/ Expulsion Rates• Promotions / Retention Rates• Graduation/ Dropout Rates• Free/Reduced Lunch Percentages• Neighborhood Characteristics• Parent Involvement• Behavior and Social Problems of
Students04/12/2023 15
What is a Program Evaluation (PE)?
What is a Program Evaluation?• A program evaluation is a
purposeful, systematic, and careful collection and analysis of programs, establishing accountability, and identifying areas needing change and improvement.
What are the goals of a Program Evaluation?
16
• The generic goal of most evaluations is to provide “useful feedback” to a variety of audiences including sponsors, donors, client-groups, administrators, staff, and other relevant constituencies.
Why Evaluate Programs?
Evaluations are conducted to:• Gain insight about a program and its operations – to see where we
are going and where we are coming from, and to find out what works and what doesn’t.
• Improve practice – to modify or adapt practice to enhance the success of activities.
• Assess program effects to determine how well we are meeting the goals and objectives, how the program benefits the community, and to provide evidence of effectiveness.
• Help build capacity by increasing funding, enhancing skills, and strengthening accountability.
17
What are the Types of Evaluations?
Formative Evaluations Evaluability assessment Implementation
evaluation Needs assessment Process evaluation Structured
conceptualization
Summative Evaluations Cost-effectiveness and
cost-benefit analysis Impact evaluation Meta-analysis Outcome evaluations Secondary analysis
18
Examples of the Types of PEs
Formative Evaluations Evaluability assessment Implementation evaluation Needs assessment Process evaluation Structured
conceptualization
Examples of FPEs Afterschool Programs Common Core Implementation Dropout Programs Helpdesk Process Literacy Program
19
Examples of the Types of PEs
Summative Evaluations Cost-effectiveness and cost-benefit
analysis Impact evaluation Meta-analysis Outcome evaluations Secondary analysis
Examples of SPEs One-to-One Laptop Initiatives Online Instruction Effects of Early Education on Kindergarten Readiness Science, Math, Engineering and
Technology (STEM) Program School Management Practices on Student Performance
20
Constituents of School Level Perception Data
04/12/2023 21
Internal Constituents External Constituents
Students and Parents Community Citizens
Teachers and Staff Community Businesses
Administrators Regional Colleges and Universities
School Board
Guiding Questions for Collecting Perception Data
• How do the members of our school community feel about our school and district?
• How satisfied are school community members about our educational programs?
• What do the members of our school community perceive to be the strengths and needs of our school?
• What do members of our business community think about the skills of our graduates?
04/12/2023 22
Educate…Previous vs. NowPrevious• ABCs• Grade-level Proficiency
focus • Different state and
federal models
Now• NC READY• Career and College-
Ready focus• Closing Achievement
Gaps• State and Federal share
AMOs and targets • Teacher Effectiveness
23
Objectives of Data Literacy Trainings
• Discuss components of NC READY state model• Review the federal and state Annual
Measurable Objectives (AMOs)• Review the 2012-13 Data Reports• Discuss implications of data and next steps• Provide additional support to schools
24
Educate… 2012-13 NC READY and AMO Guide
• NC READY Indicators: Definitions• State and Federal Annual Measurable Objectives
(AMOs) Indicators (Federal, State, and AMO Definition)
• AMOs Terminology/Definitions
04/12/2023 25
Ma
End of Grade/End of Course
(reading, mathematics, science)
ACT
Graduation Rates
Math Course Rigor
WorkKeys
Graduation Project
Δ End of Grade/End of Course (reading,
mathematics, science)
Δ ACT
Δ Graduation Rates
Δ Math Course Rigor
StatusIndicators“this year”
ProgressIndicators“trend over time for groups of students”
Growth Indicators
“combined individual
student’s growth”
26
EVAAS Growth
School-wide
Δ WorkKeys
Communicate…Our School’s 2012-2013 Data Announcement
• Performance Composite• School-wide Growth Status• Strengths • Areas of Opportunity
27
System of Supports
Communication
Do you need help framing your story?
Understanding Data
Do you need help understanding your
data?
Next Steps
Do you need help with next steps?
28
Systems of Support
COMMUNICATION
• Public Information and
Community Engagement
(PICE)
DATA SUPPORT
• Research and Accountability
NEXT STEPS
• Area Support (Elementary, Middle, High
School)
29
Next Steps
School Improvement Plan Review and
Revisions
Student Intervention-School Level Reports
in coming weeks
EVAAS Value-Added Reporting - More information after
Thanksgiving
Communicating with the Community – No
Surprise, Baseline Year, Continue Focus
on Growth
30
Research and AccountabilityDecember 4, 2013
“It’s not about giving the assessments; it’s about doing something about
the results”Dr. Doug Reeves (2005)
EVAASGrowth
AMOTargets
Proficiencies
Academic Services Data Summit
Research Articles
Data Release• After the initial release of the data, we will do
the following, as a middle school:– Review the data with our School Improvement
Team– Review student data to determine if we need to
provide intervention– Provide teachers more information about the
EVASS value-added data that is included in Standard 6 of the teacher evaluation instrument
– Host a Parent Meeting to go over our school data 32
Data Analysis Process
Essential Questions
AnalysisReports
Synthesis
Essential Questions: AMO Targets
34
Question to Guide Student AMO Targets Discussion.
What are our biggest AMO target challenges?
Essential Questions: Proficiencies
35
Questions to Guide Student Proficiencies Discussion.
1) Where are our biggest opportunities to move students to proficiency?
2) How have the new cut scores impacted AIG Proficiencies?
3) How will the new AMO’s goal impact AIG Identification?
Essential Questions: EVAAS Growth
36
Questions to Guide Education Value Added Assessment System (EVAAS) Discussion.
1)2)
In what grade levels are students experiencing academic growth? In what grade levels are students not experiencing academic growth?
3)4)
In what schools are students are experiencing academic growth? In what schools are students not experiencing academic growth?
5)6)
In what subjects are students experiencing academic growth? In what subjects are students not experiencing academic growth?
7)8)9)
In what grade are teachers not demonstrating teaching effectiveness?In what schools are teachers not demonstrating teaching effectiveness?In what subjects are teachers not demonstrating teaching effectiveness?
10)11)12)13)
In what schools are minority students experiencing the greatest achievement gaps?In what subjects are minority students experiencing the greatest achievement gaps?In what schools are exceptional students experiencing the greatest achievement gaps?In what subjects are exceptional students experiencing the greatest achievement gaps?
Elementary Area School-Level EOG Test Proficiency Questions
37
Essential Questions to Guide the Elementary Area School-Level EOG Test Proficiency Discussion
1) Are there schools with large opportunities for EOG test proficiency improvements (Focus Schools) for multiple grades levels and/or student groups?
2)Are there schools with declining EOG test proficiency when analyzing a school’s data across grades for all student groups, which could lead to a negative effect on feeder middle schools 6th grade students’ proficiency?
3) Are there student groups with opportunities for EOG test proficiency improvements in the Elementary Area?
Impact of Data Literacy on Teaching and Learning
• Data literacy capacity building will lead to continuous analysis of: NC READY, AMO, and EVAAS data, and in turn local assessment data.
• Student achievement data will be analyzed for the school and district by: grade level, test, gender, student group, and by teacher.
• Program evaluations will be completed internally and externally for many current DPS programs (process was developed and approved, Summer 2013).
DiscussionBrainstorm…
• How are you building data literacy capacity in your school/district?
• Which groups are you focusing on building data literacy capacity during this school year?
39
DiscussionBrainstorm…
• Through what types of professional development activities are you building your school’s/district’s data literacy capacity?
• What are your school’s/district’s goals in building data literacy capacity?
40
Deputy Superintendent’s Support for Building Data Literacy Amongst All DPS
Stakeholders
04/12/2023 41
Deputy Superintendent’s Support for Building Data Literacy Amongst All DPS
StakeholdersTeaming for Excellence• Documents were created and Data Release Meetings were planned in a
collaborative effort by: 1- the Superintendent 2- Deputy Superintendent 3- Academic Services 4- Research and Accountability and 5- Public Information and Community Engagement (PICE) 42
Deputy Superintendent’s Support for Building Data Literacy Amongst All DPS
StakeholdersTeaming for Excellence• The goals of these efforts were:• Ensure an understanding of the NC READY, AMO, and EVAAS
Data and that principals were well prepared when they communicated the accountability results to all DPS Stakeholders • Create a Data Release Toolkit to support the principals• Ensure that terms were understood and specific points were
clarified• Develop Data Release Guides
43
Deputy Superintendent’s Support for Building Data Literacy Amongst All DPS
Stakeholders• Develop and Offer 2013 NC READY and AMO Data Release Training
Sessions for principals by school level (area), (October-November 2013)
• Data Release Guides developed for school administrators by School Level
• Data Release Toolkit developed to educate all DPS Stakeholders about the release of NC READY, AMO, and EVAAS data
• Data Release parent and faculty meetings planned with corresponding ConnectEd messages created and sent to educate all DPS Stakeholders about the release of NC Ready, AMO, and EVAAS data
44
Deputy Superintendent’s Support for Building Data Literacy Capacity in DPS
• Develop and Offer 2013 NC READY and AMO Data Literacy Training Sessions for Principals, October-November 2013
• Develop and Offer 2013 Central Office Data Literacy Summit for Central Office Instructional Support, December 2013
45
Deputy Superintendent, Area Superintendents, and Research and Accountability2013-14
Central Office/School Administrator Data
Literacy Capacity
Identification of Area Focus Schools
2012-13 NC READYData Dives
Establishment of Area Networks
Steps to Building Data Literacy Capacity in DPS
2012-13 NC READY Data
Summits
Deputy Superintendent’s Support for Building Data Literacy Capacity in DPS
Area Networks• Implement Area Networks for the Elementary,
Middle, and High School Areas to focus data analysis and data literacy capacity building, December 2013.
• Conduct quarterly Data Dives for School Administrators and Central Office Instructional Support in our continuous data analysis, modification, and re-teaching of curriculum objectives, Spring 2014-Fall 2015. 47
48
Middle School Area Network
Area Networks
High School Area Network
Elementary Area Network
Area NetworksPurpose
Central Services departments provide rapid response services to each Area of schools through coordinated collaboration and communication around key issues. The purpose of these meetings are to remove barriers, address issues and respond to school needs in a timely and effective manner. Each meeting should be results-based with clear action items and timelines to address barriers for schools.
04/12/2023 49
50
To improve communication between Central
Services departments and
schools
Area NetworksDesired
Outcomes
To increase the level of accountability
To provide coordinated and responsive
services to schools
Area NetworksStructure
Each Area Superintendent will lead the assigned network. The Executive Leadership supervisor will assign individuals to the appropriate network to provide leadership and service for their respective departments. Every Network will have representation for content areas and departments. All networks will meet for two hours the 3rd Wednesday of the month. A standing agenda has been designed to ensure efficient, consistent and effective monthly meetings.
04/12/2023 51
Area NetworksMembers
04/12/2023 52
Network Members Network Members
Curriculum, Instruction, and Assessment
Research and Accountability
Exception Children’s Program Area Facilitators
Human Resources Student, Family, and Community Services
Leadership Development Title I
State Transformation Team
Middle School Area NetworkTeam Meeting 1
1. Establish a Middle School Area Network Working Plan2. Discussion of MSAN Data and the purpose of making
data-driven decisions3. Introduction to the MSAN Purpose4. Creation and administration of Middle School
Network Priority Survey- identify the most important problems which need immediate attention from the MSAN 53
Middle School Area NetworkTeam Meeting 2
1. Updates from the current Middle School Network Working Plan (planning days, data dives, Focus Middle School support, mentors, etc.)
2. Comprehensive Planning…what is the Middle School Area Network’s #1 problem to be addressed by the MSAN
3. Data Driven Problem Solving- problem solving in a particular area based on a data point with resulting action items
4. Establish MSAN Work Groups around major problems which need immediate attention from the MSAN.
54
Middle School Area NetworkTeam Meeting 3
1. Reporting out from Work Group collaboration
55
Data Dives
Who?
Teachers
School Admins
School-level Instructional
Support
Central Office
Instructional Support
04/12/2023 56
57
Continuous School and
District Data Analysis
Data DiveObjectives
Re-teaching for Mastery
Identification of Learning Objectives
Not Met
Deputy Superintendent’s Support for Building Data Literacy Capacity in DPS
Focus Schools• Implementation of Area Focus School Models in an
effort to provide extra resources to schools with greater academic needs.
• Focus Schools identified by an analysis of the previous year’s NC READY, AMO, and EVAAS data, Fall 2014-15.
• Six Focus Middle Schools were identified to provide additional central services support for the remainder of the 2013-14 school year. 58
Middle School Area Superintendent’s Support for Building Data Literacy Capacity in Focus Middle Schools
• Three collaborative Focus School Team Meetings between Focus Middle School School Improvement Teams and Central Services Support Staff held at the Focus School.
• The Focus Middle School team includes representatives from the multiple Central Services departments.
59
Focus School Data Analysis
60
Data Sets(CS
Depts)
NC READY(AS/
R&A/CIA)AMO(AS/
R&A/CIA)
EVAAS(AS/
R&A/CIA)AIG/EC/ESL
(CIA/EC/ESL)
Discipline:PBIS/OSS
(SFCS)
Staffing (HR)
Focus Middle Schools
61
Focus Middle Schools
CarringtonMS
Shepard MS
W.G. Pearson Magnet
MSNeal MS
Lowe’s Grove
MS
Sherwood Githens
MS
Focus School Team (School)
62
Focus School Team
ESL/LEP Facilitator
School Principal
School Improvement
Team
RtI Facilitator
AIG Facilitator
EC Facilitator
Focus School Team (District)
63
Focus School Team
Middle School Area Supt. MS Area
Facilitators(CIA, EC,
etc)
Executive Director of
ESL
Coordinator of MS
Staffing
District AIG Coordinator
District PBIS Coordinator
Focus Middle SchoolTeam Goals
1. Pre-Meeting Analysis of the Data2. Discussion of the Data3. Introduction to the Focus School Purpose4. Identification of Action Items for the Next
Meeting
64
Focus Middle SchoolTeam Meeting 1
1. Pre-Meeting Analysis of the Data2. Discussion of the Data3. Introduction to the Focus School Purpose4. Identification of Action Items for the Next
Meeting
65
Leaders and Learners Field Trips
Leaders &
Learners
MS Principals
ESPrincipals
Central Services
Staff
HS Principals
66
Leaders and Learners Field Trips
L & L Field Trips
Piedmont Open IB MS(Charlotte-
Meck)
Stories Creek
ES(Union)
Rock Rest ES
(Person)
Dudley HS(Guilford)
67
04/12/2023 68
Data Literacy Capacity Building Review
Principals
Central Office Data Summit
Area DataDives
Data Literacy Capacity Building 2013-14
04/12/2023 69
Data Literacy Capacity Building Next Steps
Leaders and Learners Field Trip (3/13/14)
(Charlotte-Meck./Guilford/Person/Union)
Char Area Networks Symposium
(Monthly)Summer Leadership
RetreatJune 2014
Spring 2013-14 and Beyond
Lessons Learned• Time must be protected for principals’/central office data literacy skills to
be developed (Monthly Principals’ Meetings, Monthly Area Principals’ Academies, Area Networks, School/Area/Curriculum and Instruction Data Dives).
• Principals must go back to their schools and develop the data literacy capacity of the assistant principals and teachers (Faculty Meetings, Grade and Subject-Level PLCs, School Improvement Teams).
• Data literacy skills must be applied to a comprehensive school and district-level analysis of NC READY and EVAAS data.
• Data literacy capacity building is a continuous process.• Data literacy capacity building will support Data-to-Action, data-driven
decisions. 70
“If you want to lift ten pounds, you can do it by yourself. If you want to lift two-hundred pounds
you’d better have a team.” ~old adage
71
T - Take time to build the foundationE - Engage in data-driven dialogueA - Act together to improve instructionM - Monitor implementation and results
Discussion Brainstorm…
• Are there questions or comments concerning data literacy capacity building in DPS, your middle school(s), or your school district?
72
Dr. Stacey Wilson-Norman Deputy Superintendent of Academic Services
Dr. Terri MozingoAssistant Superintendent of Research and Accountability
Dr. Julie SpencerMiddle School Area Superintendent
Dr. J. Brent CooperData Analyst, Program Evaluator, Coordinator of Surveys and Research
73