Upload
brice-carroll
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Integrating Student Standardized and Interim Assessments
with National Board's Teacher Development Process
The Eleventh MARCES/MSDE EventOctober 20, 2011
• Overview of how National Board assesses teacher practice and provides standardized feedback based on the Five Core Propositions
• A case study of integrating both student standardized and interim assessments with teacher professional development
• Bringing together the results of teacher and student assessments to enhance teacher practice and student learning
• Considerations for taking these lessons to scale
Session Goals
A Way Forward• The U.S. educational accountability movement has not
yet delivered desired results
– U.S. performance on the OECD’s PISA*
• 14th in reading
• 17th in science
• 25th in math
• We need to support teaching practice – the tools and methods of our learning communities
(Organization for Economic Cooperation and Development’s Programme for International Student Assessment)
• Over the last 20 years, the nation’s focus on measuring educational results has grown and intensified
• Historically, two separate conversations
• Student outcomes
• Teacher evaluation
• NBPTS has always believed that the success of teachers in promoting student learning should be a defining measure of an accomplished teachers
• Teacher demonstration of student learning is integral to becoming an NBCT
Balanced Focus
Policy - Outcome
• Student Achievement• Teacher Evaluation
Practice - Input
• Student Learning• Teacher Growth
The Challenge
How do we leverage accomplished teacher practice to address the societal need of
graduating well-educated students?
Provide states and school districts whole school transformation tools and processes that are:
– Successful– Systematic– Scalable– Sustainable
NBPTS’ Approach
Take One! Whole School Transformation
• Validate a process to improve teaching effectiveness and student learning in low-performing schools
• Assess NB’s Take One! professional development feeder pattern approach in strengthening teacher effectiveness and student outcomes
• Determine feasibility of taking to scale
• Compare two feeder school systems, using one as a control
• Through December 2013
• Initial year, 2010-11, funded by the Gates Foundation
To better understand appropriate measures of teacher practice linked to student performance, NBPTS convened a task force of educational experts in assessment, reform and measuring teacher quality.
Lloyd Bond, University of North Carolina, Greensboro
Peggy Carr, National Center for Education Statistics
Linda Darling-Hammond, Stanford University
Doug Harris, University of Wisconsin, Madison
Frederick Hess, American Enterprise Institute
Robert Linn (chair), University of Colorado, Boulder
Lee Shulman, Carnegie Foundation
Task Force Members
Greater clarity will net better outcomes for college and workforce preparedness
• Student achievement is the status of subject-matter knowledge, understandings and skills at one point in time and is commonly measured by standardized tests.
• Student learning is the growth in subject-matter knowledge, understandings and skills over time. Learning is gauged by comparing student mastery at successive points in time.
Clarity in Language
• Be grounded in student learning, not student achievement– “The only defensible way to determine teacher effectiveness is to focus on
the gains that students have realized over the period during which the teacher provided instruction.”
• Employ measures of student learning explicitly aligned with the elements of curriculum for which the teachers are responsible– “Tests may need to be differentiated to address the needs of the groups of
students being taught, including students with disabilities or language-acquisition needs.”
Recommendations for Teacher Evaluation
As a measure of teacher effectiveness, they must meet the following minimum conditions
• Curriculum-related scale with equivalent unit of measure along a considerable continuum of achievement
• Information on validity of tests for assessing special populations
• A data system that tracks students and links to teachers
• Alignment between tests and state curricula
Criteria for Large-Scale Standardized Tests
What is formally tested in core subjects only
Knowledge and learning that can be measured
All classroom learning
From Learning to MeasuringWhat is Tested Does Count,
But Much of What Counts Is Not Measured
Why Jefferson County School District??
National: NBPTS and Gates
Local:Jefferson County
Schools (Superintendent
Hammonds)
Factors That Make The Difference
• A Structured Approach• Buy-in that Creates a Shift in School Culture• Clarity of Roles• Feeder Systems
Take One Question
Alabama Quality Teaching Standards, Teacher Evaluation,
Professional Learning Plan
Response to Instruction
(RTI)
Continuous Improvement
Plan(CIP)
NBPTSStandards
ForEntry 2
2. What are the relevant characteristics of this class that influenced your instructional strategies for this learning sequence: ethnic, cultural, and linguistic diversity; the range of abilities of the students; the personality of the class?
1.5 Provides instructional accommodations , modifications, and adaptations to meet the needs of each individual learner.
4.7 Understands the characteristics of exceptionality in learning.
Tier IProgress monitoring all studentsIdentification of needs
Part I of the CIP I. Knowledge of Learners II. Knowledge of the Field of Literacy: Reading–Language Arts III. Equity, Fairness, and Diversity IV. Learning Environment V. Instructional Resources VI. Instructional Decision Making VII. Assessment VIII. Integration IX. Reading XIV. Teacher as Learner
A Structured Approach: Alignment of Teacher Tasks
Buy-in to Change School Culture• Critical Mass of Teachers
– Works collaboratively with peers around a common purpose– With colleagues, addresses issues across the feeder system – Focused dialogue and peer support
• Teacher Leader– Augments principals with instructional leadership– Facilitate collaborations, structured discussions, feedback and support
• Principals– Demonstrated leadership through changed interactions with teachers – Structure of environment that supports focused collaboration– Serves as key instructional leader
• School District– Superintendent involvement & presence– Deployment of human capital– Alignment of resources supportive of implementation
• Train-the-trainer model
– Project Director and Coordinator provide leadership and support to principals who will meet once monthly to receive training on guiding their faculty.
• All-participant group meetings– Host four meetings to build professional learning communities among the
feeder school system meetings will focus on, for example, reflective thinking.
• Small group meetings– Twice monthly, principal and facilitator will meet with faculty to guide the
process, lead peer reviews, and provide teacher support.
Project Approach
Feeder System• Permits a systems approach, creating more feedback loops
during the entire K-12 experience
• Facilitates a more long-term connection to students, enabling teachers to preview & follow-up about students
• Strengthens collaboration among teachers at different schools to build connections, strengthen practice & resolve issues
• Enables early identification of student issues, while successful intervention is meaningful
• Enables principal cohort & learning community focused on instruction & classroom improvements
• Eight low-performing schools
• 63% of schools receive Title I funding
• 58% of feeder pattern’s students receiving free/reduced lunch
• High school has failed to meet AYP for the last 5 years
• 210 teachers participating – 70% of all teachers
Feeder System Characteristics
Freq. Respondent Group
Assessment
Broader Learning
CommunityPrincipal Certified
StaffClassified
Staff Parent Student
National Board Take One! Yearly ü ü ü
Engagement Survey, Pre and Post Yearly ü ü ü ü ü ü
Standardized Tests• Alabama Reading and Math
Test/SAT-10 (Grades 3-8)
• EXPLORE and PLAN (Grades 8-11)
Yearly ü
Add’l Student Growth Measures (e.g., attendance rates, promotion/retention rates, graduation /dropout rates)
Yearly ü
Measures
StudentWork
Small Groups Video
Entry 2
Whole GroupVideo
Entry 3
DocumentedAccomplishments
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Exercise 5
Exercise 6
Portfolio Assessment Center
Take One! One Entry of NBPTS’ Certification Process
Teaching Skill
Port. #1
Port. #2
Port. #3
Port. #4
Content Knowledge
AC #1AC #2
AC #3
AC #4
AC #5
AC #6
ε
ε
ε
ε
ε
ε
ε
ε
ε
ε
NBPTS’ Certification Assessment 2-Factor Confirmatory Factor Analysis
Feedback/Comment area Overall (N=165)A. More evidence of knowledge of child development, knowledge of
students, and insight into learning38.8%
B. Evidence of knowledge of the subject as you appropriately design and deliver your classroom lessons
47.3%
C. Evidence that demonstrates your ability to employ effective instructional strategies to facilitate individual participation
63.0%
D. Evidence that you develop a relevant learning sequence 38.8%
E. Evidence of your ability to foster a purposeful learning environment in which all students are encouraged to participate
50.9%
F. Evidence of your ability to set worthwhile goals for learning. 53.9%
G. Evidence that you use a variety of appropriate resources to enhance student learning.
64.2%
H. Evidence that you employ appropriate assessments and provide constructive feedback.
66.7%
I. Evidence that you are able to describe your own practice and reflect on instructional decisions
6.1%
Summary of Standardized Feedback – Year 1
Feedback/Comment area ELEM (N=19)
ELEM (N=21)
ELEM (N=6)
ELEM (N=12)
ELEM (N=24)
ELEM (N=13)
MIDDLE (N=29)
HS (N=41)
Overall (N=165)
A. More evidence of knowledge of child development, knowledge of students, and insight into learning
26.3% 33.3% 0% 58.3% 41.7% 30.8% 41.4% 46.3% 38.8%
B. Evidence of knowledge of the subject as you appropriately design and deliver your classroom lessons
52.6% 42.9% 83.3% 41.7% 58.3% 53.8% 51.7% 31.7% 47.3%
C. Evidence that demonstrates your ability to employ effective instructional strategies to facilitate individual participation
31.6% 66.7% 50.0% 50.0% 41.7% 84.6% 72.4% 80.5% 63.0%
D. Evidence that you develop a relevant learning sequence 15.8% 33.3% 33.3% 50.0% 29.2% 38.5% 58.6% 41.5% 38.8%
E. Evidence of your ability to foster a purposeful learning environment in which all students are encouraged to participate
26.3% 57.1% 33.3% 75.0% 45.8% 30.8% 51.7% 63.4% 50.9%
F. Evidence of your ability to set worthwhile goals for learning. 52.6% 66.7% 33.3% 66.7% 54.2% 46.2% 48.3% 53.7% 53.9%
G. evidence that you use a variety of appropriate resources to enhance student learning.
84.2% 57.1% 33.3% 50.0% 54.2% 69.2% 69.0% 68.3% 64.2%
H. evidence that you employ appropriate assessments and provide constructive feedback.
68.4% 52.4% 50.0% 58.3% 70.8% 69.2% 86.2% 61.0% 66.7%
I. evidence that you are able to describe your own practice and reflect on instructional decisions
0% 14.3% 0% 25.0% 8.3% 0% 0% 4.9% 6.1%
Learning Community Engagement and Leadership SurveyFormative and Summative Administration
EngagementPerceptions About the Learning Community– Principal– Students– Parents– Certified Staff– Classified Staff– Community Members
Leadership 360Perceptions About the Principal’s Behaviors– Principal– Supervisor– Peers– Certified Staff– Classified Staff
Leadership BehaviorsCluster Behaviors
Thinking Gathering DataAnalyzing Information and IdeasThinking FlexiblyReflective Practice
Developing Valuing Others and Their PerspectivesCollaborationDeveloping Human Capital
Inspiring Cultivating Ownership and SupportBuilding ConfidenceCommunication
Achieving Planning, Implementing and EmpoweringGoal Setting and MonitoringStakeholder Centered
Mapped to the Standards in Behavioral Language
Respondent Group
Engagement Survey Dimension
Broader Learning
CommunityPrincipal Certified
StaffClassified
Staff Parent Student x2
1. Level of Academic Challenge ü ü ü ü ü
2. Student-Staff Interaction ü ü ü ü ü
3. Active and Collaborative Learning ü ü ü ü ü
4. Active and Collaborative Teaching ü ü ü
5. Enhancing Education Experiences ü ü ü ü ü
6. Student Commitment Levels ü ü ü ü ü
7. The School Environment ü ü ü ü ü
8. Parent-Learning Community Interaction ü ü ü ü
9. Work and Community Engagement ü ü ü
10. Organizational Citizenship ü ü ü
11. Work Withdrawal ü ü ü
12. Turnover Intentions ü ü ü
13. Broader Learning Community Engagement ü
Cross-Rater Items
Questions to Respondent Group
“Teachers present materials at a level our students can understand.”
“I present materials at a level our students can understand.”
“My teachers present materials at a level I can understand.”“My teachers give lessons at a level I can understand.”
“Teachers present materials at a level my child can understand.”
Pre-post Reporting
Benefits of Student Measures
• Longitudinal results• Guide student development• Positions college as a viable opportunity
Alabama Reading and Math TestThe 4th & 5th grade teachers are new
Grade 4
Grade 4
Grade 5
Grade 5
Grade 5
Grade 5
Math 2011 (post) 614 619 633 638 660 630
Math 2010 (baseline) 587 624 589 622 627 613
Math Gain +27 -5 +44 +16 +33 +17Reading 2011 (post) 611 613 624 629 653 591
Reading 2010 (baseline) 593 596 619 600 641 570
Reading Gain +18 +17 +5 +29 +12 +21
How Are We Doing?Some Interim Assessments
What is formally test in core subjects only
Knowledge and learning that can be measured
All classroom learning
Interim Integrated Classroom Evaluation
Knowing Your
Students
data
attendance
background
Engagement/efficacy
Know Students and Subject AreaThe Architecture of Accomplished Teaching
Step 1: Know Students and Subject Area
Areas to know about your students• Where they are intellectually,
socially & emotionally• Whether they have special
learning needs• How they feel most comfortable
learning• What they know & do not know• What they can & cannot do• What they are interested in • How any issues or event taking
place in their lives may affect learning
Resources • Personal observation of students• Interest inventories & surveys• Student work products• Report cards• Records from previous schools• Standardized tests scores• IEPs• Student extracurricular activities• Results from diagnostics testing• Previous teachers• Guidance counselors• Family members• Students themselves
Student Engagemen
t
Deep Questionin
g
Classroom Practice
Peer Video
Review
Teacher Monitoring
Interim Integrated Classroom EvaluationPre Interim Post
Evaluate Student LearningThe Architecture of Accomplished Teaching
Step 4: Evaluate Student LearningFollowing instruction, evaluate student learning to see if goals were met.
Analyzing Your Video• Extent of classroom involvement• Extent of student engagement in lesson• Evidence of student learning• Student success in meeting learning
goals• Types of questions you asked• Opportunities for students to ask
questions• Adjustment in lesson so learning goals
achieved by every student
Interim Integrated Classroom Evaluation
• Evaluate practice and learning against rubric
Review
Reflect on Teaching PracticeThe Architecture of Accomplished Teaching
5th
Step 5: Reflect on Teaching Practice• What would I do
differently? • What are my next steps?
Reflection• Link reflection to knowledge of your
students• Link reflection back to goals of lesson• Cite specific strengths and weaknesses• Describe appropriateness of your
instructional choices• Provide clear, consistent & convincing
evidence & examples of student learning
• Discuss how you continue with your instruction
• Discuss any modifications to advance student learning
Take One! Submission Rate Comparison
General Population
50%
Take One! Whole School Transformation
80%
2010-11 2011-12
N Total N % N Total N %
NBC N/A N/A N/A 42 277 15%
Take One! 215 310 69% 219 277 79%
NBCP N/A N/A N/A 2 8 25%
Principal Take One! N/A N/A N/A 4 5 80%
TOTAL 215 310 69% 267 290 92%
Participation Increase
Status Preliminary reactions are positive• Teachers - talking, planning, and communicating with each other• Teachers - excited to have the opportunity to work with others in their
field from across the feeder pattern, an atypical experience• Principals - connecting with teachers in new ways through Take One!• Principals - communicating about student support• Superintendent – encouraging his central office staff, principals,
teachers and other staff to build on the success• Everyone -- Identifying strengths and weaknesses across the schools in
the feeder pattern• Everyone - seeing the students as “their students,” a collective
responsibility
Where Are We Going?• Refined year one and building year two• Developing a structured process with a supporting suite
of tools that:– Defines the elements for success– Road-tested and readily available– Is affordable to implement & sustain– Facilitates continued school growth – multi-year– Is research-based
• Getting the word out– Department of Education– Alabama publications
Year One• Teacher selects favorite
lesson to teach
• External experts support teachers after school
• After school cohort meetings
Year Two• Teacher evaluates data and
selects lesson to support area of student need
• Identify and train school-based teacher leaders to build capacity
• In-school cohort meetings
Evaluation and Modification of Program
Measurement is only part of the story
Moving from measuring teacher effectiveness
to…
Building teacher capacity
Move from 30% teachers based on student achievement to 100% teachers based on student growth
Provides the ‘how to’ for developing a sustainable a teacher evaluation and support system that delivers learning for all students
For more informationon NBPTS, visit
nbpts.org