CFF•21st CTL Evaluation Results
Changes in Teaching Practices
Jeanne Vilberg & Robin ClausenPenn State University
What is CFF•21st CTL?
21st CTL EvaluationSince March 2007, the 21st CTL Evaluation Project has collected:
• 5,500 21st CTL Classroom Observations
• 55,000 Teacher Surveys
• 650,000 Student Surveys
• 18,000 PATI Surveys (Instructional Phase)
Analysis
Start of Program (Fall 2007/2008)
Spring 2010
• The same 21st CTL teachers compared at different points in time.
• Data collection periods (fall/spring 2007 - 2010)
• Cohorts (teachers with different number of years in program)
InstrumentsObservations
Teacher/Student SurveysPATI (Instructional Phase)
http://cff.psu.edu
Findings
Has 21st CTL Changed...
• complexity of class content
• relevance of class content
• instructional style
Overall
Complexity of Content
Basic Skills Higher Order
Relevance of Content Artificial Real World
Instructional Style Didactic Constructivist
Change in the Complexity of Content
21st CTL teachers now focus more on higher order topics than they did at the beginning of the program.
Complexity of Content Describe the content your class is designed to convey.
(Almost All Higher Order Skills / More Higher Order Skills than Content Knowledge Combined)
10%
13%
16%
19%
22%
Start of Program Spring 2010
Cohort 1Cohort 2Cohort 3
Differences5.89%6.85%4.65%
Teacher Survey
Complexity of Content Percent of Time Spent in Really Complex Thinking / Problem Solving
(Quite a Lot / Almost All the Time)
35%
37%
39%
41%
43%
Start of Program Spring 2010
Cohort 1Cohort 2Cohort 3
Student Survey
Differences2.52%2.00%1.74%
Observations
Complexity of Content
50%
54%
58%
62%
66%
70%
Start of Program Spring 2010
First ThirdMiddle ThirdLast Third
Overall Comparison by Class Period - Basic - Higher Order (Ratings of 5, 6, and 7 Combined)
Differences9.12%8.35%4.06%
Classroom Content is More Authentic
21st CTL teachers report students use technology to solve real world problems more often than non-21st CTL teachers.
Teacher Survey
Cohort 14 Years
Cohort 14 Years
Cohort 23 Years
Cohort 23 Years
Cohort 32 Years
Cohort 32 Years
Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10
Exercises & Assignments
17.48% 12.45% 19.71% 12.36% 16.67% 12.04%
Blend First / Last 42.48% 35.65% 45.98% 39.61% 47.97% 42.95%
Even Balance 30.22% 35.29% 25.59% 35.76% 28.73% 34.61%
Blend First / Last 7.40% 13.31% 7.40% 10.04% 6.10% 8.21%
Projects and Products
2.43% 3.30% 1.32% 2.23% 0.54% 1.65%
Fewer ExercisesMore Even Balance
More Projects
Fewer ExercisesMore Even Balance
More Projects
Fewer ExercisesMore Even Balance
More Projects
Fewer ExercisesMore Even Balance
More Projects
Fewer ExercisesMore Even Balance
More Projects
Fewer ExercisesMore Even Balance
More Projects
Relevance of Content Artificial - Real World
How would you describe the work that students do?
Daily
Weekly
Monthly
0% 5% 10% 15% 20% 25% 30%
Cohort 1 - 4 Years in 21st CTLCohort 2 - 3 Years in 21st CTLCohort 3 - 2 Years in 21st CTLNon-21st CTL
How often do students use technology to solve real-world problems?
Relevance of Content Artificial - Real World
PATI Survey
(Instructional Phase)
Observation
Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)
First Third of the Class
Start of ProgramFirst Third of the
ClassSpring 2010
First Third of the Class
Difference
Middle Third of the Class
Start of ProgramMiddle Third of the
ClassSpring 2010
Middle Third of the Class
Difference
Final Third of the Class
Start of ProgramFinal Third of the
ClassSpring 2010
Final Third of the Class
Difference
32.52%65.18%32.67%36.45%70.15%33.70%35.47%70.16%34.68%
Does the Lesson have a “Real World” Context (authenticity)? (More real world, rating of 5, 6, or 7 combined)
1 2 3 4 5 6 7 Real WorldArtificial
Relevance of Content Artificial - Real World
Instructional Style is Changing
21st CTL has changed the way many teachers teach; increasing percentages of teachers identify their style as constructivist.
Instructional Style Didactic - Constructivist
Teacher Survey
Cohort 14 Years
Cohort 14 Years
Cohort 23 Years
Cohort 23 Years
Cohort 32 Years
Cohort 32 Years
Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10
Didactic 29.63% 26.34% 33.15% 26.21% 34.01% 29.68%
Even Balance 47.64% 52.93% 48.53% 51.83% 50.41% 52.39%
Constructivist 22.73% 20.73% 18.32% 21.96% 15.58% 17.92%
Less DidacticMore Even BalanceLess Constructivist
Less DidacticMore Even BalanceLess Constructivist
Less DidacticMore Even BalanceMore Constructivist
Less DidacticMore Even BalanceMore Constructivist
Less DidacticMore Even BalanceMore Constructivist
Less DidacticMore Even BalanceMore Constructivist
How would you describe your teaching style?
Instructional Style Didactic - Constructivist
Observation (Overall)
Class Period Term Mean Difference
First ThirdFall 09 3.951
First ThirdSpring 10 4.365 0.414
Middle ThirdFall 09 4.372
Middle ThirdSpring 10 4.801 0.429
Last ThirdFall 09 4.489
Last ThirdSpring 10 4.854 0.365
1 2 3 4 5 6 7 ConstructivistDidactic
Instructional Style Didactic - Constructivist
0%
25%
50%
75%
100%
Cohort 1 Cohort 2Cohort 3
How Assignments GradedTopics StudiedTopics of Papers or AssignmentsWay Topics StudiedWorking Together or Alone
Who makes decisions about... (Teacher Completely / Teacher Mostly)
Student Survey
Spring 2010
• What instructional strategies are being used?
Instructional Strategies
Instructional StrategiesTeacher Survey
Teaching Strategies (Valuable / Very valuable)
Cohort 1 Cohort 2 Cohort 3
Teacher Led Discussion - High 1 1 1
Problem-based Learning 2 4 3
Authentic Learning 3 2 4
Teacher Lecture 4 3 2
Teacher Led Discussion - Low level 5 6 6
Multi-modal Teaching 6 5 5
Project-based Learning 7 7 7
Collaborative Learning - informal 8 8 8
Peer Teaching 9 9 9
Collaborative Learning - formal 10 10 10
WebQuests 11 11 11
Learning Centers 12 12 12
91-97%
80-90%
33-38%
42-45%
68-78%
Instructional Strategies
Teacher Survey
Teaching Strategies (Valuable / Very valuable)
85%
87%
90%
92%
95%
97%
Cohort 1 Cohort 2 Cohort 3
Teacher Led Discussion - High Level Outcomes Problem Based LearningAuthentic Learning Teacher Lecture
Spring 2010
Use of Strategies
0%
10%
20%
30%
40%
50%
Cohort 1 Cohort 2 Cohort 3 Non-21st CTL
Collaborative Learning Differentiation of LearningInteractive Instruction Independent Study
PATI Survey
(Instructional Phase)
Spring 2010
Rate the Level of Your Current Use of the Strategy (High)
Use of Strategies
0%
15%
30%
45%
60%
Cohort 1 Cohort 2 Cohort 3 Non-21st CTL
Direct Instruction Mediating Student ThinkingInquiry Experiential Learning
Rate the Level of Your Current Use of the Strategy (High)
PATI Survey
(Instructional Phase)
Spring 2010
‣ What has been the impact of 21st CTL / CFF on teaching in your school?
‣ What impact do these changing teaching practices have on student outcomes?
Questions
Jeanne Vilberg, [email protected] Clausen, [email protected]
CFF • 21st CTL Evaluation http://cff.psu.edu/y4report
Thank you.
Image Attribution:
School Bell Towerhttp://www.flickr.com/photos/road_less_trvled/2061666068/by road_less_trvled Taken October 13, 2007 in Zionsville, PAhttp://creativecommons.org/licenses/by-nc/2.0/deed.en
Classroom with Desks against Wallhttp://www.flickr.com/photos/ransford/770039665/by Ben RansfordTaken on June 9, 2007http://creativecommons.org/licenses/by-nc/2.0/deed.en