Transcript
Page 1: 21st Century Teaching and Learning Evaluation Project 2011

CFF•21st CTL Evaluation Results

Changes in Teaching Practices

Jeanne Vilberg & Robin ClausenPenn State University

Page 2: 21st Century Teaching and Learning Evaluation Project 2011

What is CFF•21st CTL?

Page 3: 21st Century Teaching and Learning Evaluation Project 2011

21st CTL EvaluationSince March 2007, the 21st CTL Evaluation Project has collected:

• 5,500 21st CTL Classroom Observations

• 55,000 Teacher Surveys

• 650,000 Student Surveys

• 18,000 PATI Surveys (Instructional Phase)

Page 4: 21st Century Teaching and Learning Evaluation Project 2011

Analysis

Start  of  Program    (Fall  2007/2008)

Spring  2010

• The same 21st CTL teachers compared at different points in time.

• Data collection periods (fall/spring 2007 - 2010)

• Cohorts (teachers with different number of years in program)

Page 5: 21st Century Teaching and Learning Evaluation Project 2011

InstrumentsObservations

Teacher/Student SurveysPATI (Instructional Phase)

http://cff.psu.edu

Page 6: 21st Century Teaching and Learning Evaluation Project 2011

Findings

Page 7: 21st Century Teaching and Learning Evaluation Project 2011

Has 21st CTL Changed...

• complexity of class content

• relevance of class content

• instructional style

Page 8: 21st Century Teaching and Learning Evaluation Project 2011

Overall

Complexity of Content

Basic Skills Higher Order

Relevance of Content Artificial Real World

Instructional Style Didactic Constructivist

Page 9: 21st Century Teaching and Learning Evaluation Project 2011

Change in the Complexity of Content

21st CTL teachers now focus more on higher order topics than they did at the beginning of the program.

Page 10: 21st Century Teaching and Learning Evaluation Project 2011

Complexity of Content Describe the content your class is designed to convey.

(Almost All Higher Order Skills / More Higher Order Skills than Content Knowledge Combined)

10%

13%

16%

19%

22%

Start of Program Spring 2010

Cohort 1Cohort 2Cohort 3

Differences5.89%6.85%4.65%

Teacher Survey

Page 11: 21st Century Teaching and Learning Evaluation Project 2011

Complexity of Content Percent of Time Spent in Really Complex Thinking / Problem Solving

(Quite a Lot / Almost All the Time)

35%

37%

39%

41%

43%

Start of Program Spring 2010

Cohort 1Cohort 2Cohort 3

Student Survey

Differences2.52%2.00%1.74%

Page 12: 21st Century Teaching and Learning Evaluation Project 2011

Observations

Complexity of Content

50%

54%

58%

62%

66%

70%

Start of Program Spring 2010

First ThirdMiddle ThirdLast Third

Overall Comparison by Class Period - Basic - Higher Order (Ratings of 5, 6, and 7 Combined)

Differences9.12%8.35%4.06%

Page 13: 21st Century Teaching and Learning Evaluation Project 2011

Classroom Content is More Authentic

21st CTL teachers report students use technology to solve real world problems more often than non-21st CTL teachers.

Page 14: 21st Century Teaching and Learning Evaluation Project 2011

Teacher Survey

Cohort 14 Years

Cohort 14 Years

Cohort 23 Years

Cohort 23 Years

Cohort 32 Years

Cohort 32 Years

Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10

Exercises & Assignments

17.48% 12.45% 19.71% 12.36% 16.67% 12.04%

Blend First / Last 42.48% 35.65% 45.98% 39.61% 47.97% 42.95%

Even Balance 30.22% 35.29% 25.59% 35.76% 28.73% 34.61%

Blend First / Last 7.40% 13.31% 7.40% 10.04% 6.10% 8.21%

Projects and Products

2.43% 3.30% 1.32% 2.23% 0.54% 1.65%

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Fewer ExercisesMore Even Balance

More Projects

Relevance of Content Artificial - Real World

How would you describe the work that students do?

Page 15: 21st Century Teaching and Learning Evaluation Project 2011

Daily

Weekly

Monthly

0% 5% 10% 15% 20% 25% 30%

Cohort 1 - 4 Years in 21st CTLCohort 2 - 3 Years in 21st CTLCohort 3 - 2 Years in 21st CTLNon-21st CTL

How often do students use technology to solve real-world problems?

Relevance of Content Artificial - Real World

PATI Survey

(Instructional Phase)

Page 16: 21st Century Teaching and Learning Evaluation Project 2011

Observation

Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)Cohort 3 (2 Years in 21st CTL)

First Third of the Class

Start of ProgramFirst Third of the

ClassSpring 2010

First Third of the Class

Difference

Middle Third of the Class

Start of ProgramMiddle Third of the

ClassSpring 2010

Middle Third of the Class

Difference

Final Third of the Class

Start of ProgramFinal Third of the

ClassSpring 2010

Final Third of the Class

Difference

32.52%65.18%32.67%36.45%70.15%33.70%35.47%70.16%34.68%

Does the Lesson have a “Real World” Context (authenticity)? (More real world, rating of 5, 6, or 7 combined)

1 2 3 4 5 6 7 Real WorldArtificial

Relevance of Content Artificial - Real World

Page 17: 21st Century Teaching and Learning Evaluation Project 2011

Instructional Style is Changing

21st CTL has changed the way many teachers teach; increasing percentages of teachers identify their style as constructivist.

Page 18: 21st Century Teaching and Learning Evaluation Project 2011

Instructional Style Didactic - Constructivist

Teacher Survey

Cohort 14 Years

Cohort 14 Years

Cohort 23 Years

Cohort 23 Years

Cohort 32 Years

Cohort 32 Years

Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10

Didactic 29.63% 26.34% 33.15% 26.21% 34.01% 29.68%

Even Balance 47.64% 52.93% 48.53% 51.83% 50.41% 52.39%

Constructivist 22.73% 20.73% 18.32% 21.96% 15.58% 17.92%

Less DidacticMore Even BalanceLess Constructivist

Less DidacticMore Even BalanceLess Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

Less DidacticMore Even BalanceMore Constructivist

How would you describe your teaching style?

Page 19: 21st Century Teaching and Learning Evaluation Project 2011

Instructional Style Didactic - Constructivist

Observation (Overall)

Class Period Term Mean Difference

First ThirdFall 09 3.951

First ThirdSpring 10 4.365 0.414

Middle ThirdFall 09 4.372

Middle ThirdSpring 10 4.801 0.429

Last ThirdFall 09 4.489

Last ThirdSpring 10 4.854 0.365

1 2 3 4 5 6 7 ConstructivistDidactic

Page 20: 21st Century Teaching and Learning Evaluation Project 2011

Instructional Style Didactic - Constructivist

0%

25%

50%

75%

100%

Cohort 1 Cohort 2Cohort 3

How Assignments GradedTopics StudiedTopics of Papers or AssignmentsWay Topics StudiedWorking Together or Alone

Who makes decisions about... (Teacher Completely / Teacher Mostly)

Student Survey

Spring 2010

Page 21: 21st Century Teaching and Learning Evaluation Project 2011

• What instructional strategies are being used?

Instructional Strategies

Page 22: 21st Century Teaching and Learning Evaluation Project 2011

Instructional StrategiesTeacher Survey

Teaching Strategies (Valuable / Very valuable)

Cohort 1 Cohort 2 Cohort 3

Teacher Led Discussion - High 1 1 1

Problem-based Learning 2 4 3

Authentic Learning 3 2 4

Teacher Lecture 4 3 2

Teacher Led Discussion - Low level 5 6 6

Multi-modal Teaching 6 5 5

Project-based Learning 7 7 7

Collaborative Learning - informal 8 8 8

Peer Teaching 9 9 9

Collaborative Learning - formal 10 10 10

WebQuests 11 11 11

Learning Centers 12 12 12

91-97%

80-90%

33-38%

42-45%

68-78%

Page 23: 21st Century Teaching and Learning Evaluation Project 2011

Instructional Strategies

Teacher Survey

Teaching Strategies (Valuable / Very valuable)

85%

87%

90%

92%

95%

97%

Cohort 1 Cohort 2 Cohort 3

Teacher Led Discussion - High Level Outcomes Problem Based LearningAuthentic Learning Teacher Lecture

Spring 2010

Page 24: 21st Century Teaching and Learning Evaluation Project 2011

Use of Strategies

0%

10%

20%

30%

40%

50%

Cohort 1 Cohort 2 Cohort 3 Non-21st CTL

Collaborative Learning Differentiation of LearningInteractive Instruction Independent Study

PATI Survey

(Instructional Phase)

Spring 2010

Rate the Level of Your Current Use of the Strategy (High)

Page 25: 21st Century Teaching and Learning Evaluation Project 2011

Use of Strategies

0%

15%

30%

45%

60%

Cohort 1 Cohort 2 Cohort 3 Non-21st CTL

Direct Instruction Mediating Student ThinkingInquiry Experiential Learning

Rate the Level of Your Current Use of the Strategy (High)

PATI Survey

(Instructional Phase)

Spring 2010

Page 26: 21st Century Teaching and Learning Evaluation Project 2011

‣ What has been the impact of 21st CTL / CFF on teaching in your school?

Page 27: 21st Century Teaching and Learning Evaluation Project 2011

‣ What impact do these changing teaching practices have on student outcomes?

Page 28: 21st Century Teaching and Learning Evaluation Project 2011

Questions

Jeanne Vilberg, [email protected] Clausen, [email protected]

CFF • 21st CTL Evaluation http://cff.psu.edu/y4report

Page 29: 21st Century Teaching and Learning Evaluation Project 2011

Thank you.

Image Attribution:

School Bell Towerhttp://www.flickr.com/photos/road_less_trvled/2061666068/by road_less_trvled Taken October 13, 2007 in Zionsville, PAhttp://creativecommons.org/licenses/by-nc/2.0/deed.en

Classroom with Desks against Wallhttp://www.flickr.com/photos/ransford/770039665/by Ben RansfordTaken on June 9, 2007http://creativecommons.org/licenses/by-nc/2.0/deed.en