Emporia State University

Emporia State Emporia State UniversityUniversity

Phil BennettPhil [email protected]

620-341-5367620-341-5367

Teacher Work SampleTeacher Work SampleModelModel

The Teachers CollegeThe Teachers College

mailto:[email protected]

Collaboration and UseCollaboration and Use

• Emporia and Olathe mentor teachers

• Emporia and Olathe elementary interns

• Emporia State University faculty

• Renaissance Group partner universities

• Oklahoma teachers and university

professors

• Kansas Performance Assessment to obtain

professional license

Current Status at ESU

• Required of all elementary and secondary

interns/student teachers beginning

Spring 2002

• “Practice” Teacher Work Sample completed

prior to Block 3/Student Teaching

• The Teacher Work Sample is a major part of

the final grade for the course EL/ED 431

General Structure of the TWS

• Contextual Information & Learning Environment Adaptations

• Unit Learning Goals and Objectives

•Instructional Design and Implementation

• Demonstration of Integration Skills

• Analysis of Classroom Learning Environment

• Analysis of Assessment Procedures

• Reflection and Self-Evaluation

Factor 1: Contextual Factor 1: Contextual Information & Learning Information & Learning

EnvironmentEnvironmentEducational Purposes• Increase student’s concept of classroom diversity• Link information about diversity to instructional design

Measurement Purpose• Provide information about student’s awareness of contextual

factors and ultimately data to examine their ability to function as a professional in a diverse setting.

Factor 2: Unit Learning Factor 2: Unit Learning Goals & ObjectivesGoals & Objectives

Educational Purposes• Promote use of more challenging instruction for all PK-12 pupils• Promote use, interpretation and application of local and state standards• Encourage student teachers/interns to avoid “knowledge only” targets unless appropriate

Measurement Purpose• Aid in interpretation of gain scores (i.e., what kind of learning

does the gain score represent; type and concentration)

Factor 3: Instructional Design Factor 3: Instructional Design & Implementation& Implementation

Educational Purposes• Foster use of assessment & context data in planning instruction• Link instructional design to learning objectives• Encourage student teachers/interns to design challenging lessons that: -impact learning for all students

-address different learning styles, -incorporate technology-incorporate a wide range of reading abilities-use learning-centered environments

Measurement Purpose• Examine relationship between gain scores & teaching approaches• Ensure that teachers understand and use a variety of appropriate instructional strategies

Factor 4 Demonstration of Factor 4 Demonstration of Integration SkillsIntegration Skills

Educational Purposes• Demonstrate the ability to integrate across and within content fields• Demonstrate the ability to teach thinking skills

Measurement Purposes• Ensure that teachers can facilitate all students’ abilities to understand relationships between subject areas

Factor 5: Analysis of Factor 5: Analysis of Classroom Learning Classroom Learning

EnvironmentEnvironmentEducational Purposes• Provide opportunity for student teachers/interns to link learning results to classroom efforts• Promote student teacher/interns reflection on the impact the unit had on individual, small group and whole group learning• Provide evidence of an appropriate classroom management plan• Provide evidence of an appropriate motivational skills

Measurement Purpose• Ensure teachers provide a classroom environment supportive of student interaction in learning activities

Factor 6: Analysis of Factor 6: Analysis of Assessment ProceduresAssessment Procedures

Educational Purposes• Promote link between learning objectives and assessments• Encourage the use of different assessment formats• Encourage the use of challenging assessments

Measurement Purposes• Encourage student teachers/interns to avoid using simple knowledge based assessments unless appropriate• Student Teachers/interns can present evidence of calculation of student gain scores

Factor 7: Reflection and Factor 7: Reflection and Self-evaluationSelf-evaluation

Educational Purposes• Promote analysis and synthesis of all activities• Promote professional development• Promote a better understanding of the implications of state assessment and accreditation process on the teacher’s classroom

Measurement Purpose• Estimate the degree to which the unit was successful• Demonstrate that the student teacher/intern can successfully evaluate the effects of his or her choices and actions on student learning

Emporia State University Emporia State University Teacher Work SampleTeacher Work Sample

Survey ofSurvey ofStudent Teachers/InternsStudent Teachers/Interns

Spring of 2002, 2003, & 2005Spring of 2002, 2003, & 2005

Spring 2002 Open-ended Spring 2002 Open-ended responsesresponses

Most important thing I gained doing the Most important thing I gained doing the TWS:TWS: Determining gain scores/student progress Determining gain scores/student progress

(13%)(13%) Being accountable for individual student Being accountable for individual student

learning (11%)learning (11%) Planning/using/pacing a teaching unit (10%)Planning/using/pacing a teaching unit (10%) Being aware of my teaching skills/instruction Being aware of my teaching skills/instruction

planning and how to improve (9%)planning and how to improve (9%) Learning how to present/analyze assessment Learning how to present/analyze assessment

data (9%)data (9%) Improving time management/organization (9%)Improving time management/organization (9%)


The one thing that would improve The one thing that would improve the TWS assignment:the TWS assignment: Decrease length (17%)Decrease length (17%) Make TWS less repetitive (12%)Make TWS less repetitive (12%) Provide examples of good Teacher Work Provide examples of good Teacher Work

Samples (9%)Samples (9%)


Most important thing I gained doing the TWS:Most important thing I gained doing the TWS: Planning and implementing units and lessons (15%)Planning and implementing units and lessons (15%) Reflecting about my experience of teaching (9%)Reflecting about my experience of teaching (9%) Learning to evaluate students’ gain scores (9%)Learning to evaluate students’ gain scores (9%)


The one thing that would improve The one thing that would improve the TWS assignment:the TWS assignment: Having more specific details and better Having more specific details and better

examples (16%)examples (16%) Reduce/shorten the number of Reduce/shorten the number of

requirements/amount of work (12%)requirements/amount of work (12%) More training sessions (12%)More training sessions (12%)

Spring 2005Spring 2005“Agree-disagree” questions“Agree-disagree” questions

The TWS accurately show-cased my The TWS accurately show-cased my knowledge/skills as a teacher.knowledge/skills as a teacher. 27% disagree; 73% agreed27% disagree; 73% agreed

The TWS was a valuable experience The TWS was a valuable experience to my professional training.to my professional training. 21% disagree; 79% agree21% disagree; 79% agree

Crocker Validity Survey Crocker Validity Survey ResultsResults

Response Percentages from TWS Response Percentages from TWS

RatersRaters

Who are TWS raters at ESU?Who are TWS raters at ESU?

Readers/scorers of TWSsReaders/scorers of TWSs Classroom teachers and university Classroom teachers and university

facultyfaculty Trained in TWS purpose and formatTrained in TWS purpose and format Undergo anti-bias training and Undergo anti-bias training and

review before each scoring sessionreview before each scoring session

Crocker validity, Spring 2002Crocker validity, Spring 2002

Does the TWS measure Does the TWS measure Knowledge/Skills necessary for Knowledge/Skills necessary for

beginning teachers?beginning teachers?Not at allNot at all Yes, absolutely Yes, absolutely (1) (2) (1) (2) (3) (3) (4) (4)

00 3.5 32.5 64 3.5 32.5 64

N=28, in %N=28, in %

How important to practice of How important to practice of beginning teachers?beginning teachers?

Unimportant Somewhat Important Unimportant Somewhat Important Critical Critical importantimportant

(1) (2)(1) (2) (3) (3) (4) (4) 00 3.5 46.5 3.5 46.5 50 50

N=28, in %N=28, in %

How often would a beginning How often would a beginning teacher engage in each:teacher engage in each:

In %In % NeverNever AnnuallyAnnually MonthlyMonthly WeeklyWeekly DailDailyy

Context to set Context to set goals and plan goals and plan for instruction/ for instruction/ assessment assessment

00 00 77 53.553.5 3939

Set significant/ Set significant/ challenging/ challenging/ varied/ varied/ appropriate appropriate learning goals learning goals

00 00 77 53.553.5 3939

Use multiple Use multiple assessment assessment modes modes

00 00 77 5050 4343

In %In % NeveNeverr

AnnuallAnnuallyy

MonthlMonthlyy

WeeklWeeklyy

DailDailyy

Design instruction Design instruction for specific for specific learning goalslearning goals

00 00 1111 5050 3939

Use analysis of Use analysis of student learning to student learning to make instruct make instruct decisionsdecisions

00 00 3.53.5 35.535.5 6161

Use assessment Use assessment data to profile data to profile student learn & student learn & community community information information

00 00 3939 5050 1111

Reflect on Reflect on instruction and instruction and student learning student learning

00 00 77 28.528.5 64.564.5

Extent that TWS tasks reflect Extent that TWS tasks reflect following INTASC standards:following INTASC standards:

In %In % Not Not at at allall

ImplicitImplicit-ly-ly

DirecDirect-lyt-ly

Knowledge of subject Knowledge of subject mattermatter

00 1818 8282

Knowledge of human Knowledge of human development and learning development and learning

00 4343 5757

Adapting instruction for Adapting instruction for individual needsindividual needs

00 1414 8686

Multiple instructional Multiple instructional strategies strategies

00 77 9393

Classroom motivation and Classroom motivation and management skills management skills

1414 3232 5353

In %In % Not at Not at allall

ImplicitlImplicitlyy

DirectlDirectlyy

Communication skills Communication skills 77 32.532.5 60.560.5

Instructional planning Instructional planning skillsskills

00 77 9393

Assessment of student Assessment of student learning learning

00 00 100100

Professional commitment Professional commitment and responsibility and responsibility

00 3939 6161

PartnershipsPartnerships 21.521.5 5050 28.528.5

Crocker Validity, Spring Crocker Validity, Spring 20032003

Does the TWS measure Does the TWS measure Knowledge/Skills necessary forKnowledge/Skills necessary for

beginning teachers?beginning teachers?Not at allNot at all Yes, Yes,

absolutelyabsolutely (1) (2)(1) (2) (3)(3) (4)(4)

00 2.8 27.8 66.72.8 27.8 66.7

N=36, in %N=36, in %

How important to practice of How important to practice of beginning teachers?beginning teachers?

Unimportant Somewhat Important Unimportant Somewhat Important Critical Critical importantimportant (1) (2)(1) (2) (3) (3) (4) (4)

00 2.8 52.8 2.8 52.8 41.7 41.7

N=36, in %N=36, in %

How often would a beginning How often would a beginning teacher engage in eachteacher engage in each


Context to set Context to set goals and plan goals and plan for instruction/ for instruction/ assessment assessment

00 00 11.111.1 16.716.7 72.72.22

Set significant/ Set significant/ challenging/ challenging/ varied/ varied/ appropriate appropriate learning goals learning goals

00 2.82.8 27.827.8 1515 66.66.77

Use multiple Use multiple assessment assessment modes modes

00 00 8.38.3 38.938.9 52.52.88


Design Design instruction for instruction for specific learning specific learning goalsgoals

00 00 2.82.8 38.938.9 58.58.33

Use analysis of Use analysis of student learning student learning to make to make instructional instructional decisionsdecisions

00 00 00 27.827.8 72.72.22

Use assessment Use assessment data to profile data to profile student learn & student learn & community info community info

00 00 33.333.3 44.444.4 22.22.22

Reflect on Reflect on instruction and instruction and student learningstudent learning

00 00 5.65.6 13.913.9 80.80.66

Extent that TWS tasks reflect Extent that TWS tasks reflect following INTASC standardsfollowing INTASC standards


ImpliciImplicit-lyt-ly

DirectlDirectlyy

Knowledge of subject matterKnowledge of subject matter 00 19.419.4 80.680.6

Knowledge of human Knowledge of human development and learning development and learning

00 2525 7575

Adapting instruction for Adapting instruction for individual needsindividual needs

00 8.38.3 91.791.7

Multiple instructional Multiple instructional strategiesstrategies

00 8.38.3 91.791.7

Classroom motivation and Classroom motivation and management skills management skills

00 55.655.6 44.444.4


ImplicitlImplicitlyy

DirectlDirectlyy

Communication skills Communication skills 2.82.8 36.136.1 61.161.1

Instructional planning Instructional planning skillsskills

00 2.82.8 97.297.2

Assessment of student Assessment of student learning learning

00 2.82.8 97.297.2

Professional commitment Professional commitment and responsibility and responsibility

00 36.136.1 63.963.9

PartnershipsPartnerships 66 55.655.6 33.333.3

TWS-Specific Survey of Recent TWS-Specific Survey of Recent Graduates after at Least 1 Year Graduates after at Least 1 Year

of Teaching, Summer 2004of Teaching, Summer 2004

Emporia-area new teachers who Emporia-area new teachers who completed a TWS during student completed a TWS during student teachingteaching

Telephone surveyTelephone survey

Guiding question:Guiding question:

What aspects of the Teacher What aspects of the Teacher Work Sample (TWS) do you do Work Sample (TWS) do you do

in your teaching job? in your teaching job?

Do you…Do you…N=10N=10 YeYe

s%s%NNo o %%

Identify classroom context factors Identify classroom context factors such as prior knowledge, adaptations such as prior knowledge, adaptations needed, etc. needed, etc.

100100 00

Plan for and teach according to the Plan for and teach according to the context factors. context factors.

100100 00

Teach different levels of outcomes, Teach different levels of outcomes, such as knowledge, skills, reasoning. such as knowledge, skills, reasoning.

100100 00

Teach to state or local curricular Teach to state or local curricular standardsstandards. .

100100 00

Do you…Do you…

Use pre-post tests Use pre-post tests 6060 4400

Utilize an assessment plan Utilize an assessment plan 100100 00

Set a mastery level for minimum Set a mastery level for minimum acceptable performance level acceptable performance level

9090 1010

Teach in multiple formats or Teach in multiple formats or strategies strategies

100100 00

Use groups Use groups 100100 00

Use technology Use technology 9090 1010

Evaluate individual or subgroup Evaluate individual or subgroup performance. performance.

8080 2020

yes no % %

Do you…Do you…

Calculate gain scores.Calculate gain scores. 1010 9900

Revise instruction based on learning Revise instruction based on learning results.results.

9090 1100

Seek ways to teach better based on Seek ways to teach better based on learning results learning results

101000

00

yes no

% %

Candidate Candidate Performance Data Performance Data

and Researchand Research

Means for Candidate Means for Candidate Scores, Learning Gain, Scores, Learning Gain,

Objective Mastery IndexObjective Mastery IndexSprinSprin

g g 20022002

Fall Fall 20022002

SprinSpring g

20032003

Fall Fall 20032003

Spring Spring 20042004

Fall Fall 20042004

Spring Spring 20052005

CandidatCandidate Mean e Mean ScoresScores

82.282.244

82.382.399

83.8983.89 82.482.466

86.1686.16 89.589.555

89.3289.32

Student Gain Scores are around 65%Student Gain Scores are around 65%

Objective Masters are around 77%Objective Masters are around 77%

Mean percentage scores for Mean percentage scores for additional semestersadditional semesters

Fall 05Fall 05 87.8% Some major changes 87.8% Some major changes were made this semester for which were made this semester for which candidates were not as prepared as candidates were not as prepared as in earlier years. This likely had some in earlier years. This likely had some impact on scores the next two impact on scores the next two semesters as well.semesters as well.

Spring 06Spring 06 89.88%89.88% Fall 06Fall 06 88.94% 88.94% Spring 07Spring 07 91.5%91.5%

Descriptive DataDescriptive Data(overall group)(overall group)

Strengths• Awareness of classroom context• Ability to write outcomes and align instruction• Use of collaborative/multi-learner environments• Use of “active learner/inquiry” models• Employment of multiple learning strategies• Use of formative assessments • Use of assessment throughout instruction• Ability to depict assessment data• Use of technology • Reflection on personal classroom successes & failures• High degree of reported learning impact

Descriptive Data Descriptive Data (overall group)(overall group)

Challenges• Assessment design (planning is all right but needs more “rigor”)• Reflection on professional development plan• Alignment of objectives, instruction, and assessments

Comparison of TWS Scores by Comparison of TWS Scores by Subject Fall 03Subject Fall 03

Subject Mean Score

Mastery Index

Gain Score

Others 95.0 92.7 35.1

Physical Sci 93.0 86.7 76.9

Psych/MR 85.0 90.6 66.3

Phys Ed 82.5 90.7 72.1

ESL/For Lang

82.4 76.3 68.7

Science 81.3 80.0 68.2

Elem/EC 81.0 73.7 73.7

Math 80.2 76.2 77.2

Business 78.0 84.3 73.7

Engl/Journal 71.5 85.3 72.2

Soc Studies 70.8 77.7 65.3

Assessing The Benefits and Assessing The Benefits and Challenges of Completing A Challenges of Completing A

Teacher Work Sample Teacher Work Sample During Student Teaching During Student Teaching

From From The Perspective Of PK-12 The Perspective Of PK-12 Student Teachers/Interns Student Teachers/Interns

and and Cooperating TeachersCooperating Teachers

BackgroundBackground

Dr. Darla Mallein, Emporia State Dr. Darla Mallein, Emporia State Social Science faculty member and Social Science faculty member and methods instructormethods instructor

Spring 2003Spring 2003 Descriptive study utilizing surveys of Descriptive study utilizing surveys of

student teachers/interns and student teachers/interns and cooperating teacherscooperating teachers

Benefits of TWS Benefits of TWS According to Student According to Student

Teachers/Interns Teachers/Interns Raised student teachers/interns’ awareness of Raised student teachers/interns’ awareness of

the impact they have on students’ learning the impact they have on students’ learning (30%)(30%)

Student teachers/interns specifically stated there Student teachers/interns specifically stated there were no benefits of completing the TWS (16%)were no benefits of completing the TWS (16%)

Helped student teachers/interns reflect on Helped student teachers/interns reflect on teaching and learning (14%)teaching and learning (14%)

Prepared student teachers/interns for real-life Prepared student teachers/interns for real-life applications (14%)applications (14%)

Required student teachers/interns to create Required student teachers/interns to create adaptations and activities to meet the needs of adaptations and activities to meet the needs of all their students (12%)all their students (12%)

Benefits of TWS Benefits of TWS According to Cooperating According to Cooperating

TeachersTeachers Prepares student teacher/intern for Quality Prepares student teacher/intern for Quality

Performance Accreditation, NCLB, and other real-Performance Accreditation, NCLB, and other real-life applications (32%)life applications (32%)

Provides student teachers/interns opportunity to Provides student teachers/interns opportunity to plan and teach a complete unit that aligns with plan and teach a complete unit that aligns with standards, objectives, activities, and assessments standards, objectives, activities, and assessments (24%)(24%)

Helps student teachers/interns reflect about Helps student teachers/interns reflect about teaching and learning (16%)teaching and learning (16%)

Gives student teachers/interns the practice in Gives student teachers/interns the practice in administering and analyzing pre- and post-test administering and analyzing pre- and post-test assessments (15%)assessments (15%)

Raises student teachers/interns’ awareness of Raises student teachers/interns’ awareness of their impact on student learning (15%)their impact on student learning (15%)

Problems with TWS Problems with TWS According to Student According to Student

Teachers/InternsTeachers/Interns The directions for completing the TWS were unclear or The directions for completing the TWS were unclear or

confusing (22%)confusing (22%) Student teachers/interns had trouble finding the time to Student teachers/interns had trouble finding the time to

complete the TWS (16%)complete the TWS (16%) Some requirements were difficult to complete (15%)Some requirements were difficult to complete (15%) Student teachers/interns had difficulty completing the Student teachers/interns had difficulty completing the

TWS because of lack of training, lack of good examples, TWS because of lack of training, lack of good examples, and lack of resource people to contact with questions and lack of resource people to contact with questions (14%)(14%)

Completing the TWS was time-consuming (14%)Completing the TWS was time-consuming (14%) The TWS detracted from other classroom duties (12%)The TWS detracted from other classroom duties (12%) The TWS was hard to adapt to grade level, subject area, The TWS was hard to adapt to grade level, subject area,

unit topic, assigned classroom, or curriculum (10%)unit topic, assigned classroom, or curriculum (10%)

Problems with TWS Problems with TWS According to Cooperating According to Cooperating

TeachersTeachers Student teacher/intern had a difficult time dealing Student teacher/intern had a difficult time dealing with time constraints (19%)with time constraints (19%)

Completing the TWS was time-consuming (15%)Completing the TWS was time-consuming (15%) Completing the TWS generated stress, frustration, Completing the TWS generated stress, frustration,

procrastination, and feelings that the project was procrastination, and feelings that the project was not valuable (12%)not valuable (12%)

Directions for the TWS were unclear or confusing Directions for the TWS were unclear or confusing (11%)(11%)

Student teacher/intern had difficulty adapting Student teacher/intern had difficulty adapting TWS to grade level, subject area, assigned TWS to grade level, subject area, assigned students, or district curriculum (11%)students, or district curriculum (11%)

Creating, administering, and analyzing Creating, administering, and analyzing assessments were difficult for student assessments were difficult for student teachers/interns (11%)teachers/interns (11%)

Suggested Changes for TWS Suggested Changes for TWS According to Student According to Student

Teachers/InternsTeachers/Interns Provide better training and resources Provide better training and resources

for student teachers/interns and for student teachers/interns and cooperating teachers (35%)cooperating teachers (35%)

Simplify the TWS by shortening the Simplify the TWS by shortening the requirements and making it more requirements and making it more practical (31%)practical (31%)

Suggested Changes for TWS Suggested Changes for TWS According to Cooperating According to Cooperating

TeachersTeachers Provide better training and resources Provide better training and resources

for student teachers/interns and for student teachers/interns and cooperating teachers (20%)cooperating teachers (20%)

Simplify the TWS by shortening the Simplify the TWS by shortening the requirements and making it more requirements and making it more practical (19%)practical (19%)

Move the TWS requirements to a Move the TWS requirements to a different semester other than the different semester other than the student teaching semester (11%)student teaching semester (11%)

RecommendationsRecommendations1.1. Explain the purpose and benefits of Explain the purpose and benefits of

completing the TWScompleting the TWS2.2. Provide more training sessions and support Provide more training sessions and support

concerning the TWS for student concerning the TWS for student teachers/interns throughout the student teachers/interns throughout the student teaching semesterteaching semester

3.3. Provide more contact people at the Provide more contact people at the university level who can answer questions university level who can answer questions about the TWSabout the TWS

4.4. Make the TWS manual and rubric very clear Make the TWS manual and rubric very clear and understandableand understandable

5.5. Simplify the TWS by shortening the Simplify the TWS by shortening the requirements and taking out the repetitive requirements and taking out the repetitive sections.sections.

RecommendationsRecommendations6.6. Provide good examples of completed Provide good examples of completed

Teacher Work SamplesTeacher Work Samples7.7. Allow flexibility in the format and design of Allow flexibility in the format and design of

the TWS to fit the student teachers/interns the TWS to fit the student teachers/interns assigned grade levels and/or subject areasassigned grade levels and/or subject areas

8.8. Encourage students with split placements to Encourage students with split placements to complete the TWS during the first placementcomplete the TWS during the first placement

9.9. Reinforce the positive aspects of the TWSReinforce the positive aspects of the TWS10.10. Require all cooperating teachers to Require all cooperating teachers to

attend training sessions and/or scoring attend training sessions and/or scoring sessionssessions

Questions andQuestions and

Maybe AnswersMaybe Answers

Documents

Emporia State University