Upload
ambrose-lawson
View
220
Download
5
Tags:
Embed Size (px)
Citation preview
ASSESSING THE CONTINUED EFFICACY OF
AUNIVERSITY CENTER MODEL OF TEACHER
PREPARATION PROGRAM
Diana LysKristen Cuthrell
Laura Bilbro-Berry
2
PURPOSE OF THE STUDY
Builds upon Strand 2 – Demonstrating Effectiveness and
Relevance by engaging in the process of continuous program improvement.
Prior research which noted the need for more valid and reliable assessments upon which to base program pathway comparisons.
As a new teacher performance assessment – edTPA – is implemented, do candidates in different program pathways continue to have comparable outcomes?
3
ANTECEDENT RESEARCH-UNIVERSITY CENTER MODEL
Boyd, Lankford, Loeb and Wyckoff (2005)– geography of first employment to hometown and/or college
Lorenzo (2005) – co-location models provide level access
Grady (2005) and Vaughan (2006) – barriers to community college transfer
Locklear (2007) and Locklear et al (2009) – university center models as comparable preparation programs
Bill and Melinda Gates Foundation; With Their Whole Lives Ahead of Them (2009) – reasons for leaving college early; lack of persistence
4
ANTECEDENT RESEARCH:TEACHER PERFORMANCE ASSESSMENTS When assessing preservice teachers, it is important to
evaluate their knowledge and skills, student learning, professional dispositions, and reflective practices (Darling-Hammond, 2006).
Performance assessments provide documentation of the teachers performance, note progress toward reaching the program goals, and dissect the program’s strengths and (Darling-Hammond, 2006).
Portfolio assessments are one type of performance-based assessment, used for formative, summative, and predictive assessment (Bannink, 2009).
Portfolio assessment may be beneficial for certain aspects of the teaching certification process, such as documentation of planning and examples of instruction, but may not be valid for the assessment of teacher competencies (Yao, Thomas, Nickens, Downing, Burkett, & Lamson, 2008).
5
TRADITIONAL ELEMENTARY EDUCATION PROGRAM OVERVIEW Includes face to face coursework in pedagogy,
knowledge, and skills . Majority of junior and senior level coursework
includes practica experiences; many supervised by faculty.
Spirally instruction is woven throughout program courses in observational skills, planning for diverse learners, research based instructional strategies, Common Core curriculum, classroom assessment, and differentiation.
Emphasis on classroom management occurs at the senior level.
Candidates participate in a year long internship: Senior 1 (1 day a week) and Senior 2 (5 days a week)
6
UNIVERSITY CENTERPROGRAM OVERVIEW
Five hub site community colleges, each with several spoke site CCs creating regional consortia
Each hub site has an IHE employee that works full-time on the CC campus, recruiting, advising, marketing in the region
Cohort model used; 14 current cohorts exist On-line delivery; same program taught by
same faculty Part-time delivery model requires 3 ½ years
to finish “2” 443 graduates; 77% teaching within N.C.;
95% teaching in rural eastern N.C.
7
IMPLEMENTING EDTPA IN A LARGE ELEMENTARY EDUCATION PROGRAMComparing Program Pathways
8
BRIEFLY, WHAT IS EDTPA? Capstone, summative performance assessment
portfolioLinks theory to practiceIncludes 3-4 tasks requiring candidates to plan,
instruct, and assessCandidates must video record themselves teaching
lessons they plan for a specific group of students. Nationally validated instrument developed at Stanford
University as a measure of teaching proficiency at the pre-candidate level
Results are summative for candidate and formative for programs
Currently over 25 states and 180 teacher preparation programs have adopted or considering adoption of the edTPA
9
HOW IS EDTPA SCORED?
2012 TPA Field Test Handbooks Evaluators rate candidates’ performances on
planning, instruction, assessment, analyzing teaching, and academic language in 12 rubrics.
Each item based on a 5 point scale: 5= stellar candidate4= solve foundation, knowledge & skills3= acceptable levels to begin teaching2= some skills, more practiced needed1= struggle candidate not ready
10
HOW WERE CANDIDATES PREPARED FOR EDTPA?
Revised teacher education curricula was aligned with: Common Core State Standards 21st Century Skills
ISLES modules developed as part of TQP Grant Curriculum Reform ISLES 3 aligns with edTPA Task2
Instructional Coaching Support for Candidates in Partner Districts
11
SAMPLE AND DATA ANALYSIS
12
STUDY SAMPLE AND METHODS Utilized preexisting integrated assessment system
databases on candidate performance, competence and descriptive characteristics.
JMP Pro 9 provided the ability to compare the two sections using matching student ID analysis.
Sample included 132 elementary education degree completers (74 Non-WPE and 58 WPE) Fall 2012 semester .
Dataset included teacher performance data: edTPA assessment scores, test scores, GPA, internship grades, and demographic data.
90 participants were randomly selected for analysis, 45 WPE and 45 WPE candidates.
14
AGE CHARACTERISTICS
15
ETHNICITYCHARACTERISTICS
16
FINAL INTERNSHIP GRADE DIFFERENTIALS
A A- B B+ C0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
WPE NON-WPE
GPA Compari-son
* WPE: 3.75
* Non-WPE: 3.310
18
ACADEMIC CHARACTERISTICS
N % of Total M SD Range Min Max N % ot Total M SD Range Min Max
Test Scores
ACT Math 1 11.38% 19 0 19 19 7 88.62% 21.143 4.880 11 16 27ECU-Math Placement
Math Lab 0 0ECU-Math Placement
Orientation 0 20 100.00% 17.042 3.651 12 12 24Elem Ed Instr Pract App
(5015) 44 54.45% 176.12 8.70 38.33 155.67 194 37 45.55% 175.23 10.36 37.67 154.33 192Fundamental Subj: CK
(5511) 1 100.00% 158 0 158 158 0
SAT Mathematics 9 19.53% 471.11 133.36 360 330 690 34 80.47% 513.82 60.05 220 390 610Sp Ed Core K Mild Mod
GC(5543) 1 100.00% 174 0 174 174 0
Grades and Test Score DifferentialsWPE Non-WPE
19
EDTPA RUBRIC SCORES
Assessed candidate’s performance in each content area by comparison analysis of each rubric score.
Evaluation includes mean, standard deviation, and percent total.
Significant results: Rubric 8- Assessment: Using feedback to
guide further learning (WPE, M=3.60. Non-WPE, M=3.36)
Rubric 12-Academic Language: Developing student’s academic language and literacy (WPE, M=3.56. Non-WPE, M=3.26)
20
WPE V NON-WPE – ALL RUBRICS
1 2 3 4 5 6 7 8 9 10 11 12 AVG3.000
3.100
3.200
3.300
3.400
3.500
3.600
Student Performance Assessment as Determined in the Evaluation of edTPA Rubric Scores
Mean
edTPA Rubric
WPE NON-WPE
22
RUBRIC 12- ACADEMIC LANGUAGE: DEVELOPMENT STUDENTS’ ACADEMIC LANGUAGE – ALL CANDIDATES
WPE (n=45) Non-WPE (n=45)0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
2.2%
8.9%
26.7%
42.2%
66.7%
44.4%
1
2
3
4
5
t-Test Results (95% CI)d-.311df-88.00P-value-.027χ² -5.149
Competence Scale
24
WPE V NON-WPE, WITH “NOT MET” REMOVED
Further analysis shifted from program pathway comparison to proficiency of candidates. To focus on proficiency, candidates who were not proficient on the edTPA were removed from the analysis.
Excluded 3 “not met” candidates to prevent skew data. WPE: N=44Non-WPE: N=43
Significant results: Rubric 8, 10, & 12
25
RUBRIC 8- ASSESSMENT: USING ASSESSMENT TO INFORM INSTRUCTION-PROFICIENT CANDIDATES
Competence Scale
t-Test Results (95% CI)d-.0263df-84.49P-value-.048χ² -2.813
26
RUBRIC 10- ACADEMIC LANGUAGE: UNDERSTANDING STUDENTS’ LANGUAGE DEVELOPMENT AND ASSOCIATED LANGUAGE DEMANDS- PROFICIENT CANDIDATES
t-Test Results (95% CI)d-.225df-76.75P-value-.062χ² -3.718
Competence Scale
27
RUBRIC 12- ACADEMIC LANGUAGE: DEVELOPMENT STUDENTS’ ACADEMIC LANGUAGE-PROFICIENT CANDIDATES
t-Test Results (95% CI)d-.034df-83.58P-value-.0017χ² -5.639
28
IMPLICATIONS FOR OTHER PROGRAMS
29
ISSUES TO CONSIDER WHEN ADDRESSING PROGRAM PATHWAYS
Impact of online deliveryFeasibility of supervised practica?
Level of support from facultyHow do we provide supports to DE
students that are provided in face to face forums?
Candidate developmentAre DE students more effective
independent learners in working through edTPA handbooks than nonDE candidates? If so, what traits could be utilized/taught in face to face instruction?
30
ISSUES TO CONSIDER WHEN UTILIZING EDTPA DATA
Program gatewaysShould we have stronger gateways in Junior
level classes that are aligned with edTPA? Candidates Characteristics
What is occurring in which WPE candidates outperform nonWPE candidates on Academic Language rubrics?
Individual Rubric Scores vs. Total AverageWhat considerations should be made when
using edTPA scores?How will this analysis change with shift to
Operational Handbooks with 15 rubrics?
31
OUR NEXT STEPS
32
WHAT’S NEXT?
1. Future research should continue to study efficacy of the model with larger population samples.
2. Future research should investigate the validity and reliability of our performance measures and assessments.
3. Future research should expand to address other recruitment and retention factors that may influence enrollments.
33
QUESTIONS?
34
CONTACT INFORMATION
Ms. Laura [email protected]
Dr. Diana B. [email protected]
For a copy of this PowerPoint presentation, please email Dr. Diana Lys
Dr. Kristen [email protected]