View
225
Download
2
Category
Preview:
Citation preview
www.FLDOE.org1
Spring MeetingJanuary 19, 2016
Tallahassee, Florida
Florida Association of Professors of Educational Leadership (FAPEL)
Florida Educational Leadership Examination (FELE)
www.FLDOE.org2
Postsecondary Assessment Team for Today
Phil Canto, Bureau ChiefDr. Suleyman Olgar, Scoring and Reporting Supervisor Lauren White, Scoring and Reporting Team Dan Moore, Test Development Team
www.FLDOE.org3
Today’s Postsecondary Assessment Presentation
Present status of current FELE program Discuss status of research for “Next Gen” FELEReview 2015 data (critical implementation year)Review closely the FELE holistic scoring processProvide sample materials and informQ & A
www.FLDOE.org4
Educator Quality‐BHAG
“Every Florida teacher and leader is prepared, developed, supported, and supervised by educators to make teaching better.”
www.FLDOE.org5
• No planned substantive changes to current FELE content, structure, passing scores, or examination fees in 2015 or 2016;
• Ongoing enhancement and expansion of M/C item pools for all three subtests‐We constantly monitor to ensure quality control;
• With recent SBE changes to school grades, there are no impacts to FELE items and prompts.
FELE Program 411
www.FLDOE.org6
• FLDOE is evaluating several options for performance based or enhanced assessment(s)
• May impact FELE and/or other exams in the future (current or new)
• End of 2016 to 2017 before pilot project could begin with statewide inputs
• Normal test development process is at least 2 years• Biggest question is, “What is the current FELE not giving us that a performance assessment might?”
FLDOE Research for Next Generation FELE
www.FLDOE.org7
• January 1, 2014 (New Test)• FELE 3.0 Examination
• Subtest 1: Leadership for Student Learning
• Subtest 2: Organizational Development
• Subtest 3: Systems Leadership
• Effective January 1, 2015 (New Format & Passing Scores)• Subtest 3: Systems Leadership
• Multiple‐Choice Section• Written Performance Section(Examinees must pass both sections independently to pass FELE Subtest 3)
FELE 3.0 Program Changes
www.FLDOE.org8
* Operational pass rates are calculated for the first-time examinees taking FELE subtests from January 2, 2015 to December 31, 2015 with the approved passing scores at the November 2014 SBE meeting which became effective by January 1, 2015.Note: FELE Subtest 3 written performance assessment is holistically scored by two human raters using a scoring rubric of 1–6 range. FELE Subtest 3 written performance assessment score is the sum of the two ratings.
FELE Subtest Recommended Number Correct
Recommended Percentage Correct
Projected Passing Rates
Operational Pass Rates*N %
Subtest 1: Leadership for Student Learning 48/70 69% 58% 1,617 63%
Subtest 2: Organizational Development 48/69 70% 55% 1,568 58%
Subtest 3: System Leadership: Multiple-Choice Section
36/55 66%
34%
1,620 50%
Subtest 3: System Leadership: Written-Performance Section
7/12 59% 1,618 46%
FELE Recommended Passing Scores, Projected Pass Rates vs. 2015 Operational Pass Rates
www.FLDOE.org9
FELE 3.0 First-Time Examinee Pass Rates (2014 vs. 2015)
FELE 3.02014 2015
N Pass Rate Mean Score N Pass Rate Mean Score
FELE Subtest 1: Leadership for Student Learning 1,495 84% 219 1,617 63% 202
FELE Subtest 2: Organizational Development 1,485 90% 225 1,568 58% 200
FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 82% 221 1,620 50% 197
FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 46% 7
FELE Assessment Level 1,475 73% N/A 1,523 22% N/A
www.FLDOE.org10
FELE 3.0 Best Attempt Examinee Pass Rates (2014 vs. 2015)
FELE 3.02014 2015
N Pass Rate Mean Score N Pass Rate Mean Score
FELE Subtest 1: Leadership for Student Learning 1,495 87% 219 1,617 70% 202
FELE Subtest 2: Organizational Development 1,485 93% 226 1,568 69% 200
FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 86% 222 1,620 63% 197
FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 59% 7
FELE Assessment Level 1,475 81% N/A 1,523 42% N/A
www.FLDOE.org11
FELE Performance by Competency (2014 ‐ 2015)
Test Name Competency Name 2014 2015
Florida Educational Leadership Examination (FELE) N % Correct N % Correct
FELE Subtest 1: Leadership for Student Learning
01 (17-18) Facilitate student learning goals
1,495
69.7%
1,572
69.8%
02 (18) Prioritize student learning via leadership 70.6% 74.6%03 (15-17) Develop/implement instructional framework 71.5% 71.6%
04 (17) School environment for student learning 68.4% 70.5%
FELE Subtest 2: Organizational Development
01 (11-12) Recruitment and induction practices
1,485
72.2%
1,522
74.1%
02 (22-23) Faculty/staff development and retention 69.5% 71.6%
03 (16-17) Develop leadership w/in the organization 70.9% 71.9%
04 (17-18) Personal/professional behavior practices 70.3% 71.0%
FELE Subtest 3: System Leadership (Multiple-Choice Section)
01 (13-14) Decision-making processes
1,485
62.7%
1,574
67.2%
02 (14) School operation management practices 69.8% 68.5%
03 (14) Financial and resource management 64.4% 66.9%
04 (13) School legal practices and applications 64.8% 66.2%
FELE Subtest 3: System Leadership (Written Performance Section) 05 (1) Effective communication practices N/A N/A 1,471 62.6%
www.FLDOE.org12
Score Verification• Score verification eligibility:
• A non‐passing score of 190 or higher• Written Performance component
FELENumber of Sessions Number of Items
ReviewedNumber of Status
Changes
2014 2015 2014 2015 2014 2015
Subtest 1: Leadership for Student Learning 11 25 119 197 0 0
Subtest 2: Organizational Development 3 22 59 163 1 0
Subtest 3: Systems LeadershipMultiple-Choice Section 23 23 254 (14) 212 1 1
Subtest 3: Systems LeadershipWritten Performance Section 0 42 0 45 0 0
www.FLDOE.org13
FELE Written Performance Assessment (WPA) & Holistic Scoring
• Interpretation of data and writing ability as it relates to the data analysis and target audience;
• Organization, sentence structure, and mechanics, as well as the accuracy of data interpretation
• An examinee response is judged for its total effect;• Rater makes an overall evaluation taking multiple performance factors into consideration
• Responses are rated independently by two raters and sent through a discrepancy resolution process if rater scores vary by more than 1 point.
www.FLDOE.org14
FELE WPA Responses & Holistic Scoring Process• Scoring Rubric identifies performance characteristics to be judged, includes the criteria for evaluating those features, and describes how performance varies across the scoring scale
• Anchor Set is used to calibrate raters to the rubric• Supplemental Rating Criteria lists the critical elements of data use by score points
• Calibration/Practice Set includes pre‐scored responses and used for rater calibration prior to operational scoring
www.FLDOE.org15
SCORE CRITERIA
6
The writing sample has a clearly established topic that the writer fully develops with specific details and examples, includingaccurate and thorough data interpretation, explanation, and application. The writer clearly describes the purpose of thecommunication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings,all relevant implications, and analysis of trends. Point of view is consistently maintained. Organization is notably logical andcoherent. The writer demonstrates superior facility in the use of proper grammar and syntax.
5
The writing sample has a clearly established topic that is adequately developed and recognizable through specific details and/orexamples, including mostly accurate and thorough data interpretation, explanation, and application with minor errors in details not affecting conclusions. The writer adequately describes the purpose of the communication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings, relevant implications, and analysis of trends. Point of view is mostly maintained. Organization follows a logical and coherent pattern. The writer demonstrates infrequent errors in the use of proper grammar and syntax.
4
The writing sample has an adequately stated topic that is developed with some specific details and/or examples, including somecomponents of data interpretation, explanation, and application. The writer describes the purpose of the communication and reflectsa basic understanding of the target audience with some unexplained terms or references. Plans for change may omit a fewnecessary details or relevant implications in aligning data findings and analysis of trends. Point of view is somewhat maintained. Organization is mostly logical and coherent. The writer demonstrates satisfactory use of proper grammar and syntax.
3
The writing sample states a topic that is developed with generalizations, with some accurate components of datainterpretation, explanation, and application. The writer somewhat describes the purpose of the communication and reflects amarginal understanding of the target audience with several unexplained terms or references. Plans for change omit somenecessary details or relevant implications in aligning data findings and analysis of trends. Explanations of some data elements may be faulty, and placement of data-related details may not be effective. Point of view is ambiguous. Organization is occasionallyillogical or incoherent. The writer demonstrates some errors in the use of proper grammar and syntax that do not detract from theoverall effect.
2
The writing sample presents an incomplete or ambiguous topic, with most components of data interpretation, explanation, andapplication omitted or inaccurate. The writer poorly describes the purpose of the communication and reflects little understanding ofthe target audience with numerous unexplained terms or references. Support is developed with generalizations with littledescription of plans for change or relevant implications in aligning data findings and analysis of trends. Explanations ofdata elements are faulty, and placement of data-related details is ineffective. Point of view is confusing and distracting. Organization is frequently illogical and incoherent. The writer demonstrates serious and frequent errors in proper grammar and syntax.
1
The writing sample has no evident topic. Numerous components of data interpretation, explanation, and application are missing orinaccurate. The writer fails to describe the purpose of the communication and does not address the target audience. Plans forchange and/or relevant implications in aligning data findings and analysis of trends are minimally addressed. Development is inadequate and/or irrelevant. Point of view has not been established. Organization is illogical and/or incoherent. The writerdemonstrates severe and pervasive flaws in the use of proper grammar and syntax.
FELE WPA Rubric & Score Scale
www.FLDOE.org20
Questions
Call850‐245‐0513
Visithttp://www.fldoe.org/accountability/assessments/posts
econdary‐assessment
Recommended