2014 Fall Assessment Conference Assessment Program Update

Melissa Fincher, Ph.D. Deputy Superintendent, Assessment & Accountability [email protected] Fall Assessment Conference Assessment Program Update 2014-2015 August 25, 2014 (Part 1), 1:00-3:00 PM September 9, 2014 (Part 2), 1:00-3:00 PM
2014 Fall Assessment Conference Sessions
2
August 5, 2014 Test Administration/General Session (Live repeat of Aug. 1) Recording: https://sas.elluminate.com/mr.jnlp?suid=M.007B8E44C6500D7C4FB0A124F3F415&sid=2 012003
August 25, 2014 Assessment Program Updates 2014-2015 – Georgia Milestones (Pt. 1) Live Session: https://sas.elluminate.com/m.jnlp?sid=2012003&password=M.3ADE93EA6A4F1 C2C725DE7FFAF7A17 Recording: https://sas.elluminate.com/mr.jnlp?suid=M.0022562966FC107A2B357D4658 DE06&sid=2012003
1:00 pm - 3:00 pm
1:00 pm - 3:00 pm
9:00 am - 12:00 pm
Coherent – consistent expectations and sufficient challenge to position Georgia
students to compete with peers nationally and internationally
– consistent signal about student preparedness for the next level, be it the next grade, course, or college/career
– consistent signal about student achievement both within system (across grades and courses) and with external measures (NAEP; PSAT; SAT; ACT)
Consolidated – combine reading, language arts, and writing into a single measure to
align to the standards
Georgia Milestones Guiding principles stipulate that Georgia Milestones: be sufficiently challenging to ensure Georgia students are well
positioned to compete with other students across the United States and internationally;
be intentionally designed across grade levels to send a clear signal of student progress/growth and preparedness for the next level, be it the next grade level, course, or college or career;
be accessible to all students, including those with disabilities or limited English proficiency, at all achievement levels;
support and inform educator effectiveness initiatives, ensuring items and forms are appropriately sensitive to quality instructional practices; and
accelerate the transition to online administration, allowing – over time – for the inclusion of innovative technology-enhanced items.
Georgia Milestones
• Grades 3 – 8
– End of Grade (EOG) in language arts, mathematics, science, social studies
• High School
– End of Course (EOC) in 9th Grade Literature & Composition, American Literature & Composition, Coordinate Algebra, Analytic Geometry, Physical Science, Biology, US History, and Economics
State Board Meeting Follow-Up The State Board of Education took action on two items at its
August meeting that relate to Georgia’s Student Assessment Program.
Rule Amendment: 160-3-1-.07 Testing Programs – Student Assessment – This rule was amended in the following manner:
• general clean-up and streamlining of definitions;
• removal of references to the former testing programs such as the Criterion- Referenced Competency Tests (CRCT), the Criterion-Referenced Competency Tests Modified (CRCT-M), the End of Course Tests (EOCT), and the Writing Assessments in grades 3, 5, and 8;
• inclusion of information about the Georgia Milestones Assessment System;
• clarifications about the assessment of students with disabilities and English learners; and
• inclusion of language to phase-out the requirement for the Georgia High School Writing Test (GHSWT) beginning in 2015-2016.
6
The GHSWT must be administered as scheduled during the 2014-2015 SY. Students must earn a passing score to earn their diploma.
State Board Meeting Follow-Up Rule Waiver: 160-4-2-.11 Promotion, Placement, and Retention sections (3)(a), (3)(b), (3)(c) and
160-4-2-.13 Statewide Passing Score sections (2)(d), (2)(e), (2)(f)
Based upon the delay of scores in this first year of Georgia Milestones due to the necessary and required technical work involved, including setting achievement expectations (i.e., cut scores), specific provisions of these two Board Rules were waived for the 2014-2015 school year – through July 31, 2015.
– Promotion in grades 3, 5, and 8 may occur based upon local discretion/policy.
– Local systems should develop policies related to the calculation of final course grades for courses requiring a Georgia Milestones end of course assessment during 2014-2015.
– Additional guidance regarding other purposes and uses of the end of course assessments, such as “test-outs”, retests, etc., will be forthcoming for the 2014-2015 school year.
7
Scores will be issues in Fall 2015, after achievement expectations have been set.
High School Course Enrollments/Completions in Fall 2014
• Students who enroll in a course associated with an EOC during Fall 2014 must participate in the Winter 2014 Georgia Milestones EOC Main Administration.
• This includes those students enrolled in a 9-week/short- duration course in Fall 2014 (for example, a course ending in October 2014). They, too, must participate in the associated Georgia Milestones EOC in Winter 2014. – Given the waiver of State Board Rule 106-4-2-.13 (Statewide Passing
Score), the following conditions apply.
1. These students should NOT participate in the Fall 2014 “Legacy” End of Course Tests (EOCT) Mid-Month Administration upon course completion.
2. Local systems may determine their policy to award final course grades.
8
Features include:
– inclusion of constructed-response items in ELA and mathematics, in addition to selected-response items
– inclusion of a writing component (in response to text) at every grade level and course within the ELA assessment;
– inclusion of norm-referenced items in every grade and content area to complement the criterion-referenced information and to provide a national comparison; and
– transition to online administration over time, with online administration considered the primary mode of administration and paper-pencil back-up until transition is completed.
Addition of technology-enhanced items beginning in 2016-2017.
Georgia Milestones: Unique Features
Blended: Criterion-Referenced and Norm-Referenced
Georgia Milestones will provide:
– criterion-referenced performance information in the form of four performance levels, depicting students’ mastery of state standards
– norm-referenced performance information in the form of national percentiles, depicting how students’ achievement compares to peers nationally
Note: To provide norm-referenced information, some norm-referenced items may not align to Georgia’s content standards. Only aligned NRT items will contribute to proficiency designations.
Georgia Milestones: Embedded NRT
• Each content area/course test will contain 20 norm- referenced items.
• The 20 NRT items will provide a national percentile score to provide a barometer of national comparison.
• Approximately 10 of these items have been reviewed by Georgia educators for alignment to the grade level/course content standards. – Only those NRT items judged to be aligned by Georgia educators will
contribute to the criterion-referenced proficiency designations of students.
• The remaining 10 or so items, while not necessarily aligned to the grade level/course content standards, will not contribute to the proficiency designation.
12
The NRT items were selected to reflect the full TerraNova subtest for each content area.
Supported uses of NRT Scores
• Supported uses:
– General comparison of Georgia to the TerraNova 2011 nationally representative sample
– Should only be used for lower stakes information
• For example, – to garner information about how Georgia students are
achieving relative to their peers nationally;
– to evaluate general trends over years at an aggregate level (i.e., school, district, state)
13
Non-supported uses of NRT Scores
• Given the NRT scores serve to provide barometer of national comparison only, they should not be used to:
– count towards course credits;
– rank students, teachers, schools, or districts; or
– to evaluate educators.
Georgia Milestones
• It is important to remember that Georgia Milestones is primarily a criterion-referenced test, reflecting the content standards for each grade and course
– teachers should teach the Georgia state-adopted content standards and not to the NRT standards
15
Remember: All important uses of the test results – for both students and educators – will be based on the criterion-referenced scores and proficiency determinations.
Georgia Milestones: Unique Features
– all content areas
• Constructed-Response
– to begin in 2016-2017
Constructed response is a general term for assessment items that require the student to generate a response as opposed to selecting a response. Extended-response items require more elaborate answers and explanations of reasoning. They allow for multiple correct answers and/or varying methods of arriving at the correct answer. Writing prompts and performance tasks are examples of extended-response items.
Examining Georgia Milestones
Multiple Choice
A
B
C
D
The content and presentation of these items are for illustrative purposes only.
Constructed Response
George and Ana each had a 12-inch pizza. Both pizzas were split into 8 equal pieces. The shaded pieces are the portion of their pizzas that George and Ana ate.
>
George Ana
George Ana
Constructed Response
12 inches
George Ana
9 inches
12 inches
>
Georgia Milestones:
Georgia Milestones General Test Parameters
• ELA will consists of 3 sections, 1 of which will focus mainly on writing
• Mathematics will consist of 2 sections
• Science will consist of 2 sections
• Social Studies will consist of 2 sections
Each section will be approximately 70 minutes.
Georgia Milestones General Test Parameters: ELA
Criterion-Referenced Total Number of Items: 44 / Total Number of Points: 55
Breakdown by Item Type: – 40 Selected Response (worth 1 point each; 10 of which are aligned NRT)
– 2 Constructed Response (2 points each)
– 1 Constructed Response (worth 4 points)
– 1 Extended Response (worth 7 points)
Norm-Referenced – Total Number of Items: 20 (10 of which contribute to CR score)
Embedded Field Test – Total field test items: 6
Total number of items taken by each student: 60
Georgia Milestones Writing at Every Grade
– All students will encounter a constructed-response item allowing for narrative prose, in response to text, within first or second section of the test.
– Within the writing section of the test, students will read a pair of passages and complete a series of “warm-up” items: o 3 selected-response items asking about the salient features of each
passage and comparing/contrasting between the two passages
o 1 constructed-response item requiring linking the two passages
o 1 writing prompt in which students must cite evidence to support their conclusions, claims, etc.
Genres Writing prompts will be informative/explanatory or opinion/argumentative depending on the grade level. Students could encounter either genre. Warning: Students who simply rewrite excerpts
from the passage(s) to illustrate their point(s) will not receive favorable scores.
Georgia Milestones
Total Number of Items: 53 / Total Number of Points: 58
Breakdown by Item Type: – 50 Selected Response (worth 1 point each; 10 of which are aligned NRT)
– 2 Constructed Response (worth 2 points each)
– 1 Constructed Response (worth 4 points)
Norm-Referenced – Total Number of Items: 20 (10 of which contribute to CR score)
Georgia Milestones
Breakdown by Item Type: – 55 Selected Response (worth 1 point each; approximately 10 of which are aligned NRT)
Norm-Referenced – Total Number of Items: 20 (approximately 10 of which contribute to CR score)
Georgia Milestones
Breakdown by Item Type: – 55 Selected Response (worth 1 point each; approximately 10 of which are aligned NRT)
Norm-Referenced – Total Number of Items: 20 (approximately 10 of which contribute to CR score)
Georgia Milestones: Rubrics
• Rubrics are item specific and therefore cannot be released. – Generally speaking, rubrics outline the expectations for the answer(s)
along with sufficient justification/explanation • Student cites evidence from the text to support answer in ELA
• Student explains reasoning or approach to problem solving in mathematics
• Student answers all portions of the item
– Remember the OAS includes rubrics and student exemplars for all formative open-ended items.
• The extended-response analytic writing rubric will be released. Students will be scored on two features: – Idea Development, Organization, & Coherence
– Language Usage & Conventions
• Content standards – frameworks, formative lessons, PARCC evidence statements
• Sample items – formative items/benchmarks via Georgia OAS→GOFAR;
– released items via PARCC, SBAC, other states (KY, NY), NAEP
– parent’s guide to Georgia’s new assessment developed by the National PTA [http://www.pta.org/advocacy/content.cfm?ItemNumber=3816 ]
• CRCT Readiness Indicators
• Technology Specifications
• Eliciting Evidence of Student Learning Modules
Focus on teaching and learning – eliciting evidence of student learning during instruction and adjusting as needed.
Lexiles Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Common Core Stretch Text – Lower Limit 520 740 830 925 970 1010
Common Core Stretch Text – Upper Limit 820 940 1010 1070 1120 1185
On Track 625 775 845 930 970 1070
Commendable 890 990 1085 1155 1210 1265
Meets 410 570 650 685 800 805
Exceeds 790 915 1040 1120 1210 1265
US Typical Reader – Lower Limit 330 445 565 665 735 805
US Typical Reader – Upper Limit 700 810 910 1000 1065 1100
Georgia 2013 Median 790 860 940 1070 1095 1210
Lexiles with CRCT Readiness Indicators
Formative Assessment Initiatives Bringing a Balanced Assessment Focus to the Classroom
Assessment Literacy
Professional Learning
Benchmark Assessments
1600 new items loaded
1140 science and social studies items will be loaded in fall 2014
Formative Instructional Practices (FIP) [ 7 foundational modules launched
in summer 2013]
24 Benchmarks developed
Additional 12 modules that expand upon the foundation
Coming soon: items for grades 1 & 2 and additional writing prompts
Overall ELA Pilot Summary Data
Incorrect or
1208 2713 1471 374 71 5837
20.7% 46.5% 25.2% 6.4% 1.2% 100.0%
1223 2593 1575 367 146 5904
20.7% 43.9% 26.7% 6.2% 2.5% 100.0%
1148 2038 2192 1054 308 6740
17.0% 30.2% 32.5% 15.6% 4.6% 100.0%
781 2427 1839 826 197 6070
12.9% 40.0% 30.3% 13.6% 3.2% 100.0%
913 2389 2310 1132 286 7030
13.0% 34.0% 32.9% 16.1% 4.1% 100.0%
1322 2673 1856 729 169 6749
19.6% 39.6% 27.5% 10.8% 2.5% 100.0%
1362 2732 1332 463 90 5979
22.8% 45.7% 22.3% 7.7% 1.5% 100.0%
1115 2407 1938 584 177 6221
17.9% 38.7% 31.2% 9.4% 2.8% 100.0%
4
5
Grade
Number of students and percent falling into each score point
Total
Incorrect or
2085 1756 894 199 83 5017
41.6% 35.0% 17.8% 4.0% 1.7% 100.0%
2118 2064 685 170 83 5120
41.4% 40.3% 13.4% 3.3% 1.6% 100.0%
1880 1548 642 188 59 4317
43.5% 35.9% 14.9% 4.4% 1.4% 100.0%
2506 1938 639 200 67 5350
46.8% 36.2% 11.9% 3.7% 1.3% 100.0%
2454 1473 453 132 46 4558
53.8% 32.3% 9.9% 2.9% 1.0% 100.0%
2513 1840 933 317 139 5742
43.8% 32.0% 16.2% 5.5% 2.4% 100.0%
3977 2696 656 165 58 7552
52.7% 35.7% 8.7% 2.2% 0.8% 100.0%
Grade
Number of students and percent falling into each score point
Total
• Overall performance shortfalls
• Many responded ‘dnk’ – as in ‘do not know’
– Students did not show their work, detail their thoughts, rationales, cite evidence to support their answer or claim
• Tendency was to cite answer only – as if a multiple-choice item
– Students did not read carefully and answer all parts of the question/item
Transition to Georgia Milestones: Resources Available Soon
• Sample items specific to Georgia Milestones
• Ancillary support resources, such as
– Assessment Guides
– Informational videos [parents & public /educators]
– Online Practice Center (to aid students in gaining familiarity for CTB’s
online test administration system)
38
Transition to Online
– Online administration will be the primary mode of administration for Georgia Milestones, with paper/pencil serving as back-up
• transition will occur over time
– Administration procedures will change
– Online practice center will be available for students
A demo of CTB’s online platform can be accessed at this link – http://learnoas.ctb.com/GA/ • Click on any one of the tests to open the Sample Test Page • Click on “Start the test” at the top of the web page • Click on “Login”, no credentials are required Note – this demo was designed for the CRCT Retest in mind so the tests that you will see are for Grades 3, 5 & 8, Reading & Math.
• Tablets supported beginning with Spring 2015 main administrations. – iPads supported beginning Winter 2014 EOC
• Transition to Online – Year 1: minimum of 30% online across the district, grades 3-12
– Year 3: minimum of 80% online across the district, grades 3-12
– Year 5: minimum of 100% online across the district, grades 3-12
All schools in all district are expected to conduct some degree of testing online in 2014-2015 . . . and of course, subsequent years.
40
Paper/pencil versions will be available through all years of the transition for the small number of students who cannot interact with computer due to their disability. Braille forms will be available as well.
Online Testing Considerations Online testing requires different thinking relative to logistics . . .
– For many systems (if not all), many lessons have been learned over time through use of the EOCT online in Main, Mid-Month, and Retest administrations and through the CRCT Retest online.
– Technology hardware, peripherals, requirements, support needs, capacity
– More than one session per day (AM and PM for instance) and testing on Mondays and Fridays (which are days of the week many systems have tried to avoid in the past)
– “Cycling” students through test settings where technology is housed
– The number of students a school can test online given the bullets noted above
– Seating/space considerations
– Contingency planning for both expected and unexpected events such as power outages (incl. weather related), Internet service interruptions, construction in or near a school, local downtime/upgrades to technology, etc.
41
Specified high school courses
Grades 3-8
Spring 2015: 04/27 - 06/05
Summer 2015: 06/15 - 07/17
Local
Testing
Window
Can span the entire EOC State Testing Window Nine day window within the EOG State Testing
Window
Structure
Paper/Pencil and online administration modes available for all content
For paper/pencil administrations an individual test booklet will be provided for each of the 8 EOCs
*Section 3 will assess writing through an extended constructed- response
Paper/Pencil and online administration modes available for all
content
For paper/pencil administration there will be one test booklet by
grade containing all 4 EOG content areas
*Section 3 will assess writing through an extended constructed-
response
Order Determined by School District English Language Arts, Mathematics, Science, Social Studies
Content Sections Minutes Per Section
Ninth and American Literature and Composition
3* 70
2 70
2 70
Mathematics 2 70
Science 2 70
General Scheduling and Administration Guidelines
• Target for online testing in Year 1 of Georgia Milestones is a minimum of 30% of students district-wide across Grades 3 – 12.
• Each school within a district must conduct some degree of online testing.
• For the EOG, students who are tested online must engage in online testing in all content areas.
• In other words, a student who takes the Grade 5 ELA assessment online must also test online in Math, Science, and Social Studies.
• The scheduled start date for an EOG grade-level content area or EOC content area must be consistent district-wide regardless of administration mode.
• Each grade-level content area (EOG) or course area (EOC) should be scheduled for completion during the same week as its start date. The exception is ELA Section 3 (writing).
44
General Scheduling and Administration Guidelines - continued
• Breaks provided to students during a test administration must conform to directions in the Examiners Manual. – For instance, a lunch break of 25 minutes does not conform to this requirement and
presents potential test security concerns.
– Note: This does not apply to EOC two-day administrations given the nature of that model of scheduling.
Makeup Days/Sessions
• Make-up days/sessions must be designated within the local testing window.
• The last day of the local testing window should be scheduled as a makeup day to capture any remaining students who need to complete testing.
• In addition to designated makeup days; makeup sessions can be scheduled, morning or afternoon, as time permits on other days during the local window.
45
Differences in Scheduling Guidelines to Consider When Scheduling the EOC and EOG
EOC EOG
Administration mode for a student can vary from one content area to another
Yes No
Schedule for individual content area administrations can span consecutive school days
Yes (for paper and/or
Online Only
AM and PM sessions of the same content area are permitted Yes
Online Only (Paper PM make-ups
allowed)
Sections 1 and 2 of a content area can be scheduled on separate consecutive days
Yes (2-day
administration model)
No
46
Scheduling for EOG Online • In order to maximize online testing capacity in grades 3-8, districts
may stagger the scheduled start date for grade-level content areas provided the scheduled start date for each grade level content area is consistent district-wide. • For instance, Grade 3 ELA can be scheduled to start on a Monday and complete with
Social Studies on Friday of the same week; Grade 4 ELA can then be scheduled to start on a later day that week.
• Because the scheduled start date for EOG grade-level content must be consistent system-wide, considerations must be given to the various levels of student enrollment (and online testing capacity) across schools within a system. Larger schools, dependent upon their online capacity, would likely take longer to complete a grade- level content area than a smaller school. • The 9-day window for grades 3-8 remains in place. However, GaDOE will be available to
talk with districts in detail regarding schedules and attempts to maximize the implementation of online testing.
47
• EOG Paper/pencil administrations must be scheduled, and administered, on a calendar that is consistent district-wide.
• The scheduled start date, by grade level content area, must be the same for both paper/pencil and online administration modes. – For instance, all Grade 3 Math students (both paper and online) start testing in
mathematics on the same day. Should there be a need for online test administrations to extend beyond one day in Grade 3 Math, paper/pencil administrations cannot begin for the next content (Science) until online test- takers are complete with Math.
– Again, the 9-day window remains in place. However, GaDOE will be available to talk with districts in detail regarding schedules and attempts to maximize the implementation of online testing.
– The collection of enrollment counts (including paper/pencil and online numbers) will not be required until January 2015 . . . So there is time to converse and plan for your Spring 2015 EOG window.
48
Considerations for Administering English Language Arts (ELA)
• The ELA assessment for the End of Course (EOC) and the End of Grade (EOG) consists of 3 sections.
• Section 3 (Writing) is comprised of an extended constructed-response.
• Because of the unique characteristics of the design of the ELA content area, special considerations MUST be considered when scheduling for administration (including make-ups).
49
Administering English Language Arts - continued
• ELA Section 3 (writing) should be the only content on which a student is assessed on the day it is scheduled, either as a main or makeup administration
– An exception to this would occur should a student need to make up multiple content areas and there are not enough days left in the local testing window.
• The administration of ELA Section 3 (writing) must be scheduled:
1) on a school day that immediately follows the scheduled completion of ELA Section 2 and;
2) before the administration of a subsequent content area (make- ups may be an exception to this).
50
Administering English Language Arts - continued
• Under no circumstances (including for make-up purposes) should a student take ELA Section 3 (writing) prior to the completion of ELA Sections 1 and 2.
• For EOG, the scheduled main administration start date for ELA Section 3 (writing) must be consistent district-wide by grade (3-8).
• For EOC, the scheduled main administration start date for ELA Section 3 (writing) must be consistent district-wide by course (9th Grade Literature, American Literature).
51
Final Points of Emphasis
• For EOG and EOC, ELA Section 3 (writing) should be the only content area a student should be scheduled to take on a single day. (Note some possible exceptions as discussed in previous slides . . . Make-ups)
• For EOG, online only, grade-level content test sessions can be scheduled for AM and PM and over consecutive school days (if necessary).
52
Final Points of Emphasis
• For EOC, Sections 1 and 2 can be scheduled over 2 consecutive days. (As has been the case in the past for the
EOCT through the 2-day administration model)
• For EOG, Sections 1 and 2 must be scheduled on the same day.
53
Administration)
– August 14 – 26, 2014
• Spring 2015 EOC and EOG Administrations – January 15 – February 2, 2015
GaDOE Data Collections Pre-ID Webpage: http://www.gadoe.org/Technology-Services/Data- Collections/Pages/FY2015-Pre-ID-Labels-Resources.aspx
Pre-ID Cycle I (Winter EOC, GKIDS, and GHSWT): August 14-26
Winter 2014 EOC Enrollment Counts Collected: August 18-29
Winter 2014 EOC Pre-Administration Webinar: October 28-30
Update available for CTB Test Delivery Client for Winter 2014 EOC: November 12
Winter 2014 EOC Paper Materials: Shipment begins Nov. 10
Winter 2014 EOC Main Administration: December 1-January 9
Spring 2015 EOG Enrollment Counts Collected: January 8-22 (Tentative)
Spring 2015 EOC Enrollment Counts Collected: January 15-29 (Tentative)
Pre-ID Cycle (Spring EOC & Spring EOG): January 15-February 2
56
Spring 2015 EOC Mid-Month Administrations January 20-January 30
February 9-20
March 2-13
Spring EOG Pre-Administration Webinars: Week of Feb. 2 (Dates TBD)
Spring EOG Paper Materials Delivery Begin: March 9-16 (Tentative)
Spring & Summer EOC Pre-Administration Webinars: March 10-12 or 17-19
Summer EOC Enrollment Counts Collected: March 12-26 (Tentative)
Update available for CTB EOG Test Delivery Client: No later than March 16 (Tentative)
57
Spring EOG Main Administration: March 30-May 1
Spring EOC Paper Materials Delivery Begins: April 13 (Tentative)
Update available for CTB EOC Test Delivery Client: No later than April 13 (Tentative)
Spring EOC Main Administration: April 27-June 5
Summer EOC Paper Materials Delivery Begins: May 26
Summer EOC Main Administration: June 15-July 17
Standard Setting: Summer 2015
Reports Available: Fall 2015

Documents

2014 Fall Assessment Conference Assessment Program Update