Upload
keitha
View
34
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Regents Scanning CFN Training Sessions. Updated April 28, 2011. Objectives. The purpose of today’s session is to EDUCATE you on the new Regents scanning procedures, and EMPOWER you to turn-key the process at the school level. After the training, you will understand:. - PowerPoint PPT Presentation
Citation preview
1
Regents ScanningCFN Training Sessions
Updated April 28, 2011
22
ObjectivesThe purpose of today’s session is to EDUCATE you on the new Regents scanning procedures, and EMPOWER you to turn-key the process at the school level.
After the training, you will understand:
> Why the DOE is implementing the new Regents scanning process and the benefits of the changes;
> The overall scanning process to be performed by schools, and step-by-step procedures for each stage;
> How schools should plan for Regents exam week, and special situations to prepare for;
> What documents and resources are available for training and support, including how to use the demo IT environment to train schools; and
> Best practices for turn-keying the scanning process at schools.
33
Agenda> New SED Regents requirement & DOE scanning solution> Regents scanning A–Z
Process overview and timeline Step-by-step review Live demonstration
> Turn-keying to schools Lessons learned from the pilot program Demo testing environment Group activity: case study and discussion
> NY SED Special Study – June 2011 Regents> Ongoing support and resources> Q & A
44
SED Regents Requirement & DOE SolutionSED Requirement: Beginning in June 2011, the SED will require all districts to collect and submit item-level data for the following Regents exams:*
DOE Solution: Leverage existing technology and systems currently used by schools to capture and report item-level data. New scanning process highlights include:
> Answer documents: student-specific documents will now be generated and printed at school level
> Scoring: teachers mark Constructive Response items only> Data capture: scan answer docs with attendance scanners> Reporting results: real-time results available in ATS immediately
after scanning; automated data transfer from ATS to STARS*Other Regents exams & RCTs will use the traditional administration and scoring process.
55
SED Regents Requirement & DOE Solution
> Time savings through automation of manual processes Less overall time required for scoring No further data entry requirements for aggregate scores
> Increased accuracy in reporting test results Elimination of manual scoring of MC questions Section scores totaled automatically Automatic application of conversion tables
> Improved instruction Item-level data available for use in informing instruction at
student, teacher, school, and district levels> Audit trail
Answer document image-capture and line-item results will aid in special inquiries
Benefits to schools under the new process:
Pre-Planning
STARSATS &
Scanner
ATSSTARS
(& Beyond)
1. Schools receive training and begin preparations for Regents Scanning process.
2. Schedule students for Regents and produce exam invitations in STARS.
3. STARS data flows to ATS. Schools test scanner and generate and print pre-slugged answer docs in advance.
6. Schools scan answer docs for upload to ATS.
7. ATS instantly applies scoring key & conversion chart for scale score. Results available immediately.
8. ATS passes data nightly to STARS for report cards and transcripts, and for reporting to NYSED.
Exam Week & Scoring
5. Schools administer exams. Answer documents for walk-ins are printed on-demand. Scoring committee marks Constructive Response items and bubbles in scores on answer documents.
Attendance Scanner
Re-Scanning (As Needed)
Regents Scanning A-Z: Process Overview
6
Logistics Planning
Manual Score Verification
4. Schools conduct Regents week planning activities for logistics, including assignment of roles, and performing checklist items in the week before exams are administered.
7A. Score verification6A. Re-scan docs.
Overnight Data Transfer
Overnight Data Transfer
77
Timeline of Key StepsDates Key Step
ASAPALs begin scheduling sessions to train schools on the new process
May 1 – 31Relevant personnel at schools assign roles, conduct planning, review materials, test scanner, etc.
May 1 – 31Schools clean and test scanner and ensure adequate supplies (toner, paper, etc.)
By May 27 Schools register majority of students for Regents in STARS
June 1–10 Schools print answer documents in ATS for most students
June 8–14 Regents week logistics planning & final preparation
June 15–24 Regents administration
June 15–24 Scanning and data validation
88
Step 1: STARS> Main changes from the traditional process:
No changes – the process is exactly the same.> Main tasks:
Register students for Regents exam, and generate exam invitations.> Task owner: typically school Program Chair> Best practices:
Complete registration process for majority of student by May 27. Middle Schools: use proper exam codes in STARS for 8th graders.
If appropriate, notify the high school where students will take exam. Only register students expected to take exams (though when in
doubt, err in favor of registration). Use “Master Schedule” function for creating a single section, and
“Master Schedule Builder” for multiple sections.> Supporting materials: Regents Scanning Handbook, School Progress
Reports available to Assessment Liaisons> Support: CFN Data & IT Liaison
99
Steps 2–3: Printing Answer Docs in ATS> Main changes from the traditional process:
Answer documents are no longer included in the Regents exam booklet to be torn out.
Pre-slugged student-specific answer documents are printed in advance, or generated on-demand at school level.
No generic answer documents may be used. Answer docs are 1 page for Algebra & English; 2 for all others.
> Main tasks: Test scanner to ensure readiness, adequate supplies of toner
and paper. Print answer docs (for pre-registered students).
> Task owner: typically Pupil Accounting Secretary> Supporting materials: Regents Scanning Handbook, School
Progress Reports> Support: DIIT Help Desk, 718-935-5100
1010
Traditional & New Answer DocumentsTraditional Process Regents Scanning
Individualized bar code
Student MC answers
CR teachers’ grades
Alt. language & accommodations
Pre-populated student info
Scorers’info
1111
Steps 2–3: Printing Answer Docs in ATS> There are five options for printing answer docs:
1. “Not previously printed” (Batch printing for registered students)
Prints batch of answer documents that have not been printed
2. “Print by student ID” (For DOE walk-in student)Prints an answer sheet for any student one at a time. This option
may be used for another DOE student taking the exam at your school. (Use ATS SBIO screen to look up student ID.)
3. “Print by name (non-pub)” (For non-DOE walk-in student)Prints an answer sheet for a non-DOE student who is taking the
exam at your school.
4. “Charter school option” (For schools not using STARS)Batch printing option for schools who are unable to register students in STARS.
5. “Reprint answer documents”Reprint previously printed docs by student ID or answer document page number.
1212
Steps 2–3: Printing Answer Docs in ATS
Sorting Options
> Use RGPR screen for printing options 1, 2, and 3.
1313
Steps 2–3: Printing Answer Docs in ATS> Best practices for printing answer documents
Answer documents must be printed on white paper only. Use the same stock paper as is normally used for attendance forms. Using colored paper or heavy stock may result in scanning errors.
Print answer docs for pre-registered students by June 10. After printing batches of answer documents, a “test page”
will print. Scan the test page to confirm ink quality for batch. When printing, you may sort print jobs by teacher, section, or
room number. Ensure that adequate supplies of paper and toner are on-
hand.
1414
Step 4: Logistics Planning for Regents Week> Main changes from the traditional process:
Educate relevant staff members on new procedures and responsibilities for Regents scanning, and organize staff roles.
> Main tasks: Distribute test rosters and page one of answer docs to appropriate staff in
advance. Train exam proctors on new procedures and policies. Ensure adequate supply of toner, paper, pens/pencils. Establish process and task owners for dealing with walk-ins. (See slide
“Instructions for Walk-Ins”) Review Implementation Checklist
> Best practices: Roles and responsibilities are clear and communicated to all relevant staff
before Regents Week. Perform all checklist items.
> Supporting materials: Regents Scanning – Implementation Checklist> Support: CFN Assessment Liaison
1515
Step 5: Exam Week & Scoring> Main changes from the traditional process:
Answer docs for “walk-ins” printed on-demand on test day. Students use pre-slugged answer documents; pencil for MC,
pen for essays and DBQ questions. Teachers mark Constructive Response items only using pencil.
> Main tasks: Generate answer documents for “walk-in” test-takers. (See next
slide.) Proctors administer exams according to best practices and
emphasize new procedures (e.g. completely fill in bubbles, no stray marks, sign student declaration when finished).
> Best practices: Good preparation will lead to smooth exam administration
> Support: CFN Assessment Liaison
1616
Step 5: Instructions for “Walk-Ins”Schools should make preparations and assign roles to handle walk-in students during Regents week.
Step DOE Student Non-DOE Student
Confirm student’s ID Check photo ID Check photo ID
School notification Check student record against notification sent from her home
school
Must present signed letter on school letterhead with exam
specified
Print answer doc in ATS “Print by student ID”* so that data flows to STARS
“Print by name” for non-pubwalk-in student
Enter exam code Use appropriate exam code Use appropriate exam code
*If Student ID is unknown, use ATS SBIO menu to look it up.
> Inform appropriate staff how to direct walk-ins upon arrival, and appoint staff to print answer docs using ATS.
> Print answer documents by following the below instructions:
> Detailed instructions on handling walk-ins is included in the Handbook and FAQ.
1717
Step 6 & 6A: Scanning Answer Documents> Main changes from the traditional process:
Scanning will capture item-level data and images of answer documents (and any ancillary re-scans) for audit trail.
Immediate results available in ATS; conversion chart automatically applied for instant score.
> Main tasks: After notification by DOE, scan docs using attendance scanner. Docs with errors in reading bar codes will be reprinted with an error
message. Docs with bubble read errors will be indicated on DFS confirmation sheet. Re-scan affected documents.
Review ATS reports for “Warnings” observed in Teacher section.> Task owner: staff members who process attendance scanning> Best practices:
Scanning tips available in Regents Scanning Handbook Ensure that scanner has been cleaned in advance.
> Support: DIIT Help Desk, 718-935-5100
1818
Step 6 & 6A: Scanning Tips> Ensure the scanner is clean; a dirty scanner may result in inaccurate
results. See Tips for Scanning appendix for instructions on cleaning.> When scanning 2-page documents, it is recommended to batch all
page 1 documents and scan separately from page 2 documents. > Schools may scan the first page of two-page documents immediately,
before scoring is complete for page two.> If only one page of a 2-page document is scanned, it will be shown in
ATS as an “Incomplete Scan” until the second page is processed.> Use the ATS Status Report to review warnings.> After scanning, a confirmation page will print indicating pages with
scanning errors. The scanner will reprint pages for which the bar code could not be read.
> File the scanned batches together with the DFS confirmation page. This will assist in retrieving papers as needed in the future.
> Daily cutoff for overnight data transfer to STARS is 9pm on weekdays and 4pm on weekends.
1919
Step 7: ATS Reports & Results> ATS reports available immediately after scanning:
Use Document Status Report to ensure that all answer docs have been properly scanned, and to flag warnings.
Use Item Report to see students’ scores and inform promotion status of individual students.• Review report for “warnings/multiples/omits” by using “Include only
W/M/O” option. Use Item Distribution Report to inform decisions on present
and future instruction.> Detailed instructions on how to access the ATS reports are
available in the Regents Scanning Handbook.> Given the length and format of the ATS reports, they are best
viewed when printed. You may not see all the data in the ATS screen.
> Schools may download the ATS reports into Excel for further analysis.
2020
Step 7: ATS Reports (REDS - Status Report)> Report uses:
Displays score, accommodations, warnings, for each student. Shows summary of documents scanned, outstanding, and warnings. Use report to track outstanding unscanned and incompletely scanned
document.
Accom.Scores Scan-specific info
Document statuses
2121
Step 7: ATS Reports (REDS – Item Report)> Report uses:
Displays line-item data for each student. Shows specific questions to review (warnings). Use to flag Teacher section questions to investigate (e.g. omits, multiples).
TEACH SECSTUDENT SECTION
Correct answers
Multiple answersIncorrect answers CR raw scores
2222
Step 7: ATS Reports (RADI – Item Dist. Report)> Report uses:
Shows school-level aggregated data for each question. Use data to flag specific subject matter, question types, etc. for instruction.
‘*’ denotes correct answer
2323
Step 7A: Re-Scoring & Score Verification> Main tasks:
AS OF MAY 24, 2011 – THERE IS NO RESCORING OF ANY REGENTS EXAM INCLUDING ALL MATH AND SCIENCE EXAMS with scaled scores of 60-64.
Verification of scanned results for approximately 5% of answer docs.
Complete Manual Scoring Data Validation Form. Consult the FAQ as Regents week approaches for updates on
scoring procedures.> Best practices:
Two people at school level perform data verification> Supporting materials: Regents Scanning Handbook> Support: CFN Assessment Liaison
2424
Step 8: Accessing Results in STARS> Main changes from the traditional process:
Data flows to STARS automatically; no manual data entry required.
Results will be automatically applied to transcripts and report cards from STARS.
> Main tasks: Provided scanning is completed by 9pm – on day after
answer documents are scanned, schools may generate a STARS reports for all registered students who sat for Regents, regardless of test location.
> Best practices: Any future score changes (through re-scanning, DOE answer
key alteration, etc.) will flow through to STARS automatically.> Support: CFN Data & IT Liaison
Live Demonstration
2626
Turn-Keying: Demo Environment
> The demo environment is available from May 2-31. After May 31, the demo period will end and all demo data will be removed.
> Detailed instructions on how to access and use the demo environment are available through a Demo Script. You will use your existing ATS username and password.
> Demo scope:
> We highly encourage schools to take advantage of the demo environment to gain familiarity with the new processes and to help ensure smooth Regents administration.
A demo environment is available for training purposes. Schools are encouraged to use this to prepare for Regents Week and gain an understanding of the required processes.
DEMO
STARS(Scheduling)
ATS(Printing)
ATS(Scanning & Reporting)
STARS(Report Card &
Transcripts)
2727
Turn-Keying: Lessons Learned from Pilot
> Schools with advanced preparation performed well during January Regents Preparations include: communication of new processes and
policies, defined staff roles, on-time scheduling/printing of answer documents, prep for “walk-in” students, etc.
> Many schools needed additional support and training around the technical systems. Many schools did not utilize ATS reports (Document Status Report,
Item Report, Item Distribution Report) for scanning validation and instructional benefits. The demo environment is intended to assist with this for the June implementation.
> Scanning technology was validated through the pilot.
DOE conducted a pilot program in January to administer the new process at select schools. The most valuable lessons learned from the pilot are:
2828
Turn-Keying: Case Study & Discussion> Review of Regents Scanning Implementation Checklist.> Case Study exercise.> Discussion: in groups of 3-5, discuss the following questions:
How do you plan to effectively turn-key this new initiative to schools?
Who will you include in the school-level training? What challenges do you anticipate? What items need to be emphasized or modified for different
size schools within your network? How will you support your schools in using the new item-
level data that is produced from this process? What questions do you have about the process itself or its
implementation? (Record for Q&A)
2929
Turn-Keying: Roles & ResponsibilitiesAssessment Liaisons Data IT Liaisons
Overall point of contact for schools for Regents Scanning initiative
Ensure schools and network staff have appropriate access levels in ATS and STARS
Responsible for coordinating and delivering training to relevant school staff within network
Support schools with questions on printing answer documents. Accompany AL during school questions.
Ensure schools are properly equipped for Regents scanning (e.g. paper, toner, printing capability, STARS scheduling)
Support schools in scheduling students for Regents in STARS and with all technical questions relating to ATS
Verify schools have performed data verification of students exams
Provide any additional training to schools on ATS or STARS, as requested
Be on-hand during Regents week to provide support as needed
Monitor STARS and ATS before & during test administration for data integrity issues and general activity. Be on hand for support during Regents week.
Escalate any unanswered policy or process questions/issues
Escalate any technical policy or process questions/issues
3030
Turn-Keying: Network & School ChecklistMain Action Items Timeline
CFNs coordinate trainings with schools May 1-31
Schools notify and train relevant staff, and make preliminary preparations (check toner, paper supplies)
May 1-31
Schools utilize demo environment to run mock exam process May 2-31
Schools schedule students for Regents in STARS By May 28
Schools print answer documents for majority of students By June 10
Schools establish & communicate Regents week plan (roles, logistics) June 1-14
Schools administer exams, scan documents, conduct data validation June 15-24
*Refer to Implementation Checklist for a complete list of tasks to be performed by schools.
3131
SED Special Study – June 2011 Regents
> Affected Regents exams: Comprehensive English Algebra 2/Trigonometry
> Affected schools: See NY SED Special Study for June 2011 Regents – Affected
Schools handout> Process highlights:
The special study process will supersede any NYC DOE administrative process – including Regents Scanning for the Comprehensive English exam.
CFNs will receive instructions on how affected schools must administer these exams.
The NYS Education Department is conducting a state-wide special study which will affect select schools for two June 2011 Regents.
3232
On-Going Support & Resources
> Documentation and updates: ARIS: https://www.arisnyc.org/connect/node/395478/community
> Technical support (STARS & ATS): CFN Data & IT Liaison
> Scanner and printer: DIIT Help Desk, 718-935-5100
> Overall process support: Borough Assessment Implementation Directors (BAID)
> Regents week emergency inquiries: Patrick Purcell ([email protected]; 212-374-5553) Marc Bush ([email protected]; 201-755-2519) Kirby Fields ([email protected]; 646-379-8422)
The following resources are available to support implementation and execution of Regents scanning.
*Please note that guidance from DOE on unresolved items will be issued on a rolling basis.