ASSESSMENTS & ACCOUNTABILITY

Assessments and Accountability v2.0

ASSESSMENTS & ACCOUNTABILITY

Suspense Resolution and

Reports


– Mute phones while listening (*6 to mute and un-mute)– Do not place your phone on HOLD– Silence cell phones– Hold questions until Q&A periods– Submit all questions via the Chat function

GROUND RULES


ASSESSMENTS & ACCOUNTABILITY (A/A)

EXPECTATIONS


A/A Expectations

Assessment and Accountability functionality in CALPADS is being implemented to support:

− Building a complete individual student assessment history for STAR, CAHSEE, and CELDT

− Improving the accuracy and consistency of data used to evaluate performance of individual students and educational entities


A/A Expectations

Achieving these objectives will require several iterations of CALPADS functionality

− Current phase of functionality is focused on identifying issues for both LEAs and CDE related to the loading of data

− Future functionality will address these issues as well as enhance the accessibility and usability of the data for LEAs and CDE


A/A Expectations

The initial CALPADS A/A functionality requires no changes in the way LEAs currently work with assessment vendors or with CDE

− Student Demographic changes will continue to be made through the testing vendor (within vendor’s test correction window)

− Testing Vendors will provide test results & reports as usual

− Pre-ID, where applicable, will continue as usual− Testing processes will continue as directed by

assessment program offices− AYP/API results will continue to use the student data you

supply to test vendors


A/A Expectations

LEAs will:−Match vendor results records for your LEA to the

correct SSID when CALPADS cannot make the match automatically

−Verify the reports−Provide feedback to improve functionality


A/A Expectations

CDE will:

− Match vendor results records to the correct SSID in CALPADS, when a LEA flags a record as “Not Mine”

− Negotiate contracts with assessment vendors that ensure the data needed for a student’s complete assessment history in CALPADS is available and timely

− Coordinate with each assessment program area to enable CALPADS to…− Provide a consistent approach to managing and

reporting assessment data − Report assessment results that reflect the testing

procedures communicated to LEAs


A/A Expectations

CDE will (cont): − Evaluate the outcome to determine the impact on LEAs

and Accountability reports of using CALPADS data instead of vendor data

− Work with LEAs to provide functionality focused on data accessibility and usability

− Communicate When and How: Results data left in suspense will impact Accountability

reporting Current vendor relationships will be impacted


A/A Expectations

LogisticsA schedule for data loads is included in the report sections

for each Program

Assessment and Accountability Coordinators receive an email when vendor data is loaded

Report content is frozen when CDE performs the last snapshot for a Program Administration after all data received from the assessment vendor has been loaded


SUSPENSE MANAGEMENT


Suspense Management

Suspense - (i.e. unsuccessful match)Matched – (i.e. successful match)

CALPADS Match Results:

When vendor results data are loaded to CALPADS, the result of the matching process can be either …


Suspense Management

Reason for Suspense Record:

Mismatch in SSID and/or student demographic data (gender, date of birth year) between the assessment results file and CALPADS

Missing CALPADS enrollment at time of testing Missing SSID Administration Date

As soon as one of these conditions is not met, CALPADS places the record in suspense.

SSID Administration Date is the date in an SSID’s assessment results record that is used in SSID Enrollment Validation. STAR = Testing Start Date; CAHSEE = ELA Test Date and Math Test Date; CELDT = Date Testing Completed. (See CALPADS User Manual Glossary for additional information)


Suspense Management

Suspense Errors:• Date of Birth Mismatch - If Student Date of Birth Year for SSID

in results record <> Student Date of Birth Year in ODSR SINF data that was valid at the time of testing

• Gender Mismatch - If Student Gender Code for SSID in results record <> Student Gender Code in ODSR SINF data that was valid at the time of testing

• SSID Mismatch - If SSID in results record does not exist in ODSR

• Admin Date Empty - If the Administration Date in the results record is blank

• LEA Mismatch - If enrollment in LEA at the time of testing could not be validated


Suspense Management

Suspense Resolution Actions:− The LEA correcting CALPADS data− The vendor correcting results data− The LEA and/or CDE assigning the suspense record to the

correct SSID

Suspense Resolution Methods:− Manual match which provides reporting data immediately− Automated match (nightly suspense, load suspense, CDE

snapshot suspense) which provides reporting data at completion of process


Suspense Management

Suspense Resolution Workflow:

LEA Workflow:• Review Suspense Records• Determine best resolution and apply• Wait for data refresh and verify suspense resolved


Suspense Management

Importance of Suspense Resolution:

A record in suspense will not be matched to a CALPADS SSID, which leaves the CALPADS assessment longitudinal record incomplete for this SSID

As students enroll at other districts their assessment data will be missing if not matched

Records in suspense do not appear in reports; therefore, your CALPADS assessment and AYP/API reports will not be complete


Suspense Management

Suspense data can be resolved both Online & in Batch

Both methods are accessed from the Assessment menu

CALPADS Functionality

Note: Batch resolution is normally used only by the largest LEAs. High level instructions are included in the Wrap-Up Documentation.


MANAGING SUSPENSE DATA - ONLINE


Managing Suspense Data - Online

Assessments > Online Maintenance > Manage Suspense Data Select filter for Program and Administration (subject to user’s access rights), and Academic Year Select Match Result = “LEA Suspense” to locate all students or an individual student with a record in suspense

SUSPENSE STEP 1: Review Student List

LEA MatchedLEA SuspenseCDE Suspense

ADMIN Date &Suspense Reason


Manage Suspense Data - Online

Condition Cause Action Resolution Method

SSID Mismatch Correct results record SSID does not match CALPADS

Correct CALPADS first

Match manually, providing report data real-time – or -Allow nightly suspense process to match automatically, providing report data the next day

Incorrect results record SSID

Contact vendor to correct before or after match

Match manually, providing report data real-time – or -Allow subsequent load of vendor data to resolve automatically, providing report data at that time 2

Demo Mismatch(Birth Date Year/Gender)

Correct results record demo does not match CALPADS

Correct CALPADS first1


Incorrect results record demo



LEA Mismatch(Enrollment Not Validated)

Incorrect CALPADS enrollment

Correct CALPADS first


Incorrect SSID in results record



Correct CALPADS enrollment but no match

Submit Service Ticket

See Enrollment Validation Rules in “Glossary of Terms and Business Rules – Assessments”

SSID Administration Date Missing

Blank Administration Date in results record

Contact Vendor to correct

SUSPENSE STEP 2: Determine Resolution

1 Manual match can be first, but incorrect data is provided to reports until CALPADS is corrected. 2 Data load confirms vendor’s change when suspense resolved.



1. If the student record in suspense does not belong to your LEA, select “Not Mine”

2. The record is placed in CDE’s queue for resolution. CDE can match the record to a student in another LEA.

SUSPENSE STEP 3: Apply Resolution1

2

1

2



If the resolution for the Suspense Reason requires …– Changes in vendor data, the suspense process that runs during a load will eliminate

the suspense when CDE receives and loads the corrected data from the vendor.– Data addition/changes in CALPADS, make the additions/changes and then either 1)

match immediately to the corrected record using STEP 4, since the changes to ODS are available real-time, or 2) wait for the nightly suspense process to recognize that the CALPADS demo has changed and now matches the results record demo, eliminating the suspense automatically

– Matching the results record with incorrect demographic data to a CALPADS record with the correct demographic data, use STEP 4.

SUSPENSE STEP 3: Apply Resolution (cont)



Select Match initiates the user-defined search for a CALPADS SSID to which the result record belongs

SUSPENSE STEP 4: Initiate Match

Annual

Select Match

John Doe

Annual



Enter SSID and/or demographics, using * wildcard if needed

SUSPENSE STEP 5: Enter Search Criteria

Enter Search Criteria



Matching results are displayed by SSID and/or Demographics, depending on the criteria entered

SUSPENSE STEP 6: Review Returns

Suggested Correction Record

Note the incorrect Birth Year



Make selection and answer “OK” to initiate the match

SUSPENSE STEP 7: Choose Record

Select the record to which suspense record will be matched



Both the suspense record and the selected match record are displayed

SUSPENSE STEP 8: Match Records

Select “Match Records”



Return to STEP 2 and determine resolution Repeat STEPS 3 – 10 (avoiding STEP 9)

SUSPENSE STEP 9: Resolve Problems

When the resolution requires changing CALPADS data, the new correction record will be available to your next SEARCH.

Valid enrollment required!!



When match is successful, return to Student List to resolve another record

SUSPENSE STEP 10: Return to Student List

The newly matched record is available for reporting immediately



The matched Student List displays the Assessment Results record and the CALPADS record to which it has been matched The source of the match is identified in the field “Match Source”. The source could be Batch, LEA, or State.

MATCHED STEP 1: Confirm Matched Records





Matched records can be ….

MATCHED STEP 2: Resolve Incorrect Match

− unmatched if incorrectly matched (UNMATCH) and re-matching will occur later− unmatched if incorrectly matched and re-matching will occur now (SELECT

NEW MATCH)− unmatched and sent to CDE for matching to another LEA (NOT MINE)


VERIFY RESOLUTION


Verify Resolution

Data refresh timing depends on the choice you made for resolution

– Manual match with immediate resolution and data refresh

– Automated match (nightly suspense, load suspense, CDE snapshot suspense) with resolution and data refresh at completion of process

WAIT FOR REFRESH

Reminder: When you make changes to data in the ODS, they are replicated to the ODSR in real time. However, the change will not resolve a suspense until one of the following:

– You match the suspense data to the corrected CALPADS record– Suspense processing clears the suspense (in nightly process,

load, or CDE snapshot)


Verify Resolution

Review Student List – Online Assessment Reports Status Reports

VERIFICATION METHODS


Verify Resolution

STATUS REPORTS

CDE Only 50.1 Assessment

Upload – Detail 52.1 Assessment

Snapshots - Detail


Verify Resolution

Description: Displays the LEA and School summary results of the Assessment Suspense process by Administration.

Data as of: Last Update field in header, which specifies the date for the last Suspense process included in this report

Data Included: For a County, the number of LEAs and Schools with data in suspense is reported. For an LEA/School displayed, the number of records in suspense is reported by Suspense Type (i.e. LEA or CDE) and then broken down within the Suspense Type by the specific conditions that generated the suspense.

Report Operation: The data can be displayed by County, LEA, and/or School for all or selected Administrations for a Program. Data can be filtered also by Suspense Type (i.e. CDE, LEA) and its appropriate Suspense Code.

Links: None

Calculations: Totals

Glossary References: None

51.1 Assessment Suspense - Summary


Verify Resolution

51.2 Assessment Suspense – Student Detail

Grey, Jordan

Abdul, Jordan

Ferrell, J

3224589451,

2274589451,

5288589451,

Description: Displays the LEA and School results of the Assessment Suspense process for individual students for a Programs and Administration.

Data as of: Last Update field in header, which specifies the date for the last Suspense process included in this report

Data Included: Individual students with results record in suspense.

Report Operation: The data is displayed for a Program and Administration by County, LEA, and/or School. Data can be filtered also by Suspense Type and its appropriate Suspense Code. Note that all student data is sourced from vendor results records, not from CALPADS ODSR data.

Links: None

Calculations: None

Glossary References: SSID Administration Date


Assessments & Accountability Reports


A/A Reports

The following Assessment reports are available (initially for 2010-2011 data)

– Summary LEA results for STAR, CAHSEE and CELDT, including state comparisons

– Individual student result detail supporting the summary results for STAR, CAHSEE, and CELDT

– Currently enrolled students missing CAHSEE – Currently enrolled students needing CELDT

The following AYP/API reports will be available– LEA and School summary of students included in the AYP/API results by

subgroup– Individual students included in the AYP/API results

The following individual student report is available– Individual student assessment history (initially 2010-2011 data)

REPORTS AVAILABLE


A/A Reports

Assessment reports use vendor score/result data for an SSID that is matched to CALPADS Student Profile data for that SSID

The Student Profile data display as follows:– For reports that present results data by ADMIN, data

valid in CALPADS at the time of testing is used for filters and sorting/displaying report content, while vendor score/result data is used for report content

– For reports that present results data for Current Students, data valid in CALPADS currently is used for filters and sorting/displaying report content, while vendor score/result data for the reported test ADMIN is used for report content


ASSESSMENT REPORTS

Report content for a Program Administration can continue to change as long as suspense processing is occurring for that Program Administration

– Students can be added when a new match is made through the Suspense Resolution Process, either online or batch, which updates the AADS immediately, as well as when new vendor data are loaded that resolves a record in suspense

– Student demographic data on the report can change when the Suspense Process, either nightly or in a data load, refreshes student data in AADS for students whose data have changed in the ODS (Note: Non-Participant only refreshed in Load)

– Students can be removed when the Suspense Process, either nightly or in a data reload, is not able to re-validate the student’s enrollment due to changes made in ODS enrollment data that were valid at the time of testing

Summary: Nightly suspense process and the suspense process in a reload impact existing matches only if data have changed. Students matched by LEA or CDE will not be unmatched unless enrollment can no longer be verified. However, CALPADS demo data changes will be incorporated.


ASSESSMENT REPORTS

ASSESSMENT MENU

Two entry points from Assessment

Menu


ASSESSMENT REPORTS

xxxxxxxxx

REPORTS


ASSESSMENT REPORTS

xxxxxxxxx

REPORTS (Cont)

CDE Only50.1 Assessment

Upload – Detail52.1 Assessment

Snapshots - Detail

Coming


UNDERSTANDING STAR REPORTS


Understanding STAR Reports

The STAR Program has one Administration, called ANNUAL

The ANNUAL Administration includes multiple Assessments:

California Standards Tests (CST) California Modified Assessment (CMA) California Alternate Performance Assessment (CAPA) Standards-based Tests in Spanish (STS)

INTRODUCTION



INTRODUCTION (Cont)

STAR vendor results for the ANNUAL Administration are loaded multiple times

– The loads are named P1, P2, P3, and Post P-3

– Loads P2 and P3 overwrite the previous load

– The Post-P3 load does not overwrite P3



GENERAL CALENDAR - STARSchedule Event Description

August

(Week 1)

P1 LOAD Includes approximately 99.9% of LEA participants Received about 8/1. Required to meet statutory reporting deadline of 8/15.

August

(Week 2/3)

AYP/ API (P1) LOAD

Concurrent with posting on DataQuest STAR aggregate downloadable files from

assessment vendor AYP/API reports and data files pre-release for LEAs AYP/API reports and data files release to the public

(August 15)

September (Week 1)

P2 LOAD Includes 100% of results, including LEA’s remaining results not included in the P1 (e.g. Year-round schools). Received about 9/1.

September

(Week 2-4)

LEA Data Review

LEAs examine STAR and CAHSEE data to identify needed student data changes. LEAs make changes in CALPADS and through vendor.

Assumption: Results data is loaded to CALPADS within x days of its receipt by CDE



GENERAL CALENDAR – STAR (Cont)

Schedule Event Description

October AYP/ API (P2) LOAD

Concurrent with posting on DataQuest AYP/API reports and data files that incorporate P2

STAR late testing LEAs, CAHSEE data corrections, appeal and exception decisions, and CAPA reallocations related to 1% cap for LEAs (October 1)

October - December

LEA Data Review

LEAs examine AYP/API (P2) results to identify needed student data changes. LEAs make changes in CALPADS and through vendor.

December/ January

P3 LOAD Includes 100% of results, including LEA student data changes made since P2.

December/ January

AYP/ API (P3) LOAD

Concurrent with posting on DataQuest AYP/API reports and data files that incorporate P3

student data changes




GENERAL CALENDAR - STAR (Cont)


January - February

LEA Data Review

LEAs examine AYP/API (P3) results to identify needed student data changes. LEAs initiate appeal process through CDE for approval to submit changes to vendor.

January - February

Post-P3 LOAD(s)

Includes only LEA specific results submitted to the vendor that have been approved by CDE through the appeals process. The number of loads and the timing will vary.

February - March

AYP/ API (Post-P3) LOAD

Concurrent with posting on DataQuest Final AYP and Growth API that incorporate all Post-

P3 changes (February) and Final Base API (March)




Test Type should be Assessment

REPORT 20.1 LEA SummarySummary of the most current STAR results for a school by grade for each Assessment. Students assessed at multiple LEAs are reported at each LEA.

NOTE: Terms in Italics are defined in the User Manual Glossary



Description: Displays the most current STAR results for a school by grade for each of the included Assessments. It includes the number and percentage of students for each performance level and the overall Mean Scale score. If multiple scores exist for a student at the same school for the same test, the best result is reported (i.e. highest scale score – along with its associated performance level). Students assessed at multiple LEAs are reported at each LEA.

Data as of: Last Snapshot Date +/- subsequent suspense processing

Data Included: School Testing Window, Total number of students in Testing Enrollment, number & % of enrolled Students Tested, number of Students Tested with valid scores, mean scale score for Students Tested with Valid Result, percentage of students with Valid Results in each listed performance level

Totals: Summary by grade for the selected schools

Links: From School Name and Grade to 20.2 Student List

Glossary References: School Testing Window, Testing Enrollment, Students Tested, Valid Result

REPORT 20.1 LEA Summary (Cont)

The following information is available in the CALPADS User Manual for each report.



REPORT 20.1 LEA Summary (Cont)

Each report heading provides information on the load and snapshot represented

Test Type/Purpose should beAssessment

Filters allow reporting by specific assessment, content area, test, student groups, etc.

P2




REPORT 20.2 Student ListDetail of the most current STAR results by primarily enrolled student for Test Participants for all or selected Assessments.

• Include Indicator - indicates how student’s record is used in reports

• Continuous Enrollment – indicates the category to which student belongs

• Test Variations - detailed in legend

Glossary References: Assessment, Test Participant, Include Indicator, Continuous Enrollment




REPORT 20.3 LEA and State SummarySummary of the most current results for an LEA and the state

Glossary References: Testing Enrollment, Students Tested, Valid Result


UNDERSTANDING CAHSEE REPORTS


Understanding CAHSEE Reports

The CAHSEE Testing Program has multiple monthly Administrations (ADMINs):

– JUL, OCT, NOV, DEC, FEB, MAR, & MAY

A CAHSEE vendor results file is loaded and stored for each monthly Administration

Cumulative results files are also received quarterly and at the end of the year

INTRODUCTION



All result files may include records for a previous ADMIN during the year (e.g. The monthly file for a DEC ADMIN may include corrected data for a student from the previous OCT ADMIN)

– If a record in a load for an SSID was in a previous load for the same Test &

same ADMIN, the most current record is reported for the ADMIN

different ADMIN, each record is reported for its ADMIN

INTRODUCTION (Cont)



GENERAL CALENDAR - CAHSEESchedule Event Description

September

(Middle)

MONTHLY-JUL Includes: JUL. Received 6-7 weeks after ADMIN

September (End)

QUARTERLY-1 Includes: JUL

December (Middle)

MONTHLY-OCT Includes: OCT. Received 6-7 weeks after ADMIN

January

(Middle)

MONTHLY-NOV Includes: NOV. Received 6-7 weeks after ADMIN

February (Middle)

MONTHLY-DEC Includes: DEC. Received 6-7 weeks after ADMIN

February (End)

QUARTERLY- 2 Includes: OCT, NOV, DEC + Q1 (JUL)

April (Middle)

MONTLY-FEB Includes: FEB. Received 6-7 weeks after ADMIN

Assumption: Results data is loaded to CALPADS within X days of its receipt by CDE



GENERAL CALENDAR – CAHSEE (Cont)


June

(First)

MONTHLY-MAR Includes: MAR (largest ADMIN). Received 7-8 weeks after ADMIN

June

(End)

QUARTERLY-3 Includes: FEB, MAR + Q2 (JUL, OCT, NOV, DEC)

Pre-EOY - Used for LEA Preview Reports

July

(Middle)

MONTHLY-MAY Includes MAY (Make-up ADMIN). Received 6-7 weeks after ADMIN

July - Sep LEA Data Review

After last ADMIN, LEAs can do corrections for all ADMINS during the year. Corrections must be complete by Oct 8.

July/August (Begin)

QUARTERLY-4 Includes: MAY + Q3 (JUL, OCT, NOV, DEC, FEB, MAR)

Preliminary Final for DataQuest & AYP/API for AYP Participation Rate

Assumption: Results data is loaded to CALPADS within X days of its receipt by CDE



GENERAL CALENDAR – CAHSEE (Cont)


August

(Week 2/3)

AYP/ API (P1) LOAD

Concurrently with posting on Data Quest … STAR aggregate downloadable files from

assessment vendor AYP/API reports and data files pre-release for

LEAs AYP/API reports and data files release to the

public (August 15)

October (Middle)

EOY Includes: JUL, OCT, NOV, DEC, FEB, MAR, MAY

Final - Cumulative for DataQuest & AYP/API

October AYP/ API (P2) LOAD

Concurrently with posting on DataQuest … AYP/API reports and data files that incorporate

P2 STAR late testing LEAs, CAHSEE data corrections, appeal and exception decisions, and CAPA reallocations related to 1% cap for LEAs (October 1)




Three types of CAHSEE reporting are available in CALPADS Census – Provides an unduplicated count of primarily enrolled 10th

grade students who took either part or all parts of the test at the enrolled school during the administration(s) being reported.

Administration - Provides overall results of CAHSEE administrations for all current students primarily enrolled in a school who took either part or all parts of the test at the enrolled school by grade level during the administration(s) being reported.

Enrollment - Provides overall results of CAHSEE administrations for all current students primarily enrolled in a school, both students who did not take the test and students who took the test, regardless of where the test was administered.

REPORTING TYPES

See CALPADS User Manual Glossary for additional definition



REPORT 21.1 Census – LEA SummarySummary of the CENSUS Reporting Type results for primarily enrolled Students Tested in Grade 10 on each part of the assessment. The number and percentage of students who met the high school graduation requirement are also included.

Glossary References: Students Tested, Valid Result, Passing Score, Requirement Met, Census Reporting Type



REPORT 21.2 Census – Student ListStudent detail of the CENSUS Reporting Type results for primarily enrolled Students Tested in Grade 10

Glossary References: Students Tested, Requirement Met, Continuous Enrollment, Census Reporting Type



REPORT 21.3 Administration – LEA SummarySummary of the ADMINISTRATION results for primarily enrolled Students Tested (grades 10-12 and AD) and the number and percentage of students who met the high school graduation requirement.

Glossary References: Students Tested, Valid Result, Invalid Result, Passing Score, Requirement Met, Administration Reporting Type



REPORT 21.4 Administration – Student ListDetail of ADMINISTRATION results for primarily enrolled Students in grades 10-12 and AD that were tested at the reporting school.

Glossary References: Students Tested, Administration Reporting Type, SSID Administration Date, Requirement Met, Continuous Enrollment



REPORT 21.5 Enrollment – LEA SummarySummary of testing results for current primary enrollment in grades 10-12 and AD

Not Tested should be No Test

Glossary References: Students Tested, Valid Result, Requirement Met, Enrollment Reporting Type, Current Enrollment



REPORT 21.6 Enrollment – Student ListDetailed CAHSEE results for individual students for current primary enrollment in grades 10-12 and AD. If multiple scores exist for a student, all scores received are displayed regardless of the school in which the score was received.

Glossary References: Students Tested, Valid Result, Requirement Met, Enrollment Reporting Type, Current Enrollment, Status (CAHSEE)



REPORT 21.7 Administration-LEA DisaggregatedSummary of CAHSEE results, by demographics and programs, for primarily enrolled Students Tested in grades 10-12 and AD with number & percentage of students who met the high school graduation requirement.

Glossary References: Students Tested, Valid Result, Invalid Result, Passing Score, Requirement Met, Administration Reporting Type


UNDERSTANDING CELDT REPORTS


Understanding CELDT Reports

The CELDT Testing Program has an Administration (ADMIN) every month:– JUL, AUG, SEP, OCT, NOV, DEC, JAN, FEB, MAR, APR,

MAY, & JUN

The Annual assessments occur from July 1 – October 31. CDE receives the data in January/early February. If an initial assessment occurred during this timeframe, it will be included in the data received.

Initial assessments occur each month as students enroll. CDE receives the data at the end of the year.

CDE is negotiating with the vendor to receive results monthly

INTRODUCTION



GENERAL CALENDAR - CELDTSchedule Event Description

January/ February

PRE-DRM LOAD

Includes (Annuals & Initials): JUL, AUG, SEP, OCT

LEAs get results to vendor by Mid-November. When CDE receives, it is checked before it goes into LEA DRM (Data Review Module)

February - March

LEA Data Review

LEAs make corrections in DRM

April (Early)

POST-DRM LOAD

Includes: JUL, AUG, SEP, OCT

Concurrent with posting on DataQuest & calculation of Title III AMAO

FINAL Includes (Initials): NOV, DEC, JAN, FEB, MAR, APR, MAY, JUN




REPORT 22.1 CELDT – LEA SUMMARYSummary of selected administration(s) for primarily enrolled students by Grade

Glossary References: Students Tested, Valid Result, Invalid Result, Criterion Met, Grade Level Mean Scale Score

Note: Alternate = # Students Tested with Invalid Result due to Alternate Assessment



REPORT 22.2 CELDT – Student ListDetail for current primarily enrolled Students Tested during the selected administration(s). If multiple scores exist for a student, all scores received at the school for the reported administration(s) are displayed.

Glossary References: Students Tested, Invalid Result, Valid Result, Criterion Met

Test Purpose = Annual or Initial



REPORT 22.3 CELDT – English Learner StatusDetail of the current year status of the CELDT for current primarily enrolled English Learners. If there is no current year data, it reports if it is Due or Overdue.

Glossary References: Students Tested, Students Expected To Be Tested, Test Status, Test Due Date, SSID Administration Date, Criterion Met

Test Status = Complete-Met, Complete-Not, Due, Overdue



REPORT 22.4 CELDT – County Trend by LEASummary of results for primarily enrolled students for selected administration(s) and selected academic years by LEA

Alternate = # Students Tested with Invalid Result due to an Alternate Assessment

Glossary References: Students Tested, Valid Result, Invalid Result, Criterion Met


UNDERSTANDING INDIVIDUAL STUDENT

ASSESSMENT REPORTS


Understanding Student Reports

GENERAL CALENDAR

Report is available at any time to the school in which the student is primarily enrolled

The data included is subject to the loads and suspense processing of each Program.



REPORT 40.1 Individual Student ProfileDisplays the entire Assessment history for individual students current primarily enrolled in the selected LEA. It displays scores for all Administrations, regardless of the LEA administering the Assessment.

Result (CELDT = Criterion Met, CAHSEE = Requirement Met, STAR = N/A)

Test Condition = condition that makes the result an Invalid Result (see Legend)

Glossary References: SSID Administration Date, Criterion Met, Requirement Met, Assessment, Administration



REPORT 40.1 Individual Student Profile (Cont)

Report Operation: This individual student report can be printed for current primarily enrolled students for all of an LEA’s schools or only selected schools. It can also print for an individual student. When printing for a school/LEA, the report displays results regardless of where the assessment was administered. It can be filtered to display the results for selected Academic Years, Programs, Assessments, Tests, etc. It can be requested for subgroups of students. The legend on the following page appears at the bottom of the report. A user selection allows it to print per student.

Test Type/Purpose should be Assessment



REPORT 40.1 Individual Student Profile (Cont)

Entries appear only if used in report



VIEW STUDENT RESULTS - Online

STAR=AllCAHSEE=10-12, ADCELDT=EL+ TBD

Results available online for current primarily enrolled students by …

•Program•Administration(s)•Result•SSID

Click Individual student for details


Understanding Student Reports (Cont)

VIEW STUDENT RESULTS – Online (Cont)

Complete history of STAR, CAHSEE, and CELDT


WRAP-UP


Wrap-Up

The rules for identifying the date CALPADS used to locate the enrollment at the time of testing may not accurately identify a student’s enrollment at the time of testing, since an actual day of testing is not available in the results data for STAR and CELDT. Consult the Service Desk if you suspect this has occurred.

Reports display results of most recent load +/- suspense resolutions until CDE notifies of final snapshot which freezes

KEY POINTS


Wrap-Up

When you make changes to data in the ODS, they are replicated to the ODSR in real time. However, the change will not resolve a suspense until one of the following:

– You match the suspense data to the corrected CALPADS record– Suspense processing clears the suspense (in nightly process, load, or CDE

snapshot)

KEY POINTS

A suspense record can be resolved in CALPADS by one of the following …− the LEA correcting CALPADS data− the vendor correcting results data− the LEA and/or CDE manually assigning the suspense record to the

correct SSID


DOCUMENTATION Frequently Asked Questions (FAQs) Key Terminology Loading Assessment Results Data Validation & Reporting Data Managing Suspense – Batch Files

Wrap-Up


Wrap-UP

Please Complete Our SurveyQuestions

http://www.csis.k12.ca.us/survey/calpads-training/


FAQs


FAQs

SUSPENSE QUESTIONS

When should I begin resolving suspense data?If suspense conditions exist in your data, the suspense records are available to you for resolution after each suspense evaluation. Data is evaluated … when vendor results data is received and loaded for a Program Administration nightly after the first vendor data is loaded for a Program Administration when CDE initiates a manual snapshot

The schedule for receiving results data differs for each Program Administration. Therefore, the data is received and loaded at different times during the Academic Year. See General Calendar in the Reports section for information about each Program’s data load schedule. Because reports are available for viewing immediately after the completion of the load process, this load schedule also indicates the schedule for the availability of suspense data.


FAQs

SUSPENSE QUESTIONS (Cont)

How will I know when I have suspense data to resolve?When Assessment results have been loaded (which means that the suspense and snapshot processes have also completed), notification that suspense items are available for resolution and that reports are available for viewing will be: Displayed in CALPADS Notifications on the Home page Emailed to the following roles:

– LEA STAR Coordinator for STAR data load– LEA CAHSEE Coordinator for CAHSEE data load– LEA CELDT Coordinator for CELDT data load– CDE Assessment Coordinator

NOTE: Notification occurs only after data is loaded. There is no notification after the nightly process or after a CDE initiated snapshot.


FAQs


Should I monitor the status of my suspense records through the operational reports?

The operational reports are available for your convenience. Monitoring can be accomplished solely through the “Student List”.

How frequently should I monitor the status of my suspense records?Once you begin resolving your suspense records, you will want to monitor your list of records to verify that data changes you made have removed the record from your list of suspense records as expected.

Since records in suspense do not appear in reports, your schedule for monitoring them should be based on the schedule that accurate reports need to be available for your LEA and the CDE. See General Calendar in the Reports section for information about each Program’s data availability for reports.


FAQs


Will I need to resolve CALPADS suspense records and work with my vendor to correct vendor results data that caused the suspense?

Yes, until CDE announces that it will use CALPADS data for Data Quest reports. When CALPADS provides data to the CDE accountability process (AYP/API), it will provide both CALPADS and vendor data for evaluation. CDE will compare the data to determine whether/when CALPADS data can be used.


KEY TERMINOLOGY


Key Terminology

KEY TERMS & RELATIONSHIPS

PROGRAM STAR

CAHSEECELDT

ADMINISTRATION

ASSESSMENT STAR = CST, CMA, CAPA

CAHSEE = CAHSEECELDT = CELDT

ASSESSMENT DATE (Reported)

TESTExamples:

English/Language ArtsAlgebra I EOC

CONTENT AREA

ADMINISTRATION DATE (SSID)

CALENDAR

Reporting and Data Storage utilize standard definitions


ASSESSMENT DATE (SSID)

PROGRAM STAR

CAHSEECELDT

ADMINISTRATION

ASSESSMENT STAR = CST, CMA, CAPA

CAHSEE = CAHSEECELDT = CELDT

ASSESSMENT DATE (Reported)

TESTExamples:

English/Language ArtsAlgebra I EOC

CONTENT AREA


CALENDAR


ASSESSMENT DATE (SSID)


Key Terminology

Program (Testing Program)

– STAR Testing Program– CAHSEE Testing Program– CELDT Testing Program

Administration - ADMIN (Program Administration) Indicator for a Program’s testing

DATA DEFINITIONS

cycle:– STAR = ANNUAL– CAHSEE = JUL, OCT, NOV, DEC, FEB, MAR, & MAY– CELDT = JUL, AUG, SEP, OCT, NOV, DEC, JAN, FEB, MAR, APR, MAY, & JUN


Key Terminology

DATA DEFINITIONS (Cont)

Calendar (Program Testing Calendar)

The schedule for each Program Administration used to indicate the valid testing days for a Program Administration

Assessment – STAR =

California Standards Tests (CST) California Modified Assessment (CMA) California Alternate Performance Assessment (CAPA) Standards-based Tests in Spanish (STS)

– CAHSEE = CAHSEE– CELDT = CELDT


Key Terminology

DATA DEFINITIONS (Cont)

Test

• Content Area (Test Content Area) Subject content area to which the Test belongs(See Documentation)

Name of testing instrument (See Documentation)


LOADING ASSESSMENT RESULTS


Loading Assessment Results

AADS Requires …– Vendor Results Data

– Student Profile Data Enrollment (SENR) Student Information (SINF) Student Program (SPRG)

LOAD PROCESS

CALPADSPortal

CALPADS Staging

Area

CALPADS Integration

Hub (ZIS, FTP,

ESB)

LEA Systems

Local CALPADS

ZIS

State Users

Other Systems

LEA

Assessment Vendors

CALPADS Assessment & Accountability

Data Store(AADS)

CALPADS Operational Data Store Replicated

(ODSR)

CALPADS State

Reporting Snapshots

CALPADS Operational Data Store

(ODS)A

BC

D

Reporting



STUDENT PROFILE DATA LOAD

What data does CALPADS replicate into ODSR from ODS and when?

Student Profile data is replicated from the ODS into the ODSR in real-time as transactions in the ODS occur. The data in the ODSR is a subset of ODS data from all enrollments for currently and previously enrolled students.



STUDENT PROFILE DATA LOAD (Cont)

When does ODSR data get replicated into AADS for reporting?

ODSR Student Profile data for a Program Administration gets replicated into AADS as a part of the Suspense Resolution Process by LEAs and CDE, as part of the Suspense Process evaluation of data (nightly and when a Snapshot is taken during a load or manually initiated by CDE i.e. update & freeze of reporting data for a Program Administration). The data include:

− For Test Participants without data in suspense: Data that were valid at the time of testing (i.e. SSID Assessment Data Records)

− For Non-Test Participants: Data that were valid during the time of testing (i.e. LEA Testing Window) Note: occurs only during Snapshot Process

− For currently enrolled students without data in suspense: Current data

SSID Assessment Data Records are the ODSR Student Profile Records (SENR, SINF, SPRG) valid at the time of testing that are used in the Suspense Process to validate a Testing Participant’s enrollment.

Test Participants are students with results records for a Program Administration (both valid and invalid results)



STUDENT PROFILE DATA LOAD (Cont)

What data does CALPADS snapshot in AADS?

data replicated into AADS from ODSR for the Program Administration.

This is the data used to report results for Program Administrations, as well as to report results/lack of results for currently enrolled students.

The Snapshot process updates and freezes both the vendor results data for the Program Administration and all of the Student Profile



LOADING RULES SUMMARY

Program Load Name Initial Load Rules

STAR P1 P2 P3

Overwrite previous load for academic year. Evaluate for Suspense

If previous record exists for UIN (student Unique Identification Number), reevaluate for suspense, except for

– LEA or CDE Resolved. Do not re-match demo

Post P3-LEA(s) No overwrite of P3. Load & evaluate for Suspense If previous record exists for UIN, reevaluate for suspense,

except for– LEA or CDE Resolved. Do not re-match demo.

Note: Multiple loads are possible for different sets of LEA(s).

• Load Name is used for Snapshot Name and Extract Name

When new data are loaded, suspense records that have been resolved stay resolved, unless enrollment can no longer be validated



LOADING RULES SUMMARY (Cont)

Program Load Name Initial Load Rules

CAHSEE MONTHLY-ADMIN NAME QUARTERLY 1-4 EOY

No overwrite of previous load for academic year. Load & evaluate for suspense

If SSID is in a subsequent load for the same Test &– Same ADMIN, replace previous record & reevaluate for

suspense, except for LEA or CDE Resolved. Do not re-match demo.

– Different ADMIN, save each record & evaluate new for suspense

CELDT MONTHLY-ADMIN NAME PRE-DRM POST-DRM

No overwrite of previous load for academic year. Load & evaluate for suspense

If SSID is in a subsequent load for the same Test &– Same ADMIN, replace previous record & reevaluate for

suspense, except for LEA or CDE Resolved. Do not re-match demo.

– Different ADMIN, save each record & evaluate new for suspense

Multiple ADMINS are stored for same test (CAHSEE & CELDT)


SUSPENSE RESOLUTION


Understanding Suspense Data

LEA SUSPENSE REASONS & RESOLUTIONS (Cont.)

GENERAL GUIDELINES for MATCHING

When results record SSID correct & CALPADS SSID incorrect:– Always correct CALPADS first

When results record demo correct & CALPADS demo incorrect:– If update CALPADS first & match after …

Subsequent match (online or batch) will provide correct CALPADS data as report data immediately or

Nightly Suspense Process will provide correct CALPADS data as report data the next day after CALPADS corrected

– If match first & correct CALPADS after … Incorrect CALPADS data will be provided as report data immediately

and Nightly Suspense Process will provide correct CALPADS report data the next

day after CALPADS corrected


Understanding Suspense Data

LEA SUSPENSE REASONS & RESOLUTIONS (Cont.)

GENERAL GUIDELINES for MATCHING (Cont)

When results record demo incorrect & CALPADS demo correct:– If correct vendor results data first …

Subsequent load of vendor data will cause a re-evaluation of the suspense record, clearing the suspense automatically, and present the correct CALPADS data as report data immediately

– If match first & correct vendor results data after … Correct CALPADS data will be provided as report data immediately

and Subsequent load of vendor data will not impact report data because CALPADS will not

identify a change in demo data

When CALPADS enrollment is correct for time of testing, but can’t be located– Contact the Service Desk

When CALPADS enrollment is incorrect for time of testing and can’t be located– CALPADS data must be corrected before a match can be made

When results record does not contain SSID Administration Date– Contact the vendor


DATA VALIDATION & REPORTING DATA


Data Validation & Reporting Data

DATA VALIDATION PROCESS - SUMMARY

Suspense Resolution Process (online & batch)– Runs limited validation– If matched, removes suspense and copies ODSR Test Participant Data to

AADS as transaction completes Suspense Process – Nightly

– For suspense records, runs full validation on those for which ODSR Test Participant Data has changed.

If new match, removes suspense and copies ODSR Test Participant Data to AADS

– For matched records, runs validation on those for which ODSR Test Participant Data has changed. (For Batch matched record, runs full validation; for LEA/CDE matched record, runs limited validation)

If still match, refreshes AADS data If no longer match, places in suspense

– For currently enrolled students, copies to AADS new data and refreshes existing if ODSR data has changed


DATA VALIDATION PROCESS - SUMMARY (Cont)

Suspense Process - In Load – For new record, runs full validation

If matched, copies ODSR Test Participant Data to AADS for Admin & Load If no match, places in suspense for Admin & Load

– For all matched records and suspense records, runs validation on those for which ODSR Test Participant Data has changed for Admin in record and on those for which the vendor results have changed. (For Batch matched and suspense record, runs full validation; For LEA/CDE matched, runs limited validation)

If still matches, refreshes AADS data for Admin & Load (Note: If multiple Admins are loaded for a Program and a load for one of the Admins includes a record for a previous Admin, the most current information about the Admin is reported when the information on the report is not related to a Load)

If no match, places in suspense for Admin & Load

– For currently enrolled students, copies to AADS new data and refreshes existing if ODSR data has changed



DATA VALIDATION PROCESS - SUMMARY (Cont)

Snapshot Process – In Load and Standalone– Always includes the Suspense Process– For Non-Test Participants, copies to AADS new data and refreshes

existing if ODSR data valid during testing has changed – Freezes AADS data (i.e. CALPADS Student Profile Data for Test

Participants with matched vendor assessment result record, as well as Student Profile Data for Non-Test Participants and currently enrolled student data). Note that Test Participants with suspense data are not included.


Definitions• Full data validation = SSID, demo (gender, birth date year) enrollment, vendor Administration Date• Limited data validation = enrollment, vendor Administration Date. SSID & demo mismatch are allowed• Test Participant = Students with results records for a Program Administration• Non Test Participant = Students who were enrolled during the time of testing for a Program

Administration (i.e. during LEA Testing Window) who do not have a results record for the Program Administration.


REPORTING DATA COPIED TO AADS - SUMMARY

For Test Participant– data valid at the time of testing (i.e. SSID Administration Data records)

For Non-Test Participant– data valid during the time of testing (i.e. most current data during the LEA

Testing Window)

For Currently Enrolled Student– data valid for the most current enrollment at the school being reported




Assessments– Load

Student profile for Test Participants – Load – Suspense Resolution Process (online & batch update immediately)– Suspense Nightly Process

Student profile for Non-Test Participant (which impacts enrollment on reports by ADMIN)

– Load

Student profile for Current Students (which impacts online and enrollment reports)

– Load– Suspense Nightly process

REPORT DATA UPDATES


MANAGING SUSPENSE DATA - BATCH


Annual

Managing Suspense Data - Batch

Suspense data can also be managed in Batch

Suspense Resolution Records can be downloaded from both …

– Reports and Extracts > Assessment Extracts– Manage Submissions > Student List

ACCESSING



1. Suspense Resolution Extract records are downloaded for a specific Program and Administration

PROCESS

2. Corrections are made offline to the extracted records

3. Suspense Resolution Response records are uploaded, using Manage Submissions from the Assessment Menu



PROCESS (Cont)

4. Uploaded response records are edited to confirm that they have been formed correctly and are evaluated, with the same logic used in the online, to determine if the suspense has been resolved

– Status of upload is viewable in Submission Status

– Details of upload errors are available on “Rejected Records” screen

GERR0001 Required Field Missing

GERR0001 Required Field Missing

ABBOTT, Jay 1231234567

7824435891Baba, Grey



BATCH ERRORSError # Error Name Error Description Severity

SUSP0001 Invalid Submitter Reporting LEA must equal institution identifier of submitter User ID Fatal

SUSP0002 Invalid Reporting School School must be a valid code in CDS and have an active CALPADS reporting relationship with the Reporting LEA

Fatal

SUSP0007 Invalid SSID SSID must be a valid CALPADS SSID in the ODS Fatal

SUSP0069 Mismatch of SSID Demographics

Student Gender Code and Student Date of Birth Year must equal existing ODS Student Gender Code and Student Date of Birth Year

Fatal

SUSP0234 Multiple Suspense Resolution Records for Participant Record ID

Only one Suspense Resolution record per Participant Record ID may be submitted.

Fatal

SUSP0235 SSID already matched for Administration

A results record for this SSID exists for this LEA with different UIN for the Administration of the Suspense Record

Warning

SUSP0236 Fields differ from Extract Record

The content of the following fields differs from the Extract Record content: Record Type Code, Program, Administration Code, Testing Validation Date, Suspense Code

Fatal

SUSP0237 Participant Record ID not found

A record with the specified Participant ID can not be found in the AADS Participants table

Fatal

SUSP0238 Invalid Test Date Assessment Record has invalid or null Test Date Fatal

SUSP0239 SSID in Participant record is different from SSID in Match record

SSID in Participant record is different from SSID in Match record Warning

GERR0001 Required Field Missing A required field for the submitted Records Type is missing Fatal

FRMT0001 Wrong field length Field length must match CALPADS File Specifications (CFS) Fatal



SUSPENSE RESOLUTION EXTRACT RECORD



SUSPENSE RESOLUTION EXTRACT RECORD (Cont)



SUSPENSE RESOLUTION RESPONSE RECORD



SUSPENSE RESOLUTION RESPONSE RECORD (Cont)



SUSPENSE RESOLUTION RESPONSE RECORD (Cont)

Documents

ASSESSMENTS & ACCOUNTABILITY