A Deep Dive Into the Online Reporting Suite

(ORS) for Interim

Assessments

August 21, 2018

1

Housekeeping

If you have problems hearing the presentation, dial in on a telephone. Dial-in information:

• in the Event Info tab at the top-left of the screen

• in your registration email (messenger@webex.com)

Dialing in for audio is recommended for best sound quality.

All attendees’ lines have been muted due to the high number of participants.

For questions or comments, use the “Q&A” function.

2

ObjectivesToday’s session will cover the following topics:

• Reporting Overview

• Online Reporting Suite (ORS)

• Probability Scores

• Interim Data File

• Interpreting Results

3

Reporting Overview

Reporting

• New Online Reporting Suite (ORS)

• Dynamic, real-time reporting system

• District, campus, and student level

• User-friendly graphs and customization

• Filterable results

• Probability results reported in the STAAR Assessment Management

System

• District data file

5

Reporting

Results Key

6

Reporting

Results Key

7

Reporting

How Results are Calculated

8

Accountability

• Optional assessments

• NOT tied to accountability

9

Online Reporting Suite (ORS)

Online Reporting Suite (ORS)

• Dynamic real-time reporting

• Immediately after test submission, ORS will display the following results

• District

• Campus

• Student

• Results filterable by student demographics

• First release of results available beginning August 31, 2018

11

Online Reporting Suite (ORS)

• Results available from the primary (first) completed attempt only

• Available immediately after test is completed

• Allow five minutes for processing

12

Expanded Online Test Administrator & Teacher Roles

• For the interim assessment, the online test administrator and teacher roles have been expanded.

• The intent of expanding these roles is to give districts an option to allow educators to view their students’ results in ORS.

• Educators who are assigned these roles by the district in the Assessment Management System will be able to view interim results for all students within their campus in the Assessment Management System and ORS.

• See the User Roles & Permissions Matrix for details on each user role.

13

Expanded Online Test Administrator & Teacher Roles

• Educators will have access to demographic information for students registered in their campus via Students > View & Edit on the Profile tab.

• Educators will be able to access the attempts history pop-up under the Manage Online Testing tab in the Assessment Management System.

• Allows user to view probability scores for all attempts

14

Expanded Online Test Administrator & Teacher Roles

• Educators will NOT be able to view any STAAR data or view any student information beyond the campus level.

• No access to summative standard reports under Reports > Results in the Assessment Management System

• No access to the Scores tab in the individual student profile where STAAR Report Cards are posted

15

Expanded Online Test Administrator & Teacher Roles

• Pursuant to the Family Educational Rights and Privacy Act (FERPA), the disclosure of personally identifiable information from education records is permissible without written consent of the parent or eligible student if the disclosure is to other school officials, including teachers, within the local education agency (LEA) whom the LEA has determined to have legitimate educational interests. Please note FERPA compliance is required when assigning roles to access the online reporting system.

16

Navigation

• From the Assessment Management System, select Reports > Online Reporting from the top dropdown menu.

17

Navigation

• From the Online Reporting Suite, select the Organization Reports tab to view organizational reports (district or campus).

18

Navigation

• From the Online Reporting Suite, select the Student Reports tab to view student level reports.

19

Navigation

• From the Online Reporting Suite, select Back to STAAR Assessment Management System from the top dropdown menu.

20

Find Interim Report

• Opens at the user’s organizational level

• Users can then navigate to campus-level and student-level reports as permitted by their user role

• Users with multiple user roles can view all their organizations

• Use the Organization Reports and Student Report tabs to toggle between organization level and student level

21

Find Interim Report

1. From the Assessment Management System, navigate to Reports > Online Reporting.

OR

2. Click the View Report icon on the Manage Session page.

The Find Interim Report appears, displaying the summary report at the user’s organizational level.

3. Select an Organization (if different from the organization displayed), Grade, and Subject.

4. Click the Find button.

5. Click a report link under Report Name to view.

22

Find Interim Report

23

Find Interim Report

• Available at district, campus, and student levels

• Summary view and expanded list views at each level

• Includes clickable graphics and other elements that expand on information

• View individual student reports at any time by using the Student Report tab

24

District-Level Reports

• Campus results in a bar chart (lowest to highest mean score)

• District results in pie chart with aggregate results

• Segmented into campuses by the following criteria

• Top 25%

• Bottom 25%

• Middle 50%

• Click Students Tested button to view a list of all campuses with data.

• Organized lowest to highest mean score

• Number of students tested in grade and subject listed

25

District-Level Reports

26

Campus-Level Reports

• Open at summary report displaying:

• Student results in bar chart (lowest to highest mean)

• Campus pie chart with aggregate results

• Segmented into groups of students by the following criteria

Top 25%

Bottom 25%

Middle 50%

• Click Students Tested button to view a list of all students (listed alphabetically) with test results including the following:

• Score

• Probabilities

• Reporting Category Scores

27

Campus-Level Reports

28

Campus-Level Reports

29

Student-Level Reports

• Allow educators to perform item analysis based on individual student performance

• Display the following:

• Overall score as a number and plotted on a scale

• Performance in each reporting category

• Approaches, Meets, and Masters probabilities

• TEKS student expectation for each test question

• Student’s response to each test question

• A pop-up window that shows the test question

30

Student-Level Reports

• Click an Info icon in “Reporting Category Performance” section or a link in the “Student Expectation” column to review student expectation information and reporting categories for each question.

• Click the View icon to see the item, student response, and correct response.

31

Student-Level Reports

32

Probability Scores

Probability Scores

• Assessment Management System will display probability results

• Available immediately after testing

• Located in the Manage Online Sessions page in the View Attempts icon

• Show percentage probability of achieving the following categories on STAAR

• Approaches

• Meets

• Masters

• Includes results from all testing attempts

34Copyright © 2018 by Educational Testing Service. All rights reserved. ETS, the ETS logo and MEASURING THE POWER OF LEARNING are registered trademarks of Educational Testing Service (ETS). 34728

Probability Scores

1. From the top navigation bar in the Assessment Management System, select Online Testing, then Manage Online Testing.

2. Select an interim administration (“INT”).

3. Select a Subject, Testing Grade, and District or Campus.

4. Click the SEARCH button.

5. In the Groups in Session list, click the View icon next to the appropriate testing group.

6. Results will appear on the Student Attempt History pop-up.

35

Probability Scores

Student Attempt History

36

Interim Data File

Interim Data File

• A preliminary data file will be available in early January.

• A final data file will be available two weeks after the test window closes.

• Available in the Assessment Management System from Reports > Results

• Process for downloading similar to summative

• Choose interim test administration (“INT”)

38

Interim Data File

• Fixed-width data file

• Refer to Appendix B: Interim Data File Form for how to find and interpret information

• Includes the following:

• Student responses to test items

• Which Section 2 the student took (higher difficulty, medium difficulty, lower difficulty)

39

Interpreting Results

Interpreting ResultsHow should districts interpret results?

• Interim Assessments Results Guide will be posted this fall.

• https://tea.texas.gov/student.assessment/IA

• Provides information on interpreting each type of reporting information including:

1. Student’s Scale Score

2. Student’s Probability of Achieving Each Performance Level

3. Student’s Reporting Category Results

4. Student’s Performance on Each Item

• Provides sample remediation strategies

41

Training

Interim AssessmentsUpcoming Webinar Trainings

Starting early, get ready for online testing (interim and summative)

• New secure browser needed

• 7/23 Bi-Weekly Email

• Online Testing > Secure Browsers

• Thursday, August 23, 2018

• 1:00 p.m.–2:00 p.m. CT

Registration links posted to https://www.texasassessment.com/administrators/training

43

Questions

44

Questions?

45

Customer Support

Texas Assessment Support Center

• Monday–Friday

• 8:00 a.m.–5:00 p.m. CT

• 855-333-7770

• STAAREOC@ets.org or STAAR3-8@ets.org

46