Title II Federal Pass Rate Reporting for AY 2015/16 For ... · Title 2 record (last name, first name, SSN, DOB, ETS Testing ID) are different than the student’s test score record

Confidential and Proprietary. Copyright © 2016 Educational Testing Service. All rights reserved.

Title II Federal Pass Rate Reporting for AY 2015/16

For 2017 IHE Report Cards

Presented by:

Sharen Virgin

Confidential and Proprietary. Copyright © 2016 Educational Testing Service. All rights reserved.Confidential and Proprietary. Copyright © 2016 Educational Testing Service. All rights reserved.

Webinar Agenda

• Find Help and the User’s Guide

• Title II User Accounts

• ETS Responsibilities and Final Reports

• IHE Responsibilities

• Entering Student Data, 6 Steps to Accurate Reporting

• Follow Up Reporting

• Resolution Period

• Important Dates

2


From Client Services Home Page, choose

All Help Documents in one place.

Guides / Webinar Links / FAQs

Help Documentshttps://tlcs.ets.org


Website User’s Guide


TITLE II USER ACCOUNTS: PRIMARY AND SECONDARY

5


User Accounts

Each institution must have 1 primary user and can have up to 3 secondary users.

ETS administrator approves primary user; primary user approves, removes, and manages secondary user accounts.

6


Primary User

7

Primary user Manages Secondary Accounts


Primary User

8


ETS RESPONSIBILITIES

ANDFINAL REPORTS

9


ETS Responsibilities

• Match students to test scores, apply state licensure requirements and calculate pass rates on tests

– Pass rate calculation: #passers / #takers• Ex: 18 passers ÷ 19 takers = 95% pass rate

• Prepare formatted pass rate reports for Program review

– Summary Pass Rate Report

– Single Assessment Pass Rate Report

10


Summary Report

11

Number of completers taking one or more assessments within their

area of specialization. In order to count as a pass, the student must

pass all tests taken for at least one license area.


Assessment Report

12

In cases where there are less than ten students taking a test, the number

passing and pass rate are not reported.


ETS Prepares flat data files for submission to Westat for inclusion in Section III of the IHE Report Card

Assessment:

Summary:

13


PROGRAM (IHE) RESPONSIBILITIES

14


• Submit a list of students enrolled between 9/1/2015 and 8/31/2016

• Assign a program type, group, and license/certification area to each student

• Verify test match results to student records

• Review final pass rate reports

• Teacher Preparation programs must submit an Institutional and Program Report Card on the Quality of Teacher Preparation (IHE Report Card) with Westat (IPRC) by April 30 each year

15

IHE Responsibilities


Entering Student Data, 6 Steps to Accurate Reporting

16


ENTERING STUDENT DATASTEP 1: IDENTIFY AND GROUP

17


Students to be Included

• All students officially enrolled in your program between 9/1/2015 and 8/31/2016

– Note: students who officially drop from your program prior to 8/31/2016 should not be included

• Initial certification only

• Classroom teachers only

– No administrators, counselors, nurses, etc.

• Regardless of testing status

18


Each Student Assigned to 1 of 3 Groups

• All students currently enrolled in your teacher preparation program must be counted in one of three groups:

• Group 1 Enrolled students who have completed all nonclinical courses (aka Enrolled, All But Clinical)

• Group 2 Other enrolled students

• Group 3 Program completers


Exception to Group 1

• If your program has coursework during or after student teaching, you will not have students in Group 1, Enrolled, All But Clinical

20

Enrolled, all but clinical


Two Prior Year Groups, 4 & 5• Group 4

– Program completers, prior year, 2014/15

• Group 5

– Program completers, 2 years prior, 2013/14

21

Group 1

Group 3

Group 2

Group 5

Group 4


One Possible Group: 6

22

Group 6When two or more completer groups have fewer than ten completers, data is combined across the three completer years into an additional completer group for the Summary pass rate report only

Group 6


Why No Pass Rates??

23

Rule of 10: When there are less than 10 takers in a group, no pass rate will be calculated and the number passing is not noted for confidentiality reasons. This occurs on both Summary and Assessment Reports.


ENTERING STUDENT DATASTEP 2: PROGRAM TYPES

24


Student Program Types

• Traditional Route within an institution– R on upload sheet

• Alternate Route within an institution– A on upload sheet

• ARO: Alternate Route Organization

outside of an institution– O on upload sheet

25

State’s Department of Education designates program type for each institution and

organization based on approval status.


ENTERING STUDENT DATASTEP 3: CERTIFICATION AREA

26


Licensure or Certification Area

• Completer and All But Clinical must be assigned a license

• Other Enrolled should be assigned a license if possible, but can be entered as “undeclared”

27


ENTERING STUDENT DATASTEP 4: EDITING RECORDS

28


Update Rolled Over Records

• Delete students who left your program (this should be done first)

• Update students’ group

• Update by use of upload sheet

• Export student list

– Live Demo

29


Update Rolled Over Records

• Change where necessary:

– Group

– Program

– Certification area

• Live Demo

30


ENTERING STUDENT DATAPART 5: NEW STUDENTS

31


Enter New Students

• Enter by upload sheet or individually through the website

• Information required– First and last name

– DOB and/or SSN

– State

– Group

– Program type

– Licensure area

• Live Demo32


ENTERING STUDENT DATASTEP 6: MATCH VERIFICATION

33


Matching Students to Scores

• Match runs occur 3 times weekly (M/W/F) during overnight hours to match scores in Praxis test database to student records in Title II.

34


Match Criteria

• Each of the 5 match criteria includes a match to the Last Name or an alternate last name. Up to 4 alternate last names can be added to a record.

– Last Name (or alternate), First Name, Praxis ID

– Last Name (or alternate), SSN, DOB

– Last Name (or alternate), SSN

– Last Name (or alternate), First name, DOB

– Last Name (or alternate), Praxis ID, DOB

35


Match Criteria, cont’d

36

The last name must be an exact match to what is on the

student’s score record.

Example: If Brown-Jones is on the score record, the

scores will not match to the below Title II record.


Match Verification

• Match verification is an important part of Title II pass rate reporting

• Failing to verify your matches could results in the need to edit during the resolution period for a fee.

37


Match Verification, cont’d

• ETS will match students to scores 3 times weekly (Mon, Wed, Fri)

• Students will show a match status

– Match not run (entered or edited since last match run)

– Not Found

– Found

– Matched to scores

38


Match Verification, cont’d

• Match results must be reviewed

• Edits can be made to improve match results

• New results should be confirmed and additional edits made where necessary

– Live Demo

39


Match Verification & License Rules

• A license rule is Boolean login and is used to determine the requirements of a license

• The license rule is applied when the data is extracted for the reports.

40


Match Verification

41

• ALL tests taken will match to student record if they are present in the license rule: ( ( ( 5623 OR 0623 ) OR ( 5624 OR 0624 ) ) AND( 0700 OR ( 5701 OR 0701 ) ) )

• The license rule is applied when the data is extracted to prepare reports.

and these tests will be dropped

When rule is applied, these tests will used in pass rates Not Passed 5623 PRINC LEARNING AND TEACHING 5-9

Passed 5624 PRINC LEARNING AND TEACHING 7-12

Not Passed 0700 AGRICULTURE (discontinued)

Passed 5701 AGRICULTURE


Same Test, Different Format (Related Tests)

Student list view

Student detail view


Applying the License Rulefor Non-Related Tests

43

( ( ( 0710 OR 5710 ) OR ( 5712 OR 0712 ) ) AND

( ( 0720 OR 5720 ) OR ( 5722 OR 0722 ) ) AND

( ( 0730 OR 5730 ) OR ( 5732 OR 0732 ) )

AND

( ( 5015 OR 0015 ) OR ( 5019 OR 0019 ) )

AND

( 5622 OR 0622 ) )


Check all Results

• Important to look at results in all match categories

– Not Found – may not have tested

– Found – may have tested outside window; review a-h list in Section 6

– Matched to scores – are they the expected scores?

• Live Demo

44


Resolution of Non-Matched Records (A-H Chart)

REASON RESOLUTION (where applicableA: Information entered in key fields in

Title 2 record (last name, first name, SSN,

DOB, ETS Testing ID) are different than the

student’s test score record

Review the student’s score report (either on file in your office or

through Test Taker Score Report service) to make sure that the

key fields of data entered into Title II match what is on the score

report.

B: The wrong license is listed for the

student

Confirm the student’s area of study with your preparation

program. Review a list of licenses for your state to make sure

you are listing the student with the correct license.

C: The student took a test not required by

the license listed

There is no resolution for this.

If a student took a test that is not a requirement of the licensure

for which your program trained him, there is nothing that can be

done on your part to resolve this; however, the student should

remain on your list as they might take the correct test at a later

date and the record will be updated at that time.

D: The student took the test outside of

the testing window (before 9/1/2011 or

after 8/31/2016, current reporting year)

There is no resolution for this.

Title II pass rate reports are not equal to state licensure.

Exceptions made by state licensing agencies are not factored into

pass rate reporting. If a student took a test one day outside of

the testing window, we will not include the score from that test

in the pass rate calculations.

E: The student took the test before its

effective date within your state

There is no resolution for this. To determine if this is the cause,

view the effective date for the test on the License Detail Page

F: The student took the test after its

expiration date in your state

There is no resolution for this. To determine if this is the cause,

view the expiration date for the test on the License Detail Page

G: The student has registered with us but

has not yet tested

There is no resolution to this. If the student is “other enrolled”

he may not have tested. If you cannot locate a score report for

the student in your files or Test Taker Score Reports, you have

done your due diligence and can assume this student has not yet

tested.

H: A student has an outstanding balance

with ETS which would prevent scores from

being available in the database.

We see this reason very seldom. If none of the above reasons fit

your student, contact us and we will check this for you.


Edits to Improve Match Results

• Confirm last name, DOB and SSN, with original score report (Test Taker Score Reports)

• If incorrect on report, still need to use that data

• Refer to Section 6 of the User’s Guide

• If all else fails, send email to [email protected]

46


FOLLOW UP REPORTINGON

PRIOR YEAR COMPLETERS

47


Follow up for Completers from2013/14 and 2014/15

• The purpose of follow up reporting is to capture any testing activity that has occurred since the close of the reporting window for the year a student is listed as a completer.

• Section 9 of the User’s Guide provides detailed information

48


Edits Allowed to Prior Year Completer Records

• change or delete an SSN or DOB

• address

• state

• add alternate last names

• change, add, or delete Praxis IDs

49


Edits Not Allowed to Prior Year Completer Records

• student group

• program type

• middle initial

• last name

50


RESOLUTION PERIOD

51


Resolution Period

The purpose of the resolution period is to allow programs the opportunity to review reports based on the student data provided and matched to scores the previous Fall. If it is believed that an error was made, an appeal should be submitted to ETS via email at [email protected].

52

mailto:[email protected]


Resolution Period

• The resolution period is 16 business days in length (2/17–3/10, 2017) and begins on the day the reports become available for your review.

• Changes to your data during this time will incur a fee of $1500.

• Section 11 of the User’s Guide provides detailed information

53


IMPORTANT DATES

54


ETS Website Dates

• September 6 through November 9, 2016 9/6/2016 Site opens for entry and edit of student data and test

score match verification

11/9/2016 Site closes to IHEs

• February 17 through March 10, 2017 2/17/2017 – Draft reports available for review; resolution period

begins

3/10/2017 – Resolution closes

• April 15, 2017 – Final reports sent to Westat for Section III of IHE Report Cards

https://tlcs.ets.org

https://tlcs.ets.org/


Student and TestDate Ranges

• Enrolled between

– 9/1/2015 and 8/31/2016

• Matched to scores from tests taken

– on or after 9/1/2011 for current year (15/16)

– on or after 9/1/2010 for 14/15 Completers

– on or after 9/1/2009 for 13/14 Completers


Students Reported& Matched to Tests


Concerns or Questions

• Email us at [email protected]

• Most issues require a careful review of the data. Please put your request in writing with sufficient information to allow us to investigate.

• Log in issues:

[email protected]

58

mailto:[email protected]