An Administrators Guide to Student Follow-Up and Data Quality Presented by: Dean Smith Lennox McLendon Kathi Polis

An Administrator’s Guide to Student Follow-Up and Data Quality

Presented by:Dean SmithLennox McLendonKathi Polis

McLendon and Polis 2

Training Objectives

To review NRS guidelines for determining what participant populations need to participate in follow-up surveys

To outline the seven-step process for conducting NRS-approved participant surveys

To examine strategies for training telephone interviewers


Training Objectives

To develop a local plan for conducting NRS-compliant follow up

To review and provide input into the development of data quality standards for local programs

To discuss and clarify MAERS-related data collection and reporting questions and issues


Introduction to Student Follow-Up

Requirements established by National Reporting System

Defines: Outcomes you must collectMethodologies for collecting themState reporting requirements


Core Measures

Educational Gains – completion of EFLsFollow-Up Measures

Entered employmentRetained/improved employmentEarned a secondary school diploma or GEDEntered postsecondary education or job

training


Two Follow-Up Methodologies

Data Match (recommended)Most cost and labor effective method

Student SurveyMichigan

2005 - 2006 = student survey2006 – 2007 = exploring data match for

employment and GED measures


What Participants to Survey

Only participants with follow-up goalsEnter, retain, improve employmentEnroll in postsecondaryEarn a GED/diploma

High school diploma completion can be verified through program records.

GED can be verified through local GED Examiner (with participant’s permission granted on Follow-Up Notice)


When to Survey for Program Year 2005 – 2006

Postsecondary, HS Diploma, GED Anytime prior to October 25, 2006 (for 2005- 2006 year)

Tips Postsecondary/job training – know the college’s

enrollment schedule to determine most appropriate time (e.g., mid-September for fall semester college enrollments)

GED – allow as much time as possible for participants to complete test and receive their test scores (e.g., mid-September for previous program year)


When to Survey – The Plot Thickens!

Participants with entered employment follow-up goals Timing is the challenge!

Must determine TWO outcomes

1. If they got a job by the 1st quarter after exit quarter AND

2. If they were still employed by the 3rd quarter after exit


When to Survey for 2005 - 2006

Exit Quarter Conduct Entered Employment Follow-Up

by end of:

Collect Retained/Improved

Employment by end of:

First QuarterJuly 1 – Sept. 30, 2005

Second Quarter Fourth Quarter

Second QuarterOct.1 – Dec. 31, 2005

Third Quarter First Quarter, Next Program Year

(July 1 – Sept. 2006)

Third QuarterJan. 1 – March 31, 2006

Fourth Quarter Not reported for 2005 – 2006

Fourth QuarterApril 1,2006 – June 30, 2006

First Quarter, Next Program Year

(July 1 – Sept. 2006)

Not reported for 2005 - 2006


Activity 1 – Match Game

Let’s see how well you do at matching the right information together.

Refer to Activity 1 Handout.


The Survey Process

Getting Ready:1. Select and train telephone interviewers.2. Determine your telephone survey schedule.Participant Intake:3. Inform participants of the survey.4. Ensure accurate contact information.Data Collection:5. Identify survey respondents.6. Conduct the telephone survey.7. Record the results.


Getting Ready: Step 1: Select and train telephone interviewers.

Selecting telephone interviewersMay be you, your teachers, volunteers,

other staffYou need folks who:

Speak clearlyUnderstand the program and reasons for follow-

upUnderstand the adult learner populationAre polite and courteousAre patient yet persistent ANDWho WANT to be telephone interviewers!


Getting Ready

Training telephone interviewersTraining should include

How to use the script, including what to say to participants to introduce the survey and get their cooperation

Ways to avoid refusalsHow to ask the survey questionsHow to record responsesHow to answer participant questions about the

survey.


Getting Ready

Training telephone interviewersGo over every question to ensure

interviewers understand: the purpose, what is being asked, and what responses are desired.

Include mock interviews and other practice.Manual – Appendix C: Training Strategies

for Telephone Interviewers


Program Survey

Appendices D1 – D6Standardized script for each follow-up

measure/goalMust follow script preciselyThree sections

1 – includes general participant information2 – relates to follow-up goal3 – requests participant feedback and

recommendations


Activity 2: Can We Talk?

Divide into pairs. One person will be the student; the other will play the role of interviewer.

For your designated follow-up goal, conduct the telephone interview. What was the result? Did you see any problems you might encounter? If so, what were they and how might you handle

them?


Step 2: Determine your telephone schedule.

Employment goals – quarterly Don’t forget

Participants with “enter employment” goals will need to be contacted twice (first for employment, second for job retention)

Postsecondary and GED goals Depends on college schedules, GED test dates,

etc. – You be the judge. All follow-up must be entered into MAERS no

later than October 25!


Step 3: Inform participants of the survey.

At intakeFollow-Up Notice – Appendix A

Explains follow-up processRequests their participationProvides the participant’s permission to be

contacted for follow-upProvides permission to verify GED completion

with state or local GED testing examinerGiven to all participants with follow-up goals

(or potential to convert to follow-up goals)


Step 4: Ensure accurate contact information.

Important to collect alternative contact information Addresses and phone numbers of friends, relatives,

etc. Use the Follow-Up Contact Information Form –

Appendix B Can input into MAERS’ Update Alternative

Contact screen (Appendix E) Encourage participants to keep their contact

information up-to-date


Step 5: Identify survey respondents.

CRITICAL – Input participant data into MAERS at least quarterly!Enables MAERS to generate a Follow-Up

Contact List Lists all participants that need to be

contacted and when.


Activity 3: Beginning Your Follow-Up Plan

Refer to the Student Follow Up PlanReview the questions for Steps 1 – 5.Complete the “Response Plan” column for

any items that you can answer right now.


Step 6: Conduct the telephone survey.

Ask for the individual by first name or nickname.

If teacher is not making the call, have follow-up staff state they are calling on behalf of the participant’s teacher.

Must use the Contact Log (Appendix F). Must attempt minimum of four contacts. No answer after four attempts – mark as a

non-respondent


Step 7: Record the results.

Record follow-up results in MAER’s Follow-Up Information screen (at least quarterly).

File program survey in participant’s paper file.Non-respondents – file blank survey

Adapt the Participant Follow-Up Checklist (Appendix G) for your use.


Activity 4: Completing Your Follow Up Plan

Refer to the Student Follow Up Plan.Complete the “Response Plan” column

for Steps 6 and 7.


Data Quality Standards

Improving the accuracy of our data U.S. Department of Education developed data

quality standards that states must address annually A guide to the policies, processes, and materials

that need to be in place for quality data Data foundation and structure Data collection and verification Data analysis and reporting Staff development

Levels of performance range from acceptable to exemplary



Michigan’s Draft Data Quality StandardsAdapted the national data quality standards

to fit local programsDraft form only – want your inputRecommendations will be forwarded to the

taskforces for further review and submission to State Office



Michigan’s Draft Data Quality Standards “Acceptable” level standards will go into

effect July 1, 2006. “Superior” level – July 1, 2007 “Exemplary” level – July 1, 2008

Training and technical assistance will be made available.



FormatThree columns1st column – states standard or process2nd column – provides examples of sample

verification3rd column – requires “yes” or “no” responseA few questions require brief narrative

response.



Process Program director submits completed checklist and

signed certification page to State Office by September 30, 2007 (for previous program year).

Written improvement plan for unmet standards Describe new policies or procedures you will put in place

to meet the standards, Identify barriers to moving to a higher quality level, and Describe the technical assistance needed to implement

the plan. DLEG will provide customized technical assistance

to help you.


Activity 5: Getting Your Input

Group 1 tables will review A and C.Group 2 tables will review B and D.Make a list of any concerns, questions,

or recommendations for the standards or verification samples.


Dean’s Corner

Let the games begin!!!!


Always willing to help…

Dean Smith [email protected]

Lennox McLendon [email protected]

Kathi Polis [email protected]


This project was developed by National Human This project was developed by National Human Resources Development, Inc. (NHRD) and the Resources Development, Inc. (NHRD) and the

National Adult Education Professional National Adult Education Professional Development Consortium in cooperation with the Development Consortium in cooperation with the

Michigan Department of Labor and Economic Michigan Department of Labor and Economic Growth and funded through a grant under Section Growth and funded through a grant under Section 222(a)(2) State Leadership Activities of the Adult 222(a)(2) State Leadership Activities of the Adult Education and Family Literacy Act, Title II of the Education and Family Literacy Act, Title II of the

Workforce Investment Act of 1998, amended. Workforce Investment Act of 1998, amended.

For more information visit:For more information visit:http:www.maepd.orghttp:www.maepd.org

Documents

An Administrators Guide to Student Follow-Up and Data Quality Presented by: Dean Smith Lennox McLendon Kathi Polis