Performance Reporting Under WIA Title 1B Candice Graham-Young ETA Performance Accountability Team

Performance Performance Reporting Under WIA Reporting Under WIA

Title 1BTitle 1BCandice Graham-Young

ETA Performance Accountability Team

WIA Performance TeamWIA Performance Team

Quarterly Report performance information

used for management purposes: budgeting and GPRA

includes information about statewide outcomes on the key indicators of performance and customer satisfaction

captures number of participants and exiters due 45 days after the end of the quarter


Quarterly Reporting

• submitted electronically from States• States responsible for verification and

validation• Regions responsible for

second validation

Submission process…


State Role

• develop standard operating procedures for data collection and handling to ensure the quality and integrity of data

• data verification equals checking accuracy• data validation equals checking reasonableness• examples of procedures for data collection are:

checking accuracy against original sourceperforming logical checks of the datarandom call-backs to participantshaving trained staff member evaluate data collection efforts

Verifying and validating data…


Regional Role

• ensure timeliness of reports• ensure number of quarters submitted conforms to Table A of the quarterly report instructions• note discrepancies in comment section• ensure reasonableness of data• become familiar with standard operating procedures which States are required to have in place

Validating reports…


fields added to collect response rates

fields added to show period of time covered by data for each measure

performance is automatically calculated

additional edits added to for data validation purposes (e.g., can’t have more exiters than registrants)

Changes to Quarterly Report Form


Common Reporting Problems

reporting correct quarters simple oversights, like entering

numerators and denominators in wrong locations

minor programming errors in edit checks

entering performance outcomes that did not match the numerator divided by the denominator


Amount of Data Reporte

d by States

Third Quarter Report


Third Quarter Performance




basis for awarding incentives or applying sanctions

includes information about negotiated levels, state and local outcomes on the key indicators of performance, customer satisfaction and state measures

numerators and denominators allow for aggregation of data

submitted electronically

Annual Report


local performance status of state evaluations comparison of results of those who received

training and those who did not outcomes for special populations other outcomes listed in the Act…

training-related employment wages at entry into employment cost of activities cost relative to impact of activities 12 month retention and earnings

Required Items for the Annual Report


Annual Report to StakeholdersWhat our customers have to

say about usTAX PAYERSappreciate increased efficiency

JOB SEEKERS reaping the

benefits from newly integrated

services

EMPLOYERSobtaining qualified

employees

1st Qtr 2nd Qtr 3rd Qtr 4th Qtr

Annual Report Format Varies

0

10

20

30

40

50

60

1st Qtr 2nd Qtr 3rd Qtr 4th Qtr

Region 1Region 2Region 3


though certain information mandated by the legislation must be contained in the report, the approach the State uses to tell its story is up to the State

States are encouraged to design reports that make extensive use of graphics and other communication devices

States may want to include “success stories”

Annual ReportState Flexibility


Annual Report

Required portions– Narrative Section– Tabular section– Designed to meet Sec. 136(d)


Annual Report

Optional portions (suggested)- Governor’s Comments

- Financial (Narrative & Graphic)

- Organizational Chart

- Resources

- Graphics

- Success Stories


Annual ReportNarrative Section

Discuss the cost of workforce investment activities relative to the effect of the activities on the performance of participants– Sec. 136(D)(2)(C)

Describe the State’s evaluations of workforce investment activities, including:– The evaluation's methodology; and– The timing of feedback and deliverables.– Sec. 136(d)(1)


Annual Report--Tabular Section

Financial Statement Example


Annual ReportTabular Section

State performance measures Sec. 136(d)(1)

– Numerator and denominator• Allows aggregation over states

– Negotiated levels

– Exclude:• Self-service & informational activities only

• National Emergency Grants

• Statewide activities excluded from performance

– Source for determining incentives & sanctions



Comparison of results of those who received training and those who did not Sec. 136(d)(2)(E)

Outcomes for special populations 136(d)(2)(F)

Other outcomes listed in the legislation– Training-related employment 136(d)(2)(A)

– Wages at entry to employment 136(d)(2)(B)

– 12-month retention and earnings 136(d)(2)(D)

– Cost of activities 136(d)(2)(C)

Local performance 136(d)(1)


Annual Report--Tabular SectionTable A - Workforce Investment Act Customer Satisfaction Results

Table A - Workforce Investment Act Customer Satisfaction Results

Customer Satisfaction NegotiatedPerf Level

ActualPerf Level -

Am Custr SatIndex

Number of CustSurveyed

Number ofCustomers

Eligible for The Survey

Program Participants

Employers

# surveyed = # with completed interviews Eligible for survey (the sample frame) =

– Participants: program year exiters– Employers: service completed in program year

Sec. 136(d)(1)


Annual Report--Tabular Section 2Table B - Adult Program Results At-A-Glance

Sec. 136(d)(1)


Annual Report--Tabular Section 3Table C - Outcomes for Adult Special Populations

Sec. 136(d)(2)(F)


Annual Report--Tabular SectionTable D - Other Outcome Information for the Adult

Program Table D - Other Outcome Information for the Adult Program

Reported Information Individuals WhoReceived Training

Services

Individuals WhoReceived Only Core

and IntensiveServices

NUM NUMEntered Employment Rate

DEN DEN

NUM NUMEmployment Retention Rate

DEN DEN

NUM NUMEarnings Change in SixMonths DEN DEN

NUM NUMEmployment And CredentialRate DEN DEN

• Does not include individuals who received only informational or self-service core activities• Sec. 136(d)(2)(E)



Table E - Dislocated Worker Program Results At-A-Glance– Performance Measures 136(d)(1)

Table F - Outcomes for Dislocated Worker Special

Populations– Performance Measures for:

• Veterans • Displaced Homemakers• Individuals With Disabilities• Older Individuals

– Sec. 136(d)(2)(F)



Table G - Other Outcome Information for the Dislocated Worker Program– Performance Measures for

• Individuals Who Received Training Services

• Individuals Who Received Only Core and Intensive Services

– Sec. 136(d)(2)(E)



Table H - Older Youth Results At-A-Glance– Performance Measures 136(d)(1)

Table I - Outcomes for Older Youth Special

Populations– Performance Measures for:

• Public Assistance Recipients• Veterans• Individuals With Disabilities• Out-of-School Youth

– Sec. 136(d)(2)(F)



Table J - Younger Youth Results At-A-Glance– Performance Measures 136(d)(1)

Table K - Outcomes for Younger Youth Special Population– Performance Measures for:

• Public Assistance Recipients

• Individuals With Disabilities

• Out-of-School Youth– Sec. 136(d)(2)(F)


Annual Report--Tabular SectionTable L - Other Reported Information

Table L - Other Reported Information

12 MonthEmployment

RetentionRate

12 Mo. EarningsChange

(Adults and Older

Youth)

or

12 Mo. EarningsReplacement

(Dislocated Workers)

Placements forParticipants inNontraditionalEmployment

Wages At EntryInto

EmploymentFor Those

Individuals WhoEntered

UnsubsidizedEmployment

Entry IntoUnsubsidizedEmployment

Related to theTraining Received

of Those WhoCompleted

Training Services

NUM NUM NUM NUM NUMAdults

DEN DEN DEN DEN DEN

NUM NUM NUM NUM NUMDWs

DEN DEN DEN DEN DEN

NUM NUM NUM NUM NUMOlderYouth DEN DEN DEN DEN DEN

Sec. 136(d)(2)(A), 136(d)(2)(B), 136(d)(2)(D)


Annual Report--Tabular SectionTable M - Participation Levels

Table M - Participation Levels

Total Participants Served Total Exiters

Adults

Dislocated Workers

Older Youth

Younger Youth


Annual Report--Tabular SectionTable N - Cost of Program Activities

Su

pp

lem

ents

fin

anci

al r

epor

t

Table N - Cost of Program Activities

Total Federal Outlays(Program Only)

Adults

Dislocated Workers

Youth

Rapid Response(up to 25%) ' 134 (a) (1) (A)

Total Federal Outlays(Include program costs. Also

include administrative costs asappropriate)

Statewide RequiredActivities (Up to 15%) ' 134 (a) (2) (B)

Program Activity Description Total Federal Outlays(Program Only)

StatewideAllowable Activities ' 134 (a) (3)

Total PY Programmatic Federal Outlays


Annual Report--Tabular SectionTable O - Local Performance

Table O - Local Performance (Include This Chart for Each Local Area In The State)

Adults

Dislocated Workers

Older Youth

Local Area Name

Total Participants Served Younger Youth

Adults

Dislocated Workers

Older Youth

ETA Assigned #

Total Exiters Younger Youth

NegotiatedPerformance

Level

ActualPerformance Level

Program Participants

Customer Satisfaction Employers

Adults

Dislocated Workers

Entered Employment Rate Older Youth

Adults

Dislocated Workers

Older Youth

Retention Rate Younger Youth

Adults

Dislocated WorkersEarnings Change/EarningsReplacement in Six Months Older Youth

Adults

Dislocated Workers

Older Youth

Credential/Diploma Rate Younger Youth

Skill Attainment Rate Younger Youth

Description of Other State Indicators of Performance (WIA ' 136(d)(1) (Insert additional rows if there are more than two AOtherState Indicators of Performance@)

Not Met Met Exceeded

Overall Status of Local Performance

A table for each local area

Sec. 136(d)(2)(C)


Time Periods to be Reported--Same as used

for performance indicators:

– Measures Based on Wage Records

• Adults, DWs, older youth, younger youth retention

• Exiters first quarter of PY and last 3 quarters of

previous PY

– ‘Real-time’ measures

• Youth goal attainment, Youth diploma attainment,

Customer satisfaction

• Program Year

Annual Report--Time Periods


ETA Performance TeamETA Performance Team

Consultant services providedConsultant services provided by…by…

Mr. Rich West [email protected] Social Policy Research Associates

Phone Phone Fax Fax 202.693.3031202.693.3031 202.693.3229202.693.3229

Mr. Bill Rabung, Team Leader [email protected]

Ms. Jacqui Shoholm [email protected]

Mr. Bradley Sickles [email protected]

Ms. Candice Graham-Young [email protected]

Documents

Performance Reporting Under WIA Title 1B Candice Graham-Young ETA Performance Accountability Team