Click to edit Master title style 1 Core Indicators, Reports, Negotiated Performance Targets, and Perkins Funds Regional Workshops Carl D Perkins (Perkins

Click to edit Master title style

1

Core Indicators, Reports,Negotiated Performance Targets,

and Perkins FundsRegional Workshops

Carl D Perkins (Perkins IV)Career and Technical Education Act

Chuck Wiseley

Spring 2008


2Regional Workshops

Agenda

• Why are we here – why do we care?

• Perkins IV– Legislative requirements

• State & Local requirements• Negotiating & Meeting Targets

– Not meeting targets & targeting funds

• Perkins funds - improving student success.

– Core Indicators, Reports, Performance Targets • Setting Targets & Perkins IV Local Application.


3Regional Workshops

Session 1 Goals

• Perkins funds - improving student success.

• What are the Rules?

• Negotiating & Meeting Targets– Tools for analysis– Not meeting targets & targeting funds


4Regional Workshops

Session 2 Goals & Objectives

• Update on development– Core Indicators, Reports, Performance Targets &

Perkins IV Local Application.Participants will:• View/Review Perkins IV Indicators & Reports

– what meaning can be derived– changing performance with Perkins funds

• Discuss and Set Proposed Targets – negotiated targets (who & when)– strategies for negotiating targets.– strategies for program improvement,– strategies for getting buy-in– Other necessary topics for inclusion


5

2004 Crystal Ball

• 109th Congress

• Some changes in Accountability– Strengthened

– Common measures• Wage gain

– Negotiated performance State & Local


6

Perkins IV

• Carl D. Perkins Career and Technical Education Improvement Act of 2006

• P.L. 109-270• August 12, 2006.• 2007-8 Transition Year• 2008-9 First Year under the Act

Regional Workshops


7

Negotiating Targets

• Perkins III– State negotiates targets USDE

• Perkins IV– State negotiates targets USDE

• Targets for 1 year and then 2 years (3rd & 5th)

– Locals either:• Accept those targets

• Negotiate local targets

– Included in Local Plans• Targets for next year

Regional Workshops


8

Sanctions in Law

• Perkins III– State

• Bundled – WIA & ESEA• Improvement plan• Two years below, Secretary MAY withhold funds

• Perkins IV– State & Locals:

• 90% of Perkins Targets• Improvement Plan• Year 2 no improvement, <90% for 3 years for any

single target– Feds or State SHALL provide technical assistance– MAY withhold funds (all or part)

Regional Workshops


9

CCC Sanction Plan:Meeting Performance Targets

• Application is Improvement Plan• Accountability Determination

– In compliance – at or above 90%• Expenditures need not be linked to the areas with the largest

performance gaps

– Focused Improvement Status - < 90%• Focused Improvement section • Year 2 – No Improvement: Diagnostic Study• Programs funded must address individual program performance gaps.

– Priority Focused Improvement – 3rd Yr < 90%• Diagnostic Study and Action Plan

Regional Workshops


10

Priority Focused Improvement

• Focused Improvement section

• Diagnostic Study

• Action Plan– Expected outcomes, – the Perkins and other funding dedicated to each

strategy, and – the implementation dates expected for each

strategy. Regional Workshops


11

Odds of being in:

• Focused Improvement? – Depends on how well the negotiation goes.

• Improving after Focused Improvement?– Depends on last year.

• Priority Focused Improvement? – Depends on how well the negotiation goes. – Unlikely to occur?

Regional Workshops


12

The Negotiation Process

• Negotiation Workbook

– Online

– Tables

– Charts

• Negotiation Worksheet

• Local Application


13Regional Workshops

Negotiation Tables

Mention Office 2003 v 2007

Problem? 90% x 55.6% = 50%

What has occurred?Outcomes & Diagnostics

Faupl


17

Local Application: District



Perkins IV funds

• Improving student success.– Nine Requirements– 18 Permissive Uses – Should address performance gaps

• Perkins IV Local Application.– Improvement Plans– Negotiating & Meeting Targets– Progressive Oversight, Improvement Plans, and

Risk Monitoring


20

With or Without Sanctions

Basic criteria for expending Perkins funds

• Expenditures must– Meet the purpose of the Act

– Be necessary and reasonable

• Expenditures may not be used for– General purposes

– Maintenance of existing programs


21

What federal rules apply?

• Carl D. Perkins Career and Technical Education Act (Perkins IV)

• EDGAR (Education Department General Administration Regulations)


22

What federal rules apply? (Continued)

• OMB Circulars (Office of Management and Budget)– Circular A-87 Cost Principals – State and Local

– Circular A-21 Cost Principals – Education Institutions

– Circular A-133 Single Audit Requirements

– Circular A-102 Grants and Cooperative Agreements with State and Local Governments

– Circular A-110 Uniform Administration Requirements for Grants and Agreements with Institutions of Higher Education, Hospitals and Other Non-Profit Organizations


23

What types of costs are generally considered eligible?

• Administrative Costs (5%)• Personnel Services (time records)• Operating Expenses • Stipends• Consultants• Instructional Materials• Travel• Instructional Equipment


24

What costs are NOT eligible?

1. Student expenses or direct assistance to students *

2. College tuition, fees, books*

3. Entertainment

4. Awards and memorabilia

5. Individual memberships

6. Membership with orgs. that lobby

7. Fines and penalties

8. Insurance/self-insurance

9. Expenses that supplant10. Audits except single audit11. Contributions and donations12. Contingencies13. Facilities and furniture *14. General advertising15. Alcohol16. Fund raising17. General administration


25

What are the 9 requirements for Perkins IV expenditures?

1. Strengthen academic & technical skills of students

2. Link secondary and postsecondary CTE programs (at least one program of study)

3. Provide students with strong experience and understanding all aspects of an industry (WBL)

4. Develop, improve, & expand use of technology

5. Professional development

6. Evaluate programs with emphasis in meeting needs of spec. pops.

7. Initiate, improve, expand and modernize quality programs

8. Provide services & activities of sufficient, size, scope and quality

9. Provide activities to prepare special pops. for high skill, high wage, or high demand occupations leading to self-sufficiency.

Funds made available to eligible recipients under this part shall be used to support career and technical education programs that-


26

20 permissive uses of funds:

1. Involve parents, business and labor in planning & operation

2. Career guidance & academic counseling

3. Business Partnerships - Work-related experience students & faculty

4. Programs for spec. pops.

5. CTE student organizations

6. Mentoring & support services

7. Upgrading equipment

8. Teacher prep. programs

9. Improving and developing new CTE courses including distance ed.

10. assist transition to BA degree programs

11. support entrepreneurship education

12. initiatives for secondary students obtaining postsecondary credit to count towards an AA/AS or BA/BS degree

13. support small CTE learning communities

14. Family & consumer sciences

15. Adult CTE programs

16. Job placement programs

17. Support Nontraditional activities

18. Automotive technologies

19. *Pooling funds –Teacher prep,

data & accountability,

assessments

20. Support other CTE programs

Regional Workshops


27

Effects of not achieving targets

• Above 90% of targets:– Freedom to fund any program improvements– Encouraged to address low performance

• Below 90% - year 1– Focused Improvement section

• Requires some analysis to write the narrative

– Must address low performance in funded Programs– Encouraged to start a diagnostic study

Regional Workshops


28


continued...

• Below 90% - year 2 or no improvement– Focused Improvement section– Diagnostic study– Propose Effective Practice solutions

• Probably a more district level approach to expenditures

Regional Workshops


29


continued...

• Below 90% of target - 3 years– Focused Improvement section– Submit Diagnostic study w Action Plan to CO– Action Plan

• Proposed Effective Practice solutions & Implementation dates

• Probably an even more district level approach to expenditures

– Risk MonitoringRegional Workshops


30

Break time

• I am the messenger – don’t kill the messenger

• Questions

• 20 minute break

Regional Workshops


31

Data: Why?

• Funding

• Outcomes Analysis

• Telling the story

Regional Workshops



Defining the Data

• SAM Codes• TOP Codes• Data Elements• Core Indicators

– “The Law”– Definitions– Negotiated Performance Targets– Measurement Approaches/Formulas

Funding



Student Accountability Model (SAM) & Taxonomy of Programs (TOP)

• Priority “A“ - Apprenticeship – Must have the approval of the Division of Apprenticeship

Standards

• Priority “B“ – Advanced Vocational– Used sparingly, no more than two courses in any one

program– “B” level courses must have a “C” prerequisite in the same

program area

• Priority "C" – Clearly Occupational– Generally taken in the middle stage of a program, detracts

"drop-ins."



Student Accountability Model (SAM) & Taxonomy of Programs (TOP), Continued

• Priority "D" – Possibly Occupational– Taken by students in the beginning stages of their

occupational programs– Can be survey course

• Priority “E” = Non-Occupational

Vocational Flag on TOP code– Designed to identify vocational “Programs” for federal

reporting (*) - see Taxonomy of Programs, Sixth Edition, July 2007



Data ElementsMIS System

• Students, Courses, Degrees, Services

• Student VTEA Data Elements– Economically Disadvantaged– Single Parent– Displaced Homemaker– Cooperative Work Experience Education– Tech Prep– Migrant Worker



Accountability Requirements Section 113(b)

5 core indicators of performance:1. Student attainment of technical skill proficiencies;

2. Student attainment of credential, certificate, or degree;

3. Student retention in postsecondary education or transfer;

4. Student placement in military, apprenticeship, or employment

5. Student participation/completion of non-traditional training

• State and Local adjusted levels of performance

– Levels of performance negotiated with USDE / State

• Results reported annually


37

Perkins III (1998)Core Indicators

1. Skill AttainmentStudent Achievement – Successful course completion

• Academic• Vocational and Technical

2. Completions Student program completion – Certificates & Degrees

3. Placement and (3b) Retention Placement and retention in employment

4. Equity -- Non-Traditional Employment Participation/Completion of non-traditional training

• 4a - Participation• 4b Completion

Regional Workshops


38

Core Indicators

1. Technical Skill AttainmentSuccessful CTE course completion

2. Completions

Program completion–Certificate, Degree & Transfer Ready

3. Persistence & TransferStudent persistence in Higher Ed

4. Placement Placement in apprenticeship, employment, military, fed gov

5. Equity -- Non-Traditional Employment Participation (5a)/Completion (5b) - non-traditional training

Regional Workshops



Cohort Definitions Used for Measurement

Participant: NT Participation

– Perkins III -Any enrollment in a CTE course (TOP)

– Perkins IV – concentrator using assigned major

Concentrator: All other indicators

• Cohort of students enrolled during the cohort year and

– Successfully completed at least one course in the middle or end of a program (SAM A-C) and 12 vocational units within a single discipline (two digit TOP) or

– Program completion as indicated by receipt of ANY vocational credit certificate or degree

Leavers: Not enrolled in year following cohort year

– 2P1 - Completions

– 4P1 - Placement (Employment)



Assigning a Program to a Student

1. Award –TOP code of CTE Certificate or Degree

2. Concentrators

• Hierarchy based on SAM Priority code

• Assigned to the TOP where most CTE units occurred


41

What is not decided

Regional Workshops

• Concentrator– How long to accumulate 12 units in a two digit TOP (discipline)

• Two, three, or five years

• Completions – Transfer Ready or Prepared

• Employment – Which period for “employed”?

• Feds want quarter 2 - Oct-Dec • Any of 4 quarter

• NT Participation• CRCC


42

Negotiating Targets

• Perkins III– State negotiates targets USDE

• Perkins IV– State negotiates targets USDE

• Targets for 1 year and then 2 years (3rd & 5th)

– Locals either:• Accept those targets• Negotiate local targets

– Included in Local Plans• Targets for next year • Negotiations complete by May 15

Regional Workshops


43

Sanctions in Law

• Perkins III– State – Bundled with WIA, ESEA, Adult Ed

• Improvement plan• Two years below, Secretary MAY withhold funds

• Perkins IV– State – Perkins – each indicator

• 90% of Target• 3 years for any single target

– Locals:• 90% of Targets• Improvement Plan• Year 2 no improvement, 3 years <90% for any single target

– State SHALL provide technical assistance– State MAY withhold funds (all or part)

Regional Workshops


44

CCC Sanction Plan:Meeting Performance Targets

• Application is Improvement Plan

• Accountability Determination

– In compliance – at or above 90%

– Focused Improvement Status - < 90%

• Focused Improvement section

• Year 2 – No Improvement: Diagnostic Study

– Priority Focused Improvement – 3rd Yr < 90%

• Diagnostic Study and Action Plan

Regional Workshops


45

Timeline for Outcomes & Outputs

• Negotiating Performance 2008-09– Spring 2008

• Cohort Year (2006-07)• +1 yr for outcomes (2007-08)

– Transfer– Persistence– Employment– Spring 2009

• Outcomes have already occurred– Target low performance now!


46

Local Application: District

Regional Workshops


47

Spring 2008 Sessions

7 Regional Sessions for District/College Teams– Content so far from this session

• TOP & SAM Codes• Data Elements• Core Indicators• “The Law”

– Definitions & Negotiated Performance Levels– CCC Strategies

– CI Measurement Approaches/Formulas/Targets – Reports

• Negotiation – Strategies• Compliance – Local App• Program Improvement

Regional Workshops


48

Core Indicators & Targets

1. Technical Skill AttainmentSuccessful CTE course completion

2. Completions

Program completion–Certificate, Degree & Transfer ready

3. Persistence & TransferStudent persistence in Higher Ed

4. Placement Placement in apprenticeship, employment, military, fed gov

5. Equity -- Non-Traditional Employment Participation (5a)/Completion (5b) - non-traditional training

Regional Workshops



Core Indicator 1Technical Skill Attainment

• Successful Course Completions – Technical Skill Proficiencies

• Vocational (CTE) Courses– SAM A-C

– Vocational TOP

• G.P.A. Concentrators– Grade reports (moved to Data Mart)


50

Core Indicator 1: Measurement &Performance Levels

SAM A - C Courses: # Students with GPA > 2.00

÷# Students with Grades A - F

Year Target Actual*2002-03 77.76% 84.01%2003-04 78.76% 84.73%2004-05 79.76% 84.70%2005-06 84.02% 83.69%2006-07 83.70% ??.??%

99.6%

* Based on Perkins III data.Regional Workshops



Core Indicator 1: Measurement &Performance Levels (Analysis)

SAM A - C Courses:

# Grades A-C & Cr

÷

# Grades A–F, Cr-NoCr, W, & I



Core Indicator 2Program Completions

• Completers (numerator)– Award in Current Year

– AA/AS degrees– Certificates

– Transfer Ready (or Prepared)– or Equivalent

• Leavers & Completers (denominator)– Left system (college) for one year and/or– Award in Current Year

– AA/AS degrees– Certificates– Transfer Ready


53

CI 2-Completions: Measurement & Performance Levels

Certificate/Degree/Transfer Ready÷

All Concentrators (Leavers & Completers)

Year Target Actual*2002-03 59.82% 76.36%2003-04 59.82% 74.89%2004-05 60.82% 73.42%2005-06 73.00% 73.73%2006-07 73.00% ??.??%

* Based on prior indicator definition.Regional Workshops



Core Indicator 3 Persistence & Transfer

Concentrators who were not leavers in the year following the cohort year

Or

Transfers (CSU/UC/Clearinghouse)

÷

All Concentrators



Core Indicator 4 Placement

• Placement• UI wage file match 1st year out

• Apprenticeship, Military, Fed Gov

• Minus Continuing in Four Year Institution – CSU, UC, Clearinghouse

• Not resolved• Four quarters in Academic Yr after leaving

• Quarter 2


56

CI 4 Placement : Measurement & Performance Levels

Leavers & Completers in UI covered employment or

Apprenticeship, Military, Fed Gov÷

All Leavers & CompletersYear Target Actual*2002-03 83.19% 84.08%2003-04 83.19% 82.53%2004-05 83.19% 82.07%2005-06 82.07% 81.76%

98.65%

99.62%

*Based on prior indicator definition.Regional Workshops


57

Core Indicator 5Gender Equity

Training leading to Nontraditional Employment

(e.g., Men in Nursing – Women in Auto)

75% / 25% from 2000 census employment data

• Job codes (SOC) mapped to 2000 Census data

• SOC codes mapped to CIP (USDE)

• NAPE developed Nontraditional CIP table

• CIP codes mapped to TOP (CCC)Regional Workshops



Core Indicator 5Gender Equity

In Programs leading to Nontraditional Employment

Nontraditional Gender Students

÷

All Students in NT Program



CI 5a: Participation Measurement & Performance Levels

Nontraditional participants enrolled in a nontraditional TOP Code

÷All participants enrolled in a

nontraditional TOP Code

Year Target Actual2002-03 27.98% 36.98%2003-04 28.98% 36.80%2004-05 29.98% 35.31%2005-06 35.31% 37.16%



Nontraditional “completers” of nontraditional programs

÷All “completers” of nontraditional programs

CI 5b Completion: Measurement & Performance Levels



61

Nontraditional “completers” of nontraditional programs

÷All “completers” of nontraditional programs

CI 5b Completion: Measurement & Performance Levels


90.6%89.8%92.8%

Regional Workshops



Do these indicators really measure what education in CCC does?



Do these indicators really measure what education in CCC does?

They are only indicators.



Are they meaningless?



Are they meaningless?

No. Student success matters.



How can we use the indicators to improve student success?




Understand “What” is happening to students.




Understand “What” is happening to students.

Why is it happening?

SLOs, CATs, Research, etc.


69

How important is setting an appropriate target ?

Regional Workshops



Accessing the Data

• Access• Report Formats• Report Structures• Report Contents• Report Examples



Reports

• New reports– Developed for negotiations

• District and College Level• Special Pops detail • 8 years history• Final version mid February 2008

• All Reports– Easier access – Excel or SQL reports– ??? Late February 2008 ???

• Progressive scrutiny on use of reports– Below 90% of Targets in



Access

• http://www.cccco.edu• Important Documentation

– Quick Reference

– Negotiation Report Instructions

– System Documentation

– System Requirements• No more Installing plug-in

• No Administrative rights requirements



Report Formats

• Print Version – For administrators, faculty, students & PIOs

• Printing from the WWW• Local Applications and Final Reports• For exporting to

– word processor– spreadsheet

• Data Download Version – For Researchers, IS & Technophiles

• CSV For Export to:– Excel

» SPSS, SAS or other Stat Package» BRIO» Access or other database software



Data Download


75

Negotiation Workbooks

• Instructions

• Counts

• Percentages

• Charts– Rates– Counts

• Data work tables



Negotiation Tables



Negotiation Charts

3P1 - Rates

80.99%

79.15%

83.32%84.07% 84.18% 84.60% 84.90%

84.08%

82.53% 82.07%81.14%

91.4%92.1% 92.7%

93.6% 94.0%94.8% 95.0% 94.6%

93.3% 93.2%93.8%

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

100.00%

1995-96 1996-97 1997-98 1998-99 1999-00 2000-1 2001-2 2002-3 2003-4 2004-5 2005-6est

Grand Total

Female

Male

Non-traditional

Economically Disadvantaged

CA Empl Rate

Est

California Employment Rate (source: BLS)

2P1 - Counts

50,50744,245

70,296 69,356 68,890 69,700

62,140 60,37664,125

68,33772,572

32,438 30,963

39,790 41,990 40,881 42,10045,166 46,101 48,028 50,172 52,296

88,22793,319 92,159

96,833

104,203 106,925

114,362118,405 117,427 119,035

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

Program Year

Total

Success

All CCC Awards

Est SP

Total 50,507 44,245 70,296 69,356 68,890 69,700 62,140 60,376 64,125 68,337 72,572

Success 32,438 30,963 39,790 41,990 40,881 42,100 45,166 46,101 48,028 50,172 52,296

All CCC Awards 88,227 93,319 92,159 96,833 104,203 106,925 114,362 118,405 117,427 119,035

Est SP 23,991 25,362 34,210 36,598 36,090 36,483 38,287 38,394 40,004 43,589 46,604

1995-96 1996-97 1997-98 1998-99 1999-00 2000-1 2001-2 2002-3 2003-4 2004-52005-6

est

Es

Es

fees increase Sp 2003from $11 - $18

fees increaseFall 2004from $18 - $26

Although CCs are helping more people succeed (including special population students), the numbers of students leaving the system without a certificate or degree is increasing at a higher rate.

Fee increases seemed to have the impact starting in 2002-3 academic year.

CA hits 6.6% high unemp rate Jan 04

Regional Workshops



Report Structures

• Negotiation Workbooks• Forms

• Perkins IC - Local Application • Perkins IC - Final Report

• Summaries– All five Indicators on one page

• Answer Sheet Style

• Detailed – Percents and counts – Detailed breakouts for each Indicator component



CI Report Contents

• CI Reports

– State, District, College

– 2-4-6 digit TOP

– Special Populations breakouts

– Identified on each report• Performance below negotiated levels

– Performance below is highlighted

– Low numbers (denominator < 10)• Blue-italics



CI Contents (continued)

1. Summary• College, District & State• Identifying Low Performance & Low Numbers• Comparing Performance across Programs

2. Detailed• Program Area (6 digit TOP) • Three Year Trends• Identifying Low Performance areas• Comparing Performance within Programs



Report Examples

• Final Report Form • College by TOP Form • Summary – College

– Summary - 5 digit TOP

• Summary – Detail • Core 2 – Completions

– Core 2 - 5 digit TOP

• Course Voc Status/SAM Code



Negotiation Report Contents

• District, College?

• Special Populations breakouts

• Numerators, Denominators, & Rates

• Charts for analysis

College Local App -Final Report Form

Regional Workshops

Dist Local App - Final Report Form

Regional Workshops

Final Report Form

Regional Workshops

College by TOP Form

Regional Workshops

Summary - College

Regional Workshops

Summary – College 5 Digit

Regional Workshops

Summary - Detail

Regional Workshops

Core 2 - Completions

Regional Workshops

Core 2 - Completions 5 digit

Regional Workshops

Course Voc Status/SAM Code

Regional Workshops



Using the Data

• Negotiate or accept state-adjusted levels – Trends over time for your college

– By program, special population, Tech Prep

• Workshops, Committees & Faculty Information– Circulate to faculty for their use & input

• Factor in allocation decision-making for Local Plans– Supplement MIS data with

your own knowledge and follow-up of students



Practice using the Data

Negotiation Exercise:• Review FAUPL Worksheet• Walk through common scenarios• Review the pre-printed report for a sample

college• Conduct a guided practice using first

indicators• Negotiate or Accept State Targets


96

Faupl


97

Negotiation Workbook


98

Discussion

• Improving Session Content:– Order – Negotiations earlier or later– Detail– Strengthen funding ties or are sanction threats

sufficient?– Other?– Resources: JSPAC, IWITTS, NAPE

Chuck Wiseley, [email protected], 916.327.5895

Regional Workshops

Documents

Click to edit Master title style 1 Core Indicators, Reports, Negotiated Performance Targets, and Perkins Funds Regional Workshops Carl D Perkins (Perkins