DOCUMENT RESUME

ED 409 957 JC 970 421

AUTHOR Rajek, Becky; And OthersTITLE Designing and Implementing Value Focused Effectiveness

Indicators.INSTITUTION Northcentral Technical Coll., Wausau, WI.PUB DATE Jun 97NOTE 25p.; Paper presented at the Annual Community College

Consortium Summer Institute on Institutional Effectivenessand Student Success (9th, Tempe, AZ, June 22-25, 1997).

PUB TYPE Numerical/Quantitative Data (110) Reports Descriptive(141) Speeches/Meeting Papers (150)

EDRS PRICE MF01/PC01 Plus Postage.DESCRIPTORS Access to Education; *College Outcomes Assessment; College

Planning; Economic Impact; *Evaluation Criteria; EvaluationMethods; Participant Satisfaction; School Effectiveness;*Self Evaluation (Groups); Technical Institutes; Two YearColleges

ABSTRACTFocusing on the measurements and traits used to assess

program effectiveness at Wisconsin's Northcentral Technical College (NTC),this document reviews the college's value-focused effectiveness indicatorsand provides data on outcomes as of 1994-95. Following NTC's missionstatement, the program assessment criteria developed at the college aredescribed, indicating that they include both measurements, obtained fromcommonly used college or state data, and traits, or information related toemployer satisfaction and program supply and demand. Measurements, traits,and related benchmarks are then described for the following indicators:customer satisfaction, including student satisfaction rate, the studentsatisfaction rate with information provided by program leavers, and employersatisfaction; economic impact, including graduate salary comparisons,placement rates, placement rates including leavers, future demand, communityeconomic return, and outcomes after 5 years; and efficient use of resources,measured by course utilization, student accessibility, retention/graduationrates, faculty utilization, and high demand among students. Following asample "report card" of 1992-93 to 1994-95 outcomes for NTC's dental hygieneprogram, outcomes by program are presented for the same period for thefollowing measures: student satisfaction rate, graduate salary comparison,placement in jobs related to students' program of study, community economicreturn, course utilization, retention/graduation rate, and facultyutilization. Overheads related to the development of the indicators areattached. (HAA)

********************************************************************************Reproductions supplied by EDRS are the best that can be made

from the original document.********************************************************************************

C7

rnO.rtQ

DESIGNING AND IMPLEMENTINGVALUE FOCUSED EFFECTIVENESS

INDICATORS

PRESENTED TO CCCD SUMMER INSTITUTE

JUNE 23,1997 -TEMPE ARIZONA

PRESENTED BY NORTHCENTRAL TECHNICAL COLLEGE

BECKY RAJEKARVES JONES

DAN McCOLLUMTOM GOLTZ

1000 CAMPUS DRIVEWAUSAU, WI. 54401

715-675-3331

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

ST COPY ANL 1 L1

2

-N.

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

D. R. McCollum

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)."

DRAFTNTC INSTRUCTIONAL TEAM LEADERSHIP COUNCIL

PROGRAM ASSESSMENT CRITERIA SUMMARY

Mission Statement: Northcentral Technical College provides quality education, training, andservices of high value to individuals, businesses, industries, and organizations--building a

competitive work force in a changing global society.

The Program Assessment Criteria includes both measurements and traits. Sources formeasurements are from commonly used (and audited) college or state data, such as six-monthfollow-up, staff accounting, client reporting system, course files, etc. Traits are measurable but notnecessarily from readily available and audited sources. Trait information is provided from programfaculty and describes such things as employer satisfaction and the supply and demand for the

program.

I. Customer Satisfaction can be assessed by:

1. MEASUREMENT "Student Satisfaction Rate"As reported in the 6-month follow-up for respondents who reported very satisfied orsatisfied with their program. The data is presented for the prior three years and as a

three-year average.

BENCIWARK: Three-year average greater than 95%.

2. MEASUREMENT "Student Satisfaction Rate including Leavers"A similar report will be developed for program leavers to be implemented fallsemester 1996 for fall and spring 1995-96 leavers.

BENCHMARK: To be determined.

3. TRAIT: "Employer Satisfaction"Exemplary programs are regularly involved with related business and industry toobtain feedback on employer satisfaction of programs.

BENChMARK: Program can produce evidence of above.

Provide evidence that the program faculty is regularly involvedwith business andindustry to obtain feedback on employer satisfaction with the program and programgraduates. This could include work with advisory committees but broader exposure

is desirable.

II. Economic Impact can be assessed by:

1. MEASUREMENT: "Graduate Salary Comparison"NTC graduate hourly average salaries compared to those reported in WTCS 6-monthgraduate follow-up expressed as a percentage. The three prior years are reportedand a three-year average.

BENCHMARK: 100% of state average

2. ME4SUREMENT 'Placement"The percent of placement into related employment by program as reported in the NTC6-month follow-up survey. The data is presented for the prior three years and as athree-year average.

BENCHMARK: 90%

3. MEASUREMENT:: 'Placement with Leavers"The percent of placement into related employment including information provided byleavers by program. The data is presented for the prior three years and as a three-year average.

BENCHMARK: To be determined

4. TRAIT: "Future Demand"An exemplary program finds ways to assess labor market needs, supply and demand,future trends and political implications.

BENCHMARK Program can produce evidence of above

A. Provide data on labor market needsSource: DILHR - State and regional (with central District assistance)

B. Provide supply dataSource: WTCS, client reporting system, graduates and enrollments

(with central District assistance)Source: Other training sources (include high schools, if appropriate)

C. Provide data on future employment trendsSource: DILHR job trends (with central District assistance)Source: Professional publicationsSource: Other

D. Describe the current political and economic environment as it may relate to majorprogram modification or discontinuance

Page 2

E. Provide out-of-district enrollmentsSource: WTCS report

NOTE: The above information could be used by the program and advisory committeefor program improvement and budget requests.

5. MEASUREMENT. "Community Economic Return"As based on the annual salary of employed related graduates divided by directinstructional cost.

Annual salary is determined by information from the 6-month follow-up survey (job -outs to be added later). Annual salary is computed by prorating hours worked forall respondents.

Direct instructional costs include salaries and benefits of full, part-time and call stafffor all program courses. Course costs are based on staff accounting information andare prorated by program students enrolled. This does include elective courses takenby program students. It does not include other costs directly attached to aninstructional area, capital equipment or general overhead.

Employed-related Graduates x Annual Salary = CommunityDirect Instructional Cost Economic Return

BENG-LI/LARK: Variable by program type (TBD)

6. TRAIT: "5 Year Follow-up"An Exemplary Program tracks graduates beyond the 6 month follow-up and usesgraduate information to improve the program.

BENCHMARK The program can provide evidence of trackinggraduates past the 6-month follow-up.

Provide information on your program students success in the labor marketbeyond the 6 month follow-up. Include salary and job advancement and retentioninformation.

Source: 5 year follow-up and other data. Note: The college is exploringthe possibility to annually survey our graduates for three years after graduation.

Page 3

5

III. Efficient Use of Resources assessed by:

1. MEASUREMENT: "Course Utilization"As a percent of occupational specific courses by calculating actual hours ofinstruction (AHI) and dividing by the capacity hours of instruction (CHI).

AHI = actual headcount as reported to the state X total course hours as defined bycourse section

CHI = capacity as defined by the course section X total course hours as defined bythe course section

BENCHMARK: 65% utilization

2. TRAIT: "Student Accessibility"An exemplary program identifies and responds to student accessibility needs ineducationally sound ways that are in sync with the market served, utilizing costeffective solutions.

BENCHMARK: Annually complete Program Accessibility Matrix

3. MEASUREMENT: 'Retention/Graduation Rate"(%) Based on the number of students who started a program in a given year

compared to successful student outcome. Successful outcomes include graduations,still in program, job-outs (in the future). Time duration covers students officially in aprogram (not pre-program students) allowing one and one half (1.5) the time fornormal program completion. Short term (Ex: a 6-week Nursing Assistant) completionis within a year. Long-term will be considered the same as two-year programs.

BENCHMARK: Retention rate greater than 70%.

4. MEASUREMENT: "Faculty Utilization"Based on actual hours of instruction (AI-II) per FTE faculty for all program courses(not electives). The data is presented for the prior three years and as a three-yearaverage.

AHI = actual headcount as reported to the state multiplied by the total course hoursas defined by course section

F7E Faculty = staff accounting instructor loads prorated by the number of programstudents in the course section

BENCHMARK: Three-year average greater than 12,000 hours

Page 4

5. TRAIT: "High Student Demand"An exemplary program has a high student demand as evidenced by a full program.

BENCHMARK: Program is full at start-up

A. Describe the student demand for your program over the last three yearsSource: NTC admissions reports

B. Describe your student market in terms of student demographics, age, enrollmentstatus, location, prior education, entry-level competencies, etc.

Source: Client Reporting System

C. If you have a non-homogeneous group, describeyour student market segments bygroup. NOTE: This information will be helpful to develop a marketing plan, ifnecessary.

Page 5

sigsnliem

6narlIV

AV

Ad00

196 '1561101.11

ISM

Silty91111111

661-6611I

GA

M 000 W

I )1111/11HO

N311

KM

us'is"gnM

I-ZM

I411m

A I'M

11113

Nw

ssImIK

I Km

16917m

pgV

AS

ILMeV

selsilPsie

"

migm

rilminm

mm

-ma

.

Im=

aff:crin3 V

W V

Wis G

M W

tld M

anus°IM

171:11'

VN

W3

61

1111.6161

II .1 01 )1HM

IHO

N311

501119IZ

1sP

H uslisssIQ

SP

Y sigssfive

smil

910191M11 V

III111111

LILT,"4

11.1111L7

ME

MF

es!O

ffS

t3

is MeV

Wv

d IsgssKIIIS

II

1111311113

00

IMP

IMO

D

51 61IS

LSA

dutam

ildewMIy

irsmic

siU

Iim

qvuAv

w3

PM

lM

w3

vesselw

(sissssi i sputa)

9111am

mo,

ssissim0

IssumosisL

61

x00'011S

tsv A

siw3

1-1111

oz5S

11O

LIL

sHM

IMI3 414115IV

,11:661

%SCti

SI

filltatil3I-LIS

I

SI

(Ivo ssisMs/0)

01611P A

VIsw

rasfdO

iH

EIV

IVH

ON

31

569'96L IIIS

MI6 is IL

H 21N

x06'66O

f LISW

ISP

16kilH

MIS

/ItK

PH

3114%

CS

S6

116 111

sP66 is

EP

IC A

VM

3114xiC

'Si

96119E

PIC

P II

ii/H'191

9S S

t!m

lisIssiIt

M6I-Z

6111N

tririH3N

311

11911 l Il1ifffH2N

311 515 96silaissy

o500 oot

L6611161

01.1LIM

:26MM

Ll

N..

1

23

4567891011

12131415161718192021

22232425262728293031

32333435363738394041

42434445

STUDENT SATISFACTION RATETOTALS

1.1 Satisfa 'on1992-199 1993-199 1994-199 Average

97.73%Accounting 100.00% 95.40% 97.80%Marketing 85.70% 96.40% 94.70% 92.27%Legal Secretary 100.00% 94.70% 94.70% 96.47%Medical Secretary 100.00% 100.00% 100.00% 100.00%Admin. Assistant 95.00% 100.00% 100.00% 98.33%CIS Programmer 95.20% 85.00% 88.80% 89.67%Small Busi Mgt. 100.00% 100.00% 100.00% 100.00%Insurance Services 100.00% 100.00% 100.00% 100.00%Supervisory Mgt. 100.00% 100.00% 100.00% 100.00%Printing/Publishing 92.30% 90.00% 91.15%Police Science 93.70% 90.00% 94.10% 92.60%Dental Hygiene 93.70% 100.00% 95.40% 96.37%Nursing AD 80.70% 90.00% 100.00% 90.23%Radiography 100.00% 100.00% 87.50% 95.83%Edu Interpreter Tec 93.30% 100.00% 91.60% 94.97%Electronics 100.00% 100.00% 100.00% 100.00%Mechanical Design 100.00% 100.00% 100.00% 100.00%Arch-Res Design 90.40% 100.00% 90.90% 93.77%Electromechanical 100.00% 100.00% 95.60% 98.53%Laser Tecnology 100.00% 100.00% 100.00% 100.00%Industrial Engineer 100.00% 100.00% 100.00% 100.00%Farm Business 87.50% 100.00% 93.75%Police Recruit 100.00% 97.30% 91.60% 96.30%Nursing Assistant 95.80% 99.10% 89.60% 94.83%EMT 100.00% 100.00% 91.80% 97.27%IPS 100.00% 100.00% 100.00% 100.00%Inventory Control 100.00% 50.00% 91.60% 80.53%Auto Body/Paint 100.00% 100.00% 100.00%Mechanical/Comput 100.00% 100.00% 100.00% 100.00%Welding 92.80% 100.00% 94.40% 95.73%Surgical Tech. 100.00% 85.70% 100.00% 95.23%Agri Power & Equip. 100.00% 100.00% 66.60% 88.87%Automotive Tech. 100.00% 90.00% 100.00% 96.67%Machine Tooling 50.00% 90.90% 100.00% 80.30%

TOTALS/AVGS 95.82% 95.42% 95.35% 95.51%

Measurement Goal1.1 Satisfaction (9591)

10 -7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

GR

AD

UA

TE

SA

LA

RY

CO

MPA

RIS

ON

TO

TA

LS

11.1

Hou

ryW

age

1992

-199

319

93-1

994

1994

-199

5A

vera

ge%

of S

tate

102.

32%

NT

CS

TA

TE

% o

f Sta

teN

TC

ST

AT

E%

of S

tate

NT

CS

TA

TE

% o

f Sta

teN

TC

ST

AT

E $8.1

8A

ccou

ntin

g$7

.49

$7.7

396

.90%

$8.6

4$8

.17

105.

75%

$8.9

9$8

.65

103.

93%-

Mar

ketin

g$6

.37

$7.0

989

.84%

$9.6

4$8

.24

116.

99%

$8.6

4$8

.09

106.

80%

$8.2

2$7

.81

105.

25%

88.4

0VLe

gal S

ecre

tary

$0.0

0$7

.36

$0.0

0$7

.12

ER

R10

3.37

%$6

.95

$7.3

7$7

.69

90.3

8%-

$7.5

3$8

.69

86.6

5%$7

.24

$8.1

9

Med

ical

Sec

reta

ry$7

.40

99.5

9%88

.19%

$6.9

3,__

__$7

.69

$8.7

8'-$

8.35

90.1

2%10

5.15

%$7

.22

$7.7

1$7

.40

$8.1

697

.52%

94.4

1%A

dmin

. Ass

ista

nt$6

.95

$7.7

689

.56%

$7.3

9--$

8.38

CIS

Pro

gram

mer

$9.4

4$1

0.65

88.6

4%$1

0.18

$11.

8785

.76%

;12.

92$1

3.13

98.4

0%$1

0.85

$11.

8891

.28%

Sm

all B

us! M

gt.

$0.0

0$0

.00

ER

R$0

.00

$0.0

0E

RR

$0.0

0$0

.00

ER

R$0

.00

$0.0

0E

RR

,10

8.46

%In

sura

nce

Ser

vice

s$8

.50

$7.1

411

9.05

%$7

.41

$8.3

388

.96%

$9.2

2$7

.70

119.

74%

$8.3

8$7

.72

Sup

ervi

sory

Mgt

.$0

.00

$0.0

0E

RR

$0.0

0$0

.00

ER

R$0

.00

$0.0

0E

RR

$0.0

0$0

.00

ER

R

Prin

ting/

Pub

lishi

ng$7

.05

$7.1

898

.19%

$8.4

5$7

.38

114.

50%

$8.3

5$8

.12

102.

83%

$7.9

5$7

.56

105.

16%

Pol

ice

Sci

ence

$8.9

9$9

.23

97.4

0%$8

.13

$9.8

082

.96%

$9.4

2$9

.75

96.6

2%$8

.85

$9.5

992

.22%

96.6

962;

Den

tal H

ygie

ne$1

6.56

$17.

3695

.39%

$17.

23$1

7.98

95.8

3%$1

7.11

$17.

3098

.90%

$16.

97$1

7.55

Nur

sing

AD

$13.

86$1

0.81

--r$1

3.46

0.44

102.

97%

104.

12%

$13.

05$1

0.75

---

$10.

50$1

3.75

$11.

001

94.9

1%10

2.38

%10

0.00

%

$14.

11$1

1.11

$11.

84

$14.

00$1

0.99

$11.

84

100.

79%

101.

09%

100.

00%

$13.

67$1

0.91

$10.

82

$13.

74$1

0.64

$10.

82

99.5

4%10

2.51

9;10

0.00

%R

adio

grap

hyE

du in

terp

rete

r T

ech

$9.6

3$9

.63

100.

00%

$11.

00E

lect

roni

cs$9

.55

$9.6

199

.38%

$9.6

1$1

0.00

96.1

0%$1

0.84

$9.9

610

8.84

%$1

0.00

$9.8

610

1.45

%

Mec

hani

cal D

esig

n$0

.00

$0.0

0E

RR

$10.

15$1

0.32

98.3

5%$9

.54

$10.

9587

.12%

$9.8

5$1

0.64

92.5

7%

Arc

h-R

es D

esig

n$8

.27

$8.2

710

0.00

%$9

.20

$9.2

010

0.00

%$1

0.11

$10.

1110

0.00

%$9

.19

$9.1

910

0.00

%

Ele

ctro

mec

hani

cal

$10.

73$1

1.00

97.5

5%$1

0.25

$11.

7087

.61%

$10.

84$1

1.81

91.7

9%$1

0.61

$11.

5092

.21%

Lase

r T

ecno

iogy

$10.

36$1

0.36

100.

00%

$8.5

8$8

.58

100.

00%

$11.

39$1

1.39

100.

00%

$10.

11$1

0.11

x13.

7110

0.00

%

Indu

stria

l Eng

inee

r$0

.00

$0.0

0E

RR

$14.

93$1

4.42

103.

54%

ER

R81

.80%

$13.

99$1

3.00

107.

62%

$14.

4610

5.47

%

Far

m B

usin

ess

$0.0

0-$

8.99

$0.0

0$1

1.76

ER

R76

.45%

$0.0

0$1

0.38

$0.0

0$1

2.69

$6.6

5$8

.81

$9.6

1$1

2.91

69.2

0%68

.24%

$6.6

5$9

.39

$9.6

1$1

2.45

69.2

0%75

.43%

Pol

ice

Rec

ruit

Nur

sing

Ass

ista

nt$6

.35

$6.2

010

2.42

%$6

.76

$6.6

510

1.65

%$7

.13

$7.1

010

0.42

%$6

.75

$6.6

510

1.45

%

EM

T$9

.11

$9.6

994

.01%

$11.

07$1

0.58

104.

63%

$0.0

0$0

.00

ER

R$1

0.09

$10.

1499

.56%

IPS

$6.1

3$6

.09

100.

66%

$7.1

6$6

.77

105.

76%

$6.7

4$6

.62

101.

81%

$6.6

8$6

.49

102.

82%

Inve

ntor

y C

ontr

ol$7

.34

$6.7

510

8.74

%$0

.00

$0.0

0E

RR

$7.9

1$7

.25

109.

10%

$7.6

3$7

.00

108.

93%

Aut

o B

ody/

Pai

nt$6

.50

$7.2

190

.15%

$0.0

0$0

.00

ER

R$6

.11

$7.8

178

.23%

$6.3

1$7

.51

83.9

5%

Mec

hani

cal /

Com

put

$0.0

0$0

.00

ER

R$7

.38

$8.2

989

.02%

$0.0

0$0

.00

ER

R$7

.38

$8.2

989

.02%

Wel

ding

$14.

38$8

.99

159.

96%

$10.

53$1

0.00

105.

30%

$8.4

1$1

0.17

82.6

9%$1

1.11

$9.7

211

4.27

96-

Sur

gica

l Tec

h.$8

.96

$9.1

3$7

.34

$7.3

1

98.1

4%89

.65%

81.6

7%

$8.7

0$6

.25

$7.3

6

$9.2

7$8

.43

$7.8

393

.85%

74.1

4%94

.00%

$9.0

2--

$5.6

2$7

.22

$9.7

6$7

.25

$7.9

2

92.4

2%77

.52%

91.1

6%

$8.8

9$6

.15

$6.8

5

$9.3

9--

$7.6

7$7

.69

94.7

4%80

.15%

89.1

2%A

gri P

ower

6 E

quip

.$6

.58

$5.9

7A

utom

otiv

e T

ech.

Mac

hine

Too

ling

$8.4

7$9

.62

88.0

5%$8

.71

$10.

31,

84.4

8%$1

0.57

$11.

0595

.66%

$a25

$10.

3389

.57%

TO

TA

LS/A

VG

S$7

.08

$7.1

898

.62%

$8.0

4$8

.40

95.6

9%$8

.41

$8.7

995

.61%

$8.6

6$9

.04

95.8

6%,

11.2

Hou

rlyW

age

(100

%of

Sta

te)

BE

ST

CO

PY

AV

AIL

AB

LE12

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

RE

LA

TE

D P

LA

CE

ME

NT

Gra

duat

es)n

ly

Acc

ount

ing

112

Pla

cem

ent

1992

-199

319

93-1

994

1994

-199

5A

vera

geR

ate 79

.82%

Ava

ilabl

e 31E

m .

loR

e!R

ate

Ava

ilabl

eE

mpl

oy R

elR

ate

Ava

ilabl

eE

mpl

oy R

elR

ate

Ava

ilabl

eE

mpl

oy R

el91

2477

.42%

4128

68.2

9%42

3992

.86%

114

Mar

ketin

g12

975

.00%

2620

76.9

2%17

1694

.12%

5545

81.8

2%Le

gal S

ecre

tary

42

50.0

0%17

1588

.24%

-9

888

.89%

3025

83.3

3%M

edic

al S

ecre

tary

1110

90.9

1%20

1890

.00%

1310

76.9

2%44

'38

86.3

6%A

dmin

. Ass

ista

nt19

1578

.95%

1210

83.3

3%10

990

.00%

4134

82.9

3%C

IS P

rogr

amm

er19

1157

.89%

2013

65.0

0%10

770

.00%

4931

63.2

7%S

mal

l Bus

t Mgt

.13

1292

.31%

22

100.

00%

74

57.1

4%22

1881

.82%

Insu

ranc

e S

ervi

ces

32

66.6

7%9

666

.67%

74

57.1

4%19

1263

.16%

Sup

ervi

sory

Mgt

.1

110

0.00

%2

210

0.00

%2

210

0.00

%5

510

0.00

%P

rintin

g/P

ublis

hing

75

71.4

3%13

1310

0.00

%10

770

.00%

3025

83.3

3%P

olic

e S

denc

e12

1191

.67%

107

70.0

0%16

743

.75%

3825

65.7

9%D

enta

l Hyg

iene

1615

93.7

5%21

2095

.24%

2016

80.0

0%57

5189

.47%

Nur

sing

AD

5449

90.7

4%60

5591

.67%

3432

94.1

2%14

813

691

.89%

Rad

iogr

aphy

1212

100.

00%

1111

100.

00%

88

100.

00%

3131

100.

00%

Edu

Inte

rpre

ter

Tec

h,15

1280

.00%

1514

93.3

3%13

1292

.31%

4338

88.3

7%E

lect

roni

cs8

562

.50%

66

100.

00%

75

71.4

3%21

1676

.19%

Mec

hani

cal D

esig

n5

120

.00%

42

50.0

0%6

583

.33%

158

.53.

33%

Arc

h-R

es D

esig

n20

1680

.00%

1817

94.4

4%20

1680

.00%

5849

84.4

8%E

lect

rom

echa

nica

l15

1386

.67%

1414

100.

00%

2116

76.1

9%50

4386

.00%

Lase

r T

ecno

logy

65

83.3

3%9

333

.33%

33

100.

00%

1811

61.1

1%In

dust

rial E

ngin

eer

10

0.00

%4

410

0.00

%3

266

.67%

86

75.0

0%F

arm

Bus

ines

s0

0E

RR

88

100.

00%

1211

91.6

7%20

1995

.00%

Pol

ice

Rec

ruit

3122

70.9

7%36

2672

.22%

108

80.0

0%77

5672

.73%

Nur

sing

Ass

ista

nt96

6870

.83%

106

8580

.19%

7759

76.6

2%27

921

275

.99%

EM

T50

3672

.00%

4831

64.5

8%31

1135

.48%

129

7860

.47%

IPS

3527

77.1

4%52

3363

.46%

3321

63.6

4%12

081

67.5

0%In

vent

ory

Con

trol

75

71.4

3%1

00.

00%

1211

91.6

7%20

1680

.00%

Aut

o B

ody/

Pai

nt2

210

0.00

%0

0E

RR

87

87.5

0%10

990

.00%

Mec

hanl

cal/C

ompu

t5

240

.00%

22

100.

00%

32

66.6

7%10

660

.00%

Wel

ding

44

100.

00%

1715

88.2

4%18

1583

.33%

3934

87.1

8%S

urgi

cal T

ech.

77

100.

00%

65

83.3

3%6

583

.33%

1917

89.4

7%A

gri P

ower

& E

quip

.3

310

0.00

%5

480

.00%

33

100.

00%

1110

90.9

1%A

utom

otiv

e T

ech.

44

100.

00%

108

80.0

0%8

787

.50%

2219

86.3

6%M

achi

ne T

oolin

g13

1184

.62%

1111

100.

00%

88

100.

00%

3230

93.7

5%

TO

TA

LS/A

VG

S54

142

177

.82%

638

508

79.8

7%50

799

878

.11%

1684

1325

78.6

8%

11.2

Pla

cem

ent

(90%

)

3B

EST

CO

PY A

VA

iLA

B5:

14

,

.".0.11

"1111

itstriplig

:;- :111111

IP 11 "," --1-1111111rain 73111 iummoilimm 111111111111

FFINIP1111 HPIMPOI1111 1111111

111111111111111 1111111111111

MENNIMPOINE

371111111111 11111111111

1111111111111111110111511111111111111111111111111111

11

a.-iNsa.rinaoggrigrusno.-aling"-IN sgax c,,

1

234567891011

12131415161718192021

22232425262728293031

3233343536373839404142434445

COURSE UTILIZATION

111.1 Course Utilization1992-199 1993-199 1994-199 Average

Accounting 60.50% 66.40% 64.80% 63.90%Marketing 68.00% 71.00% 79.40% 72.80%Legal Secretary 52.80% 58.80% 50.40% 54.00%Medical Secretary 55.30% 67.80% 55.60% 59.57%Admin. Assistant 50.90% 57.80% 47.30% 52.00%CIS Programmer 48.70% 58.80% 77.30% 61.60%Small Busi Mgt. 48.00% 64.30% 69.60% 60.63%Insurance Services 64.60% 65.20% 65.80% 65.20%Supervisory Mgt. 65.00% 81.10% 62.40% 69.50%Printing/Publishing 84.30% 70.50% 59.00% 71.27%Police Science 89.90% 91.50% 82.50% 87.97%Dental Hygiene 89.70% 91.60% 87.50% 89.60%Nursing AD 93.30% 91.70% 89.30% 91.43%Radiography

1

79.20% 81.00% 70.80% 77.00%Edu Interpreter Tec 62.60% 54.00% 59.20% 58.60%Electronics 62.00% 63.40% 59.20% 61.53%Mechanical Design 60.00% 67.10% 78.40% 68.50%Arch-Res Design 72.00% 66.60% 74.40% 71.00%Electromechanical 61.90% 58.80% 64.50% 61.73%Laser Tecnology 58.30% 68.50% 59.10% 61.97%Industrial Engineer 67.20% 67.30% 48.00% 60.83%Farm Business 39.40% 72.90% 67.10% 59.80%Police Recruit 80.40% 79.30% 79.20% 79.63%Nursing Assistant 69.90% 62.10% 47.60% 59.87%EMT 67.70% 84.60% 77.40% 76.57%IPS 38.70% 40.70% 41.60% 40.33%Inventory Control 24.40% 33.40% 46.30% 34.70%Auto Body/Paint 59.90% 60.70% 77.50% 66.03%MechanicaVComput 47.60% 38.20% 42.30% 42.70%Welding 47.00% 55.60% 52.80% 51.80%Surgical Tech. 73.70% 78.40% 59.40% 70.50%Agri Power & Equip. 30.70% 29.50% 18.00% 26.07%Automotive Tech. 67.50% 96.40% 72.70% 78.87%Machine Tooling 52.90% 77.20% 80.70% 70.27%

TOTALS/AVGS 61.59% 66.83% 63.74% 64.05%

111.1 Course Utilization (65%)

17

1 2 3 4 5 6 7 9 10 11 12 13 14 15 18 17 18 19 20 21 22 23 24 25 28 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

RE

TE

NT

ION

/GR

AD

UA

TIO

N R

AT

E

Acc

ount

ing

111.

4 R

ekni

ton

/Gra

duat

ion

1992

-199

319

93-1

994

1994

-199

5A

vera

geR

ail

Bas

eC

ontin

uing

Gra

duat

esR

ate

Bas

eC

ontin

uing

Gra

duat

esR

ate

Bas

eC

ontin

uing

Gra

duat

esR

ats

Bas

sC

ontin

uing

Gra

duat

es

136

2435

43.3

8%'

117

4411

47.0

1%0

00

ER

R25

368

4845

56i

i;

Mar

ketin

g75

1120

41.3

3%69

1115

37.6

8%0

00

-E

RR

144

2235

39.5

8%

I Sec

reta

ry28

110

42.3

1%21

69

71.4

3%0

00

ER

R47

719

55.3

2%58

.331

.6

Mic

alex

sta

310

1754

.84%

294

1482

.07%

00

0E

RR

604

31

Adr

rdn.

tant

ryA

ssis

tant

282

la'

64.2

9%26

310

50.0

0%0

00

ER

R54

'5

2657

419

6

CIS

Pro

gram

mer

734

1830

.14%

6023

848

.33%

00

0E

RR

133

2724

38 3

5%47

97%

Sm

all B

ull M

gt.

688

2548

.53%

6522

47.2

7%0

00

ER

R12

330

29

Insu

ranc

e S

ervi

ces

210

1047

.82%

92

355

.56%

00

0E

RR

302

,13

50.0

0%

Su

Mgt

.29

121

44.8

3%24

111

50.0

0%0

00

ER

R53

232

47 1

7'5

Prit

ellv

nily

sPul

!liki

ngtn

373

1754

.05%

3010

63.3

3%0

00

ER

R67

1323

53 7

3%

Pol

ice

Sci

ence

382.

.18

52.6

3%29

313

55.1

7%0

00

ER

R67

531

53 7

3%79

459

6

Den

tal F

iygl

ene

240

2187

.50%

4918

1975

.51%

00

0E

RR

7318

40

Nur

sing

#41

378

558

80.7

7%65

3222

83.0

8%0

00

ER

R14

337

8081

82%

Rad

iogr

aph

202

1185

.00%

1810

55.5

6%0

(10

ER

R38

1211

60 5

3%

Edu

Into

or T

eo36

115

44.4

4%30

121

43.3

3%0

00

ER

R66

1316

43 9

4%

Ele

ctro

n ea

200

1155

.00%

257

340

.00%

00

0E

RR

457

1446

67%

48 6

5%

Mec

hani

cal D

esig

n17

44

47.0

6%20

84

50.0

0%0

00

ER

R37

108

Arc

h-R

es D

esig

n67

317

29.8

5%61

1616

52.4

6%0

00

ER

R12

819

3340

63%

Ele

ctro

mec

hani

cal

252

1048

.00%

377

1559

.46%

00

0E

RR

629

2554

84%

Lam

Tec

nolo

p1I

32,

212

43.7

5%24

110

45.8

3%0

00

ER

R56

322

44 6

4%42

119

1

Indu

stria

l in

neer

133

238

.46%

63

50.0

0%0

00

ER

R19

62

farm

143

450

31.4

7%10

047

48.0

0%0

00

ER

R24

392

138

.27%

Pon

ce R

ecru

it51

142

84.3

1%62

045

72.5

8%0

00

ER

R11

31

137

77 8

8%

Nur

sing

Ass

ista

nt20

4,

017

485

.29%

178

115

288

.93%

00

0E

RR

380

132

686

05%

EM

T0

00

ER

R85

065

78.4

7%0

00

ER

R85

065

76 4

7%

IPS

115

342

39.1

3%11

913

4648

.74%

00

0E

RR

234

1687

44 0

2i-6

Inve

ntor

y C

ontr

ol14

04

28.5

7%21

18

42.8

6%0

00

ER

R35

112

37 1

4%

Aut

o B

ody/

Pai

nt0

00

ER

R0

00

ER

R0

00

ER

R0

00

ER

R

Mec

hani

cal/C

ompu

i15

.0

853

.33%

113

36.3

6%0

00

ER

R26

39

4615

%

Wel

ding

300

1756

.87%

330

2183

.64%

00

'0

ER

R63

038

60 3

2%a

44%

Bur

gleA

Tec

h.22

011

50.0

0%23

09

39.1

3%0

00

ER

R45

020

ft( P

ower

I &

MP

.14

06

42.8

6%7

12

r-42

.86%

00

0E

RR

211

842

86%

Aut

omot

ive

Tec

h.32

115

50.0

0%24

38

37.5

0%0

00

ER

R56

421

44 6

45,

Mac

hine

Too

ling

181

1088

.75%

222

1159

.09%

00

0E

RR

383

2163

.16%

55.5

696

TO

TA

LS/A

VG

S16

5014

067

76

.71%

1487

3264

868

.61%

00

0E

RR

9037

482

1225

111.

4 R

eten

kon/

Gra

dusk

on (

70%

)

BE

ST

CO

PY

AV

AIL

AB

LE19

1

234567891011

12131415161718192021222324252627282930313233343536373839404142434445

FACULTY UTILIZATION

111.5 Faculty Utilization1992-199 1993-199 1994-199 Average

Accounting 15640 16518 15258 15805

Marketing 15364 14733 14578 14892

Legal Secretary 14683 15071 14890 14881

Medical Secretary 15000 17829 14592 15807

Admin. Assistant 13556 15161 12981 13899

CIS Programmer 13751 13619 12553 13308

Small Busi Mgt. 12835 13740 12735 13103

Insurance Services 12756 13528 10351 12212

Supervisory Mgt. 10684 16675 11419 12926

Printing/Publishing 11748 11312 10348 11136

Police Science 17311 17176 15373 16620

Dental Hygiene 8434 8651 9770 8952

Nursing AD 9796 9735 9077 9536

Radiography 14776 14576 11583 13645

Edu Interpreter Tec 10615 9826 10781' 10407

Electronics 11752 13223 10572 11849

Mechanical Design 9763 10503 11621 10629

Arch-Res Design 14762 12809 14201 13924

Electromechanical 11814 12505 11807 12042

Laser Tecnology 11082 12193 10728 11334

Industrial Engineer 12494 2847 8386 7909

Farm Business 13586 12103 12288 12659

Police Recruit 15205 14241 11973 13806

Nursing Assistant 16726 10473 13050 13416

EMT 11434 11681 14743 12619

IPS 14697, 14204 9764 12888

Inventory Control 342 10392 13921 8218

Auto Body/Paint 8641 11714 12019 10791

Mechanical/Comput 8815 8786 9072 8891

Welding 10383 12686 11155 114Q8.

Surgical Tech. 8301 8442 6826 7856

Agri Power & Equip. 7620 10479 6753 8284

Automotive Tech. 14585 15419 12031 14012

Machine Tooling 10231 12784 12874 11963

TOTALS/AVGS 13089 12877 12121 12696

111.5 Faculty Utilization (12,000 hrs)

20

ESIGNING ANDNTING VALUE

FOCUSED EFFECTIVENESSINDICATORSJUNE 23,1997 / TEMPE, AZBECKY RAJEK, ARVES JONES,DAN McCOLLUM, TOM GOLTZNORTHCENTRAL TECHNICALCOLLEGE, WAUSAU, WISCONSIN

WHY VALUE FOCUS? (page 2)

t CARVER POLICY GOVERNANCE MODEL* WHAT SERVICE

TO WHOMAT WHAT COST

STATE MANDATES FOR COST CONTROL* CUSTOMER FOCUS

NTC MISSIONNCA - DOING WHAT WE SAY WE AREDOING

NTC VISION

NTC is the college of choice, anticipating andexceeding customers' expectations withexcellence

BEST COPY AVAILABLE

WHY VALUE FOCUS?ULTAXPAYERMOOD_

a* PRISONSk WELFARE REFORM* LOW TAXES*- ACCOUNTABILITY

0 QUALITY AT A REASONABLE COSTEl BETTER, FASTER, LOW COST PROVIDER

EDUCATION IS A MATURE INDUSTRY

NTC MISSION

Northcentral Technical College provides qualityeducation, training, and services of high valueto individuals, business and industry, andorganizations, building a competitive workforce in a changing global society.

WHO ARE THEY?

sTUpur

ttiecusititigiu

COMMUNITY EMPLOYER.

21

WHAT DO THEY BUY?

HOW DOES THE COMMUNITYBENEFIT ?

COMPETITIVEWORKFORCE

LEADERSHIP

OF LI

COMMUNITY

ECONOMY

BEST COPY AVIAiLAbi,

WHAT DO THEY BUY?

HOW DO THE STUDENTSBENEFIT?

STUDENTS

SKILLS

KNOWLEDGE

LIVINGWAGE

HOW DO THE EMPLOYERSBENEFIT?

COMPETITIVEWORKFORCE

KNOWLEDGE

EMPLOYERS

SKILLS

227-12

GUIDELINES FOR DEVELOPMENT OFPROGRAM EFFECTIVENESSINDICATORS

21 MUST BE OF HIGH QUALITYMUST BE OF HIGH VALUEMUST BUILD A COMPETITIVEWORKFORCE

Li CUSTOMER DRIVENMUST BE ADAPTABLE

El MEASURABLE AND MANAGEABLE

DEVELOPMENT

STRONG FACULTY INVOLVEMENTLI BOARD GOAL

TRAITS

LI EMPLOYER SATISFACTIONEl FUTURE DEMANDit 5 YEAR FOLLOW-UP

STUDENT ACCESSIBILITYHIGH STUDENT DEMAND

BEST COPY VA LA

PURPOSE AND OBJECTIVES

DEVELOP CLEAR, MEASURABLESTANDARDS

El ELIMINATE THE "RANKING" OFPROGRAMS

Li PROVIDE CLEAR DIRECTION TOFACULTY AND STAFFEMPOWER FACULTY TO PROACTIVELYWORK TO IMPROVE THEIR PROGRAMS

MEASURES

LI STUDENT SATISFACTION RATE11 STUDENT SATISFACTION RATE

INCLUDING "LEAVERS"Li GRADUATE SALARY COMPARISONLi RELATED PLACEMENTLI RELATED PLACEMENT WITH "LEAVERS"al COMMUNITY ECONOMIC RETURNLi COURSE UTILIZATION

RETENTION/GRADUATIOKRATE.., . ..

FACULTY 8 . ..... '."'

IMPLEMENTATION

MEASURES DISTRIBUTEDLi TRAINING PROVIDEDEl FLEXIBLE FEEDBACK

2313-18

USES

.E1 PROGRAM SCREENINGLi IDENTIFY PROGRAMS NEEDING

IMPROVEMENTBUILD FACULTY UNDERSTANDING OFCOST AND BENEFIT

Li BECAME BASE TO IMPROVE PROGRAMMIX

VALUE RATIO

txptsams,

,VtOgP"

THE POWER OF THE VALUE RATIO

Li SELECT THE BENEFIT THAT ISIMPORTANT TO THE CUSTOMER ANDTHE MISSION OF THE COLLEGE

LI POSITIVELY COMBINES SEVERALTRADITIONAL INDICATORS

0 THE HIGHER THE RATIO THE MOREPOSITIVE THE BENEFIT TO THECUSTOMER

Li EXCELLENT TOOL TO IMPROVEPROGRAM

EST COPY AVAILI46LL

VALUE FOCUS

VALUE IS THE ESTIMATE OF WORTHEXPRESSED IN MONEYVALUE equals BENEFIT divided by COST

_I VALUE IS CUSTOMER FOCUSEDJ VALUE IS CUSTOMER DEFINED

NTC VALUE FORMULA - COMMUNITYECONOMIC RETURN

2j VALUE EQUALS THE TOTAL SALARY OFEMPLOYED RELATED GRADUATESDIVIDED BY THE DIRECTINSTRUCTIONAL COST

THE POWER OF VALUE

ALLOWS STAFF TO SELECT THE MOSTIMPORTANT AREAS TO IMPROVE03, REDUCE COSTkt. INCREASE RECRUITMENTttf INCREASE RETENTION

IMPROVE WORTH OF GRADUATESt. ETC.

2 419-24

THE POWER OF VALUE

LI USES MEASURABLE DATA BUTFOCUSES ON THE FUTURE

ANTICIPATE THE NEEDS OF THECUSTOMEREXCEED THE NEEDS OF THE CUSTOMER

t. LOW COST PROVIDERJUST IN TIME DELIVERY

te. ENCOURAGES PARTNERSHIPSJ FLEXIBLE

OUTCOMES

EACH PROGRAM RECEIVEDCONSTRUCTIVE FEEDBACK

J NINE PROGRAMS IDENTIFIEDLi STATE MODEL

EMPOWERMENT HAPPENS

BEST COPY *MOLE

EXAMPLES OF PROGRAMIMPROVEMENT

_I RESULTS EXCEEDED EXPECTATIONSEITP CASE STUDY

Is. INFORMATION PROCESSING SPECIALISTREDESIGNS CURRICULUMAUTOMOTIVE TECHNICIAN PROGRAMEXPLORING NEW PARTNERSHIPS

OPPORTUNITIES FOR CHANGE

PROCESS NEEDS TO RECOGNIZEPROGRAMS THAT WORK WITH SPECIALPOPULATIONSPROCESS DOES NOT ADEQUATELYRECOGNIZE WORK IN PROGRESS

d MEASUREMENTS OF CUSTOMERSATISFACTION NEED TO BESTRENGTHENED

J PROGRAMS NEED TO BE PROVIDED WITHADDITIONAL ACCESSTOENVIRONMENTAL.SCANNING AND,LABOP fORCEDATA

:::::::::::

25 25-30

U.S. Department of EducationOffice of Educational Research and Improvement (OERO

Educational Resources Information Center (ERIC)

REPRODUCTION RELEASE

I. DOCUMENT IDENTIFICATION:

(Specific Document)

0

IC3-6 9io +2-1

Title:

Designing and Implementing Value Focused Effectiveness Indicators

Author(s): Dan McCollum, Becky Rajek, Arves Jones, Tom Goltz

Corporate Source: Publication Date:

II. REPRODUCTION RELEASE:

In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announcedin the monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproducedpaper copy, and electronic/optical media, and sold through the ERIC Document Reproduction Service (EDRS) or other ERIC vendors. Credit isgiven to the source of each document, and, if reproduction release is granted, one of the following notices is affixed to the document.

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following two options and sign atthe bottom of the page.

XXtCheck here

For Level 1 Release:Permitting reproduction inmicrofiche (4" x 6' film) orother ERIC archival media(e.g., electronic or optical)and paper copy.

Signhere-+please

The sample sticker shown below will be

affixed to all Level 1 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL

HAS BEEN GRANTED BY

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Level 1

The sample sticker shown below will beaffixed to all Level 2 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS

MATERIAL IN OTHER THAN PAPERCOPY HAS BEEN GRANTED BY

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Level 2

Documents will be processed as indicated provided reproduction quality permits. It permissionto reproduce is granted, but neither box is checked, documents will be processed at Level 1.

Check hereFor Level 2 Release:Permitting reproduction inmicrofiche (4" x 6' film) orother ERIC archival media(e.g., electronic or optical),but not in paper copy.

'I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminatethis document as indicated above. Reproduction from the ERIC microfiche or electronic/optical media by persons other thanER employees and its system contractors requires permission from the copyright holder. Exception is made for non-profitre oduc n by libraries and other service agencies to satisfy information needs of educators in response to discrete inquiries.'

Sign

.6rganization/Address:

Northcentral Technical College1000 W. Campus DriveWausau, WI 54401

Printed Name/Position/Title:Daniel R. McCollum.Raagagsh .§1.1.1.Annkozg_CoordinatorTelephone:

( 715 ) 675-3331 (715) 675-9776Address: Date:

mccollum@northcpntral.tec.wi.us

III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE):

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source,please provide the following information regarding the availability of the document. (ERIC will not announce a document unless it Ispublicly available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria aresignificantly more stringent for documents that cannot be made available through EDRS.)

Publisher/Distributor:

Address:

Price:

IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER:If the right to grant reproduction release is held by someone other than the addressee, please provide the appropriate name and address:

Name:

Address:

V. WHERE TO SEND THIS FORM:

Send this form to the following ERIC Clearinghouse:Jonathan Kelly

ERIC Clearinghouse forCommunity Colleges

3051 Moore HallBox 951521

Los Angeles, CA 90095-1521

However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document beingcontributed) to:

(Rev. 3/96/96)

Institutional Effectiveness andStudent Success in the CC

June 22-25, 1997