Assessing Student Affairs Processes: A pilot study

International Assessment and Retention Conference - 2007

Josh BrownLiberty University

Greg McCurdyCentra Health

Mark DavisCentra Health

Overv

iew

• What we did• What resulted• What we’re doing• What you can do

What w

e d

id

Context

– Assessment at Liberty University divided into Curricular & Co-curricular responsibilities

– Attained varying levels of assessment• Frequency – attendance, cost, etc.• Satisfaction – locally developed instruments• Satisfaction with GAP analysis (Noel Levitz

SSI)• Engagement (NSSE)• Focus Groups

• Process Analysis– Process Engineering, Six Sigma, ISO 9000

What w

e d

id

Six Sigma

• Roots of Six Sigma can be traced to Carl Frederick Gauss (1777-1855) as a measurement standard with the normal curve

• Walter Shewhart, in the 1920’s, used six sigma as a measurement standard in product variation

• Bill Smith receives the credit for coining the term “six sigma” while working as an engineer with Motorola

• In the early 1980’s, Motorola chairman, Bob Galvin, desired a measurement by which defects per million opportunities could be shown and the after effect resulted in $16 Billion in savings

• Since then, companies such as Honeywell (Lawrence Bossidy) and GE (Jack Welch) adopted the six sigma method as a means of doing business, not just a quality management tool like TQM (W. Edwards Deming)

What w

e d

id

Six Sigma Process: DMAIC

• Define problem from the voice of customer (V.O.C.)

• Measure extent of problem by collecting data to be able to create metrics

• Analyze data for sources of variation

• Improve process by addressing root causes, identify high-impact benefits

• Control processes through continuous improvement mechanisms

What w

e d

id

Step One: Define

• Define problem from the voice of customer • Directive came from VPSA:

– “We need to streamline the judicial life process.”

DMAIC

VP for Student Affairs

Dean of Men

Campus Recreation

Student Housing(Resident/Commuter)

Dean of Women

Parent Programs

Center4ME

Army ROTC

Student Leadership

Administrative Assistant

DIVISION OF STUDENT AFFAIRS

Campus

Programming

What w

e d

id

Step Two: Measure

• Measure extent of problem by collecting data in order to create metrics

• S.I.P.O.C. - a six-sigma tool, will be utilized to create metrics for analysis– Suppliers– Inputs– Processes– Outputs– Customers

DMAIC

What w

e d

id

SIPOC: Suppliers

• Conducted inquiry sessions with all levels of persons in the judicial process:– Session One: RA’s & RD’s– Session Two: Associate Deans

(DOM/DOW)– Session Three: Head Deans and VPSA– Session Four: Students who

experienced the judicial process at various levels

– Session Five: Administrative Assistants, Secretaries, and Student Workers overseeing data entry

DMAIC

What w

e d

id

SIPOC: Inputs

• Student Handbook• Violation & Incident reports• Data entry at RD level• Res Life staff: manually sorting reports• “Why do we need to process warnings?”• Difference between practice and policy:

confusion of appeal process• “There are too many hand-offs of

paperwork.”• “We handle data differently than the other

office.”

DMAIC

What w

e d

id

SIPOC: Outputs

• Lack of communication of appeals• Appeal process is slow/inconsistent• “I am not sure of the process.”• Not enough qualified counselors on

campus• Differing approaches: men-discipline,

women-counsel• Dean on-call schedule is confusing as it

varies too frequently• Fines are confusing and don’t seem to be

achieving their intended purpose• Too many logs! (cont.)

DMAIC

What w

e d

id

SIPOC: Outputs

• RA Official Correspondence Log • Call Slip Log• Non-Return Log • Permission Slip Log• Violation Report• Incident Report – Residence Hall• IR-Type Log• Case Load Log• Discipline Community Service Log (twice)• FERPA Log• Probation Log• AW Log• Student File Database• File Log (who has what)• Self-Reports Log• No Contact Agreement Log• Permission Restriction Log

DMAIC

What w

e d

id

SIPOC: Customers

• Students• Student Leaders: RA/RD/Deans• Res Life• Dean of Men & Dean of Women• VPSA• Sodexho – community service• LUPD• Counselors • Faculty/Staff• Campus Pastors

DMAIC

What w

e d

id

Step Three: Analyze

• Analyze data for sources of variation• Three analyses conducted:

– Process Maps – this is the “P” in SIPOC processes & is implemented at this stage

– Fishbone Analysis– SWOT Analysis

DMAIC

What w

e d

id

Liberty University Judicial Process Cross-Functional Flowchart

Violation & Incident Reports, S.I.P.O.C.

Level Two6-17 Reps RD

Level Three18-29 Reps AD

Level Four30+ non-dismissal

DRC

Level Five30+ Auto Withdrawal,

JRB/VP

Level One0-4 Reps RA

Presented by Greg McCurdy

RA issues ViolationReport to student,

copies to RD; IncidentReport to RD & RLOby 8 a.m. next day

RA completes IncidentReport, emails RD &sends to RLO by 8a.m. the next day

DOM Administrative Assistantenters IR data by Assistant Dean& IR #; DOW Administrative staffenters data by type & RH in Excel

& print.

RLO AD sortsall incoming

IRsforwarding

judicial IRs toDOM/DOW

office

If quick bypassresponse is warranted, RD calls

Dean-on-Call 30+ reps

RD meets weeklywith RAs to uphold/

overturn VRappeals; RD sends

to DOM/DOWoffice each Monday

If studentappeals in writing

(VR copy) w/i 48 hrs,RD handles appeals

0-12 reps, no staff claritywith 12-17 reps caseappeals: RD or AD?

[Only one appealallowed, nofurtherance]

RD investigatesIR, meets with

student, decidescase with VR &appropriate fine;

enters inpersonal Excelor file system

conflict ofinterest

DOM/DOWAdministrativeAssistants run

Probation Reporteach Monday for

students w/ 18+ reps

AdministrativeAssistants (DOM/DOW)

assign AD by evencaseload distribution;arrange DRC; judicial

counseling

DOM/DOWSecretary sends

call slips tostudent, RA,RD & RLOfor judicial

appointmentwith AD

If no responsefrom student to callslip, DOM remove

student fromconvocation; DOW

block student’scomputer account

AD acts asTRIAGE for 18-

29 & 30+ &<<<appeals

from RD,investigatingeach case,

deciding on 18+cases

Student fine w/ DCS;probation documentationplaced in official records

Dean’sReview

Committee(DRC) meetsto decide 30+

cases

DRC:AD,RLOAD,

Dean &VP

Conflict ofinterest:

VPSA/DOSchairs both

DRC &JRB

Student finedappropriately by DOM/DOW; placed in official

records

Student finedappropriately by DOM/DOW; placed in official

records

On-call DOW seeall cases

(commuters aswell); On-call DOM

pass case toAdministrativeAssistant who

passes it to anotherAD on Monday a.m.

Ifstudent appeals

in writing to Deanw/i 48 hrs, Dean will

decide appeal?

AdministrativeAssistant to VPSA

arranges JRBmeeting

RA completes IncidentReport, emails RD &sends to RLO by 8a.m. the next day

Student rcvs DCS + fine OR

Administrative Withdrawal and/or

non-return

RA completes IncidentReport, emails RD &sends to RLO by 8a.m. the next day

If student appealsdecision by DRC in

writing to VPSAwithin 24 hrs

The JudicialReview

Board meetsto decide thefinal appeal

case

JRB: 2 SGA, 3faculty. Associate

Dean presents case& VPSA chairs;

neither AD or VPvotes

VPSAreviews all DRC

decisions; can offeralternativediscipline

Majority vote decidesAdministrative Withdrawal

and/or non-return ORoverturn & alternative

discipline

BOTTLENECKSin highlightedboxes: RLO,

DOM/DOW & ADs

RD enters datain personalExcel or file

system

Alternative discipline;documentation in official

records

DMAIC

What re

sulte

d

DMAIC

Liberty University Judicial Process Cross-Functional Flowchart,Recommended Schematic - Greg McCurdy, 30 December, 2006

Level Two18-29 Reps ADs

Level Three30+ Reps Deans

Level FourVPSA/JRB

Level One0-4 Reps RAs/Peers

6-17 Reps RDs

RA issues 0-4 RepViolation Report to

student (hard copy), submits copy to RD

RD meetsweekly with RAs

to uphold/overturn VRappeals (SAeducational

development)

RD enters VRdata in new SA

judicialsoftwaredatabase

Centralized Office/Database of DOM/DOW/SLO Administrative Assistants/Secretaries canpull up any judicial data necessary from newsoftware system; issue appropriate fine via

interface connection with the student accountsoffice; organize files in database; prepare

template documents for DisciplinaryCommunity Service letters, Sodexho status

reports; Call Slips emailed to students

Student account will beupdated with VR fine

RA completes IncidentReport via new software

system; RD andCentralized DOM/DOW/

Student Life Officeautomatically notified; IRdata is stored in system

database

RDinvestigatesIR, meets w/

student &decides case

If student appeals inwriting to RD within

48 hrs, apredetermined RH

Student PeerGroup will decide

the appealpresented by RD

(RD does not vote)Student Affairs

educationaldevelopment opp!

Decision by print-outletter final: Fine or

overturn w/ warning

RA completes IncidentReport via new software

system; RD andCentralized DOM/DOW/

Student Life Officeautomatically notified; IRdata is stored in system

database

RA completes IncidentReport via new software

system; RD andCentralized DOM/DOW/

Student Life Officeautomatically notified; IRdata is stored in system

database

RD callsDean-On-Call for

emergent IR cases30+

Administrative Assistantsassign/schedule AD judicialcase load distribution basedon Probation Report for 18-

29 Reps; 30+ Reps to DOW/DOM-Head Dean

AD investigates,researches, meets

w/ students,decides case

Dean/DOM/DOW(or Council of all

three voting)investigates,

researches, meetsw/ students,decides case

Decision by print-outletter final: DCS + fineor overturn w/ warning

If studentappeals inwriting to aDean within48 hrs, Deanwill decide

appeal

Decision by print-out letterfinal: DCS + fine OR

Administrative Withdrawaland/or non-return

If a student appealsin writing to VP ofStudent Affairs

within 48 hours (timeconsistency),

the Judicial ReviewBoard will decide

appeal

Majority vote decides touphold Dean’s Council

decision: AdministrativeWithdrawal and/or non-return; DCS + fine ORoverturn & alternative

discipline offered

AdministrativeAssistant to

VP of StudentAffairs

arranges/schedules

JRB meeting

JRB: Deanpresents case

& VPSAchairs; 3faculty, 2

SGA & VPSAvote (Deandoes not

vote)

For studentswho have beenAdministratively

Withdrawn throughjudicial process, VP

of Student Affairs (&Dean of Students)review for Reapply/

Readmitstatus

Admissionsrequests Student

Affairs feedback forreapply/readmit

Basic FlowchartShapes

document

Storeddata

Predefinedprocess

Directdata

Manualoperation

terminator

Parallelmode

card

control

process

decision

What re

sulte

d

DMAIC

ImproveEfficiencyof JudicialWorkflow

Manpower (staff) Management

Method Mother (Human) NatureMachines

Materials

Human error

Paper gets lost

Manual work flow

Several levels/offices(Handoffs)

Data storage not linked:RA/RD/RLO/DOM-DOW

Paper filing system

Privacy a factor(FERPA)

RA/RD staff are notprovided PCs

Judicial staff use differentforms of data storage:

hard copy vs. electronic

Redundant data storage

Overburdened w/processing warnings

Education ofjudicial staff

filling out reports

Concerned w/ majorIncident Reports

Crisis issues involvingtime for student

counselingdocumentation

Keeping many logs

RLO to DOM/DOWbottlenecks

Paper reportsvs. electronic?

Scan documents?

Hard copy call slips, RA:“finding student difficult”

Email call slips used forcommuter students (DOW)?

Differing methodsbetween DOM & DOWoffices (consistency)

Data entry staff hours:Women 40 hrs week

Men 55 hrs week

Filing staff hours:Women 15-20 hrs week

Men 18 hrs week

RLO AD sortersoverloaded?

Volume of business emails:RAs 3-5 per day avg

RDs 20-50 per day avg

RDs share two PCsamong 21 RDs

RAs complete emails in computer lab; potential FERPA

violations?

Flow of reportsup & down the chain?

Consistent and timely?

Need a database with:Accessible & Real-time

information

Fishbone Anal ysis:Liber ty University

Judicial Process

TitleBased on feedback from staffvia the S.I.P.O.C. qualitativeDefine & Measure steps, thefollowing key improvementarea became a focal pointfor Analysis.

Timeliness of handlingappeals a concern?

What re

sulte

d

SWOT Analysis

• Strengths– Skilled staff– Judicial process affords student appeal– Education of student handbook

• Weaknesses– Communication breakdown– Inconsistent processes– Lack of technology to integrate

processes– Paper workload with many hand-offs

DMAIC

What re

sulte

d

SWOT Analysis

• Opportunities– Software integration upgrade– Office PC’s interconnect all Student

Affairs– Educational development through

residence hall Peer Judicial Councils

• Threats– Reactive vs. proactive– Legal aspects: FERPA– Overstressed staff, burnout, and

turnover

DMAIC

What re

sulte

d

Step Four: Improve

• Improve process by addressing root causes and identify high-impact benefits.– Critical-to-success-factor chart – Prioritizing benefits and efforts – Final recommendations

DMAIC

App

licatio

n

• You and your group members have been hired by Liberty University as judicial consultants to remedy this process.

• For the next few minutes, use the collective knowledge and experience of your group to provide at least four recommendations for the university to improve its judicial processes.

• Please place your recommendations on the provided note cards.

App

licatio

n

Critical-to-success factor chart

Ideas People

Service

Efficient

Cost Total

1

2

3

4

What re

sulte

d

Total

729

3,645

405

6,5619999Streamline theJudicialProcesses &Workflow

1599CounselingCenter withQualified Staff

5999Restructure &

CentralizeJudicial System

1999Student AffairsJudicial Software& HardwareIntegration

CostEfficiencyServicePeopleIdeas

Prioritizing Critical Success Factors

3

2

1

4

High = 9

Medium = 5

Low = 1

Critical-to-success Factor Chart

DMAIC

What re

sulte

d

EFF

OR

T

BENEFITLOW

HIGH

HIGH

4th Proposal

2nd Proposal

1st Proposal

3rd Proposal

PrioritizingBenefit &

Effort

Prioritizing Benefit & Effort

DMAIC

What re

sulte

d

Final Recommendations

• Acquire a centralized student database that can integrate judicial operations

• Streamline judicial process and structure

• Eliminate conflicts of interest in the current process

• Involve students in the appeal process

• Equip the division of SA with the necessary qualified counselors

DMAIC

What re

sulte

d

Step Five: Control

• Control processes through continuous improvement mechanisms: – Formulate action plans for

implementing strategies– Establish an ongoing QA program

DMAIC

What w

e’re

doin

g

• Since the conclusion of the Six Sigma judicial study, Student Affairs has begun the following for a Fall 2008 implementation:

– Purchased a new judicial software package– Created & implemented a student court for

judicial appeals– Revised judicial organizational chart– Redefined and clarified roles (as result of above)– Eliminated policies from student handbook– Created policies from student handbook

What y

ou ca

n d

o

Six Sigma Tips For Educators

1. Know your customers• Identify them (SIPOC)• Listen to them (VOC)• Understand and define their needs

(CTQ)

2. “Know thyself”• Examine your processes (SIPOC /

mapping)• Measure your performance (baseline;

DPMO; Sigma; statistics)

What y

ou ca

n d

o

Six Sigma Tips For Educators

3. Know what to do next• Get to the roots (fishbone; hypothesis

testing; VA/NVA)• Define the ideal state (gap analysis)• Brainstorm your opportunities (SWOT;

prioritization matrix)• Drive change (force-field analysis)

4. Know how to do it• Decide on your method (project vs. go-

do)• Open the toolbox• Start with what you have

What y

ou ca

n d

o

Recommended Resources

• Academic– Assessing Organizational Performance in Higher

Education (Miller, 2007) http://www.josseybass.com/WileyCDA/WileyTitle/productCd-0787986402.html

– Continuous Process Improvement in Higher Education (Inozu & Whitcomb, 2007) http://www.novaces.com/pdfs/CoF_NovacesWhitePaper_r1std.pdf

– Process Improvement to Achieve Institutional Effectiveness (Lake, 2005) www.ncci-cu.org/Visitors/Documents/processimprovement070905AC.ppt

• Business– Six Sigma for Dummies (Gygi, DeCarlo, Williams

& Covey, 2005) – The Six Sigma Way: How GE, Motorola, and

Other Top Companies are Honing Their Performance (Pande, 2000)

Pre

sen

ter B

ios

• Josh Brown is currently the Associate Director of University Assessment for Liberty University, coordinating the assessment of all co-curricular departments. He possesses an earned Master's of Student Development from Azusa Pacific University. Email – jtbrown@liberty.edu

• Greg McCurdy is currently the manager of the Radiation Oncology Department at Centra Health, where he utilized the six sigma philosophy and instruments to hone difficult processes in a medical setting for increased workflow efficiency. He is concluding his Master's of Higher Education at Geneva College. Email – McCurdysrus@juno.com

• Mark Davis is currently a process engineer with Centra Health, where he is assisting with the implementation of a system wide healthcare improvement initiative called CH2. He holds a degree from William & Mary and a Six Sigma Black Belt from Villanova. Email – Mark.Davis@centrahealth.com