A Comprehensive Process Linking Accreditation and

Long-term Assessment Efforts

Bill NealHelena Hannonen

Johnson MakPaul Freebairn

2009 AIR Forum, Atlanta, GAJune 2, 2009

Brigham Young University–Hawaii

• Small, 4-year comprehensive university

• About 2400 students

• Highly international campus

• Programs in art, business, computing, education and science

• Located near Oahu’s north shore

Our presentation today…

1. Nuts and Bolts of BYUH’s Assessment Process

2. Department Portfolio (a work in progress)

3. Impact of Assessment at the Department Level

4. Linking Assessment to Accreditation

5. Sharing “Best Practices”

6. Summary and Q&A

Institutional Change is ChallengingClick on the box below to view movie.

PART 1BYUH’s Assessment Process

• 1996 Accreditation & 2000 Interim Report

• Realignment– Director of University Assessment & Testing (Fall 2000)

• Search for an assessment model

• Nichol’s visit to campus (March 2001)

• University Assessment Committee (UAC) – Organized in June 2001

• Now in our eighth assessment cycle

Campus-wide assessment begins(Continuous Process)

• Searched for simplified processes and documents

• Developed an online website and portfolio system to showcase results

Accreditation Documents & Processes

Assessment Model Criteria

• Short

• Simple

• Mission-driven

• Results focused

• Leads to continuous improvement (on-going)

• Used for academic and administrative units

BYUH Annual Assessment Plan(Template)

Faculty/Staff Engagement

• Nichols workshops training (2001)

• Meetings with key faculty & admin. Leaders• Accreditation (faculty survey, focus groups & writing team)

• Assessment Website– Assessment Guidebook

– Other online resources

• Mary Allen workshops (2005)

• Assessment/accreditation Conferences & Workshops• “Best Practices” (awards, luncheon & posters)• Assessment Budget

BYUH Multi-year Assessment Plan(Template)

University Assessment Committee

• 12 members (including 4-6 faculty)

• Advisory capacity (our strength)

• Each member helps 3-4 other departments• Simplified assessment plan template• Sharing the rubric in advance (what to expect)

• Accountability– annual report

– 30-minute presentation

– What did you learn? What actions did you take?

• January to March (2-3 review teams)

Developing & Using a Rubric

• A way to formalize feedback to departments

• Last year’s results

• Next year’s outcomes & means of assessment

• Linked to the multi-year assessment plan

Developing & Using a Rubric

PART 2Assessment Website

andOnline Department Portfolios

Portfolio Development

• Internship (Web programmer)

• Input from departments

• Depository for key elements

• Tied to Faculty Portfolio system

• Access to key documents

• Display for accreditation

Key Portfolio Elements

• Assessment Plan• Multi-year Assessment Plan• Assessment Rubric• Outcomes and Matrix—Course Alignment • Self-study/Department Review• Syllabi (course alignment)• Sample Student Work/Displays• Reflection and Usage of Data• Faculty Profile

PART 3Impact of Assessment(at the Department Level)

BUSINESS MANAGEMENT

Mission Statement

Develop principled and innovative business leaders for Asia, the Pacific and beyond.

• Number of students: 518 from 48 countries– BUSM 362 (246 in January 2008)– HMT 156

• Number of full-time faculty:– BUSM 8– HTM 3

• Majors in Bachelor’s degree:– International Business Management– Hospitality and Tourism– Business Management

• Developed A.S. in Business Management in 2008-09

Introduction

BUSINESS PREREQUISITE COURSES

BUSINESS FUNDAMENTALS

BUSINESS TRACKS

Business Management Major

FINANCE HR/OB

OPERATIONS

MARKETING

GENERALBUSINESS

Business Management Tracks

1. Demonstrate a knowledge of fundamental areas of business

2. Have a knowledge of international business fundamentals

3. Have a knowledge of inter-cultural understanding basics

4. Demonstrate commitment of ethical behavior

5. Be able to solve problems

6. Work effectively in teams

7. Communicate effectively

8. Demonstrate commitment to service-mindedness

9. Demonstrate the ability to lead

10. Be able to prepare a business plan.

Assessment Outcomes: Students will . . .

• Multi-year plans cover 8-10 years– Quick matrix which summarizes outcomes and means of

assessment

• Annual plans focus on 2-3 outcomes– Detailed matrix which covers outcome, means of assessment,

findings and actions

• Outcomes are rotated every year– Report results and findings

– Plan for the upcoming year

Annual and Multi-Year Plans

• Reinstituted the national Major Field Test in 2008 (administered in capstone course)

• Added a management course to IBM major in January 2008

• Discontinued Integrated Business Core in December 2008 (refocused teaching on business fundamentals)

Outcome #1:Demonstrate a knowledge of fundamental areas of business

Major Field Test Scores 2008-09

Overal

l Perce

ntile

Accounting

Economics

Man

agement

Quantitati

ve

Finan

ce

Mark

etingLe

gal

Informati

on Syste

ms

Internati

onal0

102030405060708090

100

Winter 08Spring 08Fall 08Winter 09

Mean Scores of Major Field Test

Overal

l Perce

ntile

Accounting

Economics

Man

agement

Quantitati

ve

Finan

ce

Mark

etingLe

gal

Informati

on Syste

ms

Internati

onal0

102030405060708090

• Improve legal, quantitative, finance, and information systems categories

• Monitor testing process in HTM and BUSM– All students need to take MFT seriously

– Study differences in scores among students from different geographic areas

• Reward students for excellence– Dean’s Award of Excellence

Areas for Improvement

Major Field Testby Gender and Ethnicity

17%

32%9%

19%

10%

9%3% 1%American Female

American Male

Asian Female

Asian Male

Pacific Islands Female

Pacific Islands Male

South American Female

South American Male

Major Field Test Resultsby Ethnicity

• Provided training for faculty, teaching assistants, and students

• Used Bb as a teaching and faculty development tool

• Results: – 97% of students use Blackboard

– 100% of IBM faculty use Blackboard

– Courses are offered in F2F, hybrid, and online formats

Outcome #7: Communicate effectivelyMeans of Assessment

90% of students communicate virtually on Blackboard

Sample of Student Work

• Goal: Assessment is a way of life, not a program– Assessment is on every department agenda.– Faculty members attend assessment conference

and share lessons learned with others.– “Add assessment to lesson plans”

• Syllabi, rubrics, reflection

– Share assessment tools and tips formally and informally.• Blackboard site for sharing, e-portfolio for

administration and learning

Faculty Involvement

• Easy to access, review and revise

• Quick search of other departments’ assessment information

• Provides transparency of the process and results

• Prepares for self-study/program review

• Reduces stress

Online Department Portfolios

PART 4Linking Assessment to Accreditation

Web Resources

• Online Department Portfolios

• Assessment website

• Accreditation website

• Institutional Research website

Web Resources

• Online Department Portfolios

• Assessment website– Academic and administrative departments

– Empowers departments

– Training tools and document templates

– Encourages sharing of best practices

– Provides a user-friendly process (easy at all levels to administer, change or update)

– http://www.byuh.edu/pirat/Assessment/

• Accreditation website• Institutional Research website

Web Resources

• Online Department Portfolios• Assessment website

• Accreditation website– Emphasizes accountability (both internal & external)– Documents processes and engagement– Encourages campus dialog– Supports WASC standards– Highlights themes and reports– http://www.byuh.edu/pirat/Accreditation/

• Institutional Research website

Web Resources

• Online Department Portfolios• Assessment website• Accreditation website

• Institutional Research website– Data exhibits– Campus surveys– Defines key performance indicators– Comparison schools– http://www.byuh.edu/pirat/Institutional_Research/

PART 5Sharing Best Practices

PART 6Summary and Q&A

Support for Departments

• Working individually with each department

• Being accountable (return and report)

• Providing financial/budgetary support

• Participating in conferences, workshops and training

• Linking accreditation and program reviews

• Sharing best practices (online & awards)

Assessment Score Card(How are we doing at BYUH?)

• Completion rates of about– 65% Academic units

– 64% Administrative units

• Unofficial commendation on our assessment process (WASC visiting team)

• Challenge of major campus restructuring and faculty load

• A VERY LONG JOURNEY – (a process, not an event)

• Still much to do! Our program is growing …evolving …

What’s helped us?

• Process that is conceptually sound

• Engaged faculty

• Commitment to improvement (at all levels)

• Support from key academic and administrative leaders

• Financial support (institution and department level budgets)

• Effective organizational structure (i.e. University Assessment Committee)

• Very Important: A great web programmer

Future Directions

• Sustainable processes

• Continuous improvement

• Archiving and effective documentation

• Preparation for 2012? (when we begin our next “formal” cycle and have a special visit)

• Further ease of online data entry (assessment plans and rubrics)

Q & A

Need any help?

• Bill Neal nealw@byuh.edu

• Helena Hannonen hhannone@byuh.edu

• Johnson Mak jmak@byuh.edu

• Paul Freebairn freebaip@byuh.edu