1ExamSoft Assessment Conference | #ASSESS2015

ExamSoft Assessment Conference

#ASSESS2015

2ExamSoft Assessment Conference | #ASSESS2015

Generating Stakeholder Buy-in to Establish a Culture of Assessment

Karen A. Bobak, DC, EdDDean of Chiropractic

New York Chiropractic College

Lisa K. Bloom, DC, FIACNAssistant Dean, Chiropractic Education

Professor, Department of Clinical SciencesFellow of the International Academy of Chiropractic Neurology

Board certified, Chiropractic NeurologyNew York Chiropractic College

3ExamSoft Assessment Conference | #ASSESS2015

New York Chiropractic College is committed to academic excellence,

quality patient care, and professional leadership.

NEW YORKCHIROPRACTIC

COLLEGE

4ExamSoft Assessment Conference | #ASSESS2015

Objectives

• Identify why stakeholder buy-in is important

• Formulate effective strategies for engagement of faculty and administration

• Identify principles of Improvement Science to support a process of curriculum review

• Review longitudinal assessment data to identify curricular gaps

• Recognize how to use assessment data to develop improvement plans

5ExamSoft Assessment Conference | #ASSESS2015

In a Culture of Assessment…

• Assessment processes reinforce cumulative learning across coursework.

• Assessment processes are valued and are the normal part of decision making processes.

• This allows a program to be proactive about identifying gaps rather than reactive to sudden problems.

6ExamSoft Assessment Conference | #ASSESS20156

Culture of Assessment

• Concentrating only on compliance tends to create an accountability culture that focuses on short term data reports and does not include stakeholder voices.

• Concentrating on continuous improvement tends to create an organizational learning culture that focuses on the longer term student learning outcomes.

7ExamSoft Assessment Conference | #ASSESS2015

Why Do You Want Buy-in?

The opportunity for making improvements is greater when combining content expertise

and creative thinking process

• Regular open communication encourages faculty to take a more active and positive role in implementation of change.

• A role model from within the institution’s decision making group provides participants the opportunity to be better informed about issues and concerns.

• Having a role model demonstrate new attitudes and behaviors provides faculty with a vision of expectations and can serve as a more powerful teaching tool than formal training mechanisms.

8ExamSoft Assessment Conference | #ASSESS2015

Current Knowledge

Possible Changes

Traditional Process

Discourages engagementCreates uncertainty

Often creates compliance by fear

Any change project that does not take into account the people at the implementation level will likely fail

The Improvement Guide, 2009

9ExamSoft Assessment Conference | #ASSESS2015

Current Knowledge

Different perspectives create options and generate solutions

The result of a collaborative effort is not only a rich pool of ideas but pride in ownership of those ideas

Faculty place increased value on the ideas that they

help to create

Possible Changes

The Improvement Guide, 2009

10ExamSoft Assessment Conference | #ASSESS2015

Creating Buy-In

Know your stakeholders

• Stakeholder interpretation of the new process matters because it orients their actions

• Those in positions of formal and informal power can influence how other make meaning of change efforts

• What do they value?

11ExamSoft Assessment Conference | #ASSESS2015

Strategies to Engage Faculty and Administration

• What does each program or group need to create meaning?

What strategies have you used?

Which strategies work? Which Strategies don’t work?

Your Turn

12ExamSoft Assessment Conference | #ASSESS2015

Using the principles of Improvement Science helps clarify the process

How to reduce the gap between what is actual and what is possible

Identify your baselineWhere are you now?

• Begin data collection to define what is actual

Supporting change with dataWhere does the data come from?

• Establish your goal to define what is possible

13ExamSoft Assessment Conference | #ASSESS2015

A P

DS

A P

DS

A P

DS

A P

DS

Theories, Hunches and Best Practices

Breakthrough Results

Learning and Improvement

Very small-scale test

Follow-up Tests

Test New Processes/Conditions

Wide-scale Tests of Change

Evidence and Data

Sequential Building of Knowledge

The Improvement Guide, 2009

14ExamSoft Assessment Conference | #ASSESS2015

What Does Buy-in Look Like?

Faculty ask for more data

Changes are being documented

There is an increasing sense of pride

There is an increase in collaboration

Changes are being seen in other stakeholder groups

http://www.learningspy.co.uk/

15ExamSoft Assessment Conference | #ASSESS2015

The Warning Signs of Resistance

• Limited faculty input

• Lack of engagement

• Lack of ownership and accountability

• Lack of senior leadership buy-in

• Culture of fear

• Lack of trust

http://www.learningspy.co.uk

16ExamSoft Assessment Conference | #ASSESS2015

Workshop

What Are Your Current Review Strategies?What Are Your Current Sources of Resistance?

16

17ExamSoft Assessment Conference | #ASSESS2015

Get a process going and get into a cycle• A cycle will engage more people and reinforce the importance of

cumulative assessment of student learning

• Cumulative or longitudinal assessment data becomes the mortar that binds the individual assessments together

• Studying the longitudinal assessment results makes gaps in the curriculum more transparent and the identification more precise

http://www.a1underpinningnsw.com.au/concrete-crack-brickwork-wall-door-repair-sydney.html

18ExamSoft Assessment Conference | #ASSESS2015

Sample longitudinal data

19ExamSoft Assessment Conference | #ASSESS2015

Blooms: Evaluating % Correct

Blooms: Applying% Correct

Blooms: Understanding

% Correct

Learning Outcome 1:

Cardio% Correct

Learning Outcome 2:Respiratory% Correct

Learning Outcome 3:

Gastro% Correct

# Exams 40 35 17 11 5  3

# Items  615 282 21 71 2 14

Group Average 85.23% 84.3%  83.8% 86.0%  92.9%   86.5%

Student A 75.77%  71.84% 72.73%  50% 100% 100% 

Student B 89.71% 88%   90.91% 100% 100% 92.3%

Student C 83.82%  82% 100% 100% 84.6% 50%

Student D  77.45% 80% 72.73% 75% 100% 84.62%

WorkshopHow Has

This Helped You Develop

an Improveme

nt Plan?

How Have Identified Curriculum Gaps?

21ExamSoft Assessment Conference | #ASSESS2015

Full group discussion

22ExamSoft Assessment Conference | #ASSESS2015

Thank You!Questions? Comments?

Lisa K. Bloom, DC, FIACNAssistant Dean, Chiropractic EducationProfessor, Clinical Neurology, Department of Clinical SciencesFellow of the International Academy of Chiropractic NeurologyBoard certified, Chiropractic NeurologyNew York Chiropractic Collegelbloom@nycc.edu

Karen A. Bobak, DC, EdDDean of Chiropractic New York Chiropractic Collegekbobak@nycc.edu