Upload
dianne-johnson-wojnicki
View
53
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Final Evaluation Report EDTECH 505
Citation preview
Evaluation Report of The AIDET
SM Fundamentals of Patient
and Customer Communication Program
May 8, 2014
Submitted to Dr. John Thompson
By
Dianne R. Johnson-Wojnicki
May 8, 2014
May 8, 2014 Page 1
Summary ..................................................................................................................................................... 2
Description of the Program Evaluated .................................................................................................... 3
Context ..................................................................................................................................................... 3
Program Objectives ............................................................................................................................... 3
Program Components ........................................................................................................................... 5
Evaluation Method ..................................................................................................................................... 5
Participants .............................................................................................................................................. 5
Procedures .............................................................................................................................................. 5
Data Sources .......................................................................................................................................... 6
Results ......................................................................................................................................................... 7
Discussion ................................................................................................................................................... 8
Project Cost ................................................................................................................................................. 9
Appendices ................................................................................................................................................ 10
Appendix A – AIDETSM Patient and Customer Communication program .................................... 10
Session one: ..................................................................................................................................... 10
Session two: ...................................................................................................................................... 10
Appendix B – Pre- Training Self-Assessment .................................................................................. 12
Appendix C – Pre-Training Self-Assessment Results .................................................................... 13
Appendix D – Post-Training Self-Assessment ................................................................................. 17
Appendix E – Post-Training Self-Assessment Results .................................................................. 18
Appendix F – AIDET Communication Competency Assessment ................................................. 22
Appendix G – AIDET Communication Competency Assessment Results .................................. 23
Appendix H - AIDET Patient and Customer Communication Post-Training Survey .................. 27
Appendix I - AIDET Patient and Customer Communication Post-Training Survey Results ..... 30
Appendix J – References .................................................................................................................... 34
May 8, 2014 Page 2
Summary
This document is the final report detailing the results of the formal evaluation of a patient and customer communication program termed AIDETSM at the Archetype Ambulatory Site of the Institution. The Midwest Regional Ambulatory Care executive administration established both qualitative and quantitative objectives to determine the viability of AIDETSM success to establish an institution-wide standard of service with hospitality, respect and care. Due to the decision-making nature of the objectives the CIPP evaluation model was employed to assess the four integrated aspects of a developing program through the utilization of rigorous and authentic assessments. The primary intent of the evaluation is to provide the Midwest Regional Ambulatory Care executive administration with credible evidence substantiating a) the program will provide the knowledge transfer required to attain marked improvement in customer communication and service as reported by the institution-wide Recent Patient Survey, b) a standard to effectively communicate with customers and other representatives using language aligned with hospitality, respect and care, c) the perception that the AIDETSM program is a meaningful, useful and consequential component of employment. Pre-training and post-training self-assessments were used to establish a base-line of prior knowledge and skill levels to evaluate the context and input of the program. Post-training surveys were implemented to determine the process and perceived success. The product was evaluated using supervisory competency assessments and the Recent Patient Survey rankings were monitored over a 15 month period to examine the impact of final product. The evaluation determined in order to establish an institution-wide standard of service capable of achieving the desired customer service rankings a training program with the qualities of the AIDETSM was warranted. Delivered its original format the program achieved marked success introducing, instructing and supporting a standard of customer
communication. The program attained significant support for its merit and worthiness from the attendees, although there were individual components of the training that were deemed less than effective. While the Recent Patient Survey rankings in the first quarter; post-training, did not achieve the desired level, there was marked improvement in customer perception.
Based on the results, the principle investigator recommends the implementation of the AIDETSM Patient and Customer Communication program to achieve the objectives as stated by the Midwest Regional Ambulatory Care executive administration. Additional recommendations concerning the efficiency of the program indicates the need for slight modifications as dictated by the post-training survey.
May 8, 2014 Page 3
Description of the Program Evaluated
Context
The Archetype Ambulatory Site was established in March 2010, the primary function of the site is to address the changing nature of healthcare with respect to patient access. It currently facilitates the operations of Immediate Care services as well as 160 physicians and Advance Practice Nurses (APN) in over 30 medical specialties. Due to the increasing amounts of customer satisfaction information available to patients and their families through services such as healthcareprovider.com and independent marketing campaigns the importance of excellence in customer service is of the highest priority within the healthcare domain. In order to meet the customer satisfaction rankings advocated by the Institution it was deemed necessary to establish a training program setting an institution-wide standard of service with hospitality, respect and care. The goal is to provide a standard by which all representatives of the Archetype Ambulatory Site and eventually the Institution would be expected to adhere. The Archetype Ambulatory Site due to its independent and somewhat autonomous nature has accepted the responsibility as the Beta site for the implementation of the training program. The expectations established were to evaluate the success of the program in an independently functioning environment with a reasonable fixed-cost and timeframe. If the program is assessed as successful the program would begin a full scale rollout with the Institution. The program selected by the primary stakeholder; Director of Midwest Regional Ambulatory Care, to promote customer satisfaction and excellence is referred to as the AIDETSM Patient and Customer Communication program. The program is designed to improve customer service primarily in the healthcare industry; however adaptations have been used in similar patient and customer service concentrated environments especially those involving anxiety or concerns of personal or familial welfare. Program Objectives
The primary purpose of the program is to introduce, instruct and support a standard for representative to customer communication. The overall goal of the program is to provide a means of individual accountability while incorporating effective communication at all levels to build a culture of service resulting in productive outcomes. The definition of productive outcomes is subjective, in that productive is not always financial. It could be defined as building goodwill, respect and confidence. The activities associated with the program involve brief expositional discourse, viewing content specific vignettes, self-evaluation, testimonials, role play, post-course debrief as well as obtrusive and unobtrusive observations. The expected outcome from this program is two-fold; first and foremost is providing for an exceptional customer experience and secondly increased satisfaction for representatives at all levels. Both of these specific outcomes could culminate in increased financial reward by retaining
May 8, 2014 Page 4
existing customer base, increasing the number of new customers through positive feedback and reducing representative turnover with higher personnel satisfaction. A secondary but equally important outcome is the fostering the goodwill, respect and confidence toward the customers, staff and the public at large. The overall objectives of the AIDETSM training program as indicated by the primary stakeholder are to consistently achieve a customer service ranking above the 80th percentile as reported by the institution-wide Recent Patient Survey. A secondary objective as stipulated during the evaluation program design would be identifying components of the program that could be streamlined to increase the effect and efficiency of an institution-wide rollout. The objectives of this evaluation are stated as follows:
90 % of the Ambulatory Site AIDETSM attendees will comprehend both the
significance and importance of the Acknowledgement, Introduction, Duration,
Explanation and Thanking elements patient communication and effectively apply
them to achieve superior customer satisfaction as indicated by both the training
and competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will comprehend the benefits
associated with Managing Up to achieve superior patient communication and
customer service as indicated by both the training and competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will effectively communicate with
patients and customers using language aligned with hospitality, respect and care
in accordance with the five fundamentals of patient communication as indicated
by the competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will effectively communicate with
other representatives using language aligned with hospitality, respect and care in
accordance with the five fundamentals patient communication as indicated by the
competency assessment.
90 % of the Ambulatory Site AIDETSM attendees will possess a level of individual
accountability to deliver service required to attain a superior level of customer
satisfaction aligned with hospitality, respect and care as indicated by the Recent
Patient Survey.
90 % of the Ambulatory Site AIDETSM attendees should perceive the AIDETSM
program as a meaningful, useful and consequential component of the
Ambulatory Site onboarding and review process.
90 % of the Ambulatory Site AIDETSM attendees should perceive the AIDETSM
program as effectively and efficiently achieving all of the stated program
objectives.
Any of the AIDETSM program objectives that are deemed ineffective or ineffectual
should be reworked for a positive and successful outcome.
May 8, 2014 Page 5
Program Components
The AIDETSM training program is conducted in two sessions each approximately two hours in length and one week apart. The sessions are conducted in addition to the clinical day either in the morning or evening hours. An experienced training provider was hired to facilitate the program. The activities associated with the program involve brief expositional discourse, viewing content specific vignettes, self-evaluation, testimonials, role play post-course debrief as well as obtrusive and unobtrusive observations. A detailed description of the entire training program is included in Appendix A.
Evaluation Method
The evaluation method selected to conduct the project is an adaptation of the CIPP Model of evaluations. This model is a decision-focused approach and is designed to provide both realistic and analytical substance for management decision-making. The model utilizes all phases of a program’s cycle specifically its structure, implementation, review and revision to formulate sound effective decisions. Participants
The participants involved in the evaluation are comprised of 22 service representatives from the Archetype Ambulatory Site. They are between the ages of 25 and 57. They are all female. There are 6 representatives with less than six months experience at the site the other 16 have been at the site for anywhere between one and three years. Their experience in the health care industry ranges from neophyte to seasoned healthcare personnel. Some of have previously functioned in clinical or technical health related areas. Procedures
The CIPP Model includes four distinct components for evaluation; typically referred to as Context evaluation, Input evaluation, Process evaluation and Product evaluation. These components can be further clarified using four basic premises; what should be done, how should it be done, is it being done as planned and did it work? The initial component of the evaluation focused on existing skills and knowledge base using a pre-training self-assessment. This instrument is designed to establish a baseline for improvement. The self –assessment consists of several mechanisms the most critical was the introspective perception of the five fundamentals of customer communication. During the training program there are several opportunities for obtrusive observation. It was deemed that these observations would be used only for instructive and edifying purposes and would not be used for the analytical examination in the evaluation.
May 8, 2014 Page 6
The second element of the evaluation is the degree of knowledge transfer with respect to the perception of new knowledge gained as a result of the program. This information was gathered using a post-training self-assessment. The final set of criteria concerning the level of skill and knowledge transferred as the result of the program, were assessed during post-training unobtrusive supervisory assessment using the competency assessment. This process was explained in the training and devised solely for individual AIDET assessments to complete the AIDET training contract and subsequent yearly performance review criteria. In terms of the CIPP model’s ability to measure the formative components of a program the post-training survey captured the essence of the program. This phase of the evaluation assists in determining the overall efficiency and effectiveness as well as participation satisfaction. Data Sources
Data pertinent to the training program was collected using four specific instruments from all 22 representatives attending the training program. The data collection strategy was very specific. In order to guarantee 100 percent participation online form submission after the fact was prohibited. Each participant was required to complete the data source using paper and pencil before moving forward to the next phase of the program. The four data sources were designed specifically for one or more aspects of the CIPP evaluation method. The instruments have been provided in the Appendices to clarify the nature of the information received as well as their respective outcomes. The details of their inclusion in this document are as follows:
The Pre-Training Self-Assessment is located in Appendix B with the
corresponding results in Appendix C.
The Post-Training Self-Assessment can be found in Appendix D with the
corresponding results in Appendix E.
The AIDET Communication Competency Assessment is located in Appendix F
with the corresponding results in Appendix G.
The AIDET Patient and Customer Communication Post-Training Survey can be
found in Appendix H with the corresponding results in Appendix I.
Data from the Recent Patient Survey; a fifth data source, is compiled and reported by the Institution in quarterly cycles. The customer perception ranking for the Archetype Ambulatory Site has been consistently ranging between 81 and 84 percent in Office Staff Courtesy and Respect and customer’s Willingness to Recommend during the four quarters priors to implementing the AIDET training. These rankings fall short of the institution-wide directive to meet the 75th percentile. The details of the rankings have been extracted from the quarterly documents (see Figure 1).
May 8, 2014 Page 7
Figure 1. Previous quarterly rankings for Archetype Ambulatory Site
04/13 06/13 10/13 01/14
Intuition-wide 75th percentile
Office Staff Courtesy and Respect
83.47 83.65 84.20 82.95 94.00
Willingness to Recommend
82.30 82.41 82.46 81.31 92.50
Results
The Pre-training self-assessment (see Appendix B and C) was designed to address the first component of the CIPP model; what should be done? In essence, what skills and competencies of effective patient communication currently exist and what deficiencies need to be addressed. The pre-training self-assessment was enlightening in that 86 percent of the AIDET attendees had no prior experience with the AIDET patient communication approach. That being said, in terms of their personal assessments of communicating with patients and customers in AIDET terms, 73 to 91 percent considered themselves either meeting or exceeding the AIDET standards. A key point however, was that only 14% of the attendees initially perceived a customer as a patient, families, a physician or a co-worker, quit contrary to AIDET.
The Post-Training Self-Assessment (see Appendix D and E) was designed to address the first component of the CIPP model; how should be done? The data collected revealed once the AIDET standards of patient communication had been fully defined and explored the personal perception of current abilities appears to diminish. Only 23 to 73 percent considered themselves either meeting or exceeding the AIDET standards. This dramatic reduction in skill perception indicates the awareness that current patient communication strategies are not acceptable. On a positive note, having attended the training, 100% of the attendees now perceive a customer as a patient, families, a physicians or co-workers. While 95 percent stated confidence in successfully using the AIDET approach to communicate with patients.
The AIDET Patient and Customer Communication Post-Training Survey (see Appendix H and I) designed specifically to measure the third component of the CIPP model; is it being done as planned? This data source revealed that at least 70 percent of the attendees found the training to be useful, effective in terms of achieving the goals and ultimately beneficial to customers. Nearly all of the attendees expressed confidence in their successful use of the AIDET approach to more effectively communicate with patients and achieve positive outcomes.
The AIDET Communication Competency Assessment (see Appendix F and G) was designed as an indicator for the fourth and final component of the CIPP model; did it work? The AIDET Communication Competency Assessment illustrates that while the attendees themselves perceive their communication skills lacking the poise and refinement portrayed in the AIDET training, the supervisory staff found their collective abilities ranging between 73 and 95 percent consistent or above.
May 8, 2014 Page 8
Data from the Recent Patient Survey compiled in the quarter since the AIDET training was implemented showed a slightly positive effect with the Archetype Ambulatory Site ranking 88 and 87 percent respectively in Office Staff Courtesy and Respect and customer’s Willingness to Recommend. The details of the newly reported rankings have been extracted from the quarterly documents (see Figure 2).
Figure 2. New quarterly rankings for Archetype Ambulatory Site
04/13 06/13 10/13 01/14 04/14
Intuition-wide 75th percentile
Office Staff Courtesy and Respect
83.47 83.65 84.20 82.95 88.11 94.00
Willingness to Recommend
82.30 82.41 82.46 81.31 87.39 92.50
Discussion
The Archetype Ambulatory Site accepted the responsibility of as the Beta site for the AIDET training program and evaluation. The CIPP model is designed to assist in making decisions by providing a systematic analysis of the need, how to meet the need, whether the need was met, and could the need be met more efficient and effectively. The pre and post training assessments in conjunction with the Recent Patient Survey rankings indicate there is a need to define and establish a standard of effective patient and customer communication. The AIDET Communication Competency Assessment indicates that post training the representatives’ illustrated marked improvement in effectively communicating with patients. The assessment also indicates that participating in the training to acquire the necessary skills and comprehend the requirements of excellent patient communication the representatives can achieve more effective patient communication. The AIDET Patient and Customer Communication Post-Training Survey clearly indicate the representative perceived the program as effective. The majority of the attendees consider the program did achieve the objectives. In terms of formative evaluation the components of the program which reportedly were less effective and unproductive were the vignettes and testimonials. The components of the training which reportedly were most effective and useful were the explanations, role play and moderated discussions. The most promising element reported as a result of the AIDET program and nearly meeting the stated objective was the Recent Patient Survey compiled since the training which indicated an increase of nearly 5 percentage points. If this trend continues the percentages should soon be consistently meeting or exceeding the 90 percent requirement. In light of this evaluation’s reliable model (CIPP) substantiated through the analysis of specific data collecting instruments there is clearly acceptance and benefit derived by the training program. It is the recommendation of the principle investigator that the program is creditable and worth the investment of an institution-wide rollout. Although
May 8, 2014 Page 9
some of the percentages achieved did not meet the stated 90 percent stated in the stakeholder objective many of these “soft skills” require adequate time to measure the full impact. For that reason, the principle investigator also recommends that the AIDET Communication Competency Assessment tool should be utilized multiple times a year with clearly communicated feedback to give recipients directives for improvement. These actions coupled with periodic in-service components to revitalize the AIDET philosophy would improve eventual impact realization. In terms of streamlining the training to achieve maximum benefit and a higher degree of satisfied attendees, it is the recommendation of the principle investigator to require modification by the professional training facilitator to the following components: reducing the number of vignettes and testimonials, slightly increasing the amount of role play and moderated discussions while expanding the explanation and description components.
Project Cost
The project cost consists of four days of on-site and seven days off-site. Two on-site days were meeting with the primary stakeholder to determine the objectives requirements and solidify the evaluation method. One on-site day was spent with the AIDET training facilitator to review and document the training program process and agenda. The final on-site day was observing one of the AIDET training programs (although the training took place on two different days only one day was billed since the time consumed for each session was less than four hours). The additional seven days off-site were spent preparing data source instruments, data collection, data analysis and developing the final report. Both on and off site are billed at same daily rate however the on-site services require an additional daily per-diem of $150.00. The full invoice follows.
Task Description Days Daily Rate Amount On-site formal evaluation
(primary stakeholder meeting) 2 $500.00 $1,000.00
On-site formal evaluation (AIDET facilitator meeting) 1 $500.00 $500.00
On-site formal evaluation (attending AIDET training) 1 $500.00 $500.00
Off-site formal evaluation (data source instruments preparation, data collection, data analysis and developing the final report) 7 $500.00 $3,500.00
On-site per diem 4 $150.00 $600.00
TOTAL $6,100.00
May 8, 2014 Page 10
Appendices
Appendix A – AIDETSM Patient and Customer Communication program
Session one:
The initial session begins with a pre-training self-assessment to determine a baseline for each participant to gain an introspective view of their own customer service skills at present. The session then moves to a brief expositional discourse generalizing the fundamental purpose of AIDET and defining each of its five components. Each of the components has a specific purpose and goal. Each of these components is also assessed separately and carries its own merit and value. Several participants are then asked to assist one-by-one as the facilitator enacts impromptu scenarios to illustrate both appropriate and inappropriate AIDET based customer service examples. The viewing participants are then asked to comment on these scenarios for a candid and open moderated discussion in order to determine what components of the dialog seemed effective or ineffective and why. Each of the participants is then given a worksheet to create their own unique script using their previous skills and knowledge while incorporating their new knowledge of the five components of AIDET. The participants are aware that once complete they will be randomly paired up to practice and modify the scripts based on peer constructive critique. A question and answer period follows for the facilitator to guide any concerns, difficulty or reservations. The first session ends a brief introduction of the concept of managing up. Managing up is defined as well as examples of subordinates and superordinate that could be managed and why. Each participant has been given a worksheet to describe how they would manage up themselves, one subordinate and one superordinate. At the close of session each participant is given an assignment to actively practice the AIDET principles on the job and determine a personal script. They are also asked to contemplate the advantages and benefits of managing up. Session two: The second session convenes one week after session and begins by posing the following questions:
How have you used AIDET since the last session?
How have you observed your peers using AIDET since the last session?
May 8, 2014 Page 11
A brief series with testimonial from participants will be facilitated to highlight successes using AIDET focusing its positive effect on customers and colleagues in an effort to cultivate a buy in from all participants. The session moves to the viewing of a series of vignettes set in various scenarios used to illustrate adapting the AIDET components to any situation in an order to produce positive outcomes. After each of the vignettes the following questions will be open for discussion:
What aspects of AIDET were depicted?
What were the key words and phrases used by representatives to manage up themselves, a subordinate, a superordinate, another department, or the entire organization?
What could the representative have done better? At this point the session will turn to the topic of managing up. A brief expositional discourse will review the concept of managing up and the significance it has on decreasing patient anxiety toward caregivers and healthcare coordination efforts as well as assisting in gaining customer confidence. The participants will be asked to produce their assignments on managing up. The facilitator will ask for a series of volunteers to share their managing up worksheet. This discourse is followed by a moderated discussion on positive effects and benefits of managing up. Session two ends with an informative discourse which includes a review of the competency survey, the contract and personal AIDET responsibilities moving forward.
May 8, 2014 Page 12
Appendix B – Pre- Training Self-Assessment
May 8, 2014 Page 13
Appendix C – Pre-Training Self-Assessment Results
1. Patient Communication Technique Component – Current Proficiency
9%
46%
45%
ACKNOWLEDGES the patient/customer
Needs Improvement
Meets
Exceeds
32%
32%
36%
INTRODUCES self
Needs Improvement
Meets
Exceeds
May 8, 2014 Page 14
27%
55%
18%
Estimates DURATION
Needs Improvement
Meets
Exceeds
27%
23%
50%
THANKS the customer
Needs Improvement
Meets
Exceeds
May 8, 2014 Page 15
2. Experience level
27%
55%
18%
MANAGES UP co-workers/departments
Needs Improvement
Meets
Exceeds
14%
86%
Do you have any prior experience using AIDET?
Yes
No
May 8, 2014 Page 16
4. Most commonly missed of the True/False questions.
14%
86%
Customers include patients, family, physicians and co-workers.
True?
False?
May 8, 2014 Page 17
Appendix D – Post-Training Self-Assessment
May 8, 2014 Page 18
Appendix E – Post-Training Self-Assessment Results
1. Patient Communication Technique Component – Reflects Proficiency Post-training.
27%
64%
9%
ACKNOWLEDGES the patient/customer
Needs Improvement
Meets
Exceeds
36%
55%
9%
INTRODUCES self
Needs Improvement
Meets
Exceeds
May 8, 2014 Page 19
77%
18%
5%
Estimates DURATION
Needs Improvement
Meets
Exceeds
27%
50%
23%
THANKS the customer
Needs Improvement
Meets
Exceeds
May 8, 2014 Page 20
2. Do you feel you can successfully communicate using AIDET?
64%
18%
18%
MANAGES UP co-workers/departments
Needs Improvement
Meets
Exceeds
95%
5%
2. Do you feel you can successfully communicate using AIDET?
Yes
No
May 8, 2014 Page 21
5. Most commonly missed of the True/False questions in pre-training assessment.
0%
100%
Customers include patients, family, physicians and co-workers.
True?
False?
May 8, 2014 Page 22
Appendix F – AIDET Communication Competency Assessment
May 8, 2014 Page 23
Appendix G – AIDET Communication Competency Assessment Results
0%
64%
36%
Identifies purpose of using AIDET.
Inconsistent
Consistent
Role model
5%
59%
36%
Utilizes AIDET to ensure VERY GOOD communication with patients "EVERY
PATIENT EVERY TIME".
Inconsistent
Consistent
Role model
May 8, 2014 Page 24
5%
50%
45%
ACKNOWLEDGES the customer:
Inconsistent
Consistent
Role model
46% 27%
27%
INTRODUCES self:
Inconsistent
Consistent
Role model
May 8, 2014 Page 25
27%
41%
32%
DURATION: Gives the customer a time expectation
Inconsistent
Consistent
Role model
27%
46%
27%
EXPLANATION:
Inconsistent
Consistent
Role model
May 8, 2014 Page 26
18%
46%
36%
THANKS the customer:
Inconsistent
Consistent
Role model
4%
55%
41%
Ensures non-verbal communication conveys AIDET:
Inconsistent
Consistent
Role model
May 8, 2014 Page 27
Appendix H - AIDET Patient and Customer Communication Post-Training Survey
May 8, 2014 Page 28
May 8, 2014 Page 29
May 8, 2014 Page 30
Appendix I - AIDET Patient and Customer Communication Post-Training Survey Results
1. How would you rate this training for meeting your needs to learn the AIDET
Fundamentals of Patient and Customer Communication?
2. Was the program the right length for the material covered?
32%
45%
18%
5% 0%
Excellent
Satisfactory
Neutral
Unsatisfactory
Poor
27%
41%
18% 14%
0%
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
May 8, 2014 Page 31
3. If you disagreed, what material was not covered in enough depth?
How much time to spend with a patient (D) in AIDET
Duration and Explanation
Moderated discussion
Moderated discussion
Explanations and moderated discussions
4. If you disagreed, what material was covered in too much depth?
Explanation (E) more than a one page power point, what exactly am I explaining.
Managing up
Role playing
Videos
Videos
5. Were the program objectives fully achieved?
6. If you disagreed, which objective(s) do you feel were not met?
More Role Play to get a full concept of AIDET
23%
55%
18%
4% 0%
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
May 8, 2014 Page 32
7. Which component(s) of the program did you find most helpful to achieve the
program objectives?
8. Do you feel the communication skills achieved as a result of the program will be
of benefit to patients?
19%
9%
28%
12%
32%
Explanations anddescriptions
Testimonials
Role Play
Vingettes (video)
Moderated discussion
27%
45%
14% 14%
0%
Very certain
Certain
Neutral
Uncertain
Very uncertain
May 8, 2014 Page 33
9. Do you feel the communication skills achieved as a result of the program will be
of benefit to colleagues?
10. How confident are you that this program has given you the skills to meet the
AIDET competency requirements?
23%
18%
41%
18% 0%
Very certain
Certain
Neutral
Uncertain
Very uncertain
14%
54%
32%
0% 0%
Very confident
Confident
Neutral
Doubtful
Very Doubtful
May 8, 2014 Page 34
Appendix J – References
John, B., & Dutwin, P. (2011). The ABCs of Evaluation: Timeless Techniques for Program and
Project Managers (3rd ed.). San Francisco: Jossey-Bass.
Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C., & Misulis, K. (2011). Using the
Context, Input, Process, and Product Evaluation Model (CIPP) as a Comprehensive Framework
to Guide the Planning, Implementation, and Assessment of Service-Learning Programs. Journal
of Higher Education Outreach and Engagement, 15(4), 57–84.