Outcomes Assessment Plan 2014-2015

1

Program Assessment and Effectiveness Data

Doña Ana Community College Radiologic Technology Program

2014-2015 Glossary: CR-Central Ray SID- Source to Image Distance IR- Image Receptor RIS- Radiology Information System PACS- Picture Archiving Communication System ASRT- American Society of Radiologic Technologists

For purposes of defining the program’s timeline throughout this document, the following defines first year versus second year students:

The Radiologic Technology Program consists of 5 consecutive semesters beginning in the fall of each academic year. A student is considered a first year student for semesters one and two. A student is considered a second year student for semesters three, four and five.

Likert scales for first year students are scored out of 4 points

Likert scales for second year students and graduates are scored out of 5 points Revised 3/6/15, 3/18/15, 4/1/15, 6/10/15, 6/15/15, 6/25/15-Final

Table of Contents Outcomes Assessment Plan .......................................................................................... 3

Goal 1: Students will be clinically competent .......................................................... 3

Student Learning Outcome 1: Students will demonstrate safe operation of radiographic equipment to produce quality diagnostic radiographic images ... 3

First-Year Students ....................................................................................................... 3

Student Learning Outcome 2: Students will be able to appropriately position patients, identify radiographic anatomy and pathological conditions. ................................................................................................................. 4

First-Year Students ................................................................................................... 4

Second-Year Students .................................................................................................. 5

Goal 2: Students will communicate effectively ........................................................ 6

Student Learning Outcome 3: Students will demonstrate appropriate and effective communication skills ................................................................................... 6



2

Student Learning Outcome 4: Students will accurately document/record data in accordance with clinical site policies and procedures ....................................... 7

Second-Year Students, Third Semester ..................................................................... 7

Second-Year Students, Fifth Semester ...................................................................... 7

Goal 3: Students will use critical thinking and problem solving skills .............. 8

Student Learning Outcome 5: Students will demonstrate the ability to make decisions and use independent judgment ................................................................ 8


Second-Year Students, Fifth Semester ...................................................................... 9

Student Learning Outcome 6: Students will analyze radiographic images for technical and positioning accuracy and make modifications as needed ............. 9


Goal 4: Students will model professionalism ......................................................... 12

Student Learning Outcome 7: Students will conduct themselves in an ethical and professional manner and function effectively as a member of the healthcare team in accordance with ARRT Standards and ASRT Code of Ethics....................................................................................................................................... 12

Second-Year Students ................................................................................................ 12

Student Learning Outcome 8: Students will practice professionalism by attending professional development opportunities, joining professional organizations and/or reading professional journals ............................................ 12

Second-Year Students ................................................................................................ 12

Program Effectiveness Measures and Data .............................................................. 13

Measure 1: Students will pass the ARRT national certification on the 1st attempt ......................................................................................................................... 13

Measure 2: Students pursuing employment will be gainfully employed within 6 months of graduation ............................................................................................. 14

Measure 3: Students will complete the program within 21 months ................... 15

Measure 4: Students will be satisfied with their education ................................. 16

Measure 5: Employers will be satisfied with the graduate’s performance ....... 17

Possible Program Research Questions for 2015-2016 ............................................... 18

3

Outcomes Assessment Plan

Goal 1: Students will be clinically competent Student Learning Outcome 1: Students will demonstrate safe operation of radiographic equipment to produce quality diagnostic radiographic images

First-Year Students

Course and Timeframe RADT 102 (first year of program, second semester, after third lab exam)

Person/s responsible Lead Instructor and Lab Instructor

Evaluation Instrument Laboratory Evaluation Positioning Exam rubric -- three simulations per student

o Question 3 Select the correct SID o Question 4 Select the correct tube angle and direction o Question 5 Set correct technical factors o Question 7 Center the CR to the IR/Bucky

Benchmark Target RADT 102-The class will achieve 85% or better on the four questions averaged together.

Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change

Year One: RADT 102: Question 3- 93.75% Question 4- 100% Question 5- 100% Question 7- 81.25% RADT 102: Average of all four questions: 93.75% Benchmark target exceeded.

Action Plan No action plan is needed at this time, however, an incidental finding was that the lab instructors are not using the rubric correctly when grading. To fix this, the lead instructor will have a meeting with all lab instructors at the beginning of each semester to explain the rubric and grading for lab exams. NOTE: The data presented above was calculated from the corrected grading rubrics.

4

Second-Year Students

Course and Timeframe RADT 201,202, & 203 These courses consist of the entire second year of the program: third, fourth and fifth semesters. The competencies being evaluated (spine, skull or fluoroscopy) could be completed by the student in any of the three semesters. The data will be collected at the end of the fifth semester of the program.

Person/s responsible Clinical Instructors and Clinical Coordinator

Evaluation Instrument Online Rad School Clinical Competency Exam: Three competencies per student of a spine, skull, and fluoroscopic procedure

o Question 17 Selects appropriate technical factors, mAs, kVp

Benchmark Target 90% or higher

Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: 94% Benchmark target exceeded.

Action Plan The program will continue to monitor.

Student Learning Outcome 2: Students will be able to appropriately position

patients, identify radiographic anatomy and pathological conditions.

First-Year Students Course and Timeframe

RADT 101 (first year of program, end of first semester) RADT 102 (first year of program, end of second semester)

Person/s responsible Instructor

Evaluation Instrument Quizzes from RADT 101, and then given again in RADT 102 as formative assessments.

5

Benchmark Target RADT 101 quizzes: 80% RADT 102 quizzes: 85%

Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: RADT 101 quizzes: 84.70% RADT 102 quizzes: 85.414% While the benchmark targets were met and the class is improving, the instructor has decided to include an action plan to raise the RADT 102 quiz average next year.

Action Plan Short reviews covering the material in RADT 101 will be presented in class during RADT 102 (second semester) as a refresher, possibly as a group assignment, other collaborative projects, or iClicker exercises.


Course and Timeframe RADT 201,202, & 203 These courses consist of the entire second year of the program: third, fourth and fifth semesters. The competencies being evaluated (spine, skull or fluoroscopy) could be completed by the student in any of the three semesters. The data will be collected at the end of the fifth semester of the program.


Evaluation Instrument Online Rad School Clinical Competency Exam -- Three competencies per student of a spine, skull, and fluoroscopic procedure as a summative assessment.

o Question 28 Identify pertinent anatomy o Question 31 Identify any artifacts and/or obvious pathology


Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: 100%

Benchmark target exceeded.

Action Plan The Program will continue to monitor.

6

Goal 2: Students will communicate effectively Student Learning Outcome 3: Students will demonstrate appropriate and effective communication skills First-Year Students

Course and Timeframe RADT 102 (first year of program, end of second semester)

Person/s responsible Clinical Instructors and RADT 102 Instructor

Evaluation Instrument Clinical Evaluation Survey

o Question 6 Student interacts well with the staff o Question 7 Student communicates/interacts well with the patients

Benchmark Target 3.0 or higher out of a 4 point Likert scale (See Glossary)

Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: Question 6- 3.76 Question 7- 3.75




Course and Timeframe RADT 203 (second year of program, end of fifth semester)


Evaluation Instrument Clinical Bimonthly Evaluation Survey (average of both surveys)

o Question on effective communication: The student demonstrates appropriate communication with patients, supervisors, peers and other employees.

Benchmark Target 4.0 or higher on a 5 point Likert scale (See Glossary)

7

Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: 4.6 Benchmark target exceeded.


Student Learning Outcome 4: Students will accurately document/record data in accordance with clinical site policies and procedures Second-Year Students, Third Semester

Course and Timeframe Second year of RADT Program, Third Semester


Evaluation Instrument Clinical Bimonthly Evaluation Survey (average of both surveys) as a formative assessment

o Question 15 Effective Communication: Student documents and records data needed on patient charts, x-ray requisitions, RIS and PACS


Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: 4.4 Benchmark Target Exceeded

Action Plan The program received input from Clinical Instructors at each site with the exception of one [Carlsbad] for the summer of 2014. Note: the program lost two Clinical Instructors at Carlsbad [they left for better employment opportunities]. We now have a replacement Clinical Instructor.

Second-Year Students, Fifth Semester Course and Timeframe

Second year of RADT Program, fifth semester

8


Evaluation Instrument Clinical Bimonthly Evaluation Survey as a summative assessment

o Question 15 Effective Communication: Student documents and records data needed on patient charts, x-ray requisitions, RIS and PACS


Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: 4.5 Benchmark Target Exceeded

Action Plan The program is missing results from three clinical sites. The importance of providing feedback via the Bi-Monthly Evaluation Survey will be stressed to the clinical sites.

Goal 3: Students will use critical thinking and problem solving skills Student Learning Outcome 5: Students will demonstrate the ability to make decisions and use independent judgment Second-Year Students

Course and Timeframe RADT 202, and 203 These courses consist of the second year of the program: fourth and fifth semesters. The competencies being evaluated (trauma upper and lower extremities) could be completed by the student in either of the semesters. The data will be collected at the end of the fifth semester of the program.


Evaluation Instrument Trauma Upper and Lower Extremity Clinical Competency



9

Year One: Trauma Upper Extremity Competency: 98.8% Benchmark Target Exceeded Trauma Lower Extremity Competency: 99.7% Benchmark Target Exceeded


Second-Year Students, Fifth Semester

Course and Timeframe RADT 203 (end of fifth semester)


Evaluation Instrument Clinical Bimonthly Evaluation Survey

o Question 1 Critical Thinking and Problem Solving: Student demonstrates the ability to make decisions and use independent judgment




Student Learning Outcome 6: Students will analyze radiographic images for technical and positioning accuracy and make modifications as needed First-Year Students

Course and Timeframe RADT 103 (first year of program, end of first semester)

Person/s responsible RADT 103 Instructor

Evaluation Instrument Hands-on Lab Final as a summative assessment

10

RADT 103 (includes an unacceptable radiograph that students must correct using the appropriate technical factors)


Results and Analysis (Composite Summary) Three-year Cycle, Assess, Analyze, and Change Year One: Class average was a 71% Benchmark target was not met.

Action Plan The hands on final lab exam tends to be a high stress experience for the first year students that consistently yields lower scores than the didactic portion of the course. To address these results, the program will incorporate an open lab session where students could practice prior to the hands-on final. Another possibility is to incorporate more practice into the lab as time allows.

Graduated Students

Course and Timeframe Within one year of graduation

Person/s responsible Program Director

Evaluation Instrument Employer Survey as summative assessment Question under knowledge base: The graduate has critical thinking and problem solving skills



11

Year one:

Graduated Class of 2014 Radiologic Technology Education Program - Employer Survey

The graduate has critical thinking & problem solving skills.

Answer Options Response Percent

Response Count

5 Strongly Agree 35.7% 5

4 Generally Agree 57.1% 8

3 Neutral 7.1% 1

2 Generally Disagree 0.0% 0

1 Strongly Disagree 0.0% 0

N/A 0.0% 0

answered question 14

skipped question 1

Class average was a 4.2 Benchmark target exceeded.


12

Goal 4: Students will model professionalism Student Learning Outcome 7: Students will conduct themselves in an ethical and professional manner and function effectively as a member of the healthcare team in accordance with ARRT Standards and ASRT Code of Ethics Second-Year Students

Course and Timeframe RADT 203 (second year of program, end of fifth semester)


Evaluation Instrument Clinical Bimonthly Evaluation Survey

o Question 4 Professional Ethics: Student demonstrates ethical and professional demeanor as outlined in the ASRT Code of Ethics

Benchmark Target 4.0 or higher out of a 5 point Likert scale (See glossary)



Student Learning Outcome 8: Students will practice professionalism by attending professional development opportunities, joining professional organizations and/or reading professional journals Second-Year Students

Timeframe Data will be collected at the end of Program

Person/s responsible Program Director and/or Program Faculty

Evaluation Instrument Percentage of second year students’ attendance at a professional development opportunity and/or joining a professional organization.

13

Benchmark Target 80% or higher will participate in conference and/or join a professional organization


Year One: Class of 2014 17 were NMSRT members, one was ASRT members, and 17 went to the NMSRT conference, two went to the Today in Radiology Conference 18/18 100% were involved in professional organizations and professional development opportunities

Benchmark target exceeded. Year Two: Class of 2015 14 were NMSRT members, two were ASRT members, and 14 went to the NMSRT conference 14/16 88% were involved in professional organizations and professional development opportunities


Action Plan No action is needed; however, to further promote professional activities, the Program will incorporate a portfolio assignment in which the students will choose a professional journal article to study. They will be required to take and pass the CE Directed Reading Quiz after reading the article.

Program Effectiveness Measures and Data Measure 1: Students will pass the ARRT national certification on the 1st attempt within 6 months of graduation.

Timeframe Within one year of graduation


Evaluation Instrument ARRT 1st Time Pass Rates


14

Results and Analysis (Composite Summary) Class of 2014 18 students graduated 17 students took the test within 6 months of graduation One student failed 94% pass rate


Class of 2015 Results not available at time of reporting


Measure 2: Students pursuing employment will be gainfully employed within 6 months of graduation



Evaluation Instrument Graduate Survey


Results and Analysis (Composite Summary) Class of 2014 18 students graduated 17 students pursued employment One student has continued education 100% of the students seeking employment became employed Benchmark target exceeded.

Class of 2015

Results not available at time of reporting

Action Plan The Program will continue to monitor. The program has had 100% job placement rate for the last 5+ years.

15

Measure 3: Students will complete the program within 21 months



Evaluation Instrument Program attrition/retention and graduation rates


Results and Analysis (Composite Summary) Class of 2014 19 students selected One student failed (physics) 18 graduated 18/19 = 95% retention rate Benchmark target exceeded. Class of 2015 19 students selected Three students failed (one imaging and two physics) One withdrew for medical reasons One reentered the Program (failed physics in 2014) 16 graduated 15/19 = 80% retention rate Benchmark target met.

Action Plan o Reevaluate the selection process. o Add the Test of Essential Skills to the selection process. o Evaluate the tests’ validity. o Require all prerequisites to have a “B” or better. o Remove CS 110 and add CHEM 110G as a prerequisite. CHEM 110G is a

prerequisite To BIOL 225 and has fundamental concepts for physics and radiation biology courses.

o Require Math 120 or higher and remove AHS 116. Math 120 transfers readily to other universities and is a more rigorous course.

o Require BIOL 225, Anatomy and Physiology 1 and not BIOL 154. BIOL 225 is a more rigorous course. Increasing the standards will hopefully recruit students who are better prepared for the course work.

16

Measure 4: Students will be satisfied with their education



Evaluation Instrument Graduate Survey

Benchmark Target Overall rating of 4.0 or higher out of a 5 point Likert scale (See Glossary)

Results and Analysis (Composite Summary) Class of 2014 18 graduates 12 responded to graduate follow up survey

Radiologic Technology Education Program Graduate Class of 2014

Please rate and comment on the OVERALL quality and your preparation as an ENTRY-LEVEL radiographer.


Response Count



3 Neutral 0.0% 0



N/A 0.0% 0

Comments: 6


skipped question 0

The survey return rate was 4.9. Benchmark target exceeded.

17

Class of 2014

Class of 2015 Results not available at time of reporting

Action Plan The Program will continue to send our graduate follow up surveys online through Survey Monkey to increase the response rate. The class of 2014 had a 67% response rate which is up from a 26% response rate from the class of 2013.

Measure 5: Employers will be satisfied with the graduate’s performance



Evaluation Instrument Employer Follow up Survey

Benchmark Target Overall rating of 4.0 or higher out of a 5 point Likert scale (See Glossary)

Results and Analysis (Composite Summary)

18

Class of 2014 Radiologic Technology Education Program - Employer Survey

Please rate and comment on the overall quality of this Radiologic Technology Education program’s graduate.


Response Count



3 Neutral 0.0% 0



N/A 0.0% 0

Comments: 7


skipped question 1

The survey return rate was 4.64. Benchmark target exceeded. Class of 2015 Results not available at time of reporting

Action Plan The Program will continue to send our employer surveys online through Survey Monkey to increase the response rate. The overall satisfaction rating for the class of 2014 was a 4.64. This response rate was 77%, an increase of 21% from the response rate for the class of 2013.

Possible Program Research Questions for 2015-2016 1. Do screening criteria appropriately identify students who are likely to be successful in

the program? a. Prior to program entry, what factors had an impact on program fail rate? b. Do students who take BIOL 154 have a higher failing rate than those who take

BIOL 225? c. In the screening process, will changing the bonus points for specific courses

(BIOL 226 or 227 and PHYS 211) improve program pass rate? d. How many students got extra points for CHEM 110 in the last two years? Are

there data to support having made it a prerequisite? 2. If the program looks back at the students who fail the ARRT on the first attempt, what

grades did they earn during the program? If the program raises the passing scores from a 74% to a 78%, will that eliminate any students from the program? If so, what is more important, student retention in the program or a high first attempt pass rate?

3. Does our assessment plan show that the students are improving from their first year into their second year? Analyze and report on the data collected comparing the two classes against each other.

Documents

Outcomes Assessment Plan 2014-2015