ORIGINAL CONTRIBUTION computer-assisted instruction; paramedics, continuing education

Efficacy of Computer-Assisted Instruction in the Continuing Education of Paramedics

Study objective: To determine the relative effectiveness of lecture, video, and computer-assisted instruction (CAI) for the contirluing education of paramedics.

Design: The study was a statewide, cross-sectionaL and prospective. It used premethod, post method and 60-day evaluations addressing both cog- nitive knowledge and attitude toward the method. Subjects were required to complete a 20-question cognitive examination and a ten-point satisfac- tion inventory. Each study site was randomly assigned a method by a computer-generated table.

Setting: The study involved I1 sites, systematically distributed through- out Michigan.

Type of participants: i l l Michigan licensed paramedics. Results: CAI subjects scored an average of 79.6% on the post-test, while

lecture and video subjects scored 70.5% and 68.9%, respectively. At 60 days, CAI subjects scored 70.9%, while lecture and video subjects aver- aged 59.4 and 59.1, respectively. The difference among the scores was sig- nificant post method (P = .0I) and at 60 days (P = .02). The subject method preference was lecture, video, and CAI premethod; lecture, CAI, and video immediately post method; and lecture, CAL and video at 60 days.

Conclusion: Although lecture was the preferred method, CAI was best able to impart knowledge and enhance participant knowledge retention. [Porter RS: Efficacy of computer-assisted instruction in the continuing ed- ucation of paramedics. Ann Emerg Med April 1991;20:380-384.]

INTRODUCTION Continuing education of paramedics has been a concern since para-

medics began to provide advanced emergency care procedures in the field. 1-3 This study examined three methods of continuing education for the paramedic - lecture, videotape, and computer-assisted instruction (CAI) - and compared their relative abilities to promote knowledge acqui- sition and retention. The study also examined subject attitude toward each method initially and any changes in attitude immediately after and 60 days after the method.

MATERIALS A N D METHODS The topic selected was "Kinetics of Trauma," a content presentation

modeled after the "Kinematics of Trauma" lecture of the National Asso- ciation of Emergency Medical Technicians' (EMTs') prehospital trauma life support (PHTLS) course and the "Mechanism of Injury Due tO Motion" lecture of the Alabama American College of Emergency Physicians ' basic trauma life support (BTLS) course. 4,5 This topic and the basic trauma life support and prehospital trauma life support courses were not available in Michigan at the time of the study. The topic's inclusion in the two trauma courses and within the 1986 Department of Transportation paramedic cur- riculum identified its importance in paramedic education. 6 This topic also ensured that there was no prior training to bias the measurement of learn- ing and retention.

A presentor was selected based on recommendations of the state training director. The individual was a certified EMT-paramedic instructor whose

Robert S Porter, MA, NREMT-P Pittsburgh, Pennsylvania

From the Center for Emergency Medicine of Western Pennsylvania, Pittsburgh.

Received for publication May 22, 1990. Revision received October 1, 1990. Accepted for publication November 6, 1990.

Presented at the Society for Academic Emergency Medicine Annual Meeting in Minneapolis, Minnesota, May 1990.

Address for reprints: Robert S Porter, 5050 Bowen Drive, Nedrow, New York 13120.

20:4 April 1991 Annals of Emergency Medicine 380/73

PARAMEDIC EDUCATION Porter

students performed well on the state licensure examination. He was used as both the presentor for the lecture Variable and commenta tor for the video pro- Age (yr) duction and was also involved in the Minimum lesson plan, video script, and CAI Maximum content development and review pro- Education cess. A single individual was used to Minimum ensure that instructor variability did Maximum not affect method performance. Since education

A 0no-hour lecture lesson plan and Minimum accompanying slide series were de- Maximum veloped. The outline and slides were used t o prepare the v ideo script, Training hours which in turn was broken into the Minimum

Maximum data screens for the CAI. 3~ne average elapsed t imes for lecture, video, and Male CAI presenta t ions were all w i th in Female five minutes of each other. Full-time

The l e c t u r e l e s son plan, v ideo Part-time script, and computer screens were re- viewed by a jury of prehospital emer- gency medicine services educators to ensure appropriateness and equali ty Of content. Each of the four experts was a paramedic instructor/coordina- tor with at least four years of teach- ing experience. Three of the four had bachelor ' s degrees and o n e had an Age associa te degree. T h e y were god- Education graphical ly d is t r ibuted t h roughou t . Since education the state. Training hours

The CAI program was developed Pretest following current recommendat ions for CAI design. The resul t ing tool Post-test was basically an enhanced electronic Follow-uP page turner.7, 8 interspaced every few pages were questions regarding mate- rial presented on the previous page. A correct response allowed the sub- ject to progress, whereas an incorrect Age response reduced the par t ic ipant ' s Education score and redisplayed the previous Since education page with the object of the question Training hours highlighted. The question was asked Pretest again, and the student was not al- P0st-test lowed to progress unt i l it was an- F0110w-up swered correctly.

Initially, each subject was asked to comple te a demographic quest ion- naire regarding age, sex, educational level in years, date paramedic train- Age ing was completed, hours of continu- Education ing education for the past year, and Since education full- or part-time status. Training hours

The sat isfact ion inven to ry asked Pretest the p a r a m e d i c to r e s p o n d to ten

ques t ions tha t c o n t a i n e d the key P0st-test words "familiarity, . . . . speed, . . . . vol- F0110w-up ume," "quality, . . . . ease," "comfort ," "interest ," "pleasant ," " re tent ion ," and "frustration." The last key word was intentionally negative to ensure subjects were reading the inventory

TABLE l. Mean values for demographic variables

Overall Lecture Video Computer

32.2 30.7 31.2 35.5 18 18 21 25 60 45 60 58

13.8 13.4 13.7 14.3 12 12 12 13 19 19 15 17

4.3 2.7 3.9 7.1 1 1 1 1

11 11 11 13

20.0 16.3 20.8 25.3 0 0 0 4

99 99 90 99

81 (73%) 34 (65%) 19 (73%) 28 (85%) 30 (27%) 18 (35%) 7 (27%) 5 (15%)

71 (64%) 28 (54%) 18 (69%) 25 (76%) 40 (36%) 24 (46%) 8 (31%) 8 (24%)

TABLE 2. Pearson correlation coefficients

Lecture Since Training

Education Education Hours Pretest Post-Test Follow-up

0.1921 0.3348 0.0388 0.2066 0.2909 0.1763

0.2452

0.0370 0,2593 0.2268 0.1734 0.0364 0.1728 0.0258 0.0576 0.1549

0.3911 0.3511 0.1225 0.1013

Video

0.06531

Since Training Education Education Hours Pretest

0.1413 0.1121 0.0049 0.1078

0.3994

0.1619 0.0099 0.2660 0.1375 0.2340

0.1084 0.1271 0.0425 0.5567

0.1764 0.0649 0.0951 0.2922

Compute~Assisted Instruction

Post-Test Follow-up

0.0448 0.1784

0.4427

Since Tra!ning Education Education Hours Pretest

0.0387 0.0805 0.2450 0.2673

0.2905

0.1442 0.1516

0.2020 0.0460 0.1451 0.1066 0.3356 0.1444 0.5904

0,3309 0.3433 0.0070 0.5757

Post-Test Follow-up 0.3750 0.1878

0.4307

before p r o v i d i n g a r e sponse . Re- sponses were recorded on a modified, four-point Likert scale: 4 was the

max imum positive value, and 0 was the most negative. The "frustration" response average was reversed in de-

74/381 Annals of Emergency Medicine 20:4 April 1991

PARAMEDIC E D U C A T I O N

Porter

TABLE 3. Satisfaction inventory (means)

Lecture Video Computer-Assisted Variable Pretest Post-Test 60 Days Pretest Post-Test 60 Days Pretest Post-Test Familiarity 3,86 3.83 3.72 - * 2.85 2.69 - 2,59 - - 1.67 1.95 +

Speed 3,31 3.27 3.27 3,03 2.61 - - 2 , 3 6 - - 2.67 2.87 +

Volume 3.17 3.18 3.21 2.77 2.73 2 .27 - - 2.73 3.22 + +

Quality 3.48 3 .37 - 3.33 - 2.77 2.42 - - 2.23 - - 2.61 3.00 + +

Ease 3.31 3.33 3 . 1 8 - 2.85 2 . 5 4 - - 2 . 23 - - 2.55 3.04+ +

Comfort 3.63 3.63 3.58 3.19 2.69 - - 2.32 - - 2.48 2.96 + +

Interest 3.08 3.14 3.21 + 2.88 2.54 - 2.00 - - 2.79 2.83

Pleasant 3.08 3.24+ 3.21 + 3.00 2 . 5 8 - - 2 , 18 - - 2.70 2.83+

Retention 3.25 3,31 3.24 2.69 2.78 + 2.23 - - 2.76 3.13 + +

Frustration 2.19 3.02+ + 2.48+ 2.50 2.78+ 2.50 2.55 2 .30 -

Overall 3.20 3.12 3.15 2.75 2 .45 - - 2 . 19 - - 2.44 2.75+ +

SD (0.57) (0.43) (0.49) (0.45) (0.73) (0.53) (0.42) (0.45)

*A plus or minus indicates an increase or decrease in the Liked score from the pre-method by at least 0.1. A second plus or minus sign identifies a change of at least 0.3.

60 Days

2.15+ +

2.80 +

3.05 + +

2.90 +

3.10+ +

2.95 + +

2.95 +

2.80+

3.15+ +

2 .10 - -

2.88 + +

(0.36)

termining the overall inventory aver- age for a method.

Seventy-two questions were cre- ated from the video script and field tested by paramedic students. Ques- tions also were reviewed by the jury of experts, with the best 60 being sys- tematically divided among three ex- aminations. The three examination versions were rotated through each method at least once to serve as a pretest, post-test, and follow-up ex- amination. This ensured that method performance was not affected by dif- ferences in test difficulty levels. It also ensured that subjects were given a different examination at each test- ing time.

One hundred eleven licensed para- medics participated in 11 continuing education sessions held throughout Michigan. The sites were distributed systematically throughout the state and randomly assigned a method (by computer-generated table). Subjects were unaware of the m e t h o d in which they were to participate until they arrived at the site.

When CAI or v i d e o was t he method to be used, each student was given the program disk or videotape and the option to complete the train- ing on-site or at their leisure (within 24 hours). Most participants elected to train on-site, although a few chose to return the disk by stamped, self- addressed envelope or the videotape through the site coordinator. It goes wi thout say ing t ha t the l e c t u r e method did not offer this option.

Statistical analysis was performed by the Compute r Services Depart-

ment of Central Michigan Univer- sity. Pearson's correlation coefficient, analysis of covariance (ANCOVA), and Tukey's studentized range test were selected as statistical tools. An c~ of .05 was predetermined as the de- sired level of statistical significance.

RESULTS The typical par t ic ipant was 32

years old, had f inished paramedic training four years earlier, had at- tended 20 hours of continuing educa- tion in the previous year, and had an education of just less than two years beyond high school. Seventy-three percent of the subjects were male, and 64% were employed full-time as paramedics (Table 1).

ANCOVA failed to demonstra te significant demographic differences among the three subject groupings (~ = .05). Pearson's correlation coeffi- cient analysis identified that none of the demographic factors held a strong relation {more than .6) to the pre- evaluation, postevaluation, or 60-day follow-up evaluation scores (Table 2). P values ranged from .16 to .77.

The satisfaction inventory revealed that students initially preferred lec- ture, video, and then CAI (Table 3). When the methods were completed, the preference order was lecture, CAI, and video. At the 60-day evalua- tion, the preference remained lecture, CAI, and video. The 60-day inven- tory displayed no significant differ- ence between lecture and CAI, while a l l o t h e r c o m p a r i s o n s a m o n g methods were significant (P < .05). P values among methods were as fol-

lows: Pretest: lecture-video, .23; lec- ture-CAI, .18; video-CAI, .31; post- test: lecture-video, .39; lecture-CAI, .02; video-CAI, 03; and 60-day: lec- ture-video, .45; lecture-CAI, .02; and video-CAl, .02.

The lecture inven tory remained cons i s t en t t h r o u g h o u t the study, w h e r e a s b o t h l e c t u r e and v ideo values displayed trends (Figure 1). CAI subjects felt more positive to- ward their method at both imme- diately after the method and at 60- day evaluations, whereas the oppo- site was true of video subjects. CAI and video comparisons between pre- test and immediately afterward and between pretest and 60-day scores were found to be significantly differ- ent (P < .05). P values over t ime wi th in a method were as follows: p re tes t to pos t - t es t : lec ture , .47; video, .03; CAI, .03; and post-test to 60-day: lecture, .69; video, .02; and CAI, .04.

ANCOVA revealed no significant differences among subject groups be- fore methods were used (P = .36). However , at the i m m e d i a t e post- m e t h o d eva lua t ion , CAI subjec ts scored significantly better (79.6) than either video (68.9) or lecture (70.5) groups (P = .0136). Tukey's studen- tized range test found no difference between lecture and video groups.

At 60 days, the difference between CAI and the other two groups in- creased. CAI subjects performed bet- ter (70.9) than lecture (59.4) and video (59.1) subjects by an average of 11.5% and 11.8%, respectively. The difference was again significant (P =

20:4 Apr i l 1991 Annals of Emergency Medicine 382 /75

PARAMEDIC EDUCATION Porter

FIGURE 1. Satisfaction i~ventory by method over time.

FIGURE 2. Examination score differ- ence between methods over time.

.02). No signif icant difference was found between the lecture and video scores (Figure 2).

DISCUSSION The results of the satisfaction in-

ventory demonst ra te an initial and continued preference for the lecture fo rma t of p resen ta t ion . This m a y have been because lecture was the traditional and most familiar method for continuing education of the para- medic. The decline in preference for the video format was a surprise. It may have been because the premeth- od inventory reflected positive stu- dent feeling toward all video presen- tations, i nc lud ing h o m e video and te levis ion materials . As the video was viewed, subjects may have ad- justed their Likert scores to reflect their feelings toward videos specifi- ca l ly a s soc i a t ed w i t h e m e r g e n c y medical services training.

The con t inu ing i m p r o v e m e n t in the overal l s a t i s fac t ion i n v e n t o r y value toward CAI (2.44, 2.75, and 2.88) was possibly a result of increas- ing familiari ty (1.67, 1.95, and 2.15) and the reasonably positive experi- ence with the method.

CAI subjects were better able to acquire knowledge than either lec- ture or video subjects. This may have occurred for many reasons. The stu- dents were allowed to move through the CAI at their own pace, and data screens could be reread if the student desired. The s tudent also was not permitted to move on until a particu- lar question was answered correctly. Careful reading and understanding of the content were rewarded by unob- s t ructed progress th rough the pro- gram and posi t ive feedback in the form of a running score.

In comparison, the lecture subjects had to follow at the lecturer's pace, as he spoke to 15 to 20 persons. The video subjects were similarly limited to the pace of presentation, al though the tape could be replayed or paused to meet the needs of the paramedic. The lecture and video methods did not allow for individual feedback. They would also permit a student to a t tend and receive credi t w i t h o u t

m

...................................... iiiiiii i !!ii

! i H

!i~i~!iiii:iii!iiiiiiili:i~i!!ili~ili!i ~iii~!iOii~ii~iii~ iiOiO00iiiiiOiO0 :iiii~ii~iiiii0ii~ii~

Lecture Video Computer [ ] Pretest [ ] Post-Test [ ] 60Days

15

10

iiiiiii',iiiiiiiiii iiii

ili!:ili i iiiiiiiiiiiiiii=,!iiiiii!i ~ iiNiiNii~i" iiiOiiii iiiiOiiii! i '

;ili i iiiiiii!riiiii:iiii!!i 0!~i0"G'0 0!ii0i0i0iiii

CAI- Lecture

[ ] Pretest

iii iiiiiiiii ..............................

~,;~ ~¢ ~ -

0.3 CAI - Video Lecture - Video

[ ] Post-Test [ ] 60 Days

necessar i ly l i s tening to or concen- trating on the content.

CAI m e t h o d subjects m a y have displayed better knowledge retention than video or lecture participants for the same reasons CAI displayed bet- ter knowledge acquisition. The com- puter-based format m a y also have benefitted from the active participa- t ion required for program comple- tion.

Research in o ther discipl ines re- garding CAI predicted these results. 9 Kulik et al, in their meta-analysis of 23 studies, identified that CAI stu- dents scored in the 66th percentile compared with their traditional lec- ture counterparts, lo

It was interesting to note that over- all subjects ' scores decreased 13.5% be tween their p o s t m e t h o d and 60- day examinations. This degradation of k n o w l e d g e appeared h igh for a two-mon th period. Lecture s tudents decreased the most (15.7%), whereas

CAI s tuden t s decreased the least (10.9% }. Video subjects experienced a 14.2% reduction. (The reduct ion in test score was computed as follows: [Post-test score minus 60-day score]/ p o s t - t e s t score . ) D e g r a d a t i o n of knowledge was found to be signifi- cant between CAI (P = .02) and the other methods but not between lec- ture and video (P = .33).

It was surprising to note that the video subjects scored about as well as the lecture subjects (no s ignif icant difference) even though they felt less positive toward their method. This trend cont inued through the 60-day follow-up examination.

The lecture method was the easiest presentat ion to execute. It required the least resources: a lecturer, a slide projector and slides, and a screen. It could be given to a very large group and was the most familiar method of educa t ion to the par t ic ipants . The video was also easy to present. It re-

76/383 Annals of Emergency Medicine 20:4 April 1991

PARAMEDIC EDUCATION Porter

quired only a television monitor and a videocassette recorder. It could be played to only a small group, al- though it could be played at the con- venience of the student or students.

The use of the CAI tool was depen- dent on availability of a computer and could be used by only one stu- dent at a time. Computers were diffi- cult to find and posed the greatest obstacle to the provis ion of this training. As with the video, it could be used any time the hardware and student were available.

RECOMMENDATIONS This study examined only the ap-

plication of the three methods pre- senting a single subject. The subject was didactic in nature and lent itself to CAI application. Further research of CAI for paramedic continuing edu- cation could address the more tradi- tional topics of acute medical prob- lems, emergency obstetrics, psychia- try, and so on to determine if CAI is well suited to other content areas.

Additional study may be needed to observe n u m e r o u s topics over a longer period to determine if the su- perior performance of the CAI may

be a result chiefly of its uniqueness or other phenomenon. Such a study may also determine if lecture will maintain its preference over CAI in an environment in which computers are easily accessible and the method is used frequently.

This study suggests the potential to improve knowledge acquisition and retention afforded by CAI. It has the advantage of being available on an individual basis and can record student progress and performance for later review by the instructor or pro- gram medical director. Hence, there may be merit in the development and use of CAI to augment the initial and continuing education of para- medics.

CONCLUSION In this particular study, while ad-

dressing only one topic and while novel to the users, CAI demonstrated a superiority in knowledge acquisi- tion and retention over lecture and video presentations in the continuing education of paramedics. Paramedics preferred lecture, but acceptance of CAI improved significantly with its use and with time. At the comple-

tion of the study, the preference for the lecture method over CAI was not significant.

REFERENCES 1. Gray MS: Recertification and relicensure in the allied health professions. J AIlied Health 1984;i3:22-30.

2. Dean NC, Haug PJ, Hawker PJ: Effect of mobile para- medic units on outcome in patients with myocardial in- farction. Ann Emerg Med i988;17:1034~1040.

3. Landis SS, Benson NH, WhitIey TW: A comparison of four methods of testing emergency medical techni- cian triage skiIls. Am J Ernerg Med 1989;7:1-4.

4. Creel JH: Mechanisms of injury due to motion, in Cambell JE ted): Basic Trauma Life Support, ed 2. En- glewood Cliffs, New Jersey, Brady, 1988, p 1-20.

5. Kinematics of trauma, in National Association of Emergency Medical Technicians: Pre-HospitaI Tramna Life Support, ed 1. Akron, Ohio, Emergency Training Inc, 1990, p 1-24.

6. US Department of Transportation, National High- way Traffic Safety Administration: Emergency Medical Technician-Paramedic: National Standard Curricudum

Course Guide. Washington, DC, US Govermnent Printing Office, 1986.

7. Eisenburg Y: The effects of computer-based instruc- tion on the college students' interests and achievement. Ed Technoi 1986~5:13-15.

8. Norcini JJ, Meskauskas JA, Langdon LO, et al: An evaluation of a computer simulation in the assessment of physician competence. Evai Health Prof 1986;9: 286-304.

9. Pazdernik TL, Walaszek EJ: A computer-assisted teaching system in pharmacology for health profes- sionals. J Health Educ 1983;58:341-348.

10. Kulik CC, Kulik JA, Shwalb BJ: The effectiveness of a computer-based adult education: A meta analysis. J Educ Comptlt Res 1986;2:235-252.

20:4 April 1991 Annals of Emergency Medicine 384/77