393

Telemedicine Journal and e-HealthVolume 6, Number 4, 2000Mary Ann Liebert, Inc.

Improved Rural Provider Access to Continuing MedicalEducation Through Interactive Videoconferencing

PETER W. CALLAS, Ph.D., MICHAEL A. RICCI, M.D., and MICHAEL P. CAPUTO, M.S.

ABSTRACT

We sought to describe use patterns and user evaluation of remotely-attended continuing med-ical education (CME) programs in Vermont and upstate New York. Remote attendees wererequired to return an evaluation form to receive CME credit. The form included name anddate of the program; name, location, and specialty of the respondent; and questions regard-ing program quality, value, effectiveness, and attendee plans if the program had not beenavailable via telemedicine. From April, 1996, through December, 1998, health care providersfrom 14 remote sites used the network 927 times to attend 394 CME programs at Fletcher AllenHealth Care in Burlington, Vermont. After the start-up period, an average of over three pro-grams per week was attended, with an average of 2.4 remote attendees per program. Seventy-seven percent of remote attendees stated that they would not have attended the program if ithad not been available over telemedicine, while the remaining 23% said that they avoidedtraveling due to videoconferencing. When asked the effectiveness of telemedicine technol-ogy for attending, 73% said it was as effective as having the presenter in the room, 23% saidit was less effective, and 4% said it was more effective. Major technical problems, such ashaving the call disconnect during the presentation, decreased over time. There were contin-uing minor logistical problems common to large group videoconferencing. The telemedicinesystem has increased availability of CME programs for rural providers in Vermont and up-state New York. Most attendees have found the programs to be worthwhile, and technologi-cal advancements have improved the quality of the system.

University of Vermont College of Medicine and Fletcher Allen Health Care, Burlington, Vermont.

INTRODUCTION

AMAJOR DIFFICULTY FOR HEALTH CARE PROVIDERS

in rural areas is access to continuing med-ical education (CME).1 This in turn can lead tofeelings of professional isolation and an in-ability to remain current with medical ad-vances.2 Availability of CME via interactivevideoconferencing, or telemedicine, has the po-tential to decrease isolation and travel needswhile increasing medical knowledge and feel-ings of being part of a medical community.3,4

Television has long been recognized as a po-tential way to make CME more widely avail-able,5 but technical limitations and high costshave limited development of systems for re-mote CME viewing. Most telemedicine pro-grams attempted during the 1960s and 1970sfolded due to lack of sustained funding.6 Re-cent improvements in telecommunicationstechnology and means of transmission to pre-viously inaccessible areas have increased thefeasibility of developing viable telemedicinenetworks for medical education and patient

care. There are currently many such networksbeing deployed in the United States andthroughout the world.7–10

In rural Vermont and upstate New York, aninteractive videoconferencing network hasbeen established between 15 hospitals andphysician offices.11 A major use of this networkhas been long-distance participation in CME.This study describes patterns of use and userevaluation of CME programs attended viatelemedicine during the first 33 months of op-eration.

MATERIALS AND METHODS

In January, 1996, a desktop interactive video-conferencing network was deployed connect-ing sites in Vermont and upstate New York.The system is centered at the University of Ver-mont’s major teaching hospital, Fletcher AllenHealth Care (FAHC), a 500-bed tertiary-caremedical center in Burlington, Vermont (popu-lation 39,300).

Remote sites were added throughout 1996–1998. By December, 1998, the network encom-passed 15 sites, including the hub at FAHC.The system underwent a major redesign inequipment after approximately 12 months. Thechanges and reasons for these changes havebeen described previously.11 The current sys-tem consisted of an IBM personal computerwith a Pentium (166 MHz) processor, 32 MB ofrandom-access memory, PCI local bus SVGA,and 2 MB vRAM. A SONY Multiscan 17-inchmonitor, omnidirectional microphone, andCannon VC-C1 Mark II camera were also in-cluded. The entire system rests on a portable,custom-designed cart. The primary conferenceroom used for grand rounds has two ceiling-mounted remote control cameras and four mi-crophones—one on the podium, one lapel, andtwo ceiling-mounted for transmitting audiencequestions.

Microsoft Windows 95 was the operatingsystem. The videoconferencing codec was fromthe Zydacron Corporation (Manchester, NewHampshire). The system used the Zydacron Z-250 BRI videoconferencing card and the Zy-dacron 208 Multiple BRI interface communica-tion board. Software applications included the

videoconferencing software (Zydacron OnWan2.01) and Microsoft NetMeeting.

Standard transmission for both CME andteleconsults utilized three ISDN basic rate in-terface (BRI) lines (384 kbps). In addition, abridge capable of handling six simultaneouscalls was available for multipoint conferencing.Multiple sites within FAHC were wired forvideoconferencing, including a specially de-signed telemedicine room and multiple con-ference rooms and auditoriums.

The videoconferencing network was used forCME conferences, clinical consultations, andadministrative meetings. CME programs in-cluded regularly scheduled grand rounds,morbidity and mortality conferences, plusother educational offerings. The schedule ofprograms was available at the FAHC telemed-icine web site (http://www.vtmednet.org/telemedicine).

To receive CME credit from the University ofVermont College of Medicine, remote attendeeswere required to return an evaluation form. Theform had spaces for indicating the name and dateof the program and the name, location, and spe-cialty of the attendee. In addition to a number ofgeneral evaluation items, two questions specificto telemedicine were included:

1. Would you have traveled to attend thisCME program if it was not available viatelemedicine technology? ___ Yes ___ No

2. Compared to having the presenter in theroom, please evaluate the effectiveness of telemedicine technology for attendingthis CME program. ___ More effective___ As effective ___ Less effective

Forms were on paper; electronic evaluationsare being planned.

Completed forms were sent to the Universityof Vermont Office of Continuing Medical Edu-cation. For this report, forms were collated byCME program, responses were summarized,and descriptive statistics were computed.

RESULTS

From April 1, 1996, to December 31, 1998,health care providers from 14 remote sites used

CALLAS ET AL.394

videoconferencing 927 times to attend 394 CMEprograms at Fletcher Allen Health Care. Of re-mote attendees, 70% were physicians, 5% wereregistered nurses, 2% were nurse practitioners,2% were physician assistants, 1% were psy-chologists, and 20% were other or unspecified.

While only 10 programs were remotely at-tended during the first 6 months, there was anaverage of over three programs attended perweek after this (Fig. 1).

Three remote sites accounted for over half ofthe total number of programs attended, al-though nine sites averaged more than one pro-gram per month since being connected to thenetwork. The number of programs attended byeach site during the study period ranged fromone to 121, with a mean of 39 programs per site.

For 271 (69%) of the programs, providersfrom only one site attended remotely, whereas94 (24%) had providers from two sites attend,27 (7%) had providers from three sites attend,and 2 (1%) had providers from five sites attend.The mean number of remote attendees per pro-gram was 2.4. About 10% of the programs hadfive or more remote attendees.

Table 1 shows the number of remote atten-dees and number of programs by specialty.Most attendance was at programs from fourspecialties—medicine, surgery, OB/GYN, andpediatrics. Radiology also had a large numberof attendees, although at fewer programs.

In evaluating the CME programs, most remote attendees answered “excellent” or“good” for questions regarding the preparationand presentation of the speaker, practical valueof the program to daily practice, appropriate-ness of the topic for educational needs, effec-tiveness of learning aids used, time for discus-sion, and overall value of the presentation. Theeffectiveness of learning aids used was rated“fair” or “poor” by about 10% of the respon-dents. Almost two-thirds of the attendees saidthat the program will alter their daily practiceperformance.

For the questions specifically related totelemedicine, 77% of the respondents said thatthey would not have attended if the program hadnot been available via telemedicine, whereas theremaining 23% stated that they would have trav-eled to attend. Telemedicine was judged to be a

IMPROVED ACCESS TO CME VIA TELEMEDICINE 395

FIG. 1. Telemedicine CME by month: number of programs and number of remote attendees.

less effective way to attend the program than inperson by 23% of the respondents, as effectiveby 73%, and more effective by 4%. Interestingly,for 11 programs, telemedicine was rated as lesseffective than in person by one attendee andmore effective than in person by another at-tendee at the same program. One respondentwho felt the technology was less effective notedthat “less effective than ‘being there’ does notmean this modality is not useful.”

In open-ended comments on the evaluationform, 20 positive comments regarding telemed-icine were received, such as “Telemedicine is agreat way to help meet the needs of the smallerhospitals.” In addition, there were 121 generalpositive comments, such as “Excellent pro-gram, very useful” and “Look forward to con-tinuing to participate.”

Negative comments related to telemedicineprimarily revolved around technical aspects,such as poor audio/video transmission or un-expected disconnections, and logistical aspects,such as slides that were difficult to see or poorcamera placement. There were 48 commentsconcerning technical difficulties. Seven of thetechnical comments came in the first 26 pro-grams, whereas the other 41 were spread outover the remaining 368 programs.

A total of 85 comments on logistical issueswere received. Examples include:

� “Slides were difficult to read with smallprint.”

� “Paging speakers interfere with audio of pre-senter—can’t it be turned down during pre-sentation?”

� “Sound is difficult if speaker doesn’t look atmicrophone.”

� “Encourage lecturer to stand near lectern.”� “Camera was placed behind speaker, block-

ing slides.”� “Please allow time on telemedicine to ask

questions. I could not get a word in edge-wise since no one could quite see or hearme.”

These comments were distributed relativelyevenly across the study period.

DISCUSSION

After a start-up period, remote CME atten-dance was fairly steady. Documented use waslow for the first 6 months for two reasons: re-mote sites were being added during this pe-riod, so there were fewer potential attendees,and evaluation forms were not being returnedby all attendees.

Personal identifiers were removed from theevaluations for confidentiality, so we do notknow how many remote participants were re-peat users. However, the pattern of use by spe-cific sites and specialties suggests that therewere certain individuals who regularly at-tended particular CME programs via video-

CALLAS ET AL.396

TABLE 1. REMOTE ATTENDEES BY PROGRAM SPECIALTY

Number (%) of Mean number ofNumber (%) of remote attendees

Specialty programs attendees per program

Pediatrics 76 (19) 163 (18) 2.1Medicine 72 (18) 235 (25) 3.3OB/GYN 60 (15) 121 (13) 2.0Surgery 60 (15) 91 (10) 1.5Psychiatry 30 (8) 59 (6) 2.0Radiology 22 (6) 102 (11) 4.6Anesthesiology 21 (5) 55 (6) 2.6Family practice 14 (4) 25 (3) 1.8Neurology 7 (2) 7 (1) 1.0Nephrology 6 (2) 13 (1) 2.2Pathology 6 (2) 8 (1) 1.3Cancer center 5 (1) 10 (1) 2.0Other/unspecified 15 (4) 38 (4) 2.5

Total 394 (100) 927 (100) 2.4

conferencing. The telemedicine site coordina-tor at one hospital mentioned that two to threephysicians remotely attend medical grandrounds weekly, and that one pediatrician at-tends pediatric grand rounds almost everyweek, as her schedule permits. Nine sites av-eraged over one program per month since join-ing the system, indicating a possible increasein the number of regular users.

Although attendance may seem low by ur-ban standards, the level of use was consistentwith expectations, given the sparsely popu-lated areas served and the difficulties of ac-commodating all schedules for 15 sites. Timedemands and scheduling conflicts can ad-versely affect attendance, as has been found inother telemedicine education programs.12–14

We have now created a matrix consisting of allCME events occurring at the remote sites sothat we can coordinate programs better.

Telemedicine directly benefited rural physi-cians, nurses, psychologists, and other healthprofessionals in Vermont and upstate NewYork by increasing access to CME while de-creasing travel time and costs. During thestudy period there were 651 instances whereproviders viewed programs that they wouldnot have attended if the program had not beenavailable via telemedicine. Travel was saved in199 instances where providers used telemedi-cine to view programs that they otherwisewould have traveled to attend. Other studiesof medical education via telemedicine have hadsimilar findings of increased access15–19 and de-creased travel needs.20,21

A benefit related to this increased access isdecreased professional isolation of rural healthcare providers. This is often stated as a primarygoal of using telemedicine to bring CME torural areas.4,22–24 Surveys have found reduc-tions in perceived isolation due to telemedi-cine.25,26 Although we did not specifically askabout this, there were a few comments indi-cating feelings of greater connections withpeers. For example, one pathologist mentionedthat “it’s really exciting to be back in the uni-versity atmosphere.”27 Reduced isolation maybe one factor leading to greater retention ofrural health care providers,2,28 but long-termevaluation of this needs to be done.

Although positive feedback has been obtained

from users of telemedicine for CME,14,16,29,30

there are several problem areas that need ad-dressing. Remote attendees in our study re-ported a number of technical troubles, such ashaving the call hang up unexpectedly. Such in-cidences occurred throughout our networkduring the early months, but decreased after achange in videoconferencing hardware andsoftware.11 Other telemedicine programs havealso suffered technical difficulties which dis-rupted programming.12–14,31

In addition to technical programs, many lo-gistical problems were reported by remote CMEattendees in our study. These included poor mi-crophone and camera placement, slides andoverheads that were too small, and lack ofawareness of the presenter that there were peo-ple attending via telemedicine. Although majortechnical problems decreased over time, theselogistical problems continued throughout thestudy period. However, the feedback providedby the remote attendees and the experiences re-ported by other programs14,29,32 are being usedto improve the presentations. Room micro-phones and an additional camera have beenadded to the auditorium used for grand rounds,and lighting has been redesigned. Presenterswere originally oriented toward the in-personaudiences, but are now learning the visual ma-terial requirements of the system. Templateshave been developed that produce slides withimproved readability at remote sites. In addi-tion, FAHC has started to integrate video con-ferencing and web-based educational materials,with some lecturers incorporating websites liveinto their presentations. This merger of tech-nologies allows for interactive video discussionsbetween participants while viewing high qual-ity presentation materials on the Web.

In support of guest speakers, FAHC has pur-chased a slide to video converter (ELMO TRV-35H), which is used to provide video images ofthe slides. Mixing multiple audio-visual inputshas been addressed by the addition of an au-dio and a video mixer. The audio mixer allowsfor various inputs such as the speaker, audi-ence, and computer-generated sounds to besent over the telemedicine network. The videomixer is used to easily switch between videosources such as the computer, speaker, docu-ment stand, etc.

IMPROVED ACCESS TO CME VIA TELEMEDICINE 397

For telemedicine systems to succeed, theymust be receptive of input from the users.33–35

Technical and logistical problems such as thosedescribed above can lead to decreased interestin and use of the system, particularly amongpersons less inclined to try new technology.36

A limitation of this study is that we did nothave data on potential remote attendees whodid not participate, and their reasons for this.One approach we are using to obtain such in-formation is visiting each site to meet with themedical staff.

Another limitation is that we have not con-ducted formal cost-effectiveness or cost-benefitevaluations of providing CME via telemedi-cine. The issue of costs versus benefits is diffi-cult to evaluate in the short run. It may be thata telemedicine network is judged worthwhileeven if it does not result in actual financial sav-ings, but provides increased access to CME ata cost that is not prohibitive, particularly if ef-ficiencies in clinical care can be introduced andadopted.26,37

Telemedicine can only improve access toCME if the programs are attended. Althoughwe observed fairly steady use once the systembecame fully functional, the pattern beyondthis is difficult to predict. Some programs haveseen use increase as the system matures,9,30

whereas others have seen use decrease as thenovelty wears off.13,38 Lockyer et al.39 observedphysician use decrease during the second andthird years, and then rebound over the follow-ing 2 years. We will continue monitoring re-mote CME attendance by health care providersin Vermont and upstate New York to deter-mine the long-term effects on access to CME.Future evaluation strategies include imple-menting a system for tracking repeat use, andmodifying evaluation forms to allow compari-son of responses for in-person and remote at-tendees.

ACKNOWLEDGMENTS

This study was supported in part by HRSAgrant 3 H2A CS 00118-01 S1. The authorswould like to thank Diane Scrodin for collect-ing, collating, and summarizing the CME eval-

uation forms received from telemedicine at-tendees.

REFERENCES

1. American College of Physicians. Rural primary care.Ann Intern Med 1995;122:380–390.

2. American Public Health Association. Policy State-ment 9522(PP): Rural health goals: Guaranteeing a fu-ture. Am J Public Health 1996;86:451–453.

3. Crandall LA, Coggan JM. Impact of information tech-nologies on training and continuing education forrural health professionals. J Rural Health 1994;10:208–215.

4. Office of Technology Assessment. Brining health careonline: The role of information technologies (OTA-ITC-624). Washington, DC: U.S. Government Printing Of-fice, 1995:160–161.

5. Warren FZ. Medical television: An evaluation. J MedEduc 1956;31:413–419.

6. Perednia DA, Allen A. Telemedicine technology andclinical applications. J Am Med Assn 1995;273:483–488.

7. Ferguson EW, Doarn CR, Scott JC. Survey of globaltelemedicine. J Med Syst 1995;19:35–46.

8. Grigsby J, Sandberg EJ, Kaehny MM, Kramer AM,Schlenker RE, Shaughnessy PW. Analysis of expansionof access to care through use of telemedicine and mobilehealth services. Report 2: Case studies and current statusof telemedicine. Denver, CO: Center for Health PolicyResearch, 1994:2.1–2.20.

9. Office of Rural Health Policy. Exploratory evaluation ofrural applications of telemedicine. Rockville, MD: HealthResources & Services Administration, U.S. Depart-ment of Health & Human Services, 1997:6,28–31.

10. Wright D, Androuchko L. Telemedicine and devel-oping countries. J Telemed Telecare 1996;2:63–70.

11. Ricci MA, Callas PW, Montgomery WL. The VermontTelemedicine Project: Initial implementation phases.Telemed J 1997;3:197–205.

12. Hartman JT, Moore M. Using telecommunications to im-prove rural health care: The Texas Tech MEDNET Demon-stration Project. Volume One: Final report. Lubbock, TX:Texas Tech University Health Sciences Center, 1992:4.104,4.310–4.311,4.331–4.332,4.511–4.515.

13. Auringer ST. Radiology on the Air: A continuingmedical education experiment. Invest Radiol 1992;27:984–988.

14. Kottke TE, Finger LL, Trapp MA, Panser LA, NovotnyPJ. The Pine Ridge-Mayo National Aeronautics andSpace Administration telemedicine project: Programactivities and participant reactions. Mayo Clin Proc1996;71:329–337.

15. Chouinard J. Satellite contributions to telemedicine:Canadian CME experiences. CMAJ 1983;128:850–855.

16. Elford R. Telemedicine activities at Memorial Uni-versity of Newfoundland: A historical review,1975–1997. Telemed J 1998;4:207–224.

CALLAS ET AL.398

17. Fry CF, Baer C, Cornett S. Interactive television innursing continuing education. J Continuing Educ Nurs1976;7:26–32.

18. O’Neill JJ, Nocerino JT, Walcoff P. Benefits and prob-lems of seven exploratory telemedicine projects. McLean,VA: The MITRE Corporation, 1975:94.

19. Reid DS, Weaver LE, Sargeant JM, Allen MJM, Ma-son WF, Klotz PJ, Langille DB. Telemedicine in NovaScotia: Report of a pilot study. Telemed J 1998;4:249–258.

20. Puskin DS. Telecommunications in rural America:Opportunities and challenges for the health care sys-tem. Ann N Y Acad Sci 1992;670:67–75.

21. Schwarz MR, Schaad DC, Evans FW, Dohner CW.Communications satellites in health education andhealth care provision. J Am Med Assn 1983;250:636–639.

22. Preston J, Brown FW, Hartley B. Using telemedicineto improve health care in distant areas. Hosp CommunPsychiat 1992;43:25–32.

23. Puskin DS. Opportunities and challenges to telemed-icine in rural America. J Med Syst 1995;19:59–67.

24. Zollo SA, Kienzle MG, Henshaw Z, Crist LG, Wake-field DS. Tele-education in a telemedicine environ-ment: Implications for rural health care and academicmedical centers. J Med Sys 1999;23:107–122.

25. Hartman JT, Moore M. Using telecommunications to im-prove rural health care: The Texas Tech MEDNET Demon-stration Project. Volume Two: Attachments. Lubbock,TX: Texas Tech University Health Sciences Center,1992:Attachments 4–8, 4–10, 4–11.

26. McGee R, Tangalos EG. Delivery of health care to theunderserved: Potential contributions of telecommu-nications technology. Mayo Clin Proc 1994;69:1131–1136.

27. Callas PW, McGowan JJ, Leslie KO. Provider attitudestoward a rural telepathology program. Telemed J1996;2:319–329.

28. McCarthy D. The virtual health economy: Telemedi-cine and the supply of primary care physicians inrural America. Am J Law Med 1995;21:111–130.

29. Burleson S, Sugimoto T. Cancer education via video-

teleconferencing. An assessment of participant re-sponses. J Biocommun 1984;11:5–9.

30. Lindsay EA, Davis DA, Fallis F, Willison DB, BiggarJ. Continuing education through telemedicine for On-tario. CMAJ 1987;137:503–506.

31. Furnace J, Hamilton NM, Helms P, Duguid K. Med-ical teaching at a peripheral site by videoconferenc-ing. Med Educ 1996;30:215–220.

32. Hampton CL, Mazmanian PE, Smith TJ. The interac-tive videoconference: An effective CME delivery sys-tem. J Cont Educ Health Professions 1994;14:83–89.

33. Bashshur R, Scott J, Silva J, Neuberger N. Report of theworking conference on telemedicine policy for the NII. Ar-lington, VA: Center for Public Service Communica-tions, 1995:15.

34. Gosbee JW. The white paper on human factors engineer-ing and telemedicine. Kalamazoo, MI: Center for Ap-plied Medical Informatics, Michigan State University.1995:1–23.

35. Puskin DS, Sanders JH. Telemedicine infrastructuredevelopment. J Med Syst 1995;19:125–129.

36. Rogers EM. Diffusion of innovations, ed. 4. New York:The Free Press, 1995:263–280.

37. Bashshur RL. Telemedicine effects: Cost, quality, andaccess. J Med Syst 1995;19:81–91.

38. Rost MA, Barber GM, Frank T. Evaluation of Maine’stelelecture continuing education program: Part II. JContinuing Educ Nurs 1981;12(3):23–30.

39. Lockyer JM, Parboosingh IJ, McDowell AC. (1987).Teleconferencing and the continuing education pref-erences of physicians and nurses in rural areas.Möbius 1987;7(3):53–58.

Address reprint requests to:Peter W. Callas, Ph.D.

Biometry FacilityHills Building

University of VermontBurlington, VT 05405

E-mail: pcallas@zoo.uvm.edu

IMPROVED ACCESS TO CME VIA TELEMEDICINE 399