Computer assisted learning is an effective way of teaching endocrinology

Clinical Endocrinology (2001) 55, 537±542

Computer assisted learning is an effective way ofteaching endocrinology

Richard I. G. Holt, Pawel Miklaszewicz, Iain C.

Cranston, David Russell-Jones, P. John Rees, Peter

H. SoÈnksen

Department of Medicine, Guy's, King's & St Thomas'

School of Medicine, St Thomas' Hospital, Lambeth Palace

Road, London, UK

(Received 16 January 2001; returned for revision 7 March 2001;

finally revised 11 April 2001; accepted 20 May 2001)

Summary

OBJECTIVES Computers are a part of everyday life

and offer an exciting way of learning. The aim of our

study was to determine the effectiveness of teaching

undergraduate endocrinology using a Computer

Assisted Learning (CAL) programme.

DESIGN AND SUBJECTS One hundred and eighty-

five first year clinical medical students were randomly

assigned either to attend a series of conventional

lectures (n � 77) or to have the same material

available through a CAL programme.

MEASUREMENTS A multiple choice question exam-

ination was performed before and after the course.

Lecture attendance and individual usage of the

computer system were recorded. Students were asked

to fill in an evaluation form at the end of the study.

RESULTS There was no significant difference in the

first examination scores between the groups. Both

groups improved their scores after the course.

Students spent longer performing CAL than attend-

ing lectures. Those who scored lowest in the first

examination spent the most time on the CAL course.

Those who spent the most time on the CAL course

showed the largest improvement in examination

score. Thirty-six out of the 42 students, who com-

pleted an evaluation of the CAL programme, rated it

better than the standard lectures.

CONCLUSIONS Computer assisted learning is an

effective way of increasing knowledge in teaching

undergraduate endocrinology. The course was easy to

run and was valued more highly than conventional

lectures. The module is now running routinely in the

year 3 clinical firms at St Thomas' and has resulted in an

increase in knowledge in the end of firm assessment.

Changes in the environment in which clinical medicine is taught

have profoundly altered the experience of medical students. A

shift to out-patient care, a reduction in in-patient stay and a

reduction in hospital beds have changed the `teaching' hospital

to a degree that novel approaches to medical education are

required. In July 1990, the General Medical Council issued a

statement that `the undergraduate curriculum should seek to

promote a more self-educative approach'. With the increasing

emphasis on independent learning and early patient contact, the

time in the undergraduate medical curriculum for formal

teaching of disease morphology is decreasing. Funding con-

straints, demands for improved efficiency and low staff to

student ratio all combine to produce a situation where alternative

forms of education are needed.

Computers have expanded human potential in virtually

every possible arena. Multimedia technology has reached a

level of sophistication that allows teachers to design and create

the majority of the programme themselves and to allow

students to adapt the programmes to their individual learning

needs.

Although computers have been available in medical

education since the 1960s, their use has been less widespread

than the promise of the medium should warrant. Computer

assisted instruction enhances learning, allowing the student to

decide content, time, place and pace of instruction. Information

conveyed can take several forms and the computer could

transform the way competence is assessed. Its greatest

promise, however, may lie in providing pertinent information

at the time when, and in the place where, the student needs it.

The aim of the study was to assess the effectiveness of

teaching undergraduate endocrinology using a Computer

Assisted Learning (CAL) programme in medical students in

the first year of their clinical course.

Methods

Subjects

The entire first year of clinical medical students (n � 205) of

the United Medical and Dental Schools of Guy's, King's & St

q 2001 Blackwell Science Ltd 537

Correspondence: Richard I. G. Holt, South Academic Block Level D

(MP811), Southampton General Hospital, Tremona Road, South-

ampton SO16 6YD, UK. Fax: 144 023 8079 4154; E-mail:

[email protected]

Thomas' School of Medicine was invited to participate in the

study. One hundred and eighty-five agreed to take part and

were randomly assigned either to attend a series of conven-

tional `core' endocrinology lectures (n � 108) or to receive the

same lecture material through a CAL programme (n � 77).

The lecture course

The study incorporated six 60-minute lectures in clinical

endocrinology, based on the department of medicine's `core'

endocrinology lecture course, which had been taught in

previous years. The six lectures covered the aetiology, clinical

features and treatment of diabetes (two lectures), calcium

metabolism, the adrenal and gonads, the thyroid and the

pituitary gland. The visual aids used in the lectures were stored

on a computer in digital format and the lecturers were provided

with a portable computer and display unit for each lecture.

Each lecture was given once during the study. Although the

students were encouraged to attend the lectures, this was not

compulsory. The students were allowed to make their own

decisions regarding their requirements, having been provided

with an initial background to the course and timetable.

An observer recorded the level of attendance and any

interaction between the students and lecturer. Students who

were randomized to the CAL course were excluded from the

lectures by checking the names of students against a register.

The CAL course

The CAL lectures used exactly the same visual material as the

conventional lectures, and included an edited recording of the

lecturer's voice. The course was designed to maintain a simple,

consistent and intuitive look so that the students would not

need to spend time learning how to use the programme.

Multimedia techniques, such as simple animations (e.g.

pointing or highlighting), were synchronized with the

lecturer's voice when explaining complex graphs or tables.

However, they were only used if they did not create a

distraction from the structure of the lecture and did not require

extra time compared with the core lecture course.

The course was designed using `The Multimedia Toolbook'

software (Asymetrix; http://www.asymetrix.com). This soft-

ware offers a range of built in templates and automatic

functions and preprepared objects. It also allows the use of a

low level programming language (Open Script). Furthermore,

`Toolbook' allows unlimited distribution of the Microsoft

Windows based applications, to be used on personal computers

or networks. With certain limitations, the applications may be

exported to the World Wide Web.

The equipment specifically needed to set up the CAL course

included slide and document scanners, sound recording and a

good quality lap-top computer. The total cost of the equipment

including the software was approximately £3000. The time

taken to record and edit the lecture was 8±10 h. The skills

needed to set up the course were all available within the

department of medicine and no external help was required.

The CAL lectures were available to the students in the CAL

laboratory between 09.00 h and 17.00 h on weekdays,

throughout the study period. The students could control the

pace of the lecture, interact with the built-in explanatory or

testing features and repeat the lectures if desired. The

conventional course students were prevented from using the

CAL course by two means. Access to the CAL laboratory was

restricted by identification checks by the academic staff

working in the CAL laboratory and each student allocated to

the CAL course was given a personal password.

The time spent using the CAL course and the pattern of

usage of the course by the students was monitored by an

automatic log-in system. It was appreciated that students may

sit idly by the computer rather than studying the material. In

order to assess this, the total time logged-on was divided into

èffective' time and ìdle' time. After logging-on, the system

would record every action of the user (e.g. clicking the mouse)

and the effect of this action (e.g. navigation between the

screens). The system would also record when any of the

voiceover clips ended and stopped playing. The lecturers'

comments were divided into voiceover clips, which did not

exceed 1 minute in duration. There was no automatic

progression and so the students had to click on the navigation

buttons to progress through the lectures. These records were

saved on the system server regularly, so that even in the event

of sudden termination of the session (e.g. switching off the

computer rather then logging off), the information about the

usage patterns was preserved.

Ìdle time' was defined as the time between the end of any

recorded event and the next recorded action. Some ìdle time'

was considered legitimate to allow the students to reflect on

the course material before proceeding. However, during the

analysis of the log-in records, most ìdle time' lasted only a

few seconds in duration. In all cases, when the idle time

exceeded 2 minutes, it went on for more than 7 minutes. As

the system screen saver went up after 5 minutes, this ìdle

time' must have been caused by abandoning the lecture. In

order to disable the screen saver, user action was required,

which would have terminated the idle time.

Multiple choice question (MCQ) examination

Prior to the course, the students were briefed about the

examinations. They were told that the course would be

preceded by a validated MCQ examination comprising 34

stems, each with five true or false questions. They were also

538 R. I. G. Holt et al.

q 2001 Blackwell Science Ltd, Clinical Endocrinology, 55, 537±542

told that a further examination would be conducted after the

course but were not told that the second MCQ examination

would be identical to the first examination. Although the MCQ

examination was not based solely on the material covered by

the lectures, the answers to 90% of the questions were

obtainable directly from the course. The students were not

given the results of either examination until the end of the

study. As the CAL course was not part of the core curriculum,

it was decided that the results of the examinations should not

count towards the students' assessments and for the same

reason the second examination was not made compulsory. The

second examination was sat immediately after the completion

of the lecture course. The students from both groups sat the

examinations together.

Evaluation of the study module

Each student was asked to evaluate the course by a

semistructured questionnaire immediately after the second

examination. CAL students were asked to compare the course

with their previous experience of being taught in a conven-

tional way. They were asked to rate different features of the

CAL course in terms of how useful they found them and how

well they were implemented in the programme.

Statistical analysis

The results are expressed as mean with 95% confidence

intervals or mean ^ standard error of the mean. The two

groups were compared by the unpaired t-test. The scores

before and after the course were assessed by the paired T-test.

Results

Participation in the conventional lecture course

The attendance rate for the lectures ranged between 20 and 44

students (18±41%). The reason for nonattendance was

frequently a clash with other compulsory or important

activities but was undoubtedly influenced by the usual

08.30 h time slot. Students were frequently late and missed

the beginning of the lecture. On one occasion, the lecturer was

late because of a medical emergency, which required his

urgent attention. The lecture subsequently finished late to

ensure that all of the material was provided. Two lecturers ran

over time because they were unable to complete the lecture in

1 h.

There was little interaction between the students and

lecturers. Only three students asked a total of seven questions,

although the lecturers used questions as a means of integrating

the students. All students made notes during the lecture. Only

46 (43%) of the conventional lecture students performed the

final examination.

Participation in the CAL course

In contrast to the conventional lecture course, a significantly

higher proportion of students assigned to CAL completed the

course and performed the second examination (62 out of 77

students, P , 0´0001 vs. conventional course, chi-squared

test). Several quite different patterns of usage developed

among the students although each student had a consistent

pattern of working. This ranged from some students who

would always spend enough time to complete the lecture to

others who would never spend more than 15 minutes at a

single session. The students were able to pick up a lecture

quickly where they had left it. The median number of sessions

per lecture was two. The CAL students spent an average

effective time of 83 ^ 6´5 minutes per lecture (range: 8´7±

291) (Fig. 1).

MCQ examination

Seventy-seven CAL students and 108 conventional lecture

students sat the initial examination. There was no statistical

difference between the scores of the CAL students [37´9 marks

(34´0±41´8)] compared with the scores of those randomized to

the lecture course [37´8 marks (34´9±40´7)]. The students who

dropped out of the conventional lecture course scored

significantly lower [33´8 (30´1±37´5)] than those who

completed the course [43´9 (39´2±48´6), P , 0´05] in the

initial MCQ examination (Table 1). In contrast, there was no

difference in the initial score between those who dropped out

and those who completed the CAL course.

Both groups of students scored significantly higher in their

second MCQ examination [lecture: 69´8 marks (64´9±74´7),

CAL: 64´8 (60´3±69´3)]. The mean difference in scores was

26´3 marks (25´5±27´1, P , 0´0001 vs. initial score) for the

CAL course and 25´9 marks (25´4±26´4, P , 0´0001 vs. initial

score) for the conventional course. There was no statistical

difference in the second examination between the lecture

course and the CAL course (P � 0´32).

Those who scored lowest in the first MCQ examination

spent the most time on the CAL course (Fig. 2). Furthermore,

those who spent the most time on the CAL course showed the

largest improvement in the second MCQ examination (Fig. 3).

Evaluation of the course

56% of the CAL students and 20% of the lecture students

returned their questionnaire.

The main complaint about the conventional course was the

Computer assisted learning in endocrinology 539


inflexibility of the lecture times, particularly when these were

early in the morning. Other individual complaints included the

difficulty in making notes, speed of delivery and the quantity

of information contained in the lecture.

The students found the CAL application efficient and easy

to run. Thirty-six of the 42 respondents rated the CAL course

as better than conventional lectures. The most beneficial aspect

of the CAL course was its flexibility. The course could have

been improved further if they had been able to use the software

out of the department. The students would have preferred to

attend both the conventional lectures and the CAL course,

although half of the students (n � 23) stated that in reality they

would not attend the conventional course if both were

available.

Discussion

The current study demonstrates that the use of this computer

assisted learning module is effective and easy to use. It is at

least as effective as conventional lectures in preparing students

for a knowledge based assessment and is preferred by students

because of its flexibility. It is particularly advantageous for

below average students who might otherwise drop out of a

conventional lecture course.

This study extends the widening body of evidence, which

attests to the efficacy of CAL (Dewhurst et al., 1994; Raidl

et al., 1995; Devitt et al., 1998; Devitt & Palmer, 1999).

During both CAL and conventional lectures, the students

gained knowledge about the subjects covered by the course.

The use of CAL in medical education is expanding. In 1993,

36% of departments in Australian medical schools were using

CAL but perhaps more significantly 65% of those not currently

using CAL were planning to develop CAL programmes (South

& Nolan, 1993). However, there have been criticisms of CAL.

The computer literacy of Edinburgh medical students was

questioned as recently as 1994, where 31% of medical students

had not used a computer in the previous year (Osman & Muir,

1994). The style of CAL material is also important. When

preclinical students in Adelaide were presented with material,

which was either problem based or with a free text response,

the students fared no better than control students (Devitt &

Palmer, 1999). In contrast, the students performed better if the

material was presented in a didactic manner, as in our study.

The main difference between our CAL course and the lectures

was the ability of the students to repeat sections at will.

The high drop out rate in the conventional lecture course

was worrying, particularly as those who dropped out began

with worse marks than those who completed the course. We

believe that this reflected poor motivation by the students to

attend the lectures coupled with a fear of failure in the second

examination, despite reassurances that this would not be used

in the end of firm assessment. It appears that conventional

lectures are less likely to help students with a lower knowledge

base in contrast to CAL where similar students were motivated

to spend more time and improved their knowledge accord-

ingly. It is encouraging for a self-directed learning approach

that more time was spent on an activity without a designated

slot in the timetable.

Attendance at lectures is often a problem for both the

students and lecturers and so one of the main advantages of the

CAL package is its greater flexibility. The clinical medical

Fig. 1 Frequency distribution of the average time spent per lecture

by the students allocated to the CAL course. Eighteen students spent

no time working on the computer although three of these completed

the second examination.

Group Number Initial MCQ (95% CI) Final MCQ (95% CI)

CAL completers 62 38´5 (34´0±43´0) 64´8 (60´3±69´3)²

CAL noncompleters 15 32´2 (25´5±38´9)

Lecture completers 46 43´9 (39´2±48´6)* 69´8 (64´9±74´7)²

Lecture noncompleters 62 33´8 (30´1±37´5)

*P , 0´05 vs. lecture noncompleters. ²P , 0´05 vs. initial MCQ examination. CI, confidence

interval.

Table 1 Scores of the MCQ examinations

according to group and according to whether the

students completed the study



course is full with different attachments and small group

activities on different sites such that lectures frequently clash

with other activities. This problem becomes particularly

difficult during attachments to peripheral hospitals. Conse-

quently, the attendance by the students was poor. Although the

lecture course could have been made more acceptable by

rescheduling lectures to a different time, in reality there were

clashes in the timetable at all times and the clinical

commitments of the lecturers prevented this from being a

realistic option. It is possible that the reason for the low

attendance was that nonattendees chose to avail themselves of

the same material on the computer. This is unlikely because of

the strenuous efforts to prevent access to the CAL material and

because the attendance rates were not different from before or

after the study.

The explanation for those who scored lowest in the pretest

MCQ spending more time on the CAL is not obvious as the

students were not told the result of the first examination until

the end of the study. However, sitting the first examination

gave the students an idea of the level of understanding that was

expected by the end of the study. Furthermore, the extra time

spent may reflect the need to work slower to understand and

assimilate the knowledge provided by the CAL course.

The students found the course acceptable, efficient and easy

to run. It is possible that the low response rate (56% of the

CAL students and 20% of the lecture students) biased the

results, in that the students who completed their evaluation

forms were those who were interested in CAL and had used it

heavily. However, the positive evaluation seen in this study

concurs with previous studies (Khadra et al., 1995; Paterson &

Kaufman, 1995). In Sydney, 126 students reported a high level

of acceptance for the use of CAL (Khadra et al., 1995). In

another study, 95% of medical students felt that computers

were important to medical education (Paterson & Kaufman,

1995). Although the course material might be obtained from

books, CAL is also rated more highly than textbooks

(Sneiderman et al., 1994). Despite the positive evaluation,

the students felt that the CAL course should be an adjunct to

conventional lectures, possibly reflecting the conservative

nature of students.

The course was set up as part of a larger on-going project to

create an Interactive Learning Environment for medical

students. This project has involved the creation an interface

for the `computerization' of the lectures, which will allow

basic interactivity and browsing-searching features. Once this

project is completed, it is envisaged that the lectures will be

integrated with other resources, such as the medical school

library and databases of clinical case histories and pictures. We

designed the system to be compatible with the medical school

computer network structure. However, at the most basic level,

the endocrinology course could have been made available to

the students as a PowerPoint presentation on CDs for use on

any personal computer.

Deployment on the World Wide Web provides the

possibility of open Internet access for the lectures. Most

multimedia development software, including `The Multimedia

Toolbook', has an option of exporting the material content into

web friendly format (Dynamic HTML, Java applets, Java-

Script, etc.). As some of these features depend on the Web

server configuration, support may be needed from the local

`webmaster'. The use of the World Wide Web may be limited

by the necessity to maintain confidentiality of the patient

material used in the lectures. Furthermore, the quality of

Internet multimedia files depends on the quality of the user's

Internet connection. Although when the study was performed,

there were no commercially available alternatives, today there

are `case history' orientated programmes on the market. As

such, it may not be efficient or appropriate for every teaching

centre to develop its own CAL course.

The development of CAL courses for medical education is

often considered an expensive and time-consuming task.

Creating a virtual reality, in which a student can experiment

without a predefined path, requires a huge collaboration

between medical experts and application developers. In

contrast, discussions with our students prior to the study

suggested that they were looking for a CAL resource, which

was based on a didactic and guided approach. Our decision to

convert the `core' endocrinology lectures into a CAL course

was a response to this need. By using material already prepared

for the `core' lecture programme, the development time of the

CAL course was reduced significantly. After a modest initial

investment towards the cost of the equipment and development

of the methodology, we managed to deliver a quality CAL

Fig. 2 The amount of time spent performing the CAL course was

determined by the initial score in the pre-CAL examination.

Computer assisted learning in endocrinology 541


course in a short time and in a cost effective way. As the

relevant software has significantly improved since the course

was produced, these costs could be further reduced.

A further advantage of our course was the facility for the

lecturers to customize the course to local students. It provides a

flexible means of learning the core basics and frees staff±

student contact time. As the lecturers were available for

questions about the course content, the staff contact sessions

could take a more interactive format as directed by student

needs. The CAL course should be seen as complementary to

bedside and other contact teaching, which may be more suited

to clinical skills and attitudes.

In conclusion, we have shown that computer assisted

learning is an effective way of increasing knowledge in

teaching undergraduate endocrinology. The students found the

course easy to run and valued the course more highly than

conventional lectures. The module is now running routinely in

the year 3 clinical firms and is both well-liked by the students

and has consistently resulted in a marked increase in knowl-

edge in the end of firm assessment.

References

Devitt, P., Cehic, D. & Palmer, E. (1998) Computers in medical

education 2. Use of a computer package to supplement the clinical

experience in a surgical clerkship: an objective evaluation.

Australian and New Zealand Journal of Surgery, 68, 428±431.

Devitt, P. & Palmer, E. (1999) Computer-aided learning: an overvalued

educational resource? Medical Education, 33, 136±139.

Dewhurst, D.G., Hardcastle, J., Hardcastle, P.T. & Stuart, E. (1994)

Comparison of a computer simulation program and a traditional

laboratory practical class for teaching the principles of intestinal

absorption. American Journal of Physiology, 267, S95±S104.

Khadra, M.H., Guinea, A.I. & Hill, D.A. (1995) The acceptance of

computer assisted learning by medical students. Australian and New

Zealand Journal of Surgery, 65, 610±612.

Osman, L.M. & Muir, A.L. (1994) Computer skills and attitudes to

computer-aided learning among medical students. Medical Educa-

tion, 28, 381±385.

Paterson, G.I. & Kaufman, D.M. (1995) Medical informatics and

problem-based learning in conjunction. Medinfo, 8, 1149±1153.

Raidl, M.A., Wood, O.B., Lehman, J.D. & Evers, W.D. (1995)

Computer-assisted instruction improves clinical reasoning skills of

dietetics students. Journal of the American Dietetic Association, 95,

868±873.

Sneiderman, C.A., Hood, A.F. & Patterson, J.W. (1994) Evaluation of

an interactive computer video tutorial on malignant melanoma.

Journal of Biocommunication, 21, 2±5.

South, M. & Nolan, T. (1993) Computer-assisted instruction in Australian

medical schools. Medical Journal of Australia, 159, 175±176.

Fig. 3 The difference in score between the pre-CAL and post-CAL

examination according to the amount of time spent performing the

CAL course.



Documents

Computer assisted learning is an effective way of teaching endocrinology