Applications of educational technology at the open university

Applications of Educational

Technology at The Open University 1

DAVID G. HAWKRIDGE

INTRODUCTION My purpose here is to introduce you to the Open Univer- sity, which is a unique enterprise in higher education, and one that is new, changing, and controversial. In particular, I will say something about the work of the Institute of Educa- tional Technology (IET) within the University. First, let me sketch in some background. I myself am an educational psychologist. I say that as comfort to those few who might still suppose that educational technology has only to do with projectors and flannelgraphs. Others may wonder what an educational psychologist has to do with educational technology. Let me skirt round a formal definition, yet provide a few clues about educational technology before saying what are its applications at the Open University.

Educational technology certainly cannot be thought of as an organized branch of knowledge. There is at present no unified theory, but rather an eclectic grouping of disciplines, as is reflected in the backgrounds of the staff of the Institute. (We have psychologists, educationists, sociologists,

This article is adapted from an address delivered at the Annual Con- ference of the British Psychological Society, Education Section, at York, England, September 1971.

David G. Hawkridge is director of the Institute of Educational Technol- ogy, The Open University, Bletchley, England.

AVCR VOL. 20, NO. 1, SPRING 1972 ,5

AV COMMUNICATION REVIEW 1972 SPRING : 6

statisticians, and so on, all working together.) Technology is a better descriptor than science. We are not putting for- ward a science, but instead make use of sciences. Technol- ogy, in addition to machinery or equipment, includes pro- cesses, systems, management and control mechanisms both human and nonhuman. Further, technology espouses a problem-solving, systems approach, and has strong links with communications theory, under which are subsumed such studies as control analysis, content analysis, audience analysis, media analysis, and effect analysis. All of these areas are ones in which educational technologists frequent- ly take a keen interest (Ely, 1970).

Galbraith (1967) has suggested that technology means the systematic application of scientific or other organized knowledge to practical tasks. Indeed, members of the Insti- tute are attempting to apply what is known from a variety of sciences or disciplines to the immense practical tasks of educational development in the University. Lumsdaine (1964) thinks much the same way as Galbraith, suggesting that educational technologists are appliers of science in the sense that medical practitioners are appliers of the underly- ing biological sciences.

But what educational technology at the Open University is about becomes far clearer from example than from pre- cept. Some of you will be familiar with Maier's Law: "If facts do not support the theory, they must be disposed of" (Maier, 1960). In this case, if the facts do not support the definition, we should keep the facts. On the other hand, we claim to be professionals, not craftsmen, engaged, in White- head's words, on "customary activities modified by the trial and error of individual practice" (Finn, 1953).

To provide a framework for my description, I intend to use some categories derived, I believe, by Goodlad at the University of California in Los Angeles. First, I shall consider the University's analysis of what is involved in subject-matter competence. Such analysis should precede curriculum building and course development. Second, I shall examine the problems in diagnosing the Open University students' abilities. Third, I shall consider how the Univer- sity can create conditions for its students to acquire in- creased subject-matter competence-in other words, how

EDUCATIONAL TECHNOLOGY AT THE OPEN UNIVERSITY : 7

ANALYZING WHAT IS

INVOLVED IN SUBJECT- MATTER

COMPETENCE

can it improve its instruction. Fourth, I shall outline how the University is attempting to measure achievement, and how the measures may be improved. In all four of these sec- tors, Institute personnel are deeply engaged. The Open University has such a complex instructional system that it is quite impossible to give you an adequate picture of it in the space at our disposal. The system includes correspondence tuition [instruction via correspondence courses[, television and radio broadcasts, nearly three hundred local study centers, counselling services, and summer schools in ordinary universities, such as York.

The correspondence material is rather carefully designed. Most of the units incorporate devices intended to assist students learning at a distance, who, for example, are de- prived by their isolation of opportunities to learn quickly what Snyder (1971) at MIT has called "selective negligence": the skill of identifying and neglecting what is unimportant. Students in continuous face-to-face tuition situations pick up many clues, it seems, from their professors and class- mates about the hidden curriculum-that which the Univer- sity really expects from its students. The devices in the units include lists of objectives, conceptual diagrams, and self- assessment questions which, in programed learning style, have model answers given elsewhere in the units. Some units also incorporate frequent questions to which no an- swer is given; these serve as stimulus points, breaking up the text and challenging the student to comprehend what has gone before or what is about to be taught.

Before any of these devices, as I have called them, are built into the units there should be some analysis of what is involved in subject-matter competence. This analysis turns out to be extremely difficult. A member of the Institute staff working with, say, the professor of geography or of physics, does not find it easy to obtain from that professor an exact statement of objectives for the learner. As those of you who have tried to write behavioral objectives know well, there are difficulties in describing accurately the higher levels of behavior required in a university course. Tyler (1964) has written about these persistent problems. After nearly two years' work on objectives for university courses, there is a view in the Institute that the classical behavioral

A V C O M M U N I C A T I O N REVIEW 1972 S P R I N G : 8

objective is inadequate, although it is better than no objective at all in most circumstances. We cannot cope by using only the ideas of Mager (1962), whose suggestions stemmed from industrial training situations.

The work of Ausubel (1963) and others on advance orga- nizers encourages us to continue working with unit authors to develop objectives, but the Institute houses a study group, headed by Brian Lewis and Gordon Pask, devoted to going beyond behavioral objectives. The group has taken over work begun by Michael Nell, who produced the first conceptual diagrams, used by the Faculty of Science. These diagrams, and subsequent ones in the "Foundation Course in Science" now published, were relatively simple maps showing the main concepts to be learnt and something of the relationships between them. Michael Nell found, however, that these relationships were very complex, and that any code sufficiently specific to begin to describe their complexity soon introduced so much detail into the diagram that the exercise became self-defeating.

Indeed, it is this complexity of the structure of knowledge in higher education instructional materials that is the chief stumbling-block both for the author intent on clear exposition and for the student trying to grasp the essentials. Con- tent analysis of the units after the event can be carried out on an elementary level easily enough-words can be count- ed, reading indices computed, paragraphs summarized, analyses made of the use of qualifiers, such as adjectives, and so on. But as soon as something deeper is attempted, the re- sulting network quickly becomes awe-inspiring. Recently the lET group examined what is commonly taught about probability in elementary statistics courses, and found not the neat hierarchy of concepts that appears in the contents table of most texts but rather a number of nodes of concepts. The group also found that the subject-matter could be entered from a number of directions and traversed by many routes. There was no route that obviously recommended itself above all others. It seemed that some routes were more suited to particular learning styles, however, such as holis- tic or serialist, inductive or deductive.

Such analyses of what is required for subject-matter competence would throw up some grim surprises, we suspect,


DIAGNOSING OPEN

UNIVERSITY STUDENTS'

ABILITIES

if applied to many texts written by distinguished academics. It is all very well to say that serendipitous rides on a few of the author's hobby-horses will do no harm. Perhaps not, but if these horses take the student far away from the subject-matter in which he wishes to acquire competence and throw him in a foreign territory, the author may get no thanks. Worse still, content analysis can reveal omissions which the author's superior knowledge allowed him to make unwittingly. The student, on the other hand, struggles to bridge a chasm of unknown width. Secondly, we know far too little about Open University students' abilities. The characteristics of conventional university students are fairly well-established, it is said. With- out arguing that point of view, we can accept that at present those of Open University students certainly are not. The University's admission policies guarantee an extremely heterogeneous student body. Other institutions of higher education or adult education may give us some basis for comparison, but the Open University is different from all of them.

What are the measures available to us? In fact, there are only two kinds that may be of any use when courses are first being prepared: educational background and occupa- tion. Statistics were compiled in the University for these two, and it was the task of Institute staff to bring these to the attention of course teams. There is no average student, but something can be said to a course team about the kinds of people who have applied for the course and their prob- able range of abilities.

I should mention also at this point that the Institute houses a group that has been studying for the past two years students registered for the Preparatory Courses prepared by the National Extension College at Cambridge in collabora- tion with the BBC. These studies have been just in time for some at least of the NEC/BBC students' experiences and abilities to be assessed and the results made known to Open University course teams, who can expect similar students in their courses. Naomi Mclntosh and Tony Bates have worked on this project, and expect to produce quite a full

report later this year. Next, when courses are being produced they can be tried

AV COMMUNICATION REVIEW 1972 SPRING : 10

CREATING CONDITIONS

FOR IMPROVED INSTRUCTION

out, or developmentally tested, on groups of applicants. This testing is also the responsibility of members of the Institute staff. We cannot pretend that it is an exact and scientific process. What we can say is that many units have been changed, we hope for the better, as a result of such testing. Authors do get some impression of their students' abilities.

Another group within the Institute, led by Michael Mac- Donald-Ross, is studying ways of improving developmental testing. If the "'guinea-pigs" are regarded as error-detectors or commentators, there may be some who are better at detecting errors or commenting than others. Selection of good error-detectors and commentators will mean more efficient testing. Instead of using a randomly selected group of 30, we may instead rapidly select the best 10. (The guinea- pigs are paid for their help.) Another way of improving developmental testing may lie in the advice on studying the units of work given to the guinea-pigs. We know that error- detection flags, and that if guinea-pigs study the material in segments, the rate of detection increases.

In summary, we still know far too little about our students' abilities, but then we have not yet finished our first year of teaching. Most of you would agree that one essential for improving instruction is a good system of assessing student achievement: if students perform poorly in one section of the course, probably, although not certainly, the instruction needs to be improved. But a good system of assessment may have as its primary goal the grading of students, rather than the identification of faulty areas of instruction. We shall come back to that under the next section.

If we exclude student performance as a criterion, there are still other criteria that may be applied to judge the ex- cellence of a course. For example, the vast majority of Open University course units are sent to external assessors, academics in other universities, for comment. This is done before the written units are printed. The usual network of external examiners exists to help control standards in the examinations too.

Institute staff are primarily interested in the impact of the courses on the students, however, and an outcome of the Preparatory Course study has been the setting up of a proj-

EDUCATIONAL TECHNOLOGY AT THE OPEN UNIVERSITY : 1 1

ect to examine the background of Open University students. A Social Sciences Research Council grant to Naomi McIn- tosh has allowed work to get under way, and, in particular, money has been available for the collection of data from the University's first cohort of over 24,000 students, An ini- tial questionnaire was sent to them to cover in some detail their educational and occupational backgrounds, their work and leisure patterns, and their future plans. Data from a representative sample drawn from the first cohort will be analyzed this year to provide a basis for some of the Uni- versity's major decisions about future courses. The remain- der of the data will be banked to form the basis for a longi- tudinal study of this group of students.

Other information about the impact of Open University courses is being collected by Institute staff through what are known as course unit report forms. These are questionnaires designed for a fast document reader. Each student in the sample of about 8,000 receives a pad and is asked to send in one form for each unit he studies. The questions refer to attitudes about the unit, including the television and radio components, and to details of study habits. For in- stance, students are asked to report roughly how long they have spent on the unit. The answers do not offer a detailed picture of exactly what went wrong, but trends can often be traced. Some of the data have to be interpreted with caution: too many students may find a mind-stretching piece of exposition "boring" or "too difficult," but the instant re- action should not necessarily be to lighten the unit next year.

What room is there for improving the instruction when that instruction is recorded somewhat permanently in cold print or film or tape? The University does see itself as a self-improving system and has made deliberate financial provision for renewing courses. Some staff, including myself, do not think enough money has been set aside, but the business heads in committees point out that the needs must be balanced. In fact, some of the television and radio broadcasts can be remade each year, and about a third of the printed matter must be revised yearly. The life of a course is four years, after which the whole of it may be rewritten. One of the tasks of members of the Institute is to assist in

AV C O M M U N I C A T I O N REVIEW 1972 SPRING : 1 2

M E A S U R I N G

A C H I E V E M E N T

the accumulation of sound data to provide a basis for judg- ing where revision is necessary and how it should be done.

Besides the actual course material, the University also provides a valuable support, or fail-safe, system for all of its students who are able to reach the local study centers. The tuition and counselling system, as it is known, provides face-to-face class tuition once or twice a month, and the services of student counsellors. Inevitably, many of the students who use the system are the ones who need help. The kinds of help that they receive and the ways in which the system can be improved are the subjects of yet another In- stitute study, which I myself direct. Data about the 3,500 part-time staff have been analyzed, and studies made too of the summer schools, which may be seen as part of the tuition and counselling system.

The system also provides an informal, and rather unstruc- tured channel through which suggestions for improving the instruction can flow. These suggestions may come from the students themselves or from the part-time staff, and they are collected by the regionally-based full-time staff, in particular the Staff Tutors and Senior Counsellors. The Staff Tutors are members of the Faculties and in an excellent position to report back to the teams who designed the courses in the first place. These teams are made up of subject-matter specialists, BBC producers and Institute staff. The last area in which Institute staff are heavily engaged is that of measuring achievement of students. This area is one in which the traditional university image has never been a strong one, and the majority of the Faculty members at the Open University are of course from traditional universities. Institute staff assume that there should be some direct relationships between the objectives of units, the activities of students while they study these units, and the items used in measuring student achievement. To propose that a student should be told that the objective is for him to be able to understand philosophy is not good enough. Nor can the activities then comprise of reading a highly abstruse article orig- inally written for other professors of philosophy, nor should the test consist of a question such as "'Write an essay to show that you understand philosophy." If one bears in mind the constraints of teaching and testing at a distance,


then it becomes clear that authors' intentions must be made more obvious, and that the tests must be comprehensive and carefully controlled.

The University employs three chief modes of assessing its students. Each is also used to some extent, as I mentioned earlier, to judge the effectiveness of the teaching, particu- larly of the written components. The first mode is the traditional university examination: three hours long, covering the year's work, and consisting in the main of essay-type questions. The Science Faculty is the only one to have de- parted from this mode this first year, by producing a final examination composed entirely of multiple-choice items, suitable for computer marking.

The second mode is through tutor-marked assignments, part of the continuous assessment. These assignments vary in number but come about monthly. They are marked and commented upon by part-time correspondence tutors, using schedules that vary greatly in detail, depending on the topic and faculty.

The third mode is through computer-marked assignments, also part of the continuous assessment. Some faculties make more use of these than others, since the questions can only be multiple-choice.

Institute staff working with the course teams played a con- siderable part in item-writing for these three modes of assessment, especially for the computer-marked system, for which Michael Neil and Tony Kaye worked with the Uni- versity's data processing division to devise 21 scoring op- tions.

There was no time to test the tests, hence very little indeed was known about the characteristics of the items. On the basis of our 1971 experience we hope to be able to start up an item bank. For each course, items will be stockpiled once they have been used. With each item will be a full analysis of its characteristics during its first year of use, including its difficulty and discrimination indices. Test relia- bility will be computed, and so on. Such analysis requires much development work on the University's computing system. If funds can be found, we hope to be able to reorga- nize the data base so that all analyses required can be carried out without delay or difficulty. But that is a topic for

AV COMMUNICATION REVIEW 1972 SPRING i 14

C O N C L U S I O N

another, separate paper. Suffice to say that we are deter- mined to improve a relatively primitive examinations and assessment system. From what I have said, I think it will be apparent that the work of the Institute fails roughly into two main areas: course development and institutional research of various kinds. There is absolutely no doubt that the Open Univer- sity offers a Vast potential for work in both these areas. So many topics wait to be tackled: cost-effectiveness studies, computer-assisted instruction, course production schedul- ing and management, the design of admissions procedures for about 40,000 students annually, evaluative studies of the use of media such as television and radio, and of the impact of Open University-type institutions upon higher education, and investigations of special groups of students, like our remote ones in Scotland or those in prisons. These and many others have been talked about and in some cases may start quite shortly.

The Institute has had a favored start. It has been set up in a university where educational technology and institutional research are seen to be needed. Its members have been able to regard themselves as equal partners in the University's endeavors, rather than as staff of a service or advisory group occasionally called upon by a handful of academics. The University's courses bear a good many marks of educational technology, not in the sense of being disseminated through certain kinds of hardware but in the sense of being deliberately designed for the learner-at-a-distance. Nobody at the University, and least of all the members of the Insti- tute, will claim that the courses are more than first approxi- mations, first steps towards the ideal. On the other hand, there do seem to be good chances of being able to improve the courses.

So far, the Institute members have been proud of the fact that they work with others within the Open University far more than engaging in proselyting activities outside or talk- ing with other groups of educational technologists. In time this will change: as expertise is built up in the Institute there will be something exportable to other institutions both in Britain and abroad. There is some evidence of increasing interest in the applications of eclucational technology in


higher education. In due time, perhaps two or three years

hence, the Institute may produce its own courses about educational technology. Then it will really have to practice all that it has preached!

REFERENCES Ausubel, D. P. The psychology of meaningful verbal learning. New York: Grune and Stratton, 1963.

Ely, D. P. Toward a philosophy of instructional technology. Jour- nal of Educational Technology, 1970, 1 (2), 81-94.

Finn, J. D. Professionalizing the audio-visual field. AV Communi- cation Review, 1953, I (1) 15.

Galbraith, J. K. The new industrial state. Boston: Houghton, Mif- flin, 1967.

Lumsdaine, A. A. Educational technology, programmed learning and instructional science. In E. R. Hilgard (Ed.), Theories of learning and instruction. Part I. 63rd yearbook of the Nation- al Society for the Study of Education. Chicago: NSSE, 1964.

Mager, R. F. Preparing instructional objectives. Palo Alto;' Calif.: Fearon, 1962.

Maier, N. R. 1:. Maier's law. American Psychologist, 1960, 15, 208- 212.

Snyder, B. R. The hidden curriculum. New Yorkl Knopf, 1971. Tyler, R. W. Some persistent questions on the defining of objec-

tives, 1971. Mimeographed paper.

Documents

Applications of educational technology at the open university