Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
DOCUMENT RESUME
ED 047 497 EM 008 697
AUTHOR Meredith, Joseph C.; Ferguson, DouglasTITLE Student Feedback as a Tool in Computer-Assisted
Instruction (CAI) Frame Development.INSTITUTION Indiana Univ., Bloomington. Research Center for
Library and Information Science.; Stanford Univ.,Calif. Libraries.
PUB DATE 70NOTE 83p.JOURNAL CIT International Journal of Experimental Research in
Education; v7 n2 p221-302 1970
EDRS PRICE EDRS Price MF-$0.65 HC-$3.29DESCRIPTORS *Computer Assisted Instruction, *Feedback, Programed
Instruction, Programed Units
ABSTRACTIn order to measure the value of student feedback in
contributing to the refinement of instructional frames incomputer-assisted instruction (CAI), a stand-alone CAI frame waswritten around a question designed to elicit a wide variety ofreplies. Half of a real sample of fifty handwritten responses weresubmitted to the frame, with a match score of 96% and a sense scoreof 48%. After revision, the frame attained a match score of 100% anda sense score of 88%, on the other half of the test set. Conclusionsdrawn frcm the study were that (1) for an open question with a largenumber of legal answers and an adult, heterogeneous student set, astriking improvement in meaningful intercept can be obtained throughclose analysis of a modest number of live responses; and (2) as acorollary, the rate of improvement to be derived from close analysisbeyond this mcdest number must decline quite rapidly. In the reportof the study, selected-string-match CAI strategies and the mechanicsof the CAI "macrcframe" are discussed. Appendices contain supportingmaterials. (Authcr/MF)
O
iSCIENTIA PAEDAGOGICA
EXPERIMENTALISINTERNATIONAL JOURNAL OF EXPERIMENTAL RESEARCHIN EDUCATION REVUE INTERNATIONALE DE PEDAGOGIEEXPERIMENTALE INTERNATIONALE ZEITSCHRIFT FOREXPERIMENTALE PADAGOGIK INTERNATIONAAL TI JD-
SCHRIFT VOOR EXPERIMENTELE PEDAGOGIEK
EDITED BY - EMMA PARHERAUSGEGEBEN VON - UITGEGEVEN DOOR
R. VERBIST / Gent
WITH THE CO-OPERATION OF - AVEC LE CONCOURS DE
UNTER MITWIRKUNG VON - MET DE MEDEWERKING VAN
A. CLAUSSE / Liege E. FELDMANN / BonnJ. T. HUNT / Athens, Georgia (U.S.A.)
VII, 2
UITGEGEVEN MET DE STEUN VANHET BELGISCHE MINISTERIE VAN NATIONALE OPVOEDING,
EN VAN DE UNIVERSITAIRE STICHTING VAN BELGIE
GENT, HENRI-DUNANTLAAN, 1197o
U.S. DEPARTMENT OF HEALTH, EDUCATIONIS WELFARE
OFFICE OF EDUCATIONTHIS DOCUMENT HAS BEEN REPRODUCEDEXACTLY AS RECEIVED FROM THE PERSON ORORGANIZATION ORIGINATING IT. POUTS OFVIEW OR OPINIONS STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION POSITION OR POLICY
STUDENT FEEDBACK AS A TOOL INCOMPUTER-ASSISTED INSTRUCTION (CAI)
FRAME DEVELOPMENTby
JOSEPH C. MEREDITH (Research Center for Library and Infor-mation Science, Graduate Library School, Indiana University)and DOUGLAS FERGUSON (Editor, Automation Department, Stan-
ford University Libraries)
BACKGROUND AND OBJECTIVE
This paper represents an attempt to measure the value ofstudent feedback in contributing to the refinement of instructionalframes in Computer-Assisted Instruction (CAI). In our ownwork in CAI for graduate students (specifically for students oflibrary science) (1) we have concentrated on the development ofa « machine tutorial mode » suitable for presentation of con-ceptual material to an adult and very mixed student population.We must keep in mind students whose sole preparation is aliberal arts degree others who hold degrees in the sciencesand still others with abundant experience as practicing librarians.Primary interests range from children's libraries to the upperreaches of information theory. Ages range quite evenly throughthe twenties, thirties, and forties.
In order to be effective on an individual basis, we have. feltthat we must make the machine side of the program as versatileas possible, with a very broad capability for dealing with idio-syncratic responses. Whatever we do will fall Far short of humandiscourse; we cannot pretend to cope with meanings which havenot been anticipated; but we can at least take advantage ofobserved patterns of speech and thought in order to avoid moreformal constraints on dialogue than are absolutely necessary.
(1) Described in the interim report on Project No. 7-1085 (0EG-1-7-071085.4z86) : An Information processing laboratory for education and research inlibrary science : Phase I (M. E. MAaoN, A. 3. FIUMPHREY, and J. C. MEREDITH,DHEW (OE) July, 1969).
222 J. C. MEREDITH & D, FF,R6DSON
What about these patterns ? It is sometimes said that pre-planned conversational CAI quickly reaches an area of diminishedreturn, because of the massive intellectual and clerical effortrequired to prepare even a short sequence (r). This might betrue if the CAI author were to work in a vacuum, drawing onthe single resource of his own semantic store. But what if he wereto demand less of this for-the-most-part intuitive, remembered,and general resource and rely more on live results derived frompreliminary testing ? We know that student feedback is usefulin revising programmed learning, but we need to know moreabout the extent of this usefulness and to decide when and howbest to exploit it.
This, quite simply, was the idea behind what we thought wouldbe a straightforward exercise in recursive improvement of asingle CAI frame. If we had had an earlier indication of howstriking the results would be, we would have structured the testmore rigorously. Even so, we feel that the controls are adequateto justify reasonable confidence. Also, we did take the precautionof drafting and documenting Part I of this paper before proceedingwith the second part of the experiment, which is described inPart II.
PART I
PROCEDURE
Fifty individuals were polled in writing (Appendix A) with arequest that they indicate what single response they might havegiven to a certain statement/question, in an on-line situation ata student terminal. Individuals were chosen at random from amonglibrary school students, faculty, and technical and clerical staff ofthe Institute of Library search, University of California, Ber-keley, California. In using subjects from this environment itwas hoped that we could compensate in part for the lack of
(t) William R. UTTAL et al. represent this view in Generative ComputerAssisted Instruction (Communication #2,43 of the Mental Health ResearchInstitute, University of Michigan, January, 1969).
3
STUDENT FEEDBACK IN CAI 223
instructional preparation leading up to the statement /question.At the same time we realized that a residuum of this lack ofcontext would need to be accommodated within the frame itself,to an uncommon degree. « Success », as we viewed it, would bemeasured as much by the effectiveness of this accommodationas by the efficiency of the match and fail processes of the system.
Respondents were asked to write down their first response tothe question as worded. No explanation or comment was providedat the time. The fifty answers were serially divided into sets oftwenty-five each to make up an « A » list and a « B » list, and theformer was submitted to a previously-written program calledthe « A-Frame ». The results are described and analyzed in thesections which follow.
We then revised the A-Frame in the light of what we hadlearned, and tested the effectiveness of this version (the « B-Frame ») against the « B » list of anwers. Appendices A throughI document the steps which were taken before conducting thissecond test.
THE PROGRAM
In designing the A-Frame we assumed programming languagecapabilities corresponding with PILOT (1), the CAI language inuse in the Institute's Information Processing Laboratory, withminor technical augmentations provided in DISCUS (2). Bothlanguages employ selected character string match as the basicanswer-matching routine. That is, student input of any lengthwill be successfully matched in a scanning operation if the elementsspecified in the scan are present. The remainder of the input isignored.
(i) See R. ICAapu4sia et al., PILOT; a Conversational Language User'sGuide (Office of Information Systems, University of California Medical Center,San Francisco, California, December, 1968).
(a) Developed by Steven S. Silver, ILR staff member, and now implementedat both the Berkeley and Los Angeles campuses of the University of California.Documentation available from the Institute of Library Research, University ofCalifornia at Los Angeles, California. User's Manual in Press.
224 J. C. MEREDITH & D. FERGUSON
A successful match between a scan element and an input element(student answer) triggers a unique set of program =actions.A match does not signify rightness or wrongness of a response.It refers only to the detection of a sought element.
In practice, the reaction provided for a response which iscorrect from the teacher's viewpoint will consist of displaying tothe student a statement to that effect, then making a record of thefact for scoring and control purposes and then exiting tothe next frame. The reaction provided for incorrect matchedresponses may consist of a critical or cueing display followed by areturn to the question. Or it may be followed by a critical statementincorporating the correct answer and then exiting to the nextframe. The same scoring and control tallies can accompany thisoperation.
Lastly, the program reaction on failure to match any of the inputis usually the display of a succession of two or three cueing re-sponses ending with one which gives the correct answer. Thestudent is then passed to the next frame, or in rate cases he maybe advised to sign off and seek the help of an instructor. Fail-match tallies are also recorded for control and scoring purposes.Naturally, dependence on fail routines must be kept to a minimumif meaningful instructional dialogue is to be attained.
THE FRAME
The frame, as such, is primarily an instructional concept. Itmay be programmed in a variety of ways, but usually it is com-posed of a question or other requirement displayed to the student(and usually preceeded by a didactic statement of some kind),followed by a computer-waiting condition which allows the stu-dent to insert his response, which is in turn followed by a pro-gram reaction to that response. The program-reactive displaymay be subjoined by text which is really the beginning of thenext frame, but the transition is not always apparent to the stu-dent : only the author/instructor needs to be aware of frameboundaries, in order to assemble his planned dialogue properly.
For the purpose of this study, we envisaged a single framewith five « satellite » frames. Each of the satellites would be used
STUDENT FEEDBACK IN CAI 225
to supply an element of information requested by the student.,and to ask him if the information sufficed. If so, he would bereturned to the main frame; if not, he could elect either to passto the next frame or to sign off. We call the whole complex ofmain frame and satellites a « macro-frame ».
By calling up the satellites one by one, it is possible to provideindependent subroutines which look like frames, i.e., each has aquestion, a wait mode, and a reaction, but which would never beused as instructional units by themselves. It would be possibleto merge these mechanisms in the main frame, but this wouldrequire elaborate controls to prevent mutual interference.
Another alternative is to maintain an instructional glossary ina separate file, to which the student has access at all times. Thistype of facility can be programmed in the more powerful systems,but its greater scope needs to be weighed against the advantagesof defining a term in context.
A flowchart of the A-Frame is given in Appendix B, and anabridged version of the actual machine program which wouldimplement it in Appendix C. (Scoring devices and reiterative textare omitted throughout).
SCANSION
Student input can be scanned for certain words or parts ofwords, in specified order or (if desired) in any order. Scan specifi-cations can also be nested (DISCUS) in such a way that input istested in its entirety first for one set of elements, and, if successful,for additional sets of elements. This permits very eaboratedecision trees.
A « not » facility can also be used (DISCUS) to destroy anymatch condition when a specified unwanted element crops up inthe response. This could lead to a solution of the problem ofdealing with negatives, double negatives, triple negatives, etc.,if an effective means of dealing with negative affixes can bedeveloped.
A series of scan specifications is always designed with a seriesof answers in mind. We have found it useful to categorize thesealong the following lines :
dr,
226 J. C. MEREDITH & D. FERGUSON
AL
AI
ALP
ALP m
ALPF
ALU
F
(Answer, Legal)
(Answer, Illegal)
(Answer, Legal, Perceived)
(Answer Legal, Perceived,Matched)
(Answer, Legal, Perceived,Failed)
(Answer, Legi.l, Unperceived)
A block of student input(a) of limited length,(b) containing at least one
term relevant to thequestion or its context,and
(c) in syntactical order.Any block of student inputnot meeting the above re-quirement.
Any AL which can bepredicted by the authox /instructor, as possible input.Any ALP which the author/instructor chooses to dealwith uniquely.Any ALP which the authordeems so remote or so in-consequential or so difficultto deal with in its existingform that it should be failed.Any AL which the authorfails to predict as a possi-bility.
(The sum of failed answers) AI ALU ALpF
The type of answer which principally occupies our attentionis ALPM. It is not necessary that the author have in mind every-thing that the student might say embodying the Au, m element(s).It is necessary that he select scan elements which are likely tomatch with a broad range of responses relevant to the question.In other words, he formulates a key word or phrase which willmatch members of a set of semantically similar input, and whichwill distinguish semantically dissimilar input by failing to matchwith it.
The range of ALPM is directly influenced by the form of thequestions which precede them. A question posing a very rigidrequirement (e.g., True-False) reduces the number of ALPM toa minimum, but at the same titne it vitiates the conversationalmode. Our current goal is to open up questions and to give the
STUDENT FEEDBACK IN CAI 227
student as much freedom in formulating his response as we can,without letting match probability fall below acceptable levels.
Appendix D lists he statements which would be displayed tostudents in response to various ALPM and F answers in theA-Frame, both initially (first pass) and recursively (subsequentpasses). Other elements of the program's reactions may beconstrued by referring to Appendix B and to the supportingmaterial in Appendix C. As demonstrated by the imaginarydialogues at the end of Appendix D, it would be possible for thestudent to get through the frame very rapidly indeed. On theother hand, it would also be possible for him to flounder aroundin it for several minutes.
THE STATEMENT/QUESTION
In a pamphlet entitled Microfiche Systems Planning Guide (I)(J. M. Baptie, ed.), appear the following statements :
There are three basic types of unit microforms. These are : (f) thetransparent film sheet, or microfiche, with a moderate to high pagecapacity; (a) the opaque MicrocardR, with a moderate to high pagecapacity; (3) the aperture card, with a low page capacity... (p. 4)
andTo date, the significant microfiche applications appear to be : (I) distri-bution of technical report literature... (p. 3)
These statements were reformulated into the statement/questionon the student form (Appendix A), i.e.,
There are three basic types of unit microforms. These are :(1) The transparent film sheet, or microfiche,(z) The opaque microcard,(3) The aperture card.
Which of these do you think is the most suitable for dissemination oftechnical literature ? Why ?
In the absence of extensive preparation, this is obviously avery faulty combination. What does the teacher mean by unitmicroforms ? Is microfiche the same as transparent film sheet, or are
(z) Copyright 1965 by Microcard Corporation, West Salem, Massachusetts.
2
5
228 J. C. MEREDITH & D. FERGUSON
they offcrod as two different things ? What are the microfiche,the opaque microcard, and the aperture card really like, anyway ?By « which » does the teacher mean « which one ? ); What scopeof dissemination is meant ? Suitable for whom ? What are theconditions that generally govern the dissemination of technicalliterature ?
To cap it all, the instructor asks « Why ? », which would be anexcellent question element if it weren't preceded 5y such a con-fusing array.
It might be possible for a person armed only with an amateur'sknowledge of photography and with the nature of technicalliterature to come up with the correct answer. However, it wasnot our purpose to establish this, but rather to see whether allrespondents might be led to a degree of understanding whichwould permit them to formulate a correct response. This seemsto accord with the theory that (at the problem-solving level)learning acquired through active participation by the student ismore useful and durable than that which is passively ingestedby him.
EXPERIENCE WITH THE g A » SET OF ANSWERS
As stipulated, twenty-five answers from the 50-answer samplewere used to test the scans (SC) and their associated reactive textsin the « A-Frame ». It was expected that the « A » set of answerswould reveal omissions and unforeseen ambiguities latent in thescan specifications. For example we might have specified
(SC) CLARITY, LEGIBLE, CLEAR,EASY EYES, EASIER EYES
with the idea of picking up something likeIt's better because it's more legible.
But then the occurrence of« It's better because it's more readable »
among the sample answers would show the need to add READ-ABLE to the specification.
Also the answers were expected to turn up instances whereinthe program reaction text should be modified to accommodate
STUDENT FEEDBACK IN CAI 229
variant input expressions. For example suppose we wanted totreat alike all answers expressing preference for microfichebecause it is readily reproduceable. We might write the reactivetext in the following form :
« YES, IT IS ».
Then if one of the samples turned out to be« I prefer microfiche because s simple copy machine can make copies veryquickly and cheaply »
clearly the reactive text would need to be adjusted. (Sometimesthe reaction logic goes completely awry and new subroutineshave to be added. In other cases the change of a single word willsuffice to make the reactive text more broadly suitable).
Both of these expectations were fulfilled, but we also discoveredthat the means for dealing with the « I don't know » family werecompletely inadequate. And this led to one or the most interestingdevelopments to come out of the exptriment.
THE « I DON'T KNOW » CONDITION
The idea had been to scan for versions of « I don't know »occurring with any of the following essential elements :
DISSEMINATION (TECHNICAL LITERATURE)
APERTURE (CARD)
UNITIZED (MICROFORM)(MICRO) FICHE (TRANSPARENT SHEET)
(MICRO) OPAQUE (CARD, MICROCARD)and to deal with them in the five satellite frames. If more thanone of the sertrate elements occurred in an «I don't know »answer, an appropriate succession of satellites would be called up.
Expressions of plain « I don't know » or its variants wouldinvoke a response asking the student to identify the terms heneeded to have explained.
Expressions of « I don't know » together with unidentifiedmatter would invoke a similar response, also letting him knowthat a portion of his answer had not been understood.
Altogether, eight of the twenty-five answers fell into these lasttwo categories I
10
230 J. C. MEREDITH & D. FERGUSON
Group I Don't know no further clue#1 I don't know.#1 I don't know.#9 I really haven't the slightest idea.
Group II Don't know plus unidentified matter4h I don't know enough to decide.#4 I don't know what these are exactly. So can't tell.# 5 I don't know. They all seem about equivalent to me.
%to Not enuf info to answer this would need to know propertiesof microforms and kinds/quality of available photocopy.
# I I I've only had occasion to use the microfilm on reels, so I can'treally say.
A ninth answer, « I can't honestly compare because I am notfamiliar with all 3 of the media », really belongs in Group II,except that it was caught accidentally by the scan for « I don'tknow » « aperture card (3) ».
On revision, we sought to identify the clues in # 5, #and # I x and to modify the scans as follows :
# 5 scan for « all, same, equivalent » and provide explanatoryroutine (probably a satellite).
# io add « enuf » and « info » to the « don't know » specification.#i t scan for « reels, rolls », etc. Although microfilm on reels is
outside of the problem, it is relevant and may provide a goodinsertion point inside of ;.he problem; so we would write anadditional statement covering it.
The original programmed response to Group I was. « I'll beglad to explain any of the terms I've used, if you wish. Justtype 'Clarify' ».
The original programmed response to Group II was « I don'tunderstand exactly what you need to have clarified ».
Answer # To would have fallen through to FAIL (anotheraccident) because « enuf » and « info » had been omitted fromthe scan specifica'ion. However, as luck would have it, theprogrammed response was not too bad : « How's that again ?I don't understand your response as worded ».
But all three responses fail to differentiate between cognitivestates which would be fairly obvious to a human interlocutor.Granted that we can patch up # 5, #8, # Jo, and # I r, the
STUDENT FEEDBACK IN CAI 231
,:maining five raise some interesting theoretical points whichwe are now obliged to deal with.
INFORMATION-STATE, MOTIVATION, AND MOTIVATIONAL INDICATORS
For the purpose of this study it has seemed reasonable to makecertain assumptions regarding the information-state signalledby « I don't know ». Without really plumbing the depths ofcognitive theory, we might say that :
1. Information, in order to be recognizable as such by more than one person,must be capable of being expressed in terms which mean approximatelythe same thing to others. This set of agreed meanings is the basis forsemantic transfer.
2. Awareness of information requires a substrate, comprised of relatableunits... (t) the presence, absence, and quality of which tend to bound theultimate knowledge state.
3. The cognitive process (either self-generated or derived from an outsidesource) consists of a perception of some kind of logical structure formedby the relationships between the information units. The integrity of thisperception depends on the individual's synthetic skill and on theaccuracy/completeness of the substrate. Its transferability depends onsemantic equivalence (I, above).
4. Self-sustaining cognition can be triggered by an outside stimulus or byan influx of critically related information units. In either case it representsa learning proces6 thich engages the student's whole attention. Derivedcognition, on the other hand, while it may be accepted, probably doesnot command the student's attention to the same degree.
The order in which the first three assumptions are stated isnot meant to imply an unvarying sequence. For example, gapsin the informational substrate do not preclude formation of aknowledge structure, even though the structure may be somewhattentative and unstable until the gaps are filled in. Nevertheless, theexistence of some substrate appears to be prerequisite to structuralperception, so a progression from (2) to (3) can be stated as afifth assumption. (In fact, we might think of structural perceptions
(t) Ausubel might speak of these as « anchoring ideas ». See D. P. Ammo,Educational Pgebology ; a Cognitive View (New York, Holt, Rinehart andWinston, 1968) (esp. « Learning and the availability of relevant anchoringideas », p. 132 et seq.).
232 J. C. MEREDITH & D. FERGUSON
being packed down in use until they can be employed as in-formational units, to serve as substrate elsewhere. But the reverseof such a process is hard to imagine).
Probably none of the assumed conditions could be broughtabout in the absence of some kind of motivation, and it appearsreasonable to suppose that this element is strongest at the momentof creation, i.e., when the structure is about to be perceived.Can we, then, use motivational indicators to tell us somethingabout the student's cognitive state ? For example, from expressionsof disinterest that a semantic block exists, or that the informationalsubstrate is so fragmentary that nothing can be made of it ?
Does a request for specifics indicate that the structure thoughshaky is beginning to take shape ?
These questions impel us to see if we can detect motivationalindicators imbedded in « I don't know »s, hoping that from thesewe might get an idea of information-state. This in turn wouldpermit us to react to individual «I don't know » answers withmore discrimination.
Reverting to the Group I answers :# r I don't know. Curt. No evidence of interest.# a I don't know. Possibly none of the terms in the
statement/question connect withany information unit.
#9 I really haven't the slightest idea. Almost explicit disinterest, plusovertones of distaste for thesystem itself. There is a possibilitythat the student has some infor-mation but prefers to dismiss it,for the sake of being expressive.In any case, we have to take himat his word.
In the face of zero motivation implying a near-zero informationalsubstrate, we have two choices :
We could say « All right, let's talk about something else »,and go to another part of the program trying to find somethingwhich the student knows enough about to get off to 'a fair learningstart. Later, he might become interested and knowledgeableenough in « microphotography », for example, to allow us toreturn him to the A-Frame on a better footing. (This faculty of
13
STUDENT FEEDBACK IN CAI 233
letting the student influence his own learning path is of courseone of the most promising features of CAI technology).
We could start building, bit by bit, the necessary collection ofinformation units by literally imposing them on the student adifficult and thankless task, and one which promises no earlysurge of motivation on his part.
In this particular study we attempted a modification of thesecond tactic in order to be in a position to observe second-passperformance later on. We tried to pinpoint critical informationunits which are necessary in dealing with the question, i.e.,characteristics of microform types. Then we pointed out ourawareness of the difficulty of answering the question withoutthese units. Finally we presented a tactful offer of assistance.
« I realize that it is almost impossible for you to answer without knowingsomething about the types of microforms mentioned, and about the factorsinvolved in dissemination of technical literature. I'd like to help you tacklethe problem from the beginning by supplying the information you need.Are you willing to work with me toward a correct solution ) o
Second-pass reaction to a « yes » from the student would be :« Which item would you like me to describe first ? Type « De-scribe ». Second pass reaction to a « no » from the studentwould be a signoff, with a consultation recommended.
Turning to the answers in Group II :# 3 I don't know enough to decide.
#4 I don't know what these are ex-actly so can't tell.
Polite regret implies some wil-lingness, so there may be a fewscraps of information upon whichto build.
Rather than trying to ascertain the areas of complete ignorance,the first move was to ascertain and strengthen the areas ofknowledge, then to fill in the gaps. The following text was aimedat accomplishing this :
« Well, suppose we start by examining the items you may already have heardabout. I may be able to add some useful information about them. Howabout microfiche ? Microcard ? Aperture card ? The problem of dissemi-nation of technical literature ? Are you acquainted with any of these ?
Second-pass reaction to « Yes » would be : « Which ? » Second-pass reaction to « No » would be to step down to the text given
/4
234 J. C. MEREDITH & D. FERGUSON
for plain « I don't know » answers. Second-pass reaction to aspecific would be to branch to a satellite. Third-pass reactionto another « I don't know » would be something like this :
« Shall we take them up one at a time ? Suppose we start with the problemof dissemination of technical literature ».
This would be followed by at least four of the satellite sub-routines.
To continue with Group II :# 5 I don't know. They all seem about
equivalent to me.
# 8 I can't honestly compare becauseI am not familiar with all 3 of themedia.
Here the student is clearly makingan effort of some kind. He mayalready have a fair idea of thedissemination problem, since hisresponse shows a narrowing tothe three forms suggested.
In this case we are justified in concentrating initially on a de-finition of the three types.
A-FRAME PERFORMANCE WITH OTHER TYPES OF ANSWER
In the remainder of this section are listed the other answers,and an indication of how the program handled them. In order tosave space at this point, we simply refer opposite each to theAppendix D code number which defines the reaction that wouldhave occurred. An assessment as to whether or not that reactionwould have been appropriate appears in the right-hand column.
Group#
Answer# Student input
AppendixD
channel
Appro-priate
1st pLss ?
IV
IV
6
8 (a)
I don't know what an aperturecard is. Nor (1) or (2).
I can't honestly compare becauseI am not familiar with all 3 of themedia.
8.a
7.d
Yes
No
(a) Repeated here for statistical purposes. (The program would have interpreted the inputas meaning «I don't know what an aperture card is ». The outcome might have beensatisfactory in this case, but no credit can be taken).
15
STUDENT FEEDBACK IN 235
Group#
Answer.# Student input
AppendixD
channel
Appro-priate
1st pass ?
IV 12 Please define opaque microcardand aperture card.
8.a Yes
V 13 I believe the opt que microcardis the most useful.
4.a Yes
V 14 Transparent film sheet; justseems easy to store and use.
2.a No (b)
V 15 Aperture card too awkward for abig file, easier to enlarge micro-fiche for convenient viewing.
4.b No (c)
VI 16 None. I believe in hard copy. 10.a YesI dislike sitting in dark roomsand looking at idiot boxes.
V 17 I think the microfiche would bebest because I think it can con-tain more info than the otherforms.
2.a No (d)
IV 18 3. The aperture card can behandled mechanically.
4.b No (e)
IV
IV
19
20
Aperture card, because the cita-don infro (sic) is readable withoutresorting to a reader,
3. Aperture cards because theyare processed easily.
4.b
4.b
No (f)
No (g)
(b) Should have gone to a.b.z. « Store a was overlooked in the scan specification.(c) Program fault. On revision, something to screen correct negative judgements is needed.(d) Should have gone to 2.b.z. « Contain more info a was overlooked in the scan specification.(e) « Why ? a will be misinterpreted. A scan for the sense of machine handling and processing
is required.(f) « Why ? a will be misinterpreted. A scan for the sense of eye-legible titling is needed
in any case, and perhaps « citation » should be one of the specified keywords, althoughits occurrence seems remote.
(g) Same as (0).
110
236 J. C. MEREDITH & D. FERGUSON
Group#
Answer# Student input
AppendixD
channel
Appro-priate
1st pass ?
IV 21 The opaque microcard is ade-quate.
4.a Yes
IV 22 The microfiche offers better re-production (visually) than themicrocard.
2.a No (h)
IV 23 I would suggest the microfiche,since complete information canbe stored in a small space.
2.b.2 Yes
IV 24 #2 is most suitable. As apositive copy of the text it isleast objectionable to the user.
2.a No (i)
IV 25 The microfiche has proved quiteuseful for cartographic informa-tion.
2.a Yes (j)
(h) The answer itself is ambiguous, but suggests the need for scans covering (z) manualcopying, (a) rendering into hard copy, and (3) clarity of display in a reading machine.
(i) A scan is needed to dispose of the positive-negative issue, if possible, followed by areturn to other considerations.
(3) « Cartographic a is too far out to provide for at present. One can only hope that the« Why ? » will elicit something which the system can process more meaningfully.
EVALUATION AND REVISION OP THE A-FRAME
Appendix F shows in tabular form the characteristics of eachof the «A » answers; how it fared in the «A -Frame »; and anevaluation of the A-Frame as to each, on a scale of i -o. Thesummary I ,f this data leads to one conclusion, namely, that althoughthe percentage of matches on the first pass was high (96 %), frameperformance as a whole was not commensurately good, since programreactions were suitable in only about half of the cases. Accordingly wewere obliged to evaluate the frame as a whole at less than 5o %.
Let us examine this more closely. The A-Frame was planned andwritten by an experienced CAI programmer, and his work was
I
STUDENT FEEDBACK IN CAI 237
checked and emended by a second person, also experienced inthe field. It is not surprising that between them they should haveforeseen a range of keywords such that 24 out of the 25 answerswere intercepted. But they failed to anticipate many of the sec-ondary elements which were needed to refine the meanings of theanswers. For example, every response containing a primaryelement : microfiche, microcard, or aperture card, was identified assuch, but further processing did not occur, because of variousomissions and other faults in dealing with such things as « me-chanical » and « info ».
This in itself was not unexpected. The original purpose of thetest was to see how much of an improvement could be madein the frame, using ideas suggested by operational experience.Admitting that no one person nor even two persons in collabora-tion could imagine the whole scope and variety of possible re-sponses to a fairly open question, we wanted to gauge the directutility of student responses in augmenting the range of ourperceptions in this regard. We might even decide that the bestway to design a frame was to erect only the barest structure,without trying to cover every eventuality, and to rely on aninflux of on-line responses from live students to point the wayto the frame's further development.
But the experience with « I don't know » answers raisedanother problem. It was not that the program reactions wereparticularly bad. They just weren't particularly good. They madeno account of attitudinal overtones which to a human teacherwould have clearly indicated shades of knowledge-condition inthe student. The effort to improve this situation needs to beconsidered apart from the simple refinements mentioned in thepreceding paragraph, because, as indicated in the discussion ofcognitive states, we believe that such differentiations might provegenerally useful as tools in CAI methodology.
In order to investigate this, all the words and phrases whichmight be expected to have key status in a « I don't know »statement were listed. Items were clustered in various ways, andthe clusters (of which there were finally ten) were mapped intoeach other on a trial and error basis until it was possible to discernroughly three levels of attitude. These combinations were then
238 J. C. MEREDITH & D. FERGUSON
assigned programmatically, yielding an array of scans we call the« M-series », set forth on page 276, Appendix G.
Taking the scan labelled « MO » (motivation zero) as an ex-ample, we see that it is composed of elements from the scanlabelled «-A », or from scans « Az » plus « Bi » plus « CI ».Any answer which had not been screened out by an earlier scanwould thus be matched by « MO » if it contained any of thefollowing :
... don't know...
... can't say...
... can't imagine...you expect...
. you think...
. at sea...
... with it...search me...
or a combination of the following :... don't have...... not have...... haven't...
expect have...... think have...
plus
plus
least..... vague...
vaguest...slightest...fuzzy...fuzziest...foggy...foggiest...remote...remotest...
. idea...notion...interest...concept...conception...inkling...
STUDENT FEEDBACK IN CAI
Thus
239
« What makes you think I could possibly answer that ? »« Search me 1 »« I'm sure I don't know ».« I haven't the foggiest notion ».
would all be matched in « MO » and be treated accordingly.The « M-Series » of scans would not need to bulk as large in
the program as might be supposed from the foregoing, becauseof the common device of forgiving parts of words which maybe missing or misspelled. For example the form FOGG* (PILOT) orFOGG' (Discus) accepts « foggy », « foggiest », « foggier »,
and « fogged ». A third language, FOIL (r), forgives a specifiedpercentage of characters : FOGGIEST (50 %).
The adoption of an additional line of inquiry did not seem toprejudice the original idea of the model, because it is possible toassess performance according to AL groups individually or incombination. Thus in the A-Frame the following breakdown canbe made :
Score Possible %
Groups I and II (« I don't know ») 5 1/2 8 69Groups III, IV, & V (all others) 6 1/2 17 38
Composite 12 25 48
The A-Frame's superficial success with Group I and II restssolely on the fact that forms of « I don't know » can be ex-haustively predicted, compared with all the other things that canbe said by a student, and a response that makes some sense can
(I) HESSELBART, J. C., « FOIL a file-oriented interpretive language » inProceedings of the 23rd National Conference of the Association for Computing Machinery(ACM Publication P-68, Princeton, Brandon, 1968), pp. 93-98. See also in thesame publication the review of CAI languages by Karl ZINN, « Programmingconversational use of computers for instruction » (pp. 83-92).
20
240 J. C. MEREDITH & D. FERGUSON
be trotted out. The M-series of scans represents a new object' .ve,the hope that we could significantly refine this ongoing sense.
As for Groups III, IV, and V, we could continue to pursuethe general intent of improving scan specifications word by word,so a?. to catch and channel a greater variety of unique responses.In some cases it would be necessary to write separate new scanstatements and program reactions to deal with aspects we hadcompletely missed in the first version, such as the idea of me-chanical handling.
PART II
PROCEDURE
Appendix G (Flow Chart) and Appendix H (Program) showthe model as revised and redesignated the « B-Frame D.
Very little was done to alter the flow, except to insert theM-series of scans just ahead of « DUNNO + GARB », and to adda satellite to deal with mechanical handling. The A-Frame flow-chart omitted certain controls specified in the program, and theB-Frame does the same.
Appendix I corresponds with Appendix D, in specifyingexactly what would happen to an identifiable type of input marchedby a particular scan statement.
All this work was completed before we allowed ourselves tolook at the « B » set of answers. When we had everything ready,we proceeded to drop them one by one into our paper « hopper ».It may be asked why we didn't actually put the program in thecomputer and test it on-line, in order to avoid any possibility ofshading the results. We would have liked to do this, of course,but as mentioned in the beginning we had assumed an augmenta-tion of an existing language (PILOT) with features of one whichwas not yet operational (DISCUS). In other words, we wereusing a non-existent CAI language. More recently, DISCUS hasindeed been implemented at the University of California Berkeleyand UCLA campuses, and we expect to use it for most of ourCAI programming.
Anyone monitoring the project might have objected to the
e-I
STUDENT FEEDBACK IN CAI 241
fact that we did not seem to require an absolute fit of some inputterms with the scan specifications, as originally laid down inAppendix C and Appendix H. The reason for this is that weassumed the « forgive » convention whereby « stor* », forexample, would match « storing », « storage », or « stored», butwrote one of these out for better understanding by the reader.This is obviously misleading, though, so we revised these twoAppendices to show such a specification as « storage » with onlythe underlined characters representing the absolute requirement.Thus a reader can perceive both the real specification and theterm idea behind it. Except for this reformatting, no retrospectivechanges of any consequence were made in any of the supportingdocuments.
COMPARISON OF RESULTS
The summaries of A-Frame and B-Frame observations arecompared in Table I. Clearly a remarkable improvement tookplace in respect to what we call « sense score », i.e., the percentageof student answers to which the program would have respondedsatisfactorily.A satisfactory response in every case is deemed to be one whichnot only makes sense on the first pass but offers a good chanceof leading to a sensible response on the next pass. (This rulesout the plausible but misleading response of « Why ? » to ques-tions A-I8 and A-x 9 (see page 230 where the student is talkingabout one thing and the program is talking about another).
In these assessments the student viewpoint controls; we cannotexcuse awkward « noncommunicating » responses ongrounds that they suit our pedagogical plan or that they will leadin the end to some kind of complete exposition.
Before looking into the significance of the results, we need toask if enough similarity existed between the A set of answersand the B set of answers to validate a comparison of A-Frameperformance and B-Frame performance. I think we can say thatthe motivation scores and the proportions of Group V answersshow that the two sets were quite alike certainly close enoughfor our purposes.
a R.
142 J. C. MEREDITH & D. VERGUSON
Table ICOMPARISON OF A-FRAME AND B-FRAME PERFORMANCE
ON SIM;LAR SETS OF ANSWERS
A-Frame B-Frame Change
Match score 24 (96 %) 25 (100 %) + 4 %
Sense scoreGroups I & II 5 1/2 / 8 (69 %) 5 1/2 / 6 (91 %) +22 %Groups III, IV, V, & v I 6 1/2 / 17 (38 %) 16 1/2 / 19 (86 %) +48 %Composite 12/25 (48 %) 22/25 (88 %) +40 %
Characteristics of the sets :
Motivation to 4x0.--- 0 2x0= 0Motivation 1 4 x 1 = 4 6 X 1 = 6 +2Motivation 2 3 X 2= 6 3 X 2= 6Motivation 3 14 x 3 = 42 14 x 3 = 42
Composite 52 54 +2Possible maximum 75 75Factored 52/75 54/75 +2/75
Answer Type Groupings :
I 3 1 2II 5 5III 1 0 1IV 3 6 +3V 12 13 +1VI 1 0 1
25 25
SIGNIFICANCE
The total number of unique AL which might be entered inresponse to a given question is a function of the openness of
as
STUDENT FEEDBACK IN CAI 243
that question, and can range from a very small set (as in responseto a True-False) to one which is very large (as in response to« What's it all about ? »). Let us call this number K and charac-terize it as finite (because of the constraint of our AL definition)but indeterminate (because no matter how many different legalanswers we can imagine in response to a question, someone maycome along with one we had not thought of).
If we were to scan randomly for AL the curve of match pro-bability would be a straight line as shown in Figure I. But wedo not scan randomly, because we know that the frequency ofoccurrence of some AL is far greater than that of others, and thatit is possible to attain a better match probability for K scans bytailoring them to the :core frequent AL. The uppermost curve(line 4) represents the cumulative frequency of occurrence ofAL and at the same time serves to represent the match proba-bility (P) which a CAI programmer could obtain for N scans ifhe knew the makeup of the items to the left of N.
But of course he doesn't know this. He can only guess, on thebasis of his perception of (t) the topic, (z) the student set, (3) theposition of the question in context, and (4) the language in use,which AL will occur more frequently than others, and howmuch more frequently. The richer these variables, the harderit will be for him to predict not only the identity of the AL tothe left of N but the sha-le of the curve of their frequency. Line zshows how his prediction will always be better than randombut less than perfect.
Line 3 represents a closer approach to Line 4, attained by virtueof revising and improving upon the original, highly subjective,work of the programmer. What is the most efficient way of doingthis ? By bolstering his predictions with external data furnishedhim by the system in the course of its operations.
We would like to get a better idea of the role of this externaldata (feedback) what does it look like ? how can it best becategorized ? how many iterations should be considered beforethe utility of the process is overshadowed by the tasks associatedwith it ? These questions have a direct bearing on the refinementof conversational CAI techniques.
244 J. C. MEREDITH & D. FERGUSON
F1.0
AL
Fig. I
We are still a long way from being able to scale curves z, 3,or 4, for even one set of variables, but at least the results obtainedfrom this study indicate that
(1) for an open question with a large K and an adult, heterogenous studentset, a striking improvement in meaningful intercept can be obtainedthrough close analysis of a modest number of live responses, and
(z) as a corollary, the rate of improvement to be derived from close analysisbeyond this modest number must decline quite rapidly.
It is only fair to ask how much of the improvement in meaning-ful intercept (which is a better measure than raw match proba-bility) should be credited to the M-series of scans, representinga completely new conceptual element in the B-Frame. Lookingback to the evaluations given for responses to Group I andGroup II answers in the A-set, it will be seen that none of them
g.5
STUDENT FEEDBACK IN CAI 245
was downgraded for failure to detect nuances of « I don't know »,but only for mech inistic faults. Thus the conditions determiningsense score are the same for both frames.
ACKNOWLEDGEMENTS
Our thanks to Miss Helen Yen, ILR staff, for gathering « stu-dent » input for our model frame, and to the fifty individualswho contributed to the sample.
APPENDIX A
FORMS
1. Attachment (I) is the form of answer sheet used by all Ely« students ».
2. Attachment (2) is the form used by the authors for analysisof student responses and program reactions.
Attachment (I)From : J. C. Meredith, Institute of Library Research
(Information Processing Laboratory Project)You are asked to participate in a survey of response patterns to a singlestatement/question typical of programmed instruction, as follows :
There are three basic types of unit microform.These are :i. The transparent film sheet, or microfiche.2. The opaque microcard(19.3. The aperture card.Which of these do you think is most suitable fordissemination of technical literature ? Why ?
(Assume that the program is capable of dealing with responses consisting ofsymbols, numerals, or English words, singly or in combinat ion, up to two linesin length. Your response may be in the form of a statement or a question orboth, but should at least be relevant. Try to decide on your answer quickly.)Response :
a'
246 J. C. MEREDITH & D. FERGUSON
Thank you. Please check the appropriate box(es)
Attachment (2)
STUDENT ANSWER
PROGRAM RESPONSE
I. First passz. Likely znd pass
SATISFACTORY ?
TYPE OF MODIFICATION INDICATED
REVISED PROGRAM ELEMENT
APPENDIX 3
[11 LS
1:1 ILROther
«A -FRAME FLOWCHART
I. Attached.
z. The chart represents in abbreviated form the decision sequencesenvisaged for the «A -Frame ». The following notes pertain :
i) The «Q entry (large circle) represents student input, terminatingwith a « send » signal which causes the computer to resumeprocessing.
z) « Q » returns (small circles) imply a textual display of some kind,generated according to the specifications laid down in the programitself, and held on the CRT while the computer is waiting for thenext input. Consult Appendix D to ascertain actual text planned foruse in various situations.
3) « OUT » (small circles) signifies exit to the next frame.4) To avoid excessive complication of the flowcart, several tests
(diamonds) have been omitted. For example, the first five major
0 4.1
0 0t.0
WM
'
00
NM
I15
1K
Z?
05A
TE
LL:
"UN
I T Z
"
.11.
7 tL
e
T.. 0
1"o
r I
IIER
Z5,
331.
1!
on.
00
KA
LI.
O.o
t
tusi
f 'Le
!
00
7.4
v.,
v.. 0
FI1
11_
11.1
1 .3
(Sr
JO
73%
? 44
0, T
i. C
Z, 1
3,35
0. (
6YP
AS
SI
z3,
35. I
I, lt,
to
A-F
ram
e Fl
owch
art
11?
0
7t.
ca A,
Um
, 'N
C
Trr
ts
4
Fir
ER
!
Itt 0
248 J. C. MEREDITH & D. FERGUSON
tests to the left of the « DUNNO » diamond entail additionalsubordinate tests and decisions associated with recursive scan of thesame input string, under a series of « nested » scan parameters.These are omitted from the diagram.
5) The group of operations to the left of « DUNNO » is devoted toprovision of information intended to help the student solve theproblem. One of five information « satellites » UNITZ isshown in the large box. It should be visualized as occupying theaction box marked « CALL UNITZ » (hand).
6) The group of operations to the right of « DUNNO » is designedto accommodate a student's attempt to make the right choice. Eachchoice and the first recognized valid reason, or a combination(not shown) of the two or three best reasons, call forth a uniqueblock of text. Note that if the student fails to give a recognizedreason, the program loops through to a display of « Why ? » to Q,and his second response is scanned without requiring him to repeathis choice. (This is another case where the diagram omits specifictests).
7) The remaining operations (bottom center) are miscellaneous tests andutility operations, ending with two fail-match displays terminatingthe frame.
8) The typed numerals refer to answers in the «A » set, located so asto show how they would have been treated in a first pass throughthe program.
APPENDIX C
THE « A-FRAME N PROGRAM
i. Attached.
2. The program listing resembles actual source input in a high-level CAI programming language (selected string match type),such as PILOT. We have, however, omitted all delimiters (whichserve only to notify the compiler whether it is dealing with alabel, an operation code, or an operand) and have stylized thescan specification in logical form (e.g., a comma means « or »).
3. The following notes will assist the reader in interpreting theprogram :
x) All operations are sequential from left to right and from top tobottom.
Rq
STUDENT FEEDBACK IN CAI 249
a) For purposes of this sample, « G » signifies a display occurringonly if a preceding scan specification has been successfully matched.« F » signifies a display occurring only if all match attempts havefailed; and « T » signifies a display occurring regardless of matchcondition if execution reaches that point.
3) Reference to a label in quotes means that everything in the operandso labelled is to be inserted at that point. The expanded versions ofall labelled alphabetic strings are listed toward the end of the program.
4) The control statements in the right hand column would in manycases be constrained to operate only under a « match » condition,the same as a « G » textual display. The necessary qualifiers for thishave been omitted for the sake of simplicity.
5) Control statements ADD, TALLY, and MARK mean « add »,« add one », and « ensure one and one only in » respectively. Theyaffect the contents of numerical counters whose existence is estab-lished implicitly rather than by formal definition.
6) The sequence in which various match attempts are made reflects therule that normally scans terminate immediately on successful match
the remainder of student input is ignored. This favors puttingthe larger, more difficult specifications ahead of those which areeasier to satisfy. For example, the scan for « I » is placed at thevery end, so that it will not ensnare emphatic statements whichcontain sought-for elements specified in other scans.
7) Many of the planned unique text displays ste 3o similar in structurethat they can be assumed. or example the response to «I likemicrofiche because it's convenient » would call forth exactly thesame kind of text as « I like microfiche because it's cheap ». To avoidneedless repetition, many of these have been omitted from the listing.
8) A sample satellite is shown on the last two sheets of the program.The other four satellites would resemble it very closely. It mighteven be feasible to use just one satellite with a choice of texts to be« USED » depending on which indicators (variables such asMICROY) had been activated.
0
« A
» S
ourc
e Pr
ogra
m
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
EX
POT
(Ini
tial s
tate
men
t and
que
stio
n)
MIC
RO
.AA
(stu
dent
inpu
t)
R(«
DU
NN
O »
, CL
AR
IFY
) ±
(U
NIT
, UN
ITIZ
ED
)T
AL
LY
(U
NIT
Y, P
RO
BY
),IF
UN
ITY
2 A
DD
UN
ITX
,>...
.
TO
DU
PY
R.
(« D
UN
NO
», C
LA
RIF
Y)
+ (
TR
AN
SPA
RE
NT
, FIL
M. S
HE
ET
,FI
CH
E, M
ICR
OFI
CH
E, «
N.1
»)
TA
LL
Y (
MIC
RO
Y, P
RO
BY
),IF
MIC
RO
Y >
2 A
DD
MIC
RO
XI
TO
DU
PY
(« D
UN
NO
», C
LA
RIF
Y)
+ (
OPA
QU
E, M
ICR
OC
AR
D, «
N.2
»)
TA
LL
Y (
OPA
QY
, PR
OB
Y),
IF O
PAQ
Y e
2 A
DD
OPA
QX
TO
DU
PY
R(«
DU
NN
O »
, CL
AR
IFY
) +
(A
PER
TU
RE
, « N
.3 »
)T
AL
LY
(A
PER
Y, P
RO
BY
),IF
APE
RY
?...
. 2 A
DD
APE
RX
TO
DU
PY
IF (
UN
ITY
, MIC
RO
Y, O
PAQ
Y,
DIS
SEY
) >
2 U
SE D
UP,
AD
D«
UN
ITZ
» T
O P
RO
B U
NIT
Y,
AD
D «
MIC
RO
X »
TO
PR
OB
ON
MIC
RO
Y, A
DD
« O
PAQ
X »
TO
PRO
B O
N O
PAQ
Y, A
DD
« D
ISSE
X »
TO
PR
OB
ON
DIS
SEY
GI
UN
DE
RST
AN
D Y
OU
NE
ED
MO
RE
IN
FOR
MA
TIO
N A
BO
UT
« PR
OB
»IF
PR
OB
Y _
>...
. 2 U
SE T
AK
EU
P,C
AL
L U
NIT
Z O
N U
NIT
Y, C
AL
LM
ICR
OZ
ON
MIC
RO
Y, C
AL
LO
FAQ
Z O
N O
PAQ
Y, C
AL
L A
PER
ZO
N A
PER
Y, C
AL
L D
ISSE
Z O
ND
ISSE
Y, J
UM
P T
O M
ICR
O.A
bk
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
R«D
UN
N°
» +
AN
Y
GO
NE
OF
TH
EM
IS
SUPE
RIO
R I
N R
EPR
OD
UC
EA
BIL
ITY
, CO
ST,
GE
NE
RA
L U
SE, E
TC
. DO
YO
U N
EE
D M
OR
E I
NFO
RM
AT
ION
?IF
SO
, TY
PE a
CL
AR
IFY
0JU
MP
TO
MIC
RO
.A
R«D
UN
NO
» -
I- «
GA
RB
»
GI
DO
N'T
UN
DE
RST
AN
D E
XA
CT
LY
WH
AT
YO
U N
EE
D T
OH
AV
E C
LA
RIF
IED
.M
AR
K C
LA
RA
, JU
MP
MIC
RO
.A
R«D
UN
NO
a, N
OT
CO
MPE
TE
NT
, IN
CO
MPE
TE
NT
, NE
ED
INFO
RM
AT
ION
, RE
QU
IRE
IN
FOR
MA
TIO
N
GI'L
L B
E G
LA
D T
O E
XPL
AIN
AN
Y O
F T
HE
TE
RM
S I'V
EU
SED
, IF
YO
U W
ISH
. JU
ST T
YPE
a C
LA
RIF
Y0
MA
RK
CL
AR
A, J
UM
P M
ICR
O.A
R«N
.1 »
« P
LU
SCA
P »
a PL
USC
AP
» «
N.1
MIC
RO
CA
PG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, BU
T T
HE
RE
ASO
N I
T I
S SU
PER
IOR
FO
R T
HE
DIS
SEM
INA
TIO
N O
FT
EC
HN
ICA
L L
ITE
RA
TU
RE
IS
NO
T T
HA
T I
T H
AS
GR
EA
TE
RC
APA
CIT
Y (
WH
ICH
DE
PEN
DS
ON
TH
E R
ED
UC
TIO
N R
AT
IO
R
USE
D)
BU
T O
N T
HE
i A
CT
TH
AT
a F
ICH
UP
»
c N
.1 »
+ «
PL
USE
ASE
» +
« P
LU
SCH
EA
P »
JUM
PT T
O N
EX
TFR
AM
E
MIC
RO
BO
TH
GG
OO
D. Y
OU
CO
RR
EC
TL
Y P
ICK
ED
TW
O O
F T
HE
IM
POR
TA
NT
AD
VA
NT
AG
ES
OF
FIC
HE
. a F
ICH
UP
aJU
MP
TO
NE
XT
FRA
ME
MIC
RO
EA
SEG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, AN
D E
ASE
OF
DIS
TR
IBU
TIO
N I
S O
NE
OF
ITS
CO
MPE
LL
ING
AD
VA
N-
TA
GE
S. a
FIC
HU
P »
JUM
P T
O N
EX
TFR
AM
E
I
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
R«
N.1
s «
PL
USC
HE
AP
», «
PL
USC
HE
AP
» c
N.1
»
MIC
RO
CH
EA
PG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, AN
DC
OST
CO
NSI
DE
RA
TIO
NS
CE
RT
AIN
LY
EN
TE
R I
NT
OT
HE
PIC
TU
RE
. «FI
CH
UP
»JU
MP
TO
NE
XT
FRA
ME
R«N
.1 »
« P
LU
SEX
PER
IEN
CE
», a
PL
USE
XPE
RIE
NC
E »
« N
.1 »
GC
AN
YO
U C
OM
PAR
E T
HE
M M
OR
E O
N T
HE
BA
SIS
OF
INFO
RM
-A
TIO
N?
IF Y
OU
WO
UL
D L
IKE
TO
KN
OW
MO
RE
AB
OU
T T
HE
OT
HE
R F
OR
MS,
OR
AB
OU
T T
HE
GE
NE
RA
L R
EQ
UIR
EM
EN
TS
FOR
DIS
SEM
INA
TIO
N O
F T
EC
HN
ICA
L L
ITE
RA
TU
RE
, JU
STT
YPE
a C
LA
RIF
Ya
JUM
P T
O M
ICR
O.A
(rep
eat s
erie
s fo
r a
N.2
» a
nd «
N.3
)
RR
EJE
CT
, HA
TE
, DIS
LIK
E, «
NE
GA
T »
LIK
E
GA
SID
E F
RO
M Y
OU
R P
ER
SON
AL
FE
EL
ING
S IN
TH
E M
AT
TE
R,
CA
N Y
OU
EX
PRE
SS A
N O
PIN
ION
AS
TO
WH
ICH
MIG
HT
BE
LE
SS U
NSU
ITA
BL
E F
OR
TH
E S
TA
TE
D P
UR
POSE
?JU
MP
TO
MIC
RO
.A
R«
NE
GA
T »
EX
PER
IEN
CE
GI'D
LIK
E Y
OU
TO
CH
OO
SE O
N T
HE
BA
SIS
OF
AV
AIL
AB
LE
INFO
RM
AT
ION
, IF
POSS
IBL
E. P
LE
ASE
TY
PE «
CL
AR
IFY
» FO
R E
XPL
AN
AT
ION
OF
IND
IVID
UA
LE
LE
ME
NT
S.JU
MP
TO
MIC
RO
.A
R«N
EG
AT
»T
AL
LY
HO
PEL
ESS
, IF
HO
PEL
ESS
= 1
JU
MP
CH
AN
CE
GPL
EA
SE S
IGN
OFF
NO
WJU
MP
TO
EN
D
Lab
els
Cod
eD
ispl
ay te
st a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
CH
AN
CE
GW
OU
LD
YO
U L
IKE
MO
RE
IN
FOR
MA
TIO
N ?
IF
SO, P
LE
ASE
IND
ICA
TE
BY
TY
PIN
G «
CL
AR
IFY
D
(WH
ICH
EV
ER
TE
RM
BO
TH
ER
S Y
OU
).JU
MP
TO
MIC
RO
.A
R«
N.1
» +
« N
.2 D
+ «
N.3
» (
any
2, a
ny o
rder
)
RA
LL
, AN
Y
GSE
E I
F Y
OU
CA
N P
ICK
JU
ST O
NE
, ON
TH
E B
ASI
S O
FW
HA
T Y
OU
CA
N L
EA
RN
FR
OM
ME
ON
TH
E S
UB
JEC
T.
JUM
P T
O M
ICR
O.A
R«
N.1
»
GW
HY
?T
AL
LY
(A
NSY
, MIC
RO
B),
JUM
P T
O M
ICR
O.A
R«
N.2
»
GW
HY
?T
AL
LY
(AN
SY, O
PAQ
B),
JUM
P T
O M
ICR
O.A
R«
N.3
»
GW
HY
?T
AL
LY
(A
NSY
, APE
RB
),JU
MP
TO
MIC
RO
.A
R«
PLU
SCA
P »
IF M
ICR
OB
JU
MP
MIC
RO
CA
P,IF
OPA
QB
JU
MP
OPA
QC
AP,
IF A
PER
B J
UM
P A
PER
CA
P
R«
PLU
SCL
AR
oIF
MIC
RO
B J
UM
P M
ICR
OC
LA
R,
IF O
PAQ
B J
UM
P O
PAQ
CL
AR
,IF
APE
RB
JU
MP
APE
RC
LA
R
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
R G F F
1 RE
AL
LY
? I
'M S
UR
PRIS
ED
YO
U F
EE
L S
O S
TR
ON
GL
Y A
BO
UT
IT
.IF
YO
U'D
LIK
E C
LA
RIF
ICA
TIO
N O
F A
NY
EL
EM
EN
T O
F T
HE
STA
TE
ME
NT
AB
OU
T U
NIT
MIC
RO
FOR
MS,
OR
OF
TH
E Q
UE
STIO
N,
JUST
ASK
. OT
HE
RW
ISE
PL
EA
SE S
IGN
OFF
AN
D N
OT
IFY
TH
EL
AB
ASS
IST
AN
T
HO
W'S
TH
AT
AG
AIN
? I
DO
N'T
UN
DE
RST
AN
D Y
OU
R R
ESP
ON
SEA
S W
OR
DE
D.
WE
LL
, I'M
LIC
KE
D. G
OO
DB
Y.
JUM
P T
O M
ICR
O.A
JUM
P T
O M
ICR
O.A
JUM
P T
O E
ND
(Not
e : T
he f
ollo
win
g ar
e th
e ex
pand
ed v
ersi
on o
f th
e st
ring
s re
ferr
ed to
by
quot
ed la
bels
)
UN
ITZ
MIC
RO
X
OPA
QX
APE
RX
DIS
SEX
DU
NN
O
GA
RB
T T T T T T T
UN
IT O
R U
NIT
IZE
D M
ICR
OFO
RM
S
TR
AN
SPA
RE
NT
FIL
M S
HE
ET
, OR
MIC
RO
FIC
HE
OPA
QU
E M
ICR
OC
AR
D
APE
RT
UR
E C
AR
DS
DIS
SEM
INA
TIO
N O
F T
EC
HN
ICA
L L
ITE
RA
TU
RE
DO
N'T
KN
OW
, HA
VE
N'T
, LE
AST
, SL
IGH
TE
ST, L
OST
, OU
TO
F IT
, SE
AR
CH
, CO
NFU
SED
, CA
N'T
DE
CID
E, C
AN
'TC
HO
OSE
, CA
N'T
CH
OIC
E, C
AN
'T S
AY
, HA
VE
NE
VE
R,
(etc
.)
(alp
habe
t)
Lab
els
1
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
DU
P
TA
KU
P
POSI
T
NE
GA
T
RPT
PLU
SCA
P
PLU
SCL
AR
NE
GC
LA
R
PLU
SEA
SE
PLU
SEX
PER
-T
IEN
CE
NE
GN
OT
ES
T
PLU
SCH
EA
PT
NE
GL
OSE
I B
FUR
VE
YO
U A
LR
EA
DY
ME
NT
ION
ED
YO
UR
PR
OB
LE
M W
ITH
a D
UPY
» A
ND
I T
RIE
D T
O E
XPL
AIN
. IF
YO
U A
RE
ST
ILL
AT
AL
OSS
, PE
RH
APS
YO
U S
HO
UL
D S
IGN
OFF
AN
D S
EE
K H
EL
P.
WE
WIL
L T
AK
E T
HE
M U
P IN
TH
AT
OR
DE
R.
(For
ms
of a
ffir
mat
ion)
, MO
RE
MO
ST
(For
ms
of n
egat
ion)
, LE
AST
, LE
SS
HE
RE
IS
TH
E O
RIG
INA
L Q
UE
STIO
N A
GA
IN :
« E
XPO
»
CA
PAC
ITY
, NU
MB
ER
, VO
LU
ME
, SPA
CE
, CO
NT
AIN
, HO
LD
CL
AR
ITY
, CL
EA
R, F
INE
, FIN
EN
ESS
, VIE
WIN
G, R
ESO
LU
TIO
N
HA
RD
TO
RE
AD
, PO
OR
RE
SOL
UT
ION
, DIM
, DA
RK
, HU
RT
S E
YE
S,PR
INT
TO
O S
MA
LL
, FU
ZZ
Y
EA
SE O
F D
IST
RIB
UT
ION
, EA
SE O
F M
AIL
ING
, EA
SE O
FSE
ND
ING
EX
PER
IEN
CE
IN
USE
, KN
OW
MO
RE
AB
OU
T
HA
RD
NO
TE
S, D
IFFI
CU
LT
NO
TE
S, U
ND
ER
LIN
E, W
RIT
E O
N
CO
ST, C
HE
AP,
IN
EX
PEN
SIV
E, S
AV
E M
ON
EY
TO
O S
MA
LL
, LO
SE, L
OST
, MIS
PLA
CE
, MIS
PLA
CE
D,
LO
OSE
TR
AC
K O
F
Lab
els
NE
GT
RU
B
FIC
HU
P
N.1
N.2
N.3
Cod
e
T T T T
Dis
play
text
and
inpu
t sca
n sp
ecif
icat
ions
Con
trol
sta
tem
ents
TO
O M
UC
H T
RO
UB
LE
, TIM
E C
ON
SUM
ING
MIC
RO
FIC
HE
CA
N B
E D
UPL
ICA
TE
D V
ER
Y R
EA
DIL
Y A
ND
INE
XPE
NSI
VE
LY
, AN
D I
N F
AC
T I
S T
HE
ST
AN
DA
RD
IN
USE
BY
TH
E F
ED
ER
AL
GO
VE
RN
ME
NT
FO
R D
IST
RIB
UT
ION
OF
CL
EA
RIN
GH
OU
SE, D
OD
, NA
SA, A
EC
, AN
D E
RIC
DO
CU
ME
NT
SW
HIC
H I
NC
LU
DE
A L
AR
GE
PO
RT
ION
OF
OU
R T
EC
HN
ICA
L L
ITE
RA
TU
RE
OU
TPU
T. R
EC
IPIE
NT
SC
AN
RE
AD
ILY
USE
TH
EM
IN
A W
IDE
VA
RIE
TY
OF
RE
AD
ER
S, O
R (
UN
LIK
E O
PAQ
UE
CA
RD
S) R
EN
DE
R T
HE
MB
AC
K I
NT
O E
YE
-LE
GIB
LE
FO
RM
TH
RO
UG
H R
EA
DE
R/
PRIN
TE
RS.
WIT
H A
PER
TU
RE
CA
RD
S SE
VE
RA
L U
NIT
SA
RE
RE
QU
IRE
D T
O C
ON
VE
Y A
MU
LT
I-PA
GE
DO
CU
ME
NT
,H
EN
CE
TH
EY
AR
E M
OR
E S
UIT
AB
LE
FO
R S
ING
LE
IT
EM
S,SU
CH
AS
EN
GIN
EE
RIN
G D
RA
WIN
GS.
(A
PER
TU
RE
CA
RD
SA
RE
MO
RE
OFT
EN
USE
D I
N A
ME
CH
AN
IZE
D S
YST
EM
BU
T T
HIS
IN
IT
SEL
F D
OE
S N
OT
MA
KE
TH
EM
LE
SSSU
ITA
BL
E T
HA
N F
ICH
E, O
R M
OR
E S
O.)
1, O
NE
, FIR
ST, M
ICR
OFI
CH
E, T
RA
NSP
AR
EN
T, F
ILM
SHE
ET
2, T
WO
, SE
CO
ND
, OPA
QU
E, M
ICR
OC
AR
D
3, T
HIR
D, L
AST
, APE
RT
UR
E
(Not
e : T
he f
ollo
win
g is
a ty
pica
l « s
atel
lite
» fr
ame,
cal
labl
e at
any
poi
nt).
TA
UN
IT M
ICR
OFO
RM
IS
ON
E W
HIC
H C
ON
TA
INS
A S
ING
LE
DO
CU
ME
NT
, UN
LIK
E R
OL
L M
ICR
OFO
RM
(M
ICR
OFI
LM
) W
HIC
HM
AY
CO
NT
AIN
SE
VE
RA
L D
OC
UM
EN
TS
ON
ON
E R
OL
L, O
R R
EE
L.
EA
CH
UN
IT M
AY
BE
LA
BFT
T.F
D, I
ND
EX
ED
, ST
OR
ED
, CO
PIE
D,
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
OR
. TR
AN
SFE
RR
ED
TO
A U
SER
IN
DE
PEN
DE
NT
LY
OF
AN
Y O
TH
ER
DO
CU
ME
NT
. SO
UN
IT M
ICR
OFO
RM
IS
SPE
CIA
LL
Y W
EL
LSU
ITE
D F
OR
DIS
SEM
INA
TIO
N O
F T
EC
HN
ICA
L L
ITE
RA
TU
RE
WH
ICH
IS
USU
AL
LY
PU
BL
ISH
ED
IN
TH
E F
OR
M O
F JO
UR
NA
LA
RT
ICT
PS
OR
DO
CU
ME
NT
S O
F SI
MIL
AR
LE
NG
TH
. (U
SUA
LL
YSE
PAR
AT
E D
OC
UM
EN
TS)
.D
OE
S T
HIS
EX
PLA
IN T
HE
TE
RM
AD
EQ
UA
TE
LY
FO
R Y
OU
?A
DD
-1
TO
PR
OB
Y, A
DD
-10
TO
UN
ITY
UN
IT.A
A(s
tude
nt in
put)
R«
POSI
T »
IF P
RO
BY
JU
MP
TO
MO
RE
DU
N
GFI
NE
. IF
YO
U A
RE
IN
DO
UB
T A
BO
UT
AN
Y O
F T
HE
OT
HE
R T
ER
MS,
DO
N'T
HE
SIT
AT
E T
O A
SK. «
RPT
»JU
MP
RE
TO
UR
MO
RE
DU
NG
IN T
HA
T C
ASE
WE
WIL
L P
ASS
TO
TH
E M
EA
NIN
G O
FIF
MIC
RO
Y =
I U
SE M
ICR
OX
,IF
MIC
RO
Y J
UM
P R
ET
OU
R,
IF O
PAQ
Y =
1 U
SE O
PAQ
X,
IF O
PAQ
Y J
UM
P R
ET
OU
R,
IF A
PER
Y =
I U
SE A
PER
X,
IF A
PER
Y J
UM
P R
ET
OU
RIF
DIS
SEY
= I
USE
DIS
SEX
,IF
DIS
SEY
JU
MP
RE
TO
UR
R«
NE
GA
T »
GR
AT
HE
R T
HA
N S
PEN
D M
OR
E C
AI
TIM
E O
N I
T N
OW
, ISU
GG
EST
TH
AT
YO
U E
ITH
ER
SIG
N O
FF A
ND
SE
EK
OU
TA
N I
NST
RU
CT
OR
, OR
TY
PE 0
PA
SS
» A
ND
I W
ILL
GO
ON
TO
TH
E N
EX
T Q
UE
STIO
N.
JUM
PT T
O U
NIT
.A
Lab
els
Cod
eD
ispl
ay te
xt a
nd in
put s
can
spec
ific
atio
nsC
ontr
ol s
tate
men
ts
RE
TO
UR
R G R G F E
PASS
AL
L R
IGH
T. B
UT
PL
EA
SE G
ET
CL
AR
IFIC
AT
ION
AT
TH
EFI
RST
OPP
OR
TU
NIT
Y.
SIG
N O
FF, Q
UIT
YO
U A
RE
NO
W S
IGN
ED
OFF
BE
G P
AR
DO
N ?
(ter
min
ates
cal
l, re
turn
s to
poi
nt w
henc
e it
cam
e).
JUM
P T
O N
EX
TFR
AM
E
JUM
P T
O E
ND
JUM
P T
O U
NIT
.A
NE
XT
FRA
ME
EN
D
STUDENT FEEDBACK IN CAI
APPENDIX D
REACTION TEXT DISPLAYS - K A-FRAME
259
1. List attached.z. This Appendix lists the textual responses programmed in the« A-Frame », that is, the responses which are called forth fromthe computer as part of its reaction to student input. Romannumerals indicate « pass status » as follows :
I. Responses which can occur on the first pass (i.e., in answer to a student'sfirst input), or on subsequent passes.
II. Responses which can occur only on the second or subsequent passes(depending on certain conditions being set in an earlier pass).
III. Responses occurring on the second or subsequent pass if a satellite hasbeen called.
All student answers are classified ALPM (legal, perceived, matched)except numbers I2, 27, z8, and 31, which are F (failed), beingeither AI (illegal), ALU (legal but unperceived), or AuF (legal,perceived, but intentionally failed).3. Types of student answer are indicative only. For example, eachtype or subtype listed represents hundreds or even thousands ofdifferent inputs all containing a required key element. For theexact specification of these elements, consult Appendix C. Thedesignations « Right » and « Wrong » indicate the instructor'sviewpoint and have no mechanistic significance in the program.4. For the sake of brevity, many unique responses are left to thereader's imagination, in situations where they would requireonly slight rewording of a response which has been written out.S. None of the text should be interpreted substantively. Weexpect an expert in the field might think of many counter-argu-ments.We are concerned here with dialogue rather than with factualdistinctions, and in regard to the latter defer entirely to copy-righted reference 5 .
6. At the end of this Appendix are included two sample dialogueswhich might occur in an on-line situation, to illustrate the kindof interchange which might take place in the « A-Frame ».
z6o J. C. MEREDITH & D. FERGUSON
I. First pass.Type of answer (numbers and letters Program display reactionestablish categories referred to in (« E » = « exit to next frame »)Appendix F)
z. Right both elements
a. Reason : ease of distribution, You are correct in pickingof mailing, of sending microfiche, and ease of distri-
bution and use is one of itscompelling advantages.(use FICHUP text)FICHUP :Microfiche can be duplicatedvery readily and inexpensively,and in fact is the standard in useby the federal government fordistribution of Clearinghouse,DOD, NASA, AEC, and ERICdocuments which include alarge portion of our technicalliterature output. Recipientscan readily use them in a widevariety of readers, or (unlikeopaque cards) render them backinto eye-legible form throughreader-printers. With aperturecards, several units are requiredto convey a multi-page docu-ment, hence they are moresuitable for single items, suchas a file of engineering draw-ings. (Aperture cards are moreoften used in a mechanizedsystem, but this in. itself doesnot make them more suitablethan fiche, or less suitable). (E)
b. Reason : cost, cheap, inexpens- You are correct in pickingive, save money microfiche, and cost consider-
ations certainly enter into thepicture. Microfiche can be du-plicated very readily and in-expensively (etc. seeFICHUP in La above). (E)
At 1
STUDENT FEEDBACK IN CAI 261
c. Reason : r.a i.b
z. Right first element only
a. No reason givenb. Wrong reason given
experience (« because I'vehad more experience withit »).
(a)
(3)(4)
capacity (« because it hasgreater capacity »).
clarityknowledge (« the only oneI know about »).
3. Wrong Two first elements
4. Wrong wrong first elementa. Microcardb. Aperture card
5. Wrong Don't knowa. Alone
6. Wrong Don't know + uniden-tified element
44
Good. You correctly pickedtwo of the important advan-tages of fiche. (FICHUP) (E)
Why ?
Can you compare them moreon the basis of information ?If you would like to knowmore about the other forms, orabout the general requirementsfor dissemination of technicalliterature, just type« Clarify ».
You are correct in pickingmicrofiche, but the reason it issuperior for dissemination oftechnical literature is not that ithas greater capacity than otherunit forms (which depends onthe reduction ratio in use) buton the fact that microfiche canbe duplicated very readily andinexpensively. (FICHUP) (E)(similar to 2.b.a above) (E)(similar to z.b.z above) (E)
See if you can pick just one, onthe basis of what you can learnfrom me on the subject.
Why ?
Why ?
I'll be glad to explain any of theterms I've used, if you wish.Just type « Clarify ».
z6z j. C. MEREDITH & D. FERGUSON
a. « Don't know because( ?) ».
7. Wrong Don't know + singleidentified
a. Unit, unitized
b. Transparent, film, sheet,fiche, microfiche
c. Opaque, microcardd. Aperturee. Dissemination, technical,
literature
8. Wrong Don't know + multipleidentified
a. Any combination of 7.a, 7.b,7.c, 7.d, and 7.e)
9. Wrong Don't know general
a. Any
I don't understand exactly whatyou need to have clarified.
(First pass only)I understand you need moreinformation about « unit a or« unitized » microforms. A unitmicroform is one which...(etc.) Does this resolve youruncertainty concerning thedefinition of unitized micro-forms ?
(Second pass et seq.) I believeyou already mentioned yourproblem with « unit » or « u-nitized » microforms, and Itried to explain. If you are stillconfused, perhaps you shouldsign off and seek help.(Similar to 7.a)
(Similar to 7.a)
(Similar to 7.a)
(Similar to 7.a)
I understand you need moreinformation about (7.a, 7.b,7.c, 7.d, 7.e). We will take themup in that order. (Stacked sub-routines of 7.a-e)
One of them is superior in re-produceability, cost, generaluse, etc. Do you need moreinformation ? If so, type « Clar-
STUDENT FEEDBACK IN CAI 263
b. No experience
to. Wrong Rejection
a. Dislike
b. Expletive
x. Wrong Request
a. Clarify, define
x z. Fail
I'd like you to choose on thebasis of available information,if possible. Type « Clarify »for explanation of individualelements.
Aside from your personal feel-ings in the matter, can youexpress an opinion as to whichmight be least unsuitable forthe stated purpose ?Really ? I'm surprised you feelso strongly about it. If you'dlike clarification of any elementof the statement about unitmicroforms, or of the question,just ask. Otherwise please signoff and notify the lab assistant.
(Same as 7 and 8)
How's that again ? I don'tunderstand your response asworded. Please elucidate.
II. Second pass
13. Right both elements Not applicable (frame com-pleted on first pass)
14. Right Second Element(first element right)a. Reason : ease of distribution, of (Same as t.a) (E)
mailing of sending
b. Reason : cost, cheap, inex- (Same as t.b) (E)pensive, save money
c. Reason : x.a 1.b (Same as t.c) (E)
15. Right Second Element(first element wrong)a. Microcard 1st element
(i) experience (Similar to z.b.x, a.b.;
4 -r-A-
3
264 J. C. MEREDITH & D. FERGUSON
(z) capacity(3) clarity(4) knowledge
b. Aperture card ist element
i6. Wrong Second Element(1st element right)
17. Wrong Don't know.
18. Wrong Don't know + un-identified
19. Wrong Don't know +a. Single identifiedb. Multiple identified
zo. Wrong Don't know general
r. Wrong Rejectiona. Dislikeb. Expletive
22. Wrong Request
SUBROUTINE FOLLOWING
and 2.b.4, with special wording)(E)
(Same as i5.a) (E)
(ditto)
(Repeat 5)
(Repeat 6)
(Similar to 7)(Similar to 8)
(Repeats 9)
(Repeats io.a)(Repeats 1o.b)
(Same as 7, 8, and i)
23. Righta. « YES »
24. Wronga. «NOu
Fine. If you are in doubt aboutany of the other terms, don'thesitate to ask. Here is the originalquestion again. (repeats question)
Rather than spend more CAI timeon it now, I suggest that youeither sign off and seek out aninstructor, or type < Pass o and Iwill go to the next question. (E)
SUBROUTINE FOLLOWING 8
25. Righta. « YES »
26. Wronga. s NO
In that case we will pass to themeaning of ... (next occurringitem of requested information).
(Same as 24).
46'
STUDENT FEEDBACK IN CAI 265
27. FAIL If *12 unused on 1st (Same as la).pass
z8. FAIL If #iz used on 1st pass (Use rz (admits frame failure;may quote all instructive text)).
III. Third pass(Predictable only if a subroutine was used on the '1st or and pass)
SUBROUTINE FOLLOWING 8 and 9
z9. Wrong Continuea. « PASS a All right. But please get clarifica-
tion at the first opportunity. (E)3o. Wrong EXIT
a. « I quit » You are now signed off. (E)
3t. FAIL *27 & # 28 used (Program falls through to nextframe) (E)
A SHORT DIALOGUE AS ENVISAGED IN THE S< A » VERSION
Instructor (Statement and question)Student I don't know what a microcard is, really.Instructor (Defines microcard) Does this resolve your
uncertainty concerning the definition ofmicrocards ?
Student Yes.Instructor Fine. If you are in doubt about any of the
other terms don't hesitate to ask me. (Repeatsstatement and question).
Student Microfiche, because it is cheaper to reproduceand distribute in quantity.
Instructor You are correct in picking microfiche, andease of distribution and use is one of itscompelling advantages. Microfiche can beduplicated very rapidly and inexpensively ...(etc.)
(Exit to next frame)
4'
266 J. C. MEREDITH (.3c D. FERGUSON
A LONGER DIALOGUE AS ENVISAGED IN THE «A 0 VERSION
Instructor (Statement and question)Student I dislike all three.Instructor Aside from your personal feelings in the
matter, can you express an opinion as to whichmight be the least unsuitable for the statedpurpose ?
Student How can I ? I don't know much about anyof them.
Instructor One of them is superior in reproduceability,general use, cost, etc. Do you need more in-formation ? If so, type « Clarify D
Student O.K. Clarify what is a microcard. Also 3.Instructor I understand you need more information about
opaque microcard aperture cards. We willtake them up in that order. (Defines micro-card) Does this resolve your uncertaintyconcerning microcards ?
Student AffirmativeInstructor In that case we will pass to the meaning of
aperture card : (Defines aperture card). Doesthis resolve your uncertainty concerning aper-ture cards ?
Student YesInstructor Fine. If you are in doubt about any of the
other terms, don't hesitate to ask. Here is theoriginal question again. (Repeats statementand question).
Student The whole question seems unreasonable to me.Instructor How's that ? I don't understand your response
as worded. Please elucidate.Student Well, I'll make a guess. Microcards and aper-
ture cards.Instructor See if you can pick just one, on the basis of
what you can learn from me on the subject.Student Microcards.Instructor Why ?Student Because they're cheaper, I guess.Instructor No, as a matter of fact, they are more expensive
than either of the others. Microfiche is betterfor dissemination of technical literature than
47
(Exit to next frame).
STUDENT FEEDBACK IN CAI 267
either microcard or aperture cards, because(ctc.) ... Microfiche can be duplicated
... (etc. FICHUP).
APPENDIX E
K A » STUDENT ANSWERS AND PROGRAM RESPONSES
This Appendix lists the student answers of the « A » set,together with the reactive text displays which would have beencalled forth in the « A-Frame ». All items are grouped accordingto « sense » from the instructor's viewpoint. The several evidentinconsistencies are discussed in the body of the paper.
STUDENT INPUT REACTION TEXT DISPLAY
Group I : No clue
# 1 I don't know. I'll be glad to explain any of the termsI've used, if you wish. Just type« Clarify »
# 2 I don't know. (same as #1)
# 9 I really haven't the slightest idea. (same as # 1)
Group II : Polite regret, some narrowing of information need
# 3 I don't know enough to decide. I don't understand exactly what youneed to have clarified.
# 4 I don't know what these are exactly,so can't tell.
(same as #3)
# 7 Don't have enough experience tojudge.
I'd like you to choose on the basis ofavailable information, if possible.Type « Clarify » for explana-tion of individual elements.
4g
268 J. C. MEREDITH & D. FERGUSON
STUDENT INPUT REACTION TEXT DISPLAY
410
411
Not enuf info to answer this wouldneed to know properties of micro-forms and kinds/quality of availphotocopy.
I've only had occasion to use themicrofilm on reels, so I can't really say.
How's that again ? I don't understandyour response as worded. Please eluci-date.
(same as 41)
Group III : « I don't know » + attempt
4 5 I don't know. They all seem aboutequivalent to me.
(same as #3)
Group IV : Specific information need
4 6
4 8
412
I don't know what an aperture card is.Nor (1) or (2)
I can't honestly compare because I amnot familiar with all 3 of the media,
Please define opaque microcard andaperture card.
I understand you need more informa-don about aperture card - microfiche -opaque microcard. We will take themup in that order. Aperture cards arenot true unit microforms since they are« assembled », generally from cardstock, adhes'e, and film. They atewell suited to applications such asengineering drawings... (etc.) Doesthis explain the term adequately foryou ?
I understand you need more informa-don about aperture cards. Aperturecards are not true unit rnicroformssince they are « assembled »,... (etc.)Does this explain the term adequatelyfor you ?
I understand you need more informa-don about opaque microcard - aper-ture card. We will take them up in that
49
STUDENT FEEDBACK IN CAI 269
STUDENT INPUT REACTION TEXT DISPLAY
order. Microcard is a registered trade-mark for the form which is also calledthe a microopaque )), and is charac-terized by ... (etc.) Does this explainthe term adequately for you ?
Group V : Solution offered
#17
#18
#19
#20
4P21
4t22
#23
I believe the opaque microcard is mostuseful.
Transparent film sheet; just seems easyto store and use.
Apf tire cards too awkward for a bigfile, easier to enlarge microfiche forconvenient viewing.
I think the microfiche would be bestbecause I think it can contain moreinfo than the other forms.
3. The aperture card can be handledmechanically.
Aperture card, because the citationinfro (sic) is readable without resortingto a reader.
3. Aperture cards because they areprocessed by machine easily.
The opaque microcard is adequate.
The microfiche offers better reproduc-tion (visually) than the microcard.
I would suggest the microfiche, sincecomplete information can be stored ina small space.
Why ?
Why ?
Why ?
Why ?
Why ?
Why ?
Why ?
Why ?
Why ?
Why ?
270 J. C. MEREDITH & D. FERGUSON
STUDENT INPUT REACTION TEXT DISPLAY
*24 #2 is most suitable. As a positivecopy of the text it is least objectionableto the user.
Why ?
Group VI : Other
#16 None. I believe in hard copy. I dislikesitting in dark rooms and looking atidiot boxes.
Aside from your personal feelings inthe matter, can you express an opinionas to which form might be leastunsuitable for the stated purpose ?
APPENDIX F
(< A-FRAME » PERFORMANCE SUMMARY
I. Attached.
z. Explanation of column headings follows :
1) Student .answer number
z) Studenttnswer
3) Sense group
4) Scan fate
5) Answer type
generally in random order, except thatwhen typing the list from the answerforms, « don't know »s were entered first.
except for line breaks, a literal versionin each case.
ordered, I through VI (see Appendix Efor designations).
an identification of the particular scanspecification which is believed would haveachieved match condition, for the studentanswer given. (Note : seeming incon-sistencies can be resolved by considering(a) sequence of scans, together with (b)sequence of elements in the answers).
classification of answers according tocategories discussed on page 6.
STUDENT FEEDBACK IN CAI 271
6) First pass program reaction refers to item numbers in Appendix D.
7) Sense score an evaluation of performance (o-t) basedon the following criteria : Does the firstpass reaction make sense, and is the pros-pect of sensible second-pass reactiongood ?
8) Motivation rating assigned this is a rating intuitively assigned by theby the instructor instructor, based on the content and
overtones of the student's answer :Ma = no apparent motivationMx = polite regret, some motivationMz = fair-to-good motivationM3 = weighting factor for Group V
answers, in which good to excel-lent motivation can be assumedregardless of wording.
9) Revision required(I) + scan
(z) M-Group
decision to improve scan specification, orto scan for additional items and ideas.answers which it is expected might bebetter handled through motivationalclustering. (See pages 237-2.38)
3. Initially we tried to distinguish between sense from the programviewpoint and sense from the student's viewpoint. This raisedproblems of assigning credit for fortuitous first-pass performance,and in the assessment of partial relevance, so it was decided tocombine the two on an empirical basis. We consider that if theframe performs meaningfully for the student it is immaterialwhether or not the operation was planned in exactly the wayit takes place, and whether or not all of the expository materialone might have liked to call forth in a given situation actuallyappears. Credit is taken for situations where slight discrepanciesin meaning stand a very good chance of being corrected in thenext pass.
5 24
A-F
ram
e Pe
rfor
man
ce S
umm
ary
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Num
ber
Ans
wer
Sens
eG
roup
Scan
Fate
Ans
wer
Typ
e
1st P
ass
Sens
ePr
ogra
mq
Rea
ctio
nco
re
Mot
iva-
lion
Rat
ing
Rev
isio
n re
quir
ed
+SC
AN
M-G
roup
# 1
I do
n't k
now
.I
Dun
noA
LPM
51
0x
# 2
I do
n't k
now
.I
Dun
noA
Lpm
5I
0x
1# 9
I re
ally
hav
en't
the
slig
htes
t ide
a.I
Dun
noA
Lpm
51
0x
#11
I've
only
had
occ
asio
n to
use
the
mic
rofi
lmon
ree
ls, s
o I
can'
t rea
lly s
ay.
Iln
Dun
noA
LU
51/
22
x
# 3
I do
n't k
now
eno
ugh
to d
ecid
e.II
Dun
no +
Gar
bA
LPM
61/
21
x#
4I
don'
t kno
w w
hat t
hese
are
exa
ctly
so
can'
t tel
l.Il
nD
unno
+ G
arb
AL
pm6
1/2
1x
# 7
Don
't ha
ve e
noug
h ex
peri
ence
to ju
dge.
IID
unno
+ G
arb
AL
pm9b
1
#10
Not
enu
f in
to to
ans
wer
this
wou
ld n
eed
to k
now
pro
pert
ies
of m
icro
form
s an
d ki
nds
/qu
ality
of
avai
l pho
toco
py
Iln
Fail
# 1
AL
U12
01
xx
# 5
I do
n't k
now
. The
y al
l see
m a
bout
equi
vale
nt to
me.
III
Dun
no +
Gar
bA
LU
61/
22
xx
#12
Plea
se d
efin
e op
aque
mic
roca
rd a
ndap
ertu
re c
ard.
IVD
unno
+ o
paqu
eA
LPM
7b1
3
# 6
In'
t kno
w w
hat a
n ap
ertu
re z
ard
is.
IVD
unno
+ap
ert.
AL
p m
7d1
3N
or (
1) o
r (2
)#
8I
can'
t hon
estly
com
pare
bec
ause
I a
m n
otfa
mili
ar w
ith a
ll 3
of th
e m
edia
.IV
Dun
no +
aper
t.A
LU
7d0
2x
x
#13
I be
lieve
the
opaq
ue m
icro
card
is m
ost
usef
iil.
VW
hy 2
AL
PM2a
13
#14
Tra
nspa
rent
film
she
et; j
ust s
eem
s ea
syto
sto
re a
nd u
se.
VW
hy 1
AL
U2a
03
x
#17
#18
#19
#20
#21
4t22
#23
#24
#25
#15
#16
I th
ink
the
mic
rofi
che
wou
ld b
e be
stbe
caus
e I
thin
k it
can
cont
ain
mor
e in
foth
an th
e ot
her
form
s.3.
The
ape
rtur
e ca
rd c
an b
e ha
ndle
dm
echa
nica
lly.
Ape
rtur
e ca
rd, b
ecau
se th
e ci
tatio
n in
fro
(sic
)is
rea
dabl
e w
ithou
t res
ortin
g to
a r
eade
r.3.
Ape
rtur
e ca
rds
beca
use
they
are
proc
esse
d by
mac
hine
eas
ily.
The
opa
que
mic
roca
rd is
ade
quat
e.T
he m
icro
fich
e of
fers
bet
ter
repr
oduc
tion
(vis
ually
) th
an th
e m
icro
card
.I
wou
ld s
ugge
st th
e m
icro
fich
e, s
ince
com
plet
e in
form
atio
n ca
n be
sto
red
ina
smal
l spa
ce.
#2 is
mos
t sui
tabl
e. A
s a
posi
tive
copy
of th
e te
xt it
is le
ast o
bjec
tiona
ble
to th
e us
er.
The
mic
rofi
che
has
prov
ed q
uite
use
ful f
orca
rtog
raph
ic in
form
atio
n.A
pert
ure
card
s to
o aw
kwar
d fo
r a
big
file
,ea
sier
to e
nlar
ge m
icro
fich
e fo
r co
nven
ient
view
ing.
Non
e. I
bel
ieve
in h
ard
copy
. I d
islik
e si
tting
in d
ark
room
s an
d lo
okin
g at
idio
t box
es.
RE
CA
PIT
UL
AT
ION
:
Ove
rall
mat
ch s
core
24 (
96 %
)Se
nse
Scor
e12
(48
%)
Mot
ivat
ion
0(4
)W
eigh
t :0
Mot
ivat
ion
1(4
)4
Mot
ivat
ion
2(3
)6
Mot
ivat
ion
3(1
4)42
Mot
ivat
ion
com
posi
te52
Mot
ivat
ion
poss
ible
max
.75
Mot
ivat
ion
fact
or52
/75
VW
hy 1
AL
T T
2a0
3x
VW
hy 3
AL
U4b
03
x
VW
hy 3
AL
U4b
03
x
VW
hy 3
AL
U4b
03
x
VW
hy 2
AL
P M
4a1
3V
Why
1A
LU
2a0
3
VW
hy 1
AL
U2a
03
x
VW
hy 2
AL
U4a
03
x
VW
hy 1
AL
U2a
13
VW
hy 3
AL
U4b
03
x (s
peci
al)
VI
RE
JEC
TA
Lp
m10
a1
0
Gro
up V
res
pons
es12
(48
%)
All
othe
rs13
(52
%)
Scan
impr
ovem
ent r
equi
red
13 (
52 %
)fo
r «
B-F
ram
e »
274 J. C. MEREDITH & D. FERGUSON
APPENDIX G
B-FRAME » FLOWCHART
I. Attached.
z. The attached flowchart reflects program changes incorporatedin the « B-Frame >+. Appendix B notes apply (p. 246-248), exceptas provided below :
5) (revised) The group of operations to the left of « DUNNO » is devotedto the provision of information intended to help the student solve theproblem. If no specific information need is detected, his response istested for motivational indicators (Mz, Mi, & Ma) in three separatescans whose specifications are given in Appendix H. The Q-returndisplay for each of these differs from the other two in the way it ap-proaches the problem. Thc. general « I don't know » + GARB, and« I don't know » scans are retained in order to catch all such answersthat escape earlier match.
9) (new) Satellites are increased from five to six in number, and a slightstructural modification is provided : Instead of a single block of textending with « Does this explain the term adequately for you ? » adifferently worded ending is used for students who have been assessed« MI » and are first being reinforced on something they may have someknowledge of : «I hope this confirms your understanding of ... (arWhat would you like to follow up next ? »
55
r
00
00
CA
LL
VS
SC
L
T.. I
Tos N
IKE
Mto
kr
GA
LLA
l KR
...
5AT
EL
LiT
E "
UN
IT 2
:
Ti, C
ALL
0,4X 0
Y.I MC
RBIT lilt'
eXt
Der
enc
CA
L
To
T -
00
eAL
FA
L A
L'
CA
LLio
di X
"
4111
$*41
1* Ter
Q (
DT
ftss
l 4.11
1Itz
. ti
0
Ves
FA
:L. I
(L e
Lx.
)
B-F
ram
e Fl
owch
art
Y. 0
4.*
Wit
larL
AA
Yes
00
In 0
276 J. C. MEREDITH & D. FERGUSON
Yn
B-Frame Flowchart (Continuation)
5?
STUDENT FEEDBACK IN CAI
APPENDIX H
THE (< B-FRAME )) PROGRAM
277
I. Attached.
2. The notes to Appendix C apply (p. 248-249) except as providedbelow :
8) (omitted)
9) (new) A word partially underlined, in a scan specification, means thatonly the underlined portion is stipulated as a match requirement. Thecharacters which are not underlined are included only to show thereader what term group the programmer has in mind. For example« STORAGE » will accept « store », « stored », «storing», and «sto-rage» but not «stop». It will also accept the name of the bird, but theprogrammer is willing to take the risk.
3. It should not be imagined that a CAI programmer wouldoften invest in a single frame the amount of work which hasclearly gone into both the A-Frame and the B-Frame. As a stand-alone unit we had to provide it with features which normallywould need to be written only once far an entire course, e.g.,the « POSIT » and « NEGAT » expansions, and the M-seriesof scan specifications. Also it should be mentioned that thecommand statements can be put together quite rapidly after onehas worked with a language for a while. As a matter of record,one person designed and wrote the « A-Frame » in a little overfour hours.
58
<4
B »
Sou
rce
Prog
ram
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
EX
POT
(Ini
tial s
tate
men
t and
que
stio
n)
MIC
RO
.AA
(stu
dent
inpu
t)If
PU
TA
AD
D D
ESC
RIB
E T
O«
MIC
RO
.A »
, CL
EA
R P
UT
A
R(«
DU
NN
O s
, CL
AR
IFY
, IN
FOR
MA
TIO
N, D
ESC
RIB
E)
+T
AL
LY
(U
NIT
Y, P
RO
BY
), I
F(U
NIT
, UN
ITS,
UN
ITIZ
ED
UN
ITY
2 A
DD
UN
ITX
TO
..>...
DU
PY
R(«
DU
NN
O »
CL
AR
IFY
, IN
FOR
MA
TIO
N, D
ESC
RIB
E)
+T
AL
LY
( M
ICR
OY
, PR
OB
Y),
IF
(TR
AN
SPA
RE
NT
, FIL
M, S
HE
ET
, FIC
HE
, MIC
RO
FIC
HE
,FI
LM
S, «
N.1
»)
MIC
RO
Y >
2 A
DD
MIC
RO
X T
OD
UPY
R(«
DU
NN
O »
CL
AR
IFY
, IN
FOR
MA
TIO
N, D
ESC
RIB
E)
+T
AL
LY
(O
PAQ
Y, P
RO
BY
), I
F(O
PAQ
UE
, AIC
RO
CA
RD
, «N
.2 »
OPA
QY
_>
... 2
AD
D O
PAQ
X T
O D
UPY
R(«
DU
NN
O »
, CL
AR
IFY
, IN
FOR
MA
TIO
N, D
ESC
RIB
E)
+T
AL
LY
(A
PER
Y, P
RO
BY
), I
F(A
PER
TU
RE
, «N
.3 »
)A
PER
Y >
2 A
DD
APE
RX
TO
DU
PY
R(«
DU
NN
O »
, CL
AR
IFY
, IN
FOR
MA
TIO
N, D
ESC
RIB
E)
±T
AL
LY
(D
ISSE
Y, P
RO
BY
), I
F(D
ISSE
MIN
AT
ION
, TE
CH
NIC
AL
, LIT
ER
AT
UR
E)
DIS
SEY
2 A
DD
DIS
SEX
TO
_.-.
DU
PY; I
F (U
NIT
Y +
MIC
RO
Y ±
OPA
QY
+ A
PER
Y +
DIS
SEY
= 2
USE
DU
P, A
DD
« U
NIT
X »
TO
PR
OB
ON
UN
ITY
, AD
D «
OPA
QX
P T
OPR
OB
ON
OPA
QY
, AD
D «
APE
RX
»T
O P
RO
B O
N A
PER
Y, A
DD
« D
ISSE
X »
TO
PR
OB
ON
DIS
SEY
Lab
elC
ode
G R G R G R G R G R G
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
I U
ND
ER
STA
ND
YO
U N
EE
D M
OR
E I
NFO
RM
AT
ION
AB
OU
T «
PR
OB
»
« D
UN
NO
» +
(R
EE
L, R
EE
LS,
RO
LL
, RO
LL
S)
AS
TO
RE
EL
MIC
RO
FOR
M V
S. S
HE
ET
MIC
RO
FOR
MS
«M2
»
IN O
RD
ER
TO
EN
AB
LE
YO
U T
O D
ISC
RIM
INA
TE
... (
etc.
)...
TY
PE «
IN
FOR
MA
TIO
N
« M
I »
WE
LL
, SU
PPO
SE W
E S
TA
RT
BY
EX
AM
ININ
G...
(et
c.)
...A
RE
YO
U A
CQ
UA
INT
ED
WIT
H A
NY
OF
TH
ESE
?
« M
O »
I R
EA
LIZ
E I
T I
S A
LM
OST
IM
POSS
IBL
E...
(et
c.)
AR
E Y
OU
WIL
LIN
G T
O W
OR
K W
ITH
ME
TO
WA
RD
AC
OR
RE
CT
SO
LU
TIO
N ?
«DU
NN
O »
(A
LL
, AN
Y)
ON
E O
F T
HE
M I
S SU
PER
IOR
IN
RE
PRO
DU
CE
AB
ILIT
Y,
CO
ST, G
EN
ER
AL
USE
, ET
C. D
O Y
OU
NE
ED
MO
RE
INFO
RM
AT
ION
? I
F SO
, TY
PE «
CL
AR
IFY
a
IF P
RO
BY
> 2
USE
TA
KU
P,C
AL
L U
NIT
Z O
N U
NIT
Y, C
AL
LM
ICR
OZ
ON
MIC
RO
Y, C
AL
L O
PAQ
ZO
N O
PAQ
Y, C
AL
L A
PER
Z O
NA
PER
Y, C
AL
L D
ISSE
Z O
N D
ISSE
Y,
JUM
P M
ICR
O.A
USE
RE
EL
Z, J
UM
P T
O M
ICR
O.A
MA
RK
(M
OT
C, P
UT
A),
JU
MP
TO
MIC
RO
.A
MA
RK
(M
OT
B, P
UT
A),
JU
MP
TO
MIC
RO
.A
MA
RK
MO
TA
, JU
MP
MIC
RO
.A
JUM
P T
O M
ICR
O.A
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
R«
DU
NN
O »
« G
AR
B »
GI
DO
N'T
UN
DE
RST
AN
D W
HA
T Y
OU
NE
ED
TO
HA
VE
JUM
P T
O M
ICR
O.A
CL
AR
IFIE
D.
RN
OT
CO
MPE
TE
NT
, IN
CO
MPE
TE
NT
, IL
L E
QU
IPPE
D, N
OT
PRE
PAR
ED
, NO
T E
QU
IPPE
D, I
LL
PR
EPA
RE
D
GI'L
L B
E G
LA
D T
O E
XPL
AIN
AN
Y O
F T
HE
TE
RM
S I'V
EJU
MP
TO
MIC
RO
.AU
SED
, IF
YO
U W
ISH
. JU
ST T
YPE
« C
LA
RIF
YD
RPO
SIT
IVE
, NE
GA
TIV
E
GM
ICR
OC
AR
D I
S A
LW
AY
S D
IST
RIB
UT
ED
AS
A P
OSI
TIV
E :
JUM
P T
O M
ICR
O.A
MIC
RO
FIC
HE
S A
ND
APE
RT
UR
E C
AR
DS
MA
Y B
E H
AD
IN
EIT
HE
R P
OSI
TIV
E O
R N
EG
AT
IVE
VE
RSI
ON
S. A
SID
EFR
OM
TH
IS C
ON
SID
ER
AT
ION
, WH
AT
WO
UL
D B
E Y
OU
RC
HO
ICE
?
R(«
N.I
», «
N.2
», «
N.3
»)
-1-
(NO
T, I
NSU
FFIC
IEN
TL
Y,
INA
DE
QU
AT
E, T
OO
, OV
ER
, OV
ER
-, U
NA
CC
EPT
AB
LE
,E
XC
ESS
IVE
, UN
DU
E, U
ND
UL
Y)
+ $
, (no
t inv
erte
d)(N
ote
: $ s
igni
fies
lite
ral)
GPL
EA
SE R
EPH
RA
SE Y
OU
R R
EPL
Y, S
TA
TIN
G T
HE
PR
E-
JUM
P T
O M
ICR
O.A
FER
RE
D T
YPE
FIR
ST
R«N
.1 »
, « N
.2 »
, « N
.3 »
(re
quir
e 2,
any
ord
er)
RA
LL
, AN
Y
GSE
E I
F Y
OU
CA
N P
ICK
JU
ST O
NE
, ON
TH
E B
ASI
S O
FJU
MP
TO
MIC
RO
.AW
HA
T Y
OU
CA
N L
EA
RN
FR
OM
ME
ON
TH
E S
UB
JEC
T.
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
MIC
RO
CA
PR
« N
.1 »
« P
LU
SCA
P »,
« P
LU
SCA
P »
« N
.1 »
MIC
RO
CA
PG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, BU
T T
HE
JUM
P T
O N
EX
TFR
AM
ER
EA
SON
IT
IS
SUPE
RIO
R F
OR
TH
E D
ISSE
MIN
AT
ION
OF
TE
CH
NIC
AL
LIT
ER
AT
UR
E I
S N
OT
TH
AT
IT
HA
S G
RE
AT
ER
CA
PAC
ITY
(W
HIC
H D
EPE
ND
S O
N T
HE
RE
DU
CT
ION
RA
TIO
(USE
D)
BU
T O
N T
HE
FA
CT
TH
AT
e F
ICH
UP
a
MIC
RO
EA
SER
«N.1
» «
PLU
SEA
SE »
, « P
LU
SEA
SE »
« N
.1 »
MIC
RO
EA
SEG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, AN
D E
ASE
JUM
P T
O N
EX
TFR
AM
EO
F D
IST
RIB
UT
ION
AN
D U
SE C
ER
TA
INL
Y D
OE
S FA
VO
R I
TA
S A
ME
DIU
M F
OR
TH
E D
ISSE
MIN
AT
ION
OF
TE
CH
NIC
AL
LIT
ER
AT
UR
E. «
FIC
HU
P »
R«
N.I
» «
PL
USC
HE
AP
», «
PL
USC
HE
AP
» «
N.1
MIC
RO
CH
EA
PG
YO
U A
RE
CO
RR
EC
T I
N P
ICK
ING
MIC
RO
FIC
HE
, AN
D C
OST
JUM
P T
O N
EX
TFR
AM
E
R
CO
NSI
DE
RA
TIO
NS
AR
E A
N I
MPO
RT
AN
T F
AC
TO
R. «
FIC
HU
P »
e N
.1 »
« P
LU
SEA
SE a
« P
LU
SC.H
EA
P a
(any
ord
er)
GG
OO
D. Y
OU
CO
RR
EC
TL
Y P
ICK
ED
TW
O O
F T
HE
IM
POR
TA
NT
JUM
P T
O N
EX
TFR
AM
EA
DV
AN
TA
GE
S O
F M
ICR
OFI
CH
E. «
FIC
HU
P a
R«N
.1 »
«PL
USC
IT a
(JU
MP
TO
NE
XT
FRA
ME
OR
TO
R«
N.1
» «
PL
USC
OP
»M
ICR
O.A
, DE
PEN
DIN
G O
NR
N.1
» o
PL
USC
ON
V »
(sim
ilar
disc
ussi
on o
f ty
pes
and
thei
r as
pect
s)SE
NSE
)R R
« N
.2 o
« P
LU
SCA
P »
o N
.2 »
a P
LU
SCH
EA
P a
etc.
R«
PLU
SEX
ER
»
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
GC
AN
YO
U C
OM
PAR
E T
HE
M, N
OT
ON
TH
E B
ASI
S O
FJU
MP
TO
MIC
RO
.AE
XPE
RIE
NC
E, B
UT
MO
RE
ON
TH
E B
ASI
S O
F IN
FOR
-M
AT
ION
? IF
YO
U W
OU
LD
LIK
E T
O K
NO
W M
OR
E A
BO
UT
TH
E O
TH
ER
FO
RM
S, O
R A
BO
UT
TH
E G
EN
ER
AL
RE
QU
IRE
-M
EN
TS
FOR
DIS
SEM
INA
TIO
N O
F T
EC
HN
ICA
L L
ITE
R-
AT
UR
E, J
UST
TY
PE «
CL
AR
IFY
a
RR
EJE
CT
, HA
TE
, HA
TE
D, D
ISL
IKE
, « N
EG
AT
» L
IKE
D,
«NE
GA
T »
LIK
E, «
NE
GA
T a
CA
RE
FO
R
GA
SID
E F
RO
M Y
OU
R P
ER
SON
AL
FE
EL
ING
S IN
TH
E M
AT
TE
RJU
MP
MIC
RO
.AC
AN
YO
U E
XPR
ESS
AN
OPI
NIO
N A
S T
O W
HIC
H M
IGH
TB
E L
EA
ST U
NSU
ITA
BL
E F
OR
TH
E S
TA
TE
D P
UR
POSE
?
R«
POSI
T »
IF M
OT
A M
AR
K P
UT
A,
IF M
OT
A J
UM
P M
OT
AA
GW
HIC
H ;
IF M
OT
B M
AR
K P
UT
A,
IF M
OT
B J
UM
P M
ICR
O.A
MO
TA
AG
WH
ICH
ON
E W
OU
LD
YO
U L
IKE
ME
TO
DE
SCR
IBE
FIR
ST ?
TY
PE «
DE
SCR
IBE
aJU
MP
MIC
RO
.A
MO
TB
BG
SHA
LL
WE
TA
KE
TH
EM
UP
ON
E A
T A
TIM
E ?
SU
PPO
SEW
E S
TA
RT
WIT
H T
HE
PR
OB
LE
M O
F D
ISSE
MIN
AT
ION
OF
TE
CH
NIC
AL
LIT
ER
AT
UR
E. F
OL
LO
WIN
G T
HA
T, Y
OU
CA
NA
SK F
OR
IN
FOR
MA
TIO
N O
N O
TH
ER
EL
EM
EN
TS
OF
TH
EPR
OB
LE
M.
R«
NE
GA
T »
IF M
OT
B U
SE M
OT
BB
,C
AL
L D
ISSE
Z, J
UM
P M
ICR
O.A
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
GPE
RH
APS
YO
U S
HO
UL
D S
IGN
OFF
, TH
EN
.JU
MP
TO
SIG
NO
FF
R«
N.I
»
GW
HY
?T
AL
LY
(A
NSY
, MIC
RO
B),
JUM
P M
ICR
O.A
R«
N.2
»
GW
HY
?T
AL
LY
(A
NSY
, OPA
QB
),JU
MP
MIC
RO
.A
R«
N.3
D
GW
HY
?T
AL
LY
(A
NSY
, APE
RB
),J'
JMP
MIC
RO
.A
Re
PLU
SCA
P »,
« N
EG
AT
» e
NE
GC
AP
oIF
MIC
RO
B J
UM
P M
ICR
OC
AP,
IF O
PAQ
B J
UM
P O
PAQ
CA
P,IF
APE
RB
JU
MP
APE
RC
AP
R«
PLU
SCO
NV
», «
NE
GA
T »
e N
EG
CO
NV
aIF
MIC
RO
B J
UM
P M
ICR
OC
ON
V,
IF O
PAQ
B J
UM
P O
PAQ
CO
NV
,IF
APE
RB
JU
MP
APE
RC
ON
V
R«
PLU
SCIT
»(s
imila
r to
abo
ve)
R«
PLU
SCL
AR
»R
e PL
USC
OP
»R
o PL
USD
IST
»(s
imila
r do
uble
neg
ativ
es)
Re
PLU
SME
CH
»R
« PL
USE
XP
»
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
RN
OT
ES
(sim
ilar
to p
rece
ding
)R
LO
SE, L
OST
, MIS
PLA
CE
D(s
imila
r to
pre
cedi
ng)
GD
ID Y
OU
SPE
CIF
Y O
NE
OF
TH
ET
YPE
S ?
JUM
P T
O M
ICR
O.A
FH
OW
'S T
HA
T A
GA
IN?
I D
ON
'TU
ND
ER
STA
ND
YO
UR
IJU
MP
TO
MIC
RO
.AR
ESP
ON
SE A
S W
OR
DE
D
FW
EL
L, I
'M L
ICK
ED
. GO
OD
BY
.JU
MP
TO
SIG
NO
FFU
NIT
ZT
UN
IT (
OR
UN
ITIZ
ED
) M
ICR
OFO
RM
MIC
RO
XT
MIC
RO
FIC
HE
OPA
QX
TO
PAQ
UE
MIC
RO
CA
RD
APE
RX
TA
PER
TU
RE
CA
RD
DIS
SEX
TD
ISSE
MIN
AT
ION
OF
TE
CH
NIC
AL
LIT
ER
AT
UR
E
DU
PT
I B
FLIF
VE
YO
U A
LR
EA
DY
ASK
ED
AB
OU
T «
DU
PY »
, AN
DI
TR
IED
TO
EX
PLA
IN. I
F Y
OU
AR
EST
ILL
AT
A L
OSS
,PE
RH
APS
YO
U S
HO
UL
D S
IGN
OFF
AN
D S
EE
Y: H
EL
P.T
AK
UP
TW
E W
ILL
TA
KE
TH
EM
UP
IN T
HA
TO
RD
ER
.R
PTT
HE
RE
IS
TH
E O
RIG
INA
LQ
UE
STIO
N A
GA
IN :
« E
XPO
aM
0T
« A
l », (
« A
2 »
+ «
BI
» +
« C
I ».
!
MI
T(«
Al »
, «A
2 »)
+ «
D3
», «
El »
+ U
NL
ESS
+ (
« C
2 »,
«A
3 »)
M2
T«D
I »
+ «
C2
», «
D2
»A
lT
DO
N'T
KN
OW
, CA
N'T
SA
Y, C
AN
'TIM
AG
INE
, YO
U E
XPE
CT
,Y
OU
TH
INK
, YO
U I
MA
GIN
E, A
TSE
A, W
ITH
IT
, SE
AR
CH
ME
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
A2
TD
ON
'T H
AV
E, N
OT
HA
VE
, HA
VE
N'T
, EX
PEC
T H
AV
ET
HIN
K H
AV
E
A3
TA
BSE
NC
E, W
ITH
OU
T, L
AC
KIN
G, E
VE
R, N
EV
ER
, EN
OU
GH
,E
NT
JF, S
UFF
ICIE
NT
, IN
SUFF
ICIE
NT
, IN
AD
EQ
UA
TE
B1
TL
EA
ST, V
AG
UE
. SL
IGH
TE
ST, F
UZ
ZIE
ST, F
OG
GIE
ST,
RE
MO
TE
ST
Cl
TID
EA
, NO
TIO
N, I
NT
ER
EST
, CO
NC
EPT
, IN
KL
ING
C2
TE
XPL
AN
AT
ION
, DE
FIN
ITIO
N, D
EFI
NE
, CL
AR
IFIC
AT
ION
EL
UC
IDA
TIO
N, E
XPL
ICA
TIO
N, I
NFO
RM
AT
ION
DI
TN
EE
D, 1
1EQ
UIR
E, H
EL
P
D2
TD
EFI
NE
, FU
RN
ISH
, GIV
E, E
XPL
AIN
, TE
LL
, SU
PPL
Y,
AC
QU
AIN
T, E
LU
CID
AT
E, E
XPL
ICA
TE
D3
TK
NO
WL
ED
GE
, FA
CT
S, I
NFO
RM
AT
ION
, DA
TA
, EX
PER
TIS
E,
INC
OM
PET
EN
CE
El
TN
OT
AB
LE
, UN
AB
LE
, CA
N'T
, CA
NN
OT
, CA
N N
OT
DU
NN
OT
DO
N'T
KN
OW
, CO
NFU
SED
, CA
N'T
DE
CID
E, C
AN
'T C
HO
OSE
,U
ND
ER
STA
ND
, UN
AC
QU
AIN
TE
D, M
AK
E C
HO
ICE
, NO
T S
UR
E,
UN
SUR
E, D
ON
'T U
ND
ER
STA
ND
, WH
AT
GA
RB
T(a
lpha
bet)
POSI
TT
(for
ms
of a
ffir
mat
ion,
exc
ludi
ng w
ord
« PO
SIT
IVE
»)
NE
GA
TT
(for
ms
of n
egat
ion,
exd
udia
g w
ord
«NE
GA
TIV
E »
)
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
PLU
SCA
PT
CA
PAC
ITY
, SIZ
E, V
OL
UM
E, S
PAC
E, C
ON
TA
IN, H
OL
D, H
OL
DS,
HE
LD
, MO
RE
IN
FOR
MA
TIO
N, S
TO
RE
, CO
NV
EY
, CO
NV
EY
S
PLU
SCIT
TT
ITL
E, H
EA
DIN
G, C
ITA
TIO
N, M
EN
/11-
ICA
TIO
N, B
IBL
IOG
RA
-PH
ICA
L, R
EFE
RE
NC
ES
PLU
SCL
AR
TC
LE
AR
, VIE
IFIN
G, R
ESO
LU
TIO
N, R
EA
DE
R, D
ISPL
AY
, SC
RE
EN
PLU
SCO
PT
CO
PY, C
OPI
ES,
CO
PYIN
G, B
LO
W U
P, B
LO
W B
AC
K, B
LO
WB
AC
KE
NL
AR
GE
ME
NT
, RE
EN
LA
RG
EM
EN
T
PLU
SDIS
TT
EA
SY (
DIS
TR
IBU
TE
, MA
ILIN
G, S
EN
DIN
G, D
ISSE
MIN
AT
ION
)
PLU
SME
CH
TM
AC
HIN
E, A
UT
OM
AT
ION
, ME
CH
AN
ICA
L
PLU
SEX
PER
TI
HA
VE
USE
, I K
NO
W A
BO
UT
, I E
XPE
RIE
NC
E
PLU
SCH
EA
PT
CH
EA
PER
, CO
ST, C
OST
S, E
CO
NO
MIC
AL
PLU
SCO
NV
TC
ON
VE
NIE
NT
, « N
EG
AT
» D
EL
AY
, QU
ICK
LY
, SPE
ED
, FA
STE
Ra
NE
GA
T »
TR
OU
BL
ESO
ME
PLU
SSO
MP
TR
OO
M, C
OM
PAC
T, S
TO
RA
GE
, SPA
CE
NE
GC
LE
AR
TD
IFFI
CU
LT
RE
AD
, PO
OR
RE
SOL
UT
ION
, BL
UR
RE
D, D
IM, D
AR
K,
FUZ
ZY
, HU
RT
S E
YE
S, I
LL
EG
IBL
E, P
RIN
T S
MA
LL
, SM
AL
L P
RIN
T
NE
GL
OSE
TL
OSE
, LO
ST, M
ISPL
AC
ED
, TO
O S
MA
LL
NE
GC
ON
VT
INC
ON
VE
NIE
NT
, AW
KW
AR
D, T
IME
CO
NSU
MIN
G, D
EL
AY
,T
RO
UB
LE
SOM
E
N.1
T1,
ON
E, F
IRST
, MIC
RO
FIC
HE
, FIC
HE
, TR
AN
SPA
RE
NT
, FIL
MSH
EE
T
,,,N
ripr
. .1*
.rvi
rrtg
7":7
^
6%-
oc)
Lab
elC
ode
Dis
play
text
and
inpu
t ter
m s
peci
fica
tions
Con
trol
sta
tem
ents
N.2
T2,
TW
O, S
EC
ON
D, O
PAQ
UE
, MIC
RO
CA
RD
N.3
T3,
TH
RE
E, «
NE
GA
T »
AL
L (
3, T
HR
EE
), T
HIR
D, L
AST
TR
AN
SFIC
HE
TT
RA
NSP
AR
EN
T (
SAM
E, D
thE
bRE
NT
) FI
CH
E, F
ICH
E (
SAM
E,
DIF
FER
EN
T)
TR
AN
SPA
RE
NT
MO
TS
TI
HO
PE T
HIS
CO
NFI
RM
S Y
OU
R U
ND
ER
STA
ND
ING
OF
« M
ICR
O.A
».
WH
AT
WO
UL
D Y
OU
LIK
E T
O F
OL
LO
W U
P N
EX
T ?
AL
TE
RE
XPO
TW
HA
T I
ME
AN
IS
: OF
TH
E T
HR
EE
CO
MM
ON
EST
TY
PES
OF
NO
N-R
OL
L M
ICR
OFO
RM
, WH
ICH
LE
ND
S IT
SEL
F B
EST
TO
TH
ED
ISSE
MIN
AT
ION
OF
TE
CH
NIC
AL
LIT
ER
AT
UR
E ?
FIC
HE
?M
ICR
O O
PAQ
UE
S ?
APE
RT
UR
E C
AR
DS
? B
EA
R I
N M
IND
CO
ST A
ND
EA
SE O
F M
AK
ING
AN
D D
IST
RIB
UT
ING
CO
PIE
S.
288 . C, MEREDITH HL D. FERGUSON
APPENDIX I
REACTION TEXT DISPLAYS - < B-FRAME >>
I. List attached.
z. Appendix D comments (p. 259) apply, with the exceptionof para. 6 (sample dialogues omitted).
I. First passType of answer (numbers and lettersestablish categories referred to inAppendix K).
1. Right both elementsa. Reason : ease of distribution, of
mailing, of sending.
b. Reason : cost, cheap, inex-pensive, save money.
c. Reason : z.a t.b
2. Right first element only
a. No reason givenb. Wrong reason given
(z) experience (« because hadmore experience with it o).
C99
Program display reaction(«E» = « exit to next frame »)
You are correct in pickingmicrofiche, and ease of distribu-tion and use is one of its com-pelling advantages. Microfichecan be duplicated very readilyand inexpensively... (etc.) ...(same as corresponding« FICHUP » text in AppendixD) (E)You are correct in pickingmicrofiche, and cost considera-tions certainly enter into thepicture. Microfiche can be du-plicated ... (etc., same asabove). (E )
Good. You correctly pickedtwo of the important advan-tages of fiche. Microfiche canbe duplicated... (etc.; same asabove). (E)
Why ?
Can you compare them moreon the basis of information ? Ifyou would like to know moreabout the other forms, or
STUDENT FEEDBACK IN CAI 289
(z) capacity
(3)(4)
(5)
clarityknowledge (« the only oneI know about »)(n) (other aspects providedin the program)
3. Wrong Two first elements
4. Wrong Wrong first element
(a) Microcard
(b) Aperture card
5. Wrong Don't knowa. « don't know » with no addi-
tional element following, andno match on motivational in-dicators.
6. Wrong Don't know + (un-identified matter)
a. « don't know » with addedmaterial, but no match onmotivational indicators
b. Ms
O
about the general requirementfor dissemination of tochnicalliterature, just type « Clarify
You aw correct in pickingmicrofiche, but the reason it issuperior for dissemination oftechnical literature is not that ithas greater capacity (whichdepends on the reduction ratioin use) but on the fact thatmicrofiche can be duplicated... (etc.) (E)(similar to 1.b.z above) (E)(similar to z.b.z above) (E)
(similar to z.b.1 above, exceptthat some will exit from theframe while others will requirerecursion).
See if you can pick just one, onthe basis of what you can learnfrom me on the subject.
Why ?
Why ?
I'll be glad to explain any of theterms I've used, if you wish.Just type « Clarify >>
I don't understand exactlywhat you need to have clarified.
I realize it is almost impossiblefor you to answer withoutknowing something about thetypes of microform mentioned
290 J. C. MEREDITH & D. FERGUSON
c. Mt
7. Wrong Don't know + singleidentified
and about the factors involvedin dissemination of technicalliterattre. I'd like to help youtackle the problem from thebeginning by supplying all theinformation you need. Are youwilling to work with me towarda correct solution ?Well, suppose we start byexamining the items you mayalready have heard about, I maybe able to add some usefulinformation about them. Howabout microfiche ? Microcard ?Aperture card ? The dissemina-tion of technical literature ?Are you acquainted with anyof these ?
a. Unit, unitized I understand you need more in-formation about unit, or uni-tized, microforms. A unit mi-croform is one which... (L tc.,same as corresponding text inAppendix D)
b. Transparent, film, sheet, fiche, (similar to 7.a)microfiche
c. Opaque, microcard (similar to 7.a)
d. Aperture (similar to 7.a)
e. Dissemination, technical (similar to 7.a)literature
8. Wrong Don't know + multipleidentifieda. Any combination of 7.a, 7.b, I understand you need more in-
7.c, 7.d, 7.e formation about (7.a, 7.b, 7.c,7.d, 7.e). We will take them upin that order. (Stacked sub-routines of 7.a-.e)
9. Wrong Don't know + misc-ellaneous
71
STUDENT FEEDBACK IN CAI 291
a. Any, all One of them is superior inreproduceability, cost, generaluse, etc. Do you need moreinformation ? If so, type « Clar-ify » for explanationof individual elements.
b. No experience I'd like you to choose not onthe basis of experience but onthe basis of available informa-don, if possible. Type « Clar-ify » for explanationof individual elements.
c. Positive, negative Microcard is always distributedas a positive; microfiches andaperture cards may be had ineither positive or negativeversions. Aside from this con-sideration, what would be yourchoice ?
d. Not competent, ill equipped, I'll be glad to explain any of thenot prepared, not equipped terms I've used, if you wish.
Just type « Clarify
Jo. Wrong Rejection
a. Dislike
b. Expletive.
Aside from your personal feel-ings in the matter, can youexpress an opinion as to whichmight be least unsuitable for thestated purpose ?
(same as corresponding text inAppendix D. No evidence yetas to how this might work).
I1. Wrong Request
a. Clarify, define, info, describe (same as 7 and 8. The variationbetween request words for Meand Mi is provided to facilitatechanneling of operations calledforth by these terms, and forscoring if and when incor-porated).
1. Fail How's that again ? I don'tunderstand your response as
292 J. C. MEREDITH & D. FERGUSON
IL Seem' pass
13. Right Both elements
14. Right Second Element, firs,:element right
a. Reason : ease of distribution, ofmailing, of sending
b. Reason : cost, cheap, inex-pensive, save money
c. Reason : r.a .b
worded. Please explain morefully.
(not applicable. (Frame com-pleted on first pass)).
(same as i.a) (E)
(same as r.b) (E)
(same as x .c) (E)
15. Right Second Element first element (same as correspondingwrong tions in Appendix D)
x6. Wrong Second Element, first ele-ment right.
17. Wrong Don't know
x8. Wrong Don't know Un-identified
18. a. Wrong Don't know + singleidentified
19. Wrong Don't know + multipleidentified
zo. Wrong Don't know + misc-ellaneous
z1. Wrong Rejection
a. Dislike
zz. Wrong Request
75
(ditto)
Repeats 5.
Repeats 6.
Similar to 7.
Similar to 8.
Repeats 9.
Repeats x o.a
(same as 7, 8 and it)
reac-
STUDENT FEEDBACK IN CAI 293
SUBROUTINE FOLLOWING 7
23. Right
a. « yes
24. Wronga. a. « no »
Fine, if yon are in doubt aboutany of the other terms, don'thesitate to e.sk. Mere is the originalquestion again. (Repeats question).
Rather than spend more CAItime on it now, I suggest that youeither sign off and seek out aninstructor, or type « PASS » andI will go on to the next question.(E)
SUBROUTINE FOLLOWING 8
2 5 . Right
a. « yes » In that case we will pass to themeaning of (next occurring itemof requested information).
z6. Wronga. « no » (same as 24)
SUBROUTINE FOLLOWING 6b (Handled within main frame)
27. Righta. « yes » alone Which one would you like me to
describe first ? Type « Describe
b. « yes » with item, I understand you need more in-or item alone formation about... (item)... (etc.)
z8. Wronga. « no a Perhaps you should sign off, then.
74
294 J. C. MEREDITH & D. FERGUSON
SUBROUTINE FOLLOWING 6c (Handled within main frame)
29. Righta. « yes » alone
b. « yes s with item,or item alone
3o. Wrong.a. «no»
Which ?
« DIJNNO » is automatically in-setted in rtst,nnse, so that pro-gram goes direct to satellite
Shall we take them up one at atime ? Suppose we start with theproblem of dissemination of tech-nical literature. Following that,you can ask for other elements ofthe problem.
31. FAIL If unused on 1st pass
32. FAIL Second instance
III. Third pass
(same as 12, above)
Admit frame failure.
(Predictable only if a subroutineis used on the 1st or znd pass).
SUBROUTINE FOLLOWING 8 and 9
33. Wrong continuea. « PASS »
34. Wrong Exit
a. « I quit »
All right. But please get clari-fication at the first opportunity.(E)
You ate now signed off. (E)
35. FAIL If #31 and #32 havebeen used, program falls throughto the next frame.
STUDENT FeEDBACK IN CAI 295
APPENDIX J
B » STUDENT ANSWERS AND PROGRAM RESPONSES
This Appendix lists the student answers of the « B » set,together with the reactive text which would have been calledforth in the « B-Frame ».
STUDENT INPUT REACTION TEXT DISPLAY
# 1 I don't even know what a micro- I understand you need more informa-fiche is. tion about microfiche - (test of
MICROZ satellite) - Does this explainthe term adequately for you ?
# 2 I don't know. I realize it is almost impossible for youto answer without knowing somethingabout the types of microforms men-tioned and about the factors involvedin dissemination of technical literature.I'd like to help you tackle the problemfrom the beginning by supplying allthe information you need. Are youwilling to work with me toward acorrect solution ?
# 3 I am not familiar with programming. I don't understand what you need tohave clarified.
# 4 I have not had any experience with Can you compare them not on theany, so cannot decide. basis of experience but on the basis of
information ? If you would like toknow more about the general require-ments for dissemination of technicalliterature, just type a Clarify a
# 5 I am not sufficiently acquainted with Well, suppose we start by examiningthe characteristics of these different the items you may already have heardmicroforms to answer. about. I may be able to add some
useful information about them. Howabout microfiche ? Microcard ? Aper-ture card ? The dissemination oftechnical literature? Are you acquaint-ed with any of these ?
'7
296 J. C. MEREDITH & D. FERGUSON
STUDENT INPUT REACTION TEXT DISPLAY
# 6 Lack of knowledge prevents a con-sidered answer.
(Same as #5)
7 What's an aperture card. To tell the I understand you need more informa-truth, 1 strongly dislike microforms ton about apri.cure cards. - (text ofand readers. APERZ satellite) - Does this explain
the term adequately for you ?
# 8 Microfiche, since it is easy to handle,store, and reproduce.
You are correct in picking microfiche,and ease of storing is certainly an ad-vantage, but one which is also sharedby the other two forms. Microfiche canbe duplicated very readily and in-expensively, and in fact is the standardused by the Federal government fordistribution of Clearinghouse, DOD,NASA, AEC, and ERIC documents,which include a large portion of ourtechnical literature output... (etc.)
I don't understand all the distinctions One of them is superior in repro-among the three types. duceability, cost, general use, etc. Do
you need more information ? If so,type « Clarify ---»
#10 1. Because I don't quite know what the See if you can pick just one, on theother two are I basis of what you can learn from me
on the subject.
#11 1. as it contains more information(data) readily & quickly available,
You are correct in picking microfiche,but the reason it is superior for thedissemination of technical literature isnot that it has greater capacity (whichdepends on the reduction ratio in use)but on the fact that microfiche can beduplicated... (etc.) (same as #8)
#12 I want aperture cards. Aperture cardsare fun.
Why ?
'17
STUDENT FEEDBACK IN CAI 297
STUDENT INPUT REACTION TEXT DISPLAY
#13 1 - easy storage (same as # 8)
#14 Fiche. Equipment feasibility, cost, andeventual wide availability.
You are correct in picking microfiche,and cost considerations are an impor-tant factor. Microfiche can be dupli-cated very readily and inexpensively...(etc.) (same as # 8)
# 15 Type 1. Largest amount of experience You are correct in picking microfiche.with this type. Can you compare them not on the
basis of experience but on the basis ofinformation ? (etc.) (same as #4)
#16 It depends on the dissemination systemand the ultimate user(s), but 1 seemsbest.
Why ?
#17 Microfiche, because of Federal stand-ard, cost, and availability ,)f readers.
(same as #14)
#18 1. Density and handling are superiorto 3. copying superior to 2.
(same as #10 FAULT !)
# 19 Microfiche. Large amount of informa-tion stored by unit and a flexible »display.
(same as # 11)
#20 I have used microfiche with a projectorand it seems to be quite satisfactory.
(same as # 10)
I have not used the other two so haveno opinion of their usefulness.
#21 What is opaque microcard ? I understand you need more informa-What is aperture card ? tion on microcard - aperture card. We
will take them up in that order. (Textof OPAQZ satellite) - Does this ex-plain the term adequately for you ?
1 g
298 J. C. MEREDITH & D. FERGUSON
STUDENT INPUT REACTION TEXT DISPLAY
#22 Don't know what aperture card is. I understand you need more informa-What is microform card with all the tion on aperture card - (text of APERZpages on one card called ? satellite) - Does this explain the term
adequately for you ?
#23 What is an aperture card ? ksame as #22)
#24 I don't know what a microcard oraperture card are. I prefer hard copyanyway.
(same as #21)
#25 Microfiche, I guess. Access time in See if you can pick just one, on the basismicrofilm is too long I have never of what you can learn from me on theused opaque microcard. subject.
APPENDIX K
B-FRAME » PERFORMANCE SUMMARY
. Attached.
z. For explanation of column headings, see Appendix F (p. 270-271).
'79
Wif
ieO
RM
A-
71M
mrs
trs1
*71.
4mA
rTrt
rpt-
0--
00
B-F
ram
e Pe
rfor
man
ce S
umm
ary
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
Ans
wer
Scan
Gro
upSc
an F
ate
Ans
wer
Typ
e
1st p
ass
Pg.tn
reac
tion
sens
ese
nse
Scor
e
hlfi
°.11
Rat
ing
2 31
4
* 5
* 6
* 9
* 7
* 1
*21
*23
424
#22
*10
* 8
*11
I do
n't k
now
.I
am n
ot f
amili
ar w
ith p
rogr
amm
ing.
I ha
ve n
ot h
ad a
ny e
xper
ienc
e w
ithan
y, s
o ca
nnot
dec
ide.
I am
not
suf
fici
ently
acq
uain
ted
with
the
char
acte
rist
ics
of th
ese
diff
eren
tm
icro
form
s to
ans
wer
.L
ack
of k
now
ledg
e pr
even
ts a
con
side
red
answ
er.
I do
n't u
nder
stan
d al
l the
dis
tinct
ions
amon
g th
e th
ree
type
s.W
hat's
an
aper
ture
car
d. T
o te
ll th
etr
uth,
I s
tron
gly
disl
ike
rnic
zofo
rms
and
read
ers.
I do
n't e
ven
know
wha
t a m
icro
fich
e is
.W
hat i
s op
aque
mic
roca
rd?
Wha
t is
aper
ture
car
d ?
Wha
t is
an a
pert
ure
card
?I
don'
t kno
w w
hat a
mic
roca
rd o
r ap
ertu
reca
rds
are.
I p
refe
r ha
rd c
opy
anyw
ay.
Don
't kn
ow w
hat a
pert
ure
card
is. W
hat
is m
icro
form
car
d w
ith a
ll th
e pa
ges
onon
e ca
rd c
alle
d ?
1. B
ecau
se I
don
't qu
ite k
now
wha
t the
othe
r tw
o ar
e 1
Mic
rofi
che,
sin
ce it
is e
asy
to h
andl
e,st
ore,
and
rep
rodu
ce.
1. a
s it
cont
ains
mor
e in
form
atio
n (d
ata)
read
ily &
qui
ckly
ava
ilabl
e.
I II II II H II IV IV IV IV IV I" V V V
MO
Dun
no +
Gar
bD
unno
+ E
ver
M1
Ml
Dun
no +
all
Dun
no +
ape
r
Dun
no +
Fic
heD
unno
+ o
paqu
e+
ape
rD
unno
+ a
per
Dun
no +
opa
que
+ a
per
Dun
no +
ape
r
N1
+ N
2
N.1
+ P
lusc
omp
N.1
+ c
ap
AL
PMA
L,p
mA
Lpm
AL
p m
AL
pm
AL
PM
AL
pm
AL
p m
AL
pm
AL
PMA
Lpm
AL
PM
AL
PU
AL
PM
AL
pm
6b 6a ob 6c.
6c 9a 6d 7b 8a 7d 8a 7d 3
2b (
2)
1 1/2 1 1 1 1 1 1 1 1 1 1 1 1 1
0 1 1 1 1 2 1 1 3 3 2 3 2 3 3N 0
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
Num
ber
Gro
upSc
anG
roup
Scan
Fat
eA
nsw
ert
ass
1s p
prog
ra m
reac
tion
Sens
etio
nM
oon
Rat
ing
*12
*25
#13
*14
*15
#16
*17
#18
*20
*19
I w
ant a
pert
ure
card
s. A
pert
ure
card
sar
e fu
n.M
icro
fich
e I
gues
s. A
cces
s tim
e in
mic
rofi
lm is
too
long
. I h
ave
neon
zus
ed o
paqu
e rn
icro
card
.1
Eas
y st
orag
e.Fi
che.
Equ
ipm
ent f
easi
bilit
y, c
ost,
and
even
tual
wid
e av
aila
bilit
y.T
ype
1. L
arge
st a
mou
nt o
f ex
peri
ence
with
this
type
. Rep
rodu
ctio
n an
d en
larg
emen
tqu
ality
is g
ood.
It d
epen
ds o
n th
e di
ssem
inat
ion
syst
em &
ultim
ate
user
(s),
but
1 s
eem
s be
st.
Mic
rofi
che,
bec
ause
of
exis
ting
Fede
ral
stan
dard
s, c
ost,
and
avai
labi
lity
ofre
ader
s.1.
Den
sity
and
han
dlin
g ar
e su
peri
or to
3.
copy
ing
supe
rior
to 2
I ha
ve u
sed
mic
rofi
che
with
a p
roje
ctor
and
it se
ems
to b
e qu
ite s
atis
fact
ory.
Iha
ve n
ot u
sed
the
othe
r tw
u so
hav
e no
opin
ion
of th
eir
usef
ulne
ss.
Mic
rofi
che.
Lar
ge a
mou
nt o
f in
form
atio
nst
ored
by
unit
and
« fl
exib
le a
dis
play
.
V V V V V V V V V V
N.3
N.1
N.1
+ P
LU
Scom
pN
.1 ±
PL
USc
heap
N1
+ P
LU
S ex
p
N.1
N.1
N.1
N.1
+ N
.2
N.1
+ G
arb
AL
PM
AL
U
AL
PMA
LPM
AL
pm
AL
PU
AL
PM
AL
PU
AL
PU
AL
PU
2a 3 in lb
2b (
1)
2a lb - 3 6a
1 1/2 1 1 1 1 1 0 1 0
3 3 3 3 3 3 3 3 3
RE
CA
PIT
UL
AT
ION
:O
vera
ll m
atch
sco
reSe
nse
Scor
eM
otiv
atio
n 0
Mot
ivat
ion
1M
otiv
atio
n 2
25 (
100
%)
22 (
88 %
)W
eigh
t0 6 6
Mot
ivat
ion
3(1
4)M
otiv
atio
n co
mpo
site
Mot
ivat
ion
poss
ible
max
.M
otiv
atio
n fa
ctor
Gro
up V
res
pons
es
42 54 7554
/75
13 (
52 %
)
O
STUDENT FEEDBACK IN CAI 301
ABSTRACT
A stand-alone CAI frame was written around a question designed to elicita wide variety of replies, in order to measure the utility of reply analysis inrefining individual frames. Half of a real sample of fifty handwritten responseswere submitted to the frame, with a match score of 96 % and a sense scoreof 48 %. After revision, the frame attained a match score of too % and asense score of 88 %, on the other half of the test set.
The authors, discuss selected-string-match CAI strategies, including theuse of attitudinal scan combinations to indicate cognitive state. Mechanics ofthe CAI « macroframe A and its « satellites » are detailed.
RAstnit
A propos d'un projet sur l'instruction par les machines a calculer, un seulcadre isole fut daft sur une question precise afire d'obtenir une grande varietede reponses, et d'etre a meme de mesurer la valeur de l'analyse des reponsesfournie par les cadres individuels. La moitie d'un vrai modele de cinquantereponses ecrites a la main furent soumises au cadre, avec une congruencestatistique de 96 % et un systeme de comprehension de 48 %. Apres avoirete revise, le cadre atteignit un resultat de too %, et un systeme de comprehen-sion de 88 % sur moide du contrOle d'essai. Les auteurs discutent lesstrategies de « selected string match CAI » comprenant l'usage des combinai-sons parcourues pour indiquer Petat des connaissances. On discute les meca-niques en grand cadre et ses satellites.
ZUSAMMENFASSUNG
Fez amine Frage der Programmierten Instruktion wurde ein unabhangigerRahmen geschrieben, der entworfen wurde um eine Reihe von Resultatenzu bekommen und um die Brauchbarkeit der Antwortenanalyse zur Ver-feinerung einzelner Rahmen zu messen. Die Halfte der Originalserie vonfunfzig handgeschriebenen Antworten wurde in den Rahmen gefuttert underreichte einen Erfolgsgrad von 96 % und einen Verstandnisgrad (von derSeiten des Systems) von 48 %. Nach der Revision erzielte der Rahmen mitder anderen Halfte der Versuchsserie einen Erfolgsgrad von too % unterVerstandnisgrad des Systems von 88 %.
Die Autoren beschreiben die Auswahl-Aufeinanderfolge-Anpassung (se-lected-string-match) Strategien des Programmierten Unterrichts, einschliesslichdie Anwendung von verhaltensskandierenden Kombinationen, urn auf dasErkennungsstadium hinzuweisen. Die Mechanik des « Makrorahmens » undseines « Satelliten » werden beschrieben.
4
302 J. C. MEREDITH & D. FERGUSON
JOSEPH C. MEREDITH was born at Oakland, California on June 29th,1914. B. A. Olivet College, Olivet, Michigan (i934) and M.L.S. School ofLibrarianship, University of California (1967). Between 1941 and 1962 :
Commissioned service, U.S. Navy; studies in International Law and Strategy& Tactics, U.S. Naval War College, Newport; R.I. NATO staff duties;R&D in amphibious warfare, various commands. From 1967 to 1968 Projectstaff member and from 1968 to 1969 Project Manager, CAI component of aproject entitled « An information Processing Laboratory for Education andResearch in Library Science », (0EG-I-7-71o85-4286), Co-Director of In-formation Processing Laboratory in the School of Librarianship, Universityof California. Is now Assistant Director (Acting) of the Research Center forLibrary and Information Science, Graduate Library School, Indiana Uni-versity, Bloomington, Indiana, 47401.
Publications :« The CAI Author/Instructor an Introduction and Guide » (EnglewoodCliff , New Jersey, Educational Technology, Inc., 197o).« Machine as Tutor » in Educational Technology IX (September 1969).« DISCUS A New CAI Programming Language » (with StevenS. Silver) Institute of Library Research Technical Paper Nom (February 197o).
DOUGLAS FERGUSON was born on July 4th, 1935. B.A. (Philosophy)Macalester College (1956). M.L.S. School of Librarianship, University ofCalifornia (1967). Post-master's course work, ditto. From 1967 to 1968 TeamMember, Indian National Scientific Documentation Centre, New Delhi.Study and proposal for the National Electronics Information Network. FromJanuary to September 1969 specialist in CAI Course Development, Instituteof Library Research, University of California. Is now editor of the AutomationDivision, The Stanford University Libraries, Leland Stanford, Jr., University,Stanford, California 943o5.
43