Embed Size (px)
Citation preview
ll(WP»pwpi!pi«!JP|il«««iiP»WWPP5!WlW^P^^
■
AD/A-002 705
FIELD EVALUATION OF MODEL II OF THE COMPUTER-BASED, INDIVIDUAL TRAINER FOR THE RADAR INTERCEPT OFFICER
Joseph W. Rigney, et al
University of Southern Californi
Prepared for:
Naval Training Equipment Center Advanced Research Projects Agency
July 1974
DISTRIBUTED BY: m\ National Technical Information Service U. S. DEPARTMENT OF COMMERCE
MMBM iiHjiriMitiiliiiliT "-miiiiiiiiiii mi n iin ■■■ - -■ ' ...^**ä***1***M**~. «linii vrmiiii
""im .mi Mim*r*m**mm*mmmm iJiJii(i*u«ii,(y».».wi|iyMi|iiiB.iiMiii!j.iiii^^p» 1. Iliumii,!«^*!^ J.ll;IWHJ!l»IIM,Ili|iJ!Ult»-.«!kL(VJl,-«(l"BI|LJ,lllll,|lllPJ4U< JW,. I|"> J '^ "WJW itJlimiJII.Ul ,.l 'IHHI
j
UNCLASSll'IED StCURITY CL&SSlFi!'. ATION O'' THIS PA'.E fHfiBn /)alB RnKrsdJ
REPORT DOCUMENTATION PAGE READ INSTRUCTIONS BEFORE COMPLETING FORM
1 REPORT NUMBER
73-C-0065~2 2. GOVT ACCESSION NO.
4 TITLE ranrf Suhtlllft
FIELD EVALUATION OF MODEL II OF THE COMPUTER- BASED, INDIVIDUAL TRAINER FOR THE RADAR INTERCEPT OFFICER
7. AUTHORC«;
Joseph W. Rigney, D. Kirk Morrison, Louis A. Williams, Douglas M. Towne
9. PERFORMING ORGANIZATION NAME AND ADDRESS
Behavioral Technology Laboratories University of Southern California Los Angeles, California 90007
II. CONTROLLING OFFICE NAME %ND ADDRESS
Naval Training Equipment Center Orlando, Florida 32813
3 RECIPIENT'S CAT ALOG NUMBER
5 TYPE OF RFPORT & PERIOD COVERED
Final (1 Jan. - 31 Dec. 1974)
6. PERFORMING ORG. REPORT NUMBER
8 CONTRACT OR GRANT NUMBERCt.)
N61339-73-G-0065
10 PROGRAM ELEMENT. PROJECT. TASK AREA 4 WORK UNIT NUMBERS
2752-02P02
12 REPORT DATE
July 1974
14 MONITORING AGENCY NAME » ADDRESSC//J///»f»n( /rom Conlro/ling Of/c«;
13. NUMBER OF PAGES
IS. 15. SECURITY CLASS, (ol Ihlm import)
UNCLASSIFIED
15«. DECL ASSIFICATION DOWNGRADING SCHEDULE
16. DISTRIBUTION STATEMENT (nl this Report)
Approved for public release; distribution is unlimited.
17. DISTRIBUTION STATEMENT fo/ 'he mhitracl enf red In Block 20, It dltlerent from Report)
Reproducod by
NATIONAL TECHNICAL INFORMATION SERVICE
U S Dopartmoit of CommaKo Spnngliold, VA. 22ISI
IB. SUPPLEMENTARY NOTES
Rigney, 3. W., Morrison, D. K., Williams, L. A., and Towne, D. M. Description and Initial Evaluation of a Computer-based Individual Trainer for the Radar Intercept Observer. Los Angeles: Behavioral Technology Laboratories (NAVTRAEQUIPCEN 71-C-0219-1) , 1973. (cont.) 19. KEY WORDS CConiynu« on reverse elde It neceeeary end Identity by block number)
Computer-assisted Instruction (CAI) Radar Intercept Observer Training Adaptive Control of Instruction Computer Graphics
20. ABSTRACT ^Continue on reverse elde It neceeemry end Identity by block number)
Model II of the basic skills intercept trainer for Radar Intercept Officers was evaluated in a school environment. Model II incorporated different instructional sequencing logic, additional weaponry capability, and additional graphic features from Model I (developed under a previous project). These added features were designed to assist the student in understanding the intercept geometry and in learning to use the B-scan
display. One (N=31) of two random groups practiced on a trainer with these
DD ,: FORM AM 73 1473 EDITION OF 1 NOV 65 IS OBSOLETE
S/N 0103-014* 6601 / UNCLASSIFIED SECURITY CLASSIFICATION OF THIS PAGE (When Dmta Bnlmrmd)
•
l(i|JUiMimiiifia^B»pw»j.iiw)(!w^i"nj^,ii"i|.i!w .i">Hywii..l*l«>liii,if»»v.i''W-'!'»?«',B'".'-'"1 '• .r".'."iy,«'ii .i.^im-jiwuiwi i
1 UNCIASSIFIED
'<! I Y CL ASSIFIv- A T ION OF THIS P AGC.(t*fiBfi Oala EnOredl
acfditional graphics, the other group (N=29) used a version of the trainer without these features.
The group using the enhanced trainer took longer and required more problems to satisfy the trials-to-criterion logic used in the practice session. All students expressed predominantly favorable attitudes toward the trainer.
Two measures of transfer were used: a twenty-item post-test using different problems in a random sequence, and an inflight checklist administered during each of eleven practice flights and two training phases per student. The two groups did equally well on the post-test. The group using the enhanced trainer was rated slightly higher than the other on the inflight checklist on both of the inflight trail ing phases.
Explanations for these results are discussed, ana recommendations are offered. The principal recommendation is that the self-standing, CAI-system-in-a-terminal, having been demonstrated to be a viable and effective concept in a Naval training environment, should be developed as an attractive alternative to (1) large centralized-processor, distributed terminal CAI systems for use in remote environments, and (2) multi-million dollar simulators, for certain types of job skills training.
18. (cont )
Rigney, J. W., Morrison, D. K., Williams, L. A., and Towne, D. M. A Guide for the Application of Performance-Structure Oriented CAI in Naval Training. Los Angeles: Behavioral Technology Laboratories (NAVTRAEQUIPCEN 73-C-0065-2), 1974.
:
UNCLASSIFIED / (/*-• SECURITY CLASSIFICATION OF THIS P*0Erit7i»n D«i» Bnltrcd;
HMMMUMMaiiaMHl — i n\ »m—mmmmm ianttiti-iiii^rttiiii' Hkttitmmirti-iii-tifttiiftä,■ .*» .
Mj^ ■ ■■. ti^ w»^«* ■' vmmmjm*» ^ > I,«;I p»»"«," * ijw9*m*.im*K'iiw'iiwmlm-'i™11 »>9*m vi}} ■M1 'I .MVUWIVWI.. ■|..IWII"W)."Wt.J'iiIJ llW MiPIH V'^MW'lrWW ll'i-^HU! llfV«4IHI»Bf,WP J
NAVTRAEQUIPCEN 73-C-0065-2
FIELD EVALUATION OF MODEL II OF THE COMPUTER-BASED, INDIVIDUAL TRAINER FOR THE RADAR INTERCEPT OFFICER
ABSTRACT
Model II of the basic skills intercept trainer for Radar Intercept Officers was evaluated in a school environment. Model II incorporated different instructional sequencing logic, additional weaponry capability, and additional graphic features from Model I (developed under a previous project). These added features were designed to assist the student in understanding intercept geometry and in learning to use the B-scan display. One (N=31) of two random groups practiced on a trainer with these additional graphics, the other group (N=29) used a version of the trainer without these features.
The group using the enhanced trainer took longer and required more problems to satisfy the trials-to-criterion logic used in the practice session. All students expressed predominantly favorable attitudes toward the trainer.
Two measures of transfer were used: a twenty-item post-test using different problems in a random sequence, and an inflight checklist admin- istered during each of eleven practice flights and two training phases per student. The two groups did equally well on the post-test. The group using the enhanced trainer was rated slightly higher than the other on the inflight checklist on both of the inflight training phases.
Explanations for these results are discussed, and recommendations are offered. The principal recommendation is that the self-standing, CAI-system-in-a-terminal, having been demonstrated to be a viable and effective concept in a Naval training environment, should be developed as an attractive alternative to (1) large centralized-processor, distri- buted terminal GAI systems for use in remote environments, and (2) multi- million dollar simulators, for certain types of job skills training.
HCCBSIO» ftf
N1IS DOB
mui "■■■'" «mil
\ ".' '
COVERNMENT RIGHTS IN DATA STATEMENT
Reproduction of this publication in whole or in part is permitted for any purpose of the United States Government.
jMMHIMtf^wytfMMiHtaM* J
r'!??TT^wi'?ff^«jP!!WSBP^f!w?W'PRf^^* l-J- VT A".' •! w ui ■ J u i".-; m *»t\v*i,wm j » pmiFfjM.fy.'üfl!»*1"» wy^™m!*»*^• i .■ ■ . M M -
NAVTRAEQUIPCEN T3-C-0065-2
FOREWORD
The Naval Training Equipment. Center is engaged in a program to advance the general state of Computer-Aided Instruction (CAI). A portion of this program, undertaken in association with the University of Southern California, is described in this and two other technical reports (Rigrey, et al, 1973; Rigney, et al, 19710. The present document reports the development, field implementation and experimental evaluation of an (almost) completely automated CAI capability for teaching skills for the Radar Intercept Officer's (RIO's) job. Contributions made by this research to the state-of-the-art of CAI stem mainly from demonstrations as follows:
a. Advanced types of CAT, as employed in the present trainer, are feasible and useful in field application;;.
b. These advanced CAI features are successful when one property of the training is a simulation of a job performance situation (the RIO task) which is characterized by dynamic, interactive, real-time qualities. (This type of training material is in contrast with training materials much more typically subjected to CAI applications. With these latter materials, programmed instruction, rather than job simulation, provides the model for the CAI approach; and information acquisition rather than skilled job performance, is the major outcome of instruction.)
c. Details concerning the manner in which a given feature of CAI is utilised is critical to its effectiveness in training, and these details are not always readily apparent. Results related to the latter point permit preliminary statements to be made in this report about the most appropriate ways to utilize some CAI features in training applications.
■*♦
In addition, this report describes a valuable by-product of this research effort, viz., a new CAI device for RIO's which essentially is ready for use in operational training environments. This computer-based trainer has shown capa- bilities which suggest considerable superiority to trainers and training methods currently used for RIO instruction in terms of cost-effectiveness. Definitive evaluations of the various capabilities of the trainer, however, await its more complete development and an experimental environment more amenable to experimental manipulation and control. Under these improved conditions, the promise of superior training and transfer performance held by the computer-based aspects of the trainer would stand a greater chance to be demonstrated than under current conditions. Complete development of the trainer would permit comparisons with current training capabilities in which the present CAI program could be used as an alternative, rather than as an auxiliary, to such training. As it is, the data trend obtained, which associates a reduction in certain instructiDual components of the trainer with degraded transfer performance, can be considered as only minimally suggestive of the benefits that might be derived (and arc expected) from such features.
At present, the utility of developing CAI materials for teaching the utiliza- tion of the AWG-9 system for maintaining the V-]h aircraft is being investigated for the continuation of research in this program.
»
ARTHUR S. BLAIWES Scientific Officer
piini^wwBWiwwwwwpBwppPB^^ ! ij^iiw.iiiTiJwipjiwiiwiip p n.p .ij«»mH.ypmm.tiiiLii T^^- 1.1 IIMWUMIIS a.JH«.lfT"«VH
NAVTRAEQUIPCEN 73-C-0065-2
^ /
Section
1.
[I.
III.
TABLE OF CONTENTS
Page
INTRODUCTION 1
MODEL El OP THE RIO TRAINER AND THE EXPERIMENTAL
DESIGN 2
Added Tactica3 Procedures 2 Added Instructional Features ^ Chanr.cn in the Instructional Strategy o The Experimental Design
DATA COLLECTION, DATA ANALYSES, AND DISCUSSION 15
Data Collection ^ Data Analyses Trends Across Successive Practice Conditions 23 Comparison of Trends Between Groups 28
Student Attitude Questionnaires 30
Biographleal Data - Inflight Checklist Data ^ Summary of Results ^ Discussion of Results f[ Conclusions and Recommendations 39
REFERENCES kl
h? APPENDIX A
lift APPENDIX B
i i i
tmmmmmMimm ■MMM MHMMMMMBMI
Hwwpwpw^iiipjiippwiisiiawwipww^iiPBB^ i, a u.iwpwwfWTiBwwi^ww^wipwi^ »Mm^u-JHumiji
N/WTRAEQUIPCEN 73-C-0065-2
LIST OF ILLUSTRATIONS
Figure Päfi!
1. Approximate Geometry o£ Sparrow Attack 3
2. Graphic and Alphanumeric Knowledge of Results in the Fire Display (a) Sparrow Display, (b) Sidewinder Display, (c) ß-Scan Display '3
3. Free-Fly Mode with Optimal Path and Track History Added 7
4. Experimental and Control Groups Instructional Sequences. Dynamic-Mode Cycle Repeated for Each of Seven Speed Levels
5. Scoring and Transitioning Criteria
iv
13
14
MMMMMMIMM «,.—.. ..*jfl
iJiVIJIIIl»«ll4l),JWli.«iHW™'WIPJi|"l i",l«lli|.imJiW-i»WPIl|w|i,4li'!li iii!i-ll|ipiii -^ i , IU.I iiip«.^»»!..^«!!.! I. W,"M ."i(ii WHlili.WWgiUMU'UKHiWuil«"»'»!»«"!."^1!»,! .ijn«iKiiini.juixiimMiu^^
NAVTRAEQUIPCEN 73-C-0065-2
LIST OF TABLES
I
»
Table gagg
1. DIFFERENCES BETWEEN GRAPHIC FEATURES IN THE ENHANCED AND REDUCED TRAINERS ..
2. DEPENDENT VARIABLE LIST
3. GROUP COMPARISONS ON TRANSFER TEST DATA ... 17
4. GROUP COMPARISONS ON PRACTICE SESSION DATA- FREQUENCY VARIABLES \
5. CROUP COMPARISONS ON PRACTICE SESSION DATA- TIME VARIABLES \ 9
6. CROUP COMPARISONS ON PRACTICE SESSION DATA- MISSILE FIRING AND OTHER ACCURACY MEASURES " . .' . 22
7. OVERALL LATENCY (SEC) (MEANS AND VARIANCES) TO COMPLETE THE TOTEBOARD AS A CONSEQUENCE OF PRACTICE 24
8. OVERALL ERRORS IN COMPLETING TOTEBOARD ACROSS SUCCESSIVE PRACTICE CONDITIONS ........ 25
9. TRENDS IN SPARROW P-1HT SCORES (MEANS AND VARIANCES) ACROSS SUCCESSIVE PRACTICE CONDITIONS ' 26
10. TRENDS IN SIDEWINDER P-llIT SCORES (MEANS AND VARIANCES) ACROSS SUCCESSIVE PRACTICE CONDITIONS . . ^
U. MEANS AND VARIANCES FOR NUMBER OF TURNS USED TO MAKE AN
INTERCEPT ACROSS SUCCESSIVE PRACTICE CONDITIONS . .'. 27
12. COMPARISONS BETWEEN ENHANCED (E) AND REDUCED (R) GROUPS WITH RESPECT TO TRENDS IN SUCCESSIVE PRACTICE CONDITIONS 29
13. ANALYSIS OF STUDENT ATTITUDES TOWARD THE TRAINER ... 3]
14. DIFFERENCES IN MEANS OF PROPORTIONS OF SAMPLES EXPRESSING ATTITUDES TOWARD THE TRAINER . . . 32
15. ATTITUDES OF STUDENTS TOWARD THE FREE PLY MODE IN THE TRAINER .... 32
16. BIOGRAPHICAL DATA FOR THE GROUP USING THE ENHANCED TRAINER: APTITUDE TESTS AND PRECEDING NAVAl, TRAINING COURSE SCORES ■uwiNmi,
V
MM ■MMM
5^Bl!W^(!WIHBP?ipPW««f"«)l(*l»W.«ii|iiiJi,,I ii.ilii.liiii .iil.UHiipiili^'mwiii-WWPiBMW' l1"» ' ■' (PPiipiIIPpJ 11114.iuwp^pipimpi»«p p^mpn«11 ILII i.
NAVTRAEQU I L'CKN 73-C-0065-2
I.IST OK TAiJLI'S (CONT )
)
Table Pa^c
17. BIOGRAPHICAL DATA FOR THE GROUP USING THE REDUCED TRAINER: APTITUDE TEST AND PRECEDING NAVAL TRAINING COURSE SCORES 34
18. COMPARISON OP RANKED SCORES OF THE ENHANCED AND REDUCED GROUPS ON THE INFLIGHT CHECKLIST FOR WO INFLIGHT TRAINING PHASES 36
VI
M ■■ - ^ - -- -—~"--Mlll IWH
ppy^wp«p*»fli5^ppw(iBfp»pp?T^ v-TT^y UrUtivMui^iiMmmfummm'pm^mm^ ^'^tw^i.1. --J™ vmrwrnimi \mmm^imf
, ■•
NAVTRAEQUIPCEN 73-C-0065-2
SECTION 1
INTKODUCTION
This is the second of two reports describing the results of develop- ing and field-testing an individual trainer for the Radar Intercept Officer. The earlier field trial was a demonstration of the feasibility of using an intelligent graphics terminal as a standalone, "CAI-system- in-a-terminal" in a field-training environment remote from access to
time-sharing networks.
In the research to he described here, the field test was concerned with evaluating refinements in the instructional strategy, and with the instruc- tional value of several computer graphics features. These differences between Model 1 and Model II of the trainer, and the experimental design, were described In detail in ARPA Quarterly Technical Report for July 1973. This description is reviewed below.
Since the performance, assisting the pilot in air-to-air intercepts, is quite complex, the trainer also is complex. Thus, this complexity is necessary even though the trainer -as designed to teach only basic skills used in this performance. Therefore, chc details of the instructional strategy, and of the differential treatment conditions, require rather complicated descriptions. General requirements for a CM system of this type are discussed in a companion report. The present study is one in a series of studies designed to provide an empirical basis for deriving and evaluating these general requirements. The reader is referred to that report for information about the CAI system architecture.
I 1... Rlgncy, et al,, L973.
?.. • Rigney, et al., 1974
mätmm
ippjiuiiiwipiWP1 u,■4*3ppi«Bn"Piwpf(f^iw?is^ tm»wr1^1»^.^^w'■l»w*.J*.Jiw^'-Hiw**'.mF"'^.I'IIUMW^' '"^A.w'.A1 i"i»^m>ii!iiiwi'i1 w11mm..w\m* m,mmmmfmiw,m*''ft(tf-wv^n?!1 I^JI^WP■ii!PiPi.iwi»f 11 ii-"'•" •*..--r»■■--^ i.i.«ii^iuuvwN
NAVTRAEQUIPCKN 73-C-0065-2
SIICTION 1.1.
MODEL II OF TIIK RIO TRAINER AND THE EXPERIMENTAL DESIGN
ADDED TACTICAL PROCEDURES
Positioning Liu' Oighter for a Sicluwlnder attack requires approximately a ISO degrees turn Crom a heading near the Bogey Heading Reciprocal to achieve a final position astern of the bogey. The procedures for accom- plishing this tactic wore taught by the Model i trainer.
In Model II, the procedures for the Sparrow attack also are taught. The Sparrow is the first weapon fired, while the interceptor is still approaching and in front of the bogey. The lighter must be turned slight- ly from a collision course heading to establish a lead collision course lor the missile. Then, the fighter must he Leveled for tiring. Alter firing the Sparrow, the tighter must be turned slightly again to enter the rcattack turn for firing the. Sidewinder. Approximate geometry is shown in 1: igure I .
The student Kid's task is made considerably more difficult by the requirement to make both missile attacks during a problem, lie must be able to establish an approach that will allow the necessary maneuvers to be made in quick succession. To do tills, he must develop a high mental information-processing rate and he must be able to sniit his attention from Sparrow to Sidewinder attack requirements. This is a classic example of performance that is driven by a real-time problem. Under these condi- tions, the development of fluency, as indicate'' by short response-latencies and low error-rates, is essential.
This added tactic required several modifications to the computer pro- gram; lor scoring the student's Sparrow attack, for providing a Sparrow fire display, and tor recording values ol student response variables per- taining to the Sparrow attack phase ol the intercept. The principal new computations Involved were lead collision course error for th( Sparrow missile, and a score, called probability ol hit (pllit).
To avoid distrait in,', the student, graphic and numeric scores Tor the Sparrow attack were not displayed until he finished the reattack (Side- winder) phase of lh< problem. The fire displays for the two missiles were combined In one final display. This lire display presented both graphic and numeric knowledge ol results, as .shown In figure 2:
a. The Sparrow Display. Bogey and fighter positions and headings at time of firing are displayed. The dotted linos In the display show the firing-range •one. The diagonal line shows the lead collision course. The hit probability X 100 must be » 50 for a passing score. I.CC is Lead Collision Course in degrees. Krror 2L means LCC was 2 degrees too far left in this Instance. TA 19 means target aspect was !') degrees,
«WMIIMMIIMtk in» —^
IM^«.>WB iiM*iu^iiwii.mwiap*i!|^BiPn.ii.ui ..nmjjMmwp.-u > ywwjypwJUR«■ j.!ijun ^M| IUJimin■IJIIIWWU. ' i i) i^ 111III^iifSMUiMPW iiiiwwifwp»™,flM'a^ii^Miwpi-f*T^^^*mHi,iu^wpi!ii> LI. 1.1^+1. '-.'■ -^^.^ipHiJimi 111 JI. 1 IM. JUJ^
■
^
NAVI'KAKQU [PGKN 73-C-0065-2
BH
'
DISP1ACEMENT DISTANCE
ORIGINAL FH
The bogey is on Cixed course (BH) and speed. The fighter Lurns I'rom original Fighter Heading (FH) at position A to a Collision Course (CC). At a short range of 8-1) nautical miles (15), the fighter turns port to establish a Lead Collision Course (LCC) for the Sparrow missile. After release of the missile and prior to 6 nautical miles short range (C) the fighter turns port to increase his displacement disisnee and provide turning room for tin' Sidewinder reattack released at point 1).
Figure 1. Approximate Geometry of Sparrow Attack
I
M_ ■Ük ■ -1
Hii.uniiii i i »PPW^P«WW!BP»WSf!I^WIB!ipWPIP«IWpinpiipW?WW»WP^
NAVTRAEQUIPCIäN 73-C-0065-2
b. The Sidewinder Display. The .sLtuatLon at Lliu time for Eiring Ls displayed. The Sidewinder acquisition cone, the bogey heat cone, and the Ciriiig range zone are shown. For a passing score, hit probability X 100 must & 80. In this problem, ehe student made 4 turns after turning to collision course, took 173 seconds to complete the intercept, and turned a total of 100 degrees in hard as possible turns.
c. The Simulated B-Scan with the Optimal Path (solid line) and the Track History (dotted line) superimposed. The dotted circle marks the point at which the Sparrow was fired.
^ y
/VDDED INSTRUCTIONAL FEATURES
Model I Instructional features were described in drLail in an earlier report. They are summarized below:
Instructionai Feature When Displayed
Static, geographic plot of Lighter and bogey
in Static Node, after a computa- tional error or, on-demand by student
Computational error knowledge of results: erroneous value disappears from totcboard, arrow does not move-
Correct answer for Intercept triangle va Lue
Sum of response latencies to complete toteboard
Free-fly Node: Triangle values provided, dynamic triangle provided
Knowledge of results for Sidewinder firing
In Static and Dynamic Modes, after entry of erroneous value by student
In Static anil Dynamic Modes, on- demand by student
In Static and Dynamic Modes, after entering last value in toteboard
in Dynamic Mode, if score for Side- winder firing was <80
After firing Sidewinder
The above features were continued in Model 11 of the trainer. In addi- tion, latencies to compute each intercept triangle value were displayed, for more detailed knowledge of results about the relative difficulties of these computations. An end-of-problem lire display Cor the Sparrow missile was added. The data and the diagram in (a) in figure 2, pertinen: to the Sparrow, gave the student knowledge oE results of his Sparrow attack.
Two new graphic features in Model 11 were a bogey track history and an "optimal path." Both of these were displayed on the B-scan in the free-fly mode and in the fire displays at the end of a problem. At other times, they and the dynamic triangle were available' to the student through a func- tion key. The bogey track history and the optimal path were designed to
m**. — -- - ■ ■■■ ■ ■■ '■•■ -
I^PW^W . ^IWMPIINIÜIPI II liwii ipnipiWM.^^ »PW ■ .'!™y'WW*^PPIWWVW*»!^^''WBSWWl!WPP»S»l^»«««^W^^^»^R,-.-'WW»» ^A '^PipmwppWBPIRlljl
NAVTRAKQUIPCKN 73-C-0065-2
HIT PROP. 90 107 SECONDS Fll 319 LCC 321 KRROR 21, TA 19
II [T PROß 100 4 TURNS 173 SECONDS 100 DEGREES HAP
PROBLEM 219
Figvre 2. Graphic and Alphanumeric Knowledge of Results in the Fire Display (a) Sparrow Display, (b) Sidewinder Display, (c) I5-Scan Display. See explanation in text
-MMMM^M ^aaMMM «_
MPIIH^BB^WPH. ».i iiiJ.iBiiiaaiviii i. «^^IW^IilBSIWWfWPBWPWIIWWBPIIPPBHpHllllWPW
NAVn^AKQUIVCliN 7:i-C-0065-2
assist Liu1 siudcni in learning, lie could maneuver the Cighter to cause tliL' simulated radar return (Llie bogey) to move down the B-scan on the "oplLmal path" for l>oLli Sparrow and Sidewinder attacks. These displays arc shown in figures 2 and '5.
The dynamic true plot, the bogey track history, the optimal path, and the fire-display geometry provided visual perceptual Information to the student. I5y comparing the dynamic true plot ol bogey and fighter positions, headings, and movements with the display on the B-scan, the student could be assisted in learning to interpret tliis relative motion display. He must learn a spatial frame of reference in which the B-scan becomes a window on the world fixed to the nose of the fighter, and all motion seen through this window is relative to the fighter's position, heading, atti- tude, am' movement.
The student could superimpose the bogey track history on the optimal path display on the B-scan. As long as he kept the simulated radar blip on or very near the optimal path line, he could make a good Sidewinder intercept. A Sparrow Eire marker appears on the optimal path, but the stu- dent also had to remember to turn slightly and to level out before firing. The graphic displays of situatiuna] geometry at the time of tiring each missile provided quickly-grasped Information that the student could use for knowledge of results, lie could see immediately that he was too far away, or too far to one side, or turned too much or not enough, or had set up the wrong lead collision course, etc. lie could use tills information to modify his performance on the next problem.
However, in the U10 trainer, the student must be able to meet criteria of proficiency using the B-scan alone, without the presence of any of these aids, before the program will transition him to the next level of difficulty or will allow liim to take the end-tost. When the stucient is working with the trainer, most of the time he is practicing without the static or dynamic triangle, the bogey track history, or the optimal path. The presence of these particular stimulus conditions is not essential for correct responding. These features are only instructional aids which present supplementary information for the? standard B-scan. Therefore, they would be expected to be most valuable in the initial stages of learn- ing, to assist the student in understanding relative motion on the B-scan.
CHANGES (N THE tNSTRUCTlONAL STRATEGY
In Model I ol the trainer. Intercept problems were grouped in four target aspect (TA) categories. The trials-to-criterion logic required the student to make a passing score on four successive problems, one from each of the four TA categories. Furthermore, during practice, the student cycled through the four TA categories Ln each successive four problems.
The instructional sequence was changed in Model 11 to give the students more concentrated practice in each target aspect category, and to provide an introductory problem for each TA category. Upon entering a new TA category, a student was first given a problem in the free-fly mode.
■ ' "■ •imliilni n ■ .
f!Smmimfi*mm**9ßQ&!i^m ^imh^mm^m m» - ms im* ^v\w3i Wi x^wmwiWW*v*vr'Q^>i>'i ■ ; ■^■1^*'^ i»^. i-w opf.uüj^flwiswpf JH^^ i1 L UU U. s, wH!lHlM.w«MW-w,■ n- mWf w w^Mmmw-v^v^w' ■' ■" ^m l\j-f'W m miM. u- ^ '-«-JM
1 T.1 : , ■■■■i
NAVTRAEQUIPCEN Y 3-C-0065-2
RANGE 26 SPEED 300
Kll 120
Uli 230
B6 85
BHR 70
TA 20R
CC 95
MUA 501,
DTG 130
AG 351,
DD 4
RG 12
20
e ■>
■II 120
PROBLEM NO. 47
«• ^
Figure 3. Free-Fly Mode with Optimal Patli and Track nisiory Added
Ml—iMhMi in ■ i - —•—<— ■>——
wwiw .,l »'. ^^^^^"
NAVTRAEQULPCEN 73-C-0065-2
Then he worked practice problems until he achieved passing scores on two successive problems, lie then was automatically transitioned Lo the next TA cati'f.nry. There, the student repeated the samu sequence.
Mso, data Crom tlir tirsi Cleld trial Indicated that the original Cour tarnet aspect categories were not widely enough separated. According- ly, only three TA categories, with greater separation, were used in Model
II.
A second modification In the Instructional strategy concerned Uu" time alloted to each student. tn Model 1, every student was required by the school to spend ten hours on the trainer, it the trials-to-criterion logic transitioned him to the highest speed level, the student Oinished out his time at that level. In Model II, a student spent only as much time as was necessary to satisfy the trials-to-criterion logic and to take a special end-test of twenty problems. Thus, each student Oinished the course when the program logic said he was ready.
The end-test was a third modification in the instructional strategy. This was added, because we wished to test the abilities of the experimental and the control groups to cope with unexpected problems. Unlike those in the practice session, the problems in the end-test were in a random sequence with respect to target aspect. A student could not predict whether the next problem would be a Low, medium, or high target aspect. Also, all test problems were run at 500 knots, 40 knots faster than the highest speed level in the practice problems. The end-test was designed to be a typo of transfer test.
Other modifications related to the static triangle (a geographic plot) and the free-fly mode. When a student started on the trainer, he received three sample problems in the free-fly mode, one at each target aspect cate- gory. Then, he was sequenced to the static mode, where he practiced mental arithmetic with the static triangle visible. Upon solving two problems in succession, each with a total latency - 6U seconds and no errors, he was transitioned to the second phase of the static mode, to practice without the triangle.
THE EXPERIMENTAL DESIGN
A number of conditions, each of which could serve as an independent variable in more rigorously controllable laboratory experimentation, were lumped together in this field trial. Under Ideal conditions, a between- groups factorial design should bo used, with an independent group for each combination of variables. Under field conditions, where the rate of data collection is slow, the number of subjects available over a period of three to six months is limited, and the evaluation must be conducted on a "not to interfere" basis, (his is not feasible.
Students in the first lield trial had .•.pressed strong positive atti- tudes toward the static triangle, the dynamic true plot, the free-fly mode, and the graphic fin displays, as being helpful instructional features.
m—m mmm mm MMMHMMMi
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^BBWn^^^^^^^^^^—
I NAVTRAEQUIPCEN 73-C-0065-2
Therefore, features like Lliis miglit be considered worthwhile merely to increase the attractiveness of the trainer to students. However, there would remain the question of whether these features increase proficiency, as measured by quantitative criteria. The experimental design for the field trial ol Model II was planned to provide some inlormation about this question, in terms of the possible advantages of trials with certain of these graphic features present versus "standard" practice trials in which these features were absent. The measures to be used for comparisons between groups were numbers of problems required to reach transition cri- teria at various stages, errors per problem, latencies per response cate- gory, values of variables measured at the time of firing eacii missile, scores on tlie end-of-course test, and ratings by instructors on an inflight checklist used in two inflight training phases.
Two versions of the trainer were developed. One was enriched by a particular pattern of graphic aids and one was not. The following summarizes differences between graphic features in the enhanced and reduced trainers:
—MB^^i »W^—^ I ; ; . .■ ■■.----
............ ■ ■
NAVTRAKQU EPCEN 73-C-0065-2
a;
Q U u Q Ui K
P
«Ji
u
US u
Ui S H
'75 W
| H
PL.
w
p UJ CO
u ss
UJ Cu
M a
en
■u
u d • • t) 0) o 0) U) 6 c a. X) 0) 0) cd in
id a ,9 •r-l
)J
.-I XI
u n) JJ <: e o TD o 4J >, d H 01 U 01 u « u r-l 0J ■—i a. «j ^ u 00 0 u. ^xi d QJ <u M »j r-i u o '-I-I o
•r-l n) V-i <u r—1 o u o -^ x: o »J >u 01 m JJ a. t/i u •H vi C u -a o) rl 4J
01 .c o l-l
IM 0J r—1
d OJ
3 UJ d 0J ■a o o> ui
u B (0 id d xi rf o C J= OJ o •r-l O "JJ vi d JJ u 0) 4J XI U JJ o n) tn
T3 •H d 01 B <X • • C -rl QJ 3 u •H •u 01 OJ • r-l JJ ~ H
Q iJ rt o i-l UJ Q •o •r-l U UJ <n iJ u XI o H o oo u o) -a >■ u 01 o P r^ d o) xi d
:3 *-> (0 (U r-l u U T) ^
-o •r^ a. d cd •^ •r-4 4J XI OJ a. d O 01 )-l 01 -H CM JJ S-j U-l u O u OJ di •rJ M 3 Vi 3 OJ en a <h m d U ^-N y-i ■a W S CO XI QJ o M D j-i 0) a. >, n) 0J ~ M CO QJ oo x; -H
Q •H M TJ i—i u JJ >. n d r-l .• d ■H JJ M H HI 3 i-> UH - d n) rt 0) >, o V -ri Ui o
X u u U] 0) (U 0J --l jg "O Q -v u cd
1 01 CU ui >-i 01 T) "O a. ex s 0 CQ O 3 00 »J •o 60-a •H (U O 3 01 U) ?1 IJJ CQ ?1 xi d pu
^2 o c o >. ii-1 U ^ •^-, >J -.-i jj o •M •r-l
C XI CO
u Q
00 o •H JJ as
>,x) »J x: r-l d •-! 4J
u c u •o a •H >, d o) JJ U CQ U. cd '*-! o •H •1-4 •r-l c •H >, n xi ■H VJ (tf rt JJ u J_l CO ^J w rt VJ -H JJ JJ 0J E Vi n) ■H a) B D O d -u 3 lu tn (0 Ä QJ D QJ 00 u 0) 4-) V Q 00 >. d a fa Vl r-l JJ d t/1 d LO •v OJ ng d d u U. J3 "JJ -H
d /^ iJ ^-v M M <• *J o t: vi c jj C c -t (0 d o) <f 3 d VI 3 M 4J H o M -O •• •-^
CM I-I a.^> xj
in .•'^ •*• n ^■N n r> PI n s~\ •* I—* CM C^l CN| CM rO r-l »-J n <j- ^* ^^ ^.^ ^w^ V»^ N^ ^»^ ,w' v— v^
_ H* m 0] C • 0) I-i
00 •rl VJ )-i
•H (J 01 rt 'M OJ JJ -o a
a: 3 d d in Ö •1 f—i QJ -r-l ^ n ^-^^ cr) 3 u M o r-l X) QJ o i u oj -a ■ij
u • 0J JJ -M
H o rn
00 t/)
•—* 00
cd tn X)
ca 4J
o ^^ v»' 'M Q 01 d a. "a t—i
Ui m U •r-l rO 3 d 3 u d M U M •r-l cd W 55 • T! G *** JJ IJ >. QJ i oc o O i d JJ ■H » u
•H UI (J OJ i-l o X. 'W 1 4J SB Pf cd u UJ UJ
A «1 O UJ
a! a fJ O VJ
•r-l (d o
5 *, o. d s^ I UJ JJ OJ JJ D. QJ H-l o o H ■ r d eg tn 00
H >, O i-i 0J 01 -c >" v^' x: b cc rf •o JJ o u O
^ 4J O C'^ 3 3 u o i—1
«U rt u X H JJ a. 3 «j Ä
1 o OJ f—i
10 D.
■ r-l »-I
a UJ E
o cd c □
H W) c >» x: H w *— a rd Vl *M IJ ss c « 01 55 u JJ JJ JJ QJ UJ rt —( 00 ^: UJ UJ •r-l o •rl QJ O X) to •r-l u a U V) o •^ JJ 3 G •rJ d UJ JJ IJ XI <a UJ UJ O n •r^
os u u OS pd ■a U) •a OJ QJ 3 a. <J ,—1 i-i CU u 01 U 00 B 01
•J •r-l ä i n •r^ cd :; XI to •r-l B n >. ca o u o 00 d ■f-l u UI rt •H 0J u »tj XI d xi d Cd tn l-( « d 4-1 00 h-l 0) u QJ .r-l r* X u >. O- o ITi JJ JJ U U PUTJ fc tn a O rn pu o rt O I i—I d s i H -i f-i u. i; nj
a • i—i
• CM
• • o • ^-i CM
• • »a.
CO o
cd VJ oo
d o; G JJ cd 0) tj
d 01 E
•r-l 1-1 QJ a.
X) 01
3 4J
d o u
QJ 'J) V £ H
■M ■MM ■^M ^^ .^dfl
NAVTRAUQUIPCEN 73-C-0065-2
> x Some explanation ot the static modo, dynamic mode, and £ree-fly mode
also is necessary. The static mode is "front-loading." Before a student was given a complete Intercept: problem, he was given a period of drill in computing the six values Cor Liu- Intercept triangle: l)oj',cy heading reciprocal, target aspect, collision course, makeup angle, degrees to go,
•l1"1 anj'lr ofC. Tills drill was called the static mod,., when he could do' all six ol these values with no errors, no use of the answer k.y, and with a total latency under ()() seconds, on j_wo successive problems, he was auto- matically transitionod to the dynamic mode. in this mode, the remainder of the intercept problem was added; tin Student Still bail to compute the above six values, but the bogey and I i/,liter were now moving at a predeter- mined speed. As soon as tbe student entered the value ol collision course, the fighter started turning to that value. But, sine the fighter had been moving Irom the start ol the problem, the student's value oil collision course was "late" by tbe amount ol time it Look him to do the computations to ge t to i t.
Three introductory problems In free-fly mode started the practice session, and each time a different target aspect category was entere-' the first problem was in Tree fly. Tbe free-fly mode also was automatically entered If tbe student did not achieve tbe scoring criteria for the Sidewinder, pllit - .80, and also, above 300 knots, for the Sparrow, pllit
' .r)(). The Intent of this mode was to give the student an opportunity for free" practice, during which he was unburdened ol doing mental arithmetic,
lie bad the above graphics. In the enhanced trainer, which he could relate to the standard B-scan. It was called "Croc-fly" because the student was tree ol the burden ol computing triangle values and was free to "fly" the fighter without being scored. It was supposed to be an opportunity for the student to relax, and to take stock ol what he had done wrong in the preced- ii\y problem. This mode was supposed to reduce a disadvantage of the trials- to-crltorion logic for adapting to Individual differences; students are practicing "taking the test" all the time with this logic, which constrains the learning process to Intensive practice. No scores or other data were collected when a student was in free-fly mode.
tt will h. recalled thai adaptive control Logic Is used in the trainer* that is, each student progresses at his own rale and must meet successive trans it ion in;; criteria. The more able .students will work fewer problems to get through to the end than the less able .'■.indents. However, all students started with the .same problem sequence at the beginning ol each different speed level md target aspeei category.
A transfer test was given to all students from both groups when they were transitionod out ol the top speed level. The hypothesis was that those students who achieved an understandin}< ol the true and relative geometry involved in the Intercept tactics, i.e., who had an accurate mental picture to guide them -- would do bettor on i ii. transfer test. The characteristics ol the next problem coming up in the transfer test were not predictable from regularities in problem sequencing yet each now problem must be "sized- up" extremely rapidly and performance must be extremely Clucnl if success- ful Sparrow and Sid. winder attacks ore to be made in the short time available at the increased Irans for speed ol >i)li knots.
I I
NAVTRAEQUIPCKN 73-0-0065-2
figure A summarizes the differences In thi' Instructional sequences Eor Liu- two ^ronp.s used In the experiment.
Figure 5 summarizes scoring and transltlonlng criteria used tlirougliout the instructional sequence.
12
NAVTRAEQUEPCKN 73-C-0065-2
m t/. cu u H
O H
< 7*
*• < O
O H
Ui UJ Cd : 5 H Ui >< J ^J eJ M
O W u Ü PQ LO
u: a8 UJ 03
X
3 J"*.
g O Pi PL4
r-tp SB
V, C-, DO a; r Ti H
H (Xi Q H H w H UJ
g M X. >' ««sow o; <j o o -t; I ^1
H H to
?! LO UJ X. «H Q O in pu
Q Ii1
H H 9 k r 1 |J < ÜJ w i-i Oi UJ oi
„j U. P-i ( ) u *^~ U O HJ u
n i X X H Pä PQ 1 U) Cd HJ UJ UJ >• ?: a; -< w
< UJ J oä pe! ^ OH cm UH
[JJ H UJ Ui 1 >• Du -i: < n i H n o UJ O U ^-
o H & (i, O Ü
a: pj x. u, x ^
rr: Cd 1 ■ o U-. I-I <i; ^J oi 11 U •< UJ a UJ oi UJ
X w w a: Q ui U-' tn H ^J H KJ to >* t/1 UJ UJ L.'' .-J U H
O o " S3
< s i; Di
P-. M O |-1 Dd H X, K U
n g 2
UJ H
M Pi r-l H <
U ij Pu < w tj u c2 pi c2 x
ü t/3 H en a m ^ U) H CM H Ü ►"•
cü O *.^ UH
t-l Q
Q UJ UJ 3
H O
ä ^1
PH o UJ u, S
UJ S uj & uj o o KJ O CQ ca S -il o oi >• LO CU HJ <J
h ^ H 1 UJ U-I U UJ Pi o UJ UJ UJ Pi Du CüS n a Pi u< UJ i
o u UJ O Q o ro X. Cd Pi -i tn H Pn Pw H <
P-. X H H n H U Pi CM •-i
3 w C/3 gs 3
vD
10 V u C •1) U) 3 ^-< cr <u a) > in ai
*J i—i rt -a C (U O (U
« en U 3 C U <U (J > Ul (U C w
H Pi
(/I
Pi UJ X.
a UJ a x i X ui
X O M
H >• U J ■_J (H a 1 ü UJ pi u; H S
u, K-*
UJ x. in O H
p X us
'/J
ca c ai p-
u. H 1 u
UJ UJ UJ Cu Pi in [b < a H UJ UJ U t" X. Crt
ä < H
X.
u UJ 3 UJ Cd
In
\ /
C-I
M s u UJ y H Q/ H O
H
UJ X
Ö . DiX HO
I-I
x en H in F-l UJ
x u l/l U Pi
UJ UJ O TJ in Pi
r: x w
o X.
u UJ tfl
u H H <| H X in U
X^
U-. o Ul O ^o l-J O X.
'.J f^ X -r. H c^ o M cm Q a
U '-I
H O ,£ | H to
>
-t
m x.
a H < <
o oi o
H
O • OS
UJ pi
6U ca pi <; Id
H in i..
in *t >• •J ^J
X. pi X I-' Ul o
H Ul l-' UJ U Pi |J M U U H u
p- Pi H
r—1
< H H H UJ
•J\ UN •• PM \ • • M
u • > U H M U.' / M U
|| /
■X. PM >-i >< O
m Q
H
UJ id O Pi Pu
UJ Pü u.
o IU u X.
Si X. Ul
Ul -1
(O H
— ■
H C tn UJ UJ in H
s • . S OS o
/ gi g^ ^o H r-l
00
UJ H J in CQ 10 o td Pi 7\ a.
^ H CM UJ
X X Ü UJ I-I X Cd
»• 0Q
Q ^
" t: in
" tn < U: U -3 O I-I Cm ^J X in
cm i-1
X O UJ UJ Pi 5 o I-I P- X u.
X in
x M u,
ui O a. 3 JS o o U n) O UJ
.-l u o o U IM 4J C T3 O SJ U *J
cd
c a oi
U i-l C u
E U
41 "O a. o x S
UJ ■ u
•H • E
3 ai >, M a 3
iri vO
M^MM - ... -
NAVTRAKQll ll'CI'.N 7'3-c;-ü06r)-2
c6
H
■J :: >-* tA O u
o
>. o it -o
«) ^ 0 a. >, U r-l 'J u 'J c 01 -C u
0 to
(J OJ n ^ o u
AS en ^1 b J Q) u r-l
a. JQ 09 O in VJ
C u <U o •J. •M l-i u
I I
U)
a- 0) n)
<u oo
«J 4J
•a
^
oo O
o oo • A u •H X
OS •o UJ a p
oi UJ H I-J
O
o 2 H Pü O u
<t
(d
Q
o c E (U o
f—t lO
2A •r-l
u s: •H CL
iJ O
u ^
u '■J a gg n)
u <U 1- u o o
«1 n a» —<
O U 0,
a M M cd
n 01 d
in
•H •n tn <u u u D m
•a <u D. I/) ou
B U r-i
.Q c C •l-l
U a. e
ID o 1—4
•M J5 t; O m U c o. >. T) (1>
4J 01
(U IU c 4J •0
rt Q <u a> S
X. a. a) 0) >> u >J »—i
U-l •* •t i
a 0) 0) «i n J r-l f-i VH
<U IU •n
o s
01 u o ^-. 3 O
I
ou o 3 CM O eg
OJ ^i ^
u in
c o o >
•l-i 111
•1-1
in "O C O « QJ l-i cx H en
u 4J
■•-I
1J
C •r-l c c
—1
in C rt VJ H
•O c
00 c
•H u o tj to
m
o
oo •H
|/.
Ml ■■ .^MM J
NAVTRAEQU ll'CKN / !-(;-()ü63-2
^^ SECTION III
DATA COLLECTION, DATA ANALYSES, AND DISCUSSION
DATA COLLECTION
The ill-tails of Chi' procedures that won' used in this and the earlier field trial were described In an earlier report in this series,1 Initial storage limitations (HK of con), and the requirement for Inexpensive recording devices with relatively high reliabiilty determined the use of these procedures. During the time these field trials were being run, inexpensive floppy disk systems became available. These have been integrated with other interfacing hardware, so that all data recording and storage will be done in the future on floppy disks.
The data-collection was managed by a Navy Training Device Technician. In general, this was a feasible arrangement. The data were received from the field in the form of punched paper tapes. While each student was practicing with the trainer, a record of his performance was punched by a teletype (with a silencer) attached to the terminal. The greatest disad- vantage of this procedure is the tiim required to process tapes and to produce summary statistics.
Values of the following dependent variables, with the exception of frequency of use of on-demand keys, were recorded for both groups, both in practice and in transfer test sessions:
CABL DEPENDENT VARIABLE LIST
COUNTERS STATIC VAKIAIUI'.S DYNAMIC VARL\BLES
'■ PROBLEM NUMBER TOTEBOARD TOTAL [ATENCY LATENCY TO CC INPl'T ■ STATIC ATTEMPTS TOTEBCARD TOTAL ERRORS IATENCY TO FIRE, SP & SW STATIC COMPLETIONS USE OF TRIANGLE KEY* NUMBER OF TURNS
1 DYNAMIC ATTEMPTS HIT PROBABILITY, SW , DYNAMIC COMPLETIONS HIT PROBABILITY, SP STATIC ABORTS LEAD COLLISION ANCLE ERROR, SP
i DYNAMIC ABORTS USE OF TRIANGLE KEY* USE OF TOTEBOARD KEY*
-
INFLIGHT CHECKLIST
^Experimental group only. In addition, an attitude questionnaire and comment sheet was completed by each studcnl a) the end of the session with the trainer.
Rigney, ct a l()7i,
I i
J
NAVTKAEQUIPCRN 73-C-0065-2
DATA ANALYSKS
The values of the above variables won- used La the analyses. later mediate, summar es consisting of suras, sums of squares, and N's „ere List In the statistical program Cor visual Inspection. Subsequent analyses were done manually, using appropriate categories oJ these data. This .eUu^do sundry and analysis is Ceasible with relatively small samples (29 and 31) and has the advantage oJ allowing visual inspection of Inter e a o-level summaries to cheek tor errors and to identify additional
ways to organize the data.
Hard copy listings of the intermediate summaries, for each student for each class, and tor each of the two groups in the field trial wer made and are available for inspection. However, in the interests of saving -space, all ol these data wilt not be reproduced here.
TRANSFER TEST ANALYSES. It: will be recalled that the end-of-course transfer test «as designed to present the students in each of the two groups w an unpredictable mixture of (2,)) air intercept problems, at a greater diffi- culty level, as determined by 40 knots higher speed (500 knots) The objoct.vc was to test the students' ability to cope with unexpected varia- t on In nitial problem conditions: initial positions, headings and ranges of the bogey; and widely varying initial target aspects between bogey and fighter. All students took the same twenty problems, but in different random orders.
sli.h!LLiS ^^'/T tlU' data ^ table 3 that the enhanced group did verv si ghly better (not statistically significant) on number of problems correct out ol 20. and on both missile (Sparrow and Sidewinder) scores Some explanation ol the variables listed in table 3 is in order.
"Static problems-refers to the first part of an intercept problem in which the student must mentally compute values of six variables for the intercept "tr.angle " He must do this within 60 seconds and without erro- lor the problem to be scored correct. In the transfer test, the student
thesecfite^ia ITll*^* ^"T ^ thOU8h ,,,■ mi*ht not achieve these criteria. U he nude an error Ln arithmetic, the remainder of the intercept was scored Incorrect. if he made no errors, but did not finish
rect'T^ h "ifthln 6,) BeT6B' the ,,8tatic Problem" was «c—' l-ncor- rect, but he could go on to achieve a correct score on the subsequent (dynamic) part of the problem. There, a pHit score ^ .50 was required for
or tho'sid """V ' f aI" Tl' LeVela aboVC ]tM k"^' and a P»i score
part of tttn tV ",,SS;1" "S,) at a11 Bpeed Levels* Eor the sSbsequent part ol the problem to be scored correct. The two missile firing scores were set at different cute.is because of the differences in relative dmi- culty as judged by an experienced aviator, in making these attacks. Tor example, due to the high closing rate of the Sparrow attack approach tactic
ig^res I«? V"^ S"U,,, WindOWn lnr ^ 8tUdCnt t0 Elre t '1^, ^
Latencies to finish certain parts of th tie problem were recorded in seconds. .Number ol turn:. Included the turn to collision course and all
If,
MMaaH*>. .
NAVTKAKOll ll'CKN /3-C-Ü065-2
*i
v>
H to
H
OS U4 Pi
SQ o UJ
en m g o O pi LO Pn
Pa sä & ^. o u
o
3,> (^
u a
II ?5
a UJ u u a u oi
■J
n ii
Q
U
C/) U4 i-J to < M OS
o
00
IÜ
ra O cu u
UJ U OJ H c^; H o < '-> H to ;;
o
o
C1
o
o
o
o
-1
(X) o
o
o
o
o o o
o
o 00
o Oi
^D
in
in
o ri
O
00 CM
00
in
o tn
00
<r
o
-t
CM in
in
o
I--
in
^J CQ O Pa CU H
U U LU M Oi
a O p U
Q ;;
O Q
■ CQ U UJ UJ H tn o w H
UJ
& UJ :_) i-i <
o H
U U: UJ O to Di ^ S
a ryi x. u w H Oä
O H
UJ Q H
fJ U ^
Q >* H CJ t/l
ui w [~-i a; < H J Ui
o
o
-t fn C-l av ON .—i <f ON in O 00 <t- ^O m m ^-1
<t ^D f-1 <r r-l i-~ -t ON
-t Cl 00 <j\ vD vD r^- in
00 CM 1 ON t-J i—i O ON
vO OJ Cl vD ON 01 ni cn r^ ON CM O tn ■d1 en in
o CM
00
O ON
00 ON
ON
O
ON
cn NO ON
to >N
1)
CM
X
□S UJ Q
os o h-l UJ rf. UJ g UJ £9 |
tf. os vc < o Q o (X, u H ^J
1 ' (/i w CO tn
c c
^3 3
0
0i
a E
CO
17
—^^— ^ c
P"' I WWWBppWWiP^^PiWWWI'^f'WW^'i'PW»™^ '|,I,|:>J«MI,IIHIIJWHI.I.«I.
NAVTRAEQULPCKN 73-C-0065-2
subsequent turns. The requirement to fire a Sparrow at speed levels from JÜO knots up added to the number of necessary turns.
- >-
PRACTICE SESSION DATA ANALYSES. The sequencing Logic for the practice problems was described in figure 4. Basically, students in both groups^ bad to continue practicing until they achieved the transitioning criteria [or that mode (Static of Dynamit) or (speed) level. They then were auto- matically transitioned to the next mode or level. As explained above, the "enhanced group" was exposed to several graphic instructional aids during introductory sample problems, the first stage oC the static mode, when they made computational errors, and while they were repeating entire inter- cept problems that they did not "pass" the first time and any other time on demand. Analyses of data recorded during the practice sessions are summarized in table;; 4-12.
Frequencies of various events are summarized Ln table 4. Some of the variables require some explanation. Again, number of static problems correct refers to computations of the values Cor the six triangle variables in both the special static mode and in the subsequent dynamic mode. Number of dynamic problems attempted includes the last part of all problems after the static mode. 1'roportion of problems repeated refers to the fact that, if scoring criteria were not achieved on the dynamic, part of a problem, that problem was automatically presented in the free-fly mode. For students using the reduced trainer, this meant flying the problem again without having to compute intercept triangle variables, but with none of the other graphic aids. For students using the enhanced trainer, the problem was repeated, too. But in this case, the dynamic triangle, the optimal path, and the bogey track history were present throughout the problem. Propor- tion of problems aborted refers to those cases in which a student was inter- rupted during a problem, or he hit the missile firing key before 64 seconds
into the problem.
It is noteworthy that students in both groups aborted a much higher proportion of repeated problems (.12 and .10) than they did new problems ( 03 and .02). Recall that the students knew they were not being scored on repeated problems, which were in the free-fly mode. Evidently they often felt that they had extracted enough information at some point in the repeti- tion and decided to terminate the practice, or they did not want to repeat
the problems at a 11.
It is apparent from the data in table 'i that the students using the reduced trainer took fewer practice problems and achieved final transition- ing criteria in fewer on-line sessions than was the case Cor student;; in
the other group.
Also, students in the enhanced group generally did slightly worse than the other group in terms of proportions of problems attempted that met scoring criteria, i.e., that were scor d as correct., It should be recalled that the transitioning criteria added to the scoring criteria the requirement to successfully complete .two problems in succession in order to transition to the next speed level, or at the end, to transition to the
18
■ ■—^—^, — .......— ..- MM— . •
P;l»JJM«!iyjl»WHIl»fWBW»>'l"^W,™wiH»'l,''"lllll!»,iPlVWW-1"i"--l.11*1 .vwfmwriirt ■r,,...™TT„.,„^,,>,j,,i> un.ipgiiHi. in.ji w.^jmimmjinjugmii WHII.P. iJ "■.vmi
NA.VTRAEQI! M'CKN 73-G-0065-2
w
■it
u
Ö
S
IS
o M
cn u in
ui u H u
m 7a o to M
<; fct *■<
o u
o Da o
PQ
H
u
pd 3
n. CM o o
iXI
^ Q
II V.
Q UJ U
^> p !—< t/i ro II Xi ^-^ Q W u J -^ 2 5S s <: t^ a
00
u H ui ui P > c w Ul pei IM C o
o
UJ y, M
I !« o Ui o
CM
O
-1
m O O
o
CM
^O
lO
00
o oo o o
-r
t/)
Q UI H Ea ü H H
Pa o Ul M CO OJ S c/i Li Ul ^'. 1/1
HJ .-1 tl pa 1 ' o o o; H O«
CM
1^
oo
m
O
cs ON
o
o
m
u H M U l-i U
H c/1
o 5/3 Pd ui UJ hJ
S o UJ Pi
m
I--
1-1
00
CM
O
O
in
O
o CO
Q U Ul H H
;'. 1.1 H
a
o '/I
UJ KJ
>X3 -X) O
1—1 r-l CO
CN rn 0 c-l
o>
-t M
in
01
in m c^i
r~4 O
01
ON
vO CO -t
1— PI O
U u H H
1^ hi ! 5 W, >" Pi n o
u IM
o ;/}
a; ui w ^J m cn ;•; o ;_) pi •.'. P^
M H ^ H H U a u <! UI a ui H Pi >. pi CO Pi 0 a;
0 0 X. u sa u 0 0 M a) M tn (-1 H H H
^ g^ PL. Ul CU UJ O H Pi H SK
H p
S H
Q CM
Q U
P H UJ Pi H O
ui << CM U) oi
0) 0) tn
CM
U
Pi KJ
P-i -a! P-i <!
O H W H S pi UJ O ^J P-i m O O Pi Pi PJ CM
00
a
O cn H O H Pi Pi CM O CM <-N O U, Pi U, CM --'
a P a UJ
>i Pi P O
PQ V. < O H c/l H S Pi Ul O ^ P-i CO O Pi CM CM
i
o •H
3 D. O
CM
CM
a, §
WJ
I1»
A MMMMi .—drntflMftb M. HUg^ä
VH. im .USIIIBI! i i i i.pwa^rvwv >*^WPiqiamiP^(«9i^i^^nwnwuiiiii 'imk-ip«»!! ■■iiiu»"F"""»i"". i ■• irnmrrnifmimfn^miiw^im^m^"'"'' \iiivm.mi«mn.**w^q
NAVTRAEQUIPCEN 7:5-(;-0065-2
transfer test. In table 4, number of problems correct La based on scor- Ln^ criteria. It will be seen later, in table 12, Lbat students in the enhanced group had a slightly higher error rate on the toteboard, through- out all practice conditions, 1 i' a student made a toteboard error during the dynamic phase ol practice (220 through 460 kt), he was allowed to Cinish the intercept, and his turns and missile firing scores were included in the data. Uut, the problem was counted as incorrect. Under these conditions, the higher toteboard error rate of the enhanced group could have required this group to attempt more problems, even though their proficiency scores, turns and missile pllit scores were as good as the other group s.
The comparative analyses of elapsed times for various events are s umma rized in tab I e ri.
The various response latencies; to complete the toteboard (mentally compute six values for the intercept triangle), to fire the Sparrow, and to fire the Sidewinder, are elapsed times, in seconds, from the beginning of a problem until a student achieved these conditions.
These latencies are means computed over all problems, over the static and dynamic modes, and over all seven speed levels. Thus, they might bo regarded as summary fluency scores for each group.
It is apparent, from the data in table 5, that the enhanced group re- quired an average of 2.15 hours longer to achieve final transitioning criter- ia (^60 knot level) than did the reduced group (12.51 hours vs 10.36 hours). Means and standard deviations of these values in hours are total elapsed times students spent on the trainers, as recorded manually in a log. Thus, these times are not as accurate as those in remainder of the table, which were automatically recorded in minutes, or seconds, by the computer program. Differences between response latencies to complete major parts of a prob- lem were very small, with the reduced group being a few seconds quicker, over all.
The summary statistics in table 6 are overall values of various indi- cators of accuracy of performance. The probability of hit scores for firing the two missiles have been described above. The algorithms for computing these scores take basic parameters into account, as diagrammed in figure 2. Because firing the Sparrow accurately is a much more difficult problem for the student RIO than firing the. Sidewinder, the cutoff score was set much lower (Sparrow, pllit = .50; Sidewinder, pllit = .80).
The lead •ollision course error refers to the error in leading the bogey when "aiming" the Sparrow. The absolute error is presented in the table. Leading too much or too little is not differentiated here, although it could be of tutorial importance to do so.
Number of turns during intercept, like all the measures in these tables, is a summary mean over all speed levels and target aspect angles. It is a measure of the precision with which the student could "fly" the inter- ceptor under all these varying conditions. It includes the initial turn to collision course, and in this respect differs from the measures ot the same variable reported lor the earlier field trial of this trainer.
20
- ■■■--— ■ _. .._ ^itJ
*B»!*fiaW«l!PW>*WI!"WCTWI5S«WW^»'7l»«^»W«Wl IJWfBWW.lTOlMW.^J^WUUfflWMUWl"*!!»^^ lfP^lJ)lüpjp|iuuliliii
NAVTRAEQUEPCEN 73-C-0065-2
CQ
t s CO
CO
W CO
Ul u M H u
g o to
o CO M
Oi t; & »-. o u
o
H
CM
o
CO --x P vO a> to • CM Ol II
Q UJ ^O u ro UJ X. o < O W Ixl ,—i 05 r.
ii
o u X.
CO
CO IU
111
H -r; , i
ui CO
b to w PC
CO
r-4
li-l
ro
L'l
oi
cn
-1
CO
o CM
o
O CO
-I
-+ m
in tN
-t
o
\
(J
l-l M
u fl H I>I ' ) i-J
^ pn O
n. Pa Hi
o H
H lii
UI .-I
b u
CJ
111 H
00 00
CM O
m
CM
m o
^ E
ui
o
o 0^ iri -t vD vD CM O ro r~l
O o o-. r-l a^ o o ^D ^D r^
-.-t i^ O 1-1 m in o O O \Ü
CO r-l T m r 1
u UJ CO
O OS
p-l CO
ro in
m
o eg
in c^
in O CM
UI Pi
p H
t_) i-i
Di P
tn
.-i
a)
a) UJ in
C o •H 4J
D a. o
d to to
2]
MMHMMI laMMMMMMMaiaMuiakt MMMMHIM
pBilu.ui.^iiwiu.u^HSlBJMiWtU^ - •ii,""«"i>«mBtii>«j<.i",';!'".i ♦Ni-ut«! ii,j»A.»w»oeiw»J'i»!^,».^'«.,i-,»l,-i'"^^i'- ■ ™.u».'""i"'"T™J"
NAVTRAI'QU LPCHN 73-C-n06r)-2
< H < O
X rn o U H F^ i/i HJ LO CO
1 ^* UJ Ü >" H u H U ^ <Tl : i S! u PU u
< o DC
tn g SB H Ü o m M p. K! < ■11
h -■. o O X Ü H
f«
?* t-i Ll IJ-. o
a M
W • I/! VO M
-■.
UJ hJ pq
"^ 3
U
s-4 tn
b
,-* Q ON t/)
II
Q UJ u i—> ;'. Q < ui w
II
u
a
to iii Ql 1/1 O UI U pi
u ui
u S
o
o
as o üO
CNI
00
oo
o vD
O
O
O O
ID O
~t
o o
o o
CO
in -t
(N
oo CO
UI OS
O Qi a;
UJ
o u tn
H ^-| I
a-
q < UJ
M en ^-N oj en Ui ui O UJ u a:
O 55 uj o a
o OS
PU o en u
K o (1-, S uj
UI cd
UI CÜ Cd UI o p u
H
S3
W fl4
O
o
CM
O
c o
00
o
cä
o Qi UJ
Ö
H PM hi u p!S UJ H
O
o
o
in r^ r~* in ~^ in in ^O i^ ^t in c^ Ol ev| 00 00 CO ir. t». ^D
CO Ol rT o uo 00 rn o> in o ,-J i-^ o rn -t
-1
CNI CO
m
M H UI ^J & 6 U
;'. Q (—t rrt
5 (/) a OS Cfi o u: oä H DJ o IJ H
J
(Ti
en
QJ IU tn
CM
3
o
3 a. o tu
p«
a. a
en
22
M^M -■ I mi- h - -
»Wi|pi«ipip|l«l«!PP!!"Bi»!llll!!fWP>w»™~mfW»^w!«W!iW^^
NAVTRAEQDCPCEN 73-C-0065-2
, Errors Ln completing the toteboard concerns Liu' frequency oL" errors students made Ln Computing and entering values ot the six Intercept "triangle" variables. The frequency of these errors did diminish with practice, but some students in both groups continued making an occasional error throughout the training. It was our experience Ln the earlier field trial that many of these errors were due to the inability to tell left I rom right instead of to computational failures.
The most obvious conclusion to be drawn from inspection of table 6 is that the two groups were very similar to each other in terms of these overall proficiency measures. The enhanced group did make more toteboard errors as Indicated by the means. It will be seen that the distribution of these errors, shown later in table 12, is of importance for explaining the results.
TRENDS ACROSS SUCCESS1VK PRACTICE CONDITIONS
Strong evidence for the effectiveness of the trainer as a device for teaching basic skills was presented in the report of the first field trial. Similar evidence is available from the current field evaluation. Pertinent tables are presented below. in all cases, the trends across the tables, from left to right, reveal the effects of practice, on low, medium, and high target aspect problems, driven by increasingly higher problem speeds, from 220 to 4()0 knots. In some cases, lower and upper bounds of response latencies are Influenced by the speed of the problem. However, in the case of the mental computations of values of the six intercept triangle variables, it is always to the advantage of the student to get this mental chore done as quickly as possible, to avoid getting behind the problem.
Table 7 presents total latencies to complete the toteboard, for all the values ot the six variables. As in the previous field evaluation, there was a marked reduction In means and variances between first and last practice conditions. Over all target aspect categories (bottom row), the mean latency was reduced by a factor of 2.30, the standard deviation by a factor of .5.22.
'Uigney, et al., L973.
2:)
■MM. - "—
'■•"■"I" ^mmm^m^^m ■•" 'mmwmmmmmmiiii i\m\.i m m II mm,,. I >• I • .m,,,„mm:, ■■ill
NAVTRAEQUEPCEN 7i-C-ü()6,j ■-,-•>
TABLE 7. OVERALL lATLNCY (SEC) (MEANS AND VARIANCES) TO COMPLETE TllE TOTEBOARD
AS A CONSEQUENCE OE PRACTICE (N = 60)
.
TARGET ASPECT
STAT1C I II 220 260
DYNAMIC 300
36.02 22.73
LEVELS 340
SPEED 380 420 460
ROW TOTAL
M LOW SD
7l).74 64 .66
39.33 22.32
50.4 5 31.21
37.64 16.11
30.32 1 1 . 14
26.03 9. 30
27.37 1 7. 54
24.90 10.17
41.71 36.26
MED «D 61.32 32.71
51.22 26.60
49.33 28.34
44. 34 18.76
37.32 17.80
32.88 12.30
36. 18 18.25
32.52 15.94
28.33 9.40
41.64 23.77
M HIGH SD
57.30 28.49
61.25 35.21
56.69 37.22
4 9.36 23.31
40.04 18.66
35.66 17.08
35.38 16.33
34.91 10.97
36.29 21.05
44.07 25.74
COLS M IOT si)
67.96 49.00
51.29 30.14
52.16 32.45
44.12 20. 34
37.54 20.32
33.10 14.02
33.14 16.07
31.98 14.94
29.51 15.22
42.49 29.39
OL
!■ I I
timm^mmimmii&mmmm&immiW- r' n u ''"■ i>mMijiHPi-inipi.wivJ<w " t" l'- IHIH'"'»1 ~ !■■ • Ji' .'.WM.IIIHTOWIPSH'JUH1-»«« '.i1 ■ -1 JILWIUImil . .'I mi mwitm^fm^m
*W»^'W)-,-.~
NAVTRAEQUIPCMN 73-C-0065-
The rLsf In this Latency «P
at Liu- 220 knot level Indicates UK- student's first experience with the u-sc;m ami the requirement to koep up with the problem. The highest target aspect problems (second 1 row from the bottom) evidently presented the most difficult computations, possibly because some oi the numbers Involved were larger.
We see similar trends in the data in table 8, where frequency of tote- board errors is summarized. Students tended to make more errors in high target-aspeeL problems.
Scores for Tiring missiles obviously would be Influenced by increasing the speed of the intercept problem. It becomes more and more difficult for the student to get his aircraft into correct position for firing before it is too late. Nevertheless, the pllit scc.res lor firing the Sparrow and the Sidewinder, as shown in tables l) and 10, do indicate some improvement, in most cases, despite the increasing speeds.
TABLE rf. OVERALL ERRORS IN COMPLETING T0TEB0ARD, ACROSS SUCCESSIVE PRACTICE
CONDITIONS (N - 60)
TAKC1 ASPFX
:T
;T
STATIC 1 1 1 220
DYNAMIC LEVELS: 260 300 340
SPEED 380 420 460
ROW TOTAL
0.50 1.27 LOW M
sn 0.1)2 1,66
0.5^ 1.29
0.79 0.41 0.88
0.46 1.25
0.27 0.84
0.21 0.5 5
0.2! 0.80
0.33 1.21
ffiD M sn
0.83 1.37
().r)i)
1.04 0.55 1.15
0.54 1.00
0.44 1.04
0.26 0.72
0.59 1.77
0.35 1.20
0.31 0.83
0.50 1.19
IIC11 M SI)
0.73 1.26
0.77 1.55
0,70 1.42
0.62 i.l1^
0.40 0.87
0.47 1.26
0.4 5 1.42
0.33 0.95
0.51 1.26
0.54 1.26
;OL CO 1
M SI)
0.8'. 1.30
0.63 L.46
0.70 I.OA
0.53 1.09
0.43 0.97
0.34 1.39
0.4 3 0.()8
0.30 0.98
0.38 1.13
0.51 1.25
25
mmmitmmmaam^^mm^^^_ ■^■tM^fcrti^^nf irdu1
HWPP*l«W»W«WPi"J«P»l'»"'»"l^w,^',w ' "" ■i">P^^I»W«!>iH"Ji i HIMIW' ti>imrffmii*m**'*m^ i i ■ i mimmmnv.i^m »iiuikHiJimjmvipi
NAVTRAEQUIPCEN 7 J-C-0065-2
TABLE 9 TRENDS IN SPARROW P-IHT SCORES (MEANS AND VARIANCES) " ACROSS SUCCESSIVE PRACTICE CONDITIONS, (N - 60)
-
TARGET ASPECT 220
DYNAMIC 260 300
LEVELS: 340
SPEED 380 420 460
ROW TOTAL
LOW !JD 56.07 40.70
58.27 40.32
65.13 37.76
60.11 38.94
60.94 40.42
59.95 39.77
M MED SD
69.98 33.85
68.5 7 32.90
63.91 36.38
61.79 36.99
71.09 31.96
66.96 34.63
M HIGH SD
66. % 34.25
58.65 40.01
59.53 39.30
53.36 41.82
56.89 40.13
58.61 39.59
COI. M TOT SD
63.49 37.30
62.17 37.91
62.65 37.88
5 7.87 39.70
62.59 38.45
61.70 38.33
TABLE 10 TRENDS IN SIDEWINDER P-HIT SCORES (MEANS AND VARIANCES) ACROSS SUCCESSIVE PRACTICE CONDITIONS, (N ■ 60)
TARGET ASPECT 220 260
DYNAMIC 300
LEVELS: 340
SPEED 380 420 460
ROW TOTAL
L0W SD
74.17 40.68
88.22 29.99
88.13 30. 16
90.13 28.33
90.63 26.99
87.35 31.03
88.33 29.45
85.90 32.40
M MED SD
81.90 36.05
93.56 21.80
93.72 21.38
93.35 22.87
88.13 29.58
92.50 24.30
95.38 18.71
91.06 26.10
M HIGH S|)
86.48 31.21
89.35 29. 10
94.75 18.90
91.91 25.24
84.34 34.88
92.13 25.26
91.57 24.72
89.90 27.93
COL M TOT SD
79.90 37.16
90.46 27.16
91.69 24.96
91.91 25.37
87.28 31.23
90.82 26.89
91.33 25.44
88.88 29.12
26
-
NAVTRAEQUIPCEN 73-C-006I)-2
[t should l).> noted that students could firo the Sparrow bolow 300 knots, and a Eevi iliil this, but to bu consistont with operational pro- cedures the program did not count these scores toward transltlonlng to the next level. I'-llit scores tor firing the Sparrow decreased from 300 to 460 knots lor the high target aspect problems, :igaiii an indication that this was the most difficult of the three target aspect categories.
The high target aspect category evidently was not markedly more difficult: than the low and medium categories, in the case oi the Side- winder, despite the Increasing speeds. There was a steady improvement in scores across the successive speed conditions.
The trends in number of turns are shown in table 11. The biggest improvement occurred for low target aspect problems. These low target aspect problems also were the first to occur at 220 knots in the instruc- tional sequence. These trends corroborate the evidence in the preceding tables for the effectiveness of the trainer.
TABLE 11. MEANS AND VARIANCES VOR NUMBER OF TURNS USED TO HAKE AN [NTERCEPT ACROSS
SUCCESSIVE PRACTICE CONDITIONS (N =-- 60)
s
TARGET DYNAMIC LEVELS: SPEED ROW ASPECT 220 260 300 340 380 420 460 TOTAL
- 1 7.02 5.86 5.65 6.22 6.14 6.06 5.82 6.15 3.10 1.94 2.03 1.64 1.40 1,69 1.45 2.11
*JD M 5.47 5.80 4.in 5.67 5.58 5.49 5.01 5.51 ,u, SI) 2.02 1.95 1.50 1.41 1.5" 1.56 1.15 1.66
M men 5.77 5.28 5.26 5.72 5.89 5.77 5.54 5.63 iiu.M SD 2.17 1.71 1.38 1.62 1.86 1.52 1.72 1.73
COL M 6. J7 5.63 5.33 5.85 5.85 5.78 5.50 5.77 TOT si) 2.63 1.88 1.73 1.57 1.67 1.59 1.51 1.87
27
aaM^ataaM—„_
i>PPiM«VHW^^nmiw«m«p««RP>>pinRW«nT^mimMpmqiinp^riHniPViaP-> 'MIL iwm^^^pppiwpiiiwwip^^^^PliiiBWPPiinp^iflBwi^^Piiijwi"1 «.tu*1 ' - i «.uiu I.III,II.,I n*
NAVTRAEQU CPCEN 73-C-0065-2
COMPARISON OF TRENDS BETWEEN GROUPS
28
1L is Important to examine trends in each group, to Identify any differences between groups that might be attributable to differences in treatment. Table 12 summarizes tills Information. There are very few differences between the two groups with respect to turns and pUit scores, in terms of trends across practice conditions. The enhanced group did attempt a few more problems in every practice condition, yet their scores on the above proficiency measures were approximately equivalent to the other group. These data also reveal noteworthy differences between the two groups in terms of computational errors in computing intercept triangle values, in the static mode and at most of the speed levels. The overall mean for the enhanced group was .57 (SI) ■ 1.32) versus .44 (SI) = 1.13) for the reduced group (Table 6). The significance of this is as follows. During the static phase, the display of the static triangle may have been distracting during the performance of mental arithmetic. During the dynamic phase of the practice, 220 through 400 knots, it will be recalled that students still had to compute intercept triangle values. Then, they went on to "fly" the intercept, in the second part of the problem. If they had made a computational error, they were allowed to finish the intercept, and their proficiency scores (number of turns, missile firing scores) were included in the data. However, the problem was not counted toward transitioning to the next TA category or next speed level. Nor, was the problem repeated in free fly, nor was it counted as correct. Therefore, this slightly higher error rate in the enhanced group could have been a major reason for this group having to do more practice problems and to take more time in the practice session, even though the proficiency scores of the two groups were almost identical at all practice stages.
The latencies to complete the toteboard, in table 12, also are note- worthy. The enhanced group was much slower in the first and second parts of the static phase; 74.25 sec vs r)8.lJl sec and 55.72 sec vs 46.01 sec. A possible explanation for this is again, that the geometric display of the static triangle was actually distracting, so far as the mental arith- metic task was concerned.
■ - ■ iMim mi i wmi\
BW,- "- ,i 'I' i -I IIWH •« II'WmMIHWltgSWPPB -Tw^..,-- .. ,. ..«,,.v. Ml<^ ■•.>M!l«l)< •< ■ V, ^ « <■ » ll.pi «l.««^«.!.!.)«!!. ^-^"^1 ■ ■ ■« •
NAVTRAEQUIPCKN 73-G-0065-2
TA15LH 12. C0MP4LRIS0NS BETWEEN ENHANCED (!•:) AND REDUCED (R) GROUPS WITH RESPECT TO TRENDS [N SUCCESSIVE PRACTICE CONDITIONS
(AVERAGED ACROSS TARGET ASPECT CATEGORIES)
NAME OF VARIABLE
LATENCY TO DO TOTEBOARD
COMPUTATIONAI
ERRORS IN TOTEBOARD
TOTAL NUMBER OF TURNS
SPARROW PH1T SCORE
SD
SD
M
SD
SIDEWINDER iM!IT scoiu:
STATIC PROBS
ATTEMPTED1
COMPLETED2
DYNAMIC PROBS
ATTEMPTED3
COMPLETED^
SD
SD
STATIC SPEED LEVELS: DYNAMIC PllASE
M
1 1 2H) »60 JOD 340 380 A 20 tbO
74.2rj r)r).72 r)).2(i 42.95 36.37 33.10 33.49 32.47 2').75 r)«.Ml .',(,.01 SO.21 45.81 39.08 33.10 32.01 31.53 29.21
53.52 33.98 34.02 21.82 18.9] L4.9] L7.63 12.33 14.28 40.03 23.89 30.1b 17.97 21.1)7 12.63 13.29 17.01 16.37
.93
. 71
. 5')
.1)3
. 76
.48
1.51 1.00
. 73
.67 . 64 .36
1.38 1.1'' 1.58 0.75
.4/
. 5S
1 .08 1. 10
.34
. J3
1.08 0.80
,49 . 34
.31
.29
1.66 1.02 0.82 0.95
.39
.37
1.28 0.92
6.15 6.64
2. 34 2.97
5.60 r).(uS
1.78 2.02
5.86 5.82
5.80 5.93
5.71 5.40 5.84 5.64
1 .55 1 .95
1.63 1.66 1.48 1.68
1.53 1 .66
1.40 1.62
63.29 61.05 60.94 57.33 61.56 63.74 63.66 6 5.12 58.38 63.83
37.16 38.04 38.46 39.94 38.97 37.64 57.74 36.97 39.62 37.95
79.87 89.12 92.08 91.83 86.10 91.40 91.28 79.93 92.37 91.19 92.01 89.09 90.29 91.39
J7.09 29.07 23.88 25.80 52.49 26.48 25.07 3/.32 24.16 26.35 24.80 29.17 27.35 26.00
(MEAN NO. OF PROBLEMS PER STUDENT) 20.93 12.80 19.06 13.97 18.13 18.32 20.29 15.45 15.64 15.38 11.18 15.10 10.38 14.86 13.65 14.00 17.93 13.37
8.26 7.45 10.52 8.84 13.13 14.55 15.52 12.84 12.35 8.27 7.48 8.76 7.07 11.10 L0.69 10.96 14.65 10.37
18.42 13.87 16.97 17.77 19.77 15.03 15.35 14.89 10.31 14.07 13.45 13.86 17.45 13.10
8.58 7.94 7.,>() 8.10 9.16 8.23 7.87] 7.90 7.07 7.66 7.76 7.66 8.79 7.51
To Luis AttcmptiHl E = 4793 R - 3662
'Totals Completed E ■ 3207 R = 2595
Totals Attempted E = 36 33 K = 2817
Totals Completed E = 1791 R = 1576
29
«MM __-—
•WIW|IBW»B"WBB>P»»"»1IHH.»I-I"J,» HP.»* il!PII»P(,P!»IIIWWm^"»BB«!WII,l ■ I J.I "i II! "LUIUIIIIIUJIUJIUJJIJI» II IIIILI^ JIMJI,JJ4UII1|J,JIJ1IIMIJ!II1,,WJ1I,"..WJU.'JIIJIJ'WIW''1 ' < ' H*W M J,1UM«»IUII.«W «
NAVTRAUQUIPCKN / i-(:-()()()r)-2
STUDENT ATTITI'DK QUHSTIONNMRES
h'ssontia 1 ly Liu- same questionnaire was used in this stiuly, as was used in the first field stiuly, with the exception of several questions not applicable for the reduced group. The results of questions common to both groups arc reproduced in table 15. It will be recalled that this was a forced-choice instrument with two positive, one neutral, and two negative categories for each item. There were ten items common to both groups. Since only 23 out of the 2') students in the reduced group res- ponded to the questionnaire, versus all 31 students in the enhanced group, proportions instead of frequencies are given in the following tables (13, 14, and lr)).
It is apparent from an examination of table 13 that botli groups had generally favorable attitudes toward the trainer, as sampled by those questions. However, there were more extremes in attitudes among the reduced group. These differences are shown in table 14.
In addition to the items common to both groups, five items on the questionnaire were specific to the froe-fly mode, in which the special graphics described above were available only to the enhanced group. The responses to these items are tabulated in table 13. In addition, one questionnaire item required a yes - no response:
"In free fly you could observe the relationships between intercept geometry and radar presentation during an intercept. Did you use this feature?" Of the 31 students, 23 responded yes and six responded no.
These data indicate a preponderance of favorable attitudes toward this feature of the trainer. (Kolmogorov-Smirnov one-sample test: p<.01).
The attitude questionnaires used with the two samples are reproduced in Appendix A.
BIOGRAPHICAL DATA
Certain items of background information were collected from the stu- dents. Although not all students supplied every item requested, scores on standardized tests and prior courses in the Navy were available from the Naval Aerospace Medical Research Laboratory, Pensacola. These data are summarized in tables 16 and 17. 1t is apparent that there were no signifi- cant differences in the data for the two groups.
INFLIGHT CHECKLIST DATA
The checklist devised for use by instructors during field testing of Model 1 was revised to make it simpler for the instructors to use. A copy is reproduced in Appendix 15. The data from these checklists, which were used eleven times on each student, five during initial flight training (RT-8) and six during more advanced flight training (RT-13), were converted
30
■•»S.^«« »'"».Uli ■ iNiij^iiiWp4Ji^iiW»ipi^.!iMl.ilBpJ^iliiJIiHLHy i-N^iJwi^i^i*»«"«"«" i '■> i ■ ■ ■ "^^^r^mfm^^prrprmi^ifmi^w^'^^mm^^w^'^^^v^^m^vm^m^mi^mi^w^nFfwmr irtm «.rniviiu^niit
NAVTRAEQUCPCEN 73-C-0065-2
u
«
Ö H
a /-N
pä X 3 ;■: h o o H
</i r-< V) •.-. CO IÜ o P H II 5 H H K 5a M O H P4 H o • • < a: p-.
P4 _> g O
w OS U Pi CO n 5 i-> (U H en PU cn u Ü
u • • b KJ 2 o <J F ^ .-. C/3 UJ H hJ LT: HJ
>■ u J u ^
H
s
vO
U
o o H <c U
UJ CO
6 to
Qi
o r—4 o I-- ■n v£) OS (^1 c-O m O o
in o o
UJ HJ CO <• p^ o UJ
I> —i < 03 u^ < (-J ►l o et; ^. IU •Ij
H
p
i
O
0> r 1 cr> O O cNi - r '-' '-' o
m (^i in O O ,—i m m o o
CT« in m o ^O c-i m r-l .-J o
ro av r-J rn Cl O Cv| m r-l O
m m O^ O O cvi cn CM r-i O
vO O) ■X' vD o r- in (N o
j3 r-j m m <n O m n o O
m co o O tn .-H in C^J O O
rn «"i in o fn ^i - T CI O O
UJ
03
o ^>
OA UJ
m C^4
ii
Z
O
Q UJ u l-> Q UJ s
UJ
OS
oi UJ
UJ H
>> oi O
UJ H < u UJ to
Ö p-l t/l UJ 02
/, £1 ^D -O O >£) er« m J^ oo ^-1 m rj r-l o
O 00 CM O O O fN in rj o
O ^J vO ^ o i^i in ■—! '—, O
CO 00 VO 00 00 o o •-, o o
CM 00 00 ^ O m <N O <N O
o oo <f o CM <f CM O
O vD -t -t C ^t m tM ^ ^
'O C O CM O O c-J CM in o
00 O -* 00 o CM -t <N O O
oi -4 (N O m CM --i o
csj rsi -,t CM O r-i in CM '-' O
03
Ul
Di u:
03
O
% H
UJ
UJ J 03 i o
b
03
UJ >
V ii
en c cd a 2 «4-1 o 4J (fl (U H
0. B ca to
<u c o > o c u
•rJ E
o 1-1 o 00
s
n
■M^««. niiiMrim r"—"■
NAVTRAEQUIPCEN 73-C-0065-2
TABLE 14. DIFFERENCES [N MEANS OF PROPORTIONS OF SAMPLES EXPRESSING ATTITUDES TOWARD THE TRAINER
VF F N U VU
ENHANCED GROUP .135 .380 .361 .090 .027
REDUCED GROUP .168 .396 .256 .196 .080
DIFFERENCES -.033 -.016 + .105 -.106 -.053
TABLE 15. ATTITUDES OF STUDENTS TOWARD THE FREE FLY MODE IN THE TRAINER (CELL VALUES ARE PROPORTIONS OF SAMPLE)
(N = 31)
RESPONSE CATEGORY ITEM NUMBER 10 12 13 14 15 MEANS
VERY FAVORABLE .35 .29 .23 .23 .13 .246
FAVORABLE .42 .45 .32 .52 .32 .406
NEUTRAL .19 .19 .39 .16 .45 .276
UNFAVORABLE .03 .03 .03 .03 .10 .044
VERY UNFAVORABLE .00 .03 .03 .06 .00 .024
(Kolmogorov-Smirnov One-Sample Test of Means: p =<.01')
32
a^^M^HMflllMMH - '■ ' ■■■ ■•"-—~ -- ■ -
NAVTRAF.QU IPCKN 7,3-C-0065-2
TABLE 16. BI0GRAPHIC3AL DATA FOR THE GROUP USING THE ENHANCED TRAINER: APTITUDE TESTS
AND PRECEDING NAVAL TRAINING COURSE SCORES N = 32
STUDENT VT-1Ü VT-10 NUMBERS AFQT FAR ACADEMIC PRACTICAL
1 1 6 7 4 7.82 3.05 2 6 9 53.40 3.13 3 5 3 r)().lÜ 3.00 4 8 5 61.76 3.27 5 r) 3 37.48 3.00 6 '> 7 49.50 3.06 7 6 5 52.12 3.03 8 5 4 45.14 2.99 9 8 8 61.10 3.19
10 4 1 42.38 2.96 11 5 4 48.28 3.03 12 7 9 53.82 i.07 13 6 3 59.24 3.17 14 7 5 44.14 3.11 lri '. 4 55.84 3.04 16 'J 5 58.50 2.96 17 5 5 49.52 3.00 18 6 1 58.94 3.26 19 7 7 43.48 3.07 20 4 7 51.08 3.11 21 r. 4 51.96 3.04 22 7 8 52.80 3,08 23 5 8 54.64 3.17 24 5 4 52.70 3.05 2r) 5 6 40.58 3.08 26 8 7 62.44 3.33 27 4 3 42.76 3.08 28 7 6 51.10 3.01 29 7 8 4 9.89 3.06 30 6 6 56,28 3.20 31 A 2 46.00 3.11 32 6 r> 52.56 2,91
MEAN ^).7,) r).28 51.79 3,08 SD 1 . 1 'J 2.20 5.91 ,09 SE .21 .39 1.05 .02
33
■ imlirtlMMlMi
NAVTRAEQUIPCl'N 73-C-0065-2
, TABLE 17. BIOGRAPHICAL DATA FOR THE GROUP USING
THE REDUCED TRAINER; APTITUDE TEST AND PRECEDING NAVAL TRAINING COURSE SCORES
N = 29
| STUDENT VT-10 VT-10 NUMUERS AFQT FAR ACADEMIC PRAGTICAL
I 4 4 51.04 3.02 2 4 1 48.80 3.12 3 6 5 51.00 3.51 4 5 4 42.66 2.90 5 7 7 52.24 3.26 6 6 5 50.10 3.00 7 8 9 49.56 3.06 8 9 9 63.16 3.35 9 3 9 48.98 3.08
10 8 6 59.86 3.21 11 4 6 52.64 2.99 12 8 5 51.06 3,08 13 8 A 59.44 3.17 14 8 8 53.60 2.97 15 6 7 49.58 3.10 16 5 6 53.42 3.18 17 5 4 50.88 2.97 18 4 8 58.68 3.23 19 8 4 48.28 3.09 20 8 9 60.46 3.36 21 7 2 4 1.78 3.07 22 7 8 5 7.70 3.20 23 8 9 50.28 3.07 24 5 1 45.74 3.04 25 6 7 55.46 . 3.16 26 7 5 50.40 3.05 27 7 9 41.44 3.00 28 7 7 4 9.54 3.04 29 6 5 54.20 3.02
MEAN 6.34 5.97 51.79 3.11 SD 1.61 2.40 5.40 .13 SE .30 .45 1.00 .02
34
1_^^__ _-_ . - ■ - - ■ -
NAVTRAEQU ll'CKN / i-('.-()0()r)-2
to ranks for the Mann-Whitney U test. Ranks ami results oi comparisons between the two groups are presented in table 18. The tests of signifi- cance Indicate that the group that had used the enhanced trainer did slightly better in these two LnClight phases (p = .1423 and ,0901). Although their practice on the trainer could have contributed to this better showing; again, Lnterscssion differences inexperience, and the relative unreliability of the checklist must be taken into account in such an interpretation.
SUMMARY OF RESULTS
1. The two groups did equally well on the end-of-course transfer test, 'ihe enhanced group was rated slightly higher on the inflight check- list for both of two flight training phases (table l) (table 18).
2. The group using the enhanced trainer required more practice prob- lems, took more time, and used more on-line sessions to reach the final transitioning criteria at 460 knots.. These differences were statistically significant (p<.01), but the measures of association (u)-) between inde- pendent and dependent variables were relatively low (tables 4 and 5).
3. Although the difference between means (20.1«) of number of dynamic problems attempted (variable 4, table 4) was significant, (p : .0102), the difference between means of number of dynamic problems correct was only 3.43 (p .IK') (variable 5, iahte 4).
4. On practice problem:; the two groups' scores; response latencies, missile tiring scores, number of turns, and computational errors, were not. statistically significantly different (tables 5 and 6).
r). Practice with the two versions of the trainer resulted in marked improvement in fluency (as measured by response latencies) and proficiency (as measured by missile firing scores and number of turns) in both groups
(tables 7, 8, l), 10, and 11).
b. There were differences between the two groups in trends in accuracy measures across the successive Levels (table 12).
7. The attitudes of the two groups toward the two versions ol the trainer were in the favorable direction (p < .01) (table 13).
8. The group using the enhanced version of the trainer expressed favorable attitudes toward the special graphics features that were specific to that trainer (p <.01) (table L5) .
(). There were no d i llerences belween the backgrounds of the two random groups with respect to prior test scores (tables 16 and 17).
35
immmmm ■ - ■ - ■ --
NAVTRAEQUIPGEN 7J-C-ÜÜ65-2
rABLE 18, COMPARISON OF RANKED SCORES OF THE ENHANCED AND REDUCED GROUPS ON THE INFLIGHT CHECKLIST FOR
TWO TNFLICHT TRAINING PHASES
STUDENT RT-8 (R) RT-8 (E) RT-13 (R) RT-13 (E) NUMBER RANK SCORE RANK SCORE RANK SCORE RANK SCORE
1 7.0 37.5 10.0 33.5 2 3/. 5 4'). 5 J3.5 55.5 3 2(>.l) 30.5 16.5 13.0 4 22.0 4 1 .0 21.0 16.5 r) 19.0 -- 18.5 6 30.5 56.5 26.5 50.5 7 26.0 4'». 5 44 .0 44.0 8 5-'-.. 0 16.0 52.5 21.0 9 16.0 23.0 12.0 7.0
10 5.5 49.5 55.5 8.0 11 9.0 26.0 14.5 23.5 12 -- 5.5 4.0 13 '. 1.0 11>. 0 2.0 18.5 U 12.0 41.0 11.0 44.0 15 3't. 5 4.0 37.5 40.0 16 ./. 1 .0 19.0 44.0 1.0 17 30. 5 30.5 44.0 5.5 1« 44.5 51.0 4 8.0 9.0 19 51.0 34.5 37.5 21.0 20 54.0 44.5 40.0 40.0 21 12.0 34.5 55.5 55.5 22 12.0 -- 52.5 23 51 .0 8.0 4 8.0 33.5 24 LA.O ■'. 1 . 0 33.5 29.5 2 5 51.0 16.0 48.0 26.5 26 1.0 56. 5 5.5 33.5 27 21 .0 4'). 5 3.0 29.5 28 3.0 2.0 23.5 26.5 2() 26.0 -- 14.5 : 30 10.0 44.0 31 [ 26.0 26.5 32 34.5 50.5
K07; p = .142 3 z * 1.34; p = .0901
Scort-s computed witli corrections for ties: Mann-Whitney U Lest (one-tailed)
Scores were averages of checklist data Eor live Clighta (RT-8) and six flights (RT-13) per student. Blanks indicate four students who were dropped from tlie course after com- pleting the RT-13 phase. These data were not included in the statistical tcs ts.
36
■
MM -
N/WTRAEQUIPCKN / J-C-0065-2
Ü DISCUSS ION OF RESULTS
A Eundamental problem here la that ii Ls not possible, In field environments, to control adequately temporal variables over the period of time that a student is r.oinr, throu'.-h school. There Tore, the practice session data collected over this period undoubtedly were affected by biases of one sort or another that must be taken Lnto consideration when drawing conclusions from these data. The best data here, those from the end-of-practice transfer tost, indicate that the experimental graphics had no observed effects on the final accuracy or fluency scores of the group that used them in comparison to the group that did not use them during practice. The two groups performed equally well on tins transfer test. The data from the checklist used during the two phases of actual flight operations indicate that the students who used the enhanced version of the trainer did perform somewhat better in the air than the other group. These data covered .1 total ol eleven practice flights per student.
Several gost hoc , ilternative explanations for the in rhe practice session data, are possible
observed at 1Lerences hut further experi- among groups
mentation would be required to completely resolve the various Issues that have been raised. These are presented in order of their power to explain
the data.
1. The enhanced group's higher error rate in computing values of the Intercept triangle (Table 12) might account for the facts that tins group required an average of 28 more problems and 2,8 more hours during practice. Recall that their proficiency scores (missile firing and number ol turns) nevertheless essentially were equal In each practice condition, and that their latency scores to complete the loteboard were usually a second or more Blower. Just one computational error in the dynamic phase of practice (220 through 460 knots) would result in (I) that problem not being counted correct, (2) the student being allowed to finish the Inter- cept, (i) his number of turns and missile pllit scores being included in the data, and ('.) the problem not being repeated in free fly. Thu it appears that arithmetic accuracy was weighted too heavily in the scoring and the transitioning criteria. It is apparent that these errors did not prevent the enhanced group from making as good intercepts as the other group (Tab le 12).
2, The data in table 12 also suggest that the static triangle, as supplied continuously in static mod.. I, and after an error in static mode 11, might have been distracting to the students in the enhanced group while they were performing mental arithmetic, since they required an aver- age of 15.34 and ().71 seconds longer, and their error rates were substan- tially higher, .93 versus .71 and .76 versus .48. This supports the first explanation above.
]. 'The bogey track history, optimal Intercept path, ami dynamic triangle were displayed to the experimental group during Introductory problems, and also when problems were repeated in Pree-fly mode. Repeated problems constituted 247, to 277, ol the total dynamic problems attempted for both groups. Therefore, this mode contained part of the treatment difference between the two groups. However, the effectiveness of repeating problems may have diminished during later stages of Learning (practice for fluency).
37
■HMMMM ~m^*mm^mm
NAVTRAEQU [PCEN 73-C-0065-2
mmmt--
/,. in tM i i^l<l study, Ln whlcli a major objective was to test Llio "survivabillt oJ the trainer in a school environment, it was impossible' to control all variables that might Influence outcomes, in addition to the Influence o£ the Independent variables. I'or oxamplc, we learned that some ol the students who wer.' having unusual difficulties in grasping relationships between true and relative motion, or in learning to compute the values ol Intercept triangle variables, or in learning other basic Intercept procedures, wore given additional "alter hours" practice on the trainer by the school. This was the school's prerogative, and, In tact, LL indicates their confidence in the trainer. We can only assume the effects were random across groups. Also, it was not possible Lo schedule individual practice sessions on the trainer to rigorously control Inter- session-lntervals because Ellght schedules understandably hail priority. The enhanced group may have had a less favorable Lntersession schedule from the standpoint of distractions, Lntersession forgetting, or for motivation to learn than the reduced group. [f so, they could have required more time and taken more problems during practice.
'i. A fifth possibility is that the group using the enhanced trainer actually came from a different population with respect to some correlate(s) of the dependent measures. Although there are very slight differences in ehe AFQT scores for the two groups, the scores from a preceding naval training course, VT-10, are essentially the same (Tables 10 and 17). hi this regard, Stanley and Campbell have this Co say about the postCest-only
concrol group design:
While Che pretest is a concept keenly embedded in the thinking of research workers in education and psych- ology, ic is not actually essential to true experi- mental designs. For psychological reasons it is dilticuli to )',ive up "knowing for sun-" that the experimental and control groups were "equal" before ehe differential experimental treatment. Nonetheless, Che most adequate all-purpose assurance of lack of initial biases between groups is randomization. With- in the Limits of confidence stated by ehe Cescs of significance, randomization can suffice without ehe pretest. Actually, almost all ol the agricultural experiments in the Pisher (1925, L935) tradition are without pretest.
Randomization procedures were taught to the TD technicians who monitored the operaCion of clu trainer at the school. Each new class of students was Co be divided Lato two random groups, using a table of random numbers and Che standard randomization techniques.
1CarapbelJ and Stanley, L% K P-
US
a—tmmu—mm
NAVTRAKQUIPCEN 73-C-0065-2
CONCLUSIONS AND RECOMMENDATIONS
Conclusion.'
1. The trainer described in this series of reports was designed to develop fluency in Radar Intercept Officers in performing basic procedures in air intercepts. The two sets of data from the field trials in the RIO school amply substantiate the assertion that the trainer did what it was designed to do. Some comparisons between this aelf-standing, CAI system in a terminal and the conventional radar trainers at the school are of interest:
2. The hardware cost of the liehavioral Technology Laboratories trainer was $24,000 versus approximately 10 times that for each radar trainer.
?. The BTL trainer automatically recorded objective measures of the performance of each student. The radar trainers had no provisions for doing this.
4. Modifications and extensions in the scope of the procedures taught by the trainer could be made relatively inexpensively by program changes. For example, differential altitudes and speeds, and radar operating pro- cedures could be incorporated, thereby making the radar trainers unnecessary.
5. Although a technician-instructor was present to assist students while the trainer was in operation, further refinements in the data analysis programs would make this unnecessary. The trainer is essentially auto- matic. One instructor could monitor a room full of trainers. The radar trainers required a 1:1 instructor-student ratio for active learning.
6. The trainer incorporated error-correcting feedback and adaptive control sensitive to each student's entering skills. None of these features were in the radar trainers.
7. The hardware used for this tvainer is general-purpose. It is only necessary to load a different program to teach a different course. With the addition of a floppy disk and 10 interface, any one course from a library of courses could be loaded in a few seconds and detailed student records could be kept on cheap disks (approximately eight dollars) for as long as desired. Needless to say, the radar trainers lacked this flexi- bility or this capability.
8. The possible instructional effects of the bogey trc.ck history, op- timal intercept path and dynamic geographic displays, evidently were masked (1) by distracting effects of displaying a static intercept triangle while students were performing mental arithmetic preparatory to initiating the dynamic phase of the problem, which may have caused the observed higher arithmetic error rates in the experimental group, and (2) by insuffi- cient exposure of these instructional aids to the students in an crror-
19
■-- -
NAVTKAKQUIPCI'N 7J-(:-0065-2
remediation loop.
9. However, it also is likely that these instructional aids, exclud- ing the static triangle, would have been effective only in very early stages of practice for accuracy. In later stages of practice for fluency, external stimulus control over behavior is greatly reduced, for, by that time, the student has acquired mental representational structures capa- ble of guiding his performance without external aids.
Recommendations
1. PSO CAI .should be applied and tested in other areas of Navy train- ing, using current off-the-shelf hardware, to fully develop the concept and to accumulate more information about CAI system variables.
2. A study should be undertaken to develop a design for a CAI-system- in-a terminal, based on what has been learned here, taking into account current and forseeable advances in electronics technology, instructional technology, and computer science. The appearance on the market of a hand- held stored-program computer, complete micro-processors on a .15 inch square sillcon-on-sapphire LSI chip, and electronic watches with liquid crystal displays, are signs of the current revolution in electronics that will make this type of system feasible, attractive, and cost effective.
3. The results of the study reported hero, relative to the effective- ness of special graphics, should not be generalized. The independent variable actually was a complex of fixed-effect variables; thus generaliza- tion is not statistically possible. It appears that it would have been better to schedule these graphics entirely in the front-loading rather than in the practice and remedial sections of the PSO CAI instructional strategy. There, a period of "free practice" on problems designed to teach the stu- dent how to extract the maximum instructional value from the graphics, and during which students were reinforced for using them, possibly would have been a more effective use of these instructional aids.
Uo
-- -
NAVTKAKOIill'Cr.N 73-G-0065-2
u RKFERßNCES
Campbell, l). r., and Stanley, J. C, Exgerlmenta L and (v>u,-u; i-iixpc-rinR'nia I Üesigna Cor Research. Chicago: Kand McNally College Publishing Co., L963.
Hays, W. L. Stallst ics for Pajrchologista. Now York: Unit, Kinohan and Winston, 1%!.
Rlgncy, .1. W., Morrison, I). K., Willi.ims, L. A., and Towne, I). M. Degcrlp- tton ami lnit La I EvaluatIon oj a Computor-basod Individual Trainer Cor Lho Radar [ntcrcegt Observer. Los Angeles: Behavioral Technology Laboratories (MVTRAEQUIPCEN 71-C-0219-1), 1<)7J.
Rlgncy, .1. W., Morrison, I), K., Williams, I,. A., and Towne, I). M. A Guide for Lho A|)|)j Lcat Lon o£ PerEonnance-Stri^ture Or ion tod CA1 in Nay a 1 Training. Los Angeles; Behavioral Technology Laboratories (NA.VTRA.EQUIP- CEN 73-C-0065-1), 1974.
in
MMM -- ■ -- ■ ■ --- ■--■ ' - rf^MMMMMMUM
NAVTRARQUIPCEN / i-(;-0()6rj-2
Al'I'KNDlX A
IUO STUHKNT QUIiSTTONNAIRE - KNIIANCKD VKRSTON N = 31
in comparison with other similar learning experiences, the recent experience with the intercept trainer was:
.1. Very en joyablc d) b. Enjoyable (13) c Neutral (II) d. Uorin^ (2) e. Bxtremcly boring(l)
As a method for teaching operating procedures, the intercept training system was:
a . Very el Feet ive (M h. Effect Ivc (18) C Average (8) (I. [neffect i ve (o) e. Very Lne Pfec t i ve (1)
While using the Intercept training system to learn operating pro- cedures, I would have liked to have had the system answer inr me:
a. Very many questions (1) b. Many more questions (l) C A few more questions (II) d. No more questions (16) e. No quest inns (2)
With the presence o£ an instructor and fully operating equipment I feel that Intercept procedures would have been learned:
a. In a much shorter period ol training b. in a shorter period ol training c. In about the same time d. in a longer period of training e. In a much longer period of training
(2) («) (lb) (5) (0)
The int' •pi l IM i ner i s
a. Very easy to use (7) h. Easy to use (II) c. Neither easy or difficult to use (9) d . Hi IT Icult tO use ( () e . Very d i 1 I ieu 1 ; to use (())
MMMMMMMI J
NAV I'KAKQUIPCRN 73-C-0065-2
% r d. riic iniii.il instruction giviMi on how to usi' the intercepl trainer
was:
,1. Much moiT Mian .ul('(|ii,ii c in my needs I 1) I). Mtin I li in ,i<li''|ii.il i- In niv lU'etls CO
^ . Ail.MiM.il i I >' my MI eds ( I h )
.1. Less than adequate to my needs i'•) e. Much less than adequate to my needs (I)
7, Almut how Ion,", ili'i ii take you to learn to utilize the intercepl l ra i iu'1-.'
a. More lhau I WO hours (2) h. About two hours O) C. About 1-1/2 hour:. (■'•) (1. About 1 hour (11) e. About L/2 ol t he I I rsl liour (9)
8, The tmmedlate knowledge ol errors provided by the Intercept trainer:
a. Greatly aided learning RIO operating procedures (A) b. Aided learning (16) c. Made no dirCeroncc in learning (11) d. Hindered learning (0) R. Greatly hindered learning (0)
9, Static mode requires thai you perforni necessary arithmetic computa- tions without error Ln a specified time frame before continuing witii Intercepts. I'iil you Find ibis:
a. Very helpful In learning (9) b. HeLpful in Learning (1 5) c. Neutral (ft) d. (»i very little help O) e. 01 no help (0)
10. The Intercept trainer repeats an Intercepl Ln Free Fly Display format where you did not achieve criterion firing position in the Initial attempt. This allows you to review the Intercept and percclv« your errors. ni<i you:
a. Strongly like this capability (11) b. Like ibis capabilily (13) c. Have no like or dislike lor this capability (b) (1. Dlsl Ike this capabi lily (1) e. Strongly dislike this capability (0)
11. in Free Fly you could observe the relationships between intercept geometry and radar presentation during an intercept. I'ul you use this feature':'
a. Ve: b. No
(25)
(ft)
1.3
■MMMte - - - — - -- ■
NAVTKAKOll ll'CKN /i-C-OOb^-l
12 11 you used Free I'ly in the above way, do you believe it was:
.1. Very helpful In Learning (• I), iii'lpiui in learning ( 0 c. Neutral (b) d. Oi VLTV little help (1) e. OL no help (1)
13. In Free Fly Mode, Liu' Trainer provides an "optimal path" for the bogeys track down the scope, and a dotted "track history" of the bogey's actual path. As a method of associating intercept geometry with scope presentation these aids were:
a. Very effeet ive (7) b. Effective (10) L. Average (12) d. Ineffective (1) e. Very Lneffeel ive (I)
1V. The trainer automatically provides a sample problem In Free Fly mode when advancing or changing problem category. This feature is:
a. Very helpful (7) b. Helpful (If)) c. Neutral (5) d. Of very Little help (I) e. 01 no help (2)
13. During the progress of any Intercept you may display the intercept triangle by depressing the TRI key. When the triangle is displayed you will also see an "optimal path" and "track history" displayed on the "B" scope. En clarifying the problem solution, this aid was:
a. Very el feet ive b. Effective c. Neutral d. Ineffective e. Very Ineffective
(A) (nn
(3) (0)
16. The radar simulation provided hy the Intercept trainer was:
• i. Much more than adequate lor b. More than adequate Por learn c. Adequate for learning basic d. Less than adequate Cor Learn e. Much less than adequate tor
earning basic intercept procedure (2) ng basic intercept procedures (fa) ntercept procedures (20> ng basic intercept procedures (2) earning basic intercept procedures(1)
7 Were there any particular Intercept problems that were confusing?
hh
. . . ^
NAVTRAKQUIPCKN 73-fi-0065-2
a RIO STUDIiNT QUESTIONNAIRIS - REUDCED VERSION N = 2r)
In comparison with other similar Learning experiences, ihv lecenh experience with the Intercept trainer was:
a. Very enjoyable (5) b. Enjoyable (U) c. Neutral (6) (1. Boring (3) e. Extremely boring (())
? As a method lor teaching operating procedures, Liu' Intercept train in;/, svs tern was;
a. Very e I led ive (3) b. Effective (13) c. Average ((>) d . I neffee live (3) e. Very inelfeetLve (0)
While using Llie Intercept training system to learn operatlnt» proce- dures, 1 would have Liked to have had the system answer for me:
a. Very many questions (0) b. Many more questions (2) c. A few more questions ((>) d. No more questions (10) e. No quest ion;; v7)
With the presence of an Instructor and fully operating equipment i feel that Intercept procedures would have been learned:
a. In a mueh shorter period ol training (l)') b. In a shorter period of training (13) C, In about the same time (3) d. In a longer period of training (5) e. In a much longer period oi training (2)
The Lnlercepl irainer is:
■ i. Very easy to use (10) b. Easy to use (()) c. Neither easy or difficult to use ((>) d . I)i 11 it ul t to use (0) e. Very di CflculI to use (0)
h'.
M^MMM^^MMMMMM
i^^T^iv^nrn^Mpnn^mpimRVWIP^
NAVTRAEQUJPCRN 73-C-0065-2
6. The initial instruction given on how to use the Intercept trainer was:
a. Much more than adequate to my needs (2) b. More than adequate to my needs (5) c. Adequate to my needs (12) d. Less than adequate to my needs (6) c. Much less than adequate to my needs (0)
7. About how long did it take you to learn to utilize the intercept
trainer?
Ü
a. More than two hours b. About two hours c. About 1-1/2 hours d. About 1 hour e. About 1/2 of the first hour
(2) (6) (2) (7) (8)
8. The Immediate knowledge of errors provided by the intercept trainer:
a. Greatly aided learning RIO operating procedures (2) b. Aided learning (1') c. Made no difference In learning (4) d. Hindered learning (2) c. Greatly hindered learning (0)
9, Static mode requires that you perform necessary arithmetic computa- tions without error in a specified time frame before continuing with Intercepts. Did you find this:
a. Very helpful In learning (5) b. Helpful In learning (13) c. Neutral (M d. Of very 1Lttle help (3) e. Of no help (0)
10. The intercept trainer repeats an Intercept in Practice mode format wiiere you did not achieve criterion firing position in the initial attempt. This allows you to review the intercept and perceive your errors. Did you:
a. b. c . d. e.
Strongly like this capability (5) Like this capability (13) Have no like or dislike for this capability (4) Di sii ke th i s capab i1i ty (3) Strongly dislike this capability (0)
)iG
mammmm «MMMMMMHM J.J...J.^... ^■.^,.-M-Jfa-WM|htf;Wh...t-Jg*J
PmUCW .»MM««»«! IMJIlllWjUiillJIJilllMill^lUliniNhWJ1 ,<»!.■'• i iliiJ,! |i"J*l«lllJl(lt|IHt(l*f""J-™l. '* . 1 J-"!»!!!!«" ' " |.:|im«iWW*»SW«MJl«P-W-"llIWJW Jip(,»'Hmw"iW»»««*»s»w)M',w,w rmfivf-ii.,mw :'•? («■J.WIIH« .UUHJIJ.IJI^
NAVTRAKQUIPCEN 73-C-0065-2
; I
1/, The trainer automatically provides a sample problem In Practice when advancing or changing problem category. This roature is:
a. b. c. d. e,
Very helpful (6) Helpful O4) Neutral (5) OL very little help (0) Of no help (0)
16. The radar simulation provided by the intercept trainer was:
a Much more than adequate tor learning basic intercept procedure (0) b. More than adequate ior learning basic intercept procedures {/) c Adequate for learning basic intercept procedures Qli) d Less than adequate for learning basic intercept procedures (J)
e. Much less than adequate for learning basic intercept procedures(0)
17. Were there any particular intercept problems that were confusing?
f
1)7
i* ■■■
IMP """! i «WPWWPtttHUi JJ.viJ.W.|^ilWll*"lin!«wiWHi|iJ(|.vLlli JI.1JPIIIIM»!W* i .inJin ii.PVjiuiiJMJiwimiiuwBiMuiiii ,i«Mi«i,-)iji<i«p!iHivsnnipiPPVf ir • J:-MimmvmmiQm^fff
NAVTKAKQU[PGEN 73-C-0065-2
APPENDIX 15
r x) y.
in y
<f K en 3
CO
io y. CM '^
m Ä ■-I Z
<r gj r—t
1 ro y. H
r-J SB
H K
< ' H C-. • H > • • m
| ; *. ! ; E oä
.4-14-1. 4-1 0) •
ffi . 14-1 U-l • d -H c- H . -H -H . O 4-1 -a M . u u • 0>
^ r. • T) 13 • in d c^- !-i u • • d o U-l u
S Q . r-l U • o p . IM 3 o o • O CJ • . -r-i tn d O u
IH • U 4-1 • . 4-i d <>■ o • Ü ^#
w (U ■ . 4J G • . o o x) ,'-4 tu • 0 •3 H ? y Ul • c • . d 3 • od UJ
. (U -r-i QJ 4J ^4 0) < PH HJ Pi 1-J • u < . o o • V4 4-1 Ji .J tO c^- in u
Q en (n 3 • 3 • o u • CQ . c-. •H o tn 3 d • d i-i OJ
i O 4J • . c -a OJ 3 ■-' < • 10 0 >
o O U ^1 • o o • | (0 >J »-i CO • CU I-I
. c u 01 . u w • c« o. d •H > -a • in U u %r • o c ,-J 4-1 c in w -o 4-i tu c 01
. -t-i O C C • tu OJ tn H CO 3 O co • CU d H H . u 3 ji 3 Jj . 4J 4-1 4-1 pL, D. 0) 4-1 C C E . E u-i
01 cn CM . u <n tn • ( U O 3 • tfl (fl -^ W yi E l-i <U O . »H -H
OJ M . (11 '^ -H ■ 1 = O 4-1 • U U B U •r4 01 3 in '-1 (30 . u X ä u . i_i i-i i-" -, u • -^ oi OJ 4J > 4-1 CO XI C E
OS . -M r-l & 13 * J 14-1 Q) . C C t-1
OJ E cfl ? O co • 01 01 4J o ■U 0 tfl U - ̂ O 4-1 . U U H •c4 (U 00 0) U P6 . r-l ^-1 o u H 53 1*1
00 O -U i-J y C 3 3 0) f. W r-l > (U CU . X XI d SÜ a -a «i 30 4.) 3 o 4-' ■1-, ^ H xi x) co x: -^ V4 • cO 0 u •r^ c o OJ "a 3 M O H tO cu co C 00 in O . 4-1 U ai
T) (0 -U O ■i ri u a u -o U ^-14JC0 4->C^ CUCU^
s CO O O u JJ 01 0) XI CU XI to O ^- a> cO a) (u a) u 60 tn >%0 D- 4J c iO(uQC^-'-|>>>iU|3J
X! £9 .cue U H O O ü H 4-iuQc04-iinr-ii-iox: M to o u x; o) -i-i )-i cr> VJ <u 3 Cu u -^i j-i to co 4J u hJ C M •'■' U J-» a. to to a >-i 4J OJ CO V4 • O O O 4J cO
g o f -H O CU 4-1 JJ o o4-iinojajd3,4-i 5 •H tJ u in 3 (j 4-1 t0 "3 O T3 u d M-I d< -as-itjcoc 4J
M (0 M-l 0) ^ • rJ .H (-1 <0 = r-l M Vl <U U ^J TJ O >-l O
D- C C 01 U C 0) -t-* 4J C 1 10
u E C -^ i CO 0J CU 0) CO c X
CO cO E Ü i « CCOitnoELr-
dd>Jhi-',Huo,^c to-r4 33Xix: u v-
to 4-1 d
O U .D TJ U T-i E 4-1 O t; [ u ui o m "til 5 U 'W • a co ^ oj c -H E C4J ^^cdoo^cö
O >-. ' <" 0> r-l X: U -^ 4-1 .^U-IIM 01 D.-'-* OdOC 1 E OJ XI ^ 0) u C- 3 co tO -H p: • H O 4.) .^ -H )J t. o
^-i 0) (0 en co in i-i E in u)-i co)aj4-i>,(-i o jQ p JJ B w -4 -H 4-1 (X I O ^ uxi .3^:tjcot4-i<U4. J tO <0 'J-i «0 Oi-c) ^ in i Cuu 4J O 4-1 c U •H c 1 00 4J 3 -H Di (X U) -.-4 -.-( c OTJCUO 0-^'-4'3C i d a. to i-4 - ^ o) V4 -a o c. c OJOJ xixi-acucu^ •iH cu i -a 3 i •a 4J T) -H VJ -r- 4J4-14-l4JT4dE4J >, O 1 O 1 1 CO i-l 4-1 01 X X)OCOCfl-r4l4-l-HCOU~ J 14-1 U OJ >. VJ ^ 1 3 >, 0) -1 4J 4. S<U 5 C p C - -H to a. cu ^ o r cr oi Cu tn in •'- CVJ^^-I oai4J!-ic 1 ^-1
O M <o ^-l OJ t>0 O 0 Z i-i(-icflto= o cu d y CU •V L> O Cu c •XJ o U Cu bO 30>>cu oiuo: ' E ai in xi en •■-i X tO XI D. C 4. 4J CJ •-1 •'-, H 11 O ••-4 l-l < J rt X E o E "i : VJ (j 4-i > tn o in C tn o ew TD in -a to .n c ) ■a-OJ-iVjtntO 1 -H "0 ( 0 >•.
■H o ■-' ^ Qi I 01 '-, HI <u U ddcoco Ä PQ 'M C 3 d H O "^ > o > Xi co T: ) .. S «3 a UJ co >- - tO ,0 >, H M X U V U 'V O C J H -O ID -rJ O "3 3 at ai -u o cd 01 TJ 4J T-I E V4 ■" ■^
^■OTjaJii- uoitnTJ 5 QJ as O H U
4J 00 C ■'-, E m O O U
4J x; c x: o cu*- IM U O U M CU (
-< 3
[J 01 HI 4-1 4J 4J 01 pCNtoto-oTacj-oN
w na u P-i oi <; < u < u, < P i •3 .^ -H CU CU 0) D U 0) -J 1 -1 5 HEC--4-'CV4C.-Od u ■*" t/JpÖOOÜi-lflIW'HÖO'l 3 3
en oi j en H
r-j o4 rn ^J to vO ^^ 00 t^ o »--4 r 1 oio-'j--|to|ii'ii>o-
UJUO4-)4-lECU4J0U o ^ fc r-" r-4 r -< ZlaiQiCCiiO-CUV > ^ |l _
1 Q J
r-i CM en ^t in kD i^- oo ^ c 3 f -1
■
1.8
■ - ' -■ — ■■ ■ —■ -■ ■ i ■■■■■■
BP PW ,1 J WMt. * LWIU mi IIS« ip lIl_pi,llJ»,,4WW»'»«^*.H **■ »>• -, - 'If * WÄW^wi.,H!iHWJHi<HI^W.J(W!i.,i„!ifiill,:. ,» ill Wljlli|l|..i ■fMlW^W^l^BIBI^IIBIf^lW
NAVTRAEQUIPCEN 73-C-0065-2
JfUWiM^WftipU r*11 ■■""'• "^J)^™
.
w H
w g o
H
fa
H4
(/3
^0
a: o H U 3
CO
eg
r-4
S2
F5
^ Z
z
4-1 4J • U-l <4-l •r4 -r^ u u ■
•a -a •
Pä rH (J .
u
en c
O «J • U 4-1 . 4J d • C 3 •
« >- 3
>: 0 o •
H O 4.
PD CJ li o o • U c 0 U 4J • u C ) c 4.
oä ' o c : c (U eu en u 4- 1 -l-l " I u • JJ 4J 4J
C. en c ) 3 (B ed -H ^ 0 •H i 1 4J U W E M >- ( p- M . -.-i
r- U- i a C C ^i X o r > 4J
^ u u
00 (J c 3 3 0) C T 4. ' : c 4J 4J 4J •H C : o >■ 4 0 <fl TD « 4J ei i L 0 (J T3 ^4 rt 4. c , u oi 0) c 1 « U i > ̂ CT a 4J c
JC 4- ) c i-< O U M e 1 o 4. 4 ^ 4. I ^ OJ 3
c ». 1 ■•-1 ( 1 et et D- M 4-1
o •u a U •H 4- ' c a 4. 4. i T3 o -a co u- • a F C : c C O c,' C •H -r < u *<- ( 1 -H t c i n) r^ »■ ' u i~ F E : -rl 1 H T o ^^ . t j a c (t) C X O (. a C i »—• t. 4J o eg o * & 1 e 3 v « C -H 6
r ) > ̂ c 4 X I -i-i 4-1 o X er 7 U 3 c U «d fH Qtj
r-l < >- 0 e/ E en J3 V 4 4J •F 4 4J *r < ■ 0 3 (0 C 1 «- v. T ) * T i ac^ *J r : -H a ■i^ c a n 1 h c U C J !- li . 0 c c 01 l T c 4. 4 T ( 1 T^ M "^ O 1 )- I t U 4J OJ X U Q ) > ̂ C L. : J > -, c 1 -H 4-1 4J (0 C Li a c r a V < tn tn T-I
r ) 0 D 4. < J Q ) C 1-0 3 v> t i c (1 T 3 C a no 01 c I X ( J X T C 4J
JC X C 1 E TJ 3 tn c : a a T 1 « C : efl X O
■H r ) •-' c C J i-- •r- ( QJ o ^-i c et ► > C et 1 X et) T) -Q ? ►> u i— < ) u 4. 1 0 0) <fl C ) 4J t. 4 4- C : E H r-4 4-1 C J) c a J : c •F < O Q"-> m c ) r • r- I > c ei
7 1 u tx 0
U B 5 u c « i <J 1 fa < Du
r-l C^ i f <t ir 1 vC r- . 00 ON o 1—1
LTl
en
c-g
f-H
x;
is
IZi
x s-
c^» eu e
•H P • ■ •
4-1 OJ • • E • •
C -H . . e>' O 4-1 • . T3 • - eu tn c • • (> • ■ • H
Pi CO> U^ . . kl u o . C" t4-l . . 3 PQ •H en C O • ■ u § 4J C P- O o pN O O "O -H 0) • • o z OJ -^ 01 4J r-4 •
t-i 4-i jr td oo • p. en H o • -^ u tn 3 c ■ c u
& C-Oeu3-i<: • edO CB H M ct) • a ui
U CM o« cj- TI > -O • tn U 00 CenMT34J(uc- eu tJ to 3 o td • eu H Ou CU 4J C C E ■ E >M Z en E l-i eu OJ • -H -H M •■-I Oi 3 en i-i 00 • 4J
<U4-i>4-ica^3C. E E td 3 0 td • eu eu
•H (U oo eu M OS • lH r-l 4-1 i-l > QJ D. • X X
XTStflX^^l-i • tflO cuedcoo tao-4JM
^-1 4J tfl 4-> c ^ a. al x a. .-a to o p. (U rt IUQCX-H >, >, U eu 4-i uQlre4-ienr-i POX a. o •f-i AJ td td 4J 04 ID V4 • O O O 4J u O^Jeneueucc1*-! u c u-i a. -3 n eu td o ed td E HE
CdOJtnoECr-i ccui-ii-i'Huo^o td -i 3 3 X X U M
4J 4J o en tn 4J C4-1 HUCCOOPC .^ IM y-i u a -H o c o o
.•H O 4J -^ -H p u U p n eu eu 4J >, P
u -o • 3 X U ed "4-4 0) 4J 4-1 3 4J c o -ri a
o TD a. J 0) -i-l i-e T3 0) 4J 01 x -3 T) a eu J«:
4J .U 4-1 4-1 -^1 C E4-I ■OOeded-^^'^tduX/
eu aj 3 c 01 c □ Li i-< rH OeU4-ip(dr M h CD d : u a. c u 3 O > > a- o <u o c 4J U —1 ■ -1 0) tu O •H u o u u iJ > en u >M -a -v u M tn id •^ T3 en c c td fl X « ^ C 3
.. td td c IM al UH H •a -3 -H O T) 3 c Z T3 X» (U v z 4J oj tn 13 o uJOJeu4-i4-i 4-i OJU
Q C N td TJ T3 'S eu T3 N p -H --4 D. H E d -H •
a ai eu (J eu -4 -3 -i c u a.T3 c i)
CD M 00 U J -M (d M -H 00 T) U O •H • -i td eu oi > o -H
tij4Jej4-i4-iEO<4JOuOl X 01 tu C r eu a c w eu >l HODS-^-UDä^MPMOä^
i-'CNif-i-d-mvor-^ooejNo
_ "
T3 eu Wi tu > o u
c eu eu X 4-1 o c eu > ja td X
a
oo
>»
| cd
& cd
eu > cd X
3 O
1.9
