Cognitive and Noncognitive Determinants and …newinbre.hpcf.upr.edu/wp-content/uploads/2017/02/...Christal (1989) specifies four sources of individual differences in the acquisition

Journal of Experimental Psychology: Applied1995, Vol. 1, No. 4, 270-304

Copyright 1995 by the American Psychological Association, Inc.1076-898X/95/S3.00

Cognitive and Noncognitive Determinants and Consequencesof Complex Skill Acquisition

Phillip L. Ackerman, Ruth Kanfer, and Maynard GoffUniversity of Minnesota, Twin Cities

Integration of multiple perspectives on the determinants of individual differ-ences in skill acquisition is provided by examination of a wide array ofpredictors: ability (spatial, verbal, mathematical, and perceptual speed),personality (neuroticism, extroversion, openness, conscientiousness, and agree-ableness), vocational interests (realistic and investigative), self-estimates ofability, self-concept, motivational skills, and task-specific self-efficacy. Ninety-three trainees were studied over the course of 15 hr (across 2 weeks) of skillacquisition practice on a complex, air traffic controller simulation task(Terminal Radar Approach Controller; TRACON; Wesson International;Austin, TX). Across task practice, measures of self-efficacy, and negative andpositive motivational thought occurrence were collected to examine predictionof later performance and communality with pretask measures. Resultsdemonstrate independent and interactive influences of ability tests andself-report measures in predicting training task performance. Implications forthe selection process are discussed in terms of communalities observed in thepredictor space.

In the past, studies aimed at prediction ofperformance over skill acquisition have typicallyexamined only ability measures, only personalitymeasures, or abilities and a few personality mea-sures (e.g., for a review, see Kanfer, Ackerman,Murtha, & Goff, 1995). Moreover, no studies exist

Phillip L. Ackerman, Ruth Kanfer, and MaynardGoff, Department of Psychology, University of Minne-sota, Twin Cities.

This research was supported in part by the Office ofNaval Research Cognitive Science Program GrantN00014-91-J-4159, Air Force Office of Scientific Re-search Grant F49620-93-1-0206, and National ScienceFoundation Grant NSF/SBR-9223357. We would like toacknowledge experiment running and data coding assis-tance by the Kanfer-Ackerman Laboratory researchassistants. We also thank Dale Prediger of AmericanCollege Testing for allowing us to use the UNIACTinventory.

Correspondence concerning this article should beaddressed to Phillip L. Ackerman or Ruth Kanfer,Department of Psychology, University of Minnesota,N218 Elliott Hall, 75 East River Road, Minneapolis,Minnesota 55455. Electronic mail may be sent [email protected] or [email protected].

that jointly examine a wide array of cognitiveability (objective and perceived), affective (person-ality), and conative (motivational and interest)variables to estimate their independent contribu-tions to prediction of individual differences inperformance or the communalities among theconstructs. The first purpose of the current investi-gation is to examine five domains of variables(personality, interests, self-concept, self-estimatesof ability, and objective ability), as they indepen-dently and jointly predict performance over skillacquisition, along with variables of self-efficacyand motivation. The second purpose of this investi-gation is to examine the communalities amongthese domains of predictor variables and establishthe grounds for an integrated perspective for therole that traits and dispositions have on individualdifferences in performance.

The approach adopted in this investigation isthat the process of skill acquisition is dynamic—asperformance changes over task practice, cognitiveand noncognitive constructs may differentially de-termine individual differences in performance (e.g.,see Ackerman, 1988,1990, 1992; Kanfer & Acker-man, 1989). In addition, the task-relevant thoughtsand self-efficacy expectations of trainees may

270

COGNITIVE AND NONCOGNITIVE DETERMINANTS 271

change over the course of skill acquisition. We alsoexamined how these changes occur, how traitmeasures predict individual differences in thoughtsand self-efficacy over practice, and, most impor-tantly, whether these thoughts and expectations, inturn, predict individual differences in subsequenttask performance.

First, we briefly describe theories and dataregarding the prediction of individual differencesin performance during skill acquisition. Next, wedescribe theories and data regarding the effects oftask practice on subsequent thoughts and self-efficacy expectations.

Prediction of Individual Differencesin Performance

Abilities

Since the 1950s, substantial research attentionhas been devoted to documenting the changes inthe ability determinants of task performance overpractice (for reviews, see Ackerman, 1987; Adams,1987; Fleishman, 1972). Since the 1970s, therehave been several attempts to develop theoriesthat predict the relative importance of differentcognitive and intellectual abilities over the courseof task practice (e.g., Ackerman, 1988; Kyllonen &Christal, 1989; Kyllonen & Woltz, 1989; for areview, see also Cronbach & Snow, 1977).

Two theories in particular relate to the currentissues of task information-processing demandsand the types of abilities that determine perfor-mance at different stages of practice, namely thoseof Ackerman (1988) and Kyllonen and Christal(1989). The model proposed by Kyllonen andChristal (1989) specifies four sources of individualdifferences in the acquisition of knowledge andprocedural skills: breadth of declarative knowl-edge, breadth of procedural skills, capacity ofworking memory, and speed of processing. When atask is novel, the critical components of perfor-mance over task practice are working memory andprocessing speed, with working memory capacitymost associated with initial performance and pro-cessing speed most associated with performanceafter practice (Woltz, 1988).

Ackerman's (1988) theory segments the ability—performance relations into three broadly definedstages of practice; namely, Fitts and Posner's

(1967) cognitive, associative, and autonomousstages. The theory specifies that general and broadcontent abilities (spatial, verbal, and numerical)are most associated with novel task performance(cognitive stage), perceptual speed abilities aremost associated with intermediate practiced perfor-mance (associative stage), and psychomotor abili-ties are most associated with performance afterprotracted practice (autonomous stage). The theoryalso specifies that tasks with only consistent infor-mation-processing components will show thesethree-stage changes in ability-performance rela-tions (consistent refers to unchanging rules formapping stimuli to responses). Inconsistent infor-mation-processing task components or tasks wherethe consistency of information processing is notapparent to the subjects will fail to show changesbeyond the cognitive stage of practice (Ackerman,1988; inconsistent refers to tasks where changesoccur in mapping stimuli to responses). In addi-tion, insufficient motivation to learn will also havethe effect of precluding or slowing development oflater stages of skill (Kanfer & Ackerman, 1989).

The theories make similar predictions for ability-performance relations during practice in severaltypes of tasks because there is substantial overlapbetween measures of working memory capacityand measures of reasoning ability (Kyllonen &Christal, 1989). That is, initial task performance iswell-predicted by general ability measures, whichalso include broad content tests (e.g., verbal,spatial, and numerical reasoning) and measures ofworking memory capacity (e.g., ABCD Order Test,Alphabet Receding Test; see Woltz, 1988, fordescriptions). At an intermediate stage of taskpractice, tests of perceptual speed ability (e.g.,tests of substitution, clerical checking, and percep-tual scanning) and tests of processing speed (e.g.,encoding speed, retrieval speed, and responsespeed) are increasingly associated with individualdifferences in task performance (Ackerman, 1988;Woltz, 1988). The theories diverge in predictingthe determinants of individual differences in taskperformance at the autonomous stage of skillacquisition. However, for the task under currentconsideration, a Terminal Radar Approach Con-troller (TRACON)1 simulation, and the amount of

1 TRACON is licensed software by Robert B. Wesson,Wesson International, Austin, Texas.

272 ACKERMAN, KANFER, AND GOFF

practice accorded (less than 20 hr) both theoriesmake similar predictions of the importance ofbroad content abilities and perceptual speed abili-ties.

Personality

The history of personality test-performance re-lations in applied psychology lies beyond the scopeof this article (but see, e.g., Kanfer et al., 1995). Inan early review, Ellis and Conrad (1948, p. 421)concluded, "[Personality inventories proved gen-erally ineffective for predicting performance mea-sures (such as successful completion of a trainingcourse)." However, recent studies have supportedthe conjecture that there are some replicablenontrivial correlations between personality mea-sures and job behaviors. For example, studies byBorman, Rosse, and Abrahams (1980) and re-search conducted under the auspices of the U.S.Army Project A Team (Hough, Eaton, Dunnette,Kamp, & McCloy, 1990; McHenry, Hough, To-quam, Hanson, & Ashworth, 1990) are representa-tive of this approach. The Army Project A studiesinvolved noncognitive and ability testing of over10,000 personnel. In the Hough et al. (1990)report, analysis of the criterion space was con-ducted concurrently with investigation of the pre-dictor domain to yield a model of personality-workcriterion linkages. A noncognitive battery, de-signed to assess six predictor constructs, was devel-oped and empirically evaluated in the context ofrelationships to a multidimensional set of perfor-mance criteria. McHenry et al. (1990) examinedthe predictive validity of the noncognitive mea-sures for training and proficiency criteria. Resultsobtained in both studies indicate that two person-ality constructs, Dependability and Achievement,were found to be valid predictors of ancillary jobperformance criteria (e.g., Effort and Leadershipand Personal Discipline).

For direct measures of performance, a relatedapproach takes advantage of the Big Five factortaxonomy of personality constructs (that is, the fivebroad personality factors of Neuroticism, Extrover-sion, Openness, Agreeableness, and Conscientious-ness; see e.g., Digman, 1990, or Goldberg, 1990).For example, Barrick and Mount (1991) examinedthe relationship of the Big Five personality dimen-sions to three job performance criteria by five

occupational groups. Using applied studies appear-ing from 1952-1988, Barrick and Mount had ratersclassify the personality scales used in the researchinto the various predictor dimensions. Meta-analytic results indicate that only Conscientious-ness was substantially related to performance acrossall occupations and for all types of criterion. Incontrast, Extroversion was found to be a validpredictor across all criterion types for two occupa-tions: management and sales.

Motivational Skills

Motivational theory (e.g., Kanfer & Ackerman,1989,1990,1995; Kuhl, 1985) and previous empiri-cal research with the selection of air traffic control-lers (Ackerman & Kanfer, 1993a, 1993b) havesuggested that level of two aspects of self-regulatory skills (namely, emotion control andmotivation control) are important determinants ofindividual differences in performance over skillacquisition. In addition, a measure of these motiva-tional skills has been shown to be independent ofability and provides incremental predictive validityfor laboratory and field complex skill training.

Self-Concept

Modern conceptualizations of academic self-concept consider the construct to be competency-based and multifaceted with varying levels ofspecificity (Goff, 1994; Marsh, 1990). Such concep-tualizations use judgments of competencies forperformance of particular behaviors similar toself-efficacy judgments as their basis at the itemlevel. Aggregation of items at differing levels ofgenerality of self-concept is then possible. Severalrecent studies and reviews (Byrne, 1984; Marsh,1990; Marsh, Byrne, & Shavelson, 1988; Norwich,1987) have established that competency-based self-concept constructs have significant relations withmatching areas of achievement in the academicdomain. Although early reports on self-estimatesof ability (e.g., DeNisi & Shaw, 1977; Kelso,Holland, & Gottfredson, 1977; Levine, Flory, &Ash, 1977) indicated limited communality withobjective measures of ability, recent research hasshown that self-concept measures (based on abili-


ties and competencies) and self-estimates of abilityhave convergent and discriminant associations withother constructs of interest in performance predic-tion such as ability, personality, and vocationalinterests (Goff, 1994).

Vocational Interests

Vocational interests can be thought of as amotivational representation of vocational self-concepts (Lowman, 1991). Several vocational inter-est and choice theories suggest that vocationalinterests involve self-concepts of competenciesand motivational factors (Holland, 1973; Krum-boltz, 1979; Krumboltz, Mitchell, & Jones, 1976,1978; Super, 1957,1963). Vocational interests havebeen shown to have differential associations withability measures (for a review, see Ackerman &Goff, 1995). Such associations are thought to bebased on the ability demands of job tasks clusteredwithin each vocational interest area (Prediger,1991), including abilities not typically tapped byability tests. For instance, realistic job tasks areexpected to have particular demands on spatial,mechanical, manual dexterity, and organizationabilities. Specific task interest has been shown topredict learning performance (see Schiefele, 1991,for a discussion of the effects of content-specifictask interest on learning performance). The com-bined representation of self-concepts of competen-cies and motivation within vocational interest as-sessments may result in useful prediction of learningof complex tasks that are similar to jobs.

Self-Efficacy

Self-efficacy judgments pertain to an individual'sconfidence in the ability to successfully mobilizeand organize resources for goal attainment (Ban-dura, 1986). Conceptualized as task-specific andprospective judgments, self-efficacy may be onlyweakly related to individual differences in person-ality or other trait measures. Major determinantsof integrative self-efficacy include (past) perfor-mance accomplishments, vicarious experiences, ver-bal persuasion, and physiological state. Numerousstudies have provided support for the influence ofthese factors on self-efficacy judgments in achieve-ment and clinical contexts (e.g., see Bandura,1986).

According to social-cognitive theory, self-effi-cacy judgments may exert direct and indirectinfluences on subsequent learning and perfor-mance (see Bandura, 1986). Judgments of lowself-efficacy are associated with lower levels ofeffort and lack of persistence; high self-efficacy isassociated with higher levels of effort and persis-tence in the face of failure. Many studies show apositive association between self-efficacy judg-ments and subsequent task performance (e.g., forreviews see Bandura, 1986; Feltz, 1994; Sadri &Robertson, 1993). However, fewer studies haveexamined the unique influence of self-efficacyjudgments on performance relative to other fac-tors, such as cognitive abilities. Among studiesreporting on investigation of the influence ofself-efficacy judgments on performance relative toindividual differences in cognitive abilities, inter-ests, and other nonability traits, the pattern ofresults obtained is inconsistent. For example, Ban-dura and Wood (1989) and Mathieu, Martineau,and Tannenbaum (1993) provided evidence for theindependent role of self-efficacy judgments onperformance relative to the influence of initial taskperformance. However, Mitchell, Hopper, Daniels,George-Falvy, and James (1994) found no signifi-cant incremental influence of self-efficacy in theprediction of performance in a short skill acquisi-tion experiment.

Consequences of Skill Acquisition

Previous research has indicated that there arenumerous cognitive and affective consequences ofskill acquisition (e.g., for a review see Kanfer &Ackerman, 1995). Generally, task self-efficacy ex-pectations increase as skills are acquired, andproblems associated with emotion control dimin-ish, which would be reflected by a decrease innegative motivational thought frequency (Kanfer& Ackerman, 1989,1990). In contrast, as skills areacquired, greater demands on motivation controlare expected. Positive motivational thought fre-quencies may be expected to change in concertwith positive self-regulatory processes. Whetherthese positive cognitions increase in frequency orremain stable is as yet unclear, given competingdesires on the part of trainees to minimize effortexpenditures and maximize performance levels(Kanfer, 1987).


Summary and Overview

Six different sources of individual differencesdeterminants are considered: ability, personality,motivation, self-concept, interests, and self-effi-cacy. Although these sources may each offer signifi-cant prediction of complex skill acquisition, theyare typically evaluated in isolation. The purpose ofthe experiment described here was to examine howthese six sources of influence relate to individualdifferences in complex task acquisition both inisolation and in conjunction with one another. Byjointly examining the various determinants of per-formance, we can evaluate the evidence for therelative independence or communality of the con-structs for the purposes of efficiently predictingperformance and understanding trait overlap.

Layout of Experimental Designand Overview of Measures

This experiment contained four major catego-ries of measures: distal individual differences,proximal individual differences, concomitant proxi-mal, and criterion (TRACON) task performance.These measures, described briefly below, are dis-cussed in greater detail in the Method section.

1. Distal. Distal refers to measures that aregeneral and are typically thought to represent traitconstructs. Numerous distal measures were admin-istered in order to converge on the multipledeterminants of performance in the TRACONsimulation task. They were designed to assessindividual differences in cognitive-intellectual abili-ties, personality, vocational interests, and self-ratings of ability and self-concept (along with ashort measure of motivational skills). In addition,the Dial Reading Test and a companion test,Directional Headings Test were administered toevaluate whether these measures had assessedcommon variance among ability and nonabilityinfluences on performance.

2. Proximal. Proximal refers to measures thatare task-specific and are associated with a particu-lar situational context. A questionnaire (InterimQuestionnaire) was administered 1 or 2 days aftertrainees had viewed the instructions for perform-ing TRACON and just prior to actual task engage-ment. This questionnaire assessed negative motiva-tion and positive motivation thoughts directly

pertaining to the TRACON task and several as-pects of task-specific self-efficacy. This question-naire was also administered prior to each subse-quent TRACON session.

3. Concomitant-Proximal. Two types of con-comitant measures were used during TRACONpractice: an Interim Questionnaire was adminis-tered just prior to the beginning of each TRACONsession and a Task Perceptions Questionnaire wasadministered just after each TRACON session.Both measures assessed task-specific retrospectiveand prospective thoughts. Retrospective thoughtfrequency for the Interim Questionnaire pertainedto intrusive (negative motivation) and purposeful(positive motivation) thought frequency during thedays between TRACON sessions. For the TaskPerceptions Questionnaire, retrospective thoughtspertained to the six simulation trials in the justcompleted session. Prospective thoughts pertainedto TRACON task self-efficacy (both measures)and performance expectancies.

4. Task Performance. As in previous studies(Ackerman, 1992; Ackerman & Kanfer 1993b), themajor measures used to assess performance inTRACON related to the number of planes success-fully handled (carried to final disposition) duringeach simulation. Overall performance was mea-sured, as were separate performance componentsof handling arrival flights and overflights.

Figure 1 shows the order of administration ofthe various measures during the study.

Method

Participants

Trainees were recruited by signup sheets andflyers distributed around the campus of the Univer-sity of Minnesota. Trainees were restricted toindividuals with (a) ages between 18 and 30, (b)normal or corrected-to-normal vision, hearing, andmotor coordination, and (c) no prior experiencewith air traffic controller tasks. Ninety-three partici-pated in the experiment. Of these trainees, 42 weremen (age level for the whole sample was M = 20.4and SD = 2.75). Trainees were paid $125 each forthe 24 hr of participation, and this payment wasnot contingent on task or test performance.


Session 1 - Distal Measures (Ability)

108 min Aptitude Assessment Battery6 Motivational Skills (self-report)5 Break

53 Additional Ability Tests

Session 2 - Distal Measures (Self-Report)

25 min Personality4 Self-Estimates of Ability6 Typical Intellectual Engagement5 Break7 Vocational Interests3 Self-Concept5 Break

85 TRACON Video Training Tape(Embedded Quizzes)

Sessions 3-8 (Proximal Measures and Task Practice)

11 min Interim Questionnaire60 TRACON (two trials)5 Break

60 TRACON (two trials)5 Break

30 TRACON (one trial)6 Task Perceptions Questionnaire

Figure 1. Order of administration of ability, self-report,and task measures during the study. TRACON =Terminal Radar Approach Controller Task.

Apparatus

The computerized self-report measures and theTRACON trials were administered on IBM orCompaq 80386 computers. Information was dis-played in text mode (for the questionnaires) or incolor VGA (visual graphics array; 680 horizon-tal x 480 vertical pixels) resolution graphics (forthe TRACON trials). Input was accomplishedby keyboard responses or with a MicroTechthree-button trackball. Audio information (fromTRACON) was presented binaurally over head-phones to each trainee (with a SoundBlaster inter-face). Each computer was on a separate table andwas shielded visually and aurally by large acousticpanels that enclosed each computer. For abilitytesting, trainees were divided into groups of up to40. For TRACON task practice, trainees weredivided into groups up to 20.

Ability Tests

Two batteries of tests were administered. Thefirst was the Aptitude Assessment Battery (AAB),

the development of which was described exten-sively in Ackerman and Kanfer (1993b). The sec-ond battery was used to supplement the AAB andto assess specifically perceptual speed and verbalabilities. The AAB contains nine ability tests andan 18-item measure of motivational skills in thefollowing order: Necessary Facts, Spatial Orienta-tion, Math Knowledge, Spatial Analogy, ProblemSolving, Paper Folding, Verbal Test of SpatialAbility, Dial Reading Test, and Directional Head-ings. (The Motivational Skills measure follows theability tests in the administration order). Thesupplemental battery contained six tests in thefollowing order: Vocabulary, Letter/Number Sub-stitution, Controlled Associations, Subtraction andMultiplication, Word Beginnings, and CA-2 (Cleri-cal Ability). Descriptions of the AAB tests can befound in Ackerman and Kanfer (1993b). The threeverbal tests and the Subtraction and Multiplicationtests in the supplemental battery are from theEducational Testing Service Kit (Ekstrom, French,Harman, & Dermen, 1976). The Letter/NumberSubstitution test is a locally developed perceptualspeed test (Ackerman, 1986), and the CA-2 is aclerical checking test (Bennett & Gelink, 1951).Together, these batteries were designed to assessthe following four factors (Spatial Ability, VerbalAbility, Perceptual Speed [Simple], and Mathability). Composites for each ability are computedwith unit-weighted z scores of the component tests,and an overall general ability composite is com-puted across all five factors.

In addition to the four standard ability factors,we also collected data on the Dial Reading Test(based on a description in Guilford & Lacey, 1947;see Ackerman & Kanfer, 1993b) and a similarmeasure, the Directional Headings Test. The Di-rectional Headings Test is a test of memory,perceptual encoding, and learning that was devel-oped by the Federal Aviation Administration CivilAeromedical Institute (see Cobb & Mathews,1972). Trainees are given items that include adirectional letter, arrow, and degree heading (e.g.,S t 180). They must decide the direction impliedby these indicators (or on one part of the test theymust decide whether two of three indicators agree)or indicate that conflicting information is pre-sented in the item. We used these two tests toidentify an ability that we initially called Percep-tual Speed (Complex), which we distinguished


from an ability that is identified with more tradi-tional, simple measures of perceptual speed.

In previous predictive validity studies, both theDial Reading Test and the Directional HeadingsTest have shown high validity for predicting thesuccess of Federal Aviation Administration (FAA)air traffic controllers and for performance on ourTRACON simulation task. These two tests alsoshow a high intercorrelation (r = .74, as reportedin Ackerman & Kanfer, 1993b). In fact, similar tothe findings of the Army-Air Force with the Dialand Table Reading Test (Guilford & Lacey, 1947),our version of the Dial Reading Test and theFAA's Directional Headings Test had mathemati-cally the highest validity for TRACON perfor-mance in a large battery of cognitive and percep-tual speed ability tests. A unit-weighted z-scorecomposite was computed from these two measuresand is referred to as Perceptual-Speed-Complex(PS-Complex).

Personality Measures

In order to investigate the relations betweencriterion task performance and the broad personal-ity domain, the NEO Personality Inventory Re-vised (NEO-PIR; Costa & McCrae, 1992) wasadministered. The NEO-PIR is a 240-item inven-tory designed to assess five broad factors of person-ality. The five broad personality factors assessed bythe NEO-PIR are Extroversion, Neuroticism,Openness, Agreeableness, and Conscientiousness.The 20-item Spielberger (1983) State-Trait Anxi-ety measure was also administered under traitinstructions.

In addition to the five broad personality factorscales, a measure was included that attempts tobridge the personality and cognitive ability do-main, namely the 59-item Typical Intellectual En-gagement Scale. Typical intellectual engagement(TIE) is a personality construct that represents anindividual's aversion or attraction to tasks that areintellectually taxing and is thus related to accultura-tive and purposeful development and expression ofcertain intellectual abilities (e.g., crystallized intel-ligence). Scores on this measure have differentcorrelations with the broad ability classes of fluidand crystallized intelligence (with higher correla-tions between TIE and crystallized intelligence

(Ackerman & Goff, 1994; Goff & Ackerman,1992).

Vocational Interest Measure

Vocational interests were assessed with Ameri-can College Testing's 90-item Unisex Edition ofthe ACT Interest Inventory (UNIACT; Lamb &Prediger, 1981). The UNIACT inventory is de-signed to assess six different factors of vocationalinterests (Holland, 1959, 1985) by asking respon-dents to indicate their interest or liking for doingparticular job tasks, irrespective of their compe-tency to do them. The six scales in the battery areRealistic, Investigative, Artistic, Social, Enterpris-ing, and Conventional.

Self-Ratings of Ability and Self-Concept

For self-ratings of ability, a 31-item question-naire was used. The items represent three broadfactors of abilities: spatial-math, verbal, and self-regulation. Trainees were instructed to respondwith a self-evaluation relative to other personstheir age.

Self-concepts for competencies and knowledgein several specific areas were assessed with a32-item questionnaire developed specifically forTRACON-relevant abilities and skills. The areasassessed were self-concepts of mechanical, verbal,math, spatial, clerical, and science competencies,as well as self-concepts of self-management abilityand stress resistance.

Motivational Skills

The Motivational Skills measure was an 18-itemquestionnaire assessing aspects of self-confidencefor learning, studying, and performing in testsituations (see Ackerman & Kanfer, 1993b). TheMotivational Skills questionnaire has been shownto provide incremental prediction of TRACONperformance after abilities are partialed out.

Self-Efficacy Measures

A series of self-report questionnaires was admin-istered before and after the TRACON simulationtrials (which occurred on Sessions 3-8). Trainees


were asked to indicate their self-efficacy for compo-nents of TRACON task performance (handlingarrivals and overflights) and for overall perfor-mance (aggregate planes handled and perfor-mance relative to the population of college stu-dents). Questions were administered in an orderedfashion. In the Interim Questionnaire, questionsreferred to performance on the current session.For example, "I can land xx percentage of theplanes in the upcoming session" (where xx can bereplaced with 20,40,60,80, and 100; or "Overall, Ican perform TRACON at least as well asyyy otherstudents" (where yyy can be replaced with a few,some, many, most, or almost all). For the TaskPerceptions Questionnaire, questions pertainedonly to planes handled and overall performance.These questions referred to expected performanceon the next session. Consistent with previousmethods used to assess self-efficacy, trainees wereinstructed to indicate their confidence in perform-ing at each target level with a 0-10 scale, with 0indicating no confidence and 10 indicating certain("that I can do it"). Self-efficacy scores wereobtained by summing confidence scores acrosseach set of items.

Proximal-Concomitant Measures

In addition to self-efficacy, several other scaleswere administered either just before each TRA-CON session or just after. Prior to task perfor-mance, trainees were asked about negative motiva-tional thoughts (e.g., "Between the last session andtoday, I wished that I was done with this project")and positive motivation thoughts (e.g., "Betweenthe last session and today, I imagined myselfmaking no errors on the task"). Subsequent toeach session, trainees were asked questions aboutfrequency of thoughts related to planning, off-task,positive cognitions, negative affect, and motivationduring the current TRACON session.

Procedure

The experiment was completed in eight sessions(see Figure 1) over a 2-week period. The firstsession was devoted to ability testing. The secondsession was devoted to personality and other self-report testing and to viewing an instructional

videotape specifically designed for the TRACONtask. The 60-min videotape featured the major taskcomponents, the rules regarding operation of thecomputer interface (keyboard and trackball), dis-play characteristics, and procedures for accomplish-ing the controller task. Two quizzes were adminis-tered, one about halfway through the videotapeand the other at the end (in order to maintainmotivated attention to the task). Actual TRACONtask practice started at Session 3 and proceededthrough Session 8. Each of these practice sessionsincluded five simulation scenarios. Each scenario(trial) lasted 30 min. Order of ability testing wasconstant across trainees, but order of simulationswas counterbalanced in a Latin-square design.Breaks were given after each simulation-task trial,and self-report questionnaires were administeredat the beginning and end of each TRACONpractice session. Trainees were then debriefed andpaid. The entire experiment was completed in 24hr and included 6 hr of testing, 1 hr of task-basedvideo instruction, 2 hr of accumulated breaks, and15 hr of total time on task in TRACON.

Results

The presentation of the results from this studyhas been divided into five sections, as follows: (a)descriptive statistics of distal measures and perfor-mance measures (both overall TRACON perfor-mance and performance components); (b) distalmeasure (ability and nonability)—performance as-sociations (and a description of gender differencesin the context of task requirements); (c) concomi-tant measures, including descriptive statistics andperformance associations; (d) an integrated predic-tion model (for performance and concomitantmeasures), which uses both multiple regressionand path-analysis perspectives. In the last section,(e) we present a detailed sketch of the broaderpredictor space by means of multidimensionalscaling and factor analytic techniques.

Descriptive Statistics

Means, standard deviations, squared multiplecorrelations, and intercorrelations are providedfor 30 composite distal measures in Table 1.


Table 1Means, Standard Deviations, and Intercomlations for Nonability and Ability Measures

Variable M SD 10 11 12

Personality

1. Neuroticism2. Extroversion3. Openness4. Agreeableness

158.64 28.75 .777201.81 24.76 -.172216.11 25.06 .147198.86 26.57 -.142

5. Conscientiousness 196.05 26.93 -.3236. TIE 242.94 30.27 -.030

.727

.391

.127

.007

.223

.797

.151 .493-.173 -.083.696 .036

.652

.082 .7357. Anxiety

Interests

8. Realistic9. Investigative

10. Artistic11. Social12. Enterprising13. Conventional

Self-Concept

14. Mechanical15. Self-management16. Verbal17. Clerical18. Stress-resistant19. Math20. Spatial21. Science

Self-Estimates of Ability

22. Math/spatial23. Verbal24. Self-regulation

Ability Composites

25. Spatial26. Verbal27. PS(Complex)28. PS(Simple)29. Math

Motivational Skills

30. Motivational Skills

39.60

53.2462.9962.5066.9955.7645.89

15.1917.3519.5916.3018.5215.1517.0617.46

36.7841.6734.34

0.000.000.000.000.00

93.26

8.52

12.0214.2117.6310.3813.5117.04

4.523.223.803.972.755.953.954.43

7.486.835.60

1.001.001.001.001.00

16.16

.806

-.162-.049

.263-.087-.092-.041

-.205-.248

.164-.149-.251-.195-.316-.175

-.282.185

-.200

-.114.178

-.183-.083-.157

-.223

-.349

-.058-.108

.198

.502

.464

.014

-.071-.223

.070-.008

.223-.097-.003-.239

-.089.256.092

.048-.085

.091

.100-.189

.290

-.022

.106

.251

.672

.326-.035-.254

.023-.096

.363-.141

.055-.120

.104

.084

-.156.304

-.101

.082

.407

.097

.135-.094

.157

-.188

.042

.087

.203

.238-.125-.188

-.041.061.050

-.011-.063-.085

.107-.043

-.090-.204-.105

.251-.017-.022

.123

.133

-.028

-.332

.200-.003-.219-.077

.292

.374

.183

.506-.036

.318

.277

.276

.114

.127

.293-.002

.551

.087-.142

.102-.116

.201

.241

-.102

.114

.322

.418

.249

.046-.109

.029

.242

.349

.013

.229

.051

.045

.249

-.032.366

-.072

.062

.486

.217

.211

.120

.219

.802

-.185-.010

.147-.112-.071

.023

-.214-.198

.130-.121-.195-.148-.255-.087

-.182.133

-.224

-.156.129

-.209-.168-.140

-.300

.748

.593

.101

.184-.075

.282

.588

.115-.127

.238

.215

.382

.431

.522

.456-.223-.028

.241

.136

.354

.077

.382

.127

.727,130.145

-.326-.050

.419

.063

.063-.035

.101

.228

.364

.664

.291-.088-.207

.126

.329

.305

.121

.341

.148

.729

.275-.055-.411

-.118-.178

.396-.242-.063-.322-.052-.094

-.394.269

-.157

.010

.372-.080-.038-.202

-.030

.717

.398-.029

-.056-.028

.156-.141

.234-.163

.027-.056

-.031.235.019

-.042-.079

.015

.065-.138

.219

.771

.516

-.139.136

-.026.373.187.140

-.161-.213

.081

.106

.301

-.151-.322-.062-.053-.148

.134

Note. Diagonal entries in boldface are squared multiple correlations. Correlations greater than r ± .204 are significant at/> < .05,two-tailed. TIE = Typical Intellectual Engagement Scale; PS = Perceptual Speed.

Composites were formed by unit weighted z scoresof the tests in each content set (i.e., Spatial Ability,Verbal, PS-Complex, PS-Simple, Math). For sum-mary purposes (see below), an overall generalability estimate was also created, with a unit-

weighted composite of all 14 ability test measures.The other 16 distal measures (personality, inter-ests, self-concept, and self-ratings of ability) weresummed composites of raw items. These compos-ite measures revealed means and variances well


13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

.807

.130 .766

.168 .144 .620-.371 -.082 .020 .662.566.153.614.076.109

.325 .237 -.025

.207 .316 .164

.428 .195 -.277

.631 .150 -.118

.592 .242 .039

.692

.260 .540

.436 .224

.119 .157

.133 .269

.865

.280

.425.691.552 .746

.519 .656 .251 -.259 .376 .252 .826 .507 .560 .897-.289 -.059 .028 .663 -.167 .280 -.323 -.110 -.002 -.172 .784.247 .021 .236 .055 .384 .218 -.017 -.060 -.141 .084 .183 .654

-.005 .264 .145 -.074 -.014 .145 .231 .514 .296 .294 -.083 -.149 .690-.319 .055 .141 .546 -.056 .192 -.056 .041.138 .259 .108 .024 .169 .280 .385 .309-.071 -.015 .072 .194 -.001 .102 .137 .147.178 .352 .339 -.041 .160 .130 .552 .364

.273 -.071 .420 -.266 .258 .797

.327 .374 .005 -.199 .562 .470 .782

.091 .079 .177 -.148 .378 .439 .662 .660

.453 .582 -.100 -.145 .638 .383 .660 .464 .819

.069 .262 .320 .110 .136 .247 .315 .211 .264 .311 .213 .107 .133 .255 .200 .116 .492

within previous normative results. The self-reportmeasures generally had quite high internal-consis-tency reliabilities. In fact, only two self-reportmeasures had reliabilities below a = .80; namelySelf-Management self-concept (a = .68) and Self-Regulation self-estimate of ability (a = .77).

Skill Acquisition: TRACON Data

This section describes the effects of practice onTRACON performance. Below, the results arepresented for overall performance and are fol-lowed by the results for two important components


of overall performance, namely the handling ofarrivals and overflights.2

Overall Performance

Previous studies with this version of TRACONclearly demonstrated that the task is complex and,within the parameters of our simulations, quitedifficult early in practice (Ackerman, 1992; Acker-man & Kanfer, 1993b). On Trial 1, out of 28possible planes, the number of actual successfulplane handles was M = 5.89, SD = 4.67. Althoughwe consider this to be good initial performance, weshould note that 12% of the trainees had nosuccessful plane handles on Trial 1, even thoughall trainees had already received an explicit 1-hrvideo instruction on the task. By the end of 30trials of practice, overall performance was muchimproved (M = 19.49, SD = 6.57). At the end oftraining, 2% of the trainees were able to handle all28 of the planes. Mean performance levels acrosspractice are shown in Figure 2. A repeated mea-

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Trials of Practice

Figure 2. TRACON performance. Top: Mean totalplanes handled as a function of trials of practice.Bottom: Mean arrivals and overflights handled as afunction of trials of practice. TRACON = TerminalRadar Approach Controller Task.

sures analysis of variance (ANOVA) on overallperformance confirmed the significant effect ofpractice shown in the figure, F(29, 2668) = 145.03,p < .01.

Component Performance

Although the simulations had different numbersof arrivals (12 per trial), departures (approxi-mately eight per trial), and overflights (approxi-mately eight per trial), examination of perfor-mance on each type of flight helps illustrate thedifference in difficulty levels of the respective typesof flights. To save space and because departuresrepresent moderate difficulty (i.e., more difficultthan overflights but less difficult than arrivals),these component measures have not been in-cluded. Mean performance levels for arrivals andoverflights are shown in lower portion of Figure 2.For Trial 1, MArrivals = 1.69 and Moverflights = 2.51.At the end of practice, substantial improvement onboth types of flights was indicated: Trial 30, Mf^^^ =7.99 and Moverflights = 6.51. Although trainees werehandling a greater number of arrivals than over-flights at the end of practice, the different numberof flight types available in each trial complicatesmaking direct comparisons. That is, at the end ofpractice, trainees were handling 66% of the arriv-als but 81% of available overflights.

Distal Measure-Performance Associations

Ability-Performance Relations

Although the ability requirements of the TRA-CON task have been described elsewhere (Acker-

2 Although the purpose of the two quizzes adminis-tered during the videotape TRACON instructions wasto maintain trainee motivation for attending to theinstructional material, data from the quizzes did providesome insight into the nature of predictor-criteria rela-tions. Ability measures well-predicted a summed mea-sure across the two quizzes—all ability compositesshowed significant correlations and a multiple correla-tion indicated substantial ability overlap with the quizzes(R = .59, p < .01). In turn, quiz scores significantlycorrelated with the proximal measures of negative moti-vational thoughts (r = -.44, p < .01) and self-efficacy(r = .33, p < .01). Moreover, quiz scores were highlycorrelated with later TRACON performance across taskpractice (e.g., rSesSion i = -62; rSessjon6 = .66).


man, 1992; Kanfer & Ackerman, 1993b), for adiscussion of interrelations among performancepredictors it is necessary to document fully thesubstantial cognitive demands of the task, whentrainees (a) first confront the task and (b) haveobtained substantial task practice. For these andsubsequent analyses, we concentrated on macrochanges in performance by averaging scores onTRACON by session of practice (five simulationtrials were administered on each of the six. practicesessions). In a task that is much more quicklylearned, such large time slices obscure the learningprocess. However, for a complex task such asTRACON, this level of analysis provides addi-tional stability for the performance scores andmore manageable data for structural analysis,without sacrificing validity or obscuring the pat-terns of learning and skill acquisition. Althoughsession-to-session correlations were quite high(rs = .85, .92, .96, .96, and .95 for Session 1 withSession 2, Session 2 with Session 3, etc.), Session 1correlated with Session 6 performance (r = .77),consistent with the general finding of simplex-likepatterning of performance data over multiple occa-sions (e.g., see Ackerman, 1987; Humphreys, 1960).Although the latter result indicates that 60% of theindividual differences variance in Session 6 perfor-mance can be explained by Session 1 performance,such a relationship also indicates that even rela-tively modest differences found between predictorvariable correlates with Session 1 and with Session6 measures may be statistically significant.

The analyses below focus on overall TRACONperformance, as well as differences between perfor-mance components of handling arrivals and over-flights. First, we describe the correlations betweenthe ability composites and TRACON performanceover practice.

Overall performance. Overall performance iswell predicted across all sessions of practice by thegeneral ability composite (rs = .49, .64, .57, .56,.58, and .56 for Sessions 1-6, respectively). Exami-nation of constituent content ability measures, asshown in Figure 3, indicates that only the Mathand Spatial broad content abilities are substantial(mean r = .56 and .53, respectively) and stablepredictors of performance, the Verbal contentability shows only a modest correlation with perfor-mance (mean r = .24). The two PS abilities showsubstantial validity in predicting overall TRACONperformance with the PS-Complex (mean r = .60)

.7

.6 — ~

.5

.4 --

.3 --

.2 --

.1

.0

.7

.6

.5

.4

.3

.2

.1

.0

Broad Content AbilitiesMathVerbal

Perceptual Speed

1 2 3 4 5

Sessions of Practice

Figure 3. Correlations between ability composites andoverall performance (planes handled) on the TerminalRadar Approach Controller Task over sessions of prac-tice (five simulation trials per session). Top: Contentability composites (spatial, math, and verbal). Bottom:Perceptual speed (PS) Complex and Simple.

showing higher validity than PS-Simple (meanr = .44).3 These results are consistent with previ-ous task analyses and extant validity data in twoprevious studies (Ackerman, 1992; Ackerman &Kanfer, 1993b).

Secondary analyses can be predicated on gen-eral performance across all sessions of practice.Toward this end, a measure of overall planeshandled was computed across all 30 trials andcorrelated with the general and the constituent-content and PS-ability composites. The correla-tions between abilities and overall performanceacross all sessions of practice are as follows:General, r = .59; Spatial, r = .56; Math, r = .59;Verbal, r = .24; PS-Complex, r = .63; and PS-

3 As shown in Table 1, PS-Complex correlates substan-tially with the Spatial ability composite (r = .66) andwith PS-Simple (r = .56), suggesting that the abilityrepresented by this composite falls somewhere in-between the two more traditional reference factors.


Simple, r = .46 (all significant p < .05). A single-step multiple regression (with all ability measuresexcept for the aggregate general composite) yieldeda multiple R of .70 or an R2 = .49. That is, takentogether, the five ability composites accounted for49% of the variance in overall TRACON perfor-mance across 15 hr of task practice.

Arrivals and overflights. Separating the perfor-mance components of arrivals and overflights illus-trates the task analysis that described arrivals asmore difficult and more demanding of spatialabilities than overflights, which are hypothesizedto be more easily learned and thus relatively betterpredicted by PS abilities than broad content abili-ties. Figure 4 shows that initial performances onarrivals and overflights were equivalently well-predicted by the general ability composite. Overpractice, validity coefficients for arrivals and over-flights diverge, with general ability showing highercorrelations with arrival performance and lowercorrelations with overflights.

Figure 5 shows arrival and overflight correla-tions for four ability composites (Verbal abilityhaving been dropped as a result of demonstratedlack of substantial validity for predicting TRA-CON performance, especially relative to the otherability measures). Again, concordant with ourexpectations, Spatial and Math ability diverge invalidity for predicting arrivals and overflights afterinitial practice (with greater validity for predictingarrivals). The two PS measures show greater simi-

-i 1 1 1 r

General Ability

1 2 3 4 5 6


Figure 4. Correlations between general ability compos-ite and performance components (arrivals and over-flights) on the Terminal Radar Approach ControllerTask over sessions of practice (five simulation trials persession).

larity in predicting performance of both arrivalsand overflights.

Summary. The results described above supportthree points about the relations between abilitiesand performance on TRACON, as follows:

1. Performance throughout practice is well-predicted by measures of cognitive and intellectualability. The measures most highly associated withtask performance were Spatial, Math, and PS-Complex abilities. Verbal ability showed only amodest validity for predicting performance onTRACON at any stage of practice.

2. Across practice, ability measures accountedfor nearly 50% of the variance in TRACONperformance.

3. Arrival and overflight components of perfor-mance showed substantial and equivalent abilitydemands early in practice, but with practice, spa-tial and math abilities were more predictive ofarrival than overflight components. In contrast, PSabilities showed generally stable correlations withboth arrivals and overflights across practice.

Gender Differences

A robust finding in the ability literature is groupdifferences in performance on tasks that requirecomplex spatial information processing, with menshowing an overall performance advantage overwomen (e.g., Lohman, 1986). Our previous studieshave demonstrated similar group differences inperformance on TRACON, with male traineesshowing a substantial mean performance advan-tage relative to female trainees. Because some ofthese differences may be determined by abilitydifferences, by personality, and other nonabilitydifferences, it is necessary to review the TRACONperformance by gender.

TRACON Performance

Similar to previous studies (e.g., Ackerman,1992), there was both a significant main effect fortrainee gender on overall performance, F(l, 91) =19.66, p < .01, and an interaction between traineegender and sessions of practice, F(5, 455) = 5.49,p < .01. The means for each group over practiceare shown in the top panel of Figure 6. As shown,the interaction is indicated as a divergence in


.6

.5

.4

.3

.2

.1

.0

.7

.6

.5

.4

.3

.2

.1

.0

Spatial Ability

<~Wi m— ^_

Math Ability

T 1 1 \—ii—i 1 1 1 1 r

Perceptual Speed (complex)_| 1 1 1 \-2 3 4 5 6Sessions of Practice

2 3 4 5 6

Sessions at Practice

Figure 5. Correlations between ability composites and performance components(arrivals and overflights) on the Terminal Radar Approach Controller Task, oversessions of practice (five simulation trials per session).

2 3 4 5Session of Practice

Figure 6. Mean Terminal Radar Approach ControllerTask performance by gender over sessions of practice.

performance between the two groups with increas-ing practice on TRACON.

Concordant with the earlier ability-perfor-mance analyses of arrivals and overflights, whichshowed higher correlations between arrival perfor-mance and math-spatial abilities than for over-flights, the differences between gender groupsshowed greater advantages to men over women inhandling arrivals. The lower panels of Figure 6show these effects. A repeated measures ANOVAof arrivals showed a main effect of gender, F(l, 91) =32.25, p < .01, and a significant interaction be-tween gender and practice, F(5, 455) = 7.74, p <.01, with performance diverging between groupswith practice. In contrast, a repeated measuresANOVA on overflights showed a significant, butsmaller effect of gender, F(l, 91) = 4.32, p < .05,and a nonsignificant interaction between genderand practice, F(5, 455) = 2.05, ns. At the end ofpractice, men and women have equivalent perfor-mance on overflights (MMen = 6.44,MWomen = 6.21).Again, these results support the assertion thathandling arrivals requires more complex spatial


information processing than does handling over-flights.

Nonability Predictor-Performance Relations

Personality. Personality constructs have oftenbeen implicated in determinations of stress reactiv-ity or stress resistance in performance contexts(e.g., see Vickers, 1991). We first correlated mea-sures of the five major personality composites withoverall performance on TRACON, None of thecorrelations reached traditional levels of signifi-cance: Neuroticism, r = —.15; Extroversion, r =—.02; Openness, r = —.01; Agreeableness, r = .02;Conscientiousness, r = .02; TIE, r = —.03; andAnxiety, r — -.19. A multiple regression withthese seven composites similarly yielded no signifi-cant result (R = .24 or R2 — .06). Simple correla-tions between personality composites and dailysession measures of TRACON yielded similarresults, as did separate correlations for arrivals andoverflights. Only Anxiety revealed a significantcorrelation with performance and only for Session1 (r = -,2&,p < .Ol).4

Vocational interests. It has often been claimedin the vocational literature that emotional stress isgenerated when a mismatch exists between anindividual's vocational interests and the task thatthe individual is asked to perform (see, e.g., Dawis& Lofquist, 1984). We computed correlations be-tween theme scales of the UNIACT and aggregateTRACON performance, with the following re-sults: Realistic, r = .31; Investigative, r = .19;Artistic, r = -.14; Social, r = -.07; Enterprising,r = -.07; and Conventional r = .07. The correla-tions between the Realistic and Investigative themescales and performance are the only ones thatreached both statistical and practical significance,jointly accounting for about 10% of the variance inTRACON performance; R = .31, F(2, 90) = 4.90,p < .01.

Self-ratings of ability and self-concept. The threecomposite scales of ability self-ratings (SR) andthe eight measures of academic ability self-conceptwere factor analyzed to yield three main compos-ites. The first composite (SRI) was comprised ofitems from math, spatial, mechanical, and scienceself-concept and self-ratings of similar abilities.The second (SR2) was comprised of items fromverbal self-concept and self-ratings of generalverbal ability and more specific reading, vocabu-

lary, and writing abilities. The third (SR3) wascomprised of items from self-management, cleri-cal, and stress-resistant self-concept, and self-ratings of self-control and coping abilities. Correla-tions between these three composites andTRACON performance indicated that the firstcomposite was highly and consistently related toperformance (mean r = .46), and the remainingtwo composites were essentially unrelated to TRA-CON performance (SR2: mean r = -.08; SR3:mean r = .12). These results indicate that selectedself-ratings of ability and academic ability self-concept are effective predictors of performance ina complex task. In the aggregate, these measuresaccounted for 24% of the overall variance inTRACON; R = .49, F(3, 87) = 9.10, p < .01. Inparticular, the ratings of math, spatial, and me-chanical abilities accounted for about 20% of thevariance in TRACON performance across practicesessions.

Motivational skills. As in previous studies (Ack-erman & Kanfer, 1993b), an 18-item measure ofMotivational Skills was included in this study.Consistent with results from predicting success inair traffic control tasks (both in the laboratory andin the field with FAA trainees), the MotivationalSkills measure showed consistent, modest, andsignificant correlations with performance across allsessions of TRACON performance (meanr = .25).For aggregate performance, the correlation withMotivational Skills was r = .27, p < .01, or about7% of the variance.

Proximal Measures

In addition to the sets of distal individual differ-ences measures administered prior to trainees'exposure to TRACON, an additional set of proxi-

4 Several authors (e.g., Snow, 1989) have argued fornonlinear relations between personality measures andability measures and by implication between personalityand task performance measures. We examined qua-dratic and cubic regressions between personality andTRACON performance measures and did not find anysignificant increment in associations beyond the linearregressions. However, it is possible that because ourmeasures were geared to a normal sample, such effectsmay not generalize to a sample of tests that are bettergeared to assessment of psychopathology and/or totrainees that are more broadly sampled from the popula-tion at large.


mal (and concomitant) measures was adminis-tered. Specifically, prior to each session of TRA-CON practice, the Interim Questionnaire (16items) was completed by the trainees. Similarly, atthe end of each session of TRACON practice, aTask Perceptions Questionnaire (44 items) wascompleted by the trainees. Of critical importanceis the fact that the first Interim Questionnaire wasadministered prior to any direct TRACON experi-ence but 1 or 2 days after the video instruction onTRACON was presented to the trainees. The firstinterim questionnaire captures the proximal antici-pations of trainees for their initial confrontationwith TRACON.

Interim questionnaire. Two subscales from theInterim Questionnaire were of critical importancefor the current investigation: negative motivationand positive motivation thoughts. The Negativescale is composed of six items such as "I wishedthat I was done with this project" and "I gotdiscouraged when I thought about today's ses-sion." The Positive scale is composed of sevenitems such as "I imagined myself doing extremelywell on today's session" and "I made a specificplan for how to perform some part of the task."Both measures significantly predicted initial perfor-mance on TRACON Session 1, negative motiva-tion (r = -.29 and -.38) and positive motivation(r = .26 and .22) for arrivals and departures, respec-tively. In addition, the negative motivation itemsadministered prior to task engagement showedstability in predicting TRACON performance overthe subsequent days of task practice (see Figure 7).The positive motivation thoughts only showedstable correlations with the overflight componentof TRACON.

Self-efficacy prior to task engagement. At theend of the Interim Questionnaire, trainees wereasked to indicate their self-efficacy for componentsof TRACON task performance (handling arrivalsand overflights) and for overall performance (aggre-gate planes handled and performance relative tothe population of college students). As with thenegative motivation and positive motivation items,these measures were administered 1 to 2 days aftertrainees had viewed the 1-hr instructional video onTRACON, but just before the trainees had anydirect experience in performing TRACON.

All four of the self-efficacy measures corre-lated significantly with initial TRACON perfor-mance (SEArrivals, r = .23; SE0verflights> r — -32;

.5

.4 --

.3

.2

.1

Negative Motivational Thoughts—I 1 1 1

Positive Motivational Thoughts

1 2 3 4 5 6


Figure 7. Correlations between self-reports of task-relevant thoughts and the Terminal Radar ApproachController Task performance over sessions of practice.

SEp|anes handled* r = -26; and SEpetformmce, r = .38).

These measures were also highly intercorrelated,so that an aggregate measure of self-efficacy corre-lated (r = .32) with initial TRACON performance.These correlations increased after the first sessionand were stable throughout the remaining ses-sions. For example, the composite self-efficacymeasure correlated (r = .40, .38, .39, .42, and .43)with performance on the respective five remainingsessions of TRACON performance. The self-efficacy measure that showed the highest indi-vidual validity was the performance measure, whichcorrelated r = .48 with overall performance onTRACON, accounting for 23% of the variance inperformance across practice sessions.

In summary, measures of proximal thoughts andself-efficacy prior to engagement in TRACONperformance showed validity for predicting indi-vidual differences in task performance. Specifi-cally, greater incidence of negative motivationthoughts was significantly and substantially associ-ated with lower TRACON performance through-out practice, and to a somewhat smaller degree,greater incidence of positive motivation thoughtswas significantly associated with positive TRA-CON performance throughout practice. Also,


higher levels of self-efficacy (especially as per-ceived in comparison with other college students)were associated with TRACON performance.

Concomitant Variables andConsequences of Practice:

Measures Taken During Skill Acquisition

Negative Motivation and Positive Motivation

Repeated measures ANOVAs on the negativemotivation and positive motivation scales revealedsignificant increases for both negative motivationthoughts, F(5,455) = 4.02, and positive motivationthoughts, F(5, 455) = 9.92, over sessions of prac-tice (bothps < .01). However, the negative motiva-tion scale showed a main effect for gender, F(l, 91) =3.95, p < .01)—with women reporting higherfrequency of negative motivation thoughts. Inaddition, a significant interaction of Gender xSession, F(5, 455) = 3.08, p < .01, was obtained.This reflected a convergence of male and femalereporting of negative motivation thoughts overpractice. No gender main or interaction effectswere found for positive motivation.

Incidence of negative motivation or positivemotivation thoughts, subsequent to the initial TRA-CON practice session, showed diminished validityfor predicting TRACON performance. In the ag-gregate, these results suggest that trainees' initialmeta-motivational strategies for task learning haveimportant and persistent influences on task perfor-mance.

Task Perceptions Questionnaires

In contrast to the Interim Questionnaires (whichwere administered prior to each TRACON ses-sion), Task Perceptions Questionnaires were ad-

ministered immediately following the last simula-tion for each of the six TRACON sessions. Thesemeasures concerned the frequency of thoughtsthat trainees had during the TRACON session.Measures of planning (e.g., "I thought about howmany more planes I could handle"), off-task (e.g.,"I daydreamed while doing the task"), positivecognitions (e.g., "I thought about ways to excel onthis task"), Negative affect (e.g., "I got upset whenI made a mistake"), and motivation ("I gave thetask my maximum effort") were included. Re-peated measures ANOVAs were performed foreach of these measures with gender as a between-subject variable and practice as a within-subjectvariable. These ANOVAs are presented in Table2. Results indicate several significant differencesbetween gender groups, over practice, and interac-tions between two variables.

For frequency of planning thoughts, there wasno main effect of gender, but significant increasesin planning over the course of practice and asignificant interaction of Gender x Practice. Ex-amination of the means showed that the interac-tion was the result of women reporting less plan-ning than men early in task practice but moreplanning later in task practice.

For off-task thoughts and negative affectthoughts, only a main effect of practice was signifi-cant. The means showed that although Session 1off-task and negative-affect thought frequency washigh (at the point where performance was lowest),the frequency dropped substantially on Session 2.Off-task thoughts showed a slow increase throughlater sessions of practice, whereas negative affectthoughts remained stable after Session 2. Theseresults are consistent with earlier studies (e.g.,Kanfer & Ackerman, 1989), that indicate off-taskthoughts are often indicative of problems in nega-tive affective responses to stressful task demands,

Table 2Repeated Measures Analysis of Variance for Task Perceptions Measures Over TRACON Practice Sessions

Task perceptions measure

Planning

Variable

GenderSessions of practiceGender x Sessions

of practice

df1,905,450

5,455

MSB

669.4643.52

43.52

F

0.2014.12**

3.14**

Off-task

MSE

379.0644.94

44.94

F

0.4313.92**

1.25

Positive cognitions

MSE

1,005.6165.19

65.19

F

0.470.78

1.47

Negative affect

MSE

820.8771.83

71.83

F

1.0222.49**

0.43

Motivation

df

1,905,450

5,450

MSE F

137.80 0.009.14 2.02

9.14 0.33

Note. TRACON = Terminal Radar Approach Controller.**p < .01.


that is, emotion control problems (see Kanfer &Ackerman, 1995).

Interim Self-Efficacy Measures

Four measures of self-efficacy for TRACONperformance were administered prior to each TRA-CON session. These pertained to self-efficacy for(a) percentage of planes handled, (b) performance(relative to other students), (c) overflights, and (d)arrivals. Mean self-efficacy scores for each of thefour measures over Practice x Gender are pre-sented in Figure 8. Repeated measures ANOVAsfor each are shown in Table 3. For each of the fourself-efficacy measures, there were substantial andsignificant main effects of gender on scores (withwomen having lower mean self-efficacy for allaspects of the TRACON task), and sessions ofpractice (with self-efficacy dropping from Session 1to Session 2 and then rising after Session 2).Significant Gender x Session of Practice interac-tions were found for SEHandied and SEArrivals, with adivergence of scores for men and women and nosignificant interactions for SEPerf0nnance or SEoverflights.It is interesting that even after five sessions ofTRACON practice, the final SEPerformance and

SEArriva|S ratings by women did not exceed those ofmen at Session 1, even though mean overallperformance on TRACON showed a substantialadvantage by women at Session 6 vs. men atSession 1.

Because self-efficacy has been discussed as apotentially causal construct for predicting taskperformance and repeated measures of self-efficacy were obtained during the course of taskpractice, this study provides a means for evaluatingthe changes in predictive validity for self-efficacy astrainees gained experience and skill over the courseof the six TRACON sessions. Figure 9 representsseveral correlational perspectives toward evaluat-ing the predictive validity of self-efficacy acrosspractice sessions for each of the four presessionmeasures of self-efficacy (i.e., SEHandied, SEPerformance,SEArrivals, and SEoverflights). The first curve (Day 1)shows the correlations between self-efficacy, asmeasured prior to any direct experience withperformance on TRACON and performance onTRACON across the six practice sessions. OverallTRACON performance is the criterion variablefor SEHandled and SEPerformance; for SEArrivais, thecriterion is arrivals handled; and for SEoverflights. thecriterion is overflights handled). As shown in

~ ' SE (Performance - Normative)

1 2 3 4 5 6


2 3 4 5


Figure 8. Mean self-efficacy (SE) expectation composite scores by gender overTRACON Terminal Radar Approach Controller Task practice. Measures administeredprior to each TRACON session.


Table 3Repeated Measures Analysis of Variance for Self-Efficacy (SE) Measures Over TRACON Practice Sessions

SEMeasure Motivational thought

SEperfon ^^Overflights Negative Positive

Variable df MSE MSB MSE MSE df MSE MSE

Interim questionnaire

Gender 1,91 214.66 29.14** 289.46 31.92** 331.36 15.39** 292.92 32.46** 1,91 189.29 3.95* 345.24 0.57Sessions of practice 5,455 21.70 61.73** 23.64 13.92** 27.37 32.02** 25.91 69.58** 5,455 15.92 4.02** 25.32 9.92**Gender x Sessions

of practice 5,455 21.70 3.23** 23.64 0.70 27.37 1.14 25.91 7.06** 5,455 15.92 3.08** 25.32 1.34

Task perceptions questionnaire

Gender 1,91 317.59 33.13** 258.28 30.29**Sessions of practice 5,455 18.4826.08** 21.2699.33**Gender x Sessions

of Practice 5,455 18.48 0.74 21.26 1.74

Note. TRACON = Terminal Radar Approach Controller; SE = self-efficacy.*p < .05. **p < .01.

1 2 3 4 5 6Sessions of Practice

2 3 4 5 6Sessions of Practice

Day 1 (SE, with Performance^)Synchronous (SEn with Performance,,)

Lagged (SEn with Performance^,)Partial (SEn with Performance,, by Performance ,̂)

Figure 9. Correlations between four self-efficacy (SE) measures and performance onthe Terminal Radar Approach Controller Task (TRACON). Curves represent (a)correlations between pretask (Day 1) self-efficacy and TRACON performance overpractice sessions; (b) synchronous (same session) correlations between self-efficacy andTRACON performance; (c) lagged correlations (self-efficacy with previous sessionTRACON performance); and (d) partial correlations (synchronous correlations withprevious session TRACON performance partialed out).


Figure 9, the validity coefficients for self-efficacy(Day 1) and subsequent performance were gener-ally stable across TRACON practice, though small-est for the SEArrivals and largest for SEPerformance

measures.The second curve (synchronous) shows the cor-

relations between SE measures administered justprior to each day's session and performance onTRACON for that session. These correlationsincrease substantially as practice on TRACONproceeds and is most pronounced for SEArrivais, andleast pronounced for SEPerfonnance- The third curve(lagged) shows the correlations between each ses-sion's SE measure (except for the measure admin-istered in Session 1) and the previous sessionTRACON performance. For all of the SE mea-sures, these correlations are either larger than thesynchronous correlations or are equivalent to thesynchronous correlations, indicating that any ses-sion's SE measures may have more in common, orat least as much in common, with the previousday's performance level (a retrospective view) thanwith the upcoming performance (a prospectiveview). Indeed, if the previous session's perfor-mance is partialed out of the synchronous correla-tions (the fourth curve: partial), it appears clearthat prior performance accounts for nearly all ofthe shared variance between SE estimates andTRACON performance. The only general excep-tion to this finding was for Session 3, wheresignificant positive partial correlations are foundfor SEHandied and SEArrivals and marginally sig-nificant (p — .09) correlations were found forSEPerformance and (p = .06) for SEoverflights- That is,self-efficacy, as measured prior to Session 3, wasfound to account for a significant though small,mean (r = .22, r2 = .05) amount of Session 3 TRA-CON performance variance, not explained by priortask performance. However, at the end of TRA-CON practice, SE measures showed no significantcorrelations with task performance, once priorsession performance variance was partialed out.Although it is not possible to specifically determinethe test-retest reliabilities for the self-efficacymeasures given the changing nature of perfor-mance and self-efficacy during training, it is impor-tant to note that a correction for attenuation of theraw correlations for self-efficacy measures andTRACON performance would lead to slightlylarger partial correlations (e.g., see Stouffer, 1936,regarding the interpretation of partial correlationsbased on measures with imperfect reliability).

Task Perceptions Self-Efficacy Measures

Two measures of self-efficacy for TRACONperformance were administered immediately aftereach TRACON session. These pertained to self-efficacy for (a) percentage of planes handled and(b) performance (relative to other students) in aformat that was identical to the Interim Question-naire measures. Mean self-efficacy scores for eachof the two measures over practice by gender arepresented in Figure 10. Repeated measuresANOVAs for each are shown in the top part ofTable 3. These measures show similar patterns tothose from the Interim Questionnaire, with theexception of the absence of a drop in scores fromSession 1 to Session 2. The obvious reason for thisdifference is that the Session 1 self-efficacy mea-sures were administered after the trainees hadtheir initial experience with TRACON and thefirst Interim Questionnaire was administered priorto actual performance experience. Significant maineffects for gender and for sessions of practice werefound for both measures, but no significant interac-tions were found for either measure. Such resultsindicate general improvement in self-efficacy withpractice and persistently higher self-efficacy scoresfor men.

Integrated Prediction Model for Performanceand Concomitant Variables: MultipleRegression and Correlation Results

Performance

As in previous studies with TRACON (e.g.,Ackerman, 1992; Ackerman & Kanfer, 1993b), rawcorrelations between various predictors and taskperformance only give partial information, in that

2 3 4 5

Sessions of Practice2 3 4 5


Figure 10. Mean self-efficacy (SE) expectation compos-ite scores by gender over Terminal Radar ApproachController Task practice. Measures administered aftereach session.


each set of predictors is treated in isolation.Common variance among predictor variables servesto limit the overall predictive validity of an inte-grated battery of tests. Multiple correlation andregression procedures allow for the determinationof the incremental validity of each set of predictorvariables in a general equation for predicting taskperformance. The general logic of the analyses inthis section was first to enter variables that areassumed to be causal antecedents. That is, abilitymeasures were entered into the prediction equa-tion and then followed by the other distal variables(self-concept and vocational interests) and proxi-mal variables (motivational thoughts and self-efficacy). Finally, gender was added to the equa-tion to examine whether gender differencesaccounted for variance in performance after all thepredictor measures had been taken into account.

However, only measures obtained prior to taskperformance were included in the prediction equa-tion. We dropped the personality measures fromthese computations because they did not providesignificant raw correlations with TRACON perfor-mance. Table 4 presents these analyses of first(Session 1) and last (Session 6) session perfor-mance for TRACON performance.

Prediction of individual differences in perfor-mance was quite good. In total, the predictorsaccounted for roughly 50% of the individual differ-ences variance in overall TRACON performance.However, arrival task component was better pre-dicted than the overflight task component acrosstask practice. In fact, there was a decrease in theamount of variance predicted for the overflightcomponent over practice (from 39.2% to 32.8%)possibly because of ceiling effects on overflights

Table 4Hierarchical Multiple Regression and Correlation Results for Performance

Step R2

Session 1

Incrementin/?2 F to add

Session 6

df R2Increment

in/?2 F to add dfPrediction of overall TRACON performance

1. Ability2. Self-concept3. Interests4. Motivational thoughts5. Self-efficacy6. Gender

Total

.384

.403

.410

.457

.457

.467

.467

.384 10.58**

.020 .90

.007 .47

.046 3.31*

.000 .01

.010 1.46NA 4.75**

5, 85 .4403, 82 .4582, 80 .4662, 78 .5401, 77 .5471, 76 .568

14, 76 .568

.440

.018

.008

.074

.007

.021NA

13.35**0.940.536.32**1.563.767.14**

5,853,822,802,781,771,76

14,76

Prediction of overflight component


Total

.331

.340

.347

.390

.392

.392

.392

.321 8.41**

.009 0.38

.007 0.42

.043 2.75

.002 0.39

.000 0.00NA 3.51**

5, 85 .2263, 82 .2362, 80 .2402, 78 .3281, 77 .3281, 76 .328

14, 76 .328

.226

.010

.004

.088

.000

.000NA

4.97**0.370.195.10**0.020.072.66**

5,853,822,802,781,771,76

14,76

Prediction of arrival component


Total

.317

.351

.378

.417

.420

.457

.457

.317 7.89**

.034 1.44

.027 1.71

.039 2.61

.003 0.51

.037 5.13*NA 4.57**

5, 85 .4473, 82 .4642, 80 .4752, 78 .5051, 77 .5151, 76 .565

14, 76 .565

.447

.017

.011

.030

.010

.050NA

13.74**0.890.812.411.578.74**7.06**

5,853,822,802,781,771,76

14,76

Note. TRACON = Terminal Radar Approach Controller Task; NA = not applicable.*p < .05. **p < .01.


late in practice, and an increase in the amount ofvariance predicted for the arrival component overpractice (from 45.7% to 56.5%). Cognitive and PSabilities, admittedly the first variables entered intothe equation, also accounted for the largest amountof individual differences variance in overall TRA-CON task performance, as well as respectiveoverflight and arrival components of the TRA-CON task across extensive practice. Of the remain-ing predictor measures, only proximal negative andpositive motivational thoughts consistently pro-vided incremental validity, which was more pro-nounced in the overflight component than thearrival task component. Finally, gender accountedfor a significant incremental amount of variance inthe arrival component of task performance butaccounted for virtually no incremental variance inthe overflight component.

Concomitant measures. Considerable specula-tion appears in the literature about the determi-nants of individual differences in task self-efficacyprior to task engagement and subsequent to taskpractice (e.g., see Bandura, 1986; Feltz, 1982; Gist& Mitchell, 1992; Mitchell et al., 1994). We com-puted hierarchical regressions and correlations forinitial and final self-efficacy with the same order ofvariable entry that was used for predicting perfor-mance. We have presented here only the resultsfor the self-efficacy measure that pertained tooverall performance, although similar results wereobtained with the other three self-efficacy mea-sures. As shown in Table 5, distal measures ac-counted for 31.3% of the variance in initial self-efficacy (i.e., prior to task engagement) with ability(22.8%) and self-concept (8.3%) each accountingfor significant amounts. The proximal measures of

Table 5Hierarchical Multiple Regression and Correlation Results for Proximal-Concomitant Measures

Session 1 Session 6

Increment IncrementStep


Total


Total


Total

R2

.228

.311

.313

.472NA.515.515

.190

.296

.347NANA.360.360

.131

.279

.300NANA.306.306

in/?2 F to add df R2

Prediction of self-efficacy (performance)

.228 5.03** 5,85 .212

.083 3.28* 3,82 .297

.002 .13 2,80 .343

.159 11.68** 2,78 .375NA NA .474 -.043 6.90* 1,77 .500NA 6.29** 13,77 .500

Prediction of negative motivational thoughts

.190 4.00** 5,85 .132

.106 4.12** 3,82 .272

.051 3.11* 2,80 .283NA NA .335NA NA .336.013 1.57 1,79 .351NA 4.04** 11,79 .351

Prediction of positive motivational thoughts

.131 2.56* 5,85 .194

.148 5.62** 3,82 .306

.021 1.19 2,80 .311NA NA .495NA NA .498.006 .69 1,79 .525NA 3.17** 11,79 .525

in/?2

.212

.085

.046

.032

.099

.026NA

.132

.140

.011

.052

.001

.015NA

.194

.112

.005

.184

.003

.027NA

F to add

4.56**3.31**2.802.02

14.48**3.99*2.94**

2.58*5.20**0.583.030.161.762.94**

4.08**4.42**.30

14.21**.49

4.38*6.01**

df

5,853,822,802,781,771,76

14,76

5,853,822,802,781,771,76

14,76

5,853,822,802,781,771,76

14,76Note. NA = not applicable.*p < .05. ""p < .01.


motivational thoughts also accounted for a signifi-cant and substantial amount of variance (15.9%).Initially, it may appear that the covariance be-tween the motivational thought measures andself-efficacy was the result of their being measuredon the same questionnaire (self-efficacy was at theend). However, the fact that mitigates against thisexplanation is that the motivational thought ques-tions were retrospective (i.e., thoughts you havehad over the past l-to-2 days), and the self-efficacymeasure was prospective (i.e., your confidence inperforming during the upcoming session). Genderaccounted for an additional 4% of variance in taskself-efficacy even after the inclusion of the distaland proximal measures.

Self-efficacy prior to the last session of practice(Session 6) was also well predicted by distal mea-sures and proximal measures that were adminis-tered prior to task engagement, even though self-efficacy for Session 1 and Session 6 were notextremely highly correlated (r = .52, p < .01). Infact, ability, self-concept, and interests accountedfor similar amounts of both Session 1 and Session 6self-efficacy variance. However, initial motiva-tional thoughts added little to the prediction ofSession 6 self-efficacy, whereas Session 1 self-efficacy added a significant incremental prediction(10% variance accounted for) beyond the distalmeasures first included in the prediction equation.Such results are consistent with an interpretationof self-efficacy. That is, there are traitlike proper-ties not univocally influenced by short-term experi-ences, and, for at least some trainees, self-efficacymay have been resilient even in the face of failure.Finally, gender accounted for an additional 3% ofvariance in self-task self-efficacy even after theinclusion of the distal and proximal measures.

Hierarchical regression analyses were applied toSession 1 and Session 6 motivational thoughtvariables (also shown in Table 5). These analysesshow that—similar to the prediction of self-efficacy—distal measures (i.e., ability and self-concept) account for about 30% of the variance inmotivational thought frequency at Session 1 as wellas at Session 6. However, at Session 6, a significantamount of variance (18%) in positive motivationalthoughts was accounted for by Session 1 positivemotivational thoughts. Gender provided only incre-mental prediction validity for positive motivationalthoughts and then only at Session 6 (3%).

Path Analyses

Use of hierarchical multiple regression, as de-scribed above, provides a partitioning of varianceaccounted for in the dependent variable, which ispartly determined by the order in which variablesare entered into the equation. Even though weargue that entry of distal variables first and proxi-mal measures afterward is the only ordering thatmakes logical sense, alternate perspectives mightalso be fruitful, for example, where interests areentered prior to self-concept. One way to estimatethe relative contribution of the distal and proximalmeasures to predicting performance is to produceby means of LISREL or other modeling tech-niques a path model that simultaneously estimatesall path coefficients. We conducted a series of teststhat attempted to provide a model for predictingperformance, which was based on the variablesdescribed in the regression section above. In per-forming these tests, we found that it was useful tocreate aggregate measures for some variables (e.g.,ability and self concept). Also, several variableswere dropped from the model (e.g., interests)because of findings of nonsignificant paths. Themodels we derived are shown in Figure 11. The

Dish Session 1

Figure 11. LISREL derived path models for distalpredictors, proximal predictors, and task performance.Top: Session 1TRACON performance. Bottom: Session6 TRACON performance. TRACON = Terminal Ra-dar Approach Controller Task.


models showed reasonable fit statistics accordingto standard methods for assessing adequacy ofmodels. The model for Session 1TRACON perfor-mance, shown in the upper panel of Figure 11,yielded x2(7) = 8-5' P = -29- General Fit Index(GFI) = .97, Normed Fit Index (NFI) = .95, andNon-Normed Fit Index (NNFI) = .98. An identi-cal model when applied to Session 6 TRACONperformance (lower panel of Figure 11) yieldedX2(7) = 12.32,p = .09, GFI = .96, NFI = .93, andNNFI = .93. (See Bentler & Bonett, 1980, fordiscussion of these fit statistics and their interpre-tation.)

Figure 11 shows the coefficients for all signifi-cant paths in the model. Of specific interest, wefound similar path coefficients for ability (g) toTRACON performance (.48 for Session 1 and .52for Session 6). In addition, a significant path wasfound for ability to self-concept. From self-concept, significant paths were found to bothnegative and positive motivational thoughts, aswell as a direct path to task self-efficacy, suggestingthat self-concept serves as a mediator for theassociation between ability and self-efficacy be-cause there is no significant direct path from abilityto self-efficacy. In addition, a significant path wasfound from negative motivational thoughts to TRA-CON performance—which is consistent with andgenerally supportive of the role that deficits inemotion control have on task performance (e.g.,see Kanfer & Ackerman, 1990,1995). When viewedin the context of the hierarchical regressions re-ported earlier, these results suggest that individualdifferences in ability account for the major share ofpredicted individual differences in TRACON per-formance across practice, but that self-report mea-sures, especially motivational thoughts as medi-ated by self-concept, also account for a nontrivialamount of variance in both initial and final TRA-CON performance. Self-efficacy, as assessed priorto task engagement, provides no significant contri-bution to the prediction of individual differences inperformance that is not in turn accounted for byother measures (e.g., self-concept and motiva-tional thought frequency).

Analysis of the Predictor Space

On the basis of the wide array of distal measureswe administered in this study, the discussion in theliterature on the general nature of the predictor

space (e.g., Kanfer, Ackerman, Murtha, & Goff,1995) and the above findings that are indicative ofcommunality among several families of predictormeasures, we decided to focus more specifically onthe predictor space. We performed two differenttypes of analyses that served to illuminate eommu-nalities across the domains of ability, personality,vocational interests, self-concept, and self-esti-mates of ability. The first analysis was based on aradex-type approach (for examples, see Acker-man, 1988; Marshalek, Lohman, & Snow, 1983)and used a multidimensional scaling (MDS) tech-nique. The second was a more traditional factoranalysis approach. Each is discussed in turn below.

MDS

Although personality variables did not showindividual or incremental validity for predictingTRACON performance, there is a broader inter-est in evaluating the communality of personalityand other distal constructs, such as ability and.vocational interests. Thus we included personalitywith the other distal measures for this analysis. Inkeeping with the radex approach, we had correla-tions serve as proxies for similarity estimates.Intercorrelations among the following variableswere computed: personality (7 measures), voca-tional interests (6 measures), self-concept (8 mea-sures), self-ratings of ability (3 measures), ability(5 composite measures), and Motivational Skills (1measure) for a total of 30 variables. (For correla-tions among these variables, see Table 1.) Giventhat some of the correlations were negative (e.g.,among personality measures), a constant (1) wasadded to all correlations. The matrix was thensubjected to KYST-3 MDS (Kruskal, Young, &Seery, 1973), a two-dimensional solution was ex-tracted (Stress Formula 1 = .20), and, as is custom-ary, the solution (see Figure 12) was rotated to aprincipal-components orientation.

There are many interesting similarities and dif-ferences in Figure 12, but we have focused on a fewsimilarities that we found especially salient. First,the MDS solution recovered the Holland hexagonstructure typically found with vocational interestmeasures (and is explicitly expected from theUNIACT measures). There are several close prox-imities among interest variables and personalitymeasures (e.g., Social Interests and Extroversion;Artistic Interests and Openness; Conventional In-


• Personality• Vocational Interestsi Self-ConceptA Self-Estimates of Ability• AbBtyO Motivational Skills

Spatial.

Math * --

lAgreeableness

to,PS/Simp!e

Investigative

Mech (SCU RealisticMath (SC

Self-Management (SC)̂Clerical (SC)/ A

Conventional^Conscientiousness,

Stress-resisJcDcelSC)

Self-Re

*~~,,^ Artistic

O Hot Skills/ Verbal (SC)AMiety

/ ^Verbal

' Social---to'

Extroversion

f uterpnsng

Figure 12. KYST-3 two-dimensional solution to distalmeasures. Spat = spatial; Mot = motivational; Reg =regulation; PS = perceptual speed; Mech = mechanical;TIE = Typical Intellectual Engagement; SC = self-concept.

terests and Conscientiousness). In addition, thereare other similarities among interests and bothnonability and ability measures: Realistic Interestswere close to Math, Mechanical, and ScienceSelf-Concepts; Enterprising Interests were close toSelf-Regulatory self-estimates of ability; and Inves-tigative Interests were close to all five abilitymeasures but closest to the two PS composites.

Factor Analysis

The 30-measure intercorrelation matrix was alsosubjected to factor analysis. Although the MDSyields meaningful distances between variables, thegoal of factor analysis is to estimate a small set ofhypothetical constructs that underlie the commonvariance among the measures. For the factoranalysis, a parallel analysis criterion (Humphreys& Montanelli, 1975) was used for the determina-tion of the number of factors to extract. Theparallel analysis indicated that eight factors shouldbe extracted. A principal axis factor analysis wasused, with iterated communalities, and rotated to avarimax criterion. Oblique rotations were alsoperformed, but they provided neither an increasein clarity nor substantial positive manifold, which

thus ruled out the usefulness of a hierarchicalrotation of factors. The varimax solution is pre-sented in Table 6.

Two of the factors appear to be uniquely identi-fied within a narrow domain. Factor 3 is identifiedmainly with ability (General Ability), though mathself-concept also loads on the factor, and Factor 7(Neuroticism/Anxiety) is identified with only thetwo respective personality measures. In contrast,Factor 1 captures a range of self-concept variables(Mechanical, Math, Spatial, and Science) andstrong loadings of Realistic and Investigative voca-tional interests. Thus we identified Factor 1 asScience/Mechanical Self-Concept and Interests.Factor 2 captured a range of variables, includingSelf-Regulation self-ratings of ability and Self-Management self-concept. In addition, the person-ality trait of Conscientiousness, and the vocationalinterest traits of Enterprising and Conventionalalso had substantial loadings on this factor. Thuswe identified Factor 2 as a Self-Managementfactor. Factor 4 picked up several cultural andverbal-related traits, including Verbal Ability, Ver-bal Self-Concept, TIE, and Openness personalitytraits, and Artistic interests. Thus we identifiedFactor 4 as "Intellectence" (after Welsh, 1991). Incomparison to Factor 4, Factor 5 was contrastive,in that positive loadings were found for Verbalself-ratings of ability, Verbal Self-Concept, andVerbal Ability, but negative loadings of Conven-tional interests and negative loadings of MathSelf-Concept. In addition, a nearly salient loadingwas found for the measure of Motivational Skills.By contrasting the salient loadings between Factor4 and Factor 5, it seems that Factor 5 taps aSelf-Presentation of Verbal Ability more so thanactual verbal ability. The distinction between thesetwo factors is supported by an examination of thepattern of raw correlations, as well as the MDSresults reported above, where Verbal Self-Conceptand Verbal self-ratings of ability were generallysomewhat removed from objective measures ofVerbal ability. Factor 6 is clearly an Extroversionfactor, and it has substantial loadings of Social andEnterprising vocational interests as well. Factor 8captures mostly Spatial competence, including Spa-tial Ability and Spatial Self-Concept, though it alsohas a surprising positive loading from Agreeable-ness.

Taken together, the MDS and factor analysisresults clearly point to overlap among various trait


Table 6Factor Loadings for Measures and Composites: Varimax Rotation

Factor

Measure

NeuroticismExtroversionOpennessAgreeablenessConscientiousnessTIEAnxiety

RealisticInvestigativeArtisticSocialEnterprisingConventional

MechanicalSelf-ManagementVerbalClericalStress ResistanceMathSpatialScience

Math-SpatialVerbalSelf-Regulation

SpatialVerbalPS-ComplexPS-SimpleMath

Motivational skills

1

-.158-.105

.095-.042

.108

.108-.108

.666

.639-.067

.036-.199

.199

.810

.109-.119

.184

.214

.500

.687

.782

.715-.089-.093

.285

.084

.291-.003

.391

.261

2 3 4

Personality

-.216 -.085 .137-.065 .048 .139-.157 .042 .735-.068 .029 .215

.742 -.032 -.061

.125 .210 .706-.163 -.100 .015

UNIACT Interests

.116 .127 .188-.129 .124 386-.246 -.117 .702-.036 -.062 .282

.470 -.030 -.090

.576 .103 -.270

Self-Concept

.117 .003 -.085

.580 .122 .084

.027 .072 .424

.558 .076 -.094

.357 .157 .084357 367 -.176.007 .113 -.081.075 .160 .128

Self-Ratings of Ability

349 .268 -301.032 .052 .185.591 -.263 -.131

Ability Measures

.033 .530 -.042-.080 .528 .537

.045 .809 .069-.094 .713 .109

.219 .699 -.028

Motivational Skills

.225 .141 .047

5

.058

.091

.150-.145

.004

.181

.077

-.222-.052

.114

.106-.094-.447

.055

.099

.574-.148

.254-344

.044

.069

-.121.927.177

-.007.402.004.127

-.077

.298

6

-.093.798.292.086

-.071.111

-.158

.026-.132

.124

.621

.631

.271

-.020-.203-.014

.101

.200

.049

.037-.205

.100

.192

.093

-.003-.269

.070

.057-.206

.225

7

.807-.223

.047-.194-.212-.061

.877

-.042-.057

.182-.099

.092

.243

-.076-.173

.093

.036-.138

.049-.164-.048

-.010.143

-.145

.002

.121-.103-.127-.009

-.225

8

-.041.088.109.400.081

-.087-.121

.041-.093

.255

.057-.126-.257

.069

.070-.036-.135-.077-.188

343-.074

-.093-.118

.008

.664-.044

.025

.046

.203

.008

Note. Salient loadings are indicated in boldface. TIE = Typical Intellectual Engagement;UNIACT = Unisex edition of the ACT Interest Inventory; PS = Perceptual Speed.

domains, but especially among self-concept, voca-tional interests, and some measures of personality.However, Neuroticism and Anxiety, though highlyintercorrelated with one another, show little over-lap with other trait domains under considerationhere.

One finding from these data that has somecontroversial history in the self-concept literature(e.g., Marsh, 1986,1990; Skaalvik & Rankin, 1990)is the negative loadings of math self-concept andmath/spatial self-ratings of ability on factors thathave high salient positive loadings of verbal self-


concept and verbal self-ratings of ability. Negativecorrelations were found between math self-con-cept and math-spatial self-ratings of ability on theone hand and verbal self-concept and verbal self-ratings of ability on the other hand. That is, in thissample of college students, there appears to besome "ipsatizing" of the math-verbal continuum(e.g., if individuals believe that if they are high inverbal abilities, then they are low on math abili-ties). This may reflect trainees' preference for onetype of material or another or a more pervasiveimplicit sense of compensatory abilities. Raw cor-relations show Math Ability and Math Self-Concept to be similarly associated with one an-other (r = .55), as are Verbal Ability and VerbalSelf-Concept (r = .54), even though Math Self-Concept correlates with Verbal Self-Concept(r = .28), and Math Ability correlates with VerbalAbility (r = .38).

Postscript: PS-Complex Ability

Given the dominant influence of the PS-Com-plex ability measure in predicting performance inthe current study and in previous studies forcomplex skill performance (e.g., Ackerman &Kanfer, 1993b; Cobb & Mathews, 1972; Guilford& Lacey, 1947), we performed a set of follow-upanalyses to evaluate communality of PS-Complexwith nonabiliry distal measures. The primary ratio-nale for this analysis was the observation (from aset of performance-stress studies) that the oftenfrustrating, iterative strategy necessary for success-ful performance of the Dial Reading and Direc-tional Headings tests may yield a performance-based assessment of individual differences insusceptibility to emotion control deficits. We firstsubjected PS-complex (made up of the Dial Read-ing and Directional Headings tests) to cross-correlation analyses with nonability predictors ofperformance in TRACON. Similar to correlationswe have found in an earlier study (Ackerman &Kanfer, 1994), PS-Complex correlated significantlywith several nonability measures as follows: Anxi-ety (r = -.21), TIE (r = .22), Realistic Interests(r = .35), and Spatial-Mechanical-Science Self-Concept (r = .34). For proximal and concomitantmeasures, PS-complex also correlated significantlywith task self-efficacy (performance) across prac-tice (rs = .37, .33, .38, .39, .36, and .31) for Sessions1-6 respectively. Of considerable importance were

the correlations between PS-Complex and initialnegative motivation and positive motivationthoughts. Consistent with our interpretation of theemotion-control influences on performance in theDial Reading and Directional Headings tests,PS-Complex correlated negatively with negativemotivation scores (r = -.31), indicating that lowscores on PS-Complex were associated with greaterreporting of negative motivational thoughts priorto task practice. In addition, PS-Complex showedessentially a zero correlation with positive motiva-tional thoughts prior to Session 1 (r = .07), eventhough negative and positive motivation thoughtswere highly negatively correlated (i.e., r— -.58).Also, a further result that was consistent with ourspeculation about the tests, PS-Complex corre-lated significantly with a self-report measure ofStress-Resistance Self-Concept (r = .28). PS-Complex was the only objective ability test compos-ite that significantly correlated with this nonabilitymeasure.

Finally, we subjected PS-Complex to a genderanalysis. Even though the constituent tests havesome surface-level spatial content (which wouldlead to an expectation of higher mean scores formen), only a marginally significant difference wasfound for gender; MWomen = —.18, MMen = .22,/(91) = l.96,p = .05.

When these results are considered in light of theprevious literature on validity of the Dial Readingand Directional Headings tests for predicting per-formance, it seems clear that a class of objectivetests may provide insight into ability-nonabilityrelations and may also provide a source for bothconstruct and criterion-related validity above andbeyond so-called g measures. Such findings alsomay illuminate why some PS ability measures havehigh predictive validity for intermediate stages ofskill acquisition (e.g., Ackerman, 1988, 1990) andfor the individual-differences construct identifiedby Ackerman and Woltz (1994) as "propensity-to-learn." (See Ackerman & Goff, 1995, for a reviewof ability-nonability relations.)

Discussion

Summary

Individual differences in performance on a com-plex skill acquisition task are well-predicted by avariety of distal cognitive and noncognitive trait


measures, including ability, interests, motivationalskills, self-concept, and self-ratings of ability. Inaddition, proximal measures of negative and posi-tive motivational thoughts and self-efficacy werealso substantially related to task performance.However, substantial overlap in variance was foundamong the distal variables, proximal variables, andbetween distal and proximal variables. By simulta-neous examination of these variables in predictingtask performance, early and late in practice, wedemonstrated that much of the valid criterion-related variance among nonability predictors wasvariance that was shared with cognitive abilityvariables. Communalities among the cognitive andnoncognitive determinants of performance re-vealed by MDS and factor analysis suggest thataptitude complexes for skill acquisition can beidentified and, in turn, used in an integratedselection program (for a discussion of aptitudecomplexes, see Snow, 1989). In addition, the dem-onstrated communalities may provide a furtherbasis for building models of learner-task relationsthat better represent the interrelations of cognitiveand noncognitive traits. Models that take thesecognitive and noncognitive determinants into con-sideration may ultimately allow for a joint predic-tion of the effort allocated to a task and strategicdecisions made by trainees (e.g., choice behaviorsthat are guided by self-concept or self-efficacy).

The substantial effort in this experiment de-voted to pretask ability testing and assessment ofindividual differences in a variety of nonabilitypredictors of performance provided for a widerassessment of the predictor construct space andthe joint assessment of performance on a complexskill-learning task than has been possible in previ-ous studies. In many ways, the TRACON task isrepresentative of the kind of real-world tasks forwhich performance under time stress is observed.The task is continuous (over a 30-min watch), caninvolve unpredictable events, and has substantialinformation processing stress (by virtue of aircraftpilots giving repeated requests for permissionsover the headset and when separation conflicts arein progress and warning alerts are broadcast visu-ally and auditorally). The only salient aspects ofthe task that are not characteristic of the real-world is that no actual loss of human life occurswhen mistakes are made, and no direct threat isimposed on the individual operator. Nonetheless,we have observed behaviors in the laboratory that

are consistent with the inference that the task isstressful. Incidences of trainees who become upsetin conjunction with poor performance—who curseor become agitated with the equipment (e.g.,pounding the keyboard or trackball)—have beenobserved over the course of our experiments withTRACON. The observed negative motivationthought occurrences between task sessions areconsistent with this inference as well. Some train-ees report worry and apprehension prior to theTRACON sessions and report negative affectivereactions at the conclusion of TRACON sessions.Although there is controversy in the field aboutwhether the job of air traffic controller is of greaterstress than other jobs (e.g., see Hopkin, 1988, for areview), there is virtually no doubt that the task isstressful during the skill acquisition phase formany trainees.

Conclusions

This study indicated that many ability and non-ability predictors provide significant correlationswith individual differences in TRACON perfor-mance over practice. The nonability predictorsrange from distal vocational interests to academicand ability self-concept, motivational skills to proxi-mal reports of negative and positive motivationalthoughts and self-efficacy expectations. However,when objective ability measures are first enteredinto a multiple regression equation for predictingtask performance, little incremental validity isfound for the nonability predictors. In this study,only incidents of proximal negative and positivemotivational thoughts consistently added signifi-cant incremental prediction to the equation. Thatis, common variance among most of the nonabilitypredictors, ability measures, and TRACON perfor-mance is accounted for by several objective abilitycomposite measures.

Most prominent among these ability predictorswere the two tests identified as PS-Complex, namelythe Dial Reading and the Directional Headingstests. However, it is important to note that suchvariables also share considerable common vari-ance with the important nonability predictors ofperformance. These results support the inferencethat both ability and nonability influences of indi-vidual differences in performance under stressfulconditions may be well accounted for by a set ofobjective tests with characteristics similar to the


Dial Reading and Directional Headings tests.Although there is a literature concerning thesetests, investigators have not examined the nonabil-ity determinants of performance on these tests,even though these tests have demonstrated thehighest validity in large batteries of ability tests forpredicting job performance under stress. Our re-sults strongly suggest that future investigationsshould be conducted to broaden the constructspace underlying these measures, for example, bydeveloping a battery of tests that have similarinformation-processing characteristics. With alarger battery of these tests, it may be possible toaccount independently for the ability and nonabil-ity determinants of performance on such mea-sures. Such a result could have implications fornew assessment instruments for selection of appli-cants for jobs that involve substantial stress expo-sure.

An extensive set of analyses of the predictorspace illustrated many sources of communalityacross domains that are often implicitly consideredin the literature to be orthogonal. Specific associa-tions between personality and interest measures,between self-concept and ability measures, andbetween abilities and both interests and self-concept are particularly illuminating, especially forthose researchers who are interested in predictionsof individual differences in task performance. Thatis, in the absence of ability predictors, manymeasures of self-concept, vocational interests, andself-efficacy estimates show significant correlationswith complex task performance across task prac-tice. However, when a broad array of abilitymeasures is entered into a regression equationfirst, few of these nonability measures provideincremental predictive validity for performance.Of course, an alternate viewpoint would suggestthat these nonability predictors can serve as surro-gates for objective ability measures. For example,Math, Spatial, and Verbal Self-Concept measurescorrelated with the objective ability composites(r = .55, .51, and .55, respectively).

Traditional approaches to prediction tend toemphasize ability or nonability domains. The pat-tern of results obtained in this study suggests thatthese traditional perspectives may obscure impor-tant relations among predictors and performancecriteria. For example, path analyses indicate thesignificant role of both ability and nonability influ-

ences for predicting performance. From a tradi-tional ability perspective, the results provide confir-mation of ability influences on performance. Froma traditional nonability perspective, the resultsprovide support for the notion of significant non-ability influences on performance (by means ofproximal measures that stem from integration ofcontemporary motivation-based theories of perfor-mance). But what is typically neglected by bothperspectives is the critical linkage between indi-vidual differences in abilities and individual differ-ences in related interests, self-concepts, and proxi-mal motivational measures, such as self-efficacy.Although theorists have long posited such a link-age (Cattell, 1946; Wechsler, 1950), empiricalevidence that supports this type of ability-nonabil-ity linkage has been sparse (see Ackerman & Goff,1995, for a review).

The link between ability and nonability predic-tors has important implications for broader concep-tual issues and for prediction of performance asfollows:

1. For prediction of performance, individualdifferences in abilities obviously play a key role.Ability measures accounted for roughly 30% to45% of the variance in complex task performanceover practice. However, an individual's percep-tions of these abilities as they relate to diversedomains of behavior may importantly mediate theability-performance relations in unforeseen ways,particularly when successful performance over timeinvolves emotion control and persistence. That is,objective ability measures are moderately-to-highly correlated with self-concept and self-estimates of ability, the self-concept measureshave a direct influence on negative and positivemotivational thoughts and on task self-efficacy.How trainees view their abilities may be importantin determining how they confront a novel task, butthe objective measures of ability predict better howwell they actually perform the task.

2. Controversy over the association among mea-sures of major personality dimensions and perfor-mance has focused on matching personality mea-sures (grounded in theories of personalitystructure) to the performance (criterion) space.Results of this study suggest another fruitful ave-nue for resolution of these issues pertains to abroadening of the predictor domain to includemeasures and concepts from vocational psychology


and self-concept theories. In the integrated per-spective presented here, self-concept and interestmeasures represent constructs with important linksto broad personality dimensions, as well as perfor-mance criteria.

3. In applied settings, thorough assessment ofindividual differences in cognitive abilities is oftenprecluded for a variety of reasons. Individualdifferences in nonability predictors may provide auseful partial proxy for such assessments whenability data are unavailable. However, as shown inthe present results, when performance is complexand the demands of the task change over practice,such proxy measures may introduce systematic,non-performance-related variance into the equa-tion. Indeed, we found gender differences in taskself-efficacy, even after variance attributable toability, self-concept, interests, and the like hadbeen removed. It remains to be seen what deter-mines these residual differences in self-efficacyand whether such differences in turn influencechoice decisions on how to engage the task assign-ment. It will be important in future investigationsto discover the sources of discrepancies betweenindividual differences in task performance andtask self-efficacy.

4. Measures typically considered to be abilitypredictors (such as the Dial Reading and Direc-tional Headings tests) may indeed capture substan-tial variance in the nonability domain. Questionsof the relative effectiveness of these measures fordetermining nonability variance, compared withself-report nonability measures depend, of course,on the construct under consideration. However,these initial findings provide a potentially impor-tant direction for future research on performance-based assessment of nonability constructs. In anapplied setting such as job selection, concernsabout demand characteristics (e.g., "faking good")on nonability measures might be circumvented byadministering objective ability tests that tap indi-vidual differences in nonability constructs, giventhat it is generally impossible to fake good on anability test. The high validity demonstrated by thePS-Complex measure suggested that this particu-lar trait may represent an especially useful addi-tion to more traditional batteries of cognitiveabilities in predicting individual differences inperformance on complex skill tasks.

References

Ackerman, P. L. (1986). Individual differences in infor-mation processing: An investigation of intellectualabilities and task performance during practice. Intelli-gence, 10, 101-139.

Ackerman, P. L. (1987). Individual differences in skilllearning: An integration of psychometric and informa-tion processing perspectives. Psychological Bulletin,102, 3-27.

Ackerman, P. L. (1988). Determinants of individualdifferences during skill acquisition: Cognitive abilitiesand information processing. Journal of ExperimentalPsychology: General, 117, 288-318.

Ackerman, P. L. (1990). A correlational analysis of skillspecificity: Learning, abilities, and individual differ-ences. Journal of Experimental Psychology: Learning,Memory, and Cognition, 16, 883-901.

Ackerman, P. L. (1992). Predicting individual differ-ences in complex skill acquisition: Dynamics of abilitydeterminants. Journal of Applied Psychology, 77, 598-614.

Ackerman, P. L., & Goff, M. (1994). Typical intellectualengagement and personality: Reply to Rocklin (1994).Journal of Educational Psychology, 86, 150-153.

Ackerman, P. L., & Goff, M. (1995). Intelligence, person-ality, and interests: Historical review and a model foradult intellect. Unpublished manuscript under review.

Ackerman, P. L., & Kanfer, R. (1993a). Development ofan ability-based selection instrument for the MARC-AirTraffic Controller Training Program (Final Report toMinnesota Technical College System, Minnesota AirTraffic Control Training Center, University of Minne-sota). Minneapolis, MN: Author.

Ackerman, P. L., & Kanfer, R. (1993b). Integratinglaboratory and field study for improving selection:Development of a battery for predicting air trafficcontroller success. Journal of Applied Psychology, 78,413-432.

Ackerman, P. L., & Kanfer, R. (1994). Improving problem-solving and decision-making skills under stress: Predic-tion and training (Final Report to the Office of NavalResearch, Contract N00014-91-J-4159, University ofMinnesota). Minneapolis, MN: Author.

Ackerman, P. L., & Woltz, D. J. (1994). Determinants oflearning and performance in an associative memory/substitution task: Task constraints, individual differ-ences, and volition. Journal of Educational Psychology,86, 487-515.

Adams, J. A. (1987). Historical review and appraisal of


research on the learning, retention, and transfer ofhuman motor skills. Psychological Bulletin, 101, 41-74.

Bandura, A. (1986). Social foundations of thought andaction: A social cognitive theory. Englewood Cliffs, NJ:Prentice-Hall.

Bandura, A., & Wood, R. (1989). Effect of perceivedcontrollability and performance standards on self-regulation of complex decision making. Journal ofPersonality and Social Psychology, 56, 805-814.

Barrick, M. R., & Mount, M. K. (1991). The Big Fivepersonality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.

Bennett, G. K., & Gelink, M. (1951). CA-2. New York:Psychological Corporation.

Bentler, P. M., & Bonett, D. G. (1980). Significance testsand goodness of fit in the analysis of covariancestructures. Psychological Bulletin, 88, 588-606.

Borman, W. Q, Rosse, R. L., & Abrahams, N. M.(1980). An empirical construct validity approach tostudying predictor-job performance links. Journal ofApplied Psychology, 65, 662-671.

Buckley, E. P., DeBaryshe, B. D., Hitchner, N., & Kohn,P. (1983). Methods and measurements in real-time airtraffic control system simulation (DOT/FAA/CT-83/26). Atlantic City Airport, NJ: U.S. Department ofTransportation, Federal Aviation Administration.

Byrne, B. M. (1984). The general/academic self-conceptnomological network: A review of construct validationresearch. Review of Educational Research, 54, 173-186.

Cattell, R. B. (1946). Description and measurement ofpersonality. Yonkers-on-Hudson, NY: World Book.

Cobb, B. B., & Mathews, J. J. (1972). A proposed new testfor aptitude screening of air traffic controller applicants(FAA-AM-72-18). Washington, DC: U.S. Depart-ment of Transportation, Federal Aviation Administra-tion.

Costa, P. T., Jr., & McCrae, R. R. (1992). Revised NEOPersonality Inventory (NEO PI-R) and NEO Five FactorInventory (NEO-FFI): Professional manual. Odessa,FL: Psychological Assessment Resources.

Cronbach, L. J., & Snow, R. E. (1977). Aptitudes andinstructional methods: A handbook for research oninteractions. New York: Irvington.

Dawis, R. V., & Lofquist, L. H. (1984). A psychologicaltheory of work adjustment: An individual-differencesmodel and its applications. Minneapolis, MN: Univer-sity of Minnesota Press.

DeNisi, A. S., & Shaw, J. B. (1977). Investigation of theuses of self-reports of abilities. Journal of AppliedPsychology, 62, 641-644.

Digman, J. M. (1990). Personality structure: Emergence

of the five-factor model. Annual Review of Psychology,41, 417-440.

Ekstrom, R. B., French, J. W., Harman, H. H., &Dermen, D. (1976). Kit of factor-referenced cognitivetests. Princeton, NJ: Educational Testing Service.

Ellis, A., & Conrad, H. S. (1948). The validity ofpersonality inventories in military practice. Psychologi-cal Bulletin, 45, 385-426.

Feltz, D. L. (1982). Path analysis of the causal elementsin Bandura's theory of self-efficacy and an anxiety-based model of avoidance behavior. Journal of Person-ality and Social Psychology, 42, 764-781.

Feltz, D. L. (1994). Self-confidence and performance. InD. Druckman & R. A. Bjork (Eds.), Learning, remem-bering, believing (pp. 173-206). Washington, DC: Na-tional Academy Press.

Fitts, P., & Posner, M. I. (1967). Human performance.Belmont, CA: Brooks/Cole.

Fleishman, E. A. (1972). On the relation betweenabilities, learning, and human performance. AmericanPsychologist, 27, 1017-1032.

Gist, M., & Mitchell, T. R. (1992). Self-efficacy: Atheoretical analysis of its determinants and malleabil-ity. Academy of Management Review, 17, 183-211.

Goff, M. (1994). Understanding vocational self-conceptand related domains: A hierarchical factor-analyticapproach. Unpublished doctoral thesis, University ofMinnesota.

Goff, M., & Ackerman, P. L. (1992). Personality-intelligence relations: Assessing typical intellectualengagement. Journal of Educational Psychology, 84,537-552.

Goldberg, L. R. (1990). An alternative "description ofpersonality:" The Big-Five factor structure. Journal ofPersonality and Social Psychology, 59, 1216-1229.

Guilford, J. P., & Lacey, J. I. (Eds.). (1947). Army AirForces Aviation Psychology Program Research Reports:Printed Classification Tests. (Report No. 5). Washing-ton, DC: U.S. Government Printing Office.

Holland, J. L. (1959). A theory of vocational choice.Journal of Counseling Psychology, 6, 35-45.

Holland, J. L. (1973). Making vocational choices: A theoryof careers. Englewood Cliffs, NJ: Prentice-Hall.

Holland, J. L. (1985). Making vocational choices: A theoryof vocational personalities and work environments (2nded.). Englewood Cliffs, NJ: Prentice-Hall.

Hopkin, V. D. (1988). Air traffic control. In E. L. Wiener& D. C. Nagel (Eds.), Human factors in aviation (pp.639-663). New York: Academic Press.

Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp,J. D., & McCloy, R. A. (1990). Criterion-related


validities of personality constructs and the effect ofresponse distortion on those validities [Monograph].Journal of Applied Psychology, 75, 581-595.

Humphreys, L. G. (1960). Investigations of the simplex.Psychometrika, 25, 313-323.

Humphreys, L. G., & Montanelli, R. G., Jr. (1975). Aninvestigation of the parallel analysis criterion fordetermining the number of common factors. Multivar-iate Behavioral Research, 10, 193-206.

Kanfer, R. (1987). Task-specific motivation: An integra-tive approach to issues of measurement, mechanisms,processes, and determinants. Journal of Social andClinical Psychology, 5, 237-264.

Kanfer, R., & Ackerman, P. L. (1989). Motivation andcognitive abilities: An integrative/aptitude-treatmentinteraction approach to skill acquisition. Journal ofApplied Psychology Monograph, 74, 657-690.

Kanfer, R., & Ackerman, P. L. (1990). Ability andmetacognitive determinants of skill acquisition and trans-fer (Final Report to Air Force Office of ScientificResearch). Minneapolis: University of Minnesota.

Kanfer, R., & Ackerman, P. L. (in press). A self-regulatory skills perspective to reducing cognitiveinterference. In I. G. Sarason (Ed.), Cognitive interfer-ence: Theories, methods, and findings. Hillsdale, NJ:Erlbaum.

Kanfer, R., Ackerman, P. L., Murtha, T., & Goff, M.(1995). Personality and intelligence in industrial andorganizational psychology. In D. H. Saklofske & M.Zeidner (Eds.), International handbook of personalityand intelligence (pp. 577-602). New York: Plenum.

Kelso, G. J., Holland, J. L., & Gottfredson, G. D. (1977).The relation of self-reported competencies to apti-tude test scores. Journal of Vocational Behavior, 10,99-103.

Krumboltz, J. D. (1979). A social learning theory ofcareer decision making. In A. M. Mitchell, G. B.Jones, & J. D. Krumboltz (Eds.), Social learning theoryand career decision making. Cranston, RI: Carrol.

Krumboltz, J. D., Mitchell, A. M., & Jones, G. B. (1976).A social learning theory of career selection. Counsel-ing Psychologist, 6, 71-81.

Krumboltz, J. D., Mitchell, A. M., & Jones, G. B. (1978).A social learning theory of career selection. In J. M.Whitely & A. Resnikoff (Eds.), Career counseling (pp.100-127). Monterey, CA: Brooks/Cole.

Kruskal, J. B., Young, F. W., & Seery, J. B. (1973). Howto use KYST, a very flexible program to do multidimen-sional scaling and unfolding. Murray Hill, NJ: BellLaboratories.

Kuhl, J. (1985). Volitional mediators of cognition-

behavior consistency: Self-regulatory processes andaction versus state orientation. In J. Kuhl & J.Beckmann (Eds.), Action control: From cognition tobehavior (pp. 101-128). New York: Springer-Verlag.

Kyllonen, P. C., & Christal, R. E. (1989). Cognitivemodeling of learning abilities: A status report ofLAMP. In R. F. Dillon & J. W. Pellegrino (Eds.),Testing: Theoretical and applied perspectives (pp. 146-173). New York: Praeger.

Kyllonen, P. C., & Woltz, D. J. (1989). Role of cognitivefactors in the acquisition of cognitive skill. In R.Kanfer, P. L. Ackerman, & R. Cudeck (Eds.), Abili-ties, motivation, and methodology: The Minnesota Sym-posium on learning and individual differences (pp.239-280). Hillsdale, NJ: Erlbaum.

Lamb, R. R., & Prediger, D. J. (1981). The UnisexEdition of the ACT Interest Inventory. Iowa City, IA:American College Testing.

Levine, E. L., Flory, A., & Ash, R. A. (1977). Self-assessment in personnel selection. Journal of AppliedPsychology, 62, 428-435.

Lohman, D. F. (1986). The effect of speed-accuracytradeoff on sex differences in mental rotation. Percep-tion & Psychophysics, 39, 427-436.

Lowman, R. L. (1991). The clinical practice of careerassessment: Interests, abilities, and personality. Washing-ton, DC: American Psychological Association.

Marsh, H. W. (1986). Verbal and math self-concepts: Aninternal/external frame of reference model. AmericanEducational Research Journal, 23, 129-149.

Marsh, H. W. (1990). The structure of academic self-concept: The Marsh/Shavelson model. Journal ofEducational Psychology, 82, 623-636.

Marsh, H. W., Byrne, B. M., & Shavelson, R. J. (1988). Amultifaceted academic self-concept: Its hierarchicalstructure and its relation to academic achievement.Journal of Educational Psychology, 80, 366-380.

Marshalek, B., Lohman, D. F., & Snow, R. E. (1983).The complexity continuum in the radex and hierarchi-cal models of intelligence. Intelligence, 7, 107-127.

Mathieu, J. E., Martineau, J. W., & Tannenbaum, S. I.(1993). Individual and situational influences on thedevelopment of self-efficacy: Implications for trainingeffectiveness. Personnel Psychology, 46, 125-147.

McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson,M. A., & Ashworth, S. (1990). Project A validityresults: The relationship between predictor and crite-rion domains. Personnel Psychology, 43, 335-354.

Mitchell, T. R., Hopper, H., Daniels, D., George-Falvy,J., & James, L. R. (1994). Predicting self-efficacy and


performance during skill acquisition. Journal of Ap-plied Psychology, 79, 506-517.

Norwich, B. (1987). Self-efficacy and mathematicsachievement: A study of their relation. Journal ofEducational Psychology, 79, 384-387.

Prediger, D. J, (1991). Project CHOICE: Validity ofinterest and ability measures for student choice ofvocational program. Career Development Quarterly, 40,132-144.

Sadri, G., & Robertson, I. T. (1993). Self-efficacy andwork-related behavior: A review and meta-analysis.Applied Psychology: An International Review, 42, 139-152.

Schiefele, U. (1991). Interest, learning, and motivation.Special Issue: Current issues and new directions inmotivational theory and research. Educational Psy-chologist, 26, 299-323.

Skaalvik, E. M, & Rankin, R. P. (1990). Math, verbal,and general academic self-concept: The internal/external frame of reference model and gender differ-ences in self-concept structure. Journal of EducationalPsychology, 82, 546-555.

Snow, R. E. (1989). Aptitude-treatment interaction as aframework for research on individual differences inlearning. In P. L. Ackerman, R. J. Sternberg, & R.Glaser (Eds.), Learning and individual differences.Advances in theory and research (pp. 13-59). NewYork: Freeman.

Spielberger, C. D. (1983). The State-Trait Anxiety Inven-tory. Palo Alto, CA: Consulting Psychologists Press.

Stouffer, S. A. (1936). Evaluating the effect of inad-equately measured variables in partial correlationanalysis. Journal of the American Statistical Association,31, 348-360.

Super, D. E. (1957). The psychology of careers. NewYork: Harper & Row.

Super, D. E. (1963). Toward making a self-concepttheory operational. In D. E. Super, R. Starishevsky, N.Matlin, & J. P. Jordaan (Eds.), Career development:Self-concept theory (pp. 17-32). New York: CollegeEntrance Examination Board.

Vickers, R. R., Jr. (1991). Stress reactivity: Five-factorrepresentation of a psychobiological typology (ReportNo. 91-26). San Diego, CA: Naval Health ResearchCenter.

Wechsler, D. (1950). Cognitive, conative, and non-intellective intelligence. American Psychologist, 5, 78-83.'

Welsh, G. S. (1991). Are intelligence tests the only wayto assess intelligence? In D. Cicchetti & W. M. Grove(Eds.), Thinking clearly about psychology. Vol. 1: Mat-ters of public interest (pp. 137-157). Minneapolis:University of Minnesota Press.

Woltz, D. J. (1988). An investigation of the role ofworking memory in procedural skill acquisition. Jour-nal of Experimental Psychology: General, 117, 319-331.

Appendix

TRACON Simulation Task

The task used for this research is a uniquelymodified professional version (VI .52) of TRA-CON simulation software developed by WessonInternational. Versions of this program are in usein several locations in the United States (includingthe FAA, National Aeronautics and Space Admin-istration and Department of Defense and severalcolleges and university airway sciences programs)for training of air traffic controllers. Modificationsfor the current instantiation of the program haveallowed for the collection of a variety of datadescribed in more detail below.

Descriptions of the TRACON simulations areprovided in Ackerman (1992) and Ackerman andKanfer (1993b). The following discussion abstractsthat material with deviations in procedures specifi-

cally noted. The task requires that trainees learn aset of rules for air traffic control, including readingflight strips, declarative knowledge about radarbeacons, airport locations, airport tower handoffprocedures, en route center handoff procedures,plane separation rules and procedures, monitoringstrategies, and strategies for sequencing planes formaximum efficient and safe sector traversal. Inaddition, trainees are required to acquire human-computer interface skills that include issuing track-ball-based commands, menu retrieval, keyboardoperations, and integration between visual andauditory information channels. However, the simu-lator task represents a substantial reduction ofrules and operational demands in comparison tothe real-world job of air traffic controllers.


Display

TRACON presents the trainee with a simulatedcolor radar screen depicting a region of airspace,very high frequency omnidirectional range stations(VOR), airports, sector boundaries, and rangerings. Planes are identified by icon on the radarscope, with a data tag that indicates plane identifi-

cation and altitude information. In addition, twosets of flight strips are presented at the right side ofthe display, one "pending" and the other "active."Each flight strip contains information about aparticular flight, including identification informa-tion, plane type, requested speed and altitude, andsector entry and exit destination information (seeFigure Al). At the bottom of the screen is a

N926|L ,-''. 11*3

•:-', N5A 'V.-"' 204.6V,

AF632 D10 250k 90'SWETT BEBEE Ctr

N79GJ C425 235k 60'COFEX DPA Twr

N926L PA46 220k 110'w OBK HINCK CtrN54PX C425 190k 40'

HINCK PWK TwrN9P L23 250k 80'

THORR DPA TwrNM891 747 250k 100'

MOW COROL CtrActives

N7TF L23 250k 110'PLANO MDU Twr

.'N38G sf .100 /' / N6B C425 156k 50'

NILES PWK TwrN231K PA46 134k 60'

DPA FARMM CtrAA755 D10 250k 90'

ELGIN MDW TwrN46ER C425 205k 60'

BOJAK VEARN CtrN627S L23 212k 100'

ORD TwrN82R C425 170k 60'

LAIRD PLANO CtrN38G C425 250k 100'

DPA OBK CtrJS) /\ >&\/ I NN763 D10 250k 90'1Z><.\ -X_ MDW DEERE Ctr

N3K PA46 175k 100'DEERE PLANO Ctr

/ \

ConMunicat ions ScopeNW763 heavy. Contact pent ex1. Good day.

NU763 >iea.vy : Swi tc l i ing to center* f*me<xuency .

flpp/Dep:

5010

Figure AL Static copy of TRACON screen. There are three major components to thedisplay. The right-hand side of the screen shows pending (not under control) and active(under control) flight strips. Each flight strip lists (a) plane identifier, (b) plane type, (c)requested speed, (d) requested altitude, (e) radar fix of sector entry, (f) radar fix ofsector exit (including tower or center). The lower part of the screen shows acommunications box that gives a printout of the current (and last few) commands issuedby the trainee and the responses from pilots or other controllers. The main part of thescreen shows a radar representation of the Chicago sector. Planes are represented by aplane icon and a data tag, which gives the identifier, the altitude, and an indication ofcurrent changes in altitude. The sector is bounded by the irregular dotted polygondescribing a perimeter. Radar fixes are shown as small (+) figures on the radar screen.Airports are shown with approach cones and a circle indicating the facility proper. Acontinuous radar sweep is shown (updating at 12 o'clock every 5 s). Range rings are alsodisplayed indicating 5-mile (8.0467-km) distances.


"communications box," which shows commandsissued to planes and pilots responses, along withthe controller's score for the current simulation.When planes are about to enter the trainee'ssector (at a boundary or on the runway of anairport), this information is announced over theheadset. No flight is allowed to cross the sectorboundary or take off from an airport withoutexplicit authorization by the trainee.

Task Controls and Knowledge of Results

Trainees interacted with the TRACON simula-tion in several ways. A trackball was used for themajority of input activities, although the keyboardwas also used alone or in conjunction with thetrackball. (The trackball represents a change ininput device from our study where a mouse wasused). For each plane command, a menu of com-mand choices is displayed on the screen.

Knowledge of results is provided visually by textin the communications box and aurally with aread-back by the pilot or other controller usingdigitized speech. In addition, planes follow asnearly as possible the commands issued by thetrainee. Commands such as turn, change altitude,and change speed are processed by the computerand carried out in accordance with the limitationsimposed by each aircraft type.

When errors occur (e.g., separation conflicts,near misses, crashes, missed approaches, or hand-off errors), additional information is presented tothe trainee. In each of these cases, an alert circlearound the plane or planes in question is pre-sented on the screen, and a series of tones arepresented over the headset.

Trial Description

Trials for the task have been created and pre-tested to be roughly equivalent in difficulty. Eachtrial contained planes that were divided into threebasic categories (overflights, departures, and arriv-als). Overflights are planes that enter and exit thetrainee's airspace at cruising altitudes. Traineesare required to acknowledge these airplanes asthey approach a boundary VOR fix, monitorprogress through the sector, and handoff to a"center" controller. Departures are planes thatoriginate at one of the four airports, climb to acruising altitude, and are handed off to a center

controller. Trainees are required to release depar-tures from airports, evaluate and remediate poten-tial conflicts as the planes climbed to a cruisingaltitude and turn to intercept their intended flightpaths, and then handoff planes to the appropriatecenter controller. Arrivals enter the trainee's air-space from one of the boundary VOR fixes and areto be landed at a designated airport. Trainees arerequired to direct arrivals onto an appropriateheading and altitude to provide an acceptablehandoff to the appropriate tower controller beforethese planes can land. For all flights, the trainee isrequired to maintain legal separation, that is atleast 1,000 ft. (304.8 m) in altitude or 3 miles (5.56km) horizontally.

Each trial lasted 30 min and consisted of 16overflights and departures (with roughly equalfrequency) and 12 arrivals. The planes requestentry to the airspace at irregular intervals con-strained so as to require the trainee to be continu-ously occupied with at least one active target. Thetrials were designed so that perfect performance(handling all 28 planes successfully) was just be-yond the skill level achieved by subject matterexperts.

A successful "handle" of a flight is the appropri-ate accomplishment of the respective flight plan.That is, for a departure or an overflight, theaccomplishment is a successful handoff to theappropriate center controller. For a landing, theaccomplishment is the successful landing of theairplane.

Performance Measurement

As with a previous investigation (Ackerman,1992), overall performance was computed as thesum of all flights accepted into the sector that hada final disposition within the simulation time minusany planes that were incorrectly disposed of (e.g.,crashes, not-handed-off, or vectored off the radarscreen). This measure is concordant with measuresderived from the examination of the criterionspace for FAA air traffic controller simulationresearch (e.g., see Buckley, Debaryshe, Hitchner,& Kohn, 1983) and has acceptable reliability.

Received March 1,1995Revision received May 22,1995

Accepted May 25,1995 •

Documents

Cognitive and Noncognitive Determinants and …newinbre.hpcf.upr.edu/wp-content/uploads/2017/02/...Christal (1989) specifies four sources of individual differences in the acquisition