A0801 ARIZONA STATE UNIV TEMPE DEPT OF INDUSTRIAL AND MAMA--ETC:F6ID~EVELOPMENT OF AN EFFECTIVENESS PLANNING AND EVALUATION MODEL F-FT*C 3(U)APR 60 H H YOUNG AFOSR79-0111

UNCLASSIFIED ASUMC-R-6001 AFOSM-TR-60-0996WII- 3

scrr4LSjrkTION OF THIS PAE(hnDaaEtrd

3. PRI ORT DOCUMENTATION PAGE E RMENT MET. TOS/ GOVT ACCASSION NO. 3. RECIAIENT'S CATALOG NUMBER

O 4. COTROnG FFICE NAM AND Aso"ES

Air~~~~:_ Forc Offic of Scietifi Reerh (t-I j A

Developent of anfC~ e .tve#esPlanning 203 3I2 WAieS

.Evaluation Model for Air Force Maintenanc e S y0rganizations . .. .. .. .................

J/4TA,5AUMBERR-a)7. AUTHOR(*) .. m-, n cl sij N U B Rm

liC ewitt H.1 Young"- -

" I I AFOSR- II I /A N

0 I. PERFORMING ORGANIZATION NAME AND ADDRESS TO. PROGRAM ELEMENT. PROJECT, TASK

1 Department of Industrial'& Management Systems , AREA & WORK UNI PdT NUMBR

Engineering I/ 611Wg (/7 ',-)

Arizona State University, Tempe Arizona 8528en Evalat0 I1. CONTROLLING OFFICE NAME AND ADDRESS // , ll J"l

Air Force Otfice of Scientific Research (MuLipl R pr Ansior BoPling AFB, DC 20332 Ten in PACSu197

14. I To mAI am& ADDRES, If differnmt from Control~he Office) IS. SECURITY CLASS. (of this report)

I /// / Unclassified

15a. DEC/ ASSI F1ICATION/ DOWN GRADING

V-1 SCHEDULE

10. DISTRIBUTION STATEMENT (of this, n4e.. l I b blocnum

Approved for public release; distribution unlimited.

17 aITIBTO straTified sapeof mheaisrc ntee n &a c 0tian d ersiftersupervisorsort

IS. SUPPLEMENTARY N40TES J

T71 - Tr 7 , Frk -T

1. K E Y O R S (C on tinue o h ,, es. .,7 " ,ildIF n d ,idn tify b y M 4a ? r? ft c e P ro d u c t i v it y JMaintenance Technician Performance Maintenance Evaluation

a antenance Squadron Performance Multiple Regression AnalysisWork Performance Modeling Maintenance Technician SurveyAF Squadron Effectiveness Prediction Performance Effectiveness

Performance Evaluation Maintenance Life-Cycle Costng> W 20. A RACT (ContinC

R on rC

Seree sideIif necesaICry ad identify by block number)-A preliminary effort was made to generate a survey-supported model which

( O would (1) permit periodic evaluation of the performance effectiveness of an

. Air Force maintenance squadron and (2) highlight equipment and human resourcefactors which are contributing either positively or negatively to maintenance

uLJ squadron performance. The model is generated from survey data, collected from-J a stratified sample of maintenance technicians and their shift supervisors, ,

L. and processed by means of a stepwise, linear multiple regression statisticalpackage to provide a performnce prediction equation. Factors which surface

DD ,JAN "7 43 _ j nclassified~SECURITY CLASSIFICATION OF THIS PAGE (Wthen Date Entered)

%* Unclassified-SECURITr CLASSIFICATION OF TMIS PAGS(Wh n Data Ettetnd)

as significant in the equation indicate positive and negative contributions tosquadron performance effectiveness. The modeling effort is based on studieswith the 82nd Air Training Command Wing at Williams AFB and the 405th TacticalAir Command Wing at Luke AFB, both in Arizona. The model was validated usingimmediate supervisor ratings of maintenance technician performance in speed andquality of work, averaged across a squadron. Based on the analyses and resultsof studies covering two maintenance squadrons at Williams AFB and three main-tenance squadrons at Luke AFB, the model provides excellent predictions ofsquadron performance effectiveness and highlights significant contributingfactors..

It is recommended that the study be extended to other Air Force bases andsquadrons, with the end goal of achieving one or more survey-supported modelswhich could periodically be used to evaluate squadron performance and targetareas for improvement. The model may also be useful in evaluating individualtechnician performance within a squadron and in developing life-cycle costingfor maintenance activity on particular end-item equipments maintained byparticular Air Force Wings and squadrons.

3

Acce-.,ion -For

NT1': G:& iiD"^ TAB .

Unainnfuu tc , d

jus:t if ic. t i,)n.

By . .-----

Uncl as s ified 0||SCURITY CLASIFICATION OF

r THIS PAGC(VbM Doem Zheer*Q

+ l i l i l I l i l + 2

DISCLAIMER NOTICE

THIS DOCUMENT IS BEST QUALITYPRACTICABLE. THE COPY FURNISHEDTO DTIC CONTAINED A SIGNIFICANTNUMBER OF PAGES WHICH DO NOTREPRODUCE LEGIBLY.

oS.-TR- 8 0-0 598

DEVELOPMENT OF AN EFFECTIVENESS PLANNING

AND EVALUATION MODEL FOR AIR FORCE

MAINTENANCE ORGANIZATIONS

by

Hewitt H. Young, PhD, PEDepartment of Industrial andManagement Systems Engineering

Arizona State UniversityTempe, Arizona 85281

for

U.S. Air Force Office of Scientific Research

Life Sciences DirectorateBoiling AFB, DC 20332

AFOSR Grant 79-0111Dr. Alfred N. Fregly, Program Manager

FINAL SCIENTIFIC REPORT

April 1, 1980

UNCLASSIFIED

ASU No. ERC-R-80016

Approve f o r publle release;

distribution unlimited.

,° A..

TABLE OF CONTENTS

Page

LIST OF FIGURES .......... ............................ v

LIST OF TABLES ............ ............................ vi

GLOSSARY OF TERMS ......... ........................... viii

STATEMENT OF WORK ...... ........................... .l... 1

Objectives and Scope of Research ...... ................. 5

RESEARCH EFFORTS AND RESULTS ..... ..................... . . 10

Development of the Survey Instruments ... ............... ... 14

Selection of the Performance Measures and Measurement System 14Generating the Three Survey Instruments ..... ............ 19

Part I. Group Morale ...... ................... 22Part II. Organization Climate ..... .............. 22Part III. Occupational Attitude ..... ............. 23Part IV. Feelings While Working (Fatigue Trait) .. ..... 23Part V. Personal Traits ...... ................. 23

Overall Maintenance Technician Survey ..... ............. 25

Conduct of the Surveys ........ ...................... 25

Selection of the Sample Populations ..... .............. 25

Conduct of the Surveys ........ ...................... 27Processing of the Data ...... ...................... ... 28Results of the Study ...... ....................... ... 31

Williams Air Force Base ....... .................... 31FMS Statistical Results ....... .................... 31OMS Results ....... ........................... 37OMS Statistical Results ....... .................... 37Williams AFB Overall Results ...... ................. 38Luke Air Force Base ........ ...................... 44CRS Statistical Results ....... .................... 44EMS Results ....... .......................... .. 49EMS Statistical Results ....... .................... 49AGS Results ....... .......................... .. 52AGS Statistical Results .... .................... 55Luke Air Force Base Overall Results. .............. 60Overall Results Across 82nd ATC Wing at Williams AFB and

405th TAC Wing at Luke AFB ...... ................ 62

PLANNED PUBLICATIONS ......... ......................... 67

III

TABLE OF CONTENTS (continued)

Page

PROFESSIONAL PERSONNEL ASSOCIATED WITH RESEARCH ... ............ .... 67

INTERACTIONS WITH AIR FORCE ORGANIZATIONS .... ............... ... 68

CONCLUSIONS AND RECOMMENDATION .... .. .................... ... 69

REFERENCES .... .... .. .............................. ... 72

BIBLIOGRAPHY .... ... ..... ............................. 74

APPENDICES .... .... .. .............................. ... 79

A. Summary of Important Findings from Literature Reveiw Conductedat Wright-Patterson AFB in Summer 1978

B. Survey Instruments Used in Study

C. FORTRAN IV Computer Program for Processing Survey Data

D. Brief Summary on BMDP Statistical Programs

E. Example Output of BMDP2R Statistical Program for Williams AFB,Field Maintenance Squadron

F. FORTRAN IV Program Output for Williams AFB, OrganizationalMaintenance Squadron

G. FORTRAN IV Program Output for Luke AFB, Component RepairSquadron

H. FORTRAN IV Program Output for Luke AFB, Electrical MaintenanceSquadron

I. FORTRAN IV Program Output for Luke AFB, Aircraft GenerationSquadron

("SC)

aPprov, . ..,,

1faor1 ati Al Otrlee (7b).

iv

LIST OF FIGURES

Page

1. Illustrations of Conventional Rating Scale Formats for a Single

Item ......... .. ............................... 18

2. Rating Scale for Speed of Performance ..... .............. 20

3. Rating Scale for Accuracy of Performance ............... .... 21

4. BMDP2R Variable Identity ....... ..................... 30

II

LIST OF TABLES

Page

1. A Taxonomy of Maintenance Manpower Effectiveness ..... .............. 2

2. Summary Statistics from Study of Maintenance Job Performance on Two Auto-pilot Systems ..... ... .. ............................... 4

3. Comparison of Performance Time Curves as Estimated by 13 Human ResourcesRaters and as Generated by the SAINT Computer Simulation .... .......... 6

4. Statistical Results on Factors Affecting Maintenance Performance Speed Basedon Data Collected Across 140 SRAM Technician's AFSC 462x0 and 446x0 . . . 7

5. Statistical Results on Factors Affecting Maintenance Performance AccuracyBased on Data Collected Across 140 SRAM Technicians ... ............... 8

6. Prioritized Predictive Factors for Time to Perform Five Selected MSET TasksBased on Regression Equations ......... ........................ 9

7. Williams AFB, FM Squadron Data Showing Statistical Distribution Parameters 32

8. BMDP2R Summary for Williams AFB, FM Squadron with Performance Speed RatingA as the Dependent Variable ..... .. .. ........................ 33

9. BMDP2R Summary for Williams AFB, FM Squadron with Performance Quality RatingB as the Dependent Variable ...... ....................... ... 35

10. BMDP2R Summary for Williams AFB, FM Squadron with Transformed VariableZZ = O.4A + 0.6B as the Dependent Variable ..... ............... ... 36

11. Williams AFB, OM Squadron Data Sharing Statistical Distribution Parameters 39

12. BMDP2R Summary for Williams AFB, OM Squadron with Performance Speed RatingA as the Dependent Variable ...... ........................ . 40

13. BMDP2R Summary for Williams AFB, OM Squadron with Performance Quality RatingB as the Dependent Variable ...... ........................ . 41

14. BMDP2R Summary for Williams AFB, OM Squadron with Transformed VariableZZ = 0.4A + O.6B as the Dependent Variable .... ................. .. 42

15. Conarison of Multiple Regression Equations for Williams AFB OrganizationalMaintenance and Field Maintenance Squadrons (compared at R2 level of 78percent) ...... .. ... .................................. 43

16. Luke AFB, 405th CR Squadron Data Showing Statistical Distribution Properties 45

17. BMDP2R Summary for Luke AFB, 405th CR Squadron with Performance SpeedRating A as the Dependent Variable ....... ..................... 46

18. BMDP2R Summary for Luke AFB, 405th CR Squadron with Performance QualityRating B as the Dependent Variable ....... ..................... 47

19. BMDP2R Summary for Luke AFB, 405th CR Squadron with Transformed VariableZZ - 0.4A + O.6B as the Dependent Variable ..... ................. 48

vi

4% -. m I I II

Page

20. Luke AFB, 40 5th EM Squadron Data Showing Statistical Distribution Properties. 50

21. BMDP2R Summary for Luke AFB, 40 5th EM Squadron with Performance SpeedRating A as the Dependent Variable ..... ..................... ... 51

22. BMDP2R Summary for Luke AFB, 405 th EM Squadron with Performance QualityRating B as the Dependent Variable ..... ..................... ... 53

23. BMDP2R Summary for Luke AFB, 40 5th EM Squadron with Transformed Variable

ZZ = 0.4A + 0.6B as the Dependent Variable ...... ................. 54

24. Luke AFB, 405th AG Squadron Data Showing Statistical Distribution Properties. 56

25. BMDP2R Summary for Luke AFB, 405th AG Squadron with Performance Speed RatingA as the Dependent Variable ...... .. ......................... 57

26. BMDP2R Summary for Luke AFB, 405th AG Squadron with Performance QualityRating B as the Dependent Variable ..... .. ..................... 58

27. BMDP2R Summary for Luke AFB, 405 th AG Squadron with Transformed VariableZZ = 0.4A + 0.6B as the Dependent Variable ...... ................. 59

28. Comparison of Multiple Regression Equations for Luke AFB, 40 5th TAC Wing,Component Repair, Electrical Maintenance and Aircraft Generation Squadronsat Approximately 77% Accountability (R' = 0.77) ..... ............... 61

29. Comparison of Multiple Regression Equations for Luke AFB, 40 5th TAC Wing forComponent Repair and Aircraft Generation Squadrons at Approximately 90%Accountability (R2 = 0.90) ...... ... ......................... 63

30. Comparison of Significant Variables for Predicting Performance Speed (A)vs. Predicting Performance Quality (B) ....... ................... 66

vii

9. -- | I

GLOSSARY OF TERMS

ASCENDANCY TRAIT - Assertive in relationship with others; desire to assume anactive role.

AFSC - Air Force Specialty Code

ASSIGNMENT LOCALITY - Location, climate, community atmosphere.

EMOTIONAL STABILITY TRAIT - Well balanced and relatively free from anxieties andnervous tensions.

FATIGUE TRAIT - Subjectiveness to feelings of weariness.

GROUP HOMOGENEITY OF ATTITUDE - Common purpose and goals.

JOB CURIOSITY TRAIT - Interest in discovering and learning all facets of a job.

MSET - Maintenance Standardization and Evaluation Team; MSET tasks are thosesubject to Command Team evaluation.

ORGANIZATION CLIMATE/IDENTITY - Atmosphere which creates a feeling of belongingas a valuable member of a working team.

ORGANIZATIONAL CLIMATE/REWARD - Atmosphere which creates a feeling of beingrewarded for jobs well done.

ORGANIZATIONAL CLIMATE/RISK - Atmosphere which creates a feeling of riskinessor uncertainty about job and/or organization,including job safety hazards.

ORGANIZATIONAL CLIMATE/STANDARDS - Atmosphere which creates a feeling for theperceived importance of implicit and explicitgoals and performance standards.

ORGANIZATIONAL CLIMATE/STRUCTURE - Atmosphere which creates a feeling of groupconstraints via rules, regulations, redtape.

ORGANIZATIONAL CLIMATE/WARMTH - Atmosphere which creates a feeling of generalgood fellowship.

ORGANIZATIONAL IDENTIFICATION - Feelings of association with and support fromthe organization.

PAY AND BENEFITS SATISFACTION - Feelings of technician about these extrinsicrewards.

PERSISTENCE TRAIT - Sees work through to completion.

PROFESSIONAL IDENTIFICATION - Good feelings about job speciality as importantand necessitating special skills.

RANK - Position level in AF organizational structure.

viii

RESPONSIBILITY TRAIT - Positive and perservering in using own judgements;determined; reliable.

SAINT - Systems Analysis of Integrated Networks of Tasks; a computerizedsimulation methodology.

SOCIAL STATUS - Feelings about the importance of occupation within society; aperceived level of society acceptance and status.

SRAM - Short Range Attack Missile

ix

STATEMENT OF WORK

An important current objective of the United States Air Force is to improvethe effectiveness of weapon system maintenance and maintainability. To meetthis objective, the USAF is expending considerable resources in determining thecontent of maintenance tasks, in determining the impact of equipment designdecisions upon these tasks, in streamlining technician selection and trainingprocedures, and in providing well-documented technical information on AF airand ground equipment.

This research considers the design of an effectiveness planning andevaluation model for use by Air Force maintenance organizations at the wing andsquadron levels. The model is intended for use in (1) planning maintenanceactivity requirements on selected end-item equipments, as part of life-cyclestaffing and costing, and (2) evaluating the performance of a maintenanceorganization at a point of time and highlighting areas for improvement.

During the summer period 1978, initial phases of this research were con-ducted by the author while participating in a USAF/ASEE Summer Faculty Fellowshipat the Air Force Human Resources Laboratory, Advanced Systems Division, Wright-Patterson AFB, Ohio. The summer work included an extensive literature review ofprevious studies and proposals concerning the selection, training and perfor-mance of Air Force maintenance personnel. Factors uncovered in the literaturesearch were then developed into A Taxonomy of Air Force Maintenance ManpowerEffectiveness as shown in Table 1. It may be noted that the taxonomy is dividedinto four major categories: (1) Equipment Reliability/Maintainability factors,(2) Maintenance Equipment and Technical Information factors, (3) TechnicianExperience, Skill and Knowledge factors, and (4) Technician Productivity andMorale factors. The subfactors listed in the Taxonomy have been shown to havean impact on either (1) the performance of maintenance technicians (primarilyAF personnel) or (2) the morale and job satisfaction of maintenance technicians,in one or more research investigations. Reference is made to a report preparedfor AFHRL/ASD at the end of the summer fellowship by the author (Young, 1978).References included in the Summer 1978 Design Report are also included in theBibliography of this Final Scientific Report.

Three studies uncovered in the literature search were of particular interestto this author and suggested a possible framework for modeling the performanceof an AF maintenance organization. The first research study was an attemptto ascertain what equipment design and operating variables affect AF main-tenance performance, using survey and statistical techniques (Meister, Finleyand Thompson, 1971). The study covered organizational and intermediateavionics maintenance of two autopilot subsystems in the B-52 and KC-135 aircraft.Observations were made involving SAC personnel at March AFB and Wright-PattersonAFB and covered performance measures of time to perform maintenance, whethertechnical orders (manuals) were used or not used, the number of unique diagno-stic checks made (in trouble shooting), the number of repeated diagnosticchecks made, the number of components removed or replaced, the technician'srating of task difficulty, the number of times assistance was required by thetechnician, the number of components actually worked on, the maintenancediagnostic strategy used, and the observer's ratings of the technician relativeto understanding the problem and efficiency of work. A total of 14 subjectswere included in the study. Table 2 summarizes results from the study forperformance time production. The values stored surfaced as significant Betacoefficients (predictors of time to perform maintenance) in the multiple

II 1

#A

41Ur

04

4-) 0u 41 4-)o 4J 4- 4

> C7 4-(

Cl to 4) ) 0 4-1 )i -ai 4-) -S. 04- U- 4J 4J w0- C, 4i 0, 4.)

.0 S4- w -4U :P- 4U

- 4- UJ 4. 4j (j +4- (n- L- 0 3CL 4 4-~ ) U0 - UU) 50 c , 4- -ca m

03 a, ~ t 4-) ) UCaI 4to W 4- 40-a a,C ~ ~ ~ ~ ~ ~ V a . - C 4- - 0 U -c3 0 03 cu 0 0 0"a -0(3) ". L r, - , S 4..c

C. C O 41 w > 0 fa a)ex - = = 4- CM 0a W *- 4 0 0 0Ua,~ U 4-4- , -Uw >- CL CL a, m w w494 a4-~t 4) LaJ 0 0 a,-, 04- : a, 0 - 0 o aQ )C 0 4 0 E 0rUO -. ~ 0 0~

~ 0 4 - o~4-)u 4-) IA4- CA0. S -) 4- 4'

u, a4-)

<.0 41 4-j0 4-i

4-0

4- 0-

a9 CL C. $U4-) ) w U) - J 4J 4

%A V 4otuto)c 060 C 4, LLa. 1 0 ; M -. 0 U CL >41 4-) a, 3

(A 4-' 4-- S-)

4O M, .- 0w03 4..) V 441 U) 0 W C - . (C w) 4 - 0 . 4- . t

s- cu w. i w3~ 04- o) a, Cu (A4- 4 0. 4

41 c-. U >-U 0

3 3c i z 00 Ccm -

.41

'U4.3

41-

a 4 .1 4..

0~. *. V~n ~ -4.1 43 n4 * nv i 41 a>

UL S 41 to.e' 04L-) 41 ~ -U.0 >~0- (nz U S-=U..

4Ja- E~ 4)w 4040

0 (U (a ( "D41~ > goL. 10 0 L. 0)t W 4taL. 4

0 0a' > E CLW *NV4- .>UIA 4-1 +j .. w9. . Af09

1000 > 0u1- M- 4.U U 43-0 4.cm S- cO CL 4- 4U-) . ( 43) Q).JCU to1.0 -S- 04. to W 0a4 m

CAr- 4-) .Qo d4- Q 0 C. - L. C rW4- C9 aJ - 0 4- 4 R 41i .- S- M '4J (L) =(D > 0 t 0 0 c 4.14- (a 4J 'U4-'CL

S. .- 6 4-J 4.1 :) 5v " CUb4)4-1 ,- .14-( C V) C 4-J 04 c 4-b 43. 1

CL 0 -C VI. 0go mn 4) V c LUI ."~ S-= CJ Vi 4-) W L.-- o) 4 an- 0C4- 3C

4.1 V)r 4J 4- -4) (A 4jU 4- to r.U- 0-Q 0 0- 063c .41 1. -9 .U 4341 &- ' 4-Uv m 41 >I >4J CO 4-JE fG 4-0a cU(AU)to- ~ 0 0 C E4-0-> to 41 4j-r- C - - 0="4- 0 p MC U0- . CU C~4Afl ' to (a -4 0LUJ S- *9- > 1-..v- l (A. 'U S-.- a 4J .> 4J(a 0

toU + QJ U0 W4-3 4J W 00r-44-)0 0a .0 w 4j --to QU 0nL.U. to Q-3 CU 0v toN3 U) N = 0) to0 EU L.l4J Nf r- W N

4J 4.>J 4-U v = - f 41

L$ 4 1 10-'1E4 0 4o~ -o oM 0 >1 S_I-i a 0436CC (n

443

4j C h

'A (A 10 &4m 41 41

00

4-0 (D U-

Q 0 06

O 1.

1. V 43d

0L W -(A C 0 -

Cn 0$ '- 14-c 1. 41 043

V (A 43 U 1

~41060

x U toR 1. a 0 c0 4 4 to ti4.a (A LA. 0. U.0.4

0) 9 D9% cir 0 r - NO 000

-L& cc -~ . . L -. * * *4.- ma i I I I l i4- Al~

a U-

S- s-

0W

to 0 aD a 1 r .

aE a.'r -

WE ea U ON

CL c 4- <~ 4=4C SC o-0 '4- Wa.QOW4) 0- -)r CC0 > (D) 4-W-L aCL U~ r- a, u.'+UL-. 0> , 4j U) 0

(U - - =) Ln .- ul~ (a L w ) a C .c, .- CA a,- 0aUZ U .. a V)'CccUL

r X-~ 4) V) o m

'4-a * .0 )*~~4- 1-) 5/ d) LiLfj ..0j LW *

o9 .V- V) V) .) O

0- 4 .0 o - a 0 -4-(.) .U U) 0en

V) r 'U U)O S-Ln)

x 4) a CD

U4- - CV) I-W U) a- CD U r.

- 0 *- r.f4J 4.- 4) 09i 4) co CDUU)

&A I- U- a. 0 .4 ) 'C) . ) ) U41E.. :0.a *j c, -.a )/ U)0 CD

V)4 4a w- 4-UJ ~ a . to~ V) a,0 - J 0 *0 w-1 -> 0~ > 0 U) LO>, f -a~' 64-.-,-)' 4 .- r 4 *-nu wU)t

..-. - 0 c 19 w 4 toL *- Uow* r_ &- >,.o i.L Q'.-( *-0)-- 0- cmI= E S- Fc>- oC w -0

41r3 0 R>L.UU . *aU 0L a. 0 (U 4)E- 4) CL4 a'~ ir-- V) s M a4- 4) uJ au aC0 D 1

V)U (U cW ..- - 3.9-I s- .- s- u a!. m >> . L U) u~ 3 =' w o- * c w *' -U4- >

0 L. 4,v- ) C 4U@J >4C4 4)Z4) S-))4 .to ~ a, > -

a. o c w) &n &n 4) %n a a) *j C- U. CD .041 CU a -r- FL IO I I 11)/ wU.0 -

.- UJ4J -r--L CL a .__IA_ __-.4__ __ ___ __ __ a, ~ a

%- -l c ----- L. . . n V)-10 U. -

regression analyses. The Meister et al. study did not include motivationalfactors, nor was accuracy of performance used as a criterion measure.

The second important study was aimed at adapting a computerized networksimulation model known as SAINT (Systems Analysis of Integrated Networks ofTasks) to the maintenance task performance network on SRAM missile handlingtasks (Askren, Campbell, Seifert, Hall, Johnson and Sulzen, 1976). Twoperformance measures generated by SAINT were then compared with human resourcesspecialists. Three handling tasks in organizational maintenance of the ShortRange Attack Missile (SRAM) were included in the study and involved 120 main-tenance technicians. Dependent variables in the multiple regression analysesincluded performance time estimates and performance accuracy (or hazards)estimates. Five independent variables were measured from survey questions ordata covering (1) system proficiency (as measured by months in SRAM), (2)quality of written materials, (3) environment-temperature, (4) number of fatiguesymptoms, and (5) work motivation. Table 3 shows some results from the study.A further analysis of the results was made by the author in Summer 1978 toshow the percentile contribution of each of the five independent factors towardsprediction of performance time on Task 29, with the following results: (1)written materials - 35.4%, (2) work motivation - 25.5%, (3) environment - 17.2%,(4) proficiency - 16.8%, and (5) fatigue symptoms - 5.5%.

The third and perhaps most important study was directed towards maintenanceairman characteristics and motivational factors which contribute to the speedand accuracy of maintenance task performance (Sauer, Campbell, Potter andAskren, 1977). The survey-type study covered 140 missile mechanics andtechnicians performing organizational maintenance on 19 SRAM MSET tasks. Theseare tasks subject to maintenance standards evaluation. An additional 90 techni-cians working on the AIR-2A Genie System and/or Minuteman MKIII reentry vehiclewere included in the opinion survey. Data was collected at five AF bases.The dependent variables to be predicted were (1) supervisor ranking of techni-cian overall speed of task performance, (2) supervisor ranking of technicianoverall accuracy of task performance, (3) time to complete task as entered onMSET evaluation report (the latter data was too sketchy to be useful). Resultsof the study are summarized in Tables 4 and 5. In Summer 1978, the author furthersummarized the Sauer et al. results to provide the information shown in Table 6.The top portion of the table shows the weighted priority of the predictionvariables relative to time to perform task.

Based on the Summer 1978 review, it was concluded by the author that thesingle most important "equipment/information" predictor of performance rate(or conversely time to perform) is the adequacy and completeness of thetechnical information provided to the technician and that the "human resources"predictors of most importance are (1) technician motivation, (2) organizationclimate and group morale, (3) months in career field, and (4) assignment locality.For accuracy of performance, the human resource predictors which appear to bemost important are (1) technician motivation, (2) assignment locality, (3)months in career field, and (4) pay and benefits satisfaction. Other importantfindings from the literature review are summarized in Appendix A.

Objectives and Scope of Research

This research is aimed primarily at the human resources factors whichaffect job performance. However, technician supervisors are asked to providea limited amount of information about equipment and technical information factors

5

t|

Table 3

Comparison of Performance Time Curvesas Estimated by 13 Human Resources Raters and

as Generated by SAINT Computer Simulation(Compiled from Askren, Campbell, Seifert, Hall,

Johnson, and Sulzen, 1976)

A. Performance Times vs Number of Fatigue Symptoms for SRAM Missile Handling Task 27(Load Launcher to Aircraft)

Number of Mean Subjective Time Estimates Computer Generated TimesFatigue Symptoms of 13 Raters (Minutes) Using SAINT Simulation

0 54.6 59.95 Baseline 63.6 61.1

10 72.6 62.715 81.620 90.6 65.725 99.630 108.6 68.8

B. Performance Times vs Environment (Temperature) on Three Tasks on SRAM Missile

Task 2 ( ransport)l Task 27 (Assemble)2 Task 29 (Checkout)3

TemperatureConditions Raters Simulation Raters Simulation Raters Simulation

-40OF 22.6 15.7 114.0 70.7 483.6 271.3

0 18.3 15.0 87.6 66.0 358.8 265.5

400 14.8 14.3 66.0 62.0 275.6 261.4

600 Baseline 14.2 14.2 61.8 61.1 260.0 260.7

90 15.1 14.4 68.4 62.2 291.2 261.9

1200 19.3 15.2 96.0 67.1 397.8 266.6

1. Task 2 - Transport Payload to IMF, 463x0 crew, Mean Time 140 Minutes

2. Task 27 - Load Launcher to Aircraft, 462x0 Crew, Mean Time 60 linutes

3. Task 29 - Perform Aircraft System Checkout, 316x0 Crew, Mean Time 260Minutes

6

Y'

Table 4

Statistical Results on Factors Affecting MaintenancePerformance Speed Based on Data Collection Across

140 SRAMTechnicians AFSC 462xO and 463x0(Compiled from Sauer, Campbell, Potter, and Askren, 1977)

Significant High Performance Variable Correlation Coefficientsand Multiple Regr. Coefficients Correl. R's > 0.25462xO(B33.1879) 463xO(B70.8817) 462x0 463x0

Years of Service .2168 .28 (.17)Months in Career Field .25 .29No. of Individual Sports .28No. of Service Clubs .2132 .25No. of Interest Clubs .1702 .3109 .26 (-.Ol)Trait Anxiety Level -.36 -.32Gordon Personal Pro.'ile -2

Sociability trait -.27Emotional stability .30Ascendency trait .2218 .28

Fatigue Symptoms-Trait -.3193 -.41Fatigue Symptoms-State -.29 -.26Occupational Opinion

AF policy/practices .43Assignment locality .1933 .35 .43Social status .36

Organizational ClimateStructure .3272 (.09)Risk -.2241 (-.11)Warmth .27Conflicts -.3353 (.07)

Group MoraleSatis. indiv. motives .34 .29Homogeneity of att. .2456 .30 .27Satis. interpersonal rel .34Satis. with leader .33

LBDQPersuasiveness .28Consideration -.4230 (.15)

MotivationJob curiosity trait HP(.Ol) HP(.Ol) .37 .53Persistence trait HP(.Ol) .6173 HP(.Ol) .31 .61Prof. identification HP(.Ol) HP(.Ol) .29 .42Team attitude HP(.OI) .41Org. identification HP(.Ol) .32 .38Self-starter trait .43

7

Table 5

Statistical Results on Factors Affecting Maintenance PerformanceAccurac Based on Data Collected Across 140 SRAM Technicians

(Compiled from Sauer, Campbell, Potter, and Askren, 1977)

Significant High Performer Variables Correlation CoefficientsPredictor and Multiple Regr. Coefficients Correl. R's > .25

462xO(B14.3824) 463xO(B13.5316) 462x0 T63xO

Years of Service 34No. of Re-enlistments .26Months in Career Field .32No. of Interest Clubs .1991 .32No. of Indiv. Sports .1588 (.18)Gordon Personal Profile:

Responsibility trait .29Fatigue Symptoms-Trait HP(.0) -.40Fatigue Symptoms-State -.26LBDQ:

Representation -.36Tolerance of freedom .26

Occupational OpinionAF policy/practices HP(.0) .27Assignment locality ** .3540-HP(.01) .39Pay and benefits -.2338 (.18)Promotion opportunity ** HP(.Ol) .28

Organization ClimateResponsibility -.29Rewards -.3263 (.06)Warmth .29

Group MoraleSatis. indiv. motives * .3817 HP(.OOl) .41Homogeneity of attitude .32Satis. with leader -.3111

MotivationJob curiosity trait ** .8249-HP(.OI) ** HP(.Ol) .46 .57Persistence trait ** HP(.Ol) * .7897 HP(.Ol) .37 .71Prof. identification ** HP(.O1) HP(.OI) .32 .51Team attitude HP(.Ol) ** HP(.OI) .50Organ. identification ** HP(.Ol) ** HP(.Ol) .37 .45Self-starter trait **-.4368 HP(.Ol) ** HP(.Ol) .54

Significant at (p<.001) for high performers

* Significant at (p<.Ol) for high performers

8

4-) L. 4J 4-' a CJr-O. O )L%0 O C>r..Cto fl, . C . C4 U) 4t0 4

'UUL ). CU a. j(A. .t-...

4-G JS.-uc-- .0 4) D mrP-toqrk

'0 4- GDG(UD(VI WD

-- . *- %n .O~ IM 00 c'.i -l c c'J c 0>.- >m- MU ~CD 0C 0 J co D k coC C

-S-O0S.- L. m(L

GD '4J 4-0-~-

V) $- 2: ) (AU. . . .G -c - >1 4-) SLOG "a_ __ _ __ _ __

. IC Lz- n -V) 4- M M

L- 0 L61 *4-u'a4-) *'.'-' a Oo S- to I~ / .- U = 0 cu 4

S- (A ~ LrG U 4-)W 1.04

41u. S:. *. ( V V - 4) L-- IV__ __ _ W~ U) =)0 U-+. 4-''U

S - 00 4J L. ) 4 u - (

a- LJ 0 41 L. - to- 0 4J. V C-' ZD S-41 -a USU)A-GDU) C) 0) . I J 'U ) 4.J4.00 M .- S-(a C

o--0 .li u~). V) CL u .1.- M' 0U) C tu0 C = r V a r C4-

LL.' u U.C - GO S--' M 0I-I-o 0S- (AJ 4-E 4-.JU W) "0_ r-4_____)

'.0 -1 j-)~ W-a~ = C_ -- ,S)0- CL - 4-) (A 4- 0 -0 4

0D m -GD n M4-- )ea 4J)' (U-- 0 $- S- m- L.) 0C-" -IV

VD) 04- 4' 0U .C 10C

4-C 'O S- .- o MO

to to U U

0L. 0)2. .$ L-.4- 0 4 ) --M>- 4

CA Q CL VI. . -O LL- ') 4/ .- " e'j"CN OW( ~ c nC

4-) M 0.-.C' U)U W -W* 0t U) l ,Vd) 0 m- 4... . . .C. .

5-J C . -~ t)~-.~ 0 "

C- M -f-= .

L. 4)t 3 GD C4. 4- c w

U) 0 ) U ) u' U

oL LU. -D-4-. 0 '4. . -U ,4J' 01"'.' 4- qw. 4l> C1 LOU ON W.0'0e U0mCD

Li:I V)D'' U GD-1

-1

4- W RrMM0 WP

.U QC~ ' C V) 4--..

,t . ( u 0) X~U ' GD 'L 0204-)4J5 - 'U CO I. Ij 5-5- -5- (A .U)- 4 #GD Q* = - .1 04- = 9.0 4-0I .

W- W S- W 4) S - I

to do 4

0.9D XV- C6C AL.(

which may also be expected to impact on performance. In total, forty-eightpredictor variables were eventually included in the performance model. Thetwo dependent variables selected were technician performance rate andtechnician performance accuracy, as judged by a first line or immediatesupervisor.

The specific objectives and scope of the research are

(1) to develop one or more survey instruments, using existing and pre-validated questions when possible, for gathering data from AFmaintenance technicians and their supervisors that might be pre-dictive of technician performance (herein defined as some combinationof speed and accuracy of work),

(2) to develop a rating instrument by which first line (immediate) super-visors could compare technicians performance and establish performanceratings,

(3) to secure Air Force clearance for conduct of extensive surveys ofmaintenance personnel at Williams AFB and Luke AFB, Arizona, using thedeveloped survey and rating instruments,

(4) to conduct the surveys at the two AF bases,

(5) to summarize the survey data, including supervisory ratings, from eachof the two bases and prepare the data for statistical analyses,

(6) to subject the data from each base to extensive statistical treatmentas a means of generating useful predictive models of maintenancesquadron technician performance,

(7) to examine similarities and differences between the predictive modelsfor differing AF maintenance squadrons, and

(8) to develop conclusions, make recommendations for future work, andwrite and publish the results of the study.

The long-run objectives of the research, of which this grant and researchare a first step, is to standardize on a set of survey instruments and ratingscales which (1) would permit an AF Wing or squadron to periodically examineits performance effectiveness and (2) would permit the U.S. Air Force to moreaccurately predict maintenance requirements on selected end-item equipmentsfor life-cycle staffing and costing.

RESEARCH EFFORTS AND RESULTS

At the proposal stage of this research, a performance effectiveness modelwas postulated with the following dependent and independent variables:

(1) Average technician performance rate (or speed of work) across asquadron, a dependent variable to be predicted. For purposes ofexperimentation, supervisor ratings of technician performances wouldbe used.

i'10

(2) Average technician performance accuracy (or quality of work) acrossa squadron, a dependent variable to be predicted. For purposes ofexperimentation, supervisor ratings of technician performances wouldbe used.

(3) Average weight of subsystems or components handled by technicianswithin the squadron, a predictor variable averaged from supervisorestimates for each supervisory work group (normally a shift work group).

(4) Average difficulty of servicing assigned subsystems, across thesquadron, based on accessiblity or clearance for such tasks. A pre-dictor variable based on supervisor estimates for each work group.

(5) Average difficulty of removing and/or replacing subsystems (or compo-nents), across the squadron, based on the accessibility or clearancefor such tasks. A predictor variable based on supervisor estimatesfor each work group.

(6) Average number of internal components in subsystems serviced, acrossthe squadron, based on supervisor estimates for each work group.

(7) Operating hours between services for equipment subsystems, across thesquadron, based on supervisor estimates for each work group.

(8) Adequacy of technical information available to technicians, across thesquadron, based on supervisor estimates for each work group.

(9) Usability of the available test equipments, across the squadron, basedon supervisor estimates for each work group.

(10) Average knowledge of technicians for performing servicing requirements,across the squadron, based on supervisor estimates for each work group.

(11) Average skills of technicians in performing servicing requirements,across the squadron, based on supervisor estimates for each work group.

(12) Average (mean) shift assignment, across the squadron, based ontechnician inputs.

(13) Mean year of enlistments into the Air Force, across the squadron,based on technician inputs.

(14) Mean skill level of technicians, across the squadron, based ontechnician inputs [3,5,7,9 levels].

(15) Mean year of assignment to current duty status, across the squadron,based on technician inputs.

(16) Mean year of assignment to current equipment type, across the squadron,based on technician inputs.

(17) Average number of months on current equipment assignment, based ontechnician inputs.

(18) Mean of sex distribution (where 0 = female and 1 = male), across thesquadron, based on technician inputs.

11j

q -

(19) Average complexity level of aircraft or missile equipment (based onanalysis and using an interval scale), across the squadron, basedon technician inputs.

(20) Average military rank (or civilian grade), across the squadron, basedon technician inputs [ = military and 2 = civilian].

(21) Average number of sequential work days between days-off-periods (weekendor other), across the squadron, based on technician inputs.

(22) Average number of additional duties assigned to maintenance technicians,across the squadron, based on technician inputs.

(23) Average number of hours worked per work shift, across the squadron,based on technician inputs.

(24) Average hours per week spent on additonal assigned duties, acrcss thesquadron, based on technician inputs.

(25) Average number of clubs in which technicians particpate, across thesquadron, based on technician inputs.

(26) Average number of persons supervised by technicians (including 0),across the squadron, based on technician inputs.

(27) Average number of months technicians have served as a supervisor, acrossthe squadron, based on technician inputs.

(28) Average technician satisfaction with squadron and work group in meetingindividual motives of technicians, across the squadron, based onresponses to series of questions.

(29) Average technician satisfaction with supervision received, across thesquadron, based on responses to a series of questions.

(30) Average technician expressions concerning the homogeneity of attitudewithin the squadron and work group, across the squadron, based onresponses to a series of questions.

(31) Average technician satisfaction with interpersonal relationships amongpeers within the squadron and work group, across the squadron,based on responses to a series of questions.

(32) Average technician expressions concerning the organizational structureof the squadron and work group, across the squadron, based onresponses to a series of questions.

(33) Average technician expressions concerning the organization warmth ofthe squadron and work group, across the squadron, based on responsesto a series of questions.

(34) Average technician expressions concerning the organizational rewardsof the squadron and work group, across the squadron, based onresponses to a series of questions.

12

(35) Average technician expressions concerning organizational riskswithin the squadron and work group, across the squadron, based onresponses to a series of questions.

(36) Average technician expressions concerning organizational conflictswithin the squadron and work group, across the squadron, based onresponses to a series of questions.

(37) Average technician expressions concerning organizational identitywith the squadron and work group, across the squadron, based onresponses to a series of questions.

(38) Average technician expressions concerning their current assignmentlocality (and AF Base), across the squadron based on responses to aseries of questions (AF personnel only).

(39) Average technican expressions concerning their current pay andbenefits, across the squadron, based on responses to a series ofquestions (AF personnel only).

(40) Average technician expressions concerning their current social statuswithin the Air Force, based on responses to a series of questions(AF personnel only).

(41) Average fatigue trait of technicians, across the squadron, based ontheir responses to a series of possible identifiers.

(42) Average ascendency trait of technicians, across the squadron, basedon their responses to a series of identifiers.

(43) Average responsibility trait of technicians, across the squadron,based on their responses to a series of identifiers.

(44) Average emotional stability trait of technicians, across the squadron,based on their responses to a series of identifiers.

(45) Average sociability trait of technicians, across the squadron, basedon their responses to a series of identifiers.

(46) Average Job curiosity trait of technicians across the squadron, basedon supervisor evaluations of technicians within each work group.

(47) Average persistence trait of technicians, across the squadron, basedon supervisor evaluations of technicians within each work group.

(48) Average professional identification trait of technicians, across thesquadron, based on supervisor evaluations of technicians within eachwork group.

(49) Average organizational identification trait of technicians across thesquadron, based on supervisor evaluations of technicians within eachwork group.

(50) Average self-starter trait of technicians, across the squadron, basedon supervisor evaluations of technicians within each work group.

Thus the model included 48 predictor variables and two dependent variables:

speed of technician performance and accuracy of technician performance. The

13

q " - - _ _ _ _ _ - - - - - - - - -u- -. - -. .- - - - - I

reader may have recognized from the above list of factors that certain wellknown, existing and validated survey instruments provide information forsome of the factors and led to their inclusion. The Glossary of Terms sectionat the beginning of this report further defines some of the above characteri-stics.

Development of the Survey Instruments

The predictor factors (or variables) included in the modeling effortresulted from one or more of three conditions:

(1) The factor was a significant predictor of maintenance technicianperformance rate or technician performance accuracy in some previousresearch study (military and nonmilitary systems).

(2) The factor was suggested in the literature, or by one or more ofseveral Air Force maintenance officers interviewed, as relating in animportant way to maintenance technician performance and should beworthy of study.

(3) A measurement instrument exists, or is easily developed, for gatheringdata on the factor.

All of the 48 predictor variables initially proposed were included in thesurvey instruments developed. It was decided to separate the data collection intothree independent survey instruments, one of which would be completed by eachselected maintenance technician and two of which would be completed by thefirst line, immediate shift supervisor of each technician. The three survey in-struments developed are shown in Appendix B.

The selection of the performance measure or measures to be used in thestudy, as well as the methodology of measurements, required careful study. Thewriter was aided in this investigation by Captain Joel R. Hickman, USAF, whoundertook a graduate student project on this topic in partial fulfillment ofthe requirements for a master's degree from Arizona State University inIndustrial and Management Systems Engineering. The performance measure ormeasures would be the dependent variable(s) in the modeling effort. Certainportions of the following section are taken directly from or are summarized fromthe Hickman project report (Hickman, 1979).

Selection of the Performance Measures and Measurement System. Several evaluationschemes or rating schemes were investigated which purport to measure the perfor-mance of skilled workers. The purpose of the investigation was to select ordevelop a method for evaluating and measuring the performance of aircraft andmissile maintenance technicians in the United States Air Force. Criteriadeveloped for the performance measurement scheme may include:

(1) Be useful for describing performance to management.

(2) Be valid as a measurement of maintenance technician performance.

(3) Be applicable to different types of performance tasks such as repair,service, and preventive maintenance.

14

(4) Be applicable to military and civilian employees of the Air Force.

(5) Provide a performance measure throughout the many levels of weaponsystems maintenance.

(6) Provide valid information for statistical analysis in the form ofnormal performance distributions with constant variance.

(7) Meet time and cost limitations for use of the evaluation methodology.

Techniques for performance measurement were evaluated in terms of (1) organiza-tion structure, (2) the quality of ratings, (3) the above list of performancecriteria, (4) the appraisal methods, (5) rating scale errors, (6) scale formatand (7) the raters.

One of the restrictions on the performance measure used is that it beapplicable to different types of aircraft maintenance performed at differentorganizational levels. This is a difficult restriction to satisfy as the AirForce maintenance structure involves thousands of personnel performing a vastvariety of functions. McDonnell (1979) reports that there are forty-fivethousand Air Force members in the aircraft maintenance field alone.

Maintenance is concerned with aircraft and missiles and is performed bymilitary or civilian technicians of both sexes. The three overall levels ofmaintenance organization are known as base or organizational, intermediate, anddepot. Base level maintenance consists of inspecting, servicing, and replacingparts. Intermediate level maintenance is often indistinguishable from base levelmaintenance and consists of calibrating or replacing damaged or unserviceableparts, of modifying material, and of emergency manufacturing of unavailableparts. Depot level maintenance augments stocks of serviceable material with moreextensive shop facilities and personnel of higher technical skill level (usuallycivilian employees).

Further generality of the rating technique is mandated by the varied tasksperformed by a base level maintenance organization. A typical Air Forcebase with a mission involving aircraft might include field maintenance (FMSQ),organizational maintenance (OMSQ), avionics maintenance (AMSO), and munitionsmaintenance (MMSQ) squadrons. Meister, Finley, and Thompson (1971), Foley (1974),and Wiley (1978) have considered automatic flight control maintenance performancein the AMSQ alone, while Sauer, Campbell, Potter and Askren (1977) dealt withShort Range Attack missile maintenance in the MMSQ alone. Enlarging the scopeof a performance measurement tool to include avionics repair, fabrication, pro-pulsion, and flightline launch and recovery personnel requires a generalizedrating scale or scales applicable to many technician specialities or specific,noncomparable measures for each specialty. Separate measures, however, wouldmake analysis of overall performance within any squadron impossible.

The nature of the maintenance organization strongly favors the use ofgeneral individual performance measures. Such measures would be applicable tothe varied tasks and functions the different technicians are responsible for.Since most maintenance is performed by teams of five to ten techniciansworking under one supervisor, the supervisor could evaluate his personnel if ageneral, subjective performance measure is used. The structure, size, andcomplexity of the Air Force maintenance system thus requires the use of a new

15

subjective, and general performance measurement system for this particularresearch effort.

Barrett (1966:12) feels that a performance measure is successful only if itmeets three standards:

It must be accepteable to the people who use it; it must cover whatis important and only what is important; and a systematic examinationof the results of ratings must show that they are reasonably free fromimportant defects.

The performance data which will eventually be used to develop performanceeffectiveness models must be accepted by maintenance managers and evaluatorsas well as research personnel. The easiest way to gain acceptance might be touse existing measures such as Airman Performance Ratings (APRs) or meritratings for civilian personnel. These measures, however, are used for admini-strative purposes of promotion and wage administration and not for developmentalpurposes. McGregor (1957) and Barrett (1966) warn against mixing incompatiblepurposes in one program, as management is placed in the incompatible role ofjudge and counselor.

If a new performance measure is developed it might be advisable to solicitthe opinions of managers, using surveys or limited acceptance tests as tocriterion acceptability.

An alternative to using existing measures or soliciting manager opinionsas to acceptability would be to develop criterion-referenced test measures. Acriterion-referenced test measures what an individual can do, or knows,compared to what he must be able to do, or must know, in order to complete atask successfully (Glaser and Nitko, 1971: Swezey and Pearlstein, 1975).Such Criterion-Referenced Job Task Performance Tests (JTPT) were experimentallydeveloped by Foley (1974) for electronic maintenance tasks after much timeand effort. Such objective tests might prove to be more acceptable then othersubjective performance judgments such as supervisors' ratings.

However, acceptance is not enough; a measure that omits essentials orgives weight to trivia is defective. Barrett (1966) feels that a clearstatement of the objectives of the ratings is the first step while Guion (1965)believes that the first step is a judgment of the importance of the conceptbeing developed. Both authors agree that the second step is a clear statementof what the job requires and the kinds of job behavior that are essentialto success. As Barrett points out, punctuality may be important in anautomated office where each person's performance affects his neighbors, butit is unrelated to the success of a door-to-door salesman.

Subjective ratings or judgments are relied upon by management as criteriafor validation studies. Guion (1965:96) reports that eighty-one percent ofvalidation studies appearing in the Journal of Applied Psychology and PersonnelPsychology between January, 1950, and July, 1955, relied upon ratings.

According to Barrett (1966:33) rating scales are concerned with threekinds of concepts: personality, performance, and product. Personality isthe total of a person's characteristics. It includes emotional make-up,intelligence, and what is commonly called character. Performance has to dowith how an individual goes about doing work. Included are working hard,

16

.9.

following instructions, planning, and taking responsibility. Product is aperson's output. The quantity and quality of work are product.

The most pertient of the three is product. Management is fundamentallyinterested in sales, production of finished goods, and other factors that arevisible and inherently measurable. Product in some cases can be measureddirectly (objective measurement) and in other cases it is necessary to have arater look at the product and evaluate its quality. Measures of product oftensuffer from deficiency, as only part of an individual's output can be measuredin objective terms. They may also be contaminated since much of what is measuredis beyond the individual's contol; for example, product may be the output ofmany individuals, not one alone.

Existing ratings of individuals employed by the Air Force are of littlevalue except for administrative purposes. Airmen Performance Reviews (APRs)are inflated according to Callander (1979) and of little value as a singleperformance measure.

If products are not available for evaluation, the rater may evaluate howthe employee goes about his work instead of what he produces. Though not asobjectively measured as products, these job performance characteristics are bothratable and important. Studies by Barrett (1966) indicate that supervisorsand subordinates are quite sensitive to performance, agree on the relativeimportance of performance traits, and attach a great deal of weight to theperformance style used on the job.

In this case it appears that subjective appraisals are most applicable.There are, however, many potential traits that could be used. Lawler (1967:371)indicates that it is easy to err on the side of providing too many traits uponwhich to make ratings. Dunnette (1963:252) points out that the use of a singlecriterion is unrealistic while Rush (1953:23) indicates that between three andfive criterion factors surface in factor-analysis studies. The potential sizeof a study covering Air Force maintenance performance mandates the use of asfew factors as possible--either two or three.

Lawler (1967:371) indicates that one rating that probably should be includedis one on quality of job performance. When people are asked to make such generalratings on quality they act in a very predictable way, as efficient appraisersof critical incidentdata from their observations of an individual's performancein the past.

According to Locker and Teel (1977:246), conventional ratings constitutethe most popular form of appraisal technique. Rating scales generally haveseveral statements about employee characteristics or behavior. A continuousor discrete scale is established for each item. Figure 1 illustrates severalscaling procedures from Cummings and Schwab (1973:90). Item A is scaledcontinuously: the evaluator places a check somewhere on the scale to representhis assessment of the appraisee. Item B has a numerical discrete scalealthough letters are sometimes used instead of numbers. Item C is also scaleddiscretely with adjectives. Discrete scales generally result in greaterinterrater agreement and hence are preferable to continuous scales, accordingto Cummings and Schwab (1973).

Considerable attention has been paid in experimental psychology to theproblems of scaling to find out all that can be learned about man as a measuring

17

Item Scaling Format

A Overall job performance I I I ILow High

B Overall job performance i I I i1 2 3 4 5

C Overall job performance i I i ILow Below Aver. Above High

Aver. Aver.

Figure 1

Illustrations of Conventional Rating Scale Formatsfor a Single Item (Cummings and Schwab, 1973)

18I

instrument. Experience has shown that certain rules are favorable to effective

graphic ratings. Guilford (1954:267) lists the following rules.

(1) Each trait should occupy a page by itself.

(2) The line should be at least five inches long, but not much longer.

(3) The line should have no breaks or divisions.

(4) The "good" or "high" ends of the lines should be in the same direction.

(5) For unsophisticated raters, the "good" end should be placed first.

(6) Descriptive phrases or cues should be concentrated as much as possibleat points.

(7) Do not use end cues so extreme in meaning that they will never be

applied.

(8) Set the end cues at a little distance from the ends of the line.

(9) In scoring, use a stencil that divides each line into sections towhich numerical values are assigned.

It appears that the best scale format would follow the rules listed byGuilford. The rating standards, in this case, should be based on comparisonswith other technicians within a particular maintenance squadron. Using twoadjectives to anchor the ends of scales for quality and quantity of performanceappraisal serves several purposes: (1) the term "average" is avoided, (2)generality of the scale is maintained to make it applicable to many maintenanceactivities, (3) the possiblity of obtaining a normal performance distributionis improved, and (4) multimodal distributions grouped around descriptiveadjectives are avoided. The use of ten steps is familiar to the raters due tothe similarity with Airman Performance Ratings and, since interpersonalperformance is being rated, allows for finer distinctions between technicians.

Based on previous work the appraisal forms shown in Figures 2 and 3 weredeveloped and included in the survey instruments. They should provide usefulperformance information for valid statistical analysis.

Generating the Three Survey Instruments. The three survey instruments shown inAppendix B may now be discussed more fully. The first instrument, entitledSupervisors Technical Information and Performance Rankings Form, has two parts.The first part covers technical information needed from the supervisor to supportpredictive factors numbered 3 through 11. Horizontal interval scales wereselected foreach question with verbal descriptions at ends of the scales. Part2 contains the performance ranking scales for speed and quality of work, in whichtechnicians assigned to a shift supervisor are first ranked against each otherand then rated individually on a 1-10 scale against a near perfect "10".

The second instrument, also designed for supervisor completion, allows thesupervisor to evaluate motivation traits of each particular technician includedin the study. The Supervisors Technician Motivation Evaluation is coupledexternally to the Maintenance Technician Survey, as part of the data foreach technician selected for the study. One form is completed by the shiftsupervisor for each of the technicians (which he supervises) that are selected

19

fA I

4A

u16141111GD C C'

(U.C ......C

C w U4.

41 *J- =~G

c C 5 vi a) 'U U

o so m wU45 0 - C GDJ G

q i 4(A1~. w 4-

0) 4A 41 CL

Cm 411 3, I M fo CL 0

~C L..t 45 6

0. C 1 4

GD vi.OCL. CL

0 c -

.,4 0- u C 4.0 jc

QL4 .00 6C C

U U IA - CO31-

C C 1; 144 wb

U3w i a. = .C ..- L G.

g, .c zI.cc3 LL.ou

w~a;

U.M C.C'

4) ' L L.CGD

vIAGI.'

o0 6 .- . GD -

( 20

0 4)

413 @

) "a V

u0 4* CL 4) v

0u w. -

'a41 4'a 'A 4).c. L. EU L

Caa 0 L 0Eu

- v '0.%1 C.41 L.4-n0 41 V E ICI w

4- .

- CL M ''a WT 9S

= 3%0'c 4) 0a C, ='CL 3 c a 104.. I

41 1

'10 = X )% c 0'4 Ii .m 4~~~ 1 C)!t

lC0 4 .0 .0 0 0 @ 0

1. t @0 'a . 0 ,4 -A

C 3- 04) 0.14

'a ~ e A1. .0-~C C 1.0 I I20..

W ~O W L. 2%

"a- -C Eu~ @3 4wa Cc @3(m

LC~ *

C6 z L ou 4tj a *'

4) 412a

w 00,'Wo C, L' 0 C

0 40'I 34 ' U .1 CI *I SS.

IA 4J j 110

.C C - C C

1.21

in the sample. The survey instrument uses vertical scales, each containing5 tic marks, with a "high" description at the top of the scale and a "low"description at the bottom of the scale. The scales are a modification ofthose developed by Sauer et al. and used in a previous study (Sauer, Campbell,Potter and Askren, 1977). The motivation traits examined include (1) jobcuriosity, (2) persistence, (3) professional identification, (4) organizationalidentification, and (5) self-starter characteristics.

The third instrument is the Maintenance Technician Survey and is completedby each of the technicians selected for the sample. The body of the question-naire corsists of five areas, covering (1) group morale, (2) organizationclimate, (3) occupational attitude, (4) fatigue trait (feelings while working),and (5) personal traits. Each of these will be discussed in more detail.

Part I. Group Morale. The Bernard Goldman Group Morale Scale was adaptedfor this part of the survey with very little modification (Goldman, 1958). Thescale consists of 20 questions covering the following four factors:

(1) Group support of individual motives--questions 1, 8, 13, and 18[4 items].

(2) Group homegeneity of attitude--questions 2, 7, 11, 15 and 16 [5 items].

(3) Group interpersonal relationships--questions 3, 6, 9, 12, 14, 19,20 [7 items].

(4) Satisfaction with supervision--questions 4, 5, 10 and 17 [4 items].

As discussed in the earlier design report (Young, 1978), pros and consfor the Goldman Group Morale Scale have been presented in the literature.Further, theGI4S dates back to 1958. However, it is this author's opinionthat the GMS is still a useful and valuable survey instrument and that thequestions are well adapted to the survey of maintenance technicians with afew minor changes in wording. The GMS employs a four category answer directory:A - Strongly Agree, B - Agree, C - Disagree, D - Strongly Disagree.

Part II. Organization Climate. This section of the survey instrument wasderived from the Organization Climate Inventory (Litman and Stringer, 1968).The published OCI consists of 50 items and covers 9 factors. In previous AirForce studies which used the OCI, only the factors of (1) organizationstructure, (2) organization rewards, (3) organization risk, (4) organizationwarmth, (5) organization conflict and (6) organization identity have provensignificant as predictors of technician performance. It was therefore decidedto include in the Maintenance Technician Survey only those questions whichrelated to these 6 factors:

(1) Organization structure--questions 21-28 [8 items].

(2) Organization rewards--questions 29-34 [6 items].

(3) Organization risk--questions 35-39 [5 items].

(4) Organization warmth--questions 40-44 [5 items].

22

I9 - - - ----------- - ---- I.- - -

(5) Organization conflict--questions 45-48 [4 items].

(6) Organization identity--questions 49-52 [4 items].

The OCI is an often used and thoroughly validated survey instrument. Itwas adapted for use in the Maintenance Technician Survey by changing wordsrelating to organization form. The same response categories are used as forPart I.

Part Ill. Occupational Attitude. A survey instrument known as theOccupational Attitude Inventory was developed for the U.S. Air Force (Tuttle,Gould and Hazel, 1975). The instrument is specific to Air Force personnel;therefore, civilian technicians (employed by the Air Force) who completedthe Maintenance Technician Survey were instructed to bypass Part III. Theoriginal OAI consists of 348 items and covers 26 factors. The only factorswhich have shown significance in past research on Air Force maintenancetechnicians are (1) assignment locality, (2) pay and benefits, and (3) socialstatus. Thus the questions covering these 3 factors were extracted from theOAI for direct use in the Maintenance Technician Survey:

(1) Assignment locality--questions 53-69 [17 items].

(2) Pay and benefits--questions 70-81 [12 items].

(3) Social status--questions 82-92 [11 items].'

There are four response categories which are (1) very satisfied, (2)satisfied, (3) unsatisfied, and (4) very unsatisfied.

Part IV. Feelings While Working (Fatigue Trait). This section of theMaintenance Technician*Survey employs a series of response indicators to afatigue trait (Yoshitake, 1971). The items include some word modificationsfor proper interpretation by Air Force technicians. The factor covered bythe items included proved significant in the previously referenced study bySauer, Campbell et al.

(1) Fatigue Symptoms (Trait)--Items (indicators) 93-124 [32 items].

Two answer categories are used: (1) does describe feelings or (2)does not describe feelings.

Part V. Personal Traits. For this part of the Maintenance TechnicianSurvey, the Gordon Personal Profile was adapted (Gordon, 1963). The GPPconsists of 18 set of 4 descriptive phrases, each set being identified as atetrad. Four factors are covered:

(1) Ascendency--[18 items]

(2) Responsibility--[18 items]

1. Definition of terms are given in the Glossary at the beginning of the report.

23

Aq•m

(3) Emotional stability--[18 items]

(4) Sociability--[18 items]

Each item covers all four factors. The writer chose to use only thefirst 13 items, in order to limit the overall questionnaire to 150 questions.The matrix of selection response categories is as follows:

EmotionalItem Question Cat. Ascendency Responsiblity Stability Sociability

125 Most1 126 Least Sel B Sel C Sel D Sel A

127 Moast

2 127 Most Sel D Sel C Sel B Sel A128 Least

129 Most Sel B Sel 0 Sel A Sel C130 Least

131 Most Sel C Sel B Sel D Sel A132 Least

133 Most Sel A Sel D Sel C Sel B134 Least

6 135 Most Sel D Sel B Sel C Sel A136 Least

137 Most Sel C Sel A Sel B Sel D138 Least

8 139 Most Sel A Sel B Sel D Sel C140 Least

141 Most Sel B Sel C Sel D Sel A142 Least

10 143 Most Sel A Sel C Sel B Sel DlO144 Least

145 Most11 146 Least Sel B Sel D Sel A Sel C

147 Moast

12 147 Most Sel B Sel C Sel D Sel A148 Least

13 14 Most Sel A Sel C Sel B Sel D150 Least

Scoring on the Gordon Personal Profile is based on 2 points for a "mostlike" selection, 1 point for no selection, and 0 points for a "least like"selection. However, this writer chose to use a +1, 0, -l scale to highlightthe null position across the 13 items used. Thus an average score of "0" onone of the four factors would indicate a mean level of factor response wherein

24

the "most like's" and "least like's" cancelled out (or the factor was notindicated as a "most like" or "least like" in the 13 items).

Overall Maintenance Technician Survey. The final survey instrument, shownin Appendix B, consists of 150 questions, covering 137 items and 18 factors.In addition, the requested Biographic Information section provides input foradditional factors included in the modeling effort.

The Biographic Information section of the Maintenance Technician Surveywas designed to collect occupational information on the airmen completing thesurvey including sex, rank cr grade, squadron assigned, AFSC's, skill level,equipment assigned, extracurricular activities, etc. (see Appendix B). ForPrivacy Act compliance, the name of the technician completing the questionnairewas not requested; however, a survey control number was assigned to eachtechnician selected for the survey in advance.

The opening sheets on the Maintenance Technician Survey provide the sourcefor and purpose of the study, note the USAF assigned Survey Control Number,provide the necessary Privacy of Information statement, and give instructionsfor completion of the survey.

Conduct of the Surveys

As will be detailed later in this report, surveys were administered by thewriter and student associates at Williams Field AFB and Luke AFB, Arizona.Each AFB is approximately a one-hour trip by automobile from Arizona StateUniversity, Tempe, Arizona, where this modeling study was conducted. TheWilliams Field AFB Survey took place in October 1979 and the Luke AFB surveyin November 1979, each survey covering approximately a 3-week period. Clearancefor the study was solicited and received from AFMPC/MPCYS (Randolph AFB),prior to conduct of the surveys and survey control number 80-11 was assigned.Clearances for the study were also received from Headquarters Air Training Command(HQATC) and Headquarters Tactical Air Conmand (HQTAC), based on requests from theAFHRL, Advanced Systems Division, Wright-Patterson AFB, Ohio. Since it waselected to include civilian (civil service) USAF employees in the surveys,clearance was also obtained from the national and local union organizations.

Selection of the Sample Populations. In the early stages of this research,it was planned to pretest the developed survey instruments for construct validityand determination of sample size (based on response variability). However, theconstrained time span (9 months) for the study and the time necessary to obtainUSAF survey clearance finally ruled-out the pretests. Fortunately, an extensiveportion of the survey instruments were incorporations (with minor modifications)of previously developed and tested instruments and items. Further, theindependent factors included in the study had shown a significant relationshipto maintenance technician performance in one or more previously published studies.

The final choice of sample size and selection procedures stemmed from (1)this researcher's view of an acceptable sample size based on previous researchexperience, (2) practical constraints associated with conduct of the surveys,and (3) the maintenance organizations studied. It was decided to include 180maintenance technicians in the survey at each of the two Air Force bases.

25

- I I| I

Since the selected method of performance measurement involved first linesupervisor rankings and ratings of technicians, it was also decided that fivetechnicians should be rated by each supervisor.

Williams Field AFB has one AF Wing, the 82nd Training Wing, and involvesprimarily T37 and T38 aircraft. Maintenance is performed by two squadrons:the Organizational Maintenance Squadron (OMS) and the Field Maintenance Squadron(FMS). For the Williams study, it was elected to randomly draw 18 shiftsupervisors from each of the two squadrons. Supervisors with less than 4reporting technicians were not included in the draw. If a supervisor selectedin the draw had 4 or 5 reporting technicians, each of these reporting technicianswas included in the survey. If a selected supervisor had more than 5 reportingtechnicians, random selection was made of 5 technicians to be included in thesurvey. The number of technicians assigned to the Williams FMS squadron inOctober was 500, with 480 assigned to the OMS squadron. Thus the selectedsample design provided approximately the following percentages of the maintenancesquadron personnel (82nd AF Wing):

Squadron Size Design Sample Size Sample % of Population

FMS 500 90 18.0

DMS 480 90 18.6

Shift supervisors were drawn randomly across all three shifts and all shifts arerepresented in the sample. Technicians reporting to several of the selectedsupervisors were both military and civilian, and of both sexes.

Luke AFB has two AF Wings; the 405th Tactical Wing was selected for thesurvey study and involves primarily F15 aircraft. The maintenance organizationis defined as a Production-Oriented Maintenance Organization (POMO) and includesthree squadrons: Aircraft Generation (AGS), Component Repair (CRS) andElectrical Maintenance (EMS). Similar procedures to that used at Williams AFBwere applied in selecting the survey sample from the Luke 405th Wing. Twelveshift supervisors were to be randomly drawn from the list of shift supervisorsin each squadron, and for supervisors having more than 5 reporting technicians,a random draw was to be made of the 5 technicians to be included in the study.Because of researcher transportation difficulties, only the day and swing shiftswere included in the Luke study and thus all mid-shift supervisors were excludedfrom the draw. Further, some shift supervisors were on temporary duty assignmentsand were not available. For these and other easons, the selection of shiftsupervisors for the sample was less random at Luke AFB than at Williams AFB. Theselected sample design provided approximately the following percentages of themaintenance squadron personnel (405th AF Wing):

Squadron Size Design Sample Size Sample % of Population

AGS 800 60 7.5

CRS 463 60 13.0

EMS 330 60 18.2

26

The writer feels that the selected, stratified sample design is appropriatefor this type of research study in which comparative performance judgments mustbe made by supervisory personnel. The shift supervisors work most closelywith the maintenance technicians and are best able to judge their performances.In most cases, the supervisory work groups included between 5 and 10 technicians.Some thought was given to also having the technicians selected for the studyrated by the second-level supervisor (shop or unit supervisor). This procedurewas ruled out, however, because of the question of how to weight the performanceratings at two levels of supervision. All random draws were made from a tableof 4-place random digits.

Conduct of the Surveys

At each of the two AF bases, a classroom was available to conduct the surveys.The supervisor surveys were conducted first, over several days, followed laterby the surveys of selected technicians. All surveys were conducted in the class-room, after brief oral instructions, and questionnaires were collected as partici-pants completed their input and left the room. To minimize interference withwork requirements of the maintenance squadrons, it was necessary to have multiplesurvey sessions at each base for both shift supervisors and the technicians.Maximum attendance at any session was approximately 20, with a minimum of 2.

The writer was assisted in the survey at Williams AFB by USAF Captain JoelR. Hickman, a graduate student on temporary AFIT assignment to Arizona StateUniversity, and USAF Captain Jerry Raney of the Williams AFB maintenance organi-zation. At Luke AFB, the writer was assisted by Mr. Mark Bramlett, a graduatestudent on research assignment at Arizona State University, and USAF CaptainBob Tilton of the Luke AFB maintenance organization. All survey sessions weresupervised by one or more of the above, including evening and early morning sessionsfor swing- and mid-shift personnel. Needless to say, this involved numerousround trips from ASU.

Supervisors attending the early round of survey sessions (for supervisorsdrawn in the samples) were given a list of 5 of their reporting technicians whowere also drawn for inclusion in the study. For each name on these lists, atechnician survey control number was also assigned. The supervisors were instructedto use only the technician survey control numbers in completing the SupervisorsTechnical Information and Performance Rankings Form and the Supervisors TechnicianMotivation Evaluation questionnaires. All responses were to be placed directlyon the survey instruments (and would be hand scored). Supervisors were permittedto take the list of technicians with them at completion of the session so thatthey could assign the technicians to report to the classroom for completion of theMaintenance Technician Survey during the multi session times for techniciansurveys. Thus a supervisor only needed to part with one of his reportingtechnicians at a time, for a period of approximately one hour.

The supervisor survey and the technician survey each required one hour orless for completion. In the technician survey sessions, instructions were givento read the opening material and complete the Biographic Information page beforepreceding to the survey questions. Only one technician, of all surveyed at bothAF bases, exercised his/her right to withdraw from the survey. A few othertechnicians failed to show, however, even after extra sessions were scheduledand held. In general response was excellent.

Technicians were instructed to respond to the 150 question survey by (1)entering their assigned survey control number (written in the survey booklet and

27

14

also on a machine graded response form) in the block at the top of the formfor machine scoring, and (2) enter their responses to the 150-question surveyon the machine scoring form. A copy of the machine scoring form, IBM 1230Document No. 511, is included in Appendix B. The form has response fields forexactly 150 questions. Technicians were instructed to leave the heading informa-tion blank. The completed form as developed by each technician was inserted inthe survey booklet when submitted, and a check was made to verify the surveycontrol number (identification number) with that written inside the survey brochure.

Processing of Data

A FORTRAN IV computer program was developed for processing the input datainto printed output information and a set of punched cards with data preparedfor input to the Biomedical Data Processing Software Package known as BMDP'.The program, designed to run data for one squadron at a time, is given inAppendix C with output for Williams AFB, FM Squadron. The program is availableon punched cards and on a magnetic tape file, currently set up on the Univac1142 Computing System at Arizona State University. The program is designed toinclude Mapping and requires 1143 words of program bank and 95, 944 words ofdata bank as presently constituted. Data dimensions are as follows:

(1) Number of supervisors per squadron--up to 20

(2) Number of technicians per squadron--up to 200

The FORTRAN program utilizes three types of input data, corresponding tothe three survey instruments employed. These types of data are fully documentedin the program and may be summarized as follows:

DATAI(I,J) - Inputs from the Supervisors Technical Information and Perfor-mance Rankings Form, Part I. The five technicians reporting to eachsupervisor are first identified by technician survey control nutmbers;then the technical information data of Part I is entered. Coding isby hand directly from the survey instrument. Eventually, this data iscoupled to the survey data for each of the 5 reporting technicians.J is the supervisor counter (1-20) and I is the number of fieldpositions needed per supervisor (40 available, 15 used).

DATA2(N,K) - Inputs from Part II of the Supervisors Technical Informationand Performance Rankings Form (the speed and quality performancerankings for each technician), the Supervisors Technician MotivationEvaluation form, and the Biographic Information section of theMaintenance Technician Survey. Coding is by hand from the surveyinstruments. K is the technician counter (1-200) and N is the numberof field positions needed per technician (56 available, 29 used priorto the later data coupling operations).

1. BMDP (Biomedical Data Processing) is a software program for performing stati-stical evaluations on data, developed by the Department of Mathematics,School of Medicine, University of California at Los Angeles.

28

DATA3(I,J,K) - Input from the 150-question Maintenance TechnicianSurvey, which is machine scored. Four input data cards are usedfor each technician's data. K is the technician counter (1-200),J allows for up to 5 cards per technician ( 4 are currently used),and I is the number of field positions per input card (72).

The technician survey control number, which appears on each of the threetypes of data input is used to couple all data to each particular technician.First, the data in the DATAI(I,J) file is added to the DATA2(N,K) file bytechnician. Then the DATA3(I,J,K) file is summarized by factor, questionresponses are averaged across each factor and the factor data is added toDATA2(N,K). After coupling, DATA2(N,K) contains all of the appropriate datafor each technician in the squadron, for a total of 56 fields of information.It should be noted here that the 150 questions in the Maintenance TechnicianSurvey result in only 18 predictor variables or factors.

Appendix C shows an example run of the FORTRAN program for one squadron,Williams AFB, FM squadron. It may be noted that echo checks are made of theinput data, output data from DATA2(N,K) is printed following data collection, anda set of punched cards is output with most of the same data, prepared for useby the BMDP statistical package. The punched card output is on 2 cards, each2513 format. For input to BMDP, the technician number is unnecessary and thevariables of squadron code, supervisor code, AFSC, status code and supervisioncode were considered not directly pertiment to performance. Thus the 50variables used by the BMDP package are as shown in Figure 4, using appropriateletter names.

Output decks from the several FORTRAN data processing runs (one for squadron)were then input to the BMDP statistical package and in particular the 2R stepwise,linear multiple regression program. For each squadron, three runs were initial-ized: one in which the Performance Speed Rating (A) is the dependent variableand the Performance Quality Rating (B) is included as another independentvariable, one in which the Performance Quality Rating (B) is the dependentvariable and the Performance Speed Rating (A) is included as another independentvariable, and one in which the dependent variable is a weighted transform ofthe two performance measures and neither appear as an independent variable. Thelatter is the most significant.

Hickman made a small survey of senior maintenance officers in which he includedquestions on the relative importance of speed vs. accuracy of performancerelative to current Air Force mission (Hickman, 1979). The results favoredaccuracy, at least in peace time, over speed, though both were considered ofimportance. This writer elected to weight speed by 0.4 and accuracy by 0.6 indeveloping a transformed variable. Thus ZZ = 0.4A + 0.6B. These proportionalweightings are elective. For a summary discussion on the BMDP2R program see the1977 Series P Manual (University of California, 1977).

Appendix E provides an example output from the BMDP2R program, for the FMsquadron at Williams AFB when the transformed variable ZZ = O.4A + 0.6B isused as dependent. The punched card output data from the FORTRAN program isread in format 25F3.0/25F3.0. One variable is added to allow for the ZZtransform. Having established ZZ, variables A and B are deleted by using onlyvariables 3 through 51 in the analysis. Printing is established for thecovariance matrix, the correlation matrix, the input data distributions, thestep analysis of variances (ANOVA), the stepwise regression coefficients, thepartial correlations, the predicted values and residuals, and the summary. Plots

29

Figure 4

BMDP2R Variable Identity

A Speed Rating AA Clearance for Remove/ReplaceB Quality Rating BB No Internal ComponentsC Job Curiosity CC Hours Between Equipment ServicingD Persistence DO Quality Technical InformationE Professional Identification EE Quality Test EquipmentF Organizationa Identification FF Technician KnowledgeG Self Starter Tract GG Technician SkillH Work Shift HH Satisfaction Ind. MotivesI Enlist Year II Homogeneity of AttitudeJ Skill Level JJ Sat. Interpersonal RelationshipsK Current Duty Assignment Year KK Satisfaction with SupervisionL Current Equipment Assignment Year LL StructureM Months in Current Assignment MM RewardsN Sex NN Risk0 Equipment Type 00 WarmthP Rank PP ConflictQ Hours/Shift QQ IdentityR Days Between Breaks RR LocalityS No. Additional Duties SS Sat. Pay and BenefitsT Hours Additional Duties TT Social StatusU No. Clubs UU Fatigue TraitV No. Persons Supervised VV AscendencyW Months Supervision WW ResponsibilityX Weight Handled XX Emotional StabilityY Clearance for Service YY Sociability

30

,1 N -

requested include the normal probability plot of residuals and a detrended(trend removed) normal probability plot of residuals.

The BMDP2R computer program computes estimates of the parameters of amultiple linear regression equation in a stepwise manner. Variables are entered(forward stepping) or removed (backward stepping) one at a time according to anyof four possible criteria. As may be noted in the BMDP2R sample output inAppendix E, control limits on the stepwise procedure are established by speci-fying F-levels to Enter and Remove and a Tolerance based on whether an entryor removal of a variable will produce an RI (accountability level) change,when compared to previously entered independent variables, exceeding thetolerance. The first numbers in ENTER and REMOVE establish entry limits and thesecond numbers in ENTER and REMOVE are used as remove limits. The particularnumbers selected for use have been found to work well in providing qood predictorvariables with a very low level of multicollinearity; i.e., a high level ofconfidence can be placed on the variables selected for the prediction equationat the termination of the stepwise procedure.

At each step in the stepwise regression, an ANOVA is developed and thefollowing parameters are printed: F ratio, R (multiple correlation coefficient),R2 (which in percent form accounts for the variation in the dependent variable whichcan be accounted for by the independent variables included at the step), and theStandard Error of the Estimate. In general, R2 values of 0.70 and better areconsidered very good in subjective research [70 percent accountability].

Results of the Study

Williams Air Force Base. For the Field Maintenance Squadron, data wascollected from 18 supervisors and for 89 technicians. Sixty-three of thetechnicians worked day shift, 26 worked swing shift, none were mid-shift. Sevenof the technician respondees were female, 18 of the technician respondees werecivilians, 25 of the technician respondees supervised others in some capacity.Four of the technicians evaluated by their supervisors were "no shows" for thetechnician survey. The computer output from the FORTRAN data processing programis provided in Appendix E.

FMS Statistical Results. As described earlier, three statistical computerruns were made from the data for each squadron, using the BMDP2R stepwise linearmultiple regression model. The selection of dependent variable was varied.

Table 7 shows the FMS data after processing for statistical distributionproperties. It may be noted that the mean speed of performance rating bysupervisors, across the squadron, is 6.5467 with a standard deviation of 2.1008.Likewise the mean quality of performance rating, across the squadron, is 7.3067with a standard deviation of 2.0466. Letter names were applied to the variablesas shown earlier in Figure 4. Coefficients of variation were high for M (monthsin current assignment), S (number of additional duties), T (hours on additionalduties), LL (number of service clubs), V (number of persons supervised), W(months in supervision), VV (ascendency trait), WW (responsibility trait).Using the stopping criteria discussed earlier, the stepwise forward and backwardregression included 26 steps when A (performance speed rating) was used as thedependent variable and resulted in the summary table shown in Table 8. Thefinal model included 10 significant predictor variables, including the performancequality rating. The resulting prediction equation is

31

fml a00 N6~aNQV(?0@4N= 7Ncoot OM 4~INOf.e.pa

3, . i ... i T .- - - - - I -- - - - - NN~-. -

0-01 40 ,NP C = .!0 t7 0 IA7z d040 .

4 N~~N IMNNN I Nln-v*I Il I ~ N N N N NN - - N N IzQaaaaaaa ils6i 1 'I'll am #ga., aaaaam saammaau1aua

UA

Aw Coc cc0c0oom000C aQQOOZmQO~m =00 0OZZ0O000CCCCC 0jz QQ 0Q~O aaZO Q .000mOOOOo~oooo oo oooooooooc 0

CLI

0JaZ.ZZ2 Z .. aOjZ 4.a Q= =Qa amoO QOM =0.0.0OZ 0.0..0 ZZMo.Ooooooo.oooo -aZ 0

Jla.J ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Z zz.0 0 Z~ 0 0 0 0 0 0 0 o o0 . . 0 0 0 . 0 0 0 0 0 . . z 0

i r C C C O O MC O O O C O C C O O O . 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 (P. 0 o c o 0

M -n4-)'

fu

............... .. .. ...

'U *.- cb N .z .

.. ~~~,-aamllN N N --N (( - - aa - - a I

3 .7ZC-672 j .- JZ2L3%rI . .

~ ON-NNNN aNNO.0.4N

0.3-2

-. c

42)

x-A.

-4-j

4 )

NN mr rN,1'.

LQ)

LL. 4-)

-C C

:N-M0-r - NA-CZI

'U t.A33

A = -0.669 + 0.6088 + 0.358F + 1.107H - 1.151N + 0.148P + 0.382S + 0.025X

- 0.790Y + 0.003CC + 0.253EE [78% prediction]

The multi'lc R for the above equation is 0.8848 and 78 percent of thevariation in A is accounted for.

Table 9 shows similar results when B (performance quality rating) is thedependent variable. The final model included only 5 significant variablesafter processing through 22 steps. The resulting prediction equation is

B = 0.768 + 0.566A + 0.780C - 0.OOL + 0.279R - 0.109V [76% prediction ]

The multiple R forthe above equation is 0.8722 and 76 percent of the variationin B is accounted for.

Table 10 shows similar results when the transformed dependent variableZZ = 0.4A + 0.6B is used and A and B are deleted. After 29 steps, only 3predictor variables remain and the resulting equation is

ZZ = 1.599 + 1.300E + 0.219P + O.O09X [65% prediction]

The multiple R for the above equation is 0.8077 and 65 percent of the variationin ZZ is accounted for. Of course, if we accept a larger tolerance relativeto the partial correlations, and more predictor variables, more of the variationin ZZ can be accounted for. For example, after 18 steps, there are 14 pre-dictor variables which account for 78 percent of the variation in ZZ. At thatpoint the equation is

ZZ = 1.856 + 0.936E + 0.336G - 0.418H + 0.139J - 0.012L + 0.187P + 0.214S

- 0.340U + 0.017X - O.340AA + 0.377GG + 0.05611 - 0.039PP

- 0.020SS [78% prediction]

It may be noted from the above that 11 more predictor variables (and thequestionnaire data to support them) are needed to provide a 12 percent improve-ment in accountability. Further, multicollinearity may be present.

Although not included in this report, the prediction and residues tablesin the computer reports show that when A is used as the dependent variable, onlyone time in 75 predictions evaluated did the number of standard deviationsbetween actual and predicted exceed 3 and in only one other trial did it exceed2. Thus the prediction capability is quite good. When B is used as thedependent variable, in 75 predictions the standard deviation between actualand predicted exceeded 3 once and exceeded 2 on three other trials. With ZZas the dependent variable, in 75 trials the standard deviation between actualand predicted exceeded 3 once and 2 in four other trials.

34

a.

S--

.. J

LL.JCA

o

-z

(. 7 . . . . . .

C

4j

ex

-o ~

35

• • • i• •11 . • • i I Ii i d i l 0

iS-4- MV04N04-O t ! 01-V0

0 .40 .0 f: 0t

0 Vi l4 S0 N-6A~i 0

U--

3 main *1416= -'- 0-i

2 cc lw .. .. .-LL. toh4.- N~N N

cm Olf PU0NPdfO-.PUsf" 6 Nel

-... .... ..l C!940.C0N004NsNiNW

0 40 ~ il oe~f-.4- hrh

LLvit 0N'IN-N.NNN N

064

to- . ~ A.....

IIILtoS *g~I~IO

co'"I hl 1

I.j

10hI

a.t N- ~NN-l3

OMS Results. For the Organizational Maintenance Squadron at WilliamsAFB, data was collected from 18 supervisors and for 70 technicians. Thirty-twoof the technicians worked day shift, 27 worked swing shift, and 11 worked mid-shift; 40 of the technician respondees were female. All were military. Twentyof the technician respondees supervised others in some capacity. Four of thetechnicians evaluated by their supervisors were "no-shows" for the techniciansurvey. The computer output from the FORTRAN data processing program isprovided in Appendix F.

OMS Statistical Results. Table 11 shows the BMDP2R data after processingfor statistical distribution properties. It may be noted that the mean speed ofperformance rating, across the squadron, is 6.7714 with a standard deviation of1.8971. This may be compared with parallel figures of 6.5467 and 2.1008 for theFMS. For quality of performance for OMS the values were 7.3143 (mean) and 1.8537(standard deviation). Comparable figures for the FMS were 7.3067 and 2.0466.Coefficients of variation were high for some of the same variables as for FMS.When A was used as the dependent variable, the stepwise regression involved30 steps and resulted in 8 significant predictor variables. The resultingprediction equation is

A = -1.846 + 0.547B + 0.388F + 0.256Q - 0.345R - 0.389LL + 0.784FF

+ O.lI2LL - 0.089QQ [72% prediction]

The multiple R for the above equation is 0.8479 and 72 percent of the variationin A is accounted for. The summary is provided in Table 12.

Table 13 shows similar results when B is the dependent variable. The finalmodel includes 15 predictor variables after 29 steps. The resulting equationis

B = 0.811 + 0.283A + 0.655D + 0.747F - 0.468G + 0.569J - 1.266N

+ 0.5190 - 0.137P - 0.371R - 0.007X + 0.278EE + 0.422GG

- 0.081PP + 0.132UU + 0.152XX [81% prediction]

The multiple R for the above equation is 0.8983 and 81 percent of the variationin B is accounted for.

Table 14 shows results when the transformed dependent variable ZZ = 0.4A + 0.6BIs utilized. Stopping after 27 steps the multiple regression model gives 11predictor ,ariables and results in the following equation:

ZZ = 1.142 + 0.364D + 0.727F + 0.815J - 1.680N - 0.405R - 0.282AA

+ 0.268EE + 0.723GG - 0.04811 - 0.047PP + 0.133X (78% prediction]

The multiple R for the above equation is 0.8846 and 78 percent of the variationin ZZ is accounted for.

37

Computer program prediction tables show that when A is used as thedependent variable, no trials err by 3 or more standard deviations, threetrials err by 2 standard deviations. With B as the dependent variable, thesame results are obtained. With ZZ as the dependent variable, in only twocases do the prediction errors approach two standard deviations.

Williams AFB Overall Results. The goal of this research study is to providea measurement instrument which will model performance effectiveness within anAir Force squadron. Performance is a combination of rate of work and accuracyof work. For the peacetime Air Force, the weightings of 40% on speed and 60%on quality seem reasonable. Therefore, the model which predicts ZZ (O.4A + O.6B)seems most useful.

Similarities and differences may be observed for the ZZ prediction modelsfor the FMS vs. OMS at a 78% level of accountability:

FMS: ZZ = 1.856 + 0.936E + 0.336G - 0.418H + 0.139J - O.012L + 0.187P

+ 0.214S - 0.340U + 0.017X - 0.340AA + 0.377GG + 0.05611

- 0.039PP - 0.020SS [78% prediction]

OMS: ZZ = 1.142 + 0.364D + 0.727F + 0.815J - 1.680N - 0.405R - 0.282AA

0.268EE + 0.723GG + 0.04811 - 0.047PP + 0.133XX [78% prediction]

Variables J, AA, GG, II and PP are common between FMS and OMS. However, severalfactors which surfaced as significant for FMS are different from those whichsurfaced for OMS. Table 15 lists the factors which are involved for eachsquadron.

It is also of interest to note what factors for OMS would provide approximatelya 65 percent prediction level (Multiple R2 ), similar to the 3 factor FMS model.This reduced equation, obtained at step 5 for the OMS data, provides the predictionequation

ZZ z -0.187 + 0.907F + 0.864J - 0.466R + 0.215EE + 0.633GG [67% prediction]

For the equation, the multiple R is 0.8163 and 67 percent of the variation inZZ is accounted for. However, the Beta coefficients may be weak.

The results in Table 15, based on R2 comparisons at approximately 78 percentaccountability, show the significant Beta coefficients for the prediction equations,the mean values of the variables from the data collection, predicted values forZZ from the equations and the actual mean value of ZZ based on supervisorevaluations across each squadron.

38

N

hiM. O -.O~'P4446 4ii~t 200 NNA dI4 4

it 'A

V.04 N-0 -ON. -- WNOt CP Nb'. Ae.. In joj !-3c T2-- '' N1l v14 wVI

Ko ils if iBs ,of IIIi o i i i l i i i i i l

.. i"ZO 0:00 ZOMOZ 0000 ZQOO00000 00=0=00000Z 020mzm ZD 3

to zS_' ~ 0 0

0NN

SoCt l0A

w...

ea Ad 2ggi

A

ea

0 N ?I -7

N .4T2 c''c 11 = POJ(

0.)

I..z

2x04-4r

-C Z0 ~ I@ 044A 2-0O ?.O~ rp

o -0iOC NC -r*'*4@-4 0- iI4O~.N~J N39~.

.4

0 W.ILW

S 0-44

024 wi

-- 0-

0 r_

V)a)-s ar*;-zN 040 cN A~

C) I N .0 wr0v m

IA.

ea ..

100)i

4e -4-@-.0-4i N w a-

J 9fnem- IP,9-0- 4ca .

U4-)m.. .. . .. . .. . . .. . . .

x 01 W 61 JZ-9 00 a0 00c.-w.-N- go4 0 00 0 0 0 0 0 00 0 0 0 0 0

04

4o

cc

Jz

- 00

cc c

M.J:D

73 00

S- Q.

ICI XGD -L

4-)Ln

4J

E 0-00noo" & om

c

A A O

.j

a-wI0w=.

N - 1MNC ~ l-0.4.Qc uL"Z-16 4

WMW Mf

44

W.

ta

U'a

o -o

NN2

-:zwg0-

6' m04 co 000

M c r - e " . w . N . 0 0 N - 1 M M O00DM

Cwai . .... N~

4-i

-a *av

.0~ ~~ Li... i

x- o

z 'a.N~hi W.. NN W U4N lO? W

LA Aor-.i iz 'a~es42

Table 15

Comparison of Multiple Regression Equations for Williams AFB,Organizational Maintenance and Field Maintenance Squadrons

(Compared at R2 Level of Approximately 78%,Dependent Variable is ZZ)

FMS OMSBeta Beta

Factor Coeff. Mean Contr. Coeff. Mean Contr.

1. 0 - Persistence +0.364 3.114 1.134

*2. E - Prof. Identification +0.936 3.040 2.845

3. F - Organ. Identification +0.727 3.043 2.212

4. G - Self Starter Trait +0.336 2.760 0.927

5. H - Work Shift (performancelower in swing and midshifts) -0.418 1.280 -0.535

6. J - Skill Level +0.139 4.560 0.634 +0.815 4.857 3.959

7. L - Current EquipmentAssignment Year -0.012 65.573 -0.787

8. N - Sex (performancelower for male) -1.680 0.943 -1.584

*9. P - Rank +0.187 4.947 0.925

10. R - Days Between Breaks(performance lower withlonger work weeks) -.405 5.057 -2.048

11. S - No. of Additional Duties +0.214 0.613 0.131

12. U - No. of Clubs -0.340 0.613 -0.208*13. X - Weight Handled +0.017 39.133 0.665

14. AA - Clearance forRemove/Replace ofComponents -0.340 2.467 -0.839 -0.282 2.929 -0.826

15. EE - Quality of Test Equip. +0.268 1.143 0.306

16. GG - Technician Skill +0.377 3.467 1.307 0.723 3.857 2.789

17. II - Homogeneity of GroupAttitude +0.056 23.947 1.341 +0.048 26.971 1.295

18. PP - Organizational Conflict -0.039 22.547 -0.879 -0.047 22.200 -1.043

19. SS - Satisfaction with Payand Benefits -0.020 18.843 -0.377

20. XX - Emotional Stability +0.133 -1.786 -0.238

ZZ Intercept 1.856 1.142

Predicted Mean Performance Level ZZ for Squadron 7.006 7.098

Actual Mean Performance Level ZZ for Squadron 7.003 7.097(Based on Supervisor Evaluations)

*Present in FMS final 3 factor model.

43

Luke Air Force Base. For the Component Repair Squadron, data was collectedfrom 12 supervisors and for 58 technicians. Thirty of the technicians workedday shift and 28 worked swing shift, 8 of the technician respondees were female,3 of the respondees were civilians, 17 of the technician respondees supervisedothers in some capacity. Five of the technicians evaluated by their supervisorswere "no-shows" for the technician survey. The computer output from the FORTRANdata processing program is provided in Appendix G.

CRS Statistical Results. Table 16 shows the CRS data after processing bythe BMDP package for statistical distributions properties. The mean speed ofperformance rating by supervisors, across the squadron, is 7.0508 with a standarddeviation of 2.4735. The mean quality of performance rating is 7.8136 with amean of 2.3229. High coefficients of variation are noted for LL (number ofclubs), W (months of supervision) and WW (responsibility trait). Using the samestepwise stopping criteria as for the Williams AFB data, the stepwise regressionincluded 13 steps when variable A (performance speed rating) was used as thedependent variable, resulting in the summary table shown in Table 17. The finalmost efficient model includes only 3 predictor variables and results in thefollowing prediction equation:

A = -0.442 + 0.481B + 0.201V + I.002GG [70% accountability]

Multiple R for the above equation is 0.8338 and 70 percent of the variation inA is accounted for.

A maximum R of .8691, with 76 percent explanation, is obtained with an 8factor model after 8 steps.

A = -1.104 + 0.2698 + 0.510F - 0.337U + 0.138V + 0.249AA

- 0.004CC + 1.145GG - 0.131XX [76% accountability]

Table 18 shows similar results when B (quality performance rating) is usedas the dependent variable. The efficient model includes 7 variables after 31steps, resulting in the equation

B = 1.014 + 0.266A + 0.623C + 0.5900 - 0.039K - 0.960N + 3.6550

+ 0.359DD [77% accountability]

The multiple R for the above equation is 0.8747 and 77 percent of the variationin B is accounted for.

Table 19 shows the results when the transformed variable ZZ = 0.4A + O.6Bis used as the dependent variable and A and B are deleted. The model stopsiteration only after 42 steps and includes 18 variables as indicated in thesummary. The prediction model is

44

0.!!r Plu N'00 d%0 O~ D.rNI i P .

at -pr ! !11 hp TT",4 - T'"NNf -1Ie

140.is-

A

.Jem4. . .. .N.. . .. 1 .. .' .~~uO 4~0 s~ .N .W~rN .. . .N .4 N~$ ~~N '.JN N 5 -N-NN.4 0 1 4 - N5CNNNN.N.NN

San

S_ N 0 000000c~a00o0~0O~ooOOO000~OO~~ai 0 01,0 =0000ooO o0coo~o~ooooe00ocoo ,0 o00O0000oo0o 0

C. W2e000 ooOOo000oooo000o000000009000oo w -, 09c! 0OC0ooo0 oooo0 C 0oc C, 0O ccO0oO

'Cow VW VI o r7M A01,0Ora'

- co~o coooocomoOOoooooo~cOO00OOOOOOOO~OooO 000000mizAoo CO O0O0OOOOOOO00OO0OOOO00)OOOOOO0OOOOCOOO 000000~

4-) J.JOO~OOOeOOOOOOOOO000000000o00o0O0o00000 00000'A _J'K.......

7.14

Ing a-N a 11 " 4 A :20,I

Uisafl NV N r -a.. V a -541- 00Q 44 r O -. r Nw N0 O -o V

I N- I

C

0 3

0a W- 'a1. M2!5 'A In 0gsgsgg-asgmgsmg 0 " NI0M

V)..........

McrfI- n 0 N CO .'. c M:NaO O4 '. N V ??0 .0N I N:nS 4J.N

U. 2..NNN9 N-9. 54 N-a l01

_jA

-1?110 0: 1 -I -' '-0I= "

Li.Z""--- mm-40 NN.0 T NfZ..-9-0 ;41 V a N =,A -N.0 NN N55 NNiii- -9-

jz

-- 4

co. z

N

S wxLz

0).

.0 .

00

Lo a) ..

ea a

I", x ...

4j

q0.)4. N~ r~9 -0-mN0

.~ C 2.n-00000~Joe

.j f" N4ANNj

tv 0-0194 N N~~. N

haWNNN.4

-e46

.4

4

* zo0 ccN z =

w Wz

4- cc

z

aK 0

CP.0

CL00

4JN N flN4 a5j.

U- .cohA0)4..W -N g.4m O Nm N

ra uI-. t

0 ,in ma4p;waa m

4N0,'f N c U:f l ZIPIN 0; ze~m m 11 fl OW 4p0M 1J

uj R

CAV)M V~4OI Q 4 .W-4n W N

Z j

zz

O -JD

Wa

-J.JZ Nw.'44NS

IL c

2

44

CV.

La w

S-i

L.

~ ~ V~fl4~@O~O-O--Nv9?~V. N.O-

C-10

.4.4 ,4 , 1. 1 H 4 , O vN TAN

C

cow. 0.z...

-4

MA N Z .0 =^"O NM 7 -10 A -a. I CEO c (P ? YamL.

0 0

co

-IP

412 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ T r N 11 2P0O?0 41 1N~ 13 ?00

4 N N 6 6.c~ ~ ~ ~ 3~ A4NO'?.11 3 . O N N ~ P N ..

-n -4 r

='N" 0*z .=; al r*IZ3fm NN.. NN

~ '0 0 48

ZZ = 2.369 + 0.805C - 0.0881 - 0.031K + 0.048M + 1.382Q - 0.631R

+ 0.209T - 0.387U - 0.136V - 1.015Y + 1.181AA - 0.011CC + 0.483EE

- 0.41OFF + 0.754GG + 0.06911 - 0.055RR + 0.278WW [92% accountability]

The multiple regression R for the above equation is 0.9566 and 92 percent ofthe variation in ZZ is accounted for.

Approximately 77 percent of the variation in ZZ can be accounted for byonly 7 variables and the resulting prediction equation

ZZ = -0.481 + 0.748C + 0.303D + 0.298J - 0.017L + 0.446AA

- 0.005CC + 1.095GG [77% accountability]

At the 92% level of accountability for ZZ, 3 residuals varied by 2 standarddeviations, none varied by as much as 3 standard deviations.

EMS Results. For the Electrical Maintenance Squadron, data was collectedfrom 11 supervisors and for 51 technicians. 29 of the technicians worked dayshift and 22 worked swing shift. Four of the technician respondees were female,all of the respondees were military, IS of the technician respondees supervisedothers in some capacity. Three of the technicians evaluated by theirsupervisors were "no-shows" for the technician survey. The computer output fromthe FORTRAN data processin S .-ogram is provided in Appendix H.

EMS Statistical Results. The mean speed of performance rating by super-visors, across the squadron, is 6.5769 with a standard deviation of 2.4443 asshown in Table 20. For the quality of performance ratings, the mean is 7.4038with a standard deviation of 1.9328. When A (speed of performance) was used asthe dependent variable, the BMDP program iterated through 43 steps and producedthe following prediction equation with 17 variables:

A = -0.112 + 0.343B + 0.4750 - 1.076H + 17.0350 - 1.680Q

- l.OOlS + 0.440T - 0.009X + 0.384AA + 0.9580D

- 0.310EE - 0.895FF + l.lOlGG - 0.252JJ + 0.146NN

- 0.284WW - 0.316XX [90% accountability]

The multiple regression R for the above equation is 0.9476 and 90 percent of thevariation in A is accounted for. Table 21 shows the summary results.

At an accountability level of approximately 77 percent, the resultingequation contains 10 predictor variables

49

"COW 1- ONWO.a.. -4e440 -p- b 00 4b..coN. ft

4404 . ... t.. N 0 4.-. N@ Q b4.i. * : = C44-.~ C.N AN . U.V.U i

LFICIC .W oov rf ==CO jbb9mN. klrNT .009 a,=*- -lNNNCP=b02w OC.tW044U4In0 Nf 0N00 UI -N;-4 in N 04U 0db0 UNj4U01" OMNwle...N N

49 Nt" 0-NNN MM fqNN I MM -f"N I I I I a I N~tN MfM NM I ~N I

0

0

N b- 000000000000000000000000000000000000000200000000000 a0 0 N O000000000000000000000000000000000000000000000000 Cz

N'aCj

w 00000000000000000000=000000000000o000a~ 0oooooJ-CC V!T ooTo o!CC! o oo9o 91 CM o !oo9 o o9l! o In e 9 e!o99 9

.4.41 '1*4/b1

.4-) ....... N.. .. 0 .3t! .. tT 4 43iN . .044 id C..i

V ~ ~ bNNN-04S~?090.N029. r-N-?C4400 ,*- - NNN N~N CC

C Z

-c w............ .. 4..... o ue oe -. i 4c.A N .1b 9 4-?C

0A

CDo, -z oa o

V --.400~'..O~3~-4@ N CA.Jb2 WJ9...~ 3:-b4U

,Of *AA 0ON -N0- M 4ntr4M 4MMN OV0GTIN- %d-4- .D 4t4113 sr IN1

CL W.

4..)

* 0 50

.0

wi"

.0

I-

0 c

L v

r_ - 0

0 a NNN N N NN N N NN

0 04-4L 2

z

3' i 4 A 1 a ;3-I.?9n

4 0? 0 0 4 @

CC 4-J*

2r 0 0 N4)~

.)

UEC INTO1-0-- 9.9 - ?04N.O 40 0 T mL (.J m W 0 4 4 0 ? 4 N 4 4 1 0 oln

L.9-

-00 0 00

1M ?W4 NO.Q. N N 0 0 ? N N N ?.4M4N 0-0 e.. 001>1 t llttt . t O~a: Ot . 00

W 4l0 4 N -N - 0 0 0 0 0 0 0 0 0 0 0 0 0 0

23. *.

S..o ?.40 00 G.9P. .44 N~p44P1.AN0-..*NM 4ft*

C L _4 wV T 40* P40 * A 4ANNN Aft -0 'J; N "43 4 0 - 0 0 M N-4 F .4"a- ---- ---- m.9 *- 4 4 3 @ 0 -- N N N 9 9 9 ? ? 9 N

>1 U. I

S.. ci h51

A = -0.563 + 0.751B + 0.750D - 0.360E - 1.128H - 0.521S

+ 0.266T - 0.192EE + 0.711GG - 0.379XX - 0.245YY [77% accountability]

Table 22 shows the results when B (performance quality) is used as thedependent variable. After37 iterations the resulting model contains 13variables, with the prediction equation as

B = 0.646 + 0.188A + 0.651E + 2.376H + 0.409P + 0.115T - 0.121X

- 0.288Y - 0.012CC + 1.290DD + 0.213HH - 0.08111

- 0.177JJ - 0.269WW [85% accountability]

The equation provides a multiple R of 0.9225 and 85 percent of the variation inB is accounted for.

At an accountability level of approximately 77 percent, 11 variables arepresent in the prediction equation. Thus the inclusion of only two additionalvariables in this case increases the accountability from 77 to 85 percent.

Table 23 shows the results when the transformed variable ZZ = 0.4A + 0.6Bis used as the dependent variable. After 26 steps in the multiple regressionprogram, the prediction equation contains 10 variables.

ZZ = 1.173 + 0.939H + 2.6820 + 0.383P - 0.338R - 0.825S + 0.332T

+ I.lOlGG - 0.14511 + 0.193UU + 0.324VV [70% accountability]

The multiple R for the above equation is 0.8363 and 70 percent of the variationin ZZ is accounted for.

To expand the accountability to approximately 77 percent requires the 14variables shown below:

ZZ = 0.944 + 0.933H + 3.4010 + 0.324P - 0.365R - 0.705S + 0.255T

- 0.624U + 0.106W + 0.299AA + l.Ol6GG - 0.17611 + 0.070QQ

+ 0.114UU + 0.267VV [77% accountability]

Examination of the predicted values and residuals shows only one residualwith a standard deviation of 2, none with 3.

AGS Results. For the Aircraft Generation Squadron, data was collected from11 supervisors and for 41 technicians. Twenty-three of the technicians workedday shift and 18 worked swing shift. Five of the technician respondees werefemale. All of the technicians were military. Nine of the techniciin respondeessupervised others in some capacity. Three of the technicians evalueted by their

52

a. I

.-

09.

Lu 0.

ri

o 0-

-CA 00 0 -m

4 -01

v O.vO- 4 0NMO- 40 T -0

00 00 , n

x 00 4 -0 00 0 . '9 ,

C' ~ a O.CZ va3? 3 ~ 0N 4 f O N3

N0 .0 2AL. 3

I-c

C42 t - .N --------- 00 00..-

w3.-000000000000000000530a

z -

44 Ja.z

L h

o *

14.. 4-Z

l.A (A

U- +

r- 4 -NA-i

L4..rrp

S- -o

j 4mPNP-41.04NN0.-.SOAP-N.-

... . . .. .. ..

0"-)l

L ~ ~ ~ ~ ~ .M NN-0fN ~ -NoeC~

-Ja5

supervisors were "no-shows" for the maintenance technician survey. The computeroutput from the FORTRAN data processing program is in Appendix I.

AGS Statistical Results. Statistical distributions are shown in Table 24.Table 25 shows the results with A as the dependent variable. The programstopped after 48 steps with a model containing 40 of the 49 possible dependentvariables. Only variables F, H, K, L, M, P, AA, BB and WW did not enter theprediction equation, the equation providing a perfect (R = 1.000) prediction ofA.

With only 3 variables, the multiple R is 0.9003 and 81 percent of thevariation in A is accounted for. This equation is

A = -0.899 + 0.765B + 2.6900 - 0.105W [81% accountability]

By adding variables N, S, V, XX and YY, a 90 percent accountability for A isobtained.

A = -0.113 + 0.703B - l.115N + 3.4930 - 0.625S + 0.755V

- 0.188W - 0.l13XX - 0.228YY [90% accountability]

With B as the dependent variable, the BMDP program iterates through 43 stepsand produces a prediction equation containing 23 variables, while giving an Rof 0.9970 and a 99.4% accountability for B (Table 26).

With only 4 variables, an accountability of approximately 80% is obtained.

B = 1.785 + 0.851A + 0.515E - O.031L + 0.104W [80% accountability]

By adding variables 00, SS, V and J, an accountability of approximately90 percent is obtained.

B = 1.065 + 0.795A + 0.708E + 0.496J - 0.069L - 0.390V

+ 0.151W + 0.16800 - .U11SS [90% accountability]

In Table 27, results are shown when transformed variable ZZ 0.4A + 0.6Bis used as the dependent variable. After 30 stepwise iterations, the resultingprediction equation contains 12 predictor variables:

ZZ = -0.044 + 1.458C + 0.795D - 1.294E + 0.848J - 1.520N

- 0.535U - 0.546AA + 0.737DD + 0.440EE + 0.17900

- O.321RR + O.105TT [90% accountability]

55

W a in w4 VO4 NN 4ft 0 m ("ft4 ifN0P 0.0- Z,

An -S - - - - -- --N -

cc

ftWw UQS N QW .O = a 04

loo 4&o W O P4o o m

W. .. hlNO N~N..@VV~NO~9NN70N4 N' 4O. mw ffniolot -N b.4304.N40: C,0 rmS.00.Wf.Si~ 0 4 4 N V N 0~4

wo

(A 0

41 N 00 oooe00ooo0O000000000000000000000cooccoo0000000 cL- 0 wsa..000000000

000

0 0 0 0 0 000

00

0 0 0 0 0 0 0 0 0000

000

0 0 0 0 000

0 0 0c

oL o

S.. 4 4.OQO QOVOVOVO 0 0000000 0000W~0000000 00US1J

4-) .. .. . .. . .. .. .. ....... .......

00

M. O noOO OON o N ro m I ooZ0 00 00 OO 20 O 00 00 00 002 O GDN-00 -

.0 -0 *0000000 000 00 0 0 2 0 0 0 0 0 0 0 0 0 2 0 0 0

cc I'm N 000000 0 0 000cc 0000S 00MOW 000000000000000000CW000 C

(A 43r,

eaea

U.

-Nd

A n r - r 1N2..A a44 l - 0 :0A -3C42

P. W?9~ 4 N U 4N 7 4 A N T~ OCONl4.N

2 NN NNNN

'0* (56csJ

.c

a.J

oI-Ww z0

0 0

N Ow

a..

C~~~j <Z=m-4zo ~~ ~~~ A--0:fl N2 . . .4 N.~ A N4.0~.

4-) N N N N N N N NNN~

U' 4ea C

4. -4MN 0N O -NN;W ;; Vr tCP2: %Q; 00;

00 r0(V 0- cp V1 * 00 00t:*

0JM. rr rl 1 a 1.*,lo ",(00 0x rr t0 o n0. a cCOa .000 . .00 - a ,0 l ,C.0 P0

~ ~c

co

(A WIZ- 0-A-VVN.9~0mNN 0N0-4A N N A~(046~ N N.3

AL\j A I 0A .0 -9,2 AA,7WNVNCCAA N4 A40A - 0 -

I 42N AV.

Z A0

I-57

WA

o c

N Z

0 W-0S- 4-1 x

o0 r-I W.N N N N N N NIN N N

<L M O O

4

o~ U) 4-T0 i ti- V!4

-I - o ~ 0 O4N i~ 0 0 NOO ac=000 000O

o

a- 0. 0

41 N Wr (0NI M40.go ~ 3 O 0. wm " r 1 .4 *N000 0M0CONN

AA x

3n= 4;7r

0 N46*0 -r3. N00. r.O

CI;Na IO NozwNNNNN mmmmIr

A AI 3

3 3 ~ I z~,-58

C-i

N Z

- 4 2

4-)0 a

4) La

CD 4J. N0

a) .7j nN1o

U-)

tC)i a) I- T - 04N o Q e ; =

r-4 'a

C, .Ila0

ea

27

CID)

a--

z~ A

597

The above equation gives a multiple R = 0.9481 and 90 percent of the variationin ZZ is accounted for. For 77 percent accountability, the factors of E, N,U, AA, EE drop out but H is added.

An examination of residuals showed no cases with 2 or more standarddeviations.

Luke AFB Overall Results. Table 28 provides a summary comparison for thethree maintenance squadrons supporting the 405 th Tactical Wing at Luke AFB.Comparisons are made at an R2 level of approximately 0.77 (77 percent or higheraccountability). Similarities and differences between results for the threesquadrons may be observed below, when the transformed variable ZZ is the dependentvariable.

CRS: ZZ = -0.481 + 0.748C + 0.3030 + 0.298J - 0.017L + 0.446AA

- 0.005CC + 1.095GG [77% accountability]

EMS: ZZ = 0.944 + 0.933H + 3.4010 + 0.324P - 0.365R - 0.705S

+ 0.255T - 0.624U + 0.106W + 0.299AA + l.Ol6GG

- 0.17911 + 0.070QQ + 0.ll4UU + 0.267VV [77% accountability]

AGS: ZZ = -1.055 + 0.671C + 0.604D + 1.202H + 0.313J + 0.297DD

+ 0.13600 - 0.252RR + 0.l03TT [77% accountability]

A similar comparison may be made at an R2 comparable level of approximately0.90 (90 percent accountability). Again ZZ is the dependent variable.

CRS: ZZ = 2.369 + 0.805C - 0.0881 - 0.031K + 0.048M + 1.382Q

- 0.631R + 0.209T - 0.387U - 0.136V - l.015Y + 1.181AA

- 0.011CC + 0.483EE - 0.41OFF + 0.754GG + 0.06911

- 0.055RR + 0.278WW [92% accuracy]

EMS: Not Possible [Maximum R2 obtained = 80%]

AGS: ZZ = -0.044 + 1.456C + 0.795D - 1.294E + 0.848J - 1.520N

- 0.535U - 0.546AA + 0.7370D + 0.440EE + 0.17900

- 0.321RR + 0.I05TT [90% accountability]

60

L:. coO 00a~ 00 ~Lo Ltn 0D m4-) r" O~- -)0 ".rL LO) -Wc ( 0 0 cnqr C O 0 ION t0

4C CI m. .00Cjk

V.) 10 C.1*) - 0o r CLo cu . .

d-r 0- O enr- %D "

(3 . 0 tC)0\JI CD1. 0D C 1J

.- r jM-- LOC)U) LOlt- 0) k 10 0 -

0- -: C0 ")0-000 -c~ -O 0D Lo I"4-.C cmC 0. I I

LO Q) m .-ML .0a '" - ~ eI u") 0C CDC4 u) c CD ciwDw 0 0 ~Ln l m 0 ~ 1.0 - 0 C

4-* 4c C Ct" CtLl .O C14 00 0 kLL.- to C'1.C\ 0..- 0 14

U~ to we od e L)o o0 0 -O V)o MI N C

4) - 0n 1- "~ tnf (D 0t -l -N-I-

-u 4- 4)0a) Ca Q C ) C'" 1.0 1.0 C q C

C r_. C C~

4- 0 S: m w .. 00 0 (M" 0 r-.4- (r-) - 00 -rmC c 0

.0 t01 u Q 0.l* 0" Cc" s

co~+ C CU) 0D'-r LA) M U")im- 4C- ) c l rLO. Cf). 0 1.) 0 u")

t ) (-.) x M')C" M Cs U) '"

Cr4- fa0 4- m 0 O -. 0(A uc" or- ea 4") U")cl = C

a)- to" 4- Q)0.. 0L. - L 14- 0~C" c;J q ~ 0 0

o) 41I0 C) cc u + + I + I +

4) ul 41 u 4- c(at S- 1-

- L. o C. 0 4-' r-4 4-to04- =Ca 1. ., 4J Stvo =c 4J -c - P4

CL C- 1- s 0 t > (1 r-4IL

4)t 01 0> =1 -.C- , (L - (A4-> - U") *- 0 J 04J I- (L) .-

o (7' 4A >~ > - 1

o) C W (1 >~& C9-A to ate - u -. 10W

tAC cc L1.,- 1- c ~ 1. .1 U A4 -4

too E- 4J 0 1 4 m4A= -011 .0.I 0 0C- ., 41to> 0 aoea0.S- 0 4).J4 =1 .0 (414 35C W 4J --- +4 to I t

o (Au 4) 4 03( 4CL)0 t- t 4j " C- 4j E -4-J 0 C4- >0J4 C C V - 4

LL. A V 030 4 0 - L O Q dI- 0 C C Mt MC O.KIC-L J L~ 01 F1x < OCmL a - )c to

3C:*~~ ~ ~ U C5 C10j ()4 1

C C 1 0 7 4 U Lf1.r- O"0 Mc' N4 1. u-.-- - -I - 4 C

61

From the above it may be observed that to increase R2 from 0.77 to 0.90requires 11 more variables for CRS and 4 more variables for AGS. For the datacollected from EMS, the maximum multiple R2 is approximately 80 percent.Comparison in Table 28 is therefore made at the 77% accountability level. Nofactors surfaced as significant for all 3 squadrons. Factors C, D, H, J, AAand GG surfaced as significant in two of the three squadrons.

Differences in squadron technician responses relative to supervisorperformance ratings are clearly evident. Across the squadron, 23 of the48 dependent variables surfaced as significant predictors of performance at theR2 level of 0.77 (77 percent accountability).

In Table 29, results for CRS and AGS are compared at approximately the 90%level of accountability. We note that the eleven additional factors provide asomewhat better estimate of the mean ZZ performance score for CRS. The predictionerror at the mean is 0.08 percent with the 18 variables included as compared toa prediction error at the mean of 0.30 percent with 7 variables included. ForAGS, however, the addition of 4 variables worsens the prediction error of themean. It may also be noted in Table 30 that the factors of M (months incurrent assignment), T (hours on additional duties, U (number of service clubs),V (number of persons supervised), and WW (responsibility trait) contribute littleto the prediction of squadron performance ZZ.

Overall Results Across 82nd ATC Wing at Williams AFB and 40 5th TAC Wingat Luke AFB. A comparsion between the summary regression results in Table 15(82nd ATC) and the results in Table 28 (405th TAC) shows the following (based on77-78 percent accountability and ZZ as the dependent variable):

(1) Twenty independent variables (predictor factors) surfaced as significantfor one or both of the two maintenance squadrons of the 82nd ATC; 23independent variables surfaced as signficant for one or two of thethree maintenance squadrons at the 40 5 th TAC.

(2) Eleven of the same independent variables surfaced at the two differentWings, namely:

D. Persistence [3 squadrons - OMS, CRS, AGS]H. Work Shift [3 squadrons - FMS, EMS, AGS]J. Skill Level [4 squadrons - FMS, OMS, CRS, AGS]L. Equipment Assignment Year [2 squadrons - FMS, CRS]P. Rank [2 squadrons - FMS, EMS]R. Days Between Breaks [2 squadrons - OMS, EMS]S. Number of Additional Duties [2 squadrons = OMS, EMS]U. Number of Clubs [2 squadrons - FMS, EMS]

AA. Clearance for Remove/Replace [4 squadrons - FMS, OMS, CRS, EMS]GG. Technician Skill [4 squadrons - FMS, OMS, CRS, EMS]I. Homogeneity of Group Attitude [3 squadrons - FMS, OMS, EMS]

(3) Nine factors surfaced for the 82nd ATC that did not surface for the40 5th TAC, namely:

62

q

|U

Table 29

Comparison of Multiple Regression Equations for Luke 405 th TAC Wingfor Component Repair and Aircraft Generation Squadrons at

Approximately 90% Accountability(R2 = 0.90 or higher)

CRS AGSBeta Beta

Factor Coeff. Mean Contr. Coeff. Mean Contr.

1. C - Job Curiosity +0.805 3.328 2.679 +1.458 3.095 4.5132. D - Persistence +0.795 3.143 2.4993. E - Prof. Ident. -1.294 2.952 -3.8204. I - Enlist. Year -0.088 68.897 -6.0635. J - Skill Level +0.848 4.381 3.7156. K - Duty Assign. Year -0.031 67.328 -2.0877. M - Months Curr. Assign. +0.048 5.448 0.2628. N - Sex -1.520 0.786 -1.1959. Q - Hours/Shift +1.382 7.310 10.102

10. R - Days Between Breaks -0.631 4.466 -2.81811. T - Hrs. Add. Duties +0.209 0.862 0.18012. U - No. of Clubs -0.387 0.293 -0.113 -0535 0.238 -0.12713. V - No. Persons Supervised -0.136 1.190 -0.16214. Y - Clearance for Servicing -1.015 2.086 -2.11715. AA - Clearance for

Remove/Replace +1.181 2.035 2.403 -0.546 2.500 -1.36516. CC - Hours Between Equip.

Servicing -0.011 75.603 -0.83217. DD - Quality Tech. Infor. +0.737 3.429 2.52718. EE - Quality Test Equip. +0.483 3.138 1.516 +0.440 1.786 0.78619. FF - Technician Knowledge -0.410 2.983 -1.22320. GG - Tech. Skill +0.754 3.552 2.67821. II - Homogeneity of Group

Attitude +0.069 27.035 1.86522. 00 - Organization Warmth +0.179 25.167 4.50523. RR - Assign. Locality -0.055 20.996 -1.153 -0.321 24.000 -7.70424. TT - Social Status +0.105 24.905 2.61525. WW - Responsibility +0.278 0.569 0.158

ZZ Crossing 2.369 -0.044

ZZ Prediction 7.644 6.905

ZZ Actual Per Supervisory Evaluations 7.638 6.933

63

V

*E. Professional Identification [positive][shows for AGS at R2 .90]F. Organizational Identification [positive]G. Self Starter Trait [positive*N. Sex [negative contribution for males][shows for AGS at R2 .90]X. Weight Handled [positive]

*EE. Quality of Test Equipment [positive][shows for CRS and AGS atR2 = .90]

PP. Organizational Conflict [negative contribution]SS. Satisfaction with Pay and Benefits [negative contribution]XX. Emotional Stability [negative contribution]

(4) Twelve factors surfaced for the 40 5th TAC that did not surface for the82nd ATC, namely:

C. Job Curiosity [positive]0. Equipment Type [positive]T. Hours on Additional Duties [positive]W. Months in Supervision [positive]

CC. Hours Between Equipment Services [negative contribution]DD. Quality of technical information [positive]00. Organizational Warmth [positive]QQ. Organizational Identification [positive]RR. Assignment Locality [negative contribution]TT. Social Status [positive]UU. Fatigue Trait [positive--affecting performance negatively]VV. Ascendency Trait [negative contribution]

(5) The total number of factors which surfaced as significant in one orboth AF Wings is 32 of the 48 possible. The following factors includedin the questionnaires did not surface in the regression equations[for 77-78% accountability].

I. Enlistment YearK. Current Duty Assignment YearM. Months in Current Asisgnment*Q. Hours Per Shift*V. Number of Persons Supervised by Technician*Y. Clearance for Servicing Equipment

BB. Number of Internal Components*FF. Technician KnowledgeHH. Group Satisfaction of Individual MotivesJJ. Satisfaction with Interpersonal RelationsKK. Satisfaction with SupervisionLL. Organization StructureMM. Organization RewardsNN. Organization Risks*WW. ResponsibilityYY. Sociability

Some of the above factors did surface as significant predictors wheneither A or B alone were used as the dependent variable, namely:

*Factors show significance at Luke 40 5 th at 90% accountability level.

64

Y, V, Q, FF, LL, NN and WW. Also, at the 92% accountability levelfor ZZ with data from the CRS, Q, V, Y, FF and WW surfaced.

(6) It is of interest to observe which factors surfaced when (1)performance speed (A) was used as the dependent variable and when(2) performance quality (B) was used as the dependent variable. Table30 shows the comparison. Some of these variables did not show signi-ficance, however, when the transformed variable ZZ = O.4A + 0.6Bwas used as the dependent variable.

(7) Correlation and covariance analyses were also made from the BMDP outputdata. In addition normal and detrended normal probability plots wereobtained. For all of the survey da'-.a, the assumption of normalityis by-and-large supported by the data plots, with very few outliersPlots are essentially linear on the normal probability graphs.

Multicollinearity across independent variables is indicated by largecorrelation coefficients in the correlation matrix. One of theadvantages of the stepwise linear forward and backward multipleregression procedure is the dependent variables are withdrawn from theprediction equations that are correlated with one or more other variableswhich contribute a larger amount to the predicted quantity. Thus multi-collinearity is minimized with proper choice of the stepwise parameters.Remaining correlations in the matrices were mostly between + 0.2000,i.e., -0.2000 < R < +0.2000.

65

Table 30

Comparison of Significant Variables for Predicting PerformanceSpeed (A) vs. Predicting Performance Quality (B).

A and B are Excluded as Predictor Variables.

Predictor Dependent Variable Dependent VariableVariable is Performance is PerformanceSpeed (A) Quality (B)

C - Job Curiosity -- X(FMS, CRS)D - Persistence X(EMS) X(OMS, CRS)E - Prof. Ident. X(EMS) X(EMS, AGS)F - Organization Ident. X(FMS, OMS, CRS) X(OMS)G - Self Starter -- X(OMS)H - Work Shift X(FMS, EMS) X(EMS)I - Enlis. Year -J - Skill Level X(OMS, AGS)K - Duty Assignment Year -L - Equip. Assignment Year -- X(FMS, AGS)N - Sex X(FMS, AGS) X(OMS, CRS)0 - Equipment Type X(AGS) X(OMS, CRS)P - Rank X(FMS) X(OMS, EMS)Q - Hours/Shift X(OMS) X(OMS)R - Days Between Breaks X(OMS) X(OMS)S - No. Add. Duties X(FMS, EMS, AGS) --T - Hrs. Additional Duties X(EMS) X(EMS)U - No. Clubs X(OMS, CRS) --V - No. Persons Supervised X(CRS, AGS) X(FMS, AGS)W - Months Supervision X(AGS) X(AGS)X - Weight Handled X(FMS) X(OMS, EMS)Y - Clearance for Service X(FMS) X(EMS)

AA - Clear. Remove/Replace X(CRS)CC - Hrs. Between Servicing X(FMS, CRS) --DO - Quality Tech. Information -- X(CRS, EMS)EE - Quality Test Information X(FMS, EMS) X(OMS)FF - Technician Knowledge X(OMS) --GG - Technician Skill X(CRS, EMS) X(OMS)HH - Sat. Indiv. Motives -- X(EMS)II - Homogeneity of Attitude X(EMS)JJ - Sat. Interper. Rel. -- X(EMS)LL - Organization Structure X(OMS) --00 - Organization Warmth -- X(AGS)PP - Organization Conflict -- X(OMS)QQ - Oranization Identity X(OMS) --SS - Assign./Locality -- X(AGS)UU - Fatigue Trait X(OMS)WW - Responsibility -- X(EMS)XX - Emot. Stability X(CRS, EMS, AGS) X(OMS)YY - Sociability X(EMS, AGS)

66

q

PLANNED PUBLICATIONS

A summary article on this research is being prepared for submission eitherto the TRANSACTIONS of the American Institute of Industrial Engineers or HUMANFACTORS, the latter a publication of the Human Factors Society.

PROFESSIONAL PERSONNEL ASSOCIATED WITH RESEARCH

The principal researcher is Hewitt H. Young, P.E., Ph.D., a Professor ofEngineering at Arizona State University. Professor Young currently teaches with-in the Department of Industrial and Management Systems Engineering and directsteaching and research activities in human engineering. He holds the BSME andMSIE degrees from Case Institute of Technology and the Ph.D. degree (Engineering)from Arizona State University. His most recent publication was in HUMAN FACTORS,1979, 21(4), p. 399-407 in an article entitled, "The Impact of Environment on theProductivity Attitudes of Intellectually Challenged Office Workers." ProfessorYoung was an ASEE/USAF Summer Faculty Fellow in 1978 assigned to the AF HumanResources Laboratory, Advanced Systems Division, Wright-Patterson AFB, Ohio.This current research is an outgrowth of research begun during the summer 1978period.

The 1978 summer study was supervised by Mr. Robert C. Johnson, researchpsychologist for AFHRL/ASR. Mr. Johnson has continued his active interest inthis research area by serving as program monitor for the current research pro-ject. He was particularly helpful in getting Air Force clearance for the studyand the effort is much appreciated.

Two graduate students at Arizona State University were also involved incertain aspects of the research effort. Captain Joel R. Hickman, an AFIT graduatestudent, considered the problem of developing suitable supervisor performancerating scales for technicians as an engineering report effort. His work resultedin the scales for supervisory ratings of technician speed and quality of per-formance which were used in the research. Captain Hickman received his B.S. degreefrom the University of California at Los Angeles. He is a career Air Forceofficer who has now completed the MS degree with industrial engineering majorfrom ASU. Parts of Mr. Hickman's master's engineering report are included inthis report. A complete copy of his engineers report can be obtained at repro-duction costs from the Department of Industrial and Management Systems Engineering,ASU. Captain Hickman was of invaluable help to the principal researcher becauseof his knowledge of Air Force organizations and procedures. He also helped withthe data collection at Williams AFB. His current assignment is at K.I. SawyerAFB, Michigan.

The second graduate student involved in the project is Mr. Mark J. Bramlett.Mr. Bramlett helped the author in the conduct of the survey at Luke AFB and withsome of the analytic work. He holds B.S. and M.S. degrees from Oklahoma StateUniversity and is pursuing the Ph.D. degree at ASU. He also holds a position asgraduate research assistant.

67

INTERACTIONS WITH AIR FORCE ORGANIZATIONS

The research involved extensive surveys of maintenance technicians and theirimmediate supervisors at Williams AFB and Luke AFB, Arizona. The survey instru-ments were developed during the early phases of the project and then reviewed forcontent with AFHRL/ASR. Mr. Johnson and his staff provided valuable inputs forsurvey instrument modification. The modified survey instruments were then sub-mitted to AFMPC/DPMYPS (Randolph AFB) for approval by Dr. Gordon A. Eckstrand,Technical Director for ASR Laboratory.

Prior to operational activation of the project, letters were addressed toHeadquarters, Tactical Air Command and Headquarters, Air Training Command throughLt. Colonel James A Cline, USAF, Chief, Advanced Systems Division. These lettersrequested the assistance and cooperation of the Commands relative to conduct ofthe surveys at Williams and Luke AFB's. A request was also directed to the DeputyCommander Maintenance at each of the two AFB's for local support of the project.As part of the survey review procedure at AFMPC/DPMYPS, Captain David Gambrellrequested letters from Williams and Luke AFB's indicating agreement to partici-pate in the surveys, having reviewed the survey instruments. Clearance was alsorequested from the two Chief Base Personnel officers, since both military airmenand civilian employees were to be included in the sample. At Williams AFB,meetings were held with Lt. Colonel Lanier (CBPO), Lt. Ford of the personneloffice, Mr. Standquist (CCP) in charge of civilian personnel, and local Unionrepresentatives. At Luke AFB, meetings were held with Major Love (CBPO), andMr. Cody (CCP). Cooperation was excellent in getting approvals, which were thenforwarded to AFMPC/DPMYPS at Randolph. The researcher also directed a letter toMr. Galloway of AF/MPKE, Pentagon, to expedite national clearance for the use ofcivilian employees in the survey, and he in turn provided an authorization letterto AFMPC/DPMYPS.

Approval for conducting the surveys was received from AFMPC/DPMYPS in earlyOctober, 1979, and arrangements were initiated to conduct the surveys throughColonel Thomas E. Walker, Deputy Commander Maintenance for the Williams AFB 82ndwing, and Colonel James W. Vorhees, Deputy Commander Maintenance for the LukeAFB 405th Wing. The study was conducted first at Williams AFB, with the invalu-able help of Captain Jerry Raney of the DCM's staff. Captain Raney devoted agreat deal of time to working with this writer and Captain Joel Hickman in theselection of the sample, providing a suitable classroom for the surveys, followingup to see that proper notifications were given to personnel and soliciting attend-ance, and personally attending all of the survey sessions over a two-week period.The researchers also met with Colonel William J. Breckner, Jr., who is a recentincumbent as 82nd Wing Commander, to explain the purpose of the study and receivehis authorization.

At the survey orientation session at Luke AFB, Captain David Bump representedHQTAC and Captain Bob Tilton was assigned by the DCM to work with the researcherson the project. Colonel Charles A. Horner, Wing Commander for the 405th, providedauthorization for the study. As at Williams AFB, Captain Tilton took on theimportant tasks of helping with the sample selection, providing a classroom forthe surveys, getting supervisors and technicians to the survey sessions, andpersonally attending some of the sessions.

The researcher wishes to thank all of those mentio( 1 above, the maintenancesquadron commanders and supervisors at both Air Force L ses, and the many main-tenance technicians who participated in the study for their courteous and valuablecontributions to this research effort. Letters of appreciation were sent at anearlier date to Colonel Breckner and Colonel Horner.

68

CONCLUSIONS AND RECOMMENDATIONS

The proposed Air Force Maintenance Squadron Performance Prediction Model,which will be labeled AFMSPPM and consists of a variety of survey inputs,provides a very good estimate of a squadron's performance level based on resultsof this research study at Williams and Luke Air Force Bases. The multifactormodel, based on survey inputs from a sampling of maintenance technicians andtheir immediate (shift) supervisors within a squadron, will predict a squadronperformance level with a 78 to 92 percent accountability when compared to theaverage of supervisor performance ratings for the same sample of maintenancetechnicians. In addition, the survey responses of any one technician, when usedas inputs in the AFMSPPM, will provide a good prediction (correlate well) withthe supervisor's performance rating for that technician. In this latter case,the coefficient weightings used with the factor responses by a technician aredetermined across the squadron sample.

The squadron survey model consists of

(1) a representative sample of technicians responses to 150 survey questions,covering 138 items and 18 factors,

(2) selected biographical data on each of the sampled technicians

(3) immediate (shift) supervisor inputs on motivation traits for each ofthe sampled technicians, and

(4) immediate supervisor inputs on equipment and environmental factorswhich may influence each sampled technician's work.

In the research study performance ratings were made, for each of the techni-cians sampled, of ipeed and quality of work. These two performance ratings werefirst independently used as dependent measures in the multiple regression analysesand were then combined into a single performance measure such that speed ofwork was weighted by 0.4 and quality of work by 0.6. Based on limited inputsto the researcher, these weightings appear to be representative of maintenancecommander (DCM) feelings in a peacetime situation. In time of war, speed ofwork would probably be weighted more heavily.

In the multiple regression analyses, differences in the factors which surfacedas significant for different squadrons may be observed. These differences maybe attributed to

(1) particular aircraft equipment serviced by a squadron,

(2) organization, construct and activities of the particular maintenancesquadron,

(3) conditions within the squadron at the time of the survey which mayinfluence technician responses,

(4) personal attitudes and traits of the technicians involved, and

(5) extent to which an averaging of supervisor performance ratings of asample of technicians within a squadron actually represent a squadronperformance level.

69

For each particular maintenance squadron, the factors and weightingswhich surfaced in the multiple regression analyses, when combined in theprediction model, gave an excellent prediction of averaged supervisor performanceratings. Thus the survey model can be employed as a substitute for supervisorperformance ratings and has the distinct advantages of (1) involvingthe technicians themselves, through a sampling process, in rating squadron per-formance and (2) highlighting those particular factors or conditions which arecontributing either positively or negatively to the squadron performance level.Such input provides squadron and wing management with valuable information forimprovement and control.

Most of the factors included in the developed AFMSPPM have shown significanceas predictors of maintenance squadron performance in at least one of the fivesquadrons studied. Several of the factors were significant in two or three ofthe squadrons. At this juncture, therefore, no portion of the survey model shouldbe eliminated. However, additional studies at other Air Force bases and for othersquadrons may well suggest that the inclusion of survey questions in support ofcertain factors is not worthwhile. The general survey model could then bereduced and require less survey input and time.

Based on differences observed to date in applying the AFMSPPM, it wouldappear that only certain portions of the survey model might be used for particulartypes of squadrons and/or particular equipment types serviced. That is, allField Maintenance squadron's in the Air Force may have sufficient similaritiessuch that some factors in the AFMSPPM would often prove significant and otherfactors would seldom or never prove significant. Likewise, Air Force maintenancesquadrons servicing a particular type of aircraft may have particular characteri-stics which would emphasize certain factors in the model and deemphasize others.Additional research is needed at other Air Force Bases and for a variety ofmaintenance squadrons to clarify whether either of the above is true.

In conclusion, this research has been a limited effort to (1) develop acomprehensive survey instrument for use with maintenance technicians and theirimmediate supervisors, combining factors (independent variables) which have shownsignificance in previous maintenance performance studies, such that the combinedresponses for a squadron would be predictive of a squadron performance level, and(2) by means of survey data input to a stepwise, linear multiple regression modelto generate for a particular squadron a prediction model, which would indicateboth squadron performance level and the factors which were contributing to suchperformance level at a point of time. The results across two Air Force Wings,at two AF bases and involving a total of 5 maintenance squadrons, are veryencouraging. The study should be extended to other AF bases.

The AFMSPPM eventually offers the promise, probably in a somewhat reducedform, of

(1) permitting an AF maintenance squadron or Wing to periodically assessits overall performance level by direct input from the techniciansand shift supervisors,

(2) permitting an AF maintenance squadron or Wing to assess the performanceof each technician in a squadron against squadron performance, basedsolely on inputs from the technicians themselves and selected super-visory inputs on equipment, environment and technician motivation traits,

70

(3) establishing a performance predictive model for each squadron typeand/or each equipment type,

(4) permitting a squadron to compare model inputs at two or more pointsof time and ascertaining those factors which are increasing ordecreasing in their contributions towards effective squadron perfor-mance, and

(5) establishing a means for accurate life-cycle costing of maintenancetechnician activity for a particular type of end item equipment,placed at particular AF bases, and supported by given maintenancesquadrons.

71

REFERENCES

Askren, W. B., W. B. Campbell, D. J. Seifert, T. J. Hall, R. C. Johnson, andR. H. Sulzen Feasibility of a Computer Simulation Method for EvaluatingHuman Effects on Nuclear Systems Safety, AFHRL-TR-76-18, AFWL-TR-76-15, WPAFB,1976.

Barrett, Richard S., Performance Rating. Chicago: Science Research Associates,1966.

Callander, Bruce, "Hikes: 1 out of 3 Makes E-7." Air Force Times, July 2, 1979,p. 4.

Cummings, L. L., and D. P. Schwab, Performance in Organizations. Glenview,Illinois: Scott, Foresman, 1973.

Dunnett, M. D., "A Note on the Criterion." Journal of Applied Psychology, 1963,41, 251-254.

Foley, John P., Evaluating Maintenance Performance: An Analysis. AFHRL-TR-75-57(l),AD-053-475. Wright-Patterson AFB, Ohio: Advanced Systems Division, AirForce Human Resources Laboratory, December 1974.

Glaser, R., and A. J. Nitko, "Measurement in Learning and Instruction." In R. L.Thorndike (Ed.), Educational Measurement. Washington: American Councilon Education, 1971, 625-670.

Guilford, Joy P., Psychometric Methods. New York: McGraw-Hill, 1954.

Guion, Robert M., Personnel Testing. New York: McGraw-Hill, 1965.

Hickman, Joel R., Air Force Maintenance Technician Performance Measurement,Unpublished Master of Science Engineering Report, Arizona State University,August 1979.

Lawler, :. E., III, "The Multitrait-Multirater Approach to Measuring ManagerialJob Performance." Journal of Applied Psychology, 1967, 51, 369-381.

_,er. Alan H. and K. S. Teel, "Performance Appraisal - A Survey of Currenttices.' Personnel Journal, 1977, 56, 245-247.

aA-e. A.,"Distaff Mechanics Doing OK." Air Force Magazine, 1979,

r. ,,, _ok at Performance Appraisal." Harvard Business

t.. '~iv's'ip Between System Design,- ' .. ,-ra~ce ,) rwo Autopilot

4r4 oht-Patterson AFB,

,,-++.i -***. , aboratory,

Li

Sauer, D. W., W. B. Campbell, and N. R. Potter, Human Resource Factors andPerformance Relationship in Nuclear Missile Handling Tasks. AFHTL-TR-76-85,AFWL-TR-76-30. Wright-Patterson AFB, Ohio: Advanced Systems Division,Air Force Human Resources Laboratory, 1977.

Swezey, R. W., and R. B. Pearlstein, "Developing Criterion-Referenced Tests."

JSAS Catalog of Selected Documents in F ychology, 1975, 5, 227.

University of California BMDP 77, Series P.

Wiley, L. N., Task Level Job Performance Criteria Development. AFHRL-TR-77-75,AD-055 694. Brooks AFB, Texas: Occupational and Manpower Research Division,Air Force Human Resources Laboratory, December 1978.

Young, H. H., Performance Effectiveness in the Air Force Maintenance System:Preliminary Report and Design Report. Wright-Patterson AFB, Ohio: AdvancedSystems Division, Air Force Human Resources Laboratory, 1978. (InternalReport)

BIBLIOGRAPHY

Alexander, R. A. (and others), Editors, Relationship Among Measures of WorkOrientation, Job Attribute Preferences, Personality Measures, and Abilities,Technical Report #7, ONR Contract NOOOI4-74-A-0202-OOl, August 1975.

Arbiet, F. P., and Scheafer, H. A., A Study of Physical Demands Placed on PersonnelAssigned to an Air Force Maintenance Specialty, AFIT Master's Thesis LSSR-37-77B, 1977.

Argyris, C., Personality and Organization, Harper and Brothers, 1957.

Barlow, E., Annotated Bibliography of AFHRL Technical Reports, 1968-75, AFHRL-TR-76-50; Annotated Bibliography of AFHRL Technical Reports 1976, AFHRL-TR-78-1,1978.

Barrett, G. V., Forbes, J. B., Alexander, R. A., O'Connor, E. J., and Balascoe,The Relationship Between Individual Attributes and Job Design: MaintenanceTasks, Technical Report No. 5, ONR Contract N 00014-75-C-0985, NR 151-377.March 19, 1977.

Batezel, A. L., "Getting What Your Recruiter Promised, "Air Force Maoazine, June 1978.

Bayroff, A. G., H. R. Haggerty, and E. A. Rundquist, "Validity of Ratings as Relatedto Rating Techniques and Conditions." Personnel Psychology, 1954, 7, 93-112.

Bendig, A. W., "Reliability and the Number of Rating Scale Categories." Journal ofApplied Psychology, 1954, 38, 38-40

Beu, N. J. and R. C. Nichols, More Maintenance in OMS, Air University Report, MaxwellAFB, ADB 019707, 1977.

Bittner, R., "Developing an Industrial Merit Rating Procedure." Personnel Psychology,1948, 1, 403-432.

Blood, M. R., "Intergroup Comparisons of Intraperson Differences: Rewards from theJob, "Personnel Psychology, 1973, 26, 1-9.

Breglio, R. A. and R. F. Wright, Cost Estimating Relationships for Aircraft Jet-CoreEngine Overhaul Costs, AFIT Master's Thesis LSSR 3I-77B, WPAFB, 1977.

Brogden, H. E., and E. K. Taylor, "The Theory and Classification of Criterion Bias."Educational and Psychological Measurement, 1950, 10, 159-186.

Campbell, J. P., and others. Managerial Behavior, Performance and Effectiveness.New York: McGraw-Hill, 1970.

Campbell, J. T., E. P. Prien, and L. G. Brailey. "Predicting Performance Evaluations"Personnel Psycholocty, 1960, 13, 435-440.

Carpenter-Uffman, P. and B. Rostker, The Relevance of Trainino for the Maintenanceof Advanced Avionics, Rand Corporation Report R-1894-AF, AD 047707, 1976.

74

Cascio, W. F., Value Orientation, Oroanizational Rewards and Job Satisfaction,Unpublished PhD Dissertation, University of Rochester, 1973.

Chenzoff, A., Evaluative Study of Content and Display of New and Existinq Tech Datato Support AF Maintenance, AFHRL-TR-73-47, 1973.

Connelly, E. M., Study of Adaptive Math Models for Derivinq Automated Pilot PerformanceMeasurement Techniques, ,^FHRL-TR-69-7, 1969.

Cowell, P. H., and Jones, F. M., Jr., Analysis of Cost Center Performance Measure-ment System, AFIT Master's Thesis LSSR-2-77A, 1977.

Cowell, P. A. and F.M. Jones, A Cost Center Performance Measurement System, AFIT

Master's Thesis LSSR 02-77A, WPAFB, 1977.

Critics, C. D., Video Event Reporting, AFHRL-TR-69-16, 17, 18, 1969.

Cureton, E. E., and Sargent, B. B., Factor-Analytic Reanalysis of Studies of JobSatisfaction and Morale, WADD-TN-6D-138, July 1960.

Drake, K. L., M. S. Sanders, W. H. Crooks, and G. Weltman, Comparative Studies ofOrganizational Factors in Military Maintenance, Perceptronics CorporationReport PTR-1043-77-10, ADA 050403, 1977.

Fischer, C. D. and Pritchard, R. D., Effects of Personal Control, ExtrinisicRewards and Competence on Intrinisic Motivation, AFHRL-TR-78-20, 1978.

Flanagan, J. C., "The Critical Incident Technique." Psychological Bulletin, 1954,51, 327-358.

Foley, J. P., Jr., Hard Data Sources Concernina More Cost Effective Maintenance,AFHRL-TR-76-58, 1976; Executive Summary on Impact of Advanced MaintenanceData and Training Technologies on Maintenance Personnel and Trainino Systems,AFHRL-TR-78-24, 1978.

Foley, J. P. Jr., Performance Measurement of Maintenance, AFHRL-TR-77-76, WPAFB, 1977.

Foley, J. P. Jr., Some Key Problems Concerning the Specification, Development and

Use of Task Identification and Analyses, AFHRL-TR-76-57, 1976.

Gant, C. R., Economics of On-The-Job Training, AFHRL-TR-76-83, 1976.

Guinn, N., and Truax, S. R., Comparison of Volunteer Attitudes and Career MotivationAmong Officer and Airman Personnel, AFHRL-TR-73-28, Lakeland AFB, 1973.

Guthrie, W. J., A Study of Job Characteristics and Job Attitudes at a Tactical AirCommand Fiahter Complex, Master's Thesis, School of Engineering, AFIT/GSM/SM775-5, ADA 045998, 1977.

Habbe, S., "Marks of A Good Worker." Management Record, 1956, 18, 168-170.

Hendrix, W. H., and Ward, J. H. Jr., Preferred Job Assignment Effect on Job SatisfactioAFHRL-TR-75-77, Lakeland AFB, 1977.

Herzberg, F., B. Mausner and B. Synderman, The Motivation to Work, John Wiley andSons, 1959.

75

Hollingworth, H. L., Judging Human Character. New York: Appleton, 1922.

Holly, W. H., H. S. Field, and N. J. Barnett. "Analyzing Performance AppraisalSystems: An Empirical Study." Personnel Journal, 1976, 55, 457-459.

Homich, G. H., Deoot Maintenance Overhaul Costs for Aircraft Engines, AFIT Master'sThesis LSSR 3-71B, 1971, WPAFB.

Howard, E. F., and Lipsett, H. G., Naval Sea Systems Operational AvailabilityQuantification and Enhancement, Naval Underwater Systems Center TechnicalReport 5211, February 1976.

Hubbard, C. D., and Lindke, C. R., The Identification of a Precedence NetworkAssociated with Base Level Aircraft Maintenance, AFIT Thesis SLSR-12-75B, 1975.

Johnson, R. C., Thomas, D. L,, and Martin, D. J., User Acceotance and Usability ofthe C-141 Job Guide Technical Order System, AFHRL-TR-77-31, WPAFB, 1977.

Jones, L. V., and L. L. Thurstone, "They Psychophysics of Semantics: An ExperimentalInvestigation." Journal of Applied Psychology, 1955, 39, 31-36.

LaRue, R. D. and G. D. Metzinger, Motivation of Maintenance Personnel to WorkParticular Shifts, Master's Thesis, School of Logistics Management AFIT/SLSR15-77A, ADB 019707, 1977.

Lewin, Arie Y., and Zwany, A. "A Model Literature Critiqueand a Paradigm For Research." Personnel Psychology, 1976,29, 423-447.

Lintz, L. M., Relationship Between Design Characteristics of Avionic Systems andTraining Costs, Maintenance Performance, AFHRL-TR-72-20, 1972.

Lintz, L. M., Loy, S. L., Brock, G. R., and Potempa, K. W., Predicting MaintenanceTask Difficulty and Personnel Skill Requirements Based on Desian Parametersof Avionics Systems, AFHRL-TR-72-75, 1972.

Lopez, Felix M. Evaluating Employee Performance. Chicago:Public Personnel Association, 1968.

McCarty, D. S., and Moore, R. L., Aircraft Maintenance Cost Elements, AFIT Master's

Thesis LSSR-17-77B, ADA047640, 1976.

McGregor, D. The Human Side of Enterprise, McGraw Hill, 1968.

Meister, D., D. L. Finley, and E. A. Thompson, Relationship Between System Design,Technician Training, and Maintenance Job Performance on Two Autopilot Subsystems,Bunker-Ramo Corporation Report AFHRL-TR-70-20, WPAFB, 1971.

Millard, Cheedle W., F. Luthans, and R.L. Otteman. " A NewBreakthrough for Performance Appraisal." Business Horizons1976, 19, 66-73.

Miner, J. "Management by Appraisal: A Capsule Review andCurrent References." Business Horizons, 1968, 11, 83-94.

76

Naylor, V. and Briggs, G., Long-Term Retention of Learned Skills: Review of Literature,ASD-TR-61-390, WPAFB, August 1961.

Obradovic, J. "Modification of the Forced Choice Method asa Criterion of Job Proficiency." Journal of AppliedPsychology., 1970, 54, 228-233.

O'Connor, E. J. (and others), Editors, Organizational Policy Decisions as a Function

of Individual Differences and Task Desion: Maintenance Tasks, Office of NavalResearch Contract N 00014-75-C-0985, WR 151-377, Washington D.C., March 1977.

Potter, N. R., Thomas, D. L., Evaluation of Three Types of Technical Data forTroubleshooting: Volume I - Project Results and Summary AFHRL-TR-76-74(I), 1976.

Pritchard, R. D. and R. V. Montagno, Effects of Specific vs Non-specific and Absolutevs Comparative Feedback on Performance and Satisfaction, AFHRL-TR-78-12, WPAFB,1978.

Ronan, W. W., C. L. Anderson, and T.L. Talbert. "A PsychometricApproach to Job Performance: Fire Fighters." PublicPersonnel Management, 1976, 5, 409-422.

Saleh, S. D., "A Study of Attitude Change in the Preretirement Period", Journal of

Applied Psychology, 1964, 48, 310-312.

Sauer, D. W., Campbell, W. B., Potter, N. R., and Askren, W. B., Human Resource

Factors and Performance Relationships in Nuclear Missile Handling Tasks,

AFHRL-TR-76-85, AFWL-TR-76-301, WPAFB, 1977.

Seniawski, C. D., and Seres, F. E., An Analysis of Resource Manaoement in the WingAircraft Maintenance Orqanization, AFIT Master's Thesis, SLSR-l-73B, WPAFB,

1973.

Schumacher, S. P., Pearlstein, R. P., and Martin, P. W., A Comprehensive Key Word

Index and Bibliography on Instructional System Development, AFHRL-TR-74-14,WPAFB, 1974.

Shriver, E. L., Video Approach to Evaluating Maintenance Performance, AFHRL-TR-

74-57, 1974.

Shriver, E. L., and Foley, J. Jr., Evaluatino Maintenance Performance: Electronic

Maintenance, AFHRL-TR-74-57, 1974.

Stevens, S.N.,andE,F. Wonderlic. "An Effective Revision ofthe Rating Technique." Personnel Journal, 1934, 13, 125-134.

Taylor, E.K., R.S. Barrett, J.W. Parker, and L. Martens."Rating Scale Content: II. Effect of Rating on IndividualScales." Personnel Psychology, 1958, 11, 519-533.

Taylor, E.K., and Grace E. Manson. "Supervised Ratings;Making Graphic Scales Work." Personnel, 1951, 27, 504-514.

Thornton, G. "The Relationship Between Supervisory and Self-Appraisal of Executive Performance." Personnel Psychology,1968, 21, 441-455.

Travers, R.M. "A Critical Review of the Validity and Rationaleof the Forced Choice Technique." Psychological Bulletin,1951, 48, 62-70.

77

Uhrbrock, R.S. "2000 Scaled Items." Personnel Psychology,1961, 14, 375-420.

Umstot, D. D., Bell, C. H., and Mitchell, T. R., "Effects of Job Enrichment ar'

Task Goals on Satisfaction and Productivity," Journal of Applied Psychr76, 1, 379-394.

U.S. Air Force. Air Force Manual (AFM) 66-1, Maintenance

Management. 1O vols. Washington: Government Printing

Office, 1975 and 1977.

Valentine, L. D. Jr., Prediction of AF Technical Trainino Success from ASVAB andEducational Background, AFHRL-TR-77-18, Laxeland AF8.

Vanzelst, R., and W. Kerr. "Workers Attitude Toward MeritRating." Personnel Psychology, 1953, 6, 159-172.

Vineberg, R., and Taylor, E., Performance of Men in Different Mental Cateaories;Development of Worker-Oriented and Job-Oriented Ratino Instruments in Navv

Jobs, HUMRRO Tech Report 76-1, October 1976.

Wanous, J. P., "Individual Differences and Reactions to Job Characteristics,"Journal of Applied Psycholoqy, 74, 59, 616-622.

Watson, T. W., and Zumbro, P. A., Job Enrichment: Evaluation With Imnlications fo-

AF Job Design, AFHRL-TR-77-56, 1977.

Whitla, Dean K., and John E. Tirrell. "The Validity ofRatings of Several Levels of Supervisors." PersonnelPsychology, 1953, 6, 461-466.

Wikstrom, W.S. Managing by and with Objectives. NationalIndustrial Conference Board, Personnel Study No. 212, 1968.

Wiley, L. N., Comparing Prediction of Job Performance Ratings from Trait Ratinasfor Aircraft Mechanics, AFHRL-TR-68-108, 1968.

Wiley, L. L. and C. P. Hahn, Task Level Performance Criteria Development, AFHRL-TR-77-75, WPAFB, 1977.

Wollack, S., Goodale, J. G., Wijting, J. P., and Smith, P. C., "Development of theSurvey of Work Values," Journal of Applied Psychology, 1971, 55, 331-338.

78

APPENDICES

c

I

APPENDIC A

Summary of Important Findings from the Literature Reviev

Conducted at Wright-Patterson AFB in Summer 1978.

Som e

Findings of Past Research

1. Maintenance performance measures are significantly affected by

hardware system desgin.

2. For jet engine depot maintenance (turbofans and turbojets), over-hall costs are significantly affected by engine design and operating

parameters and average flight hours between engine overhauls.

3. 22% of the variance in time to perform maintenance on autopilot

avionics can be attributed to the design of test equipment.

4. PROMIS job selection provides a small increase in job interest and

in airman felt utilization of talents and training where assignmentis made to first choice at time of enlistment.

5. There is a strong interaction between measured individual differences

(and attitudes) and maintenance task performance.

6. Technician skills, knowledge and overall capability are primarilya function of training. Formalized on-job training is more effectivethan classroom/theoretical training.

7. Measured abilities and values of technicians are useful in accer-taining what particular task charactersitics will appeal to certain

persons.

B. ASVAB test grades and educational backgrounds of airmen are highlycorrelated with final school grades (training center).

9. Long-term recall of learned skills is a function of the type of task,learning parameters, retention interval parameters and recall parameters.

10. Persons differ widely in their reactions to location and work environ-

ment. Fears can be diminished with familiarity gained throughtraining and experience.

11. Management decisions at the Wing level can significantly affect

sypply costs.

-- mII ii4

12. Lack of spares is not a serious deterrant to maintenance performance

effectiveness.

13. The Cost Center Performance Measurement System (CCPMS) has not

been well received by the Commands, since the output measures are

not considered particularly useful. Further, for similar output

measure values across units, the actual performance costs vary widely.

14. Style and quality of supervision have highly significant affects on

technician performance and job satisfaction.

15. Supervisory conditioning of maintenance tasks has an important

impact on technician job satisfaction.

16. Important airman satisfiers, in addition to the quality of supervision,

include:

job security

opportunity for technical training

accomplishing work that gives a feeling of achievement

17. Job Guide improvements which incorporate more illustrations, logic

and procedural charts, completeness of information, clarity and

dual-level of presentation are well-received by maintenance per-

sonnel and can significantly affect both performance time and per-

formance accuracy.

18. Human errors by maintenance personnel in the Air Force can be cate-

gorized as:

40% - failure to follow procedures

10% - incorrect diagnosis

I0% - misinterpretation of communication

20% - inadequate support, tools, test equipment, environment

20% - insufficient attention or caution

19. Environmental temperature and fatigue characteristics both affect

performance capability.

IO

.'4.

20. Other conditions which may impact upon technician performance

include:

Organizational climate, including team cohesive-

ness, structure, warmth, etc.

Personal traits of technician, including job

curiosity, persistence, etc.

Assignment location

Impact of Pay and Benefits

Social Status of Occupation

21. Maintenance airmen have a lower degree of job satisfaciton than

for other AFSC's.

22. Reducing or eliminating present disincentives for maintenance

personnel probably is more cost effective than to add new incentives.

23. Adequate feedback on performance is needed for both airmen and

supervisors.

IQ

APPENDIX B

Survey Instruments Used in Study

biII

I

IL

It

ii . .. .

MAINTENANCE TECHNICIAN SURVEY

The attached survey is part of a research effort being conductedby Arizona State University under contract with the Air Force Officeof Scientific Research, and with the cooperation of the Air Force HumanResources Laboratory, Advanced Systems Division, WPAFB, Ohio. Thepurpose of the survey is to further identify factors which influenceperformance effectiveness in maintaining Air Force aircraft and missilesystems.

Your participation in the Survey is voluntary but stronglydesired. Your responses will be held confidential and in no way willimpact upon your career nor upon the squadron to which you are assigned.Headquarters USAF Survey Control Number 80-11 has been assignedto this survey.

The value of this research effort is dependent upon the effortyou make to provide open, honest responses to each question. Pleaseturn the page and read the Privacy Act Statement and Instructionsbefore proceeding. Thank you for your valued cooperation.

-A088 061 ARIZONA STATE UNIV TEMPE DEPT OF INDUSTRIAL AND MANA-,-ET¢ Ffs 19/5

- G EVELOPNENT OF AN EFFECTIVENESS PLANNING ANO EVALUATION POOEL F--ETC(U)APR 60 H H YOUNG AFOSR-79-OII

UNCLASSIFIIO ASU-ERC c-na0| AFOS-T R-o0-0o NL

1"" II

PRIVACY STATEMENT

Survey of Air Force Maintenance Technicians

a. Authority:

(1) 5 U.S.C. 301, Departmental Regulations; and/or(2) 10 U.S.C. 8012, Secretary of the Air Force, Powers Duties,

Delegation by Compensation; and/or(3) DOD nstruction 1100.13, 17 Apr. 68, Surveys of Department

of Defense Personnel, and/or(4) AFR30-Z3, 23 Sep 76, Air Force Personnel Survey Program

b. Principal purposes: To collect information from Air Force andcivilian squadron maintenance personnel concerning their per-ceptions of factors which influence their performance effective-ness. To initiate the development of an Air Force MaintenancePerformance Effectiveness Model based on the survey results andother inputs.

c. Routine Uses: Data will be used for research purposes ininitiating a predictive model of maintenance performanceeffectiveness.

d. Participation is voluntary. However, your cooperation isrequested.

e. No adverse action of any kind may be taken against any individualwho elects not to participate in any or all of this survey.Please return the survey booklet to the Project Monitor ifyou wish to withdraw.

INSTRUCTIONS

Your responses to the survey questions are to be machine scored byOptScan technology. Accordingly, all answers are to be marked onthe separate IBM answer sheet loosely inserted in this survey booklet.Remove the answer sheet from the booklet and use it to record youranswers by blackening in the appropriate answer rectangle, markedA,B,CD and E, for each question. Cleanly erase answers you want tochange. DO NOT make stray pencil marks on the sheet. Check to besure that only ONE answer rectangle is blackened for each question.Use only the No. 2 pencil provided or a comparable pencil withmedium soft lead.

For control purposes, an identification number has been preassignedto each survey booklet and appears in a large block on the bottomof the next page. This five-digit identification number should bemarked onto the first five rows of the coding section entitledIDENTIFICATION NUMBER at the top right of the answer sheet. Pleasemake sure that the number is correctly reproduced for machine scoring.Ignore all other heading information.

Note that the answer sheet treats question numbers from left to right,with questions 1 to 4 on the top row. Be sure to use the correctanswer block for each question, by question number.

Now turn the page and complete the Biographical Data before beginningto respond to the questions.

Later, after you have finished responding to the survey questions,reinsert the answer sheet in the survey booklet and turn both intogether to the project monitor. Be sure that you have providedyour Identification Number on the answer sheet. Thank you.

2BZOGRAPHIC INFORMATION

(Please complete before responding to survey questions)

gate of enlistment or employment by USAF I

List your primary AFSC O.Nt AFSC Secondary AFSC

List your skill level l ,349.7.9

Whe were you assigned to youru ty ASC Mn.._ /

WMen were you first assigned to your current atrcraft or missile system:

Tear

If you work with a tam, hom long have you been with your present team

Meonts

Who is your imediate supervisor?

Circle your se: ale Female

Circle your USAF mployment status: Military Civilian ART

What squadron do you belong to_ _ _

What type and model aircraft or missile do you maintain

List your rank or civilian mploymeont grade_ _

New many hours do you currently work In a shift (circle one)

I $ 10 12 14 16

Now my ang do you work beteen breaks (circle one)

4 6 6 7 8 10 12 14 Othr ,.

New _%n additional duties do you have (circle one) 0 1 2 3 Other

Now may hours a week do you currently devote to your additional duties

(circle mae) 0 2 4 6 A @tr___

Now many service or Interest clubs do you participate in?

Do you supevise the work of ethers? yos_ NO

If yes, hw many Others do you supervise?

If As, bw lon have you been a supervisor? , onths

WA "OUR sitg suI coi mien is

PLUSE HARK THIS NUMBER ON THE ANSWER SHEET BY BLACKENIHG IN THE

CORRECT NUMBERS, WORKING TOP TO BOTTOM.

i4 ..... . ._ ___ __

MAINTENANCE TECHNICIAN SURVEY

Part I. Group Morale

This first portion of the questionnaire requests your frank feelings

about your Job, your immediate su pervisor and your fellow workers. There

are no right or wrong answers. The best answer is your honest personal

opinion.

You may agree or disagree with each of the following statements. If

you strongly agree with a statement blacken rectangle A on the answer sheet

(by question number). If you just agree with the statement mark rectangle E.

If you mildly disagree with the statement mark rectangle C. If you

strongly disagree with the statement mark rectangle D. Rectangle E is not

usec. Following question 1 is an example of the answer categories applicable

to all questions in PART I.

1. I feel that what I am doing here gives me a chance to make friends.

A. Strongly Agree B. Agree C. Disagree D. Strongly Disagree

2. 1 believe that all my associates in my work group hold beliefs that areunreasonable.

3. Most of my associates here would help me if I needed help.

4. My Immediate supervisor Is out for his own advancement; he doesn't careabout me.

5. My supervisor can always be relied upon to do the right thing.

6. I just tolerate the people I associate with here.

7. All of my work group associates are a dull lot and don't think seriouslyabout important issues.

8. I feel that there is plenty of chance to get ahead in what I am doing now.

9. I would never make friends with any of my associates here.

10. My supervisor is out to help me as much as he can.

11. 1 seldom pay attention to what other people say; I believe in making myown decisions.

12. I feel that I have made some lasting friends among my associates in mywork group.

~~1

4

13. 1 believe that the work I do now keeps me in a rut.

14. 1 feel that I can ask advice of most of my work associates.

15. Most of my work associates are stubborn, no amount of argument willchange them.

16. Just a few of my work group associates are open-minded; most of themhave biased points of view.

17. My supervisor got ahead because of his connections, not because of hisability.

18. Sometimes I like what I am doing here, but most of the time I hate it.

19. Most of my work associates would risk their own security it It werenecessary for the good of all.

20. 1 believe that most of my work associates would "stab me in the back"if it meant they could get ahead that way.

Part II. Organization Climate

This portion of the questionnaire requests your feelings about the

local organization within which you work. As in Part I, you may agree or

disagree with each of the following statements. If you strongly agree

with the statement, mark rectangle A. If you Just agree with the statement,

mark rectangle B. If you disagree mark C, and mark D if you strongly dis-

agree. Begin with answer block 21. Following question 21 is an example of

the answer categories applicable to all questions in PART 1I.

21. The jobs it this squadron are clearly defined and logically structured.

A. Strongly Agree B. Agree C. Disagree D. Strongly Disagree

22. In this squadron it is sometimes unclear who has the formal authorityto make a decision.

23. The policies and organization structure of the squadron have beenclearly explained.

24. Red-tape is kept to a minimum In this squadron.

25. Excessive rules, administrative details, and red-tape make it difficultfor new and original ideas to receive consideration.

'1 . ... . .. _____________________________

I

5

26. Our productivity sometimes suffers from lack of organization and planning.

27. In some of the projects I've been on, I haven't been sure exactly who myboss was.

28. Our management isn't so concerned about formal organization and authority,but concentrates instead on getting the right people together to do the job.

29. We have a promotion system here that helps the best man to rise up tothe top.

30 In this squadron the rewards and encouragements you get usually outweighthe threats and the criticism.

31. In this squadron people are rewarded in proportion to the excellence oftheir job performance.

32. There is a great deal of criticism in this squadron.

33. There is not enough reward and recognition given in this squadron fordoing good work.

34. If you make a mistake in this squadron, you will be punished.

35. The philosophy of our management is that in the long run we get aheadfaster by playing it slow, safe, and sure.

36. Our business has been built up by taking calculated risks at the right time.

37. Decision making in this squadron is too cautious for maximum effectiveness.

38. Our management is willing to take a chance on a good idea.

39. We have to take some pretty big risks occasionally to meet the sortierequirements of the wing.

40. A friendly atmosphere prevails among the people in this squadron.

41. This squadron is characterized by a relaxed, easy-going working climate.

42. It's very hard to get to know people in this squadron.

43. People in this squadron tend to be cool and aloof toward each other.

44. There is a lot of warmth in the relationships between management andworkers in this squadron.

45. The best way to make a good impression around here is to steer clear ofopen arguments and disagreements.

46. The attitude of our management is that conflict between competing unitsand individuals can be very healthy. f

14(

6

47. We are encouraged to speak our minds, even if it means disagreeing withour superiors.

48. In management meetings, the goal is to arrive at a decision as smoothlyand quickly as possible.

49. People are proud of belonging to this squadron.

50. 1 feel that I am a member of a well functioning team.

51. As far as I can see, there isn't very much personal loyalty to the squadron.

52. In this squadron people pretty much look out for their own interests.

Part II. Occupational Attitude

(To be completed only by Air Force Personnel. Civilian employees should skip

to Part IV of the questionnaire).

This third portion of the questionnaire is intended to explore your

feelings about your job with the Air Force relative to assignment locality,

pay and benefits, and social status.

Different srales are to be used in Part III. If you are Very Satisfied

with the statement given as it pertains to your position as an airman in

the USAF, blacken A on the answer sheet. OR mark B for Satisfied, C for

Unsatisfied, D for Very Unsatisfied. Give a true picture of your feelings

and respond rapidly without going back to previous items marked. Begin

with answer sheet block 53. Following question 53 is an example of the

answer categories applicable to all questions in PART III.

53. The geographical area to which you are assigned.

A. Very Satisfied B. Satisfied C. Unsatisfied D. Very Unsatisfied

54. The attitudes of civilians around your base toward the Air Force.

55. The educational opportunities provided by the surrounding community.

56. The IX and Comissary facilities at your base.

7

57. The cost of living in the area to which you are assigned.

58. The similarity between your assignment and your assignment preference.

59. The facilities provided by the base.

60. The distance to your home of record.

61. The on-base housing.

62. The size of your base.

63. The size of the surrounding community.

64. The additional duties associated with your job.

65. The cultural opportunities provided by the surrounding community.

66. The recreational opportunities provided by the surrounding community.

67. On-base and off-base transportation facilities.

68. The quality of base quarters, barracks, or civilian housing in which you live.

69. The quality of food and availability of eating facilities at your base orlocation.

70. The amount of money you can make in the Air Force.

71. Your pay compared to what you could make on the outside.

72. The protection provided by the Air Force Life Insurance program.

73. Your fringe benefits compared to fringe benefits offered by a civilian job.

74. The advantages provided by the commissary and BX.

75. The opportunity for you or your family to travel at military rates.

76. The standard of living which your income provides.

77. The quality of medical care provided by the Air Force.

78. The retirement income you would receive from an Air Force career.

79. The benefits provided by the Air Force.

80. The cost of TDY versus the payment received.

81. The extent to which yourmilltary pay covers your living expenses.

82. The respect that results from your rank and job.

(4 -

8

83. The opportunity to meet and work with important people.

84. Your social position in the Air Force as a result of your job.

85. The status you have in the civilian community because of your job..

86. The prestige that goes with your position.

87. The status given a military man by the civilian community.

88. The pride your family has in your work.

89. Your prestige in the military community resulting from the type of workyou do.

90. The prestige your family receives as a result of your job.

91. The feelings you get from wearing the Air Force uniform.

92. The status your job gives compared to the status you would expect as acivilian.

Part IV. Feelings While Working

The statements listed below describe how a person may feel while working.

Reac each statement carefully and decide whether it is applicable to how you

generally feel when you are at work. If it Does describe your feelings,

blacken A in the answer block. If it Does Not describe your feelings, mark

rectangle B. Rectangles C, D, E are not used in Part IV. Begin with answer

sheet block 93. Following question 93 is an example of the answer categories

applicable to all questions in PART IV.

93. Can't seem to think.

A. Does describe B. Does not describe

94. Lack patience.

95. Feel a little hoarse.

96. Have a headache.

97. Feel unsteady on my feet.

98. Body feels generally tired.

9

99. Can't think clearly; have "cobwebs".

100. Lack self-confidence.

101. Feel thirsty.

102. Want to lie down.

103. Don't want to talk anymore.

104. Seems hard to sit or stand up straight.

105. Find it hard to breathe.

106. Feel drowsy.

107. Feel sick to my stomach; nauseous.

108. Feel stiff and cramped in the shoulders.

109. Eyelids twitch.

110. Seem to have no interest in things.

111. Feel like yawning.

112. Feel anxious about things.

113. Feel dizzy.

114. Eyes feel strained.

115. Seem to forget things.

116. Legs feel tired.

117. Hard to holc my head up; feels heavy.

118. Arms and legs feel Oshaky.

119. Feel aches and pains in my back.

120. Feel clumsy and rigid when moving around.

121. Unable to concentrate for very long.

122. Feel nervous.

123. Feel bored.

124. Keep watching my watch or a clock.

10

Part V. Personal Traits

In this final part of the survey are a number of descriptions of personal

characteristics of people. The descriptions are grouped into sets of four.

Following each set of four descriptions are two questions: (1) Which is

MOST LIKE YOU, and (2) which is LEAST LIKE YOU. Mark the answer sheet in

rectangle A if the A description applies, mark B if the B description

applies, mark C if the C description applies and mark D if the D description

applies. Begin with answer sheet block 125.

A A good mixer sociallyB Lacking in self-confidenceC Thorough In any work undertakenD Tends to be somewhat emotional

125. In above statements, the one most like you is (mark A,B,C or D on answer sheet,

126. In above statements, the one least like you is(mark A,B,C or D on answer sheet,

A Not interested in being with other peopleB Free from anxieties or tensionsC Quite an unreliable personD Takes the lead in group discussion

127. In above statements, the one most like you is

128. In above statements, the one least like you is

A Acts somewhat jumpy and nervousB A strong influence on othersC Does not like social gatheringsD A very persistent and steady worker

129. In above statements, the one most like you is

130. In above statements, the one least like you is

---------------------- w---------

A Finds it easy to make now acquaintances3 Cannot stick to the $me task for longC Easily managed by other peopleD Maintains self-control even when frustrated

131. In above statements, the one most like you is

132. In above statements, the one least like you is

-------------------------------

- m oe---.---o --.-.o -.--.------e---------------oo- ----------- -_ __ __

A Able to make important decisions vithout help 11B Does not mix easily vith new peopleC Inclined to be tense or high-strungD Sees a job through despite difficultes

133. In above statements, the one most like you is

134. In above statements, the one least like you is

----------------------------

A Not too interested in mixing socially with peopleB Doesn't take responsibilities seriouslyC Steady and composed at all timesD Takes the lead in group activities

135. In above statements, the one most like you is

136. In above statements, the one least like you is

----------------------------A A person who can be relied uponB Easily upset when things go wrongC Not too sure of own opinionsD Prefers to be around other people

137. In above statements, the one most like you is

138. In above statements, the one least like you is

A Finds it easy to influence other peopleB Gets the job done in the face of any obstacleC Limits social relations to a select fewD Tends to be a rather nervous person

139. In above statements, the one most like you is

140. In above statements, the one least like you is---------------- -----------

A Doesn't make friends very readilyB Takes an active part in group affairsC Keeps at routine duties until completedD Not too well-balanced emotionally

141. In above statements, the one most like you is

142. In above statements, the one least like you is----------------------------

A Assured in relationships with others

B Feelings are rather easily hurt

C Follows well-develope work habits

D Would rather keep to a small group of friends

143. In above statements, the one most like you is

144. In above statements, the one least like you is

---------------------------

12A Becomes Irritated somewhat readilyB Capable of handling any situationC Does not like to converse with strangersD Thorough in any work performed

145. In above statements, the one most like you is

146. In above statements, the one least like you is

-------------------------

A Prefers not to argue with other peopleB Unable to keep to a fixed scheduleC A calm and unexcitable personD Inclined to be highly sociable

147. In above statements, the one most like you is

148. In above statements, the one least like you is

A Free from worry or careB Lacks a sense of responsibilityC Not interested in mixing with the opposite sexD Skillful in handling other people

149. In above statements, the one most like you is

150. In above statements, the one least like you is

END OF SURVEY. PLEASE RETURN BOOKLET AND ANSWER SHEET.

t

MAINTENANCE TECHNICIAN SURVEY

SUPERVISORS TECHNICIAN MOTIVATION EVALUATION

All first and second level supervisors of participants in theMaintenance Technician Survey are requested to complete a SupervisorsTechnician Evaluation Form for each technician on the enclosed listing(persons either directly under your supervision or under the super-vision of those whom you supervise). The enclosed number of formsprovided in the packet should match the number of technicians onyour list.

The Maintenance Technician Survey is being conducted as part of aresearch effort being conducted by Arizona State University under con-tract with the Air Force Office of Scientific Research, and with thecooperation of the Air Force Human Resources Laboratory, AdvancedSystems Division, WPAFB, Ohio. Headquarters USAF Survey ControlNumber has been assigned to this survey. Participation ofthe airman and civilians selected is voluntary and each subject willbe provided with a Privacy Statement and have an opportunity to declineparticipation.

The purpose of the Maintenance Technician Survey, to be conductedat only Williams and Luke Air Force Bases at this time, is to collectinformation from Air Force and civilian squadron maintenance personnelconcerning their perceptions of factors which influence their performanceeffectiveness. A second objective is to initiate the development ofan Air Force Maintenance Performance Effectiveness Model based on thesurvey results and other inputs. The research data is intended forgeneral Air Force use and is not intended as a means of performancemeasurement at either Base.

Your thoughtful cooperation in completing the SupervisorsTechnician Evaluation Form for each of the technicians on the enclosedlist is requested. Then please complete the separate SupervisorsTech-ical Information and Performance Rankings Form covering all of theequipment and personnel under your supervision. If you are a shop chief,please rank the airman and civilians who report to the supervisorswhich you direct.

The enclosed list of personnel who were selected for the MaintenanceTechnician Survey either directly report to you or report to a supervisorwho in turn reports to you. Next to the names on the list are assignedSurvey Control Numbers. Only the Survey Control Number is to be placedon the Supervisors Technician Evaluation Form (one number per form) inkeeping with the Privacy Act of 1976.

However, the separate Supervisors Technical Information andPerformance Rankings form, one required for your entire area ofsuperision, is to treat all of the equipments and personnel underyour supervision. Names of all of the personnel reporting to you,or to the supervisors under you, may be used in the part 2 perform-ance rankings. Note that the rankings are not to be limited to onlythose technicians on the selected list of subjects who completed themaintenance technician survey.

Thank you for your cooperation in this research effort.

SUPERVISORS TECHNICIAN EVALUATION FORM

for Maintenance Technician Survey

September 1979

Maintenance Squadron

Your Name Your Supervisory Position

Survey Control Number of Maintenance Technician to be Evaluated IInstructions: You are requested to evaluate the above referenced

technician on five important traits which are motivation indicators.

For each item, two written statements of motivation behavior are

provided at the extreme ends of a vertical line scale. For each item,

you are to place an X somewhere along the line, at the point which

in your opinion best represents this technician's typical behavior.

The type of motivation which each item is designed to measure is written

along the vertical rating scale. Note that your responses are for

research purposes only and will not become part of the technicians

personnel record. Place an X on each of the five trait scales.

1. Job Curiosity Trait of Technician.

When working, this individual would be most likely to:

seek out information about other parts of the aircraft and try tofind out how his/her tasks fit into the whole system, even if suchinformation was not essential to his/her task performance.

Scale1

W

work only on his/her task and would not care how his/her workrelates to the whole system.

2. Persistence Trait of Technician

While working on a long task, the technician's work group ranout of a crucial lubricant and no additional supplies wereavailable in the shop. This individual would be likely to:

willingly go to another shop to get the lubricant necessary to:omplete the job, whether asked to or not.

Scale2

use this as an excuse to stop work on the task and leave theproblem for someone else, or complain if asked to get the lubricant.

3. Professional Identification Trait of Technician

This individual would be likely to:

show pride in his/her AFSC, training, and job, and considerhis/her daily work worthwhile to the Air Force.

0Scale3

consider his/her AFSC *worthless" and possibly degrading. Also,Ahe/she would consider his/her job unnecessary busy work whichdid not use his/her talents.

4. Organizational Identification Trait of Technician

This individual would be likely to:

take pride in his/her participation in the squadron and/orwork group, and cooperate harmoniously with supervisor and/orassociates.

U0

Scale

4

U

U

IUU

continually complain about the squadron and/or work group anddisplay alienation towards supervisor and/or associates.

5: Self-Starter Trait of Technician

An urgent work order has unexpectantly come down to the shop.If this individual had a "routine" dental appointment, he/shewould be likely to:

ask the supervisor for a few minutes to Call and cancel ordelay the appointment so he/she could become Involved inthe uaetwork.

Scale .

4

complain to the supervisor that he/she could not possible re-schedule the appointment and ask to got out of the work.

- U i .-.. . . .

SUPERVISORS TECHNICAL INFORMATION AND PERFORMANCE RANKINGS FORM

for Maintenance Technician Survey

September 1979

[One form to be completed covering your area(s) of supervision.]

Maintenance Squadron

Your Name

Your Job Classification or Title

Part I. Technical Information

Instructions: Both first and second level supervisors are requested to

describe the technical systems, test and repair equipments, and technical

information with which their technicians work. Please provide your best

estimate on each of the following 9 items by marking an X along the

horizontal scale at an appropriate position.

1. What is the average estimated weight in pounds of the subsystems(assemblies) which are serviced by the group(s) you supervise?

0 pounds 40 80 120 140 200 or more

2. How difficult is it, on the average, for your technicians to serviceassigned subsystems, based on the accessibility or clearance forservicing?

simple (adequate clearance) most difficult (very tight

0 1 2 3 5 clearance)I I , I

3. How difficult is it, on the average, for your technicians to removeand/or replace subsystems (or components), based on the accessibilityor clearance forsuch tasks? most difficult (very tightsimple (adequate clearance)0 1 2 3 4 5 clearance)

4. What is the average number of internal components your group(s)deals with in servicing subsystems?

0 10 20 30 40 50 ormore, parts

5. Estimate the average hours between servicing actions for thesubsystems serviced by your group(s).

0 25 50 100 150 200 250 300

6. Rate the adequacy of the technical information available toyour group(s) for servicing subsystems.

poor excellentQ 1 2 3 4 5IIiI I I

7. Rate the usability of the test equipment available to your group(s)for the subsystems serviced (leave this blank if your group(s) doesnot use test equipment).

low high0 1 2 3 4 5

8. Estimate the average knowledge of the technicians you supervisefor the subsystems serviced.

low high0 1 .2 3 4 5

9. Estimate the average skills of the technicians you supervisefor the subsystems serviced.

low high0 1 2 3 4 5

Part 2. Performance Rankings

Instructions: Two separate rankings are desired for all of the

technicians whom you supervise. Since you have worked

directly with the technicians as a first-level supervisor, or have

observed the technicians in action as a second-level supervisor, you

are well qualified to rank the speed (work productivity) and the

quality (work accuracy) of each technician under your direct super-

vision [or direct observation]. The first performance ranking is for

Speed, the second for Quality. In each case, after the technicians

have been rank ordered by Speed or by Quality of performance, you are

requested to scale the expected level of performance from 1 to 10,

with 10 being used for the top-ranked technician in each category.

4;

E .0

a34.. 0~ f jE0

- ,o U CL a

M0 V 0 E

-0 0L- 0.C..o- a)~ a, 7z EUED 44-Ooc

Ea 41 C d L0.)

04 L. I J

CM CM 0 014c a1, 0 EU ZL01 EU VA CO Eas-

(A 0 ~L0 wf c )

.0 4J0 = aS , =

41 = - "4) 0 UEm 4) EU

acs a, . ) J 0L

4, Ca)- ,S TUA 4- 4 En 0 0 EU 3gIQ- 06E En=-a

L. cUao 441n.. 0-.,-A

4j L s-n..0E IVI 0wr0)a) 3 u a 4) 4)

ZL J Z4-) r- 4at IE4J. 4 m

0_4

Eni EU EnE 0 .

Go& 4) u )ul w 0

c 06 4 EU

04. fVE 4- v-

4.- 44J-P- c w0

a, b- L. .. 0 94

4JJ W 0 u cmLfl aao 41 EU0. C . "a0L.0

M~. C.0nU

EUW W 4

CA,~, ~ a .0 IA=

0 0- cmCW W Vn L. a, C,~ c00 .4.1 .4 . e d W--

uS L. 4J0

0~L034J'V

F9

3m~ ~~ 4- ., AU>

0

0 . " 0l L

0 - W 4 U 04J.* 4,

"@ - E " 833 @3:

0 4a'. .. . . . . .....m,- 0 ', p"" . .. . .

U)*i 4- r-.T :T*- qL- L .

0I @3 CL . 0 : CO

Cl ., S.*C- -. .-. S.

'4 a) 4J 1

,L 0 4 ..m

20 4j4Ar- 0'5 .3 igE >.i

"f L C, L 41

= rm, .," a

'OW 4)C O4)15'5. C 1.u 41

i fUa 0(u - Lo

41 41 1-

0- " "-4) m

U.0 > 410 ,

C 3 0) 0 4 C 0 D

o s5~ L - "I. -'

W* '4-) +J - 039-

w Gn r &A u 41, c

0 L0 .0 0 ;, , :: to

= :0 4) 41 =.

LA m -U 0 C0

0 41 06 C4

LO@3=.1%c c vCDJ

LA I. toLi. :O Fm WC L. 4

fu W V . 4 C7 1U-I'0 W5 CC 4A )oi LJ O C U)-W =) 06i

- w .4' 41 0 ) .... 4n0 404 uU15

I. C . a O. 21.

r- V 0 -W4) C L.C= OA.31 .C'g5f 4

4141.'uq. - 43LCI 40 L 0C0 m .5 w.C0C

CO z 4 aCLU a

41'5441W U14

m 04) to 4 .0-W'4JV L4J 4J C C .5

00 = S uU4)'C '4-4) 414) 4) a. c 4 me

CL a 4 L.41

a. _a &00 CL 4U1 # 4A

C &n4' . -Z 3ac41 4)1 0 (A 0 V14)2

'm 40 10 - 0 n . CL 4)4)

c vC1 V tC-C C,0 C '. r

lotU0UiW44

APPENDIX C

FORTRAN IV Computer Program forProcessing Survey Data (including acomplete example output). Programprepares data input on cards forBMDP%_R Statistical Package.

Data for Williams AFB, Field MaintenanceSquadcron

.4

-- - -

-- A = 4A AT-e. I..Z z- -AW 2 01

A 2 MW !m of C 1-W *3 20-A - -2 . 44Cr92f. t

.1n 0 33 2 'A !.4:O. 2 = ' .1 2

z4 Z 40 Q AAA A " 1A AA - Z,44 04 ;. - ~ I, z . 2 :S2Z2 vo X

4A Zz = Z O-302j z zW N* 2 2 -,z .2 44 w - 0 aw~w w 2 44 4 Z0 4 :"220,4 A. .42 t4 " w 4-

44 a ! -_; -- a J.144.j4 -i 3

2~~~ ~a 2WN4-

X.-Zf (4 t.1 az% SA 4w~~ = 1.4 1a4 X4 ., zz224244~u 4W2W -* 2-N-A

- - .4. ~l AW A 14c fA~

- -4 i222 zz 222-222 22 C 20.% 4 c2g - 1 44 - - 4 - 4-41,4J.4144 w W-1,

.42402 242xA

L .,1 3224. -- X2 2 25.2 425 22-S 2 422.2

=-2C.42 4 C43 2 1 a2 -v 1 - 4s..- 2 4o .14 i X- win4 4 44 --Cz . w

Ol =24.4 -34 2 'a (2

4 4 4

44

- 2

Li .- 2 v 32 -4- Z,4 2-0 44 4 2 4*5.22 2a2-42 - 22. 4. 24134 3 2-8 ^41 A4 N'N -I "in4442,2

-Z A . .2 22, -as4 2x4 A 423' 144 4 24 4W4224

..... ... .. ... .. ... .. ... .. ... .. ... ..~~*402~~~~3pe-,- Z2.44 404 444

4 2L

2-. Z: z I.

Cl. - JZ

2 .' a : 2. 0 4

2x 44444 222 l: 42 3-1Z o0 c vJ 2. It z a ;A0

.444 223 - J2w -2 M. IA

U U zi cc 442.20.. Ma. I4 2 * u 20 0 I .43

2~ IJJ J -a. 4 .42 20 4 2 -... -.. w ze --

2.4 222'JZ.22..4 2- ' 2.411 ag z *42 'U-- At .22 =1' 213 - -- ajxx a Us !J: mw, "W42.4 4 a

.4a 2.. a Z--W4 32- 2- AZ4xW 4 32 2 z4 - a.-'~~~~~~4za WX ^*,Az42.4S jma =am21 A. 24U24.Si

.4?r " 24a.4. 2.5 42 0X 40 20,6- tto - SS; 4 24.4.tm.42 ~ ~ ~ ~ ~ ~ ~ "t a222220 V2 V.2 2-5 . 2 2..2 44-~24 1 2 2 2 2 2.4..a.2 2 ~ .4 422. "A44. . . a1. ? 1 -a . . . I s 2 41

4 N - vv O22. 23.. AA 2aZTO 4 4 2 ~ 4-.- ~ ~ ~ ~ ~ ~ ~ ~ ~ " A2222~4.0 .. 25 2 .'1'. l . 2 a - -X

332223P2?O A4 4 04442 -- 0a.2 -- 22422 SO4040@P 2. - 2 24ZZ 22.

a- . C 2 V4

.. 4461

t4A 2

43Z34C 3- Z7 ..!A . 2

2 334.343- -

Z-2 ~ ~ ~ - oa a Z01a233 4 1Z

34N433 - Nf-

I

Ao cm

U- -4- L- .

:-4 :.-.:

.34 4 4 4

4a 4 4

---------

.1

A A

-~

0

03

2

14

2

2

2 -

2. . . . ...- -.

22 U

:: ..: .3. 8, - 4 * ): d2 I OO " Q D4 0 O 0OO 0

N -

*4 0- - - - - 4

.40

- 4- .A .- Nr N N. 4

.40.

CA 2.c

4.22 . - 4 .3~

8.1 v. 0 8 S

ow-io

S %S N ~ sa..' ZAZxzwd 4 42

V84 -a s 4.8

'A . 4 ' 34

'A .4: 1;2AA

..... ..... CO12N'X *.. 23.4.V

ZS 44. N.8 v A434 4 m V

34 **4-sag2 22

oA L-P N* 0t..N

~~aZ .Cw.. a: - ZZZ-00 zz 2 N.. 'a Oza 4N2

oa,. N4 -44a. a% a 04

a cau za.I -,

00

24 1:; !z 5, 4 4 f 4

C2 2 a

44 '4 *V- .0r S rt -a rN

a. . C W4 AA .

41~

&,o Cjz~ er ICGC

.%t~T4 4. .NV4.S4

-40- cccc c c4cc

4ZA2zct a c a,0,2020.201

1~ '1Zcztzt)

zcz z z 1 A' .IZ,

LX a*= aszI msa 2Ozao~oAvsa~ mlaa alm

ocNA. NN- r wNNfN~tMsN a Va ftNN NN NN N

1 .4 11 4, 1,4 4 3 4N~'N' *, 4 41 N'C''N'A4N'..N4N~2 'NZ;Nz NN1 N A .1 I N 41 N

& 4 - -- 4 sPd - 44W88 4* 349A 9* 0 4A.644U 8O

sPNNSz.Nl.N N- N dNN 4

4r~pv IM. M-4e~-NIs.-94C f NCU 00tN 4 ,O6.NN9d4VPp..% N S N AI N A % ffrArNA- - PNNNN 4 MNM N ".hJ

AAJAOA AAAAJA A 2 AA.A AV.AA A A

as 'sNse ,.PP.a

2

222.33'~C~2~ :223ift024 3020

3002 00334 32003* 03.flJO@O

N...d.%%NNNftN44dP.4%N~P4N.4%hft

02:2

33A444 34~A.CAAA4A.~4~4.4.i~

*Cn~aCA33 2U*3G****USU*SSSSU8*E0

AS NP.N** WU UP. ~4 UP. VU

.%Z24N44 d~NNN.%NNN~Nt9~ANNNft

N. 0-ft~..4-ft -

022 332323-

**2.s4.~e.ft4.-..W..r.,,...*0P.,. S4N N. NN6~4S JP.NN4-.P.N%*

.. *0.344NN~446U4~4 0W*t.~.4S.0N~P.2P.Np U4- *4 P.4 3

3.1

., 22AP.A.*44A4SJ4A~ 345440 3- .s0..A.lA-~0U00e00Crv0220 441-2".0 2~ *~45 1AJ45N~4.D4SA A '45 A04BA~A A

~ C~ '4 '4'4 4 ~ft4P '4'43 eS 4?VIWPWVW S 45 4 rev err, r,.S 3£ ~L4NNS.A5...NN Ap.NN~ dP.NA.4S'.NP.S..N 4

4'NftftSftftN-$.fl5tft* ~NCflUttVtAA.DAA44444N''N*3S*S a

S NN4--NNSNNNNftP(5N4"ftftftflNfl C*

P %-N'.%. 15tttNi.~~~UNt??.1.N4~t2 ,NS.9 prpvrNnnSr~n9 ~*CWtWVNflt- ~sn-s-.r rr.vrrn.~n~r'Snr9"vrP.'rt U

IA .3995*J U 3445P.3 UJeNaE0ft a.

I* pp.1 C*p..pp*Ud,4U 8'.ft3.8W4~S4UUAftAd..d

3.2 :25 502302222333323J232 2ft332033233ft 2S *A 4-454*A5r, SAG 4ftUO3-.~~~AA4ft *0 A2 4**4SS.S' UJUUS*UUv.433.3.3.3. 2C ~ £

wA a ICf--z C3A:ZA-o aMC .4N

'a-I-3 zm"z1 N

zz=2r==Nz2;! zo I-Z

w r 3 .3 3 w 3 w 2 9 . -w 2 w Ow q -t 2 l 3 N C 1 fZl3 79.,~nr~nn~nnaftO~n~anba~sOC.naO'.CN CgC~lCN

-- : .,a.nyae~~ na-a~wO~aa'4ana.A CO n O'flC-47.C.C39N330 f3N 03.3f. OW @ Mzz-204-c -O N-3rNZ

2 N t 3 N 3 2 9 3 2 t 3m '4 3 3 3 w 33z 9 3 N 3 * 4 4 3 3 3 2

P 3 4 Nz-z-cn 3-anan Oa.aw.Z'na00 N§3g.403flCNw

Nn:'NC n C2N : '2aN C3 '3:Nz239 3:rr43N C cN

C3-,CC.z -0c:NN:: O3NC'A"fl333NC3SV33?4fl jt2 NN3zW2W C :Z nrC , znnCNZCNcan-nwcnw CwO -C4 -0c31 c N 11 N3 0

:: '2n Can atCONN2:aCa"NNCC3NCN lN%-c3-::NCZ COaN:N:CN-:NCfCzNNOnN 3 3 C~N C N C N

9-::N::.N~9.-N::'.en~z~N::.Nc~n3,d.c3.N ,:N34N a

?Cflc2N CW-0 aC-C.fc Na anCCs-CC-NGC."NCCW-Ca .zz a -N a . a " 3 N a w a N c N n N C 9 0 . 0J N3 JO O

-, N A at-C- 3 -n07 a~9~ NC0 SON93C NC9N~:zN:2339.3:jN2.9.3:933N.Z3 N~anJN~.-133 N o2 N-. r9.:?:r43'4-:f,-39N0N:39N299:: -cwCo9'-::-NC

N'.:'..'22'NCJftft'.C3N9009' .2993 .99:a N a490' w aN 3C-NO

w-CCNCCNO~wCc'-NCCNC~wC C NCCNC ft900C CSN0cCNNCC-C

?23'N923% OCNNO3NN3O ogNJONCONN ON2Y OC N ON C-r 2-ftf 2NN 90N0N0NOn unc a Vaaft0ft- anGO NzOO.wm

Pt :Z 009 03-v90 C9 C PW C NCoI@0a'900000900SN- ~ ~ ~ ~ ~ zr 2.42 - :4w~ N CNC'C Y :Nc n.3 NCw OAt lNO E N

AP= .AZ a -:~v-, 3NlCfl '93 3w C" OE4O O OO ."3CO N O', ON4' 3X~l4 ,CC r. : 39 0N3N43 0939N3w0.f@ -

2 2 % 2:-4232,; a 09a23a9a 3n300 93w 0-3 -9039aZO 0 3032 a!3- a3 2.0*~~~ ~~ r 'IJr2'3'9 2 2-3I 3 w r 3'3 2 w 2:3'-'r

A 3 '49 N9 W 9ft-.9w3P 0 NN0939N0NCfN A N9ANA97P:ANwA:P P x. Pww P V A A A30wAnA.A-C-nr4 A

300 I O MN

2 I -=

:rr::9:z:.::,::: 2 SO =3W342n =04r3r3rgm =.

3.n ::c ccc aca eaa ncc =aa-o c.-nzm c q~ co.2 rQ~ C 234 ::.C3 C~w C~COfca N~n~cnfc93c cz: ,-Z -a 3

2:3-2 ~ ~ ~ ~ Z i z z-z22=NOZC N-3rzN=n.2..N- . ..-22722.2r2'4t3fV2 N2 r2-323:fl2 92 2N Z'2 .22 -C 2 C2t224.- 2 m C -C N 2Nf2-2zae2':qe2

'.'::4':2'322'3:r2 z932% 2 9:-4 N 3::12.2 -34:N-73r22-22 zo -22-2.93'4324J2f3t NO22r:2,-2'1 22::9:~r23TC392n43n9==-.Nn3n e34-zc.4::-N:'r.3373:r2:-:z-33:2N-2%.N24 03rr9-3,933 c

ZN:S-22 4

3-2

232-2 N 3 3zr fl4N C2S'02-33Nz Nlzr2r': %r2 cr2 3r2:r~r.2 cr22 ,22 N33 r:Z-32nNz-N:

* ~ ~ ~ ~ ~ ~ N--z N--3r~NCN~~nOO.'C3NNCNN mc~~n~

Z: c- = -'2-:r%:z:, O~N JMN zZ3=Zz: NC S 2PN:N2t2:9

.. N-433N'4N '.N29 N N -,0 i -3230QN0C Oc NQN2g'934 a3 4 w 0 2 Q

%'432o No . Z .r4 9 7 2 .' 3.4 f33,Z . 3 9 Z~r9N4 ,3 f C',42 r .3 , Z ZG ft '4 29 4 39 4. 3 323432 4 4 2 3 2 2 r

*N~N2V'N393 22-344244.'NC.394c .c,42743-9'42~~. Z~2N4i3N O N 3-2'3~ ON C N N3.'4.No 0.

* 7 '2~~. 2 N 4 93 20 024 3..3-c9 4 0 43 0 9 c3.c.34.4 27429422 :2N03NCZ~a2 NC3~o:432934:933:N.z3

-.. 4SOr4 23f*337403343.Cazc a303~racn

2~NNZACf-fCCV nfQr Zae4Cnn Nra- a -NAaftn.A3N anea

2 j %2 - -'. CO 000dlC604@ 40 r cnN@ Z t03Z V347020aNo-N ..AN9" z"Z ,f

.3-i_ M220a32930293234 o ZO 2 9.3N 2349 c -3 -c nt Ccf -c3rcn a

:r::r,3 SftC 3a z 0 3Ca ncrcw.crsac3ca33aN"3ja 32 oa~anrc.3.azaa.j-~ ~ A- it A04 CC C CN 3 M@3224A090 -eCArc't Acc

Zv~~~ ~ ~ N.C -co UC ccIc I c.acC04

4 ~ :qqt - C

.5ct,0 V 1S ; z0-* . t =Zz

2. p ZN 5 1'

N2 =Z g .-

zzw.zcw za c Z 50-ZMS zmc

M IO0 .2 3 IN r

rnnw~~~~a'4w2.5.4 lzz co5~e2 tN32Zc-aacca wgfz. a32w

.::' ~ ~ ~ ~ ~ ~ ~ ~ =f .::wnns ON2z5 -. : 3.42C 234r .. J4ttf J2~~.:Z~~.N32~~~NflwW -:aN:-3e4.22'4C :- 3ft0c2NsN ::'cN~

NJN C t NC 2N Zft N.N32CNNCJ n%5 ZC3.N3.N-

- 1NZflNC ZN2Czcm lZO @cm.Nc c3..N =.N. N ca- n3ftJwaNNCCNCnNCC' CwNfn w Cfl C Z Cw CO__ GN0C N-a ~ ~ ~ ~ ~ ~ ~ c .5%,-.'NQ %3Z33.%cnl r0Cn3Mn3NCNc3N amQ.fNNl- z". 4:C=wc. cwc N.cc~c ~ . w

:' n c --r t n 2?ln NcZt c n C n az s . . S n 9 N C m l

Z A a 5*z= N zmN 3a-=O^a3N 23nnt a eNf 2.'car 3O-a-.c2N.%2 MNOO2NN.&N -a,2aaw.:1 cr.4C'cnCN:n.NJN4 aJN

.NC,3i0%-@.N3Cr.~NO 4 2? 02N43%72244-, Nflfl n~z~na 2~ -02NO 2sCNZ~ 3a'Ofl da4.lNC3fN0... "ZNJChnC-cn0w2ncn=nnc CV43~0N

:4eZ4jc 2. n == 2305l32 n2f33n3fl.N."-3N5a9N.534'a: ZZA 2 .N392S2a9 '4 C.0l04N cc NtN344-2'a f.34a

Z.1N2J.%.N2 ~ oof --2-w-4c a~caN3f04.29 0"CfNSfNN:'NNCNZZ2-c4Znc44nZNc-a~07ec.3.G~ll

l-.-70f.43 'a 2q.t1c3.1: cwj ec2 .4ol Z . .. Z 4a3NncJ. 3f -3vl.C3 -lflO2a;flN4NC Za9-ccn.4cf nc3nnc30 aaql@Z Z.onAc402Af20-3=" ccflna ofto355,

N J'42N2~20442N 020'3,lf02a40 032fl0fll07.J ~ ~ ~ ~ tfz"OZtC4 ZJNN_ fllZl03ln0z ZnOnnJfl0 meQ N- 2220".02w 320 0n0 01 -0nf@0-,.C2~ 0 20 3rc4 0 a-2:3w 22~ ~~~ 23fl0 ? 2n0 w O ntfgcfl3 2-: -

,::? Z:%.4::N c 2aca:.c:,- . n crM=z 2 c . zr c a a a422N'22Nn@0*n03 Z"2Z*0*IFq-ft Z. 0n anallnO cnOrc

f4 4, . ", 3N .Z3Aaw43222 22= 2m-.J'.%3 .. 22jndlfJ'q 32 '42 Q COOZO.5"flflC4flw aaaa a23 3a cc02falN"a 33f202-n22wMa'2oz4aw2

z ',.5:N' 2n3] *r2 aa 4 2Z c...-a 2 naftnZ2ac 4a'~c3c222Z2- a"J-.ao3NZ3

3. ~ ~ ~ ~ ~ ~ ~ ~ * A :.74rnq4n.0047f~q~~.AM02A A AqiAdAnAnc4-. --1 -~-5 .* - -:. -, 2 ---w- wr awn - - --@- - --gw - - -w0 -

2 - J:-'S: 3.1 ~ w a 3 z -- , a 2 m.2 2 -C 12-aS ON 4

.Z* Z

=ON Nzrzz O

=z = t= a.'SS cNO - z 'QNcc -3= OCN'SZ

z - . = m ^_ Or-''SN aS'SS N_,20'S _QC Z- .N ZN 'aNZ NN20NCICZN ar r -2~N.2 .NO am SNC 0 a N o m I N 'N ZMZ'CNNcz

-QVN 2

IC- Z~l c0S N0'09 '0 -fzQ _Q N02N'.2z C-cC.30rN J.Z amN N,22 ftN C.NN C0 ' a ~ t20 0rN1~ ~ 'S~..S3V 'S ~W 2S~. 3SC SJ~ 2~~'S'S''S V NS 'N~'S Nm

'S N~ N''N 'S NCS N 'SNO $~.0N O'N 00 NC~ 0 -~'Cc.2amSC~ C N CC N C~ CNC NC.CNC

'S ~ ~ ~ ~ ~c9az CZQS9OS NQC'S N Q NC 0$~ C~.C~.C @SN

z- z _ ZOO-,S' 'S'SS. zm~o C'SNC'Sz ::Oz-Tl3 m~ _Nz NQ ;at a'C N C O SS O ~ O 0 0 'C~C~.Y O N

~~~~~No . o m ,;;-cc cc.'''et2f3 SNSS3C ~ ~ ~ ~ ~ ~ ~ ~ cf 1.0S'SO'Z''C 4=000='S ZOOCQ LoftCN~

00 Z. gzz.m a ca",.G.: %'S-Of2 aCNA Q9 a a',CC mc a mc f'cNA ~ ~ ~ ~ ~ ~ ~ ~ ~ c 'S' 9 : -Q9J - 2 J'' -. tJMtS ' N am SN

S 'S*J"S 'S'S~t2JCC^'

Z; J z 3*.2 Z5* Z .- a -3 aaP a Or P PAPA A A P M Z, S''4 am 4Q 4 ONN.- aNo N __ N N a a 4 4 4 6 U CA A

ZI,~rf zr _wt " C

, o JV=,&a0SMt4emoZtamG

:,4 - - - - __ _ _ _ _ _ _

4M

00401 . .0MC ,M

4azt.z3,2.-Im: c c - z -4 a C

M e

12 =f2z~zm

r O~~~rr~~'9fl W 922r~~O aC 3i r NvS CcCr2 fI C 4 2 7 2 9zn 2 iz 1 C r e caa w a n o f a n a n a C W C C z ma w~

.CCI%-42-zh9C =2-m CCNC.OnNCVCnC*CNdCI.aNz

' a ZNZzO4:-O-:?~ NQN49:.42 C:r a212NC2r 2Ic ccn 201 C9NOfw 09W04707 09 0-. -21-3:NZ:NZ:- :ON:2-C=-:N:-a:-9 a- o 4 ~ -N2.N722.Nr ftczr2zPN:-:220r:Nr237272?r: N2N 24Z c .2C2zN-22 -2N 22-9::4,:N -32 N ... 9 447=ft..:Cl Me2N-O

I 2=N 29Nz 7MN - =NC zt2 N dC ZNZ7NNCCwZ N.:w Z =2249t2 3 N~f 24 :2- :42 f:N Oz; 4 2 74 2 ??2.N

1 2272l

.N z-NZzN-:.Mz7N22NN z N.:0NZZ4NZ:9: N~rcI-: NN:ZN-::N-00-~~~~~2mcNN 2 t.QNZ0NNCCNa=2. N:.24-:

N2.N 2N 2. ON 24N2.- OJN V. -r.00. - N244N 4

-. CN CC-C NNCC.C-gCsN ON C-= ..- Cz ^42074:27 M=44

t:Z-%:: =NN:.N0.7 .ZN -,244N:2.NZ N 227N:N:M2N4:JV N SZ42NZ.ZJNCZN : 4 aCZ Nf O9 C9NC2 N2C1NCZN 2N:iN'N

... CNNC9NCNN2242Z9N0.NN2NNOPN29NC rv2C, = N229NZ

31:4 2 4 - N2 or -CZNON -c.NZ9NN C9 0 N C V C .z 4 0 SA Oz z.Z ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ f aN4 CZ.Jf 2.N09C4404 -NON2942729437

-. Z.z0-9.QNC .3-2-NV 1 I 09-240 021N04.2I NC Q.' C2.44%224 ~ ~ ~ ~ ft Ca2J1N40zft~c cc2-204401204N04402NO--3o9420-e:44C244moc

,.Zzmt mNO.M4 a9 OA 3f4m -32 .027 22N GO-2442-.0 2 02-ft 0-24 A.22.IZ ONMMOON27N24 zc4"0444z.MzQ44 22N2N2,94

N13,2 -0049 l -CC -C N C-C9C..cw CC--CON-tC~wR..CCCC. CCNNCaN 49:2IN~name40 02N 490040940-0Q440Z aftC.94

-a2.i mOO Z 0m : M z r" 0raCa -O fn O 4N 2an~ c9 w NanN7CarNn0OJ -A0a-J. 4a I

NO2NNO2-NO CA m00z=.34 NC fCa.9440044.aze NOJ-NCZNNW No lrNOOI ."cc- NC

z2.9' ZZ00044r00.2 zw -0.940499C00-00-NO44.00-400NOI$a0.49Z ~ ~ ~ ~ ~ ~ ~ ~ -o ow4.4424 040444244044040'4C04C40790490'.2 :'-.::r-:or ~ ~ ~ ~ oe.4 w c.0 CO 04002 7N -00 0-m900.M90s

.22 N Z2 Y 2 N C $ 2 -4 0 4 10 1 0 20 4 4 2 41 3 4 C 0 7 2 44 0 N- ~ A are4N0~:4:r.O~.a4ar09-0- 0N09049094

N2-I4049034.r 0 r 404. ct0440 =AN3 2340-9 M94 3N

.Z09. C~~ae.O r4 C en C '2a roq o n a- 0044.90044#9Zo0MAT= 9a00744C*.240490NO4424CU.0Nr N~C244WorNOCNWaO49YCCNN -v.a3..'s~~n'y 2Na.9C - fl C-A2- f? 2:- M2m24.3:

a 0 -3:J.av 31 Neam4.900-e.CO'9.923-400-4-4409 raw0n 027922472 .249-4-~I4 N94444 -44-940Nl4 7.f0 9f7 4~

N' a 3 a7 O742l907..r9r~04~N 247r749'NN~24-4 24 940J N442944 M4490 NW---2r4..N4F

3cc

r Ictc 3C c OO c o c~tQ

ZN-Zr "33Nr23--j tN- N Z N2NJ3NN3N 3 V0NN 13.,9zP 2 2 C -N ~ - 2 r 24- 00' 3N N CO NO C N9'

4?33N,33NP3J3 ------ P23N32NW3Jz NPN N2~3 N 3 N 2 7

z'2'.. 2N-53 N,3 NN 3r 5N' fN, ==NN. NN3.. SICN'S 3 0 rNN 3 N

3.-c:..,=~rzN~ a:. w3N 30VOe.Cn 3r2 - C NCCzl- r C22 N =2N = z

:%rJ~3J r33 r:nt3NrJ~ZO NCJ V~NZ~rft nP3ZNNCJN .- Oz

ZN ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ I :a.3~39:r:NC.9C~2~139.:Nq 3N :2-=:J;:.r2'. Z:N 3.a N:: -:2 r:39~JZ.NJ:ZNt3:NZZN .2~:

3NZ2NNJ~~~ntCN~~tNNZCNC2NNZ3N CN C NC N Z N OJ N ZNZ

NN: N NZNZJNN e4:N c~rzJ'N-CC, S C:t-NZ

Z.;,N '40 1 Z.~J N 3'N..C3N$4CCN z ICflNaC 'NJ cNNt 0t4NNo0I0N0N N03fl N

JN3 NN3NzN OCN Z 0 OC, NNO ZNN CJ NOz NnNJZ~3JN3Z'-33N c3..N~c-C~-C3~-3Z-3JrCJN Z,4N4.

- .Z -. ZJ9N33NNOZNNZZ NCZP 3NN ZNN ZN :NN Z N

.OJqz N oz3ftNC-NO Naft" 2Nc2N G o -NJaN CN rNJC.,.2 -zOJN '.2o 2.N 2f% NC ONNO aNOON 3.-0.. ZNC-.N3oCNaN2NoNqNG-N; o No cwfC2Nz

399N 31'N.33N N33- N 3.C N ." 02N33 I NW 1 czN. 3cQ3Zc N

cr z- oft. 7 3 N 3 W 3 9N N Jt C W N 37 33 N N C N 3 N~zPNcZNC3 2 %JzNN2.OaCNNJNC N: 4QN~CCSCo NC.

2=CC NCC4%CCft C CC N C C N C NCCf.~ cC-NC~tZN.Z. C2 CJCC.N CN CJ NC NN r.Cc-NaC-Ncc. cz.NC3'NJ

3 .NZ.9 '.j ~ ~ ~ ~ ~ ~ ~ c 2..O NNZ O N C N CN CNNJN O N NczNczC:ZNC3NnCC-nC nNcC-OcC-CCoz 'NCHOO C-Na43=3C NNJ

2' t3 N3* J 3NNCONJONt.NNCN'N ONNO W N3W.J~t4C NN0c. aft3N ~ ~ ~ ~ ~ ~ ~ 4 33'20NCo.CO' 2CNCN$CCNCfl.JJNJNNefN NNC C

Z%'23N.3'.C 3N CC N 3 2# C fO NJf C CW Z N .5.3.4-* NCOzZr z I t am aI aCNCN OQNC N O2N O C NN 'C N C N C-N3C N cNN.NNz~:..NCr~cCC NZ:c~.:3.C N33'N c~ No.n cJNNCJ~ 3.9 C N.30.N 3 JNNC.Nq ~dINC ,'9CC ,2 NNO-NOJNCCNfJCN

.t ~ ~ ~ ~ ~ ~ ~ ~ 13 , 22 C.f1Of C9.C2N3A3NN21NNCCNJ

3 Z J J! 332 2 3-3 1sq; -2C ' J 22~ n: Z -N3C!.Z.3 C2NN2-N2-NCn.N CN.cJ

* ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ r r- 9N 3'93 N 3N 3.'N 3 N2 3J4 rN2'NN3 .NJ7 3n3* 3N.N.4ZNNO....NON'N ~ ~ ~ f NN C-lN -fl- N~lC~* ZN ~ NN JJ.

Z.3 ]ZNZ -N Qz.z m

.1 . I

* I-=

W3it3n lN-ZntI .. c±4r r Z-z = r, zr: rr33W-N z

zl-zZCW =CN 5Z:5 C W3z nN CNN34'nC3 XNCmNC39NCNz~Z .. W Z n .33.4 S-=W -S3l 33 W N N C, .3 3 3N3 Z'4 33.4339 '339 433 '.332 33N"33N'433993-N 3.9.2W 33? 3''

-34 C N.zN-33- 33 N Z N 3 9 3 -e N: N Z W 3.-

J..493N934W 2P~..ZN.93N3ZNC23lNz? 3.4 Z93NCN* ~ ~ ~ ~ ~ ~ ~ ~ ~ Q or'34 334 3' W3-3 NN23.433 3 r 3 72 ? 34

'4Z .'3 94 3'Z3 f3 N 3 N C. N 3 73- 3 1 3 '3 N 3 9V J fC nr C W 33 W 3 3 3 N 3 0 N.3I ~ 3 - C W ~ Z ? 3

-Z39 C3N CZ 4N3 '% %3 3 4n N .Z W 4 3 n -CN,ZW'S C I N-c

_C=, = 0 .-. ZZ'4Z 3.'3Z -3t N4Z34~-.3 W4Z

N:3 t~c N~3N~Z~e C~ N S-zNN 3 .413- Z ZZ 2 nCNZ.C

'ZZ'CZNN I ONN 00 ZZ N332 -N3933. Z.4NZfNZ'4 ZZ .'3 " .-t jZ.%93z Z -33-a Z .CP4N33 93 r3 3 cz " 33 94 ' , 3, C3 N W4 C N -3

%33%' z I I - Z9.4 z Z. 3:.'.Z- 2 1N.J~%ZZ. =.3 C N3.1.2 r c Z, -. N

'433993Z .c -r o N.99 3 N33990,rI Ncz9 f.3 33 N3 .. 30 03.4t393

0j Z" C4ZNNC. znNZZWCz- CN.a0NO3-.CMNra'.03-23w, N W9a3Z.

al ZNN Zn- 0- 3.- ZN l 3oftCNNfn.C -30 33 -A33 9m CZ 9039-C,;.433.4.4339 a39 N Z'3 N 3 4Z NN 3 N 4 Z ?3 ~ 4 2

039 39'Zr.-Z N3 .3n42n43Z.eZ~..tZ N'43-3 9-Z:.-33Y 33N'4 3.49 3W433 '4OZ003' 2=00"93. 02~ 4334.3 N 433N.C39 33N 394039 3 9394 3'43~N3' 39-

'33.4Z.9..9.4 3.'409939'40 % 0993W 33? 034CN~Co9'422-02. Or zwn :N3- 33 N3 e3N0.n3n43.N3 N3

ZNC 394 N'39439-3NrCrN3 -CSNA9 20. Or3C z 93-.C3.-CC -C3~-Z--C W.4ZnN : NC * C WN Z CCQZ'C t-

.339 ~ ~ ~ z '4 3 -C 9 4 3n 3 n-3 4 N3.. N3-n 3. C 9

0~ eo N 3-21'CNG3-29'0-43N 0N0-o9C3 Z 33NNQQN03N.N.N mN N Z 3.. .4C0I@3NC -P900390 NQPIlQQt4Q9rgfQ3240.0C

'433 N0394C3'-03-g3Nt3;943.43r- 3..'og.''4.3.-.aW-0r.-

a ' 221a3 j443NN 349 3 3N-z3nz4a 04'a'400a a r -. 3z9=a a Z .2 aZazm3-NQ a -aa zz N-CZ33Y39 39N3NNCN9CnNCNNC-r39r03N~g.4CON90Nn3~~~~0399039~~~~ r3-.3 N~..~C N Cn C-9 O A 00A

'33*903? ~ ~ ~ ~ ~ ~ ~ ~ 1 I4ZNZ N Z9:9 C~. 31:VC393. CCr C

.2z

2~t::t33 t::t~t~:~tzra t o tzt are rZ tC t .a-- NCCN CCNCO~NCZ-4C fl-C2nOQC t Vcn-OCN O ,VM32

= ?0 == ,.r3.Zr2. -3NZN-2C-432cz- ZZ ,r=tSC:ZZ .3.N :

t ~ ~ ~ ~ ~ ~ ~ l IC. IN CC CC IN a NtC.S33, 0 tS~T3'NCN NC:

Z Z IrCZN r c-rWC zZC, CCW CNCC Z E C Er52

7, z ZC:NCCNZ2NC CNNC Z 22 Co NC5,N2-C :-N ,C:Ncl 2

::CrzCC2=:"ON

.. a:.a:,%CZOCNza-N.ZcJNNCzNOcOO20 N.CcNOOM-N NZO

N- Z.. ICQN ZC -- N2 CON N 90NNQZ-NC ZN O NONN

N aCN~JN..ZNNCCW0ZNc ZNOCNC3N03 N30NNCNNONCN%~z~~~2,C aNC W C4 C N-C N S-

NZ. NZ2'.CJNN z:N-NNNO -CN-C NzN ZZNC NCO 00N

N3 O ,.. C '. Z- I N . NCZNNC I N o -CN NC, Z NNCQNZZONCCNO QN ~ ~ ~ ~ ~ ~ c -C 22 CCCN NC N N C2 N S -O,.CS CO C 'C N.

Z; -NC t z Ncc. N3CO N0CNCOC-CN OrC CtN ' CCrNCoNCC=Na= r a.No .;n.:a.C W2OZ Q~N:ZN zNNC CC NO ZMONOZ-N.lO

2V Cr NCONNO 2:.'CZ it map CNN co~c C2N N.CNCNCNCC' CNNcZN'02 ~~CN0N-N NC0NNCNNCCNCZ--C On0CcCON-CC=CC- ON

-Zn.~ ZCWCtNZNC -ZNZNN a QCNNCCIC -. t0NOWNC==.C.rC= Crp C~r.~~~lO~~fC-CtC NCitNCN nCNCN= OZ,%Z CN4ZN0 .ZrCC N.CNN~N

4CZZ NC3ZO..2NN2..NC0N C0~.Z~d C~tO2N CCNN C-.lMaN:CNNCCNNC~ anNC0~00 N0CnnCS= ecc .N Z N t C nN n

'.02 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ f I=C CNZ. O ~3W-3 r-O N CNN CN N C NN O- 2 O NJ- a a N ~ N ~ - 3 - o t . . 2 N r 0 0 l . C . S

'03 FC N C - C C S NN C 'Z l Z.2 -* 0 C N C N,'CCca =3tCo CC Ctlz OzW~r CC-0NAO P.- .CNN d"

PC CnNCN.CC- C~t CC~c:N Z~N C~$CC~C:N :CN CotNao

2: ... CN SZNZtWA2 3N4 3cN 22'.' aU NCN z~ N2 NJ N CNW ZZZ ' V Z Nz 3N4:-Z N3."ZNN-wZ4V' r3N z ZNZNCNN

2CCN'.02i..nCC...NC~rrSNNCZC CCACZ WCCSZ Mwao- 0. C N-. C

N ~ ~ ~ ~ ~ ~ ~ 4 C0nCN 1~t ZN4iZNt 44NZ-E ON -'yC OE..

Z-, ~ I -0= 0 oa Ic=vcz.sMI N

rj 19

*z cw *:

:2 I ~ c = m c _N35 7S, =2W C3 W rWC=52rc cB..c Z"=32lN r53 .

eec-cc..- c10nccc~znC. eecnc nc..o.ccZccs-c ncc

2' 5- 2W, 22?- 2 W232 N n 27 -=N N5r I 52-32N 2% "==rI=22N O O cN2N2N cZN.N Na -NN03Nzm cOSNNSS .CSN 350Wj Njz roe Q cnco oc-ca Qzftar~ Z C O srec0

SN ~ ~ ~ ~ c -cc-2: 'c5-z C=-2n 2-3N5czftoz -C5--65-r 2--:'.-czrcW~~~~~~~~~~~~ c3, Ir5N32 2er203055205N'S W5?

Z'.N 2NN3N2Z N 33 N5SN~5 N S N5N 2 N 3 N 23'. ~N:w~~~~~~ Q:'5 N:N 3 WHTCsae s-.sr 5r l 53'2

5'c?:S'.N ~c'Z~N 2N".22N0274455's ...225 z25z5N '. 0 '.zZ 2- 2'2-2 N ~ 2 N z- ==IN3 GN 252 N N5 -

'. 'N'.'.%Z 2WN 3N25N25 N53 N5Z 2 2N~ N ZN.2N c5ee -c=,ft5N-Q CSNS-TN5a2c.5C 5-C-C-CZ-.'.SC~~~~~ ~~~ 02:=NZN.2 SN S N 7 a1eCeCn. N 5 N5.C N

- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ - IQCC N c I 0 N S~ c~ N r-cnC N C N SN ONN5E~N~tNC4,430fl2S NCCN rtc3NCeN2NCN3N

zz-CCNNCS*-NC:NQ cm NCcnnCC cNcfnCNC3ftczNmcO=^moorN CON CcWSNc m '.2ecSecctcOncnmcner'ceeosNQ~ Ncn'eaS.~~~~~~~~ ~~~~ zC- S - 5 cN ~~ C5~C W 2.e5 N 52 e5

N s s -- e s s n ~ s ~ n e r -5 S N N C' a eZcsrC me 02 N C e e 5 n C W 5 N C c - OIN.-42''Zr -es~eON 3N S NNnecn5 N 2 N 3 -2wlN.

- .2 'SNN 2?.3 223527 25ee 2NNN 5 cC , r730lN5N43'N2ZN5 -3-. NS-.NZ 5.'.3:eN03~ee3-eI0M ONSO r0 a.0-e 5et57NZ.NoC-

IN ~ ~ T zcJ N Z ' 2 N~ -00N 2 N C N O 23 0 M 2N 2 ? 20CZ0tze r :2-122N~2 NN W'zz ? 22 I2Na 25 5ee.N52N'402." 22r4aSnNN C .sncZN~~ Ne~nc~ojco~e0N5~Q N

S N C Cz C C ?.C N N C C -C c N N C C N cWft c. n c n N cAec re cs N rZNNe.CweffNtN^ll^2ScN nceOcceO~c~~0.l2-22N2-NONO2-O-N flNO- n.cenew Q c-flO=WflN OO--0-ecsc-e :@-cccncc nennc NanaeNcncIanc-c c-5NVN r4cWN2.e0e =co. c c ~4NCNft~~~cncr4--r r zznrIr zrm c=mmcNc.OO NONN O NC~ccc-

cm m a nWC2=1rcfcS%nc2Nccc--Nccn c2 ccc co-a4 .ec cr ~ ~ ~ ~ ~ ~ N '.seecfrccccq-cnc Z AccC..= ce-oo.30nccq c-

5N35.eC nnzrc rocnc-rncce.oc Q^0 ccc? scocnccnn ccn

PN,5, C2 e~~c eec ccc4ccc Vc wzc I ~ ~ ~ n c -to-~ ~~~~~~~ 552 m.2 n c n "s~ c 2 5 ~ 2.cr 5 r c N ~ ~~ s n s

- 0AA044 52N C442l 33.SN4'N2 2323~3 2N5 N5A 55*'2n~~0r33 N20S720m5023N22N~cn0-

44

V am a

-N~~ ~ ~ -- ZIZ= ZV z2n g

ZW.2r.CWr32-CC W22W C2C 2ZN C2WW20? ZZWW~flC~flC~i*02ZI=ZZNNCCCN CCWCNNCNNNNz I,=MN :cCW.%C=fNC=fNCCM CWNOCNZNC~lNCCNCOWMCCCCNOC.900WNCNNCC NCflONCWNA

imN

Z N -42,--2N-QcZr"a2ZN22r2-,ZJNCO-22aNO2.. QV=%N::NZC:N-22--MCNNZ:2 C RA O NCOV;~CrZ :2M-CCN CON

NCO,?NZCcc2 Nt n zn'.ft:z NNC25.O2NNZNW 2N 2 CNICZa

Zr-:: % W ZZN ZVZZ W9O2zN ZZ N3NZzr. CZ CZCCN 0,O

ZNZZ =CN 22rW2",= Z NCCNC.z22NCZ Z rCCNW0 22-W2N 33.19NCNz

ZWN: ~ ~ ~ ~ ZVm rZ N N NWC=-fZf~r2l2 NAN QzNN ZZ.'a,:-%:Nr:NN:-~:N2 nN :Z9:NN :N ::r 'aN:: N N 2 z NO

'2-l ZN. 4ZZAom QNat NNzI- Z

ZN::.-Z: N::NC:N :ZNNZN:NN:-.CZ 9c:2CONN:NNZ NOZ'N=-rNOO'.7C3fAN2 ON Z2vN53N S9NNZ.T IN2NZfNCNC NNZ'N z-Z2? A. ,9N~n O % MZN ccNC9N O'49 ~ 4

j2?o4 c . 222? 2 N C9C N24N~l2 -CNZNCEQNO zCZP 1 NOON:-NZZ- ZNa:-aONCC a ccz34949 c 23C22.NQ 2 NO9CCNNvC Waf= rN Z r Z N r O O N O O C N N C W N O C W C W O W N C N Z

2~N:NC .Nz CNC9NC N CCNONCNC N.CNCNNONOOZ~%Z:~2ZrZ:9-2NCC9-C ~ SOCSNaN-C NN22 N::4 Ncff O.

2v2.-0NN:nC29-o n..owno.-MACNC NCN aC

I or NN2A NCZCZ N mNCO N C NNcCNNO N 2-NZ N earcc,":%:4N-.2:AC:NNOcznNC2z~NC~nNc:NNOcN0: cC49NoNNOC49N ~CC,9 Cw NO or

2? N2 9-2 2 --2.2 - . 3.NN QN22N 44 =,I ZN NC N'4 Cc,' t"9- . NON

NNCCW.CC~fNA.COSf -C~ft.NNCCO'SONOa CNN CftC l40NC &4Cft~b2NN 2WNC r NONN ZNOC NC NN C'9NOC CCNCC-NCC9NCZNv z-NM

ANNA-,C2NNOAmC2CCN aCC . !CNC =AV ONN CwCC-N CN. ;zV~qZ NMcC z , o-NC N C N C NN C C9 C. AC oMNCCN4-MONNCCN~mCCWNCC

ZNN29NOWN2C~22 N2N. N2 NC2 -CNNCN2N N C.=ACON.CtwC23N-CN 2CN CCNC2 NC 3JN CC NC2 NCN MANNC!:C.N =CE 3CC

NtC C2 ~ CA.CC? N C InCC N N C2.9 W C VNN C O aN C 1 flCC N.9CCw N O N7NM C.2~ C.NCNCNCCCONCWOCCA CNC.. CN2

- ~ ~ ~ ~ ~ ~ ~ ~ ~ I r:rC n CQ:'2 n~::N'c- on c~99:Nnr cav' z

* S

a *, £

-; -:

N32N0NV 32.422N-::NrC2

NrJ2--222

-r 2MC 2CNCN.~2 2

'49 239r :2

-N:2--22

-N23--::

2NC2-NN22r422N~tN2

'4NJflNZ 2

.NN3-N.2 2'C - N NC: V -c c

NN..3rNZJ

N~%~9NCZ

NN2N2-N::NN32-'N3.N02'-.329NC2NNJJ...33

'~2tN~J2N4Jt42NN~VN3 2

N CC'. N CC VC C'9'C2.9NCQ9'.33-00-0.4--Ca--co

~4C2NN229-22%.20NN0203

N 2 -rag.0 JNNCCNWC 2

'4.N2 2~9t 24~443 2-900

2 2~C 2'NC 2t~CC- N 21W::N 422.9? 24

N 23eV 2 3N.'22flV20

CJWV22CW 2nre.4

WN'~.9WW 3N Nt Vt~

-. 2..r.-2'9tr3N 2-r3Je Nt 2* NW 2

tN:NSV:N N. . 2 .NW9 2* .S 2~jtV a*9' 39 V2:22023:

* 22222222323

2 N N N N N£ V V Pr V22

I

,4

aao u a a a a a a * a a a a

o.o

taN 3 N r N r m N

I -

- ~~ ftN m -t a4 aM

a U..

4 I

21 AZ ft

A~ :1

2.I 0.

4 ~ ~ ~ ~ - ft ~ a ! 0 0 2

* - v

40 a . a a N 0 - a 2 0 9 0

41

J4a.

a a a * N a a a - a a a a a a a a a

a a a . * a a a ia a a a a a a a a a a

a - '4 a N a a a a - - A N a a a a a 2

a a a N * 2 4 2 N 4 3~ a a a a a N a

a a - a - N N 2 - 2 - - 2~ a a a a a a

A .i A A A A A A A A A A A N A *A 4 A 4

4 a a a a a S a a a a 4 a a a a a a a

- * a a a N V V S ~. S~ S * -. N N -

N N N N N N N N N N N N a N N N N N N

- N-------------N N N

- - a a a a a - a a a a a a a a 4 2

A 2 '4 8 S - 4 ~ ~ 2 t a A - - 5 4----------------

* '1 S S - A N N 4 N 4 4 V 4 2 N 4 44 S - N *

A .1 A A 2 .5 A A A a a A A A A

~ ~ N N N N .N a *a a a a - a - N- A A - A A 1

- NNNNN N N N NN *4 N N N N 4 N 4

- r r r V V ~ a V V

4 N 4 N 2 - 4 A N 4 2 4 4 A 4 N N 4- - 4NNN S N 4 4 4 4 - N N N

5 4 4 * 4 4 4 4 ~ c C a C-------N

N N N N N N N N N N N N N N N N N N

V - V * N N N V N N N N N V - U -

- V - V V N N N N N n N N - N 4 N V N

2

a V N V V U N N N N N V N N - V V N V N

1.

* V * V V - - N a N N - N V N S N

S V - V V N N - N V V a a - N V - V VIa2- 4 5 2 4 * a 4 N A N 2 4 2 N U 4 4 4 5

- .' - a 2 a . A 4 4 5 4 4 4 V S S A 4 -

- a a a a a a a a a a a a a a a a a S aV A 8 5 ~ a - N a p A 4 N U P 2 N V

a V V V V V A I A A A A A A A 1 4 4 4 4S * * , . 4 ~ p a a a a--------------------------N

A

a' 7. ft

4.

2 a a a aia ' a a a -N '4 4 .N a a - - a . -= a 2 -

-, .7 A A A .1 A1 a1 i..7 4 A A A A

U A 9 A A S U U a a a a a a j

3On

ft N N N N N N N N

a~ ~~~~~~~ A as a - a ' a- a -

s ~ ~ ~ ~ ~ ~ ~ . 4 3 4 2 0 S A 4 A V 2 4

- - - -- - - - - - I a - I -9 - - I

20 A7 .7 2 a , N 2 .

3 1 .41 -1 it A 4 A A A A. .

- - - - -7 - - - N N -4 N

NN N N ft N4N f N ft

V~ ~~~~ V 7 N V

m 161~ ft M ;r N 0

- a,. I2

1601

Z::o A N C Aa ft Ia A 02~ N N '. fN t . Nft N N N

AN. ON M - - N 1 A C N IN N

- -7 N N 9 N N N -.4 N .14 N N

- 2 4 4 N 4AA A A

2 a N N N N N N N N -N N N N N N

r ft - -N m t N ft t A A

- c A NfN

~~~ 22 N N N N Nft ' -

%..5 ft ft ft f- - - - -ft ft N

2 X: ft ftaA

- -a0 0 1

A

-~~~ ~~~ 4 2 0 2 70

~ 4 2 2 A 2 zN z 2 z

I S

N N N -N N

i z

NA AN N -, a a N

-N N N N N

A NzNA 2 N 2 N N N NiA 2

S I 'N I

- I I I Z

II 0 ' I I r

2 .4 A 2 Z I , I

4 ~ ~ A A 2A 4 22 1 4 N

A

. - '7 2 a V N " - 3 " - a -" ' -' 3' ' " A N

* * S -

-- : a - S N a a N V 7

- .. S-• .m $ S"• • • • • $m ,. ,.

- 2 3 - 3 N - N -*

. ~ S S

7

7

, -, . - a a -, ~ , a - -a a aI p

' 4 - -, -. - a a -, a a ~' a- I I I I I

a V - - N C V - N N N - w aS p I. -

- - N - .q aa I I S I I S I Sa

4 4 7 7 4 7 7 4 7 S * C

a a 4 2 2 7 N a ~ - a - ~ aN N a. N fi N

a a a a a - a e *1 ~ a.~ ~ CN a. 7 N N

a a a a a v A 0 - a a w - aN .4 a a N N

A A Al C a a A d N A A a A 0a a -a N N N N a N .4 .4 a

a .i a a a A 2 N a aa a a - a a ~ a a a a a a

a A A N A N - A N - N CN N N - a N N N - a -a N .a

i ,a AN Na; E~. N j~ C a C N a a - A a . aa a N -~ a N a a a #4 N

a Z - - V A - A A Ca a N a a a a a a N N N a

A N - N A A A A A N N a aN N N N N N N N N N N 4,

a a N A . a A A A N A C A Ca a * N a N N N N a a a

V V 7 7 7 7 7 7 7 7 7 7 V 7

* * V V V V 7 7 7 7 7 7 7

4i 1. t N - N 4, N 7 7 7 7 7

A A A A 7 V 7 V * a a a a 0

a a a a a a a C a a ~ ~ a b£ a a a a A A A A A N N N a a* N N N N N N N .4 N

- S S S S S S S 8 5 A A A A AV p. r

* V 7 7 7 * V 7 7 7 N N N N N

3a- 7 7 7 V N a N N N N N N a a

2 2 a a a C 2 a a a a a4 7 a V V V V V V P 7

L'.4.. 3, 4

I-.

* I.~ .1

-. 4

APPENDIX D

Brief Summary on BMDP Statistical Programs

t

f

Sample page from BMDP 77,Series P ManualHealth Sciences Computing FacilityDepartment of BiomathematicsP2R School of MedicineUniv. cf California at Los Angeles

13.2 Stepwise Regression

P2R computes estimates of the parameters of a multiple linear regressionequation in a s.te.r ie manner. That is, the variables are ent-ered (fomvardstepping) or r v"ping) from the equation one at a accor-ding.o any of =L'-- s riter1a The order of entry or removal can bespecq-fied entirely or in part. The regression equation can be estimated withor without an i~t~erarpt.

S RESULTS

The Werner blood chemistry data (Table 5.1) are used to illustrate theresults produced by P2R. In Example 13.7 we request that a stepwise regressionbe performed with CHOLSTR as the dependent variable. Only the REGRess para-graph is specific to_=. The remaining Control Language instructions aredescribed in Chapter 5

Example 13.7

/PROBLEM TITLE IS 'WERNER BLO00 CHEMISTRY OATA,./INPUT VARIABLES ARE 9.

FOOMAT IS '(A4.5F4.0,3F4.I)1'./VARIABLE NAMES ARE IO,AGE,HEIGHT,WEIGH?,BRTHPILL,

CHOLSTRLALBUMI N.CALCIUM,URICACI 0.MAXIMUM IS 161400.MINIMUM IS 161150.BLANKS R MISSING.

LABEL IS 10.

/REGRESS OEPENOENT IS CHOLSTRL.

/ENO

The Control Language must be preceded by System Cards to initiate the analysis by PZR.At KSCF, the System Cards are

P/Jobname JOB nooo,yourname/ EXEC BIMED,PROG-BMOP2R/$SYSIN O0

The Control Language is immdiately followed by the data (Table 5.1). The analysisis terminated by another System Card. At HSCF. this System Card is

E/-

The results of the regression analysis are presented in Output 13.7. The

circled numbers below correspond to those in the output.

Q Complete cases only are used in the computations; i.e., cases that haveno missing values or values out of range. Therefore only 180 of the

original 188 cases are used. All variables are checked for invalid values

-399-

J4

APPENDIX E

Example Output of BMDPZR StatisicalProgram (Multiple Linear Stepwise Regression).Shows Output for Williams Air Force Base,Field Maintenance Squadron Data.

- If

'1 I

0 I

bw sm

.411

1. W* aI

119.

af .40 ex" I* I~~O I ~ I I

*1 0**o.meo

04 WI.wx= 4

46 a 46 4

-10,

* I .

wat

&-A

. . L

'a'

*~e

i~ ... K .1

*.. .

.. ..... ,

WOO -4

40!'Noo ..... * i 0

a. m .a400..o~~ .v T:~.~. ......v~a . Iv .....e .. ... v.

uts a 0o4A * ' 0-a1 "AP M 0 -- " -0-4 0 0.4.W4 0.40.aewta." ft: Mo.ae C@C00-a-4-0.0 a00*9 vl.^0m -O9 -----

I M.a ..

pa".InI

w .000 a ec"Coaao caaooocaa oaea@00000000000.a ~~00000000..000 mPaOOO4 44.PI d .a - - P . . . . . . . . . . .v a v a ~

.00 00000010:eo 0400aaao00o @800000

-"Gaao 0=000000000coca oo ocOOa c aOOO@000a.......................................................

Ae~a mama

X_"U tO 00

a - -

4*.ap. -at0-4,;4:' 40-0 vt- 00 0 =41f ...aviP *"-~

.. *Moo 6000ftv4008a40 4P4a*44 v '.laaad~aae

w.~~~~~~~~~~tltt 0.f #*a f00.a vu v.-0.oaav0000--00-@

Al .v4aa .~..aas0a a004Neaaa44v40- ~.j .................... .................................

%t OM ii "- 1~ nt. 11 ". CMI

.71

a.6

'ItI tI

10

.... ....._.o.... _.o...........

4 - U 4-o00 4.',0 40 -4 0 0 0.O4'4 0 0

Go

@.4 4e00900800000000000 ,r-, €m -O - -0 0

v *...

owe

a * * L g O*X 0 0 am ON

o00408 14* O0@ m 1W

- U **4-f4-44----44310

'A 4 0

1 ... a % 4. -A

"4bmu-. II4 Vq

*a--- ;

4 .f . awe .,

96 - -4a ao-x' 8H

am am

Z 46a .a 2

aa

IL~~ -.4 f V-!*

a

00000002M *m a

hi 404*44404.4404

:, .00 000-0A

P-

a0 4

-ft W%

wv.4A WeWe"imf

lh

0 o w a

A-4

i~~~~' i~dqi Ott ?vio- i4P,: !-i I

f 'o

16i

Z 0-e fo -eo ee a -eo * 1e

;W

ce

.. . ....*4# . .44...~0... ..

- .. .I

an%

%mo ,Cob,

hi *P

s a w t

i~ol "I"""

.49 I .03 -a

a so W

40hi~ib.4 *Z0~Un-e30Oo

IL4 weno04 hiM 0,Z

8.6 j..w

Is gA

01.00 .. v O

W4*^ COW CS.CWW :::

# e i ! I $

46

,:*

W-0:

g . Idas*

xm4

wa4N n

U4. .

.90

:,*o wo 1

me a4d 4a i RI

A- u. w @ .......... 5 0 V @ 40 O W O Wfl VNfl ** O @ N . -- 0 CV

00 ; WW40 0.0 -1

V a044O40.NVUPt * N-OOV~0 Nof O 8 4 0 ~ 0 .. 0 @ @ e 4 . .. . . . . .. - 0 0 0 .

..ew........ .... 0o0........................

ov

S.a

~ W@*4WU0P~* SOW 0N0

0 40404dp.V004

oft-

-. a

#40000 ~ ~ ~ ~ ~ ~ ~ ~ bl .01W. 480@ 9 o8* a*0 uN.@.4S4 8.

--190 u ' 0..9 OUVV VW V .

40U~hdZ~V~t16- 1b20U

a a

.. 0.0 0," 0 a fI0we 4..

0

0 w @ftwvmOwA4.~w @ -. 40 "0Wu.c a"44O WWW0 .

4 "-a

& *-4 -!C

"0..

as.. a

0 -'

Vol U 4.. V * @ 44 O P C W 0. 46~. @1% W~U444P 4WW 4P~0 @440W

* 2 4~UWV~.@4 W 4404W0.W00. WUW@WW0@O. O~ 4W@ On

e. 0U *40...0*UUW* W4 40W4W.. .1.44 0-@ -~.04W44IW

- 4 P~W44440.40.0*0.09cow- a *0

.&

0-4 zo".500

0.

3 a

.. . .. . .. . .. . .. .... ..... . . . . . .

7 -

1.0

- .0av

24- 0.4- @5 ply

o . WOWo"e *.a@ .N44. e.w4.. @.4....444

it T S4009 ae94-ra W4O5~ 9-4.V~.@4Ii 4 4e 6 63.@ - 4 - - i 0 4 ~ 4 4 W 1

444444M4A4W WW W WWan

gas. 24M~vd65

ma. " .

4 4c

ft(ft*A 42 400 -- 0 -

040w ft*- 4 N . 0.4r~,..04

IS . . ... . . . .to s

f s

4p-- @a-aoaooaa oao

%.4

mawZ

0M . I .... a 4 ua Z,~aZ &.~

8. t. 0

46 a

a,.

a4 *

4A A 1 4d446.0

o'sa. 6 699

.. . .. . . ...

-j1

-00 R06- 4l4-.*wc.4 O046~ V4V0~-#.

fta . 3.4

a it

7" M

,- ".4 1*4 !Am*V.l49.4*4 644*.44.9.- l I. a. . b04

of 32 61o

1.

44.

Iwo

W - 1

W

ca I4c-

WM. .4 "vv

a4

.0-4 o .

Me 4:a 4

* 6 Be44.2

.0Wt. 3. ,

a.4 &-f

N.As -6

a 2

344 .164* os~

a

a49-. ~~ i-- a 9 W - "Odor - -- --

i4 M O.!.A : ..

Wv- C8..4 44 P 4 4V.@ * "4o.W 4e t

- .

19

amm-4

§ @44444444 4 .........40--4 4 . q..........

ZIAI4 Ib.

MOf - A.40

4' @3

go 40 4 4 * 04 4 4 4

44 4 4 0 44 4 44 4 44.4 4 444an4 4~~~~*~o ** 4 O ftno 4 S 4- 40V ~ ~ v 's04 44 4 4 0 0 ..aaoaa-e** a 4Z:% -- ae=1400- - §§o oe* . 4.4

Z..

444444440e w : 444@04440.4 444-444

46 . wW~ on.43 1

A -- w

Z 44

A-4 ao

w* 4x WWA

a .9ft ac Waft 'a 4

= W009.o

a.a "V O0 v6.

-U.O

..W....... ~ 0. .. .. .. . ........W~ . V.....

- a

a =.0

hi It.5 aIW- mo *1 :11

.20; *-

00.5

ale

A.A a-0 h

SO. 89U1W

* I M

Is a '~a 0 :=

4~~~0 4ft.4@ @VDO 3o"G CW@440 M@4@0r04

.. . .. . ... .. ..

a

a 04.944000 a 3004V 0P@.,44 a*4

- 4 ** 24 0 204 V. 04 0 ~ W . @........ 444440 *44 4 W4e 4S04a4P .

- . .5

-

X..1

a as,0@O*4@44 ~ ~ ~ am 404 4P4vft*44 4 ~B 4

44 A . ..

.0m5obpbo

ww 4 w aVf-~ ,

ow ..m

23 41

XNA 4 4 xd~

-- - - -- - -

- - - , -, *' o

i ' i

lb.

W- m w O W O. . ..4 ..1

W ft...O a..i € wa. ,I - .d 0¢ *40 (IO IJ. eo0P

I- ~~ 04m" *2=0 oa n

4-4 .w aA

.. .. .... .... ...... ....

a a -

* C~qWftvi a.odviceo O m . 2 : 4 Pt O ae~Efq.I t'.4y oan tv* Q Of ~t* .4d

- A

-°- "-°al. I%" -° ° °° °°-.. ..

Go I

z,' o- %P .. ..-..,.

S. -. ",'";'":~ 4U*t. r

,., O.. -. 0.

* . o ,~ e 40s e

"00 6 IM^11

41 4

I0= lo a

moft -00 4-

0'" ft#" w o l

aa

o.. soS%.Wv mA

a=0 v a OC3A.1Aafo

.4 0 .o Oad

44

a"a

1. mwat. ~ "

ce I

007 1t4 N~00~

IL

04444 4p 4W44#40600 ..444e @4400-

19Z a

-9J330a.06A4Jd 40 4j4j.'03 3 3 -2.

1.~WO r.4..*3 _

0. i

*5 i

.1 .

I h

we I

Z.

aaW606

*d0P~de4Phiwi*40 I hihi ~40ihi4W440h00hih9 hiW. 4e4-hig~h44~hi 499 -~~hi*~ p.4

0 *4uhi44~000CW43 Go

- @~0hii090i44i~Oe 40~4WN.4 Wti

- .hihi44.4444hi660 ~Poo."h'hi549

hi ' -~ I I;a *0044hi4 4*-.0 .

444 4V 4 ~ *1* 4V4.

Id 4 *4 ~ hi8 44jq

aaaaou-a---aaaaI UAn

MA .4 X ILWV.4w

I A5

WA .......

* ---- ---- VA

A 000 W 4 '00,0SmO0~0*S

4pUP.6 00.U ~ mm -A -:om

-~~~. . ..... .. ... P .. **

ox2..... . ... ..... ......................

Dal ~ PS 0Sm.0S

4 * -- vI

C.5

4

.z-,~.zgm.. .3..

wo- 4 -. 6 ---

!.* We hi ~ fi " 0A ,

w '44mP P WU= 0 PP UV S

hiXmPS0USShiA

4 . . ..

Zo,

a ~ ~ A = = Vm....awla '0 A~ *-wieap a " a

"1 -. 0 .ea N. -04 em a--a * -. AV0.0~

0*0.1

0- ac a ao ~ .4..Ia0- .. ... .......... ao~

-aavwwaww~eaa wwvw

........ .......

a a A-AHt.it@14 a

o 3M

.. m..0.4.aaaa.....Osa a m *a0e. . a in

aftoo-a-aaaa404-a4 40 me a-aaaa -o 14P @@

am, so a". **Iat a-

a 01

a,,

X.J "a

@00 a

4- M- ."007 -w "new*

W.k a-qaWOa& 4 w' @4pC.,

* .- ... .. .......

4

.....a4 .......... ~ o * ................ .....

ma

a .........

'N 0'

go 0!.-~.4U 4 000 4~~aeooeaaaa-aa 4 *" -a aaaaaaa e44P ~ ~ ~ ~ ~ ~ 09 0.2P* F 4 ~~ W~ -. w 4

awaan 49 4

a~ I.a"I&low 10xa.

a %sm

a Rw

#~ )f .. P* WSw4 o-r

aa

.owwft..4.A46*

a .4*U44 00044p.4 0000-- 40-4U 0- 0

00 00 0 _ 0U

* ~ ~ ~ ~ ~ ~ %j 4044..44* 04044Z.4WO"08-4* 40W444446 40046,........... ..............................

4*544 ~ ~ ~ ~ ~ ZA 4004444.0000.-44 4 0

44

Iss

30.

* *o-vv =0#000w sb. -0-4W44

4bao 0

004 01 4of; 44 i 440*044

"=0 OO S S 46

a abo0

46,U

Q:. ............ o ..

C a 41M

000004 a - -a0=0= 004coo-cocaaa~a

Is

-. Z

&o"3*t *-44*aa..~af 2 qkX-%.ZO4&K

... ... .. ... .. ... .. ... ..

a .0"0-n 0 . DOO -n

* em0o 2S-

* Ca .

bi 0 a aw'

=..

a-%#2w

ma

.. .. . . .

aa lee4wUec eaa-4440bC4.00 U @

a,-5904 S

1.0 1.

--- '406. cowev*O * S f* o.

toV -- 44.Q a . m U..... ... 77

IN Go- .. sm.*44

I. I

o ~ ~ ~ ~ ~ ~ f .a .4......-...-....e- ,4 @i w8

* 43 4~ ~aP...eemowV

lga., MO-O

V40 -W A*e 4 -* V 9 4 ~~ I 4 ~ *

*4P1. CD- a 1 4U 0 4..-@0 . Of W

*4 a a-Ca

UP.. me4.4 .4 .f1UE4.6U.4 4.

46 4We* 4.1

J~dft 48 4- Ub"

- *. .- ~ ..... ,, ... itt.....-

4w

a.

aft %AGOa'm

* I

dab

hOk

.em

*0 a OMW--.Aw;---- 0-004Nftf-00---------A

- - ................. ..~l c f... . ...

*P w

02-

as

an. a

004 gad @ S@ V V 4W 0 V e

ea z*05 t 4

a a a

a t4ta~4*--O- ~~O 4O4v~

16.22-5

%I bo

-. 0-. - --- - -- - - --

I 'W

0. -0X

Ab.

-w

o ~ ~~~~~~ 4 2 44 f'44C4VW4m.~94

-~~w, a .. .. . .

-~w -A * II. -I

0t .

044ft440V~~I,. j8 0 40404f440a ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Z -4NN444s~ll- 4 ... f.. .

0 -4 F

UI'd

0

20.

- u -ft.-a ... =oWa..0 w * -44.0 04-

- 4 *M I

Z. - - -Q c O o 0- ,d ... . .. ...

'A 4. 4, 0.040I W.40 '. .

a- lowF

ft 0 F

I VA

4W40 ~ 0 0 44 44400-4V00p~~44 4444~4I VWW V WVWWV"iM

-J. imx .A awsaa

J f hl

ZZ..0!-

.. . .. . I . . . . . . . . . . .

a

*TTIll I~ I4 ~ ~ ~ ~ ~ 6 1 6i P.044 4 be. 4 04 iP

......5,@ .. . ..0 4 ... .......... .......

4 0441 --Z am 2rfcw

* ittll Id4.qW ,ha,. 4'4444 5

hi U *45, 5i 'P.44i45,i5,V~*4P 44a44g4*4 0hi O'45,444 00-45, 44~6440544 *05,5 iAJ a .W~~~w'4 W 44hi~4 04*044hiW440"* 4 00- OOUO 00- 0000 00 : - -@-OhiI .04w

44o IWW4dwf

.46 -ot k

0 1

46Im

a 4 a an

hi4 wee

. . . . . .. . . . . . ..... . . . . . .

6*6 41 111 N.

.. -.... . .. . .. .........4w.8* a N . e .-.....

SO.

04. .0- -" -. 0

WO . . .. .a.w ;we4~~ 9.. W .4.~4 v

*~W A .

43~~~~3 3 -4 wo24943.0. 440h

.J ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ . "a~iNqN 9 2 0Y9 4.0.N~9 fN40.44 a " id-a o -a a aa ~ a- - a- -

-Aj .jI

- ha 0 4 -a -

Ia S f

..1 . ....

oawl* 4 ~ h~ h hah *4 a .... a~e .. ... .- ...

4 U' -.ft-sU

oj o-AhachraIAa~4v$

4 ~ ~ ~ ~ ~ ~ ^A aaa-aaoo-aOO @ 8@8@OO I-w -@

ha. I ISO. &S * 16S ... .. .. . .. .. .. . .... .. . .. .. . 16~~t -

* we

WIN.

40* 0-v% *.ha 0 4

WO ... ...

hI 2ow I

IF'C86O -

.- W.

02b.0 0 10

- 0 ;:ccOOcOO 00Oto2

9 .....

4 0'Ut C A " . - 0 0 - * q0-1000 ftM42ft= , o

OW.

U--

A0 .

APPENDIX F

FORTRAN IV Program Output for WilliamsAFS, Organizational Maintenance Squardon

L

S]

3t

a

* .3

C.3~ .3 N N .3 a a .3 .3 U .3 .3 .3 .3 .3 .3. .3 -.3.994

.3 4 - .3 .3 .3 .3 .3 N .3 .3 .3 .3 .3 A 2 U

b -

a .3 .3 .3 .3 '3 .3 94 - - - .3

:a'4 2 2 .3 .3 S f .3 4 4 4 .3 .4 .3 .3 .3 ~ .3

.3 'a .3 ~ .3 - '.3 - .3 - .3 .3 .3 - .3

A .9 .9 - -~ .9 A .3 .39 .9 .1 .3 A A

S a C 3 0 * 4 a Z C S £ .3 A .3

* '. 'a -.4 N 94 94 * * '4 A .3 .3 .4

C-

.9 .3 -

.3~3

a 9 .3 i a .3 .3 A 'a N .9 '. . - -

-- A *S~*N.3.4%N

.3 ., ., .9 a .9 .~ .9 ., ., ., .,

.3* - - *( .9 .~ .9 4. 9 .3 9 .3 9 4' .9 .9

- . a r - a - a- a- a-

- .3 4 0 9 - 9 .3 4 -

- - - N '4 .4 94 94

.4 .3 .3 - - - - . 3. 4 - -'4 - '3 . . -'a . 'a

- . - . 4 5 a- P *' 3' - a- - 3'

.3' a - r a- P .3 .3 3' .3 - A - - -

N - 3' *4 'a S N S A A 1 A - - - * S

- A - 3' 3' -A 3' a 3' 3' .4 A p p a

0.3A 3' '3 * * P4 p * S 3' -, 3' 3' - N S

.3 -

.3 A --* .3 .4U 4 .3 .3 U 4 U N U p .3 U .3 4 .3 3' 4 S

- .3.3 -

.3U .4N 3 4 U N U A 4 A N P .3 4 4 .3 N 4 U- .9*

- :a .3 .3 5 2 .3 2 .3 .3 .3 2 2 2 .3 .3 .3 .3 5A - ~ 3' A .3 .3 '3 A 4 N .3 '3 .4 .1 4.3 -. 3 .3 .3 .3 .3 .3 .4 ~* .*

.3 ~3 .3 .3 .3 .3 .3 .3 2 2 2 - - .3 34 34 '3 9- N N .4 N '4 '4 -'4 N N a- N N

I

I

.44 -. -- - - - . -.-.-.----..-.--. 4---- - -. ~

, A .0 A 2 A. 2 1 A it - % 'A A .0 A A

- -3 - - - - - - - - - - - - - 0 - - -3

-3 - -3 -3 - - - - - - - -3 - - - - - -

-3 N 14 m m v N N N N

- - - - - - - - - - - - - - - - - - 2

-' ..1 .3 A A ~I A A A A 4

*~~~ A 13 43 1 * 3

- - - - -, - - -7 - -, - - - -7 - 7 4

4, 7 - - N 7 . 7 - - - - - - - - - -, -

N~~ A

- - - - , -N -, -. - 7 - - -

z z 4, 4 4, a a 4,z

21

- - -2 - - - - - - - ~

- -4 - - - - - - - - - - - - -

* 42 A 42 .1 A 42 A A A A A

2 A 2 8 8 8 * 4 4 8 4 4r

- C: a a

*o 4 0 - . C 3... . .-...

*oN.

' -

I' C 2 N 2 : -3

I

7 4 4 3 3 N - 4 N A N - - N' A N N N. N

U I

4 - - 3 4, - N - 2 - -. N 3- - 7

- U I I I I I U I -U I U I I I

N - N N V N - - N N 3 - - 4 N - - N N '- U I I I I

- - - ' V - 2 N A 3 - N C N - - r: a: -I U I I I I I t* I

** ' = 4 3 4 N 3 4 6 3 4 3 ~I 314---------------------------------------------------------------------------------------------

* 7 4 3 - 3 A 4 - - 2 - 3 2 - NN NNNNNN N N N N N N N - N N N

- 3 4 - ~ ~ * 74 Vt N- V - N A N A

N N N N N' N N N NI N N- N N N N N N. -

7 - N 3 3 4 A *N V N - t A - 3 - N 4 3N N N N N -, N N N N N N N -N N *N N N

N A A .% N A 74 3 74 A 4 A 4 N A A N 3N N N N N N N N N N N - N N N N N N N

N 3 3 2 3 A 3 N N N 2 A 2 3 N - A N 3N N 4 N N N N N N N N

--------------- - 3 A - - . 2 A 2 -* N N N N N N N N A N N N N N N -% N -

- - - - N * N 3 3 -N 3 N .2 N N

N N N N N N N N N N N - N N N N N

- C 4 N 3 N 3 N N - 3 A N 3 4 4 - CN N N N N N N N N N N N N N N N N N

- 3 N---------N N 4 3 4 ' N .2N N N N N N N . N N N N N N N N N

-. A N 3 3 CNNNANNN C N C N* N -, N N N N N N N N - N N N N N N N

- N - .9 3 7NNNNN S N 7 4 3 N* . N . N N N N N -- - N N N -, N

.1 ; N N A A N A N

N N N N N

-- C -, 7 3 3 N A 3 A N 3 A A 3 A - A C

* 7 S A A A - - 4 4 4 4 5 - - - - -

N - 3 4, .1 A V 7 N N A N N N N N N N

- - 4 4 * * 2 3 2 3 3 3 = 3 = 3 0 2

- 7 7 -' A A A N N ~--------- S 7 r r

N- - 3 A A A A A A -7 A A A A A A A A A A* 4NN N .N N N

* - . 3 A .7 .~ 2----------A P A A A

- - 7 7 3 7------------N N - N -

S N V 7 4 7 4 7 4---------N N A N

----------- 3 2~~4* 3 3 3 4 4 4 * 4 2 3 2 3 3~ N N N N

32

I

V

I

~4

- N - N N 2 V N - 2 * 2 -, 2 N - - 2 V

S I

* - N - - r 2 4 A N - 4 3 2 0 N 3 - N N- 5 I S I I I I I I I I

N - N N C N 2 N N - - - 3 V 2 4 V N N -I a I

- - '4 N N 3 3 2 - N N N 2 - 3 - - V ,N -* I I I I I I I p

- * * 4 2 N - ~ - - 2 4 2 2 4 S

- - - - - N 3 4 N .1 2 V 2 2 fl N N C- N N N -N N -N N N N N N -N -

- - 4 2 2 V 2 V A - 3 N. 3 4 4 2 A -N N N N N N N N N N N N N N N N It

- N N A 2 2 2 0 N N 2 3 -A - N V NN N N N N N N N N N - N N N N N

A A A N 3 A A A A N 2 CI 2 A A N A NN - N N - N N N N - N N N N N N

4 A N 2 2 .1 N N 2 - 2 N A 2 2 2* N N - N N N 4 - N N N N N

- 2 A 2 N - A - 3 - 2 N A 2 2N N -N N N N N N N -'4 N N N N N N

.9 - N N - 3 2 N 3 N - 2 A 2 .1 3 - -N N N N N N N N N N N N N N N N

- 2 2 N - A - A 2 4 2 - N A 3N - N N N N N N N - N N N N -N N

-. - A 2 3 4 N - - 2 - 2 N 4 A 4 3N N - N N N N N N N N

N N - N = N 2;; ~ ~N N -N

- 4 3 V 2 4 A 4 3 A 3 2 N 2N -, - -N N - N -N N N N N

2 - 2 2 3 3 2 N 2 -- N N -N N - N N t N N N -N N -

-, ., ~ 3 2 A - N N N 3 N 2 A .1 3 .1 NN N N N N N N N N N N N

* t VNNNNNNN- N V V V V V V V 1 N

r r P 7 N .9 N N N N V P P 1 N N N N N

*--------------.. N N N - -.9 N - N C

S P 7 V V V P V N - N p P 7 P V V S A

.4 .1 .~ -, A 2 2 2 3 2 3 3 2 2 3 3 2 2 2 2- - 2 0 * 4 3 2 2 3 '4 3 3 2

3. N N N -N N N N -N N N

------------ A I S 2 2 2 2 -% '4 N N 2- * N N N N

N '4 N N N - N N - N N N N V

3

N N N N N N N N

3 2 2 2 2 -

* N 9 ~ 4 4 P P V V 4 4 4 4 N

2

I

- I - ----- ------

-10001 ARIZONA STATE UNIV TENPE DEPT OF INDUSTRIAL AND MAMA--ETC F,/6 IS'SAPR 80 N N YOU"$ APOSAT79-01I1

UNCLA7SFIED ',ASU4CRC-fl00I6 AOSR-TA-SO-059aN

3-N3

II *

- 3 3 - t N N - 4 - -, N N *

* .,.

* - 3 N-------------~ ~

I a I I S I I I I I S I

~ -~ 3 N N - $4 4 N - - N 3 a N N- I I I

- 3 3 3 - V N - N - - - - N N -I I I U * U

3 A N 3 N 4 5 4 ~ 3, 0- N N -

3 N a N -, 3 3 3 3 A- a a a a a N N N N N N

V 3 N 4 A * 3 N 6 0 6. N~.3a a a a N a -. N N rI N,

- 3 N 5 3 3 A 3 4 0 3 3 - 3 4N a N - a N N N N N N N

A 3 A N A - N 6 N N 0 4 N -N N a N - N N N N N N N N N

A 3 .5 N 0 N 3 N A N A* V N N - N N N N N N a N N

N * N a N N N N N N N N N

-. a ~ 3 3 * ~ * 3 A 3 - -~ N N * N N N a N a a

3 4 3 A 4 3 4 S A - 3 4 4-. - N N ~ N N N N N N N

3 -, - N 4 N A - 3 N A AN N N N N N N N N N N N N

N,~AA 3 A 3 A NN N N - N N N N N a a a

~ N A .1 .ft = A 0 3 N ~ 4* N .. .4 N N N N N N N

3 N J - - A. 3 -N - a - N N a a a a a a a

3 N .~ V A A N A N 3 - N N 3N N - N N N N N a N - ~ N

N -. V S V V V V V V * 9 - . 9

N r V r V N a a a V V V V

- - - 3 3 2 3 4 5 V V a a - a

- ~ 4 4 A A I V V V 4 1

- - - 3 3 3 3 3 3 3 3 3 3 3 30 V V A A V N N N N A A A A

- - N N N N

I A A A N N N N- - N N N N N V P S

- V V N N N N N - a a a N N

*- -. N N N N N *N a a N N N N N N

* N N * N N N N ~ A A A a a a a

S

J

I

S

---------------------------------------------------------------------------------.-,--.-------

i-I

APPENDIX G

FORTRAN IV Program Output for Luke AFB,Component Repair Squadron

I

200' 0 0 0| r0.

ft .

*a.Ob..0wl* -'l 0 a ' 9 0 U .U V U U, U. i I ' - .2.94' .9 - . 19 .

2SlZ19 4s U .9 Ve U' 9 0 01 U * 0 0' 09o U V *I .9

a SC

so2 a.. a4 M A, a q . m,; M a q sf .ft f 'a

64W

Aid

10.9 U. . 0. . .. . U" U' 094 U .9' a. V.9 - - - - - - -o p

i a.*~ ~ a51 a a a U a 0 t a a a a' a.19

v4

001

v ft ft "I " t I i AI '-Abe A~ W AI a . 0'.a

2* 40 .4 .9 P.~.9 F

-A

4-

*Ad

sod *4 ft ft -4 4o ft 14 ft at ft ft d%.

lbs

4. 19 v

19S .9 .9 .9 1 . 9 r9 A9 19 Us a. a a

V:0 0 4 4 0 0 ' 6 O U N U U V 4

G.9 4l N Nft N N .

402

4Uf , 'I I

a w . Ik 0 0. at M: I a

-~ ~ a - a o co ao .o a

a a - a a o, a - a a . a Q! a Na a 0 , a a a. a a NqN, N a

'00 'am '001oll ft a. a O16 P. ft 19 !4D e t e. I t l

- -a -a a -- -. -0 a a a

a - a a - - aa a. ,

B a -e • a a a a = a ae a 0 a 4 a Ia

* .. * . * a . . a . ..- ..

- - - a - - - - a a -- - - -- - - -It

A 4 0. a a 4 . a *4 * I

S Nt N N NftN NO ft ft NO Al N N "N N

- a - - - - - N - a a - -. N - - N N N

a , aa a a , a

aS + a ol o

;of ,4wv- ,' a* S I !!

- I - a a a a a, a a 06 SM I ft t4

14 1ft

- - - - o -- - - - - -

- l a al - a a -0 - - , 0 , SF - -. -. - -

a% .1 aI . 5 a1 a a a * a 4a . a1 a al ai I.

.9 - • N0• • 1 .9I, le 0 I.

,N 0 5 eN N ,5 #9 .9 0 0 0 OS S' UF U UF U U 41

4 4, - U" UP U 4 4 41. 41 .4 --

N. Ni i N .9 1W #9 #9 lP .9 .9 #9 .9. N. #P UF2 ' . 9 . 9 9 . 9 # 9 #N a a S S S S U N .9 4 a S * 4 #

N N N N -N N N N N N f

* mm t

* 0 Sl

*

* .11

*i 6 4 *1ji

4 4P

Il

II 4 4 ,6 i

.I ,

4 a

w on ftr v f oIWO o 0 a - N -4

4-9o a-a °. a ' - • a , .j N: V r

a -.- . , , Ia.I P. 0. I I S Io

Umb-. a N S r w a 4' -44 to 0 0-

04 . I ft ' I .s, M #a

uo ld ~, * . . @ r . . #. ,I *. e' i..

A r s . . . . v' S - . It o .

a a oft ....

m 4,- -.

U04- 0 0 I W i . U * * 1

- g @;up NA INN N ~ N I N~ r ~ U N I 1

HA49 ' a a' '. * ~ 1 f ~ ~ * N

21w"i NI N N 0 V. N' N ' ' N ' n

I 'A-a

a z. a 'A f t 0 .a ta a t e 'a a l N

feN N . fe (4 1$' N . N. N Nr

:dp

3% 0 0 a 4 a a A N 0 N N

46 I

64 a a a a a a - a a a a a 0. a a a a

"44.

2 A - 4 A @ a 0 -- -' - a a -- -

0 ~ ~ ~ ~ ~ ~ ~ ~ ~ 4 4E0 '

W -a .. , or a I: Or Ta: .

2 T

01

of ...

- v, S. a 0 4 4 0 0 - r 0- '6

om a pa

a -- 0 - 4 C - - 0 0 oft 0 0' 4 -. # 4

*~~~ ~ a* a I a *aa S '4 I 1 , 1

"I

#4 a ft a, a. Aa o .a.~ , . a

- 4 0 4P 0 4 Jo 0. 0 A* 28 0,4 0 I6

- 0 % A foo fo 0 4 4 4 , 4 or r

4 ~ -#4l #4 Ow A4 R4 -0 #4 # 4 #5 #4 # 4

#43 4 # 4 45 0 a 0 P 4 #

aa

!0 :E ! *.

"W WO v! a. -. .-1.

ow, .4 . q

10 Ii

§ 8 N U 4 0 cal Ul 0 0 -

a 0 a 0: a.8

0..g

- N -0' - ' 8 U N N 1 UA

a aj o

U~4 * N 8 * e a . s. * * @

4 4 p r * 8 - 8 ~ I I G ,

a- . -. ~

a#6

I I -,

40.

I J'0 #4 -

* #4 6 -* 0 6Si- #4 #4 #4 II* S II II

- .. H4 9 0 S II

o H- - - 3

flUb H- *1 I H .~L

I -.* 0 6 I .~ 3#4

* a a I ., . .

0. S- - #4

% U 4-#6 4

#4.-#4 - #4

4-#6 #4

* - 4#4 #4 #4

* 3 A#4 #4 #4

- N N#4 #4 #4

* - a#6 #4 #4

#4 #4 -#4 #4 #4

0. * #6#4 #6 N * a

I .11 I , . j .~4.. 3

VIW

* 6* P

P 9' 9' 5' *1A'

* 6 6 6I 0. * 0.0. - -

*6

£ '* #4 #6 #4

S

I a* 1

S

S

APPENDIX H

FORTRAN IV Program Output for Luke AFB,Electrical Maintenance Squadron

46

0 2, a 4 0 a a ai at i a 0 a 4

OlN -

014

Qz i

C a - a a 0 0a, 0 0

*v) 0a 0~ 0' 0 0' 4 4 01

A'Al A 0 a a 0 0, W 0 0 ta 0 o a . 00

am

- ,1. a a o a 0 0 W 0 - a a, to .

4'4 4. ,:

@4i ,,

* .1

ozv

20t a a4 a a4 a a4 4 4 - . 0 a a. .

3'-

@L 0 0 • 0 0 0, 0a '- 200

2. -0 - -. - - - - -.

- a -W

2ft Oft* I" th N N* N4

S-0 M ft V - 4 V4 0 0 V V 4 W

no IF t4- - 4 4 4 4. 4 4 .a- 4 4 4 4 4 4 4 . 4 4 4 .

2- C C C C C C C

2 A~aAIVI

%I A~An4 . 4 4 44 41 4 4 C C

o 4 ft 4 4 4 4 4 4 4 4 4 4 . 4 4 4 4 41 .

Am04 4 4 . 4 .4 . 4 . 4 4 4 4 . 4 4 , 4

a . I I

4q . W* 4 0 @ a 0 a, 0' 0 0 a a, 0 0 o 01 a

- - o - " a ,a a . a a of 094

o . a a 0, 4 Of 0 0a

a a '4 a - a a a ca e - 61 A; Is

a a a a A. a a a .a a a a a a'.a * a a a ' * a. a a OF

-~~~ - - - - - -4 - - -- - -.4 4 4

- - - -. - - - -

a a a* a - a a -a a 4 A a a -A

* * a a ~ U 4 a ft ft a".f m ft ft.

ft ft 4 a N 4 IV .4 ft $% 0.10 N

NI

a O e a a ao a *. a a a a ai a a a. 4

: a a a V 4•

N . A

4

a o° a a a o a a a .0 i1

a a a a a r a a a 0 0ia

a a am U a a, a:o a a%6 al *a la

a a wa a, 4. a a o p o a

c a a a -a a .

Sa d 4 a I Oa a al wl a

* a* aa a' * ft . * e.

A A 0 a0 A a a AA a a0 a a.

a - a a

13 -00

* P. -' e a p" 0 a . a' * '4 4 *

-. -. - P a a a a - - - -

a a aa a a aa a

a 4 4n

A4 I

aa.. I Of a I

iUJi .t @ a - ,0 - U 0, a4 a, 90 . .

o @4. r S .4 - . . . '.

.4I Pe M!F

0' 41, I N 0 0 "

1 cl I

sf 1 l. 14 1

ft 6., v' Uo' a o",

S .0* o I '

I.4 S 4 . 4 6 4 . .. 9 a ,: 6 "4 -,t F. .

I 'F

'JO, .4 . 4 4 .dI S S OF 9? 5 4' .4 I

034 ' . 3 4 S. .. 4 P 4 .41 F S .4 .4 .4 .4

4*.4 .4 4 .4 .4 4 .4 6 . . .4 I .4 .4 "4 .4 . 4

o sawno

oA .46"

,J.6

i. . . . . . . . . .

M A4

I I aw 4 . - .4 .4 . .4 a a a. 4 4 a a

-~ 64

F; a r . f a. ato

o. - a w 0 a -9 a a 0 W4

n t A 1 z ft 8'r 4 f

I' * I 'S !-' ii .,

, 'a 4 I I' I I

a~1 *4 A -4 S a4a . S 4 1

*ft N 1 .. N t N 5 9 1

N

-- ' "." = "

4 o e a. - " * SI,, - 4 Z, N -

r -1 -0 A 4 a r v N4 of 4q I5 vI v.N a

v r "

Nt - N N -'N' NI ' i

. .a a .a a . .a . . . . ... o, . a

N N N. N N N . N N'= N. N = N = N N

A NqN

4 N N N

N - a N a N -a a a a . - - -

N N N N N N N N N N N

4 4 04M

ft I ' ' I

ft ft Mt ft

~~: " *"

- ~ ~ ~ ~ ~ ~ ~ f ftt4 4 4 * * t 0 4 f

- - - -ft

P P P 0 P 0 P ft ft ft4w

ft~ ~~ ~ ft ft 'At f t ft t f t ft t t f

a 'A 0 *v 4 4 v r * r ' f

ft f t f t f t ft f t f t f t f

p - P t 6 - 4 t * C 6 P f

APPENDIX I

FORTRAN IV Program Output for Luke AFB,Aircraft Generation Squadron

1-1"11 Nil

014

vim

a.0a~ 0 0 0 ' 0 to ft

zu. a a a a a 0 0 ab 0 a 0 a a. a #4

~ - a a0 4 4 0 0 0 0 0 40a0a 4

#40

&"-4 ft ft 0 a a. a a a a #43-

me0 - #4 4 - -40 40 "4 "4 .0 "

ARJ#O1.'mj11 4 4 0 #4 w4 - a " .4 .4 .4 4 0 4 4

0 .'

'~~ ~ In4# a 0 a a a A4 0 a a16 # 0

66 S

.do

=4 . .. ..

~~-f ft a~ a1 OF v3 0 v 1 4 I 4 1 4 4

w two - -4 - - - -4 . .4 .4 . .4 .4 .4 .4

.4 0 OR 0 do 0 1 .0 01 01 - 4

41 - -s - t #4 #4 Op v4 OR v4 #4 #4

P. a ft :os # 4 # # -3

- 4 # 4 #4 # 4 4 #4 # 4 #

ft 0 a 0

at op 0 0 U 0 0 0 0

v v ft &~ a a U r 0 0 ft ON 0l a

v v ft 0 0 6 6 v 0 0 0 6 l 6 ft ft

v I

P.

s p

a 0

*00 0 .0

t * 00 0

" w0o

o 4 o 0MO

*

4 4 4

- 4 -

4 4 o 0 4

* 4 - 4, 4

" 0 '44 S 0 0 0

4, 4 4,

.?' v . .. , . .. . . . . . . .. . . .. . ..... . . .

ca.

S W

- AU 44 44 4* P4 11 a 4 A 0* p- ftI f 0 ft M fta . t 1aMc

.

'1 UO 0r a A1 40 11 1, 14 1, 4 -1 4 " 0

&P0 ft 0 4 A 0 ft 41 A ft 4 0 0 a 0 411AVI44 4 M 4t 0 44 M4 M 0 ft ft ft M

04JI

"=,A 4 4v M 4 .

66 4(f 0 0 11 4 4 1" In 0 4 4 A n 0 , 4 a-a* X 1I4 4t 4t 4 M r4 C 40 M 4

o"I 4 4 .4 4b 4l 4b 4p *I 4 4 4

ir

W VA op 41 A A A ft 0v W AP 4 4 4 A A

. .. Z . . . 4 . 4 . . .4 . . A4 .4 p

no ftf £4t

a 42

.4 - NN 3

- ~ ~ m as .4, a. a4 . 4 4 4 . 0 04 . 4 4 .

0 a a 0 a a a 0 S 0 0 8 0 4

v4 &4 i4 a4 a4 .4 op .4 ow op In .

p..a

wI.aS.

* a ~ S **

* a a ~ a 4* S B Saa- a a a a a* S

hi- a a - a aa S Sa

* a I- S P- a 0 a

* 4 a a 0~S U 0 a

a P 0 0 -a a a

* 4 0 9 4a a a a

0 0 0 S fta a a a

* p. * 9 0a a a a

p. a a S aa a a a

a 0 * 0 -a a a a

s ~ * a a~j .~ 4 a

- - - 5 4a a a a

a a 0 5 aa a a a

* 0 D ~ aa a a a

a a 0 9 -- - a a

* a a S 0a a a a

a a a a a

a a a a a

a a a a

D U S * U

o 0 0 0 0 0US.

I. 0 0 0 0 0a a -i a a a a a

p. * a 0 a aIz a a 0 S 0* U U U U U

* a a - a a1 =2

S

#4

DAT

DI