HEURISTIC SIMULATION OF PSYCHOLOGICAL DECISION PROCESSES.pdf

7/28/2019 HEURISTIC SIMULATION OF PSYCHOLOGICAL DECISION PROCESSES.pdf

1/6

Journal oj Applied Psycholoty1.,, \ol. 52, No. 4, 325-330

HEURISTIC SIMULATION OF PSYCHOLOGICALDECISION PROCESSES 1ROBERT D SMITH

Departm ent oj Man agemen t, Pennsylvania State UniversityComplex thought processes used by a skilled psychologist dealing with per-sonnel selection and placement are analyzed and programmed in a computersimulatio n m odel Heu ristic me thods used to limit the tota l possible set ofdecision branches in the model are defined and illustrated. Research resultsindicate a strong relationship between human and machine output with a 94%level of agreement between simulated psychological inferences and humandecisions with identical ultimate employment recommendations in 22 of 24 testcases. Implications of research findings on psychological instruction, experimen-tation, information retrieval, test validation, and general decision making arediscussed.

Recent developments in the study of manand machine systems have resulted in signifi-cant interdisciplinary achievements. The areasof psychology, industrial management, andoperations research have been increasingly in-tegrated through quantitative and scientificmethods which aid in the definition, explana-tion, and prediction of human behavior. Oneof the most useful of these methods is simula-tion, a research methodology which facilitatesthe design and testing of models of physicaland human systems. Through the use ofsimulation, the models can be programmed toyield results which closely approximate real-world phenomena.Simulation models are of varying types.Some are applied to the study of physical sys-tems such as oil refineries and railroad switch-ing yards. Others deal with man-machine sys-tems such as the Rand Air Defense Simula-tion in which military personnel and hard-ware exhibit integrated reactions to simulatedchanges in the air environment. Yet anothertype involves human cognition and can beused to replicate the manner in which manmakes decisions in solving complicated prob-lems. The purpose of this article is to presentthe methodology, results, and , most imp ortant,the implications of research dealing with thislatter form of simulation.

1 This article is based on a paper presented at TheCollege on Management Psychology Session of TheInstitute of Management Science Fourteenth Inter-national Meeting, Mexico City, August 26, 1967.

Considerable work in the development ofcognitive simulations has been conducted byuniversity and privately sponsored researchgroups. Colby (1964) at Stanford, M aruyam a(1966) at Berkeley, Kleinmuntz (1963) atCarnegie-Mellon, Swenson (1962) at the MayoClinic, and Finney (1967) at The Universityof Kentucky have been instrumental in de-signing computerized simulations which de-scribe individual psychological characteristicsbased on some type of test scores.The present research represents an exten-sion of work previously accomplished. Spe-cifically, this study investigated in depth andsubsequently simulated the thought processesof a psychologist involved in the analysis andinterpretation of test results, job requirements,and personal characteristics of potential cleri-cal employees. In this type of decision making,the psychologist is faced with ill-structuredproblems not amenable to solution by algorith-mic techniques in which the researcher isable to express an objective function andconstraints in well-defined ma them atical terms.Instead, it is necessary to rely upon the con-struction of simulated decision networks whichreplicate as closely as possible the psycholo-gist's thinking processes.

M E T H O DAs shown in Figure 1, the experimental model ex-tended the number and types of variables includedin previous studies by adding job specifications andpersonal characteristics of applicants. These addi-tional variables made the task of simulation moredifficult since inferences derived from test scores had

325


2/6

326 ROBERT D. S M ITHCOMPUTE**T R A N S F O R M S

DATA

TEST SCOftCS * . OOMPISTERT R A N S F O R M SDATA

O U A U T A T tV E IN T T H W C T A T IO

CMPIOTMENT DECISION

T S T S C O R E S

W C H O U M S C A LJOB RCOUREMCN

COMPUTERT R A N S F O R M SOATA

Q U A L IT A T IV E IN T E R P R E T A T IO N

EMPLOYMENT DECISION

Fic . 1. Evo lution of model complexity.to be viewed in light of specific job requirements andpersonal characteristics of the individual such as herage, experience, and length of time in her presentposition.It was hypothesized that a simulation model couldbe developed which would replicate the thoughtprocesses of a skilled psychologist as he performed thedecision task. The Protocol Method was used togather data (i e, the an alyst's v erbalization ofthoughts were recorded as he evaluated a series ofcases dealing with actual job applicants) The a p-plicants were all female and were applying for vari-ous types of clerical positions requiring varyinglevels of skill. These positions were billing clerk,statistical clerk, clerk typist, receptionist, administra-tive assistant, and executive secretary. The protocolwas tape-recorded, transcribed, and analyzed in depthprior to the development of a computer flow chartdefining the decision processes Finally the model wastested by utilizing a set of cases which were analyzedby the human and processed by the computer

RESULTSIn all, 24 cases were evaluated by both hu-man and machine methods. In 22 cases themachine selected the same employment recom-mendation, which was one of the four clas-sifications: hire, reject, hire as a fair risk, orcheck background further. In one case themachine was not programmed to handle theinput data and consequently the human ana-lyst was called in, and, in the other case ofnonagreement, the human recommended hir-ing as a fair risk while the model suggested afurther background check of the applicant.Accuracy of the simulated interpretative

statements generated by the computer wasjudged to be 94% by a skilled analyst not in-volved in the collection of the protocol. Fur-ther, statistical tests of hiring recommenda-tions showed that the probability of obtainingsuch results by chance was so small that thehypothesis was accepted, and it was con-cluded that the model did simulate veryclosely the human analyst's psychological in-ferences.Limitations

The preceding paragraphs have very brieflypresented the methodology and results of theresearch.2 It should be noted that no attemptwas made to improve upon the decision-mak-ing process of the analyst and it would havebeen preferable to test a larg er sam ple ofcases had they been available. It might betrue that a more valid and reliable modelwould have resulted if the combined decision-making capabilities of a group of analysts hadbeen used. However, rather than spendingtime in a discussion of the limitations of thestudy, more significant issues involve the useof heuristics in development of the model andthe implications of results on general psycho-logical decision making.Heuristic R eduction oj the Decision Space

In developing the computer model, thematter of relevancy of information became allimportant. Consider the clerical selection bat-tery used in this experiment. It is comprisedof the Otis Mental Ability Test, the ShortEmployment Tests, the Washburne S-A In-ventory, and the Gordon Personality Inven-tory and Profile. This battery includes 20different test scores, some of which can rangefrom 1 to 99 such as the Gordon scores whileothers range from excellent to maladjusted(7-point scale) on the Washburne Test. Inall, there are 1,151 possible psychological testscores, in addition to job and personal data,which can be obtained for a single applicant.Thus, it is plain to see that the number ofcombinations of relationships and elementsthat can be chosen quickly exceeds manage-able proportions. In fact, using the variables

2 For a complete discussion of research design referto Smith and Greenlaw (1967).


3/6

SIMULATION OF DECISION PROCESSES 327in this study, it would require 1070 outcomesto provide a model encompassing all possibleevents. This figure approaches the estimatednumber of atoms in theknown universe (107C)representing a programming task of impos-sible magnitude. To avoid this situation, itis necessary to restrict theboundaries of thesystem and consider only those relationshipswhich are most relevant to the solution of thedecision problem .

To simplify development of the experi-mental model, the experience of the decisionmaker was organized into a graphic represen-tation of thought processes. The principlesused in the simplification of the model (i.e.,to limit the boundaries of relevance) wereactually applied by the psychologist andmerely transformed into more quantitativeterms by the researcher. These principles areoften referred to in the literature as "heuris-tics" taken from the Greek heveiskein whichmeans to limit search. Heuristics provide gen-eral decision rules or "rules of thumb" whichassist researchers in the development of work-able models of systems which are within theboundaries of human and machine capabilities.Heuristic solutions to problems are not neces-sarily the best or optimum solutions since cer-

STMEMEHTSi 9*m*\ Nntrwcy tcnwifrttat cwiiotu

b *t7 fttviitg M tubimttsnv, noi stxHd for |t t >

4 A|lManr apprrart totMCrnintrrrMrt,nrrr rarme orrt M T M I U I M

FIG 2 Partial heuristic decision map for sociability

FI G 3 Partial heuristic decision map for intelligenceand abilitytain alternatives are excluded from considera-tion which hopefully are irrelevant but, infact, may not be. Clarkson's (1962) Port-folio Selection and Tonge's (1961) AssemblyLine Models were both heuristic in nature asthe authors turned their approach from asearch for theoptimum solution to one that,with high probability, would be suitable fortheir purposes.

The same heuristic approach was adoptedin the present research. Instead of providingfor each possible value of a particular vari-able, relevant ranges for most variables weredefined. This type of system definition in-volved the use of heuristics very similar tothose used by the analyst in his interpreta-tions. An example will help to clarify thispoint. The sociability score on the GordonTest could range between 1 and 99. Theanalyst, however, treated only particularranges for certain job types. For the statisticalclerk, a sociability score of 95 or greatercoupled with a low cautiousness score led toa negative response since the individual couldconceivably be a "social butterfly" and in-capable of maintaining the prolonged con-centration required for the statistical work.


4/6

328 ROBERT D. SM ITHFigures 2 and 3 provide examples of heu-ristics as well as the general methodology usedin mapping the thought processes. Note that,in Figure 2, the sociability score has beensubdivided into three broad ranges at thelower end of its scale, and within one ofthese ranges (i.e., 6 through 10) it is com-pared with certain job specifications. This isnot the only place in the program where thesociability score is considered, but it is evi-dent here that a low score coupled with workrequiring high levels of verbal proficiency anddecision making lead to a negative recom-mendation.3The psychologist placed his initial and

greatest emphasis on scores which appeared atthe extremes of the rating scale. That is, hefirst recognized scores above 90 or below 25on the Gordon Tests and this information alsoprovided a useful heuristic for the modelbuilder.If certain combinations of variables wererelevant but not provided for in the model,the computer was instructed to branch to thenext case and leave the exceptional case for

human interpretation. In this manner, heu-ristics were used to reduce the number ofcombinations of variables within the systemto those most relevant for solving the major-ity of cases.Another type of heuristic was employedwhich added efficiency to the search process.It was discovered during the analysis of proto-col that the psychologist tended to divide his

problem into four general components paral-leling the four tests used in the battery. Hechose mental ability as his first area of in-vestigation and seemed to weigh the Otisscores heavier than other factors for mostjobs. Next, he referred to the clerical aptituderesults followed by the emotional stabilityscores, and finally the personality profiles.This does not mean, however, that he fol-lowed an unbroken sequence from one testto another. If, for example, he found a girlwho performed remarkably well on the OtisTest, he would investigate the Gordon scoreon "original thinking" as well as the type ofjob for which she was applying. The intent

'Proficiency levels required in specific jobs wereexpressed on a nu me rical scale ranging from 1 (lowestlevel) to 3 (highest level)

here was to prevent very intelligent girls frombeing placed in positions where their abiliuand interest could lead to poor performanceon repetitive or otherwise unchallenging jobsThe broad approach to sequencing, how-ever, did lead the researcher to design separatecomputer subroutines for each major area asa starting point in the development of theintegrated system. Naturally, there was muchsubsequent interaction among the subsystems,but the heuristic technique of fractionatingthe total problem further simplified the workand, more importantly, permitted the simula-tion to follow a path through the networkwhereby the most heavily weighted variableswere given highest priority. Items whichtended to eliminate candidates most oftenwere considered almost immediately, and timewas saved since many unacceptable applicant*were discovered relatively early in the net-work.Implications oj Research Results

One of the most important implications ofthe results of this research is the knowledgethat it is possible to program fairly complexpsychological decision tasks. This means thattime normally spent by highly skilled analystsin decision making which is fairly routine forthem can be made available for more signifi-cant problem solving and research. Second,the probability that boredom might influenceresults may be lessened as the psychologicalevaluator concentrates on those cases of un-usual nature which require diverse and flexibleskills found only in the human analyst.

With computerized processing of routinedata comes a marked reduction in the cost perinterpretation when, of course, there is suf-ficient volume to introduce economies of scaleWhere the psychologist normally took about20 minutes to give a complete analysis, themachine performed the same type of task inabout 3 seconds.4 In this manner, expertanalyses may be provided to those organiza-tions which previously could not afford the

4 The decision model designed for the experimentwas programmed to branch to the next case wheneveran applicant was rejected. The psychologist, however,continued with a complete analysis even though acombination of factors led him to formulate a nega-tive recommendation somewhere during the course ofhis evaluation.


5/6

SIMULATION OF DECISION PROCESSES 329services of skilled psychologists. This may beparticularly true in personnel offices andguidance centers which sometimes offer sub-marginal services in the field of psychologicalanalysis.Simulation models of this type can be usefulin the field of computer-assisted instruction.A laboratory approach in basic psychologycourses might be established whereby studentswould verify their beginning and intermediateattempts at analyses against the results of anexpert. With the information storage andretrieval capabilities of today's computers,teaching efforts might be made more efficientthrough the use of feedback data. Professorscould evaluate the most common errors madeby their students during the laboratory anal-yses and thereby concentrate upon these areasof weakness in subsequ ent le ctures.

Computerized simulations of psychologicaldecision processes offer other advantages froman information-utilization stand poin t. It isnow possible to study the effects of policychanges on end results without disrupting thepresent system. For example, in the personnelmodel discussed in this paper, one could de-termine the effect on mental ability standardscaused by a labor shortage which required abusiness to increase the acceptance ratio by20% (i.e., how much would one have to lowermental ability scores to gain a 20% increasein the number of applicants accepted). Inthis type of model it would be fairly simpleto determine the effects of changing the valuesof individual parameters on final decisions aswell as to determine which factors were re-sponsible for the greatest percentage of re-jected applicants. If any of these rejectionfactors were amenable to correction throughtraining, it would be possible to estimate thetrade offs between the costs of additional per-sonnel search versus the costs of trainingpersons not quite acceptable under presentstandards.

From the view of a science of decision mak-ing, this experiment lends some credence tothe theory that human thinking can be dennedas a complex network of simple binary choices.It is difficult to determine whether the ana-lyst's decisions were or were not based onsimultaneous consideration of multivariatecriteria. But, by sequencing his thought proc-

esses in a series of binary alternatives, it waspossible to replicate his final decision eventhough it was based on relatively complexinterrelationships among variables.Research of this type gives deeper insightinto the manner in which people resolve prob-lems. The methodology allows the researcherto map an equivalent thought process at aparticular point in time and could permit thestudy of the effects of aging and experienceon decision-making capabilities of individualsand groups.Suggestions jor Further Research

Work should be done to determine whetherthere is a significant difference in job per-formance between those personnel selected byhuman methods and those chosen through theuse of a model. Of course, the problem stillexists that a representative portion of the ex-perimental group is lost since profit-orientedorganizations usually do not hire applicantswho have been rejected by skilled interpreters.The practice of obtaining "objective" evalu-ations of employee performance still permeatesall forms of organizations. Even with the de-velopment of modern information systems,managers of all types are faced with the un-pleasant task of judging the performance oftheir workers. Halo and leniency effects pre-dominate. Perhaps it would be possible to de-sign a simulation model whereby the recordsof each employee could be subjected to a kindof skilled, nonbiased, objective evaluationwithout fear of political pressure or subjec-tive interpretations.Results of the study indicate that psycho-logical and other factors can be integrated ina workable decision model Th is fact leadsone to believe that computer-assisted psy-chological clinics for low-income classes mightbe feasible. Admittedly this idea connotes aworld of cold impersonalism but there is thepossibility that many people would willinglyavail themselves of expert services eventhough these services were not obtained

through direct human contact. Further ex-perimentation along these lines would requirethe development of a model encompassing psy-chological test results and environmental fac-tors. The model could then be used to screenlarge numbers of people, some of whom might


6/6

330 ROBERT D. S M I T Hexhibit symptoms of present or potentialmental illness. A computer-assisted screeningcenter would be used to diagnose cases inearly stages for referral to a specialist, thusproviding minimum-cost psychological ser-vices to broader segments of our society.

REFERENCESCLARKSON, G P. Portfolio selection: A simulation oftrust investment Englewood Cliffs, N J : P r e n -tice-Hall, 1962COLBY, K. M, & GIL BL RT , J B. Programming acomputer model of neurosis. Journal of Mathe-matical Psychology, 1964, 1, 405-417FINNEY, J. C. Methodological problems in pro-grammed composition of psychological test re-

ports Behavioral Science, 1967, 12, 142-152

KtEiNiiUNTZ, B. Profile analysis revisited: A heu-r ist ic approach. Journal of Counseling Psychology1963,4 ,315-324MARUYAMA, M. The use of computers as industrialcounselors Computers and Automation, July 196634-39.S M I T H , R. D., & GRE E NL AW, P. S. Simulation of apsychological decision process in personnel selec-t ion. Management Science, 1967, 13 , 409-4 19.SWE NSON, W. M (Chm.), R O M E , H., MAT AYA, P

M C C A R T H Y , C, PEARSON, J., K E A T I N G , F., & HAT H-AWAY, S. Symposium on automation technics mpersonality assessment Proceedings of the SlafMeetings of the Mayo Clinic, Vol. 37, No. 3, Janu-ary 1962, 61-82.

TONCE, F. M.A heuristic program for assembly linebalancing Englewood Cliffs, N. J.: Prentice-Hall.1961.

(Received September 12, 1967)

(Continued from pape 320)Skimming Lists of Food Ingredients Printed in Different Sizes E C. Pou lton* Med ical Resea rch Cou ncil, AppliedPsychology Research Unit, 15 Chauc er Road, Cam bridge, En glan d.Knowledge of Score and Goal Level as Determinants ofWork Rate- Edwin A. Locke* and Judith I'.Bryan De-

pa r tmen t of Psychology, University of Maryland, College Park, Maryland 20740Organizational Factors and Individual Performance: A Longitudinal Study George I" Farns* Massachusetts

Ins t i tu te of Technology, Sloan School of Management, Room 52-590, Cambridge, Massachusetts 02139Conceptual and Operational Problems in the Measurement of Various Aspects of Jo b Satisfaction Ma rtin G

Eva ns*: School of Business, University of Toronto, 119 St. George Street, Toronto 5, CanadaRelation between Birth Order and Being a Beautician Philip S. Very* and Joseph A. Zannini: Rhode Island

College, 600 Mount Pleasant Avenue, Providence, Rhode Island 02908.Psychological Concomitants and Determinants of Vocational Choice K enn eth M Kuner t* . Depa r tmen t of

Psychology, University of Detroit , Detroit , Michigan 48221.Ratee Relevance in Peer Nominat ions : Frank T. Passini* and Warren T. Norman : Depa r tmen t of Psychology

The Universi ty of Michigan, Ann Arbor, Michigan 48104.Pun itive Supervision and Prod uct ivity : An Exp erime ntal Analog David R Schmit t : Depar tment of Sociolog\,

University of Washington, Seattle, Washington 98105Marginal Productivi ty Procedure for Staff Selection Purn ell H Benson*- 123 Milligan Place, South Orang e.

New Jersey 07079.Interaction of Achievement Cues and Facilitating Anxiety in the Achievement of Women W J. McKeachie*Depar tment of Psychology, University of Michigan, Ann Arbor, Michigan 48104* Asterisk indicates author for whom the address is supplied.

Documents

HEURISTIC SIMULATION OF PSYCHOLOGICAL DECISION PROCESSES.pdf