A model for diagnostic problem solving

Actu Psychologica 36 (1972) 370-380; 0 North-Holland Publishing Company Not to be reproduced in any form without written pemission from the publisher

A MODEL FOR DIAGNOSTIC PROBLEM SOLVING

J6ZEF KOZIELECKI

University of Warsaw, Institute of Psychology, Poland

ABSTRACT

A formal model for diagnostic problem solving is proposed which comprises a goal block and an operational block in short-term memory, together with an environment information block and a psychoinformation block in long-term memory.

The strategies covered by the model are discussed. Finally, a system of statements on the course of diagnostic problem solving is presented.

People are frequently confronted with diagnostic problems (DPs), notably in such fields as medicine, technology, education and manage- ment. The issue has been taken up by many investigators in recent years (ELSTEIN, 1971; KOZIELECKI, 1969; PHILLIPS and EDWARDS, 1966). In the present paper an attempt is made to develop a formal model for DP solving.

DIAGNOSTIC PROBLEMS

Notwithstanding their enormous variability, DPs have some features in common:

(1) There is a set of states of nature, H = {hl, hs . . . . h,), which shall be called hypotheses. In a majority of real DPs, set H is fairly large; for instance, in technical problems the operator may be faced by the question which of up to 104 elements is the source of the trouble.

(2) Each hypothesis in set H has a definite subjective probability, which shall be denoted as y(hi), y(hs), . . . . y&J. This probability corresponds with the degree of belief that a hypothesis is true.

(3) There are sources of information which generate the set of data D = {di, dz, . . . . dn}. Data such as the results of medical examinations, the outcome of technical tests, or the data of economical statistics, have some diagnostic value, and as such they serve to modify the initial probability of hypotheses about the state of nature.

Being contained in the diagnostic situation, the information originates from two sources: the environment and long-term memory. The incoming information, as for instance in medical tests, will be called environment information (El). The remaining information is derived

370

DIAGNOSTIC PROBLEM SOLVING 371

from man’s long-term memory as, for instance, the operator’s knowledge of the structure of the technical system, and this will be called psychoinformation (PI). EI and PI may be utilized in different degrees. In one DP most of the information will originate from the environment, while in another the dominant source is long-term memory.

It could be conjectured that in laboratory experiments as those of PHILLIPS and EDWARDS (1966) or KOZIELECKI (1969), people employ primarily environment information. However, in real-life DPs physicians or technicians tend to rely mainly on knowledge they have stored in their memory, which, if correct, would limit the relevance of laboratory experiments.

Diagnostic problem solving consists of the evaluation - in accordance with definite rules - of the posterior probability of the hypotheses, yr(h‘ 1 d). This probability ~(hr 1 d) is the degree of belief that hypothesis ht is true given data d. Hence the diagnostic process consists in a revision of the probabilities associated with the hypotheses.

The outcome of this process is the diagnosis itself. The diagnosis can be defined as a system of hypotheses of definite posterior probability. In this system a singular role is played by the most probable hypothesis, which shall be called the modal hypothesis and is denoted by @r* 1 d);

from our definition it follows that

y(h*Id)>y(hgId) for i= 1,2...m.

When the modal hypotheses has reached a definite posterior probability, the final diagnosis is formed. The degree of probability at which the modal hypotheses is accepted as the solution to DP shall be denoted as the diagnostic threshold.

THB STRUCWRB OF THB DP SOLVING MODEL

Most experiments in DP have aimed to provide an answer to the question whether people form their diagnoses in accordance with Bayes’ theorem, or whether they apply the half-split strategy. In other words, the investigator’s goal is to check whether the subjects’ behavior was optimal or not, rather than to explore the psychological mechanism of diagnosing.

In working out the model of DP solving I took recourse to the follow- ing developments in modem psychology: first, the principles of function- alism which underline the role of the goals that tend to guide human behavior; second, the results of investigations in the field of short-term and long-term memory (MCGAUGH, 1969; SIEREDA and SNOPIK, 1970) -

372 J. KOZIELECKI

assuming that these are two discrete and relatively independent systems (KONORSKI, 1967); third, the findings of the psychology of thinking.

Environment I Short-term memory 1 Long-term memory I I I

I ! I I

I I I I

I I I v I

I I I

I I I I I I

-Path of the control of the information processing

- Path of information flow

Fig. 1. The block diagram of the diagnostic problem model.

A block diagram of the DP solving model is shown in fig. 1. The structure of the model will be discussed first. It is assumed that the diagnostic processes take place primarily in short-term memory. The same assumption - which will be substantiated in due course - has been made by several other authors, e.g. ELSTEIN (1971). Short-term memory contains the goal block and the operational block, both of which determine the structure of diagnostic processes. These blocks utilize information from outside as well as from long-term memory.

The goal block comprises the diagnostic goals, arranged in a tree-like hierarchy. At its top stand the general goals, followed by subgoals and sub-subgoals. For instance, take the main goal of identifying the fault in a broken-down machine: subgoals may consist in advancing a preliminary hypotheses, performing a technical test, and evaluating the probability &ha 1 d). The subgoals are generated in the course of DP solving.

The goal tree serves to guide the diagnostic process in the operational block: it decides which information from outside and from long-term memory is introduced into that block and which data are to be eliminated.


In line with SIMON (1966) it is assumed that the goal tree is located in short-term memory. The goal tree is a dynamic structure; once a subgoal is reached it is discarded and new subgoals are generated. As soon as the final diagnosis is made, the whole tree may disappear from memory. The dynamic goal tree, therefore, functions only during the limited amount of time in which the DP situation is actually present, and for this reason is related to short-term memory.

The operational block is the kernel of the diagnostic process. Through- out this process the operational block receives data from the environment and from long-term memory in order to arrive at a system of hypotheses, and subsequently to modify their probabilities. There are a number of reasons why this block ought to be placed in short-term memory as well. Diagnosing is a dynamic process divided into phases. During phase t

the only thing that matters is knowledge of results of the previous phase, t-l, or knowledge of the posterior probability of the hypotheses as determined during t-l. The preceding history operations and the results of the preceding phases (t-2, t-3, etc.) are basically irrelevant and may be eliminated from the operational block. In effect, nothing but the outcome of the final phase, if the problem is solved, the final diagnosis has to be preserved in memory for any length of time, whereas the process of DP solving itself may be duly forgotten. This dynamic character of diagnosing makes it quite plausible that the operational block is also part of short-term memory. Incidentally, some authors (e.g. NOSAL, 1971) believe that information processing is mainly effected by what they call operational memory; the latter concept has not been included in our model.

The psychoinformation block contains the previously accumulated information of the states of nature. Located in long-term memory this block comprises a great deal of data, usually arranged in a hierarchic structure.

The environment information block handles the information received from outside. People resort to various methods of securing such information; the most widely used one is exploration. The incoming environment information is subject to a selection process; only those data which are relevant for the given goals or subgoals are admitted into the operational block.

THE FUNCTIONING OF THE DP SOLVING MODEL

Diagnostic processes involved in DP solving are to a large extent

374 J. KOZIELECKI

codetermined by the properties of the blocks described above and their interconnections. The assumption that the diagnostic blocks, i.e., the goal block and the operational block, exist in short-term memory is particularly important in this respect. This memory system is known to have a rather limited capacity for information processing. MILLER (1956) argued that no more than 7 f 2 chunks of information could be stored in short-term memory; investigating medical diagnosting, EL~TEIN (1971) found that the human being may handle 4 f 1 hypotheses at one time. From KOZIELECICI’S experiments (1969) it follows that the number of hypotheses held in short-term memory ranges from 3 to 6. Although these data are somewhat divergent, the findings underline the limited capacity of human information processing.

Subgoal 1: Develop a system of war= clude the problem

Fig. 2. The diagnostic problem solving by means of the catchall strategy.

This immediately raises the question how DPs are handled in con- sideration of the limited capacity for simultaneous data processing. The investigations to date suggest that this is accomplished by the adoption of various artful strategies. One such strategy was reported in earlier studies (KOZIELECKI, 1960, 1970). Known under the name of catchall strategy, this device is of particular use in a difficult diagnostic situation, when a large set of hypotheses is accompanied by probabilistic information.

The way problems are solved by means of the catchall strategy is shown in fig. 2. In a simplified manner it could be described in terms of a sequence of subgoals.


Subgoal 1. Develop a system of working hypotheses for subsequent checking.

To carry this subgoal into effect one splits the large set of hypotheses on the states of nature, H, into two subsets: a small subset of working hypotheses Br, and a large subset of remaining hypotheses called the catchall, Cr. It was shown in the earlier studies that Rr comprises no more than 3 to 6 hypotheses. Basically, people include into fir the hypotheses of greatest prior probability, although they may resort to other criteria as well, such as the value of the hypotheses, the satisfaction associated with testing it, etc. For instance, a physician may start his diagnosis by checking if his patient suffers from an illness that presents a danger to his life @STEIN, 1971).

The small system of working hypotheses nl is put into the operational block, while the catchall Cl is usually held in long-term memory.

Subgoal 2. Find information di required to verify system Br.

Once the system of working hypotheses has been set up, a search is started in environment as well as in long term memory for information needed to define their posterior probabilities. Used information found to be superfluous at the current stage, is at the same time eliminated from the block.

Subgoal 3. Estimate the posterior probability of system i7r in the light of information dl.

This subgoal included a revision of the probabilities of the working hypotheses. The system y(hr 1 dl) constitutes the first diagnosis.

Subgoal 4. Check if the modal hypotheses in the above diagnosis does reach the diagnostic threshold.

At this stage a comparison is made between v(h* 1 dl) and what is considered an acceptable probability.

Subgoal 5a. Problem solving is concluded if the probability of the modal hypotheses is equal to, or greater than the diagnostic threshold,

Subgoal 5b. Otherwise one goes back to preceding subgoals.

Accordingly, the subject modifies the system & by: (1) elimination of those working hypotheses which are found to have low probabilities. and which are discarded into the catchall; (2) introduction of new

376 J. KOZIELECKI

hypotheses into the system that were outside the operational block so far and stored in the catchall. This amounts to a two-way traffic of hypotheses between 81 and Cl. As a result, a new system of working hypotheses, fls, and a new catchall, C2, come into being. Thereupon, information d2 is being searched for and the posterior probabilities of the gs hypotheses are defined. There may be many such recurrences to the earlier phases of the procedure.

The catchall strategy consists in the successive feeding of small portions of information from other blocks containing the remainder of the data, into the operational block, and in eliminating the useless information from that block. Thus the operational block can be said to draw data from the environment and from long-term memory, in order to return them later to those blocks. It is this information exchange that makes it possible for the human being to proceed within the operational block, with its limited capacity, a theoretically unlimited amount of information.

The catchall strategy is particularly useful in a difficult diagnostic situation when there are a large number of hypotheses and when the only information available is of probabilistic and ambiguous nature. In simple diagnostic problems, when there is a limited number of hypotheses, or when the available information is unambiguous, human beings tend to apply different strategies. For instance, in Edwards’ urn experiments, where no more than 2 to 4 hypotheses come into question, Ss can control them simultaneously; hence there is no need to separate the working hypotheses from the catchall. Supplied with absolutely unambiguous data, like in the study of machine trouble shooting by GOLDBECK et al. (1957), people may employ the split-half strategy. Thus it is a matter of the nature of the diagnostic problem and its degree of difficulty that determines which particular strategy is adopted.

THE PROPERTJES OF DIAGNOSTIC PROCESSES

Numerous experimental studies have resulted into specific statements about how hypotheses are developed, to what extent available information is utilized during DP solving, and what diagnostic errors are committed. But where most of these statements are formulated on the basis of studies conducted in specific diagnostic situations, there is no guarantee that they also apply to other classes of DP solving.

DIAGNOSrIC PROBLEM SOLVING 377

Statement 1. People tend to make better use of positive than negative information in the diagnostic process.

A number of studies (BRDNRR, 1956; DONALDSON, 1959; WERNER, 1961) indicate that hypotheses are formed (and their probability modified) on the basis of positive information; this remains true even if the negative information is of greater diagnostic value. Accordingly, the physician will refrain from utilizing negative results which deny the presence of a given symptom in the patient. This waste in dealing with negative data often prevents people, from arriving at a correct diagnosis.

Statement 2. People tend to make better use of confirming than of disproving information in the diagnostic process.

Studies by BRUNRR (1956), KOZIELECKI (1969) and others suggest that data which confirm the adopted hypotheses are more effectively used than those which disprove them. Having received disproving information, people sometimes are affected by a kind of cognitive blindness. Rather than discard the discordant hypotheses or reduce their probability, they tend to ignore the information and keep to the status quo.

Statement 3. There is a primacy effect in the diagnostic process.

This means that people overestimate the diagnostic value of the earliest information. Many subjects quickly modify the posterior probability of the modal hypothesis as soon as they receive the first quantity of information (KOZIELECKI, 1969).

Statement 4. People tend to extend the system of working hypotheses in the earlier phases of the diagnostic process and to reduce its scope in the final phases of the process.

An earlier study by KOZIELECKI (1969) indicates that people tend initially to extend the system of working hypotheses by including in it new hypotheses retrieved from catchall. In subsequent phases, subjects gradually eliminate certain hypotheses from the system Hi. This suggests that people first want to register a set of possible alternatives, and only later seek to discover which of these is actually true. We do not know whether this expansion and contraction of the system of working hypotheses is typical for all diagnostic situations in which H is large.

378 J. KOZIELECKI

Statement 5. There are certain factors that cause the open system of working hypotheses to become a closed system.

The system is open as long as new hypotheses can be admitted while rejected ones are discarded. No such traffic of hypotheses is possible in the closed system (ELSTEIN, 1971; KOZIELECKI, 1968). The greater the probability of the working hypotheses, the further advanced the diagnostic process, and the stronger the rigidity in the subject’s thinking, more likely it is that n will become a closed system.

Statement 6. People tend to overestimate the probability of hypotheses in the working system and underestimate that of the hypotheses held in the catchall.

KOZIELECKJ (1970) has shown that people overestimate the combined probability of the working hypothesis system as compared with the probability calculated from Bayes’ theorem. In other words, hypotheses currently handled in the operational block appear to be more certain than Bayes’ theorem would justify.

Statement 7. In some DP situations people reveal diagnostic conservatism.

This uniquely well substantiated theorem states that in diagnostic problems with only few (2 to 4) hypotheses, people tend to underestimate the Bayesian probability of the modal hypothesis and hence to be overcautious in their diagnosis (PHILLIPS and EDWARDS, 1966). This is called diagnostic conservatism.

The last two statements do not contradict each other, despite appearances. The former (6) refers to the system of working hypotheses, the latter (7) to the modal hypotheses alone. In addition, these statements refer to what are in effect different types of DPs.

Statement 8. The smaller the set of hypotheses in H and the more informative the data received, the higher the diagnostic threshold.

The diagnostic threshold is the probability at which the modal hypotheses will be accepted as the final solution. As such it is dependent on a number of factors. According to RADZICKI (1963) one of these factors is the extent of set H. With a small set people tend to conclude DP solving at a higher probability of the modal hypotheses than for a large set. For instance, a probability of 0.80 may be acceptable in a problem involving eight alternative solutions, but is likely to be insufficient for


the final solution in a situation with only two alternatives. In a similar way, the diagnostic threshold is raised in situations with highly valuable information.

CONCLUDING -Ks

The paper presents a preliminary model of diagnostic problem solving to account for the specific solving process. DPs ought to be distinguished from different problems, such as construction problems where the goal is to create a new state of nature. The model under discussion applies only to the former type of problems and may not be extrapolated to other types. The present state of psychology of thinking and memory does not favour the development of a general theory of problem solving that can be applied to every kind of problem. It seems a much better approach to develop models for the solution of particular specific subsets of problems.

The above outlined DP model has certain corollaries. It effects changes in the actually present hierarchy of experimental queries. A crucial issue of investigation then becomes: how does the dynamic goal tree guide the diagnostic process, how does it control the exchange of information between short-term memory on one hand, and the environment and long-term memory on the other. At the same time, research on short-term memory becomes of increasing importance.

(Accepted March 16, 1972.)

REFERENCES

BRUNER, J., J. GO~DNOW and G. Aus-rm, 1956. A study ofthinking. New York: Wiley. ~NALDSori, M., 1959. Positive and negative information in matching problems.

British Journol of Psychology 50, 253-262. Ew, A., L. S-, N. KOOAN and H. JASON, 1971. A theory of medicul inquiry.

Lansing: Michigan State University. GOLDBECK, R., B. BERNSTEIN, W. Hum and M. MARX, 1957. Application of the

half-split technique to problem solving tasks. Journal of Experimental Psychology 53, 330-338.

KONORSKI, J., 1967. Integrative activity of the bruin Chicago and London: The University of Chicago Press.

KOZIBLBCKI, J., 196% Zagaahienka psycho&ii my&enio. (Problems of the psychology of thinking) in Polish, Warsaw: PWN Polish Scientific Publishers.

380 J. KOZIELECKI

KOZIELECKI, J., 1969. Psychologia procesdw przeddecyzyjnych. (The psychology of predecisionalprocesses) In Polish. Warsaw : PWN Polish Scientific Publishers.

KOZ~LECKI, J., 1970. Psychological characteristics of probabilistic inference. Acta Psychologica 34, 480-488.

MCGAUGH, J., 1969. Facilitation of memory storage processes. Reprinted from: The future of the brain sciences. Plenum Press, 355-370.

MILLER, G., 1956. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review 63, 81-97.

NOSAL, Cz., 1971. Przetwarzanie informacji w procesie myalenia tworczego. (Trans- formation of information in the process of creative thinking.) In Polish. Przeglad Psychologiczny 22, 51-65.

PHILLIPS, L. D. and W. EDWARDS, 1966. Conservatism in a simple probability inference task. Journal of Experimental Psychology 12, 346354.

RADZICKI, J., 1963. Wplyw liczby alternatyw i tempa doplywu informacji na proces podejmowania decyzji. (Information seeking in decision-making situations.) In Polish. Warsaw: University of Warsaw.

SIEREDA, G. and B. SNOPIK, 1970. K problemie jedinstwa mechanizmow kratkow- riemiennoj i dolgewriemiennoj pamiati. (On the unity of short-term and long-term memory mechanisms.) In Russian. Woprosy Psichologii 6, 60-74.

SIMON, H., 1966. Scientific discovery and the psychology of problem solving. Pittsburgh: University of Pittsburgh Press.

WARNER, H., A. TORONTO, L. VEASEY and R. STEPHENSON, 1961. A mathematical approach to medical diagnosis. Application to congenital heart diseases. Journal of the American Medical Association 177, 177-183.

Documents

A model for diagnostic problem solving