Embed Size (px)
Citation preview
MITIGATING COGNITIVE AND NEURAL BIASES IN
CONCEPTUAL DESIGN
by
Gregory Matthew Hallihan
A thesis submitted in conformity with the requirements
for the degree of Master’s of Applied Science
Graduate Department of Mechanical and Industrial Engineering
University of Toronto
© Copyright by Gregory Matthew Hallihan (2012)
ii
Mitigating Cognitive and Neural Biases in Conceptual Design
Gregory Matthew Hallihan
Master’s of Applied Science
Graduate Department of Mechanical and Industrial Engineering University of Toronto
2012
Abstract
Conceptual design is a series of complex cognitive processing tasks and research seeking to
further understand design cognition will benefit by considering literature from the field of
psychology. This thesis presents two research projects that sought to understand and mitigate
design biases in conceptual design, through the application of theories from biological and
cognitive psychology. The first of these puts forward a novel model of design creativity based
on connectionist theory and a neurological phenomenon known as long-term potentiation. This
model is applied to provide new insights into design fixation and develop interventions to assist
designers overcome fixation. The second project seeks to establish that cognitive heuristics and
biases predictably influence design cognition. Two studies are discussed that examined the role
of confirmation bias in design. The first establishes that confirmation bias is present during
concept generation; the second demonstrates that decision matrices can mitigate confirmation
bias in concept evaluation.
iii
Acknowledgements
I would like to thank my supervisor, Professor Li Shu, for her contributions to this research and
her mentorship. I would also like to thank Professor Birsen Donmez and Professor Greg
Jamieson for their contributions as members of my thesis committee.
I am grateful to my parents, Mike and Terry, who have always supported and encouraged
fulfilment through education, and to Laura, for her patience and support.
I would also like to thank Hyunmin, Jay, and Jayesh, for their advice and friendship over the last
two years.
The funding for this research was generously provided by the Natural Sciences and Engineering
Research Council.
iv
TABLE OF CONTENTS
Abstract ......................................................................................................................................... ii Acknowledgements ..................................................................................................................... iii List of Tables ............................................................................................................................... vi List of Figures ............................................................................................................................. vii List of Appendices ..................................................................................................................... viii THESIS INTRODUCTION ....................................................................................................... 1 CHAPTER 1: Long-Term Potentiation and Design Creativity ............................................. 3
1. Introduction ............................................................................................................... 3
2. The Biology of Information Processing .................................................................. 4 2.1. Neural Transmission ...................................................................................... 4 2.2. Long-Term Potentiation ................................................................................ 6 2.3. The Biology of Creativity .............................................................................. 7
3. Connectionist Theories and Creativity ................................................................... 9
3.1. Information Processing in Connectionist Networks ...................................... 9 3.2. Network Activation and Associative Creativity .......................................... 10 3.3. Summary ...................................................................................................... 12
4. The Role of LTP in Design Creativity and Design Fixation ............................... 13
4.1. Design Fixation ............................................................................................ 13 4.2. LTP and Design Fixation ............................................................................. 15 4.3. Practical Contributions Regarding Design Fixation ..................................... 17
4.3.1. Incubation and Insight in Design Fixation ................................... 17 4.3.2. Enhancing Awareness of Design Fixation .................................... 19 4.3.3. Using Physical Activity to Enhance Defixation ........................... 23
5. Studying Physical Activity and Defixation ........................................................... 24
5.1. Methods and Procedure ............................................................................... 24 5.1.1. Participants ................................................................................... 24
5.1.2. Procedure ...................................................................................... 25 5.1.3. Quantifying Fixation ..................................................................... 27
5.2. Results ......................................................................................................... 27 5.2.1. Rater Reliability ............................................................................ 28 5.2.2. Physical Activity ........................................................................... 28 5.2.3. Education ...................................................................................... 29 5.2.4. Concept Quantity Among Engineering Students .......................... 29
5.3. Discussion .................................................................................................... 31 5.3.1. The Effect of Physical Activity .................................................... 31 5.3.2. The Effect of Education ................................................................ 32
v
5.3.3. Concept Feasibility ....................................................................... 34 5.3.4. Quantifying Fixation ..................................................................... 35 5.3.5. Concept Quantity .......................................................................... 36
6. Summary and Conclusions .................................................................................... 37
CHAPTER 2: Cognitive Bias and Confirmation in Design .................................................. 39
1. Introduction ............................................................................................................. 39
2. Cognitive Heuristics and Biases in Design ........................................................... 40
2.1. Heuristics in Design and Psychology .......................................................... 40 2.1.1. The Use of Cognitive Heuristics in Design .................................. 41
2.2. Design Relevant Cognitive Heuristics and Biases ...................................... 42 2.2.1. Design Relevance ......................................................................... 46
3. Confirmation Bias ................................................................................................... 57
4. Study 1: Confirmation Bias in Concept Generation ........................................... 47
4.1. Observational Research in Design ............................................................... 48 4.1.1. Verbal Protocols as Observational Data ....................................... 48 4.1.2. Analyzing Verbal Protocols .......................................................... 49
4.2. Study 1: Methods and Procedure ................................................................. 50 4.2.1. Participants ................................................................................... 50 4.2.2. Procedure ...................................................................................... 50 4.2.3. Qualitative Coding......................................................................... 51
4.3. Results ......................................................................................................... 54 4.3.1. Ratio of Confirmation to Disconfirmation ................................... 54 4.3.2. Qualitative Observations .............................................................. 55
4.4. Protocol Analysis Limitations ..................................................................... 57
5. Study 2: Mitigating Confirmation Bias in Concept Evaluation ......................... 58 5.1. Participants .................................................................................................. 58 5.2. Problem 1 ..................................................................................................... 59
5.2.1. Procedure ...................................................................................... 59 5.2.2. Results .......................................................................................... 60 5.2.3. Discussion ..................................................................................... 61
5.3. Problem 2 ..................................................................................................... 62 5.3.1. Procedure ...................................................................................... 63 5.3.2. Coding Confirmation and Disconfirmation .................................. 64 5.3.3. Results .......................................................................................... 65 5.3.4. Discussion ..................................................................................... 67
6. Summary and Conclusions .................................................................................... 70
vi
CHAPTER 3: Thesis Summary ............................................................................................... 72 Long-Term Potentiation and Design Creativity ............................................................. 72 Cognitive Bias and Confirmation in Design ................................................................... 73 Conclusion ...................................................................................................................... 75 REFERENCES ......................................................................................................................... 77
vii
LIST OF TABLES Table 1.1 Participant demographics .................................................................................. 25
Table 2.1 Design relevant cognitive heuristics and biases ............................................... 46
Table 2.2 Participant groups and assigned design problems ............................................ 50
Table 2.3 Number of confirmatory and disconfirmatory cases per group ........................ 54
Table 2.4 Participant responses in Problem 1 ................................................................... 60
Table 2.5 Conditions and data for Problem 2 ................................................................... 66
viii
LIST OF FIGURES Figure 1.1 Structure of a neuron and a network of neurons with an enlarged synaptic connection ........................................................................................................... 5
Figure 1.2 Associative hierarchies around the word table (Mednick, 1962) ..................... 11
Figure 1.3 Hypothetical associative map based on the FBS ontology ............................... 21
Figure 1.4 Relationship between participants’ reaction times when judging stimulus familiarity with the stimulus’ solution frequency ............................................. 23
Figure 1.5 Example concept shown to participants to induce fixation .............................. 25
Figure 1.6 Mean fixation scores before and after defixation by experimental condition ... 28
Figure 1.7 Mean fixation scores before and after defixation by educational background.. 29
Figure 1.8 Engineering students’ mean fixation scores before and after defixation by experimental condition ..................................................................................... 30
Figure 1.9 Mean number of concepts generated by engineering students before and after defixation by experimental condition ....................................................... 31
Figure 1.10 Concept resembling a circuit diagram from an electrical engineering student . 33
Figure 1.11 Functional decomposition with no unified concept .......................................... 33
Figure 2.1 Protocol analysis of a confirmatory case .......................................................... 53
Figure 2.2 Ratio of instances of confirmation to disconfirmation by group ...................... 54
Figure 2.3 Alternatives in Wason’s (1968) confirmation bias experiment ........................ 59
Figure 2.4 Problem 1 alternatives ...................................................................................... 60
Figure 2.5 Problem 2 evaluation concepts and example concept ...................................... 63
Figure 2.6 Coded participant matrix .................................................................................. 65
Figure 2.7 Coded participant notes .................................................................................... 65
Figure 2.8 Confirming and disconfirming instances evaluated between matrix and no matrix conditions .............................................................................................. 67
ix
LIST OF APPENDICES Appendix A Demographic Questionnaire ............................................................................. 89
Appendix B.1 Design Problem Before Defixation Task .......................................................... 90
Appendix B.2 Design Problem After Defixation Task ............................................................ 91
Appendix C.1 Defixation Task Instructions ............................................................................. 92
Appendix C.2 Defixation Task Story Material ........................................................................ 93
Appendix D Recall Test for Defixation Material .................................................................. 95
Appendix E Task Difficulty Questionnaire .......................................................................... 97
Appendix F Fixation Coding Instructions ............................................................................ 98
Appendix G Rater’s Raw Scores for Fixation Coding .......................................................... 99
Appendix H.1 Participant Concepts Ranked Low in Fixation ............................................... 105
Appendix H.2 Participant Concepts Ranked High in Fixation .............................................. 107
Appendix I Design Problems and Biological Analogies ................................................... 109
Appendix J Coded Verbal Protocol ................................................................................... 111
Appendix K Confirmation and Disconfirmation Coding Scheme ...................................... 118
Appendix L Confirmation Bias Problem 1 ......................................................................... 120
Appendix M Confirmation Bias Problem 2 ......................................................................... 122
Appendix N.1 Treatment Group Instructions for Concept Evaluation .................................. 124
Appendix N.2 Control Group Instructions for Concept Evaluation ....................................... 125
Appendix O.1 Example of Participant Concept Evaluation Matrix ....................................... 126
Appendix O.2 Example of Participant Concept Evaluation Notes ........................................ 127
1
THESIS INTRODUCTION
Concept generation and evaluation are critical phases of the design process; concept
generation affords the opportunity to develop a diverse and novel set of concepts, and concept
evaluation determines which of these will be subject to additional development and ultimately
production (Dieter, 2000). While conceptual design only accounts for a small proportion of the
cost incurred to bring a product to market (~5%), decisions made in this phase determine the
majority (70-80%) of the manufactured product cost (National Research Council, 1991).
Equally important is the fact that poor decisions early in the design process can become
compounded at later stages and lead to negative process and product outcomes. Therefore,
research aimed at enhancing design methods and designer capabilities during concept generation
and evaluation significantly benefit product design generally.
Creativity and decision-making are two integral components of successful conceptual
design; creativity is required for the generation of novel and viable concepts, and rational
decision-making is required to move towards optimal design outcomes. Both creativity and
decision-making have been directly studied in the field of psychology long before formal
branches of design science took similar interest. Because of this, and the growing desire to
better understand design cognition, recent trends in design research reveal an increasing
application of literature from psychology to design (e.g., in learning, analogical reasoning,
computational design, problem solving, etc.). However, valuable contributions still remain to be
made with respect to creativity and decision-making in design. For example, the fields of
biological psychology and neuroscience offer insights into the biological mechanisms of
cognition, however they remain largely overlooked in the design literature. In addition, the study
of cognitive heuristics and biases has greatly contributed to the understanding of human
2
decision-making, but literature pertaining to design biases seems to have developed
independently of these insights.
The purpose of this research is to further understand and mitigate obstacles to design
creativity and decision-making, by applying research from the fields of biological and cognitive
psychology. Practical applications of this research could contribute to enhancing design research
paradigms, developing new design tools and methodologies, and informing design education.
The remainder of this thesis is divided into 3 chapters. Chapter 1 discusses the role of
Long-Term Potentiation (LTP) – a mechanism of neural plasticity, in design creativity. A
theoretical discussion is presented regarding LTP’s role in Design Fixation – an unintentional
adherence to a limited set of concepts or problem strategies during design. This is followed by
the discussion of an experiment that was informed by the proposed model, which attempted to
alleviate design fixation via the incorporation of physical activity with a defixation exercise.
Chapter 2 discusses the role of cognitive heuristics and biases in design decision-
making. The emphasis is placed on understanding and mitigating the negative influence of
Confirmation Bias – a tendency to search for or interpret evidence in such a way as to maintain
pre-existing beliefs. Two research projects that sought to empirically evaluate the role of
confirmation bias in concept generation and evaluation are presented. This research incorporated
naturalistic observation and qualitative analysis, as well as controlled experimentation and
quantitative analysis. The first project examined verbal protocols collected from an
observational study, in order to identify whether or not confirmation bias was present during
concept generation. The second project was an empirical study examining the effect of using
formalized decision matrices to mitigate confirmation bias during concept evaluation.
Finally, Chapter 3 summarizes this research in its entirety. While two distinct research
projects are presented, the unifying theme throughout is to understand designer cognition from a
psychological perspective to enhance design creativity and decision-making.
3
CHAPTER 1 LONG-TERM POTENTIATION AND DESIGN CREATIVITY
1. INTRODUCTION
Creative thought is the process by which individuals and groups advance society,
whether through incremental changes to existing technologies, or through innovations that
change the perceived limitations of systems. Developing a better understanding of the cognitive
mechanisms of creativity supports concept generation in the design process (e.g., through
development of more effective tools and methods for innovation [Cagan, 2007]). While it has
been argued that any credible understanding of creativity must be consistent with a modern
understanding of brain function (Pfenninger & Shubik, 2001), researchers have only just begun
to examine the biological foundations of creativity and the implications this could have for
designers. This chapter will discuss the role of long-term potentiation (LTP) in design; this
neuro-biological phenomenon has been implicated in learning and memory development but is
under investigated with respect to creativity.
First, a theoretical explanation of creativity emphasizing LTP’s role in modifying neural
networks will be presented. Connectionist and associative theories of creativity are discussed, as
they serve as a foundation for understanding creativity in complex information-processing
networks. This theoretical discussion then turns to the role of LTP in design fixation, which is a
well-studied phenomenon that has been demonstrated to inhibit creativity during conceptual
design. Finally, the results of a study that arose from considering LTP’s role in creativity (the
effect of physical activity on fixation) are discussed. The goals of this research are to: 1)
Establish that a theoretical-neurological model based on long-term potentiation can contribute to
4
better understanding design creativity, 2) describe how this model could be applied to enhance
creativity in design, and 3) test a practical application of this model to mitigate design fixation.
2. THE BIOLOGY OF INFORMATION PROCESSING
The currently accepted view of information transmission in the brain can be largely
attributed to the work of Nobel laureate (1906) Santiago Ramon y Cajal. Cajal was a principle
proponent of the Neuron Doctrine, which states that neurons are discrete units responsible for
the processing of information in the brain (Andres-Barquin, 2002). Over the past century the
understanding of the central nervous system has been advanced, and there have been moderate
revisions to the neuron doctrine (e.g., signal transmission between neurons is more complex
then the unidirectional transmission of impulses between adjacent neurons, [Bullock et al.,
2005]). However, for the purposes of this research to understand the importance of LTP during
creative cognition, the following discussion is limited to the fundamental established principles
of neural transmission.
2.1. Neural Transmission
A basic understanding of how the human brain transmits information is needed before
beginning to understand how LTP could influence creativity. The following is largely adapted
from the discussion of neural transmission by Breedlove et al. (2007a).
The brain is composed of billions of interconnected processing units called neurons,
which consist of a cell body, axons, and dendrites (Figure 1.1). Neurons are the basic cellular
unit of the nervous system, and transmit information through electric and chemical signals.
Within a neuron, an electric signal transmitted is called an action potential. The action potential
is generated when a neuron’s membrane potential is altered in a process called depolarization.
Depolarization results from the flow of ions in and out of the cell body, and if the resultant
5
membrane potential of the neuron exceeds its threshold level (~ -55 mV) an action potential is
generated. The action potential is ionically propagated from the cell body, down the axon (an
extension of the cell body) to the axon terminals. Axon terminals are structures that form
synaptic connections with other neurons, and contain synaptic vesicles with various
neurotransmitters (molecules that chemically signal changes in neurons). The arrival of the
action potential at the axon terminals triggers the release of neurotransmitters, which diffuse into
the synapse, the gap between one neuron’s axon and another’s dendrites, where they bond with
receptors on the post-synaptic neuron (Figure 1.1). Dendrites are also extensions of the neuron’s
cell body, but unlike axons are specialised to receive signals from other neurons. Receptors are
structures on dendrites that bind with specific neurotransmitters, and produce specific responses
in the post-synaptic neuron upon successful binding. Based on the receptor characteristics of the
post-synaptic neuron, and the neurotransmitters released by the pre-synaptic neuron, the post-
synaptic neuron may or may not reach the threshold level of depolarization required to generate
an action potential and continue propagating the signal. Neuron physiology is variable, and there
are multiple types of neurons and neurotransmitters, however this introductory explanation is
sufficient to understand the role of LTP in information processing.
FIGURE 1.1. Structure of a neuron (Left) adapted from Breedlove et al. (2007a) and a network of neurons with an enlarged synaptic connection (Right) adapted from Young
(2007).
6
2.2. Long-Term Potentiation
Bliss et al. (2003) provide the following description of LTP:
In long-term potentiation, the strength of synapses between neurons in the central nervous system is potentiated for prolonged periods following brief but intense synaptic activation (pp. 607).
This description defines LTP as a mechanism responsible for increasing the likelihood that
activation from a pre-synaptic neuron will lead to activation in a post-synaptic neuron following
the co-activation of those neurons. If the synapse is the point of information transmission
between neurons, then LTP is responsible for an increase in the efficiency or likelihood that
information/signals originating at a pre-synaptic neuron will successfully propagate in the post-
synaptic neuron. Short-term potentiation (STP) refers to more rapidly occurring potentiation that
subsides quickly (i.e., 5-20 minutes [Malenka & Nicoll, 1999]) as opposed to LTP, which can
last from 30 minutes (Bliss & Lomo, 1973) to months (Barnes, 1979). Since the effects of LTP
and STP on neural connectivity are similar with respect to how they influence the spread of
activation in a neural network, this discussion uses the term LTP generally to describe neural
potentiation.
The first evidence for LTP was provided by Bliss and Lomo (1973), as well as Bliss and
Gardner-Medwin (1973), who observed long-lasting changes in the neurons of rabbits after
externally stimulating large groups of neurons. Their research demonstrated that the external
stimulation of neurons could result in an increase in their synaptic efficiency, even after the
stimulation was removed. Further research revealed that LTP was not limited to instances in
which neurons were stimulated artificially. Thompson et al. (1983) demonstrated that LTP
occurred in rabbits induced to exhibit a conditioned eye-blink response. The researchers were
able to electrophysiologically measure a change in synaptic efficiency between neurons in
response to the rabbits’ behavioural conditioning. Additional examples of the documentation of
LTP in response to behavioural conditioning can be found in Teyler and DiScenna (1987).
7
Breedlove et al. (2007b) provide an overview of some of the possible mechanisms
through which LTP occurs, including alterations in neurotransmitter release, neuron receptor
characteristics, synapse size, or enzymes that modulate neurotransmitters. LTP may also be the
result of changes to neuronal structure, such as an increased proliferation of dendrites and
subsequently an increased number of synapses between neurons.
Because LTP results from behavioural responses to external stimuli, it is not surprising
that it has been proposed as a mechanism for memory formation (Bliss & Collingridge, 1993)
and learning (Martinez & Derrick, 1996; Cline, 1998; Van-Praag et al., 1999). In a distributed
memory system, information is stored in a network of neurons through the connections and
changes in synaptic function between those neurons (Martinez & Derrick, 1996). Therefore, the
connections between neurons and the patterns of transmission between them are responsible for
the encoding and storage of memories. For new memories to be formed, the network as it exists
needs to alter the communicative distribution of activation, which is precisely what LTP
achieves. For a comprehensive explanation of the role of LTP in memory refer to Martin et al.
(2000). Most importantly, LTP is a mechanism through which the connections in a biological
information-processing network are altered, which has direct implications for how such a
network can “think” creatively.
2.3. The Biology of Creativity
Given that LTP has been proposed as a mechanism of learning and memory, it is
reasonable to assume that it is also involved in creativity. However, although psychologists have
studied creativity as a distinct cognitive process since the early 1900’s (MacDougall, 1905;
Perky, 1910), Jung et al. (2009) argue that, until recently, there has been little advancement in
developing a neuro-biological explanation of creativity. This section will discuss the existing
biological theories of creativity in addition to the proposed theory based on LTP.
8
The most well known theories involving neural correlates of creativity are likely those
involving hemispheric specificity. These theories assume that the two cerebral hemispheres have
functionally distinct roles; the right hemisphere of the brain is relied on for creative cognition
and the left is relied on for analytic cognition (Martindale, 1999; Heilman et al., 2003).
However, Dietrich (2004) has suggested that specific neural circuits, independent of
hemispheric specialization, are responsible for different “types” of creativity. Dietrich’s
argument is based on research indicating that different levels of cognitive regulation from pre-
frontal cortical areas dictate which neural circuits are relied on for information processing. Even
more recently Gabora (2010) proposed a neurological model of creativity in which “atypical”
neural structures are activated during creative thought; this activation supposedly allows
individuals to form new neural connections resulting in novel associations. Gabora’s theory
focuses on the role of connectivity between neurons in creative cognition, which highlights the
importance of understanding how that connectivity changes.
There has been little discussion on how mechanisms of neural plasticity such as LTP
influence creativity. Although Lippin (2001) credited Greenberg, an evolutionary biologist, for
supporting LTP as a possible biological mechanism involved in creativity, no detailed
explanation is provided or published. Others have discussed the influence of neural plasticity on
creativity more generally (Haier, 1993; Heilman et al., 2003), but do not directly implicate LTP.
Yet, as mentioned, LTP has been proposed as a mechanism of memory formation (Bliss &
Collingridge, 1993) and learning (Martinez & Derrick, 1996; Van-Praag et al., 1999); this in
combination with the fact that LTP moderates the connection strength between neurons suggests
that it could be involved in creative cognition as well. However, to fully appreciate the
implications of changing neural connectivity in creative cognition it is useful to consider
connectionist theories on the subject.
9
3. CONNECTIONIST THEORIES AND CREATIVITY
Connectionist theories are aimed at understanding and modelling cognition through
interconnected networks of simple units. Martindale (1995) provides two reasons to consider
connectionist models when developing cognitive theories: 1) these models unify multiple
psychological theories, and 2) directly relevant to the role of LTP in creativity, connectionist
models parallel models based on biological information-processing networks. Connectionist
theories also have practical applications in design. For instance, design techniques that
encourage distributed thought processes (e.g., brainstorming [Osborn, 1963], or the use of
random stimuli [de Bono, 1979]), are fundamentally tied to connectionist theories of associative
creativity (Mednick, 1962). This section provides an introduction to connectionist theory, in
order to demonstrate the clear link between LTP and creative cognition.
3.1. Information Processing in Connectionist Networks
Fodor and Pylyshyn (1988) describe connectionist systems as networks comprised of
simple but highly interconnected processing units. The terminology for these units varies, but
will be consistently referred to here as “nodes.” Fodor and Pylyshyn further note there are
various levels of connection strength between nodes in a network, and the connection strengths
and input activation at each node determine how information is transmitted through the system.
Parallel distributed processing theory (Rumelhart et al., 1986), which also describes the spread
of activation in connectionist networks, has been largely integrated into the modern
connectionist perspective and will not be discussed independently.
Spreading activation theory describes a model for searching connectionist networks
(Quillian, 1962; Collins & Loftus, 1975) and can be used to predict the likelihood that specific
nodes will be co-activated. As a network is searched activation spreads from a starting point
outwards, directed by the connection strength between nodes. Activation spreads out along
strong connections and is resisted at weak connections. Nodal connection strength therefore
10
dictates the spread of activation in the network and the frequency with which specific nodes are
co-activated after some input stimulation. In a biological connectionist network attentional
processes also dictate the spread of network activation. It is generally agreed that less focused
attention allows for a wider distribution of activation, whereas focused attention leads to the
activation of fewer nodes (Mendelsohn, 1976; Sternberg & Lubart, 1999). Dietrich (2004) has
also proposed that when attention is unfocused, the processing of information occurs in neural
circuits that are not regulated by conscious thought.
The fact that the distribution of activation in a network is dictated in part by nodal
connectivity highlights the importance of mechanisms that can alter this connectivity. Donald
Hebb (1949) was an early, if not the original, proponent of connectionist theory. Even before
LTP was documented he proposed that neurons that fired together would become more efficient
information processing units. For example, if an input entering the system triggered node A to
fire, and node A firing caused node B to fire, the connection between nodes A and B would
become more efficient as B fires more “easily” in response to activation from A. While this was
originally a theoretical proposal, research demonstrating LTP provided the mechanism through
which this change occurred in the brain.
Flexibility is critically important in developing a biological model of design creativity
because, as Helms and Goel (2012) argue, in conceptual design the design problem and solution
co-evolve. Designers must integrate and process information in novel ways if they are to
generate concepts that are truly innovative. This necessity to formulate new concepts is
facilitated by the ability of a flexible network to form new associations between nodes due to
changing connection strengths.
3.2. Network Activation and Associative Creativity
Mednick (1962) proposed a model of creativity based on associative hierarchies (which
can be considered as generalized networks) composed of various nodes that represent an
11
individual’s knowledge. The activation of any one node will, by association, lead to the
activation of others in the hierarchy. Mednick (1962) proposed that the associative strengths for
certain words or ideas would be stronger than others (e.g., “Table – Chair” vs. “Table – Food”),
and dictate the “slope” of an associative hierarchy. Steep slopes indicate strong associations
between relatively few concepts, whereas flat slopes indicate weaker associations between many
concepts (see Figure 1.2). Mednick proposed that the probability of developing creative
solutions was proportional to the number of associations made; therefore creativity becomes
dependent on how widely distributed activation is in the network, which is dependent on the
connection strengths between nodes.
FIGURE 1.2. Associative hierarchies around the word table (Mednick, 1962).
Despite Mednick’s (1962) belief that a more distributed spread of activation (or flatter
associative hierarchies) contributed to more creative ideas, diversity of associations alone is
unlikely to lead to enhanced creativity in the design context. In engineering design, and to a
varying degree in psychological research (see Dietrich, 2004; Dijksterhuis & Meurs, 2006 for an
example of the variability), creative ideas must be novel and feasible given established problem
constraints. One can imagine that a very distributed spread of activation could lead to the co-
activation of typically un-associated nodes, which in turn could contribute to extremely novel
0
2
4
6
8
10
12
14
Chair Cloth Wood Leg Food Mable
Asso
ciat
ive
Resp
onse
Sre
ngth Steep Associative
Hierarchy
Flat Associative Hierarchy
12
associations. However, having completely unrelated associations does not ensure that the
resulting concept is feasible.
To account for this, more recent research has focused on the role of graded activation
distribution within a connectionist network. Gabora (2010) proposed that widely distributed
activation in a network is necessary to develop concept originality, but more restricted or
focused activation in the network is necessary for developing concept feasibility. This proposal
seems reasonable if it is accepted that the novelty of an idea is positively correlated with the
number of associations formed in an information-processing network. However, Gabora
provides another explanation based on the frequency with which certain nodes are co-activated,
and suggests that a widely distributed spread of activation is more likely to lead to associations
between nodes that are weakly connected; these nodes would therefore have a lower probability
of being co-activated. In this model, creativity is not dependent on the number of associations
made but on the co-activation of weakly connected nodes leading to novel associations.
3.3. Summary
The spread of activation within connectionist networks provides an explanation for the
generation of creative thoughts. Strongly connected nodes will be frequently co-activated, which
may be required to develop the appropriateness or feasibility of concepts, however these
strongly connected nodes are less likely to contribute to the development of novel associations
and original ideas. Widely distributed activation, or the distribution of activation between
typically un-associated nodes, is likely required to generate novel concepts. Most importantly,
connection strengths in a biological information-processing network are altered by LTP. One
possible implication of this is that as the connection strength between one set of nodes is
enhanced, it becomes probabilistically less likely that activation will spread out along alternate
connections and lead to novel associations.
13
4. THE ROLE OF LTP IN DESIGN CREATIVITY AND DESIGN FIXATION
Alterations in the synaptic connection strength between neurons alter the spread of
activation between them. The spread of activation in a neural network and subsequent nodal
associations may influence creative cognition. The following section explores how LTP could
influence creativity in the design process. Specifically, the discussion focuses on design
fixation, as this phenomenon is a known obstacle to creativity during concept generation and
generally manifests in designers as a restricted diversity in associations and concepts.
4.1. Design Fixation
Generally defined, design fixation is “a blind, and sometimes counterproductive,
adherence to a limited set of ideas in the design process” (Jansson & Smith, 1991). However,
fixation can be discussed in more specific terms. Cardoso and Badke-Schaub (2011) refer to
three types of fixation: 1) Design fixation - the “reuse of previously seen features or principles,”
2) Mechanised thought – the adherence to a constant frame of thought (indicating designers are
essentially fixated on a problem-solving strategy), and 3) Memory blocking – fixation resulting
from “a cognitive obstruction during memory retrieval.” The present research is primarily
interested in the first category of fixation (i.e., the reuse of previously seen features and
principles) however mechanised thought is also discussed in terms of strategy adherence.
Fixation has been a phenomenon of interest in psychology since at least the 1940s, when
Duncker (1945) performed an experiment demonstrating that the initially perceived function of
an object unconsciously influences its subsequently perceived uses. Duncker presented
participants with: a box, tacks, and a candle; and asked them to affix the candle to the wall. In
one condition the tacks were presented in the box, while in the other condition the tacks and the
box were presented separately. In the latter condition, participants were more likely to use the
box as a platform to support the candle (i.e., tack the box to the wall and place the candle on the
box) than participants in the former condition, who were more likely to attempt and tack the
14
candle directly to the wall. This specific phenomenon is often referred to as functional
fixedness, defined by German and Barrett (2005) as a difficulty considering an item for a
function other than its most typical one.
Fixation is also often induced by exposure to stimuli presented during the concept
generation process, and can lead individuals to incorporate elements of those stimuli into their
own concepts (Jansson and Smith, 1991; Linsey et al., 2010; Perttula and Liikkanen, 2006).
Individuals may also become fixated on strategies they utilize (Simon, 1966; Cardoso & Badke-
Schaub, 2011), which can lead to ineffective or unnecessarily restricted problem solving.
With regards to the influence of fixation on design outcomes, Jansson and Smith’s
(1991) definition implies that it is not always counterproductive. An individual could be fixated
on a strategy that leads to a creative solution or positive design outcome. Therefore, assessing
the benefit or detriment of fixation is largely based on the outcome of the design process.
However, the goal of concept generation is typically to develop numerous ideas (Dieter, 2000)
that can be subsequently evaluated to determine which ones should be refined and selected for
development. Therefore, the influence of fixation is more likely to be detrimental when design
outcomes are dependent on the consideration of a diverse set of alternatives. In addition,
because design fixation can arise from exposure to pre-existing design solutions, it often inhibits
concept novelty.
Further complicating the effects of fixation, individuals are typically unaware that they
are fixated (Marsh et al., 1999), which makes it more difficult for them to mitigate the possible
influence of fixation; even experienced designers have been shown to fixate after explicit
instruction not to (Linsey et al., 2010). While it may seem surprising that experts exhibit this
behaviour, Chase and Simon (1973) demonstrate that experts often have difficulty processing
information from a perspective incongruent with their domain of expertise. This could be a
result of experts’ acquisition of a large body of knowledge related to their domain, which
15
reflects the development of a neural architecture specialized in efficiently processing domain
specific information. Mednick (1962) suggested that experts have steep associative hierarchies
(with respect to associations involving their field of expertise), which may in turn make it
difficult for them to form unconventional associations.
Finally, fixation is a universal phenomenon. For example, in an experiment by German
and Barrett (2005) members of an un-industrialized Ecuadorian tribe were primed with the
function of completely novel objects, and then asked to solve problems using those objects. The
researchers found that priming individuals with the function of an item, even when they had
never encountered it before, heavily influenced subsequent use of that item in a fashion
congruent with the primed use (relative to a control group that was not primed with the function
during problem presentation). This suggests that fixation has a biological foundation and is not
solely a result of socio-cultural learning or current educational paradigms. What remains is to
compellingly establish that LTP is at the root of fixation’s biological origins.
4.2. LTP and Design Fixation
From a connectionist perspective, fixation can be considered as the inability for
activation to spread to nodes other than those associated with the source of fixation. This view is
consistent with Ward’s (1994; 1995) theory of Structured Imagination and Path of Least
Resistance. Ward proposed that during the generation of novel ideas, central attributes of novel
concepts are determined based on the frequency they are associated with known representations
of similar concepts.
Groups of neurons that are frequently activated together develop stronger, more efficient
connections due to LTP. As a result, the spread of activation to nodes infrequently associated
with the fixation target becomes less likely. For example, in functional fixedness, repeated use
of an object for a specific function reinforces connections relating the concept of the object to its
typical use. This simultaneously makes it less likely activation will spread to a set of nodes
16
associated with atypical uses of the object. Nodes in this context must embody mental
constructs, such as functions, and are best represented by complex assemblages of neurons and
the patterns of connections between them (as opposed to individual neurons).
LTP could also be responsible for fixation arising from brief exposure to stimuli, such as
example design solutions. Bliss et al. (2003) reported that even the brief stimulation of neurons
could lead to LTP, and Malenka and Nicoll (1999) asserted potentiation could be triggered in
seconds. If this is the case, LTP could be responsible for fixation even when it results from brief
exposure to fixating stimuli. The presentation of an example solution activates neural pathways
linked to nodes in the brain responsible for internal representations of the stimuli, enhancing the
efficiency through which those pathways communicate information.
Fixation is not permanent. The discussion to this point may appear to suggest that LTP
could cause individuals to become permanently fixated as network connections become
progressively more efficient. However, this is not the case in reality and is not incongruent with
a theory of fixation involving LTP. First, as previously discussed, although LTP leads to long
lasting changes (with an upper limit that remains unknown) the duration is variable and typically
subsides over time (Bliss et al., 2003). In addition, there are biological mechanisms that act in
opposition to LTP, one of these being long-term depression (LTD). Long-term depression is
neurological mechanism that reduces the efficiency of synaptic connections and may normalize
synaptic weights (Derrick & Martinez, 1995). Therefore, while biological mechanisms of neural
plasticity may contribute to fixation, they are not so long lasting or dominant as to lead to
permanent fixation. In addition, individuals possess the ability to consciously moderate the
focus of their attention, and are constantly bombarded with a variety of stimuli that would lead
to the activation of different nodes. If the spread of activation in a connectionist network
originates from a point of stimulation, as the diversity of the input increases so too must the
distribution of activation. Therefore, the distribution of activity in the network may be
17
intentionally or unintentionally moderated based on the input. Considering these factors, as well
as the fact that there are billions of connections in the brain, it seems probabilistically unlikely
that any one group of connections could become permanently dominant.
4.3. Practical Contributions Regarding Design Fixation
The discussion to this point has focused on the theoretical role of LTP in design
cognition and fixation from a connectionist perspective. However, this theory has the potential
to make practical contributions to design theory and methodology. The following section
discusses how a cognitive model that incorporates LTP can be useful with respect to better
understanding and possibly mitigating design fixation. This will be followed by the discussion
of an experiment that sought to empirically test one of these derived applications.
4.3.1. Incubation and insight in design fixation. Incubation and insight are often
sequential phases in the ideation process; incubation is a period during which an individual stops
consciously working on a problem, after which moments of insight (i.e., enhanced problem
solving or creativity) occur (Wallas, 1926). In many instances, insight reflects the spontaneous
relief of fixation, as individuals are able to find a solution for a problem that was previously
obstructed by some unnecessary adherence to a problem strategy or concept. Incubation and
subsequent spontaneous moments of insight are well documented in anecdotal reports (e.g.,
Kekulé’s insight regarding the ring-shaped structure of benzene following a daydream of a
snake chasing its tail) as well as scientific evidence (e.g., Wagner et al. [2004] empirically
demonstrated that sleep could inspire insight and enhanced problem solving).
While there is strong support for the relationship between incubation and insight, debate
remains over the mechanisms responsible for this phenomenon. A popular belief is that insight
following incubation is the result of unconscious cognitive processing. For example, Dietrich
(2004) suggested that creativity arising from “spontaneous generation,” (e.g., daydreaming)
results from the down-regulation of executive control in pre-frontal cortical areas, allowing for
18
information processing using different neural circuitry associated with more divergent or less
constrained thinking. However, some researchers disagree that the unconscious mind processes
information in parallel with the conscious to develop new insights.
Smith and Blankenship (1989) suggest that forgetting sources of fixation, and not
unconscious processing, is likely responsible for insight following incubation. Similarly, Simon
(1966) argues that incubation allows individuals to forget problem strategies they previously
used to interpret problem-relevant information, once these strategies are forgotten new ones can
be developed or referenced leading to moments of insight. These perspectives hinge on the
assumption that forgetting obstructing information (defixating) facilitates insight. However,
because unconscious processing may utilize neural circuitry separate from conscious processing,
it is possible that both unconscious processing and forgetting fixation play a role in insight.
Interestingly, incorporating LTP into a theory of design cognition can inform this debate.
LTP is a mechanism responsible for the formation of memories and is therefore relevant
to the forgetting fixation hypothesis advocated by Simon (1966) and Smith and Blankenship
(1989). Adopting connectionist terminology, as network connections related to inefficient
problem strategies or information weaken or become less efficient, subsequent insight may
result from the spreading of activation to previously un-associated nodes in the network. When
individuals are not consciously directing attention to the problem during the incubation phase,
potentiation between previously activated nodes may subside. This proposal is congruent with
those empirical observations regarding the impermanence of LTP (Malenka & Nicoll, 1999;
Bliss et al., 2003). Alternatively, mechanisms such as LTD could be responsible for altering
synaptic weights. Therefore, if LTP were responsible for fixation, it would also lend support for
the forgetting fixation hypothesis (however it does not refute unconscious processing).
Whether or not insight following incubation is the result of forgetting fixation or
unconscious processing has implications for designers. If the unconscious processes information
19
in parallel, to offer insights up to the conscious, designers simply need to stop consciously
thinking about the problem and wait for subsequent creative insights. However, if forgetting
fixation is responsible, designers can pursue insight more actively. For example, forgetting
information works because it allows for the adoption of a new design strategy, therefore
designers may attain insights through actively discussing design problems with peers, in order to
gain a new perspective. Perhaps the most interesting application would be to help designers
detect when they are fixated, and subsequently facilitate the adoption of new strategies to
encourage more creative conceptual design.
4.3.2. Enhancing awareness of design fixation. A first step towards mitigating
design fixation (or fostering insight) is to help designers detect when they have become fixated
on non-optimal design strategies and concepts. A seemingly obvious approach in the current
context would be to directly measure LTP or neural activity (if the hypothetical relationship
between LTP and fixation were accepted). However, it seems unlikely that imaging techniques
used to detect patterns of neural activity, such as fMRI, could be practically incorporated into
the design process. While there are areas in the brain that have been shown to be responsible for
processing well learned information (e.g., the Fusiform Gyrus and Fusiform Face Area are
specialized in processing visually familiar information and developing category expertise
[Gauthier et al., 1999]), limitations in the resolution of imaging systems to detect LTP, in
addition to the cost and unwieldy nature of these devices, prevent this from being a practical
solution. Therefore directly measuring neurological changes or LTP in order to enhance
designer awareness of fixation is not currently a feasible solution. However, since repeated
activation of neural connections leads to LTP, and LTP is implicated in memory, more practical
methods to detect fixation could involve the use of memory metrics.
Associative maps. It may be possible to detect fixation by having designers generate
associative maps, which could be construed as rudimentary externalizations of an internal neural
20
network. If an individual generates one externalization and then attempts to regenerate it at a
later time, identifying overlap between the two externalizations may reflect sources of fixation.
Conversely, asymmetries between the two externalizations (e.g., forgotten nodes or new
connections) may indicate information that is poorly remembered and less likely to be a source
of fixation, which would be useful to expand on. Applying connectionist theory, externalized
maps could take the form of a network of nodes.
The content of these maps, including the information encapsulated in a node and how
connections are formed, would likely be subject to the desire of the designer and the nature of
the design task. However, existing design ontologies may be useful to inform the content. For
example, the content of the nodes could be dictated by Gero’s (1990) function-behaviour-
structure (FBS) framework. Functions describe what the object is for, Behaviours describe what
the object does, and Structures describe components and relations that encapsulate what an
object is; functions are derived from behaviours and behaviours from structures (Gero &
Kannengiesser, 2006). A designer could examine the concepts they have generated and use the
FBS framework to decompose concepts in order to identify nodes and connections for
associative maps. Figure 1.3 shows the beginnings of such a map, based on a design problem for
an automated plant watering system (see Appendix B.1). Other alternatives for the content of
these maps could involve externalizing more complex concepts in nodes, such as problem
strategies, however more complex approaches would be difficult to standardize.
In the example shown in Figure 1.3, there is only one structure associated with
administering water (sprinkler head), however there are two for regulating water flow (timer
valve, perforated membrane). This in itself may indicate that the designer is fixated on using a
sprinkler head to administer water and should consider other options. Alternatively, if the
designer were to generate another map later on, and realize that they were no longer considering
21
the use of a perforated membrane to regulate water flow but only a timer valve, it could indicate
that the individual had become fixated on that structure over time.
FIGURE 1.3. Hypothetical associative map based on the Gero’s (1990) FBS ontology. (Left
to Right: Function, Behaviour, Structure) Further research is required to determine how to best apply this techniques to detect
fixation in design (in addition to whether it is actually effective). Statistical testing could be used
to determine the degree of overlap between two externalizations indicative of fixation and not
average memory performance. In addition, overlap between any externalizations regardless of
the content may not always indicate inefficient problem solving, since fixating on problem-
relevant information may lead to successful design solutions. It may also be difficult to
standardize the method, which would be useful to validate it experimentally. Analyzing
concepts using the FBS ontology is a subjective task, and even researchers experienced with the
method do not exhibit perfect reliability. For example Gero and McNeill (2006) used the FBS
ontology to analyze design protocols, and reported discrepancies in the FBS coding performed
by one rater between design protocols. There is also the risk that repeatedly externalizing fixated
material could further reinforce fixation, if it encourages recall for fixating information that had
the potential to be forgotten. However, if little progress is being made this method has potential
to be used to identify areas of fixation. The process also involves taking a break from actively
Water plant
Regulate water flow
Timer valve
Perforated membrane
Administer water Sprinkler head
22
working on the problem (in the time between when the two maps are created), which may foster
insight through incubation.
Detecting fixation using recall speed or accuracy. An alternative method to use
memory to detect fixation, which may be easier to experimentally validate, was inspired by
psychological methodologies. While participants’ subjective reports of confidence can be used
as a metric of memory strength, these reports do not always match objective measures of
memory (Qin et al., 2011). Therefore, objective measures such as recall accuracy and speed of
recall, may more accurately reflect actual memory strength (and synaptic efficiency) than
subjective measures. Presumably information that is recalled more accurately or more quickly is
associated with stronger synaptic connections in areas responsible for encoding that information.
The relationship between fixation and memory could therefore be examined by correlating
participants’ recall for specific design features or strategies with the degree of fixation they
exhibit for those same features.
This hypothesis was examined in a pilot study that will be partially discussed.
Participants were shown a set of pictorial and verbal stimuli representing structures that could
be incorporated into solutions for a design problem that they would later solve. After being
exposed to the stimuli list, they were given the design problem and asked to generate multiple
solutions. They were then given a forced reaction time test for the same set of stimuli they had
been previously exposed to. It was hypothesized that the strength of fixation (i.e., frequency
stimuli were incorporated into design solutions) would negatively correlate with reaction times.
This experiment proved difficult to control experimentally. While negative correlations
were observed between participants’ reaction time to stimuli (r = -0.86 images, r = -0.87 words,
n = 8) and the frequency those stimuli were incorporated into their design concepts (see Figure
1.4.), participants’ interpretations of the stimuli list varied greatly. This made it difficult to
accurately interpret participant behaviour and responses. Essentially, participants brought an
23
outside knowledge base to the design problem that could not be incorporated into the recall test,
but was relied on to complete the design task. It was also unclear if memory strength for design
structures induced fixation, or repeatedly referencing a design structure (fixating on it) improved
performance on the reaction time task. Therefore, this research program did not proceed beyond
the phase of a pilot study.
FIGURE 1.4. Relationship between participants’ reaction times when judging stimulus
familiarity with the stimulus’ solution frequency (use in design solutions). 4.3.3. Using physical activity to enhance defixation. In a review of the effects
of physical activity, specifically aerobic exercise, on cognitive function, Kramer et al. (2006)
found that exercise was associated with enhanced cognitive processes including: planning,
working-memory, focused attention, and multi-tasking. Other researchers working with animal
models have found that physical activity is associated with improved brain plasticity, decreased
neuroatrophy, and enhanced LTP (Van-Praag et al., 1999; Cotman & Berchtold, 2002; Farmer et
al., 2004). These results indicate that there is an interaction between physical activity and LTP.
If LTP influences creative cognition and design fixation, it begs the question of how to
incorporate physical activity into the design task to provide some positive benefit for designers.
This research assumes that fixation occurs when the network activation has a restricted
0
50
100
150
200
250
300
350
400
450
500
0 1 2 3 4
Reac
tion
Tim
e (m
s)
Number of Structure Occurrences
Images Words
24
distribution, preventing the stimulation of nodes un-associated with the source of fixation. If an
individual were to engage in a defixation task (i.e., a task promoting the spread of activation
between nodes un-associated with the fixated material) and the connections between those un-
associated nodes were strengthened, then any subsequent spread of activation within that
network should be more likely to spread in a novel pattern, making fixation less likely.
Hypothesis. If physical activity were combined with a defixation task, it could lead to
enhanced defixation; LTP would enhance the synaptic efficiency between nodes that are not
associated with the fixation stimuli, while simultaneously allowing for the subsiding of LTP
between nodes related to the fixation stimuli. This would encourage the spread of activation
along new pathways in the neural network. In this case, the defixation task must engage the
individual in some activity unrelated to the design task, and could be viewed as a form of active
incubation. This hypothesis was tested in the following experiment.
5. STUDYING PHYSICAL ACTIVITY AND DEFIXATION
A study was conducted to examine the relationship between defixation and physical
activity. The initial hypothesis was that coupling physical activity with a defixation activity
would foster a novel distribution of activation in a neural network, mitigating fixation.
5.1. Methods and Procedure
5.1.1. Participants. Twenty-four University of Toronto students (16 males, 8 females)
volunteered to participate in the study. Participation lasted approximately 45 minutes;
participants were compensated monetarily ($15) for completion of the study. Table 1.1. displays
demographic and experimental information for the participants included in the analysis. The
majority of the participants were from the faculty of engineering; the remainder were from other
faculties with no one faculty constituting a second majority. The University of Toronto’s Social
25
Sciences, Humanities and Education Research Ethics Board granted approval for this research.
Participants were recruited through poster advertisements placed around the University
of Toronto campus and through e-mails sent to individuals known through the research labs
participant network. Participants were required to speak English fluently, to have no previous
experience speaking or reading Swedish (the defixation activity involved Swedish phrases), and
to have no previous experience with the design activity provided.
TABLE 1.1. Participant demographics Condition N Gender
Ratio Mean Age (SD)
Engineering Students (%)
Control 12 5♀ : 7♂ 22.4 (4.9) 7 (58) Treatment 12 3♀ : 9♂ 23.6 (5.9) 9 (75) Total 24 8♀ : 16♂ 23.0 (5.3) 16 (67)
5.1.2. Procedure. After participants read and signed the informed consent form they
completed a demographics questionnaire (see Appendix A). They were then given a design
problem that they were instructed to generate solutions for (see Appendix B.1); the design
problem was adapted from Perttula and Liikkanen (2006). The designer was asked to generate
concepts for an automated watering system that would administer 1/10th of a litre of water per
week to a potted plant. An example solution for this problem was shown to participants to act as
a source of fixation (see Figure 1.5). Participants were given 10 minutes to generate as many
concepts that solved the design problem as they could.
FIGURE 1.5. Example concept shown to participants to induce fixation (image and description from Perttula & Liikkanen, 2006).
A timer valve has been attached to a house water main. At predetermined intervals, the valve opens and allows the desired amount of water to flow through a showerhead onto the plant.
26
Defixation. After generating concepts for 10 minutes participants were stopped and
instructed to perform the defixation task (see Appendix C.1), which they were told was
“divergent thinking” activity meant to activate the other hemisphere of their brain. The
defixation task lasted 10 minutes, and required the participants to read a short story written in
Swedish with English translations (see Appendix C.2). Participants were told that they would be
tested on the material at a later point in the study, and were instructed to memorize the content
of the story as well as the translations of the Swedish phrases. The defixation task was intended
to stop participants from thinking about the design problem.
Physical activity. Participants were randomly assigned to either the control or treatment
condition. In the treatment group, participants were required to engage in physical activity in
combination with the defixation task. The physical activity required participants to perform
aerobic exercise by stepping up and down from an aerobic step-block (6-inches high) at a self-
determined rate; participants were asked to keep their rate of stepping comparable to climbing a
flight of stairs. Participants in the treatment condition performed the physical activity for the
entire 10 minutes that they took to complete the defixation task, the written material was
provided attached to a clipboard to allow them to step and read at the same time. In the control
condition participants performed the defixation task without physical activity and remained
seated at the workstation. This manipulation was the only change between conditions.
Design and recall tasks after defixation. Following the defixation task, participants
were given another 10 minutes to generate additional solutions for the design problem. They
were given a description of the problem again without the example solution (see Appendix B.2).
After participants completed this second round of concept generation, they were given the recall
task to test their performance on the defixation task. Participants were asked to translate 15
phrases used in the Swedish story and answer seven contextual questions (see Appendix D).
Then participants were asked to rate the perceived level of difficulty of each task (i.e., the level
27
of exertion of the aerobic exercise, the difficulty of the language defixation task, the difficulty of
the design problem both before and after the defixation task) on a scale of 1 (easy) to 7
(difficult) (see Appendix E). Finally, they were debriefed and paid.
5.1.3. Quantifying fixation. Two independent raters were instructed to evaluate
fixation in 123 participant-generated design concepts (see Appendix F). Fixation scores were
based on the similarity of each design concept to the example concept. Similarity was rated on a
scale from 0.0 - 1.0 (1 indicating high fixation/similarity and 0 indicating no fixation/similarity),
along 4 functional dimensions identified by Perttula and Liikkanen (2006): 1) water source, 2)
regulation of flow, 3) water transfer, and 4) energy source. A single fixation score for each
concept was calculated by averaging the scores across the four functional dimensions, resulting
in a concept score between 0 (not-fixated) and 1 (completely fixated). A participant’s fixation
score for both before and after the defixation task was calculated by averaging both raters’
concept scores across each concept generated from the respective design period.
The structure of the rating task conforms to Amabile’s (1982; 1996) Consensual
Assessment Technique for evaluating creativity; the raters: 1) had familiarity with the domain,
2) made their assessments independently, and 3) viewed the design solutions in random order.
The design task provided also conforms to guidelines for evaluating creativity outlined by
Amabile in that the task: 1) produces a clearly observable product and 2) allows for flexibility in
responses of the designer. Measuring fixation through concept similarity a generally accepted
method (Jansson & Smith, 1991; Purcell & Gero, 1996; Linsey et al., 2010)
5.2. Results
The focus of the analysis was to determine if the combination of physical activity with
the defixation task had an effect on participant fixation scores relative to the control condition.
However, additional analyses were performed to examine the effect of educational background.
28
5.2.1. Rater reliability. Average fixation scores were used as the metric for
comparison between the treatment and control group. Five of the 123 concepts (Numbers: 26,
27, 64, 95, 102) were excluded from analysis due to participant or rater error. The reliability of
the fixation measure was assessed using intra-class correlation (ICC). This method is used for
cases involving multiple raters evaluating multiple un-ordered observations in order to
determine absolute consistency and has been recommended by Shrout and Fliess (1979) for
similar rating scenarios. The computed intraclass correlation was statistically significant,
ICC(3,1) = 0.660, F = 5.23, p < .001, and indicated moderate consistency between the raters’
fixation ratings. The raters’ raw scores can be seen in Appendix G. Example concepts scored
high and low in fixation, before and after defixation, can be seen in Appendix H.1 and H.2.
5.2.2. Physical activity. A 2X2 (Physical Activity: Yes, No) X (Defixation: Before,
After) mixed-model ANOVA was used to examine the effect of physical activity on fixation
scores before and after the defixation activity (see Figure 1.6). There was no significant
interaction between defixation and physical activity, F(1,22) = 0.21, p = .65. Fixation scores
were not significantly different before or after defixation, F(1,22) = 2.89, p = 0.10. There was
no significant effect of physical activity on fixation scores, F(1,22) = 0.03, p = 0.86.
FIGURE 1.6. Mean fixation scores before and after defixation by experimental condition.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Physical Activity No Physical Activity
Mea
n Fi
xatio
n Sc
ore
Before After
29
5.2.3. Education. The sample used in this study consisted of 16 engineers (14 males, 2
females) and 8 non-engineers (6 females, 2 males), roughly balanced across conditions (see
Table 1.1). The analysis in section 5.2.2. was repeated including participant education
(Engineer, Non-Engineer) as a covariate (see Figure 1.7). A significant effect of education was
observed, F(1,21) = 5.82, p < 0.05. Non-engineering students had lower fixation scores overall
(M = 0.41, SE = 0.04) than engineering students (M = 0.63, SE = 0.05). There was also a
significant interaction between education and defixation F(1,21) = 5.00, p < 0.05.
FIGURE 1.7. Mean fixation scores before and after defixation by educational background. Follow up comparisons were performed using paired samples t-tests to examine the
effect of the defixation activity among engineers and non-engineers separately. Non-engineers’
fixation scores were significantly lower, t(8) = 3.38, p = 0.01, after the defixation activity (M =
0.33, SE = 0.07) than before (M = 0.49, SE = 0.06). However, engineers’ defixation scores did
not change significantly, t(14) = 0.07, p = 0.95, and were almost the same after the defixation
task (M = 0.63, SE = 0.06) as before (M = 0.63, SE 0.06).
5.2.4. Concept quantity among engineering students. The difference between
engineering and non-engineering students warranted the examination of the engineers’ data
separately. There was no significant difference in fixation scores between the treatment and
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Engineering Non Engineering
Mea
n Fi
xatio
n Sc
ore
Before After
30
control conditions, (see Figure 1.8), however cursory examination suggested a difference in the
number of solutions generated between groups. Therefore, a 2X2 (Physical Activity: Yes, No) X
(Defixation: Before, After) mixed-model ANOVA was performed comparing the average
number of concepts engineering participants generated.
FIGURE 1.8. Engineering students’ mean fixation scores before and after defixation by experimental condition.
There was no significant effect of the defixation activity F(1,13) = 3.27, p = .09 on the
number of solutions generated, however there was a marginally significant effect of physical
activity, F(1,13) = 4.52, p = .053 (see Figure 1.9). Engineers in the physical activity condition
tended to generate more solutions after the defixation task (M = 3.38, SE = 0.26) than before (M
= 2.75, SE = 0.37). However, engineers in the no physical activity condition tended to generate
the same number of solutions after defixation (M = 2.14, SE = 0.34) as before (M = 2.14, SE =
0.34). A paired-samples t-test revealed engineers in the physical activity condition generated
significantly more solutions after defixation than before t(7) = 2.38, p < .05.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Physical Activity No Physical Activity
Mea
n Fi
xatio
n Sc
ore
Before After
31
FIGURE 1.9. Mean number of concepts generated by engineering students before and
after defixiaton by experimental condition. 5.3. Discussion
The initial purpose of this experiment was to test the effect of physical activity on the
effectiveness of a defixation activity. Follow up analyses revealed potential confounds, and
prompted additional analyses accounting for education as a covariate. Interpretations of the
observed effects are presented.
5.3.1. The effect of physical activity. There was no significant effect of physical
activity on the effectiveness of the defixation task with respect to fixation. One possible reason
for this is the limited aerobic exertion and duration of exercise participants in the treatment
condition experienced. Participants were asked to step up and down from an aerobic step block,
set at 6 inches high, at a brisk and consistent pace. However, physiological measures, such as
heart rate, were not recorded and variability in the aerobic exertion actually experienced by
participants cannot be accounted for. In addition, studies in which physical activity has been
shown to enhance LTP have involved longer and more intense periods of activity. For example,
Van-Praag et al. (1999) observed physical activity enhancing LTP in mice that ran an average of
0
1
2
3
4
Physical Activity No Physical Activity
Conc
epts
Gen
erat
ed (#
) Before After
32
4.78 km per day for 30 days. The physical activity manipulation in this experiment was
therefore unlikely to significantly influence LTP.
Another potential factor accounting for the absence of a treatment effect relates to the
design task and example concept used to induce fixation. One possible cause of fixation is
strategy adherence (Smith & Blankenship, 1989; Simon 1966) (i.e., mechanised thought) and
participants may have been more fixated on the strategies they used to solve the problem than
the example concept. It seems unlikely that neural connections developed and strengthened over
a relatively long period of time (such as problem strategies) would be easily changed by
interventions occurring over a relatively short period (the defixation activity).
The lack of an observable effect could therefore be attributed to the fact that neither the
treatment (physical activity) nor the defixation activity was likely to encourage participants to
forget the way in which they approached the design task before the defixation task. Additional
measures (i.e., perceived difficulty of the design task, subjective exertion of the aerobic exercise
task, perceived difficulty of the defixation task, and performance on the defixation task) were
considered as possible covariates, however none of these variables differed significantly
between the treatment and control groups.
5.3.2. The effect of education. There was a clear effect of education on defixation,
which necessitated splitting the sample into two distinct groups. This observation is consistent
with previous observations, which suggest specific skill differences between individuals
contribute to complicating the assessment of creativity (Amabile, 1996). While the engineering
students were examined separately, the nine non-engineering participants could not be grouped
under any single faculty description.
Engineering students’ fixation scores were quite different than non-engineering
students’; engineering students tended to show few signs of defixating whereas non-engineering
students appeared to become less fixated after the defixation activity, independent of physical
33
activity. One possible explanation for this effect is that subject matter expertise contributed to
fixation in engineering students; qualitative evidence of this was seen in many of these
participants’ design concepts. For example, one electrical engineering student sketched concepts
that resembled circuit diagrams (see Figure 1.10), another used functional decomposition
without actually developing an explicit solution (see Figure 1.11).
Based on these qualitative observations, one factor contributing to engineering
participants’ fixation was likely an adherence to strategies they had learned for solving design
problems. This is consistent with Purcell and Gero’s (1996) finding that differences in
educational approaches between disciplines (mechanical and industrial engineering) can account
for differences in the individuals’ degree of fixation while solving design problems.
FIGURE 1.10. Concept resembling a circuit diagram from an engineering student.
FIGURE 1.11. Functional decomposition with no concept by an engineering student.
Another difference between engineering and non-engineering participants was their
performance on the language recall task. Non-engineering participants performed significantly
34
better on the recall task than engineering participants. Two possible explanations for this effect
are that engineering participants found the task more difficult, or they put less effort into it. The
former assumption is supported by a comparison of participants’ subjective ratings of difficulty
for the language task. Engineering students’ reported difficulty scores for the language task were
significantly higher than those of non-engineering students. This indicates that the two
educational groups experienced the defixation activity differently, and demonstrates the
importance of accounting for skill differences in the sample population during the development
of design experiments.
These educational differences have indirect implications when considering the role of
LTP on fixation. Engineers’ training may increase the strength of neural connections relating to
specific solution strategies, and as Simon (1966) argued, incubation effects and forgetting
fixation may be a result of forgetting these problem specific strategies. LTP would make it more
difficult for activation to spread along alternate paths for individuals whose pre-existing paths
are strongly connected through educational training. This finding is consistent with research
showing experts have difficulty generating solutions that are incompatible with their domain of
expertise (Chase & Simon, 1973). It also suggests that individuals lacking domain-specific
expertise relevant to the problem may be easier to defixate.
5.3.3. Concept feasibility. It was briefly considered that engineering students might
have been more fixated because they were focused on developing more technically feasible
concepts. However, engineering students did develop concepts that were not feasible or ignored
the limitations outlined in the problem description (e.g., hiring a gardener to water the plant).
There was no strong evidence to suggest that non-engineering students were more or less likely
than engineering students to ignore problem constraints. Therefore, the lower fixation scores
observed in non-engineering students were not likely a result of those participants sacrificing
feasibility for novelty. In fact, non-engineering participants’ solution feasibility showed no
35
significant evidence of being negatively impacted, relative to engineering students, by a lack of
domain specific expertise.
5.3.4. Quantifying fixation. Several concerns arose during data analysis due to the
difficulty of quantifying fixation. Although the intraclass correlation was significant, the
strength of the correlation (ICC(3,1) = 0.66) was moderate and raters did disagree over the
fixation ratings of many concepts. The design task was structured to allow participants
flexibility in generating solutions, however many participants’ solutions were ambiguous
regarding the functional categories used to rate fixation. For example, participants often
indicated that a reservoir would serve as a water source, however they did not specify how the
reservoir would be filled (e.g., municipal line or collected rainwater). The raters may therefore
score that functional dimension as partially fixated (0.5/1.0) if it is assumed the reservoir is
filled from a residential water line (the example source), or unfixated (0.0/1.0) if the rater
assumes the water is from a different source. The value of scoring fixation based on these
functional categories is dependent on whether or not participants explicitly address functional
requirements in their own solutions, which was often not the case.
Assessing fixation often involves evaluating concept similarity relative to the example
concept along some objective criterion (e.g., shared design features [Jansson & Smith, 1991;
Purcell & Gero, 1996; Chrysikou & Weisberg, 2005; Perttula & Liikkanen, 2006; Linsey et al.,
2010]). However, other metrics used include originality or novelty (Kurtoglu et al., 2009;
Linsey et al., 2010), concept quantity (Jansson & Smith, 1991; Purcell & Gero, 1996; Linsey et
al., 2010), or self-reports in verbal protocols (i.e., explicitly or implicitly evidencing difficulty in
developing new ideas [Nicholl & McLellan, 2007; Cheong et al., 2012]). The data collected in
this experiment could be reanalysed to determine if participants were fixated on problem solving
strategies and not the example provided by considering measures such as concept novelty, or
concept similarity between concepts (i.e., concept categories). For example, a participant who
36
repeatedly used solar power and rainwater in their concepts could be said to be fixated on
environmental solutions, but because the example concept provided did not utilize these
structures, the raters would have scored this participant as not at all fixated. The design
community has yet to agree on any single measure of fixation for concept generation studies and
challenges to construct validity need further research to address.
With respect to increasing the reliability of fixation ratings, the design task should be
structured to eliminate ambiguity regarding the fixation categories. Encouraging participants to
focus on developing complete solutions, or discarding ambiguous solutions, would increase
consistency. In addition, in discussions with colleagues, and the raters involved in this study, it
was determined that detailed training is necessary to ensure that the raters have a shared
understanding of fixation. Arbitration between raters could also be used to enhance this
understanding (Kan et al., 2010). However, although these approaches could enhance construct
validity, they could also introduce systematic individual biases into the rating task.
5.3.5. Concept quantity. There was a statistically significant relationship between
physical activity and the number of solutions generated by engineering students. Engineers who
performed the defixation activity combined with aerobic exercise generated significantly more
concepts after the defixation activity than before, whereas engineering participants who
performed the defixation activity without aerobic exercise generated, on average, the same
number of concepts before and after the defixation activity. In addition, the subjective reports of
task difficulty revealed that individuals in the physical activity condition found it significantly
easier to generate solutions after the defixation activity than participants in the control condition.
However, generating more solutions did not equate to generating less fixated solutions.
These results parallel findings regarding the traditional brainstorming technique.
Although brainstorming tends to lead to an increase in the quantity of ideas generated, it does
not contribute to increased idea quality (Stein, 1975). In the context of the methodological
37
limitations discussed so far, the reliability of these results can justifiably be viewed with
scepticism, in part owing to the number of analyses run and the inflated possibility of
committing a Type I error. In addition, based on the limitations discussed in section 5.3.1. it is
not being suggested that the effects observed were due to an interaction between physical
activity and LTP. However, the trends observed warrant future investigation to better understand
how combining physical activity with defixation tasks influences the perceived difficulty of
design tasks and how that may relate to solution quantity.
The fact that there was a difference in the number of solutions generated between groups
highlights a potential problem with averaging fixation scores across concepts. An important
challenge for future fixation studies is to accurately quantify fixation in a way that is not biased
by solution quantity. Reinig et al. (2007) identify 4 metrics for assessing the quality of ideation:
1) Number: the number of ideas generated, 2) Sum of Quality: the summed quality of ideas, 3)
Average Quality: the average quality of ideas, or 4) Good Idea Count: the number of ideas
generated that exceed a threshold quality. Using reliability analysis, Reinig and colleagues
determined that of the four methods, only good idea count was a valid measure of idea quality.
However, they acknowledge that this method ignores the number of poor ideas generated as
well as variance in the quality of good ideas. Future fixation studies would benefit by
considering these factors, especially, as was the case with the present study, when averaging
fixation values across multiple design concepts.
6. SUMMARY AND CONCLUSIONS
Previous design research has sought to explain creativity at different levels of
abstraction, from the influence of environmental factors to localizing creativity in the brain.
However, previous research does not explain how biological mechanisms that allow for
flexibility in a neural network can influence creativity in conceptual design. The present
38
research began with the assertion that concept generation and evaluation is biased by the
efficiency of connections in the brain, and that the efficiency of these connections is subject to
change via the neurological mechanism of LTP. LTP offers a possible explanation for how
connections between neurons become more efficient, contributing to biased cognition (e.g.,
design fixation).
Contrary to anecdotal evidence and intuitive expectation, an experimental study did not
reveal statistically significant effects of physical exercise on defixation. However, other
interesting results were discovered and discussed, such as the role of educational differences on
fixation in design. Methodological limitations were identified that could account for why the
expected effects were not observed. These limitations gave rise to a discussion of
methodological recommendations for future experiments, as well as other ways to re-examine
the data collected from this experiment.
This chapter offers a neurological explanation for design creativity and identifies a new
research area that can contribute to enhancing concept generation and creativity in design. In
addition, the role of LTP in design may become more relevant as techniques of moderating and
measuring LTP advance. The desire to understand the neurology of creativity is reason alone to
justify further research. However, to truly establish a causal relationship between LTP and
creativity in the context of design, studies that can assess neurological function directly are
necessary. Even had a strong effect of physical activity been observed in the study presented, it
would not definitively prove that LTP plays a causal role in fixation. Therefore, the research
focus in the following chapters shifts away from the neurobiological perspective.
39
CHAPTER 2 COGNITIVE BIAS AND CONFIRMATION IN DESIGN
1. INTRODUCTION
Design researchers have established an eclectic body of literature regarding design
cognition, with research interests ranging from cognitive science and a theory of design (Gero,
2009) to socio-cultural determinants of creativity (Liu, 2000). Psychological research on
cognitive heuristics and biases offers another relevant body of knowledge for application.
Cognitive heuristics are intuitive information-processing strategies that have been shown to, in
some instances, contribute to irrational judgments and cognitive biases (Tversky & Kahneman,
1982). Cognitive biases are, from a normative decision-making perspective, inherent judgement
errors in human information processing. Design researchers have only recently begun to explore
the role of cognitive bias in design (e.g., Viswanathan & Linsey [2010] studied the role of the
sunk cost bias in physical prototyping). However, cognitive heuristics and biases have been
studied in the field of psychology since at least the 1940’s (Asch, 1946) and can provide useful
insights to further understand design cognition and information-processing biases in design.
This chapter first introduces cognitive heuristics and biases, emphasizing their relevance
to design. Two studies are then presented that examined the role of confirmation bias – a
tendency to seek and interpret evidence in order to confirm existing beliefs, in design. The first
study analyzed verbal protocols from a biomimetic-design lab exercise to determine if
confirmation bias was present during concept generation. The results suggest that confirmation
bias is present during concept generation, and offer additional insights into the influence of
confirmation in design. The second study was a controlled experiment examining the
effectiveness of decision matrices as tools to mitigate confirmation bias during concept
40
evaluation. While the results indicate that decision matrices can effectively reduce confirmation
bias during concept evaluation, possible confounds in the study, as well as limitations to the
adoption of formalized decision-making procedures are also discussed.
2. COGNITIVE HEURISITICS AND BIASES IN DESIGN
There are notable differences between the heuristics of interest to psychologists and
those commonly discussed in the design literature. The following section discusses these
differences, and elaborates on the relevance of cognitive heuristics and biases in design.
2.1. Heuristics in Design and Psychology
Guindon and Curtis (1988) define design heuristics as broadly applied principles “that
reduce the complexity of a design problem.” Aronson et al. (2006) define psychological
heuristics as “mental shortcuts people use to make judgments quickly and efficiently.” Although
heuristics in design and psychology are superficially similar, further comparison reveals
fundamental differences in their origin and application.
Heuristics in design are formal rules or procedures deliberately developed for designers
to use during the design process. For example, Cormier et al. (2011) developed heuristics for
designers seeking to design products that satisfy consumer variation (e.g., a product to be used
by a both left and right handed population). Design heuristics are essentially tools that designers
can use when the situation is appropriate. Although their use may eventually become less
cognitively demanding with practice, the initial acquisition and application are conscious and
intentional. Cognitive heuristics differ in that they are not developed for application but are
observations of natural occurrences. They are innate information-processing strategies that
psychologists have “discovered” humans rely on. In addition, they are relied on without an
individual’s conscious intent (Gilovich et al., 2002) and are not explicitly learned. In fact,
cognitive heuristics are often discussed in terms of their adaptive benefit from an evolutionary
41
perspective. Because individuals do not consciously apply cognitive heuristics, they are often
unaware of how they could lead to cognitive biases and irrational judgments.
2.1.1. The use of cognitive heuristics in design. Cognitive heuristics are used
during cognitive processing, under which design is logically subsumed. These phenomena have
been shown to influence a diverse set of complex decision-making tasks in interpersonal
relationships, medicine, economics, politics, etc. (Gilovich et al., 2002). Given that decision-
making is a key component of design (Gero, 1990), design decision-making, at the least, will be
subject to a reliance on cognitive heuristics.
In addition to having an opportunity to rely on cognitive heuristics, designers also have
an incentive. An innate drive to conserve cognitive effort has been proposed as a hallmark of
human information processing (Fiske & Taylor, 1984). This desire is one reason why
individuals rely on cognitive heuristics, even when they have an incentive not to (Gilovich et al.,
2002). It is worth noting that design researchers have found evidence indicating designers are
also motivated to conserve cognitive effort (Guindon, 1990; Cheong et al., 2012). Therefore,
designers may unconsciously rely on cognitive heuristics to minimize cognitive effort, even
when they are highly invested in the design task.
There has been little previous research directly investigating the role of cognitive
heuristics and biases in design. While Viswanathan and Linsey (2011) have argued that the sunk
cost bias (i.e., a tendency to pursue a strategy because of previous investment despite the risk of
further losses) contributes to fixation during physical prototyping, this is only one design task
and one cognitive bias. Determining how designers use cognitive heuristics and biases requires
additional research into each phenomenon individually. This chapter focuses primarily on
confirmation bias in concept generation and evaluation, however a discussion of multiple
cognitive heuristics and biases that were deemed especially relevant to design will be presented.
42
2.2. Design Relevant Cognitive Heuristics and Biases
Although cognitive heuristics allow for efficient information processing and are
generally beneficial, reliance on them may contribute to cognitive biases. A Wikipedia search
(as of May 11, 2012) revealed an impressive number of empirically described cognitive biases:
86 in decision-making, 24 in social judgment, and 51 in memory. However, not all of these are
directly relevant to design cognition (e.g., social biases that relate to judgments in interpersonal
relationships). The following is a discussion of the heuristics and biases that were thought to be
the most relevant to design (see Table 2.1 at the end of Section 2 for a brief summary), for
further information on cognitive heuristics see Gilovich et al. (2002).
The Availability Heuristic. Tversky and Kahneman (1973) report that the availability
heuristic is relied on when making judgements based on the information that most readily comes
to mind. This can lead to biased information processing when the “availability” of information
is overly influenced by factors that do not reflect its actual diagnosticity (e.g., overestimating the
occurrence of shark attacks because they are highly salient incidents and are thus more available
in memory [Plous, 1993]). The availability heuristic can influence simple judgements based on
frequency estimates, but has also been shown to influence more complex and serious
judgements in real life (e.g., medical decisions involving complex surgery [Gifford–Jones,
1977]). The availability heuristic directly relates to the previous discussion on design fixation; it
could easily be one factor accounting for designers’ tendency to rely on well-learned strategies
when designing, or to incorporate elements of example solutions (which would be highly
available stimuli) into their own designs.
The Representativeness Heuristic. The representativeness heuristic biases judgement by
leading individuals to assume that a member of a category is a prototypical representation of
that category as whole (Kahneman & Tversky, 1972). This often leads to drawing inaccurate
conclusions about large groups from small samples, or small samples from large groups (e.g.,
43
stereotyping). This heuristic can also lead individuals to ignore base-rate information. For
example, the base rate for a coin coming up heads or tails is p = 0.5. However, after seeing a
coin toss come up heads 5 times in a row, most individuals intuitively feel that the next toss has
a higher than 50% chance of coming up tails. This is because of a belief that a small sample of
tosses should be representative of the outcome of a large number of tosses, even though the base
rate for the outcome of each individual toss is still p = 0.5 (Plous, 1993).
The Anchoring Heuristic. Tversky and Kahneman (1982) observed that individuals rely
heavily on initial reference points during estimates of frequency or probability. This is referred
to as the anchoring or adjustment heuristic; essentially individuals automatically adjust their
judgements relative to a reference point that may or may not be relevant. Tversky and
Kahneman illustrate how the anchoring heuristic can lead to biases in evaluating the outcome of
compound events. For successful product development, a series of events must occur; even
when the individual likelihood of success for each of these events is high, the overall likelihood
of each of them occurring can be very low. The anchoring heuristic can lead to overly optimistic
estimates for the outcome of conjunctive events, like product design, because the success of an
individual event is an anchor that biases the perception of the overall likelihood of success.
The Effort Heuristic. The effort heuristic leads individuals to evaluate alternatives based
on the amount of effort that went into developing them, as opposed to relying on more
diagnostic evaluation criteria. For example, if individuals believe something took a great effort
to develop they will have difficulty disentangling the actual value from this perception of effort
(Kruger et al., 2004). Reliance on this heuristic could lead individuals to make decisions that do
not take into account the true value of an alternative.
Sunk Cost Bias. The sunk cost bias refers to a tendency to maintain a course of action
due to previous investment (e.g., money, action, time) despite the fact that the prior investment
should no longer logically be influencing the decision (Arkes & Blumer, 1985). Viswanathan
44
and Linsey (2011) examined the potential effect of sunk cost bias on fixation, suggesting that
the act of building a physical prototype represents an investment, which in turn leads designers
to fixate on the current design strategy to avoid a loss of the invested effort.
The sunk cost bias can also lead to more harmful design outcomes than fixation.
Designers and manufacturers may consciously decide to launch products with known design
flaws to avoid losses associated with re-designing the product or launching late. While these
decisions are usually planned to be cost-optimal they do not always result that way (e.g., the
Ford Pinto’s unsafe fuel tank [Birsch & Fielder, 1994]).
Framing Bias. It has been repeatedly shown that even when given a choice between
normatively equivalent outcomes, individuals’ decisions are heavily influenced by how the
choice is framed (i.e., whether negative or positive outcomes are emphasized). Tversky and
Kahneman (1981) originally discussed framing as it applied to risky choice problems involving
gains or losses. They found that given the choice between normatively equivalent outcomes,
individuals strongly tend to be risk averse when dealing with positive outcomes, and risk taking
when dealing with negative outcomes. Levin et al. (1998) provide a summary of research
demonstrating the influence of framing on risk preference, attribute evaluation, and behaviour
adoption, in numerous contexts. Generally, individuals are more likely to act if the action
prevents a loss, as opposed to providing a gain. These framing effects can influence behaviours
with serious implications. For example, Meyerowitz and Chaiken (1987) found that women
were more likely to perform a self-breast exam when informed of the negative consequences of
avoiding the exam, than women who were informed of the positive consequences of the exam.
These effects could contribute to design fixation; for example a designer may be less likely to
abandon their current course of action if they are focused on the associated gains, instead of the
potential losses of committing to a new design strategy.
45
Hindsight Bias. Agans and Shaffer (1994) define the hindsight bias as the “unjustified
increase in the perceived probability of an event due to outcome knowledge.” This results in a
false sense of confidence when making judgements relating to outcomes that individuals have
knowledge of. This confidence is unjustified because without knowing the outcome, the ability
to predict it is severely limited. The hindsight bias can lead individuals to discredit others who
were unable to predict seemingly obvious outcomes, as well as preventing individuals from
learning from past events (Fischhoff, 1975).
Illusory Correlation. Chapman (1967) reports the illusory correlation bias can lead
individuals to report correlations between events that are not actually correlated, overestimate
correlations, or report correlations in the opposite direction. The illusory correlation bias has
been hypothesized to result from the availability or representativeness heuristics (Mullen &
Johnson, 1990). For example, Chapman (1967) presented participants with a series of word
pairings and asked them to rate how frequently each pairing occurred. Participants reported that
semantically linked words (e.g., bacon-eggs) co-occurred more often than words with no
semantic link (e.g., tiger-notebook) even though the number of pairings was equal in all cases.
Chapman and Chapman (1969) also demonstrated that pre-conceived beliefs often lead people
to perceive correlations that confirm those beliefs (e.g., personality traits and physical
appearance, such as untrustworthy people have tiny eyes).
Topical Mental Accounts. When making judgements involving multiple attributes,
people generally have difficulty accurately integrating all the relevant attributes at once.
According to Kahneman and Tversky (1984) only those attributes obviously and directly
relevant to the current aspect of focus are considered. Topical mental accounting has been
demonstrated to lead to biased purchasing behaviours. For instance, Thaler (1980) found that
people would be more willing to exert the additional effort of driving to a different store to save
$5 on a $15 calculator, than to save $5 on a $125 dollar coat; even when they are told they are
46
purchasing the two items together. In this example, individuals only consider the value of
savings relative to the item cost, instead of the multi-attribute purchase cost.
2.2.1. Design relevance. These biases and heuristics were discussed because it is
believed that they have the potential to influence design cognition. For example, as
hypothesized in Chapter 1, and by Hallihan and Shu (2011), the associative strength of design
stimuli in an individual’s memory could be predictive of fixation on that stimuli; this could also
be explained by a reliance on the availability heuristic. Tversky and Kahneman (1973) point out
that while memory works by strengthening connections between events that frequently co-occur,
availability works inversely to that, using the “strength of associations as a basis for the
judgement of frequency.”
This section is primarily intended as an introduction to the literature and to highlight the
heuristics and biases that were seen as particularly relevant to the design. The remainder of this
chapter will focus exclusively on the role of confirmation bias in the design process. This
decision was partially a result of confirmation bias being described as one of the most prevalent
information processing biases (Nickerson, 1998), as well as the difficulty of studying the other
phenomena described in a controlled setting. For example, to understand whether or not
designers are making decisions/judgements based on the most available information in memory,
the researcher must control or ascertain what the most available information in memory is.
TABLE 2.1. Design-relevant cognitive heuristics and biases. Heuristic or Bias Description Availability Making judgements based on the most available information in memory Representativeness A belief that a single instance of a category represents all instances of
that category Anchoring Using a baseline stimuli as a reference point for evaluating all other
stimuli Effort A belief that the value of something is attached to the amount of effort
put into it Sunk Cost Pursuing a strategy because of previous investment, despite the risk of
further losses Framing Allowing the frame (positive or negative) of a problem to influence
decisions Illusory Correlation Perceiving correlation where none exists Topical Mental Accounts Failing to accurately integrate all attributes of multi-attribute decisions
47
3. CONFIRMATION BIAS
Confirmation bias refers to a tendency to seek out evidence, or interpret evidence in such
a way, that is consistent with pre-existing beliefs, at the expense of considering belief-
inconsistent information (Nickerson, 1998). A confirmatory bias is evident even when
individuals have no vested interest in the belief being evaluated. For example, Koriat et al.
(1980) show that individuals typically attempt to find out if a belief is true, rather than prove
that it is false. Nickerson (1998) reports that confirmation bias can lead individuals to fail to use
disconfirming evidence to adjust beliefs, accept confirming evidence too easily, misinterpret
disconfirming evidence, and fail to consider the diagnostic value of supportive evidence.
Based on recent work (Cheong et al., 2012) it was hypothesized that confirmation bias
could be prevalent among novice designers during concept design and contribute to design
fixation, (i.e., prevent designers from fully considering the value of alternative design solutions
or strategies). Therefore, two studies were conducted in order to better understand the role of
confirmation bias in design cognition.
4. STUDY 1: CONFIRMATION BIAS IN CONCEPT GENERATION
The purpose of this first study was to determine whether or not confirmation bias was
present during concept generation, and to gain insights into its influence and possible
antecedents. The research described in Chapter 1 highlighted methodological difficulties
associated with quantifying fixation in the products of controlled concept generation
experiments. Therefore, a new methodological paradigm focusing on the naturalistic observation
of designers was adopted to examine the influence of confirmation bias in concept generation.
An observational study conducted by Cheong et al. (2012) provided the data collected and
analyzed in Study 1. This analysis, also discussed by Hallihan et al. (2012), will be presented
following a discussion of the use of observational methods in the study of design cognition.
48
4.1. Observational Research in Design
While experimental studies test the validity of hypotheses or interventions, observational
studies are well suited to formulate hypotheses and develop interventions for future
experiments. According to Dunbar (1995), an important benefit of an observational study is that
it allows researchers to observe more natural and real-world behaviors of participants, whereas
those behaviors may be restricted in experimental studies. Observational methods have become
increasingly common in design research, and have been previously used to examine:
biologically inspired design (Vattam et al., 2008; Helms et al., 2009), iterative design (Adams &
Atman, 1999), collaborative design (Tang & Lee, 2008), and design fixation (Nicholl &
McLellan, 2007). Cross (2001) provides a more detailed discussion of studies in design research
that employ an observational paradigm, including the advantages and limitations of this
approach; one challenge that will be discussed in more detail is objective data collection.
4.1.1. Verbal protocols as observational data. One method of collecting
observational data is to encourage participants to “think aloud”; these dialogues can be recorded
and transcribed to generate verbal protocols, which can be analyzed to offer insights into
participant cognition (Wickens et al., 2004). Gero (2010) supports the use of protocol analysis in
design research, arguing that it has been used extensively as a method to assist researchers in
understanding design cognition.
Ericsson and Simon (1993) discuss validity issues that should be considered when using
participants’ verbal reports as data (e.g., subjectivity in the coding process), but suggest that
instructing participants to think aloud does not significantly alter the cognitive process.
However, Chiu and Shu (2010) observed that the use of a concurrent think-aloud method in
design studies can be perceived as unnatural and may place an additional cognitive demand on
participants; this can contribute to results that may not reflect real-world performance. As an
alternative to actually vocalizing thoughts, designers can be encouraged to participate in design
49
processes naturally and talk aloud as they normally would. While this approach may not capture
cognitive mechanisms in as much detail, the process is more natural, may better reflect actual
design practices, and is better suited for collaborative design scenarios in which participants
contribute simultaneously.
4.1.2. Analyzing verbal protocols. Merriam (2009) recommends that the analysis of
protocols should ultimately be tailored towards meeting the needs of the researcher. One of the
most common methods of analyzing protocols in psychological and design research, and the
most relevant to this research, is qualitative coding.
Qualitative coding involves segmenting a protocol based on categories of interest to the
researcher, such as individual statements (Ericsson & Simon, 1993). Those segments are then
analyzed based on a predetermined coding structure. Miles and Huberman (1994) suggest that
researchers should develop meaningful and clearly defined categories for coding. Gero (2010)
has suggested that the Function-Behaviour-Structure (FBS) ontology provides a common
framework for representing design knowledge and allows consistency in coding verbal
protocols. However, while this may offer consistent and clearly defined categories, the
meaningfulness of the data is still dependent on the research objective. For example, the FBS
ontology is poorly suited to provide insights into the use of cognitive heuristics in design.
The current research is primarily concerned with biased cognition during concept
generation and evaluation, which is not necessarily well examined using any existing
methodologies. Instead the approach taken here, and by Cheong et al. (2012) and Hallihan et al.
(2012), combined structured coding and qualitative observation to gain new insights into the
cognitive processes of interest. This approach, of studying creative phenomena from the bottom
up, has been supported as a legitimate means to develop new insights in design research
(Brown, 2010). The following section presents research that relied on the qualitative analysis of
verbal protocols collected from participants engaged in a realistic design task.
50
4.2. Study 1: Method and Procedure
4.2.1. Participants. Thirty engineering students (28 males and 2 females) from a
fourth-year mechanical design course at the University of Toronto volunteered to participate in
the study. Participation was part of a voluntary design-by-analogy lab exercise during the
course. All data collected came from students who consented to have their design session audio-
recorded and to have the data used for research purposes. Participation lasted approximately 60
minutes; participants were not monetarily compensated, instead they were remunerated with the
design experience gained during the exercise. The University of Toronto’s Social Sciences,
Humanities and Education Research Ethics Board granted approval for this research.
4.2.2. Procedure. The lab exercise required participants to generate solutions for an
engineering design problem using a biological analogy as a source of inspiration. Three design
problems were provided, with each problem having a corresponding description of a biological
phenomenon as the source of analogy (see Appendix I). Three to four students were assigned to
a group and each group worked on a single design problem. There were three lab stations with
three groups at each station (see Table 2.2). While one group worked on the design problem, the
two observing groups at the lab station were instructed to categorize the designing group’s
activities (e.g., design fixation, correct/incorrect analogical transfer).
TABLE 2.2. Participant groups and assigned design problems. Lab Station Design Group # of Students Design Problem
1 4 Promotional Mailing 2 3 Authorized Disassembly A 3 3 Wet Scrubber 4 3 Wet Scrubber 5 4 Promotional Mailing B 6 3 Authorized disassembly 7 3 Authorized Disassembly 8 4 Wet Scrubber C 9 3 Promotional Mailing
Each group was given 20 minutes to generate solutions for the design problem. At the
beginning of each 20-minute session, each member of the design group was provided with a
51
written copy of the design problem and relevant biological phenomenon. One group (Group 9)
used only 12 minutes and stated they could not generate any more concepts.
The order of problems was counterbalanced (using a Latin Square) between each lab
station to control for problem effects. However, it is reasonable to expect the presence of a
learning effect for the second and third design groups at each station, since they had the benefit
of observing the preceding groups.
Design session mediators. A research assistant was assigned to each lab station to
facilitate and audio-record the design sessions. To control for any confounding effects
introduced by the research assistants, they were provided with a script to handle potential
questions from students, and were instructed not to contribute to the design process. The
research assistants were only to intercede when design progress slowed or the students had
settled on a design solution. After 20 minutes, the research assistant stopped the design session
and provided the next group with the corresponding design problem.
Design protocols. Participants in each design group were instructed to discuss their ideas
during the design process; these verbalizations were audio-recorded and transcribed for analysis,
however participants were not asked to verbalize all of their thoughts.
Two researchers transcribed the audio recordings from each design group. After each
transcript was generated, it was cross-reviewed by the other researcher to verify its accuracy.
Some audio data was not interpretable (e.g., multiple designers speaking at once, designers
murmuring very quietly, etc.) and this data was excluded from further analysis. These transcripts
constitute the data source used for the qualitative analysis; an example transcript can be seen in
Appendix J.
4.2.3. Qualitative coding. As previously mentioned, the coding scheme used for
qualitative analysis should be structured to inform, in an unbiased fashion, the research question
of concern (Merriam, 2009). A central component of confirmation bias is that it manifests itself
52
in a tendency to seek out or interpret evidence in a way that will confirm pre-existing beliefs.
Therefore, this analysis was structured to first identify designer beliefs, and then evaluate
instances of designers seeking or interpreting evidence pertaining to those beliefs as either
confirmatory or disconfirmatory (an example of a coded transcript can be seen in Appendix J).
Coding beliefs. The protocols were segmented based on individuals’ statements, as
recommended by Ericsson and Simon (1993). A belief was coded in a segment as any instance
when a participant verbalized a statement that conveyed his or her intent to influence the design
process in a desired direction (e.g., suggesting a design strategy or providing feedback regarding
the current design solution). Knowledge of the participant’s intent within the context of the
design process established the nature of the belief. Given that beliefs are subject to change,
instances when a participant stated conflicting beliefs were identified, and subsequent coding of
confirmation and disconfirmation focused on only the most recently affirmed belief.
Coding confirmation and disconfirmation. Once belief segments were identified,
subsequent segments were identified in which a participant either sought or interpreted
information that was relevant to the previously identified belief. That instance was then coded as
either confirmatory (an attempt to validate or support the belief) or disconfirmatory (an attempt
to invalidate or criticize the belief); ambiguous cases were excluded to mitigate bias. While
Ericsson and Simon (1993) generally recommend that a segment be coded based on information
contained within the segment itself, they acknowledge that segments can be coded based on
information from preceding or subsequent segments to offer additional context and resolve
ambiguity. In this case, coding a segment as confirmatory or disconfirmatory is clearly
dependent on information from previous segments that contain beliefs.
An example of a confirmatory case is seen in Figure 2.1; the biological phenomenon was
an Emperor Penguin’s thermoregulatory capability, the problem asked participants to improve
the efficiency of a wet scrubber that removes pollutants from exhaust gases (see Appendix I).
53
Designer A makes a statement coded as a belief that the shape of the penguin’s feet (part of the analogy) should be incorporated into the design solution.
A: I’m thinking that the penguin’s feet really looks like the scrubber, I’m not really sure of the shape of the scrubber, but I, I, [sic] I think the scrubber looks like the feet of a penguin.
For the next several minutes the group discusses potentially relevant features of the analogue. Designer A repeatedly mentions the importance of penguins’ feet and is criticized by Designer C.
C: I don’t think the penguin’s feet is uh important, like in this example. It’s actually not relevant, like relevant is the vein and the, and the [sic] artery, how they create the heat transfer…
Designer A temporarily stops discussing this aspect of the analogy. However, a moment later he makes a statement that is evidence he is reinterpreting the problem scenario to confirm his belief.
A: Yea, well we can also bring outside knowledge to this, to this design problem. Um, I think the wet scrubber looks exactly like a penguin’s feet. I’ve, I’ve [sic] seen one of them in the, (interrupted)
Designer B interrupts and questions the belief. B: You’ve seen one of them? Well, well [sic] what do they look like?
Designer A validates his belief. A: They look like a penguin’s feet. FIGURE 2.1. Protocol analysis of a confirmatory case.
There were instances when group members would disagree with each other and present
evidence aimed at disconfirming someone else’s belief (e.g., Designer C in Figure 2.1).
However, these cases were not coded as disconfirming because the designer presenting
disconfirming evidence could be doing so to support his or her own beliefs. Instead, the
evaluation focuses on how the designer being presented with the conflicting information reacts
in terms of evaluating the new evidence (e.g., accepting disconfirming evidence is failing to
exercise a confirmatory bias and coded as disconfirmation).
Analytical validity. It is possible that participants internally vet their ideas before
vocalizing them. This was not a true talk-aloud experiment and participants were working in
groups, which may have resulted in pressure on individuals to only vocalize ideas they felt
confident about. In addition, participants may have felt pressure to not externalize
disconfirmations to avoid appearing critical. Together, these factors could have biased the
dialogue towards confirmation. However, Ericsson and Simon (1993) state that under high
cognitive load, participants often stop verbalizing during think-aloud protocols. Considering that
the design task is presumably cognitively demanding, as it involved analogical comparisons
between novel biological entities and mechanical design problems, it is likely participants would
have had difficulty verbalizing their thought processes aloud regardless. In addition, the scenario
54
developed represents a realistic design situation. Therefore the results observed are believed to
offer an adequate representation of the influence of confirmation bias during concept generation.
4.3. Results
The following sections reports on the results of the protocol analysis, including
descriptive statistics and a discussion of insights gained through observation.
4.3.1. Ratio of confirmation to disconfirmation. A total of 107 instances were
identified as confirmatory or disconfirmatory. Table 2.3 shows the number of confirmatory and
disconfirmatory cases for each group. Figure 2.2 compares the ratio of confirmation to
disconfirmation for each group.
TABLE 2.3. Number of confirmatory and disconfirmatory cases per group. Group Number 1 2 3 4 5 6 7 8 9 Total Confirming 11 13 8 10 6 3 16 13 4 84 Disconfirming 3 5 1 3 0 0 5 6 0 23 Total 14 18 9 13 6 3 21 19 4 107
FIGURE 2.2. Ratio of instances of confirmation to disconfirmation by group. A chi-square goodness-of-fit was used to determine whether cases of confirmation and
disconfirmation were equally likely, based on the total number of observations with the
expected value for each cell set at chance assuming no bias existed (53.5). The results indicated
that confirmation was significantly more likely than chance, and disconfirmation was
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1 2 3 4 5 6 7 8 9
Perc
ent o
f Cas
es (%
/100
)
Group Number
Disconfirm
Confirm
55
significantly less likely than chance X2 (1, N = 107) = 33.64, p < .001. The average ratio across
all groups was 83% confirmation and 17% disconfirmation (SD = 12.1%). The data indicate that
participants’ discussions were biased towards confirmation during the lab exercise.
4.3.2. Qualitative observations. While qualitatively coding the protocols,
observations were made that offer additional insight into the nature of confirmation bias in
concept generation. While these observations cannot be statistically validated, they may be of
interest to practicing designers as well as design researchers.
Discounting factual evidence. When designers hold beliefs that can be contradicted by
factual evidence, it is reasonable to assume they will fail to demonstrate a confirmatory bias.
However, instances of belief perseverance in the face of contradicting evidence were observed,
which often led participants to misinterpret or ignore relevant information. For example, in one
instance a participant thought collecting demographic data would be a useful strategy to solve
the promotional mailing problem (see Appendix I). A group member stated that the design brief
specified demographic data was not available (which was correct). Still the former participant
attempted to persuade the group that gathering demographic information was a useful strategy, a
non-optimal response given the feedback he had received. In this way, confirmation bias could
contribute to design fixation or an unwillingness to compromise on design ideas; if designers
discount or ignore the criticisms of others in order to maintain a belief, they will be unable to
appropriately consider the value of alternatives.
Confirming analogies. Participants were given a design task that required them to use a
pre-determined biological analogy to inspire solutions for a specific design problem. When
participants developed solutions that utilized some aspect of the analogy, they frequently failed
to consider if the analogy was being applied inappropriately. This tendency may have
contributed to improper analogical transfer. The ability to identify relevant differences between
a target design and source analogue may facilitate analogical reasoning.
56
Seeking validation. Confirmation bias may influence the way designers question each
other. Participants frequently asked affirming questions (e.g., “What part of this idea do you
like?”). However, validating the strengths of existing concepts does little to better the design
situation, since these questions do not solicit information that would be informative to improve
concept quality. If designers sought information that highlighted flaws in their ideas they would
be better equipped to resolve those issues, resulting in improved concepts.
Confidence. An individual’s perceived confidence regarding their knowledge of the
design problem or biological phenomenon was believed to influence their reliance on the
confirmation bias. Participants who were seen as highly confident seemed more resistant to
disconfirming evidence. However, relying on confirmation bias during decision-making has
been demonstrated to inflate confidence (Nickerson, 1998). Given the observational nature of
this analysis, determining the direction of this relationship is not possible. In addition, the
assessment of confidence was determined by discussing this trait with the research assistants
who observed the design sessions, and is relatively subjective. However, the established
relationship between confidence and confirmation bias suggests that these factors would likely
interact and have the potential to influence designers.
Disconfirmation. Instances of disconfirmation were observed when participants
accepted evidence that contradicted a design belief, or when they identified limitations of their
own ideas. In both of these cases disconfirmation was associated with a perceived lack of
confidence in the belief in question. Designers who lack confidence in their ideas might be
quick to self-criticize, or be less likely to defend their ideas.
Design criticism. Participants were often hesitant when criticising the ideas of other
group members, which could arise from multiple causes (e.g., lack of self-confidence, courtesy,
etc.). However, even when criticisms were expressed, they were often vague or irresolute. This
unfortunately makes it easier for a designer to dismiss criticisms, and perpetuates a confirmatory
57
bias. Designers were regularly observed failing to see a flaw in their design strategy until they
were criticized multiple times with the flaw explicitly pointed out (i.e., specific criticism was
more effective than general criticism).
A possible antecedent of this behaviour is perpetuated through the principles of
brainstorming. A central principle of brainstorming is that criticism is not allowed (Dieter,
2000). However, criticism facilitates the identification of opportunities for improvement, and is
an integral part of knowledge construction in design (Bardzell et al., 2010). Encouraging
designers to withhold criticism could be fostering a culture that is ineffective in offering
valuable criticism (i.e., providing sufficiently detailed criticism, communicating criticism
effectively) and one that is unable to respond to criticism appropriately (i.e., recognizing the
value of criticism, maintaining a sense of self-efficacy in the face of criticism). While deferring
judgment in brainstorming may encourage divergent thinking, designers will also benefit if they
recognize the value of criticism and effectively offer and respond to it. In addition, the value of
criticism is inherently tied to the design process being used. Design-by-analogy is a unique
situation because concepts that incorrectly apply the source analogue are easily identified,
however the absence of this distinction in brainstorming may limit criticism’s value.
4.4. Protocol Analysis Limitations
It is necessary to acknowledge that because the researcher coded the verbal protocols,
instead of relying on independent raters, the validity and reliability of the reported analysis is
subject to researcher bias. However, even when using independent raters to code design
protocols, the raters must be trained to the point that they can accurately identify the phenomena
of interest. Part of the reason being that the phenomena of interest are abstract concepts and
interpreting them within the context of a design session is a difficult task. Regardless, the
present protocol analysis could be improved upon by having independent but trained raters
perform the coding in order to improve the reliability of the reported observations. Instructional
58
material to facilitate this was prepared (see Appendix K) but not implemented. Having
acknowledged this, the coding structure was developed and adhered to so as to attempt to
mitigate researcher bias, and the results of the analysis are robust in indicating that confirmation
bias is present during concept generation.
5. STUDY 2: MITIGATING CONFIRMATION BIAS IN CONCEPT EVALUATION
The results of Study 1 suggest that confirmation bias is present among novice designers
engaged in a biomimetic-design concept generation exercise. The purpose of the second study
was to examine the effectiveness of decision matrices as tools to mitigate confirmation bias
during concept evaluation. This study was motivated by previous research demonstrating that
formalized decision matrices can improve forecasting accuracy in information analysts by
reducing cognitive biases (Brasfield, 2009).
The study is divided and presented based on two problems participants were given.
Problem 1 was intended to replicate an earlier experiment on confirmation bias (Wason, 1968)
in order to determine if the participant sample in this study showed a similar propensity towards
confirmation as samples in previous experiments. Problem 2 was intended to examine the value
of using decision matrices to mitigate confirmation bias during concept evaluation.
5.1. Participants
Sixteen participants (2 female, 14 male) from the University of Toronto participated in
the study (completing both problems). Participants were recruited from a University of Toronto
graduate residence, as well as from the department of Mechanical and Industrial Engineering.
Participation lasted approximately 45 minutes and participants were compensated $10 for their
time. The University of Toronto’s Social Sciences, Humanities and Education Research Ethics
Board granted approval for this research.
59
5.2. Problem 1
Problem 1 was based on Wason’s (1968) card task, in which participants were asked to
test the condition: If a card has a vowel on one side, it has an even number on the other side.
Participants are shown 4 cards with: a vowel, a consonant, an even number, or an odd number
on the side facing up (see Figure 2.3), and asked to select the cards they think are necessary to
test the condition. This task simplifies to a test of the condition If P (vowel) then Q (even
number) by selecting among four alternatives that represent: P, NOT P, Q, and NOT Q. The only
choice that exclusively allows participants to falsify (disconfirm) the rule is NOT Q, as selecting
P could be motivated by either a positive or negative test strategy.
P
NOT P
Q
NOT Q
FIGURE 2.3. Alternatives in Wason’s (1968) confirmation bias experiment.
Wason (1968) observed that all participants selected P as necessary to test the condition,
and approximately 77% of participants selected Q. Selecting Q only allows one to confirm If P
then Q, as observing If NOT P then Q does not invalidate the condition. Very few participants
(28%) selected NOT Q and fewer still (17%) selected NOT P. This is evidence of confirmation
bias in the evaluation of a condition that individuals have no vested interest in and demonstrates
a tendency to attempt to determine if a belief is true, rather than if it is false.
5.2.1. Procedure. Wason’s original task was modified to provide participants with a
test condition more relevant to design. Participants were given Problem 1 (see Appendix L),
which instructed them to evaluate the belief: Washing machines that are highly water efficient
are also highly energy efficient. Participants were shown a set of stimuli (see Figure 2.4)
representing the conditions P (Water Efficient), NOT P (Water Inefficient), Q (Energy
Efficient), and NOT Q (Energy Inefficient). They were then instructed to pick two of the four
E K 4 7
60
machines that they would want to learn the remaining information about (the relative energy or
water efficiency) in order to optimally evaluate the belief. Participants exhibiting a confirmatory
bias were expected to test the belief by examining the P and Q conditions. Participants must
select the NOT Q to demonstrate an exclusively negative (disconfirmatory) test strategy.
Participants were instructed to record their selections on a blank sheet of paper, in
addition to listing any relevant considerations that contributed to their decision. Participants
were given as much time as they felt was necessary to make their decision. After they had made
a decision, the researcher asked them to verbally relate their decision-making process to clarify
any ambiguity in the written reports and ensure they were properly interpreted.
P
NOT P
Q
NOT Q
FIGURE 2.4. Problem 1 alternatives; lettered conditions (P, NOT P, etc.) not shown. 5.2.2. Results. The data collected from participant responses to Problem 1 can be seen
in Table 2.4. All participants selected the Water Efficient (P) alternative to evaluate the belief,
however the remaining decisions were distributed between the other alternatives. One
participant decided to select only the Water Efficient machine, and none of the others, resulting
in a total of 31 selections from 16 participants. The frequencies observed are reported, however
formal statistical testing is not performed as a cursory examination indicates participants were
no more or less likely to select NOT P, Q, or NOT Q.
TABLE 2.4. Participant responses for Problem 1. Water Efficient
(P) Water Inefficient (NOT P)
Energy Efficient (Q)
Energy Inefficient (NOT Q)
Participant Selections (%) 16 (100) 6 (37.5) 5 (31.3) 4 (25.0)
61
Only 25% of participants selected the NOT Q case in their decision. The remaining 75%
(including the participant who selected only the High Water Efficiency machine) chose
instances that would allow them to confirm the belief, or were irrelevant to evaluating the belief.
5.2.3. Discussion. The results presented in Table 2.4 show that 4 of 16 participants
selected the combination of P and NOT Q, in the experiment of interest from Wason (1968), this
ratio was 4 of 18. However, Wason allowed participants to select more than two alternatives,
and selections of three or four alternatives accounted for 1/3 of all participant selections.
Therefore, while the frequency of selections evidencing disconfirmation is relatively similar to
the previous study, the observed effects are in slightly different contexts (i.e., participants in
Wason’s experiment had more combinations of alternatives to choose from).
The differences here are not statistically analyzed, as the observed frequencies suggest
that no statistical difference exists between the NOT P, Q, and NOT Q, alternatives. However,
the primary purpose of this problem was to determine if participants in the present sample
behaved similarly to participants in previous ones, and based on a comparison of the observed
selection of the P and NOT Q combination this seems to be the case, although participant
selections for the NOT P and Q alternatives were quite different, possibly a result of limiting
participants’ selection to 2 of the 4 alternatives.
Pragmatism. Participant notes indicated that of the 12 individuals who failed to select
the energy inefficient machine to disconfirm the belief, 11 explicitly stated pragmatic
considerations as a factor (e.g., establishing a causal relationship between water and energy
efficiency, gathering evidence to identify correlation). Pragmatism has been forwarded as a
possible reason for individuals’ preference for confirmatory test strategies. Friedrich (1993)
gives multiple reasons for the value of pragmatism in decision-making (over detecting objective
truths) including the maladaptive nature of adopting negative test strategies. Friedrich suggests
that individuals demonstrate a confirmatory bias because it allows for better decision-making in
62
real life. For example, if an individual has a belief that a certain berry is poisonous it would
seem unnecessarily reckless to adopt a negative test strategy to prove the belief wrong. The
participants’ written reports support the pragmatic explanation, and although the majority fail to
consider disconfirming evidence, their decisions are not specifically maladaptive in this context.
Education effects. Interestingly, three of the four participants who selected the
disconfirmatory case were law students. Post-experiment interviews revealed that these
participants recognized that a statement in the form If P then Q does not imply If Q then P. This
eliminates the Q alternative as an option. These participants also all reported that they selected
the NOT Q condition because they understood it could provide evidence to disprove the
statement. Cosmides (1989) hypothesized that individuals are better at detecting disconfirming
evidence when it can be perceived as the violation of a social contract. The participants’ legal
education may help them to perceive contractual violations in a wider range of tasks than other
participants. However, this effect did not seem to influence performance in Problem 2.
5.3. Problem 2
Problem 2 (see Appendix M) was developed to provide insights into the effectiveness of
decision matrices in mitigating confirmation bias during a concept evaluation task. The
instructions given to participants for using a decision matrix were adapted from the Analysis of
Competing Hypotheses (ACH) methodology. ACH was developed by Heuer (1999) as a
decision-making tool to improve the forecasting accuracy of information analysts. The 8-step
method helps analysts generate a matrix that facilitates the comparison of alternative hypotheses
and the evaluation of the relevance and diagnostic value of gathered evidence. It has been
demonstrated to reduce reliance on cognitive biases, including confirmation bias, in complex
decision-making tasks with uncertain outcomes. In addition, it was demonstrated that ACH aids
participants in evaluating more information regarding a decision than participants relying solely
on intuition (Brasfield, 2009).
63
The Modified Analysis of Competing Hypotheses (MACH) procedure was developed for
participants to use in Problem 2. The modified version was reduced to 5 steps and instructed
participants to generate a matrix to compare and evaluate conclusions regarding the decision
task with respect to the available evidence (see Appendix N.1).
5.3.1. Procedure. Participants were first provided with a brief background on design
fixation (Jansson & Smith, 1991) and directed to evaluate the validity of the belief: The
presence of an example causes designers to fixate and incorporate elements of the example in
their solutions. To evaluate this belief they were shown the example solution for the design
problem discussed in Chapter 1, along with six concepts developed to solve the problem (see
Appendix M and Figure 2.5). Participants were told these six concepts had been generated by
individuals who had been exposed to the example solution. Two of the concepts (1 & 3)
incorporated multiple elements of the example solution, while the others (2,4,5,6) did not. These
design concepts were selected because they provide the participants with substantial evidence to
disconfirm the belief, and an ideal evaluation should reflect this ratio.
Example
1 2 3
4 5 6 FIGURE 2.5. Problem 2 evaluation concepts 1-6 (Hallihan & Shu, 2011) and example concept (Perttula & Liikkanen, 2006). Concept descriptions can be seen in Appendix M.
64
Participants in the control group were provided with instructions to evaluate the concepts
intuitively (see Appendix N.2) and to record relevant considerations in evaluating the belief as
point form notes on a blank sheet of paper. Participants in the treatment group were provided
with instructions to evaluate the belief using the MACH matrix (see Appendix N.1).
Before beginning to solve each problem, participants were questioned to ensure they
understood the problem as intended by the researcher. After completing each problem,
participants were interviewed, which provided an opportunity for the researcher to ensure they
were properly interpreting the participants’ written notes.
Participants were told that the average completion time for this task was 15 minutes, but
that they would have as much time as they wanted to reach an optimal conclusion. Their
performance was timed to allow for comparison of the duration of problem solving between the
treatment and control conditions. Timing began once participants read and indicated they
understood the instructional materials and began problem solving.
5.3.2. Coding confirmation and disconfirmation. Participants in the control
group were instructed to use blank sheets of paper and point form notes to record any relevant
information that they considered during their evaluation. Participants in the treatment condition
externalized their evaluation using the MACH matrix. These self-generated records were
analyzed to measure confirmation and disconfirmation. Written documentation indicating the
consideration of evidence, or argument for, confirming the fixation hypothesis was counted as
one instance of confirmatory evidence. Similar documentation that disconfirmed the fixation
hypothesis was counted as one instance of disconfirmatory evidence. The total number of
instances were counted for each participant. Examples of coded data can be seen in Figure 2.6
(matrix) and Figure 2.7 (no matrix); the original sheets can be seen in Appendices O.1 and O.2.
65
Degree of Fixation Features of Design From Example Features of Design from Outside
Sources
Concept 1 High Fixation
- Overhead release of water(C) - Fed by water line(C) - Sprinkler head(C) - Periodic release at intervals (requiring timer)(C) - Valve of some kind(C)
- Ball float valve(D)
Concept 2 Medium Fixation
- Fed by water line(C) - Overhead release of water(C)
- Water wheel release(D) - Continual release of water at fixed tempo (no timer required)(D)
Concept 3 High Fixation
- Overhead release of water(C) - Sprinkler head(C) - Fed by water line(C) - Periodic release at timed intervals (requiring
time)(C)
- Natural cloud source /fed by rainwater(D)
Concept 4 Low Fixation - ? [sic]
- Dripper release(D) - Continual release of water at natural tempo(D) - Soil fed stream(D) - No water line(D) - No timer required(D)
Concept 5 Low Fixation - Timer required(C)
- External movement brings plant to water (instead of bringing water to plant)(D) - Hydraulic lift required(D) - No flow of water stream(D) - Higher relative energy required(D)
Concept 6 Low Fixation - ? [sic]
- No water stream(D) - No timer required(D) - No external movement(D) - Sponge fed(D) - Soil fed hydration(D)
Figure 2.6. Coded participant matrix: 12 disconfirming(D) and 18 confirming(C) instances.
Top Left: incorporates water line(C) and a similar looking sprinkler head(C)
Top Middle: incorporates a house water line(C)
Top Right: incorporates many elements(C), except the water line(D)
Bottom Left: seems to incorporate no elements(D)
Bottom Middle: incorporates predetermined intervals(C)
Bottom Right: seems to incorporate no elements(D) Figure 2.7. Coded participant notes: 5 disconfirming(D) and 3 confirming(C) instances.
5.3.3. Results. The data collected from Problem 2 are seen in Table 2.5. Three
participants exhibited behaviour that was believed would unduly influence the analysis.
Participant 7 was assigned to the treatment group, but did not follow the MACH procedure as
outlined. Participants 6 and 12 were assigned to the control condition, however they utilized
matrices to formalize their decision process in a way that simulated the treatment condition.
While it was originally intended to test the hypothesis that the MACH manipulation would
66
mitigate confirmation bias, these three cases confounded the original comparison. Therefore, the
comparison focused on participants who utilized matrices to formalize their decision process
with participants who relied on intuition without a matrix (Matrix: Yes, No).
TABLE 2.5. Conditions and data for problem 2. Sub No. Group Matrix Major Confirm Disconfirm Time (min)
1 MACH Yes Zoology 8 10 15.7 2 Control No Genetics 4 4 6.3 3 MACH Yes Sociology 7 8 14.4 4 Control No Medicine 3 1 6.8 5 MACH Yes Law 9 9 20.2 6 Control Yes Law 12 18 20.5 7 MACH No Law 1 2 9.7 8 MACH Yes Law 4 1 10.2 9 MACH Yes Eng. 11 16 35.9 10 Control No Eng. 8 6 19.3 11 MACH Yes Eng. 5 11 17.9 12 Control Yes Eng. 6 18 22.8 13 Control No Eng. 5 3 11.8 14 Control No English 7 11 11.6 15 Control No Law 4 2 5.3 16 Control No Eng. 2 3 15.2
Effect of matrix. A one-way multivariate analysis of variance (MANOVA) was used to
examine the differences between groups (Matrix: Yes, No) with respect to evidence evaluated
(confirming, disconfirming). There was a statistically significant difference between groups,
F(2,13) = 4.95, p = 0.025, Wilks’ λ = 0.57, partial ε2= 0.43.
Follow-up comparisons (see Figure 2.8) were performed using independent samples t-
tests, with the Bonferroni Correction (α/2 = 0.025). There was a statistically significant
difference, t(14) = 2.69, p = 0.018, in the amount of confirming evidence evaluated: Matrix (M
= 7.75, SE = 1.00), No Matrix (M = 4.25, SE = 0.84). There was also a statistically significant
difference, t(14) = 3.15, p = 0.07, in the amount of disconfirming evidence evaluated: Matrix (M
= 11.38, SE = 2.05) No Matrix (M = 4.00, SE = 1.13).
67
Figure 2.8. Confirming and disconfirming instances evaluated between matrix and no
matrix conditions. Effect of time. There was a strong and statistically significant correlation between the
amount of time participants spent solving the problem and the quantity of evidence evaluated:
confirmatory (r = 0.72, p < 0.01), disconfirmatory (r = 0.76, p < 0.01). There was no interaction
between time and the type of evidence evaluated. Although the Matrix group identified
significantly more evidence than the No Matrix group, there was no statistically significant
difference in the number of items evaluated per minute.
Participant conclusions. With respect to the belief participants were asked to evaluate,
twelve of the sixteen concluded that there was evidence available to both support and reject the
fixation hypothesis. The remaining four participants (4, 8, 10, & 16) concluded that the fixation
hypothesis was well supported by this data. Participants 4 and 8 evaluated more confirmatory
evidence than disconfirmatory, and participants 10 and 16 evaluated more disconfirmatory
evidence than confirmatory. The latter case may be a result of those participants failing to
properly evaluate the diagnostic value of contradictory evidence, a known consequence of
confirmation bias (Nickerson, 1998).
5.3.4. Discussion. The primary purpose of this analysis was to evaluate the
effectiveness of decision matrices as tools to mitigate confirmation bias in concept evaluation. A
0
2
4
6
8
10
12
14
Matrix No Matrix
Evid
ence
Eva
luat
ed (I
nsta
nces
) Confirming
Disconfirming
68
number of methods already exist in design textbooks that utilize matrices to facilitate concept
evaluation (e.g., Pugh’s Concept Selection Method, Weighted Decision Matrices, Analytic
Hierarchy Process [Dieter, 2000]). The current findings indicate the use of formalized matrices
resulted in participants identifying more disconfirmatory cases than participants that did not use
matrices. Because the concepts evaluated (see Figure 2.5) provided more evidence against the
fixation belief than for, and after comparing the ratio of confirmatory to disconfirmatory
evidence evaluated between groups (see Figure 2.8), it can be concluded that the use of matrices
allowed participants to perform a less biased or more thorough evaluation.
The effect of time. It is possible that the observed effects were due to the increased time
spent on the task in the Matrix condition relative to the No Matrix condition. However,
participants in both cases decided themselves when they “reached an optimal conclusion.”
Therefore, the observed differences in the time spent on the task could have also been facilitated
through the use of a matrix. Future experiments could control for this variable to determine
whether time itself could produce the observed effects. However, while it would be relatively
easy to require all participants spend the same amount of time evaluating the concepts, the
method used here is more realistic as participants decided when they had reached an adequate
conclusion. In addition, based on observations of participants generating concepts in the
experiment described in Chapter 1, if too much time is provided participants may stop working
on the problem once they are content with their conclusions, regardless of time remaining.
Resistance to formalized methods. Three of the participants reported that using the
MACH matrix was an unnatural way for them to think, including Participant 7 who actually
seemed incapable of using the MACH table as an evaluation tool. Previous research has also
indicated that individuals often resist the use of formal decision-making methods, instead
preferring to rely on intuitive methods (Brasfield, 2009). This resistance limits the utility of any
formalized method and may negate its potential benefits if participants apply it incorrectly.
69
Cognitive effort. The resistance to use a procedure, and the failure to properly apply it,
may arise if the procedure requires increased cognitive effort to utilize. Given a limited
information-processing capacity, any method that requires additional processing may result in
decreased cognitive effort allocated to other concurrent tasks. However, even among novice
matrix users, the written record of the decision process could lessen working-memory load,
freeing up cognitive resources. Future research could examine the cognitive demand imposed on
participants using these methods through measures that assess cognitive workload (e.g., NASA
TLX [Hart & Staveland, 1988]). Anecdotally, the two participants in the control condition who
used matrices spontaneously evaluated the most evidence out of all the participants, and were
both above average in the number of items evaluated per minute. This suggests that individuals
who can use matrices intuitively have an advantage over those who cannot. Educating
individuals on the use and benefit of these methods would likely increase their utility.
Design relevance. The use of matrices to formalize the process of concept evaluation is
not new to design. However, this research highlights another benefit of their use, namely the
mitigation of cognitive biases, specifically confirmation bias. Comparing the effectiveness of
different existing concept evaluation methodologies in mitigating cognitive bias is an interesting
area for future research. One of the benefits of matrices in mitigating confirmation bias is that
they allow individuals to see how arguments that support the selection of one concept may apply
equally well to an alternate concept. A benefit of the ACH procedure specifically is that it
encourages users to generate disconfirming evidence; this formalizes the process of criticising
ideas and may make it easier to both administer and respond to criticism.
Additional insights from the protocol analysis suggest that successful methods should
compensate for individuals’ avoidance of criticism, preferential treatment of initially generated
concepts, false sense of confidence, and failure to consider disconfirming evidence.
70
Empirical limitations. Although statistically significant differences were observed
between experimental groups, the relatively small sample size limits the statistical reliability of
these findings. In addition, the sample included both non-engineering students and engineering
students. While it was not the initial intent to sample non-engineering students, their inclusion in
the study provided valuable insights (e.g., the possible moderating effect of a legal education).
However, as mentioned, (and demonstrated in the findings presented) in Chapter 1, engineering
students likely view design problems differently than non-engineering students.
Several participants (6, 12, 7) were included in groups for analysis that they were not
originally assigned to. One option would be to leave these individuals out of the analysis
completely; if this is done the same trends are observed, however the probability of these effects
being due to chance approaches p = 0.07. Considering the Bonferroni correction was applied,
this trend is far from statistically significant, however the direction of the relationship remains
unchanged and the decrease in sample size is somewhat accounted for by a decrease in the
sample variance once these participants are removed.
6. SUMMARY AND CONCLUSIONS
It was hypothesized that confirmation bias was an influential bias in design cognition.
The presence of confirmation bias during concept generation was identified through the analysis
of design protocols collected from engineering students engaged in a biomimetic design
practical session. Results of this analysis revealed that confirmation bias could contribute to
design fixation, a failure to identify the misapplication of analogies, and a tendency to ignore
relevant contradictory information. In addition, the influence of confirmation bias may be
magnified by overconfidence and a hesitance to be critical of others. Finding ways to encourage
designers to voice criticisms clearly and handle criticism effectively may mitigate confirmation
bias during concept generation. Negative feedback and increased personal accountability for
71
decisions have both been shown to decrease overconfidence (Arkes et al., 1987). Interestingly,
avoiding criticism reduces the presence of both of these factors, and overconfidence is
hypothesized to contribute to an increased reliance on confirmation bias.
The results of the second study offer insight on the use of decision matrices as concept
evaluation tools. Although these methods can mitigate confirmation bias, they can be met with
resistance and misapplied, limiting their benefit. In addition, the cognitive effort required to use
a method could hypothetically lead to an increased reliance on cognitive heuristics to offset this
demand. Adequate training on the use of any new methodology will likely address these issues.
Jin et al. (2006) proposed that formalized methods could enhance the concept generation
process. The focus of this study was on how formalized methods may be used to mitigate the
influence of cognitive heuristics and biases in concept generation, however these tools may have
other benefits. One possibility is that the use of tools like ACH can help individuals determine
the depth and breadth (quality and quantity) of the concept generation process and adjust
accordingly. A matrix that over emphasizes alternative concepts without incorporating relevant
evidence or supporting arguments for those concepts, may indicate that the concept generation
process is overly focused on quantity and not quality. On the other hand, too few or highly
similar concept alternatives may indicate fixation.
Finally, while the majority of the research in this chapter is concerned with confirmation
bias, the application of literature regarding all cognitive heuristics and biases is a promising area
for future research. This chapter outlines a number of these that are thought to be particularly
relevant to design.
72
CHAPTER 3 THESIS SUMMARY
The goal of this research was to gain a better understanding of design cognition through
the application of psychological theories. While the theories discussed (i.e., cognitive heuristics
and long-term potentiation) are distinct and may seem unrelated, they both afford a means to
understand designer biases in concept generation and evaluation. It is through this understanding
that new and more effective methods to mitigate biased design cognition can be developed.
Long-Term Potentiation and Design Creativity
In Chapter 1, an argument is forwarded that emphasizes long-term potentiation as a
phenomenon directly relevant to design cognition. This argument in itself is not particularly
novel, as LTP must influence cognition if it changes the way neurons communicate information.
What is novel is that this argument is expanded to discuss creativity specifically, which has yet
to be publically reported. Then, in an attempt to demonstrate the actual value of considering
LTP in design cognition, a theoretical explanation of design fixation arising from LTP is
presented. The principal value of this theoretical discussion for the design community is to
provide a new perspective that can further the understanding of biased design cognition. In
addition, hypothetical interventions for mitigating fixation through enhancing awareness, based
on detection using metrics of memory, are presented. While these are not empirically validated,
the link between fixation and memory is supported by previous research.
The empirical study conducted did not validate a hypothesis based on one application of
this theory (i.e., the influence of physical activity on the efficacy of a defixation exercise).
However, the results still offer valuable insights to contribute to the research community’s
73
understanding of design methodology and design fixation. It was demonstrated that individuals
with engineering backgrounds behave differently from participants with non-engineering
backgrounds; although not surprising or novel in itself, the result emphasizes how educational
differences may contribute to design fixation. In addition, the finding that engineering students
in the physical activity condition generated more solutions without generating higher quality
solutions supports the independence of concept quantity from quality. Although the reliability of
this finding is limited due to issues associated with running multiple statistical tests. Finally this
research demonstrated, unintentionally, how the metrics used to assess fixation dictate the
outcomes from fixation experiments. For instance, even when evaluating fixation on an example
with a relatively simple coding scheme (i.e., similar or not), raters show surprisingly high levels
of disagreement. This is likely because the concepts participants generated were not always
congruent with the coding scheme based on functional categories. Metrics used to quantify
fixation should be sensitive to variability in participant responses during concept generation
exercises. While there are multiple ways to assess fixation, the design literature offers no one
definitive and accepted method. Establishing measures that are not influenced by participants’
interpretation of the design task, or concept quantity, are promising areas for future research.
Cognitive Bias and Confirmation in Design
Chapter 2 discusses cognitive heuristics and biases that may be of particular relevance to
understanding design cognition, with an emphasis on the role of confirmation bias. The
overview of heuristics deemed to be the most relevant to design is largely meant to be an
introduction to designers unfamiliar with the literature. The primary focus of this chapter was to
illuminate the influence of confirmation bias during concept generation and concept evaluation.
The first study presented analyzed verbal protocols to determine whether or not
confirmation bias was present among designers during a biomimetic concept generation
74
exercise. The results provide evidence for the presence of confirmation bias, however the
reliability of this conclusion cannot be discussed from the perspective of reliability coefficients
between independent raters. While this is not optimal, previous researchers have performed
qualitative analyses in a somewhat similar fashion. Chrysikou and Weisberg (2005) used non-
independent raters to analyze written protocols in order to identify fixation, afterwards
recruiting an independent rater to code 30% of a single protocol in order to provide a measure or
reliability. Christensen and Schunn (2007) analyzed verbal protocols from an analogical-design
session in a similar fashion, using an independent rater to analyze 18% of the total data in order
to provide some measure of reliability. While these cases still partially rely on independent
raters, they rely more heavily on the authors performing qualitative analyses themselves. It is
this researchers’ belief that qualitative analyses provide more valuable and accurate insights into
the phenomena of interest if the raters used have domain expertise, which may require
substantial training for independent raters to acquire. In addition, while performing an analysis
independently may enhance the reliability of the analysis, arbitration and discussion of
discrepancies that arise (methods also previously used in design [Gero & McNeil, 2005]), will
enhance its construct validity.
Qualitative observations made during the coding process are discussed, and while they
cannot be validated statistically they offer additional insights. Perhaps the most interesting
observation relates to the role of criticism during concept generation. It is often espoused that
criticism should be avoided during this process to facilitate group creativity, however the
observations here suggest criticism may serve the beneficial function of mitigating cognitive
bias during design. Reliance on confirmation bias may inflate problems associated with ignoring
or avoiding criticism during the conceptual design process.
The second study examined the role of confirmation bias in concept evaluation. The first
problem presented was originally intended to compare the participant sample to previous ones
75
with respect to their propensity towards confirmatory test strategies. However, it was observed
that educational differences influenced how participants perceived the problem. Law students
seemed more likely to view a problem and attempt to disprove a statement than to seek to
confirm it, however this educational training did not translate into avoiding confirmation bias in
the second problem. Additionally, the majority of participants adopted a pragmatic problem
solving approach, which indicated that they were evaluating a belief that was not necessarily
maladaptive. However, the results observed in Problem 2 demonstrated that reliance on
confirmation bias during concept evaluation could have a detrimental influence.
The second problem was intended to evaluate the effect of using formalized decision
matrices on confirmation bias during concept evaluation. Three participants exhibited behaviour
that confounded the analysis, and as such the comparison did not specifically evaluate the
effectiveness of the MACH procedure developed for this study. However, the results
demonstrated that the use of decision matrices mitigated confirmation bias, contributing to a
more thorough and less biased concept evaluation. One limitation on this finding is that the time
participants spent evaluating the concepts was not controlled for. However, previous research
has demonstrated that using formal decision matrices does mitigate confirmation bias (Brasfield,
2009), and in this study it can be argued that the decision matrix at least the very least facilitated
the increased time spent during concept evaluation.
Conclusion
While this research is based on well-established psychological literature, its application
in the field of design theory and methodology offers both novel and practical contributions.
Theoretical observations on the underlying biology of creative design phenomena await
validation, however the argument presented provides a new perspective on creative design
phenomena. This research has hopefully convincingly demonstrated the value of considering
76
cognitive heuristics in design cognition, as well as a method to mitigate negative outcomes
associated with confirmation bias. Obstacles and design biases that hinder creativity in
conceptual design, such as design fixation, must have psychological origins. Understanding the
psychology of designers is a necessary step in the development of a complete theory of design.
77
REFERENCES
Adams, R., & Atman, C. (1999). Cognitive processes in iterative design behaviour. Proceedings
of the ASEE/ISEE Frontiers in Education Conference, San Juan, Puerto Rico, 11a6-13 –
11a6-18.
Agans, R., & Shaffer, L. (1994). The Hindsight Bias: The Role of the Availability Heuristic and
Perceived Risk. Basic and Applied Psychology, 15(4), 439-449.
Amabile, T. (1982). Social Psychology of Creativity: A Consensual Assessment Technique.
Journal of Personality and Social Psychology, 43(5), 997-1013.
Amabile, T. (1996). Creativity in Context. Boulder, CO: Westview Press Inc.
Andres-Barquin, P. (2002). Santiago Ramon y Cajal and the Spanish school of neurology. The
Lancet Neurology, 1(7), 445-452.
Arkes, H., & Blumer, C. (1985). The Psychology of Sunk Cost. Organizational behavior and
human decision processes, 35(1), 124-140.
Arkes, H., Christensen, C., Lai, C., & Blumer, C. (1987). Two Methods of Reducing
Overconfidence. Organizational behaviour and human decision processes, 39, 133-144.
Aronson, E., Wilson, T., Akert, R., & Fehr, B. (2006). Social Psychology (3rd ed.), Upper Saddle
River, NJ: Pearson Prentice Hall.
Asch, S. (1946). Forming impressions of personality. Journal of Abnormal Social Psychology,
41, 258-290.
Bardzell, J., Bolter, J., & Löwgren, J. (2010). Interaction Criticism: three readings of an
interaction design, and what they get us. Interactions, 17(2), 32-37.
Barnes, C. (1979). Memory deficits associated with senescence: a neurophysiological and
behavioural study in the rat. Journal of Computational Physiology and Psychology, 93, 74-
104.
Birsch, D., & Fielder, J. (1994). The Ford Pinto Case: a study in applied ethics, business, and
78
society, Albany, NY: State University of New York Press.
Bliss, T., & Gardner-Medwin, A. (1973). Long-lasting potentiation of synaptic transmission in
the dentate area of the unanaesthetized rabbit following stimulation of the perforant path.
Journal of Physiology, 232, 357-374.
Bliss, T., & Lomo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate
area of the anaesthetized rabbit following stimulation of the perforant path. Journal of
Physiology, 232, 331-356.
Bliss, T., & Collingridge, G. (1993). A synaptic model of memory: long-term potentiation in the
hippocampus. Nature, 361, 31- 39.
Bliss, T., Collingridge, G., & Morris, R. (2003). Introduction: Long-term potentiation and the
structure of tissue. Philosophical Transactions of the Royal Society B; Biological Sciences,
358(1432), 607-611.
de Bono, E. (1970). Lateral Thinking. New York, NY: Harper & Row.
Brasfield, A. (2009). Forecasting Accuracy and Cognitive Bias in the Analysis of Competing
Hypotheses. (Master’s Thesis). Mercyhurst College, Erie, PA.
Breedlove, S., Rosenzweig, M., & Watson, N. (2007a). Biological psychology: an introduction
to behavioural, cognitive, and clinical neuroscience (5th ed.), Ch. 3-4, Sunderland, MA:
Sinauer Associates.
Breedlove, S., Rosenzweig, M., & Watson, N. (2007b). Biological psychology: an introduction
to behavioural, cognitive, and clinical neuroscience (5th ed.), Ch. 18, Sunderland, MA:
Sinauer Associates.
Brown, D. (2010). The curse of creativity. Design Computing and Cognition’10, Stuttgart,
Germany: Springer.
Bullock, T., Bennett, M., Johnston, D., Josephson, R., Marder, E., & Fields, D. (2005). The
Neuron Doctrine, Redux. Science, 310(5749), 791-793.
79
Cagan, J. (2007). A look at the emerging science of innovation. Artificial Intelligence for
Engineering Design, Analysis and Manufacturing, 21, 13-14.
Cardoso, C., & Badke-Schaub, P. (2011). Fixation or Inspiration: Creative Problem Solving in
Design. The Journal of Creative Behavior, 45(2), 77-82.
Chapman, L. (1967). Illusory Correlation in Observational Report. Journal of Verbal Learning
and Verbal Behavior, 6, 151-155.
Chapman, L., & Chapman, J. (1969). Illusory correlation as an obstacle to the use of valid
psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271-280.
Chase, W., & Simon, H. (1973). Perception in Chess. Cognitive Psychology, 4, 55-81.
Cheong, H., Hallihan, G., & Shu, L. (2012). Understanding Analogical Reasoning in
Biomimetic Design: An Inductive Approach. Design Computing and Cognition’12, College
Station, TX: Springer.
Chiu, I. & Shu, L. (2010). Potential limitations of verbal protocols in design experiments.
Proceedings of ASME IDETC/CIE (DETC2010-28675), Montreal, QC: ASME.
Christensen, B., & Schunn, C. (2007). The relationships of analogical distance to analogical
function and preinventive structure: The case of engineering design. Memory & Cognition,
35(1), 29-38.
Chrysikou, E., & Weisberg, R. (2005). Following in the Wrong Footsteps: Fixation Effects of
Pictorial Examples in a Design Problem-Solving Task. Journal of Experimental Psychology,
31(5), 1134-1148.
Cline, H. (1998) Topographic maps: Developing roles of synaptic plasticity. Current Biology, 8,
R836-R839.
Collins, A., & Loftus, E. (1975). A Spreading-Activation Theory of Semantic Processing.
Psychological Review, 82(6), 407-428.
80
Cormier, P., Literman, B., & Lewis, K. (2011). Empirically Derived Heuristics to Assist
Designers with Satisfying Consumer Variation in Product Design. Proceedings of ASME
IDETC/CIE (DETC2011-48448), Washington, DC: ASME.
Cosmides, L. (1989). The logic of social exchange: Has natural selection shaped how humans
reason? Studies with the Wason selection task. Cognition, 31, 187-276.
Cotman, C., & Berchtold, N. (2002). Exercise: a behavioural intervention to enhance brain
plasticity. TRENDS in Neurosciences, 25(6), 295-301.
Cross, N. (2001). Design cognition: Results from protocol and other empirical studies of design
activity, In C. Eastman, W. Newstatter & M. McCracken, (Eds.), Design Knowing And
Learning: Cognition In Design Education, pp. 79-103, Oxford, UK: Elsevier.
Derrick, B., & Martinez, J. (1995). Associative LTD at the Hippocampal Mossy Fiber-CA3
Synapse. Social Neuroscience Abstracts, 21, 603.
Dieter, G. (2000). Engineering Design: A Materials and Processing Approach (3rd ed.), New
York, NY: McGraw-Hill.
Dietrich, A. (2004). The cognitive neuroscience of creativity. Psychonomic Bulletin & Review,
11(6), 1011-1026.
Dijksterhuis, A., & Meurs, T. (2006). Where creativity resides: The generative power of
unconscious thought. Consciousness and Cognition, 15, 135-146.
Dunbar, K. (1995). How scientists really reason: Scientific reasoning in real-world laboratories,
In R. Sternberg & J. David, (Eds.), Mechanisms of Insight, pp. 365-395, Cambridge, MA:
MIT Press.
Duncker, K. (1945). On Problem Solving, Psychological Monographs, 58(5), Whole No. 270.
Ericsson, K., & Simon, H. (1993). Protocol Analysis: Verbal Reports as Data, (Revised ed.),
Cambridge, MA: MIT Press.
81
Farmer, J., Zhao, X., Van-Praag, H., Wodtke, K., Gage, F., & Christie, B. (2004). Effects of
voluntary exercise on synaptic plasticity and gene expression in the dentate gyrus of adult
male Sprague-dawley rats in vivo. Neuroscience, 124, 71-79.
Fischhoff, B. (1975). Hindsight ≠ Foresight: The effect of outcome knowledge on judgment
under uncertainty. Journal of Experimental Psychology: Human Perception and
Performance, 3, 349-358.
Fiske, S., & Taylor, S. (1984). Social cognition (1st ed.), Reading, MA: Addison-Wesley.
Fodor, J., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis.
Cognition, 28, 3-71.
Friedrich, J. (1993). Primary Error Detection and Minimization (PEDMIN) Strategies in Social
Cognition: A Reinterpretation of Confirmation Bias Phenomena. Psychological Review,
100(2), 298-319.
Gabora, L. (2010). Revenge of the “Neurds”: Characterizing Creative Thought in Terms of the
Structure and Dynamics of Memory. Creativity Research Journal, 22(1), 1-13.
Gauthier, I., Tarr, M., Anderson, A., Skudlarski, P., & Gore, J. (1999). Activation of the middle
fusiform ‘face area’ increases with expertise in recognizing novel objects. Nature
Neuroscience, 2(6), 568-573.
German, T., & Barrett, H. (2005). Functional Fixedness in a Technology Sparse Culture.
Psychological Science, 16(1), 1-5.
Gero, J. (1990). Design Prototypes: A Knowledge Representation Schema for Design. AI
Magazine, 11(4), 26-36.
Gero, J., & Kannengiesser, U. (2006). The situated function-behaviour-structure framework.
Design Studies, 25(4), 374-391.
Gero, J., & McNeill, T. (2006). An Approach to the Analysis of Design Protocols. Design
Studies, 19(1), 21-61.
82
Gero, J. (2009, March 26). Developing Principles for a Theory of Design or What Can
Cognitive Science Contribute to a TOD. Meeting of the Design Society Advisory Board,
Cambridge, MA.
Gero, J. (2010). Generalizing design cognition research, In K. Dorst et al. (Eds.),
DTRS8:Interpreting Design Thinking, pp. 187-198, Sydney, AUS: DAB Documents.
Gifford-Jones, W. (1977). What Every Woman Should Know About A Hysterectomy. New York,
NY: Funk & Wagnalls.
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of
intuitive judgment. New York, NY: Cambridge University Press.
Guindon, R. (1990). Designing the Design Process: exploiting opportunistic thoughts. Human-
Computer Interaction, 5, 305-344.
Guindon, R., & Curtis, B. (1988). Control of Cognitive Processes During Software Design:
What Tools are Needed? Proceedings of CHI’88, Washington, DC.
Haier, R. (1993). Cerebral glucose metabolism and intelligence, In P. Vernon (Ed.), Biological
approaches to the study of human intelligence, pp. 415-426, Norwood, NJ: Ablex.
Hallihan, G., & Shu, L. (2011). Creativity and Long-Term Potentiation: Implications for Design.
Proceedings of ASME IDETC/CIE (DETC2011-48595), Washington, DC: ASME.
Hallihan, G., Cheong, H., & Shu, L. (2012). The Role of Cognitive Bias and Confirmation in
Design Cognition, Proceedings of ASME IDETC/CIE (DETC2012-71258), Chicago, IL:
ASME.
Hart, S., & Staveland, L. (1988). Development of NASA-TLX (Task Load Index): Results of
empirical and theoretical research, In P. Hancock & N. Meshkati (Eds.), Human Mental
Workload, pp. 139-183, Amsterdam, The Netherlands: Elsevier Science.
Hebb, D. (1949). The Organization of Behavior: A Neuropsychological Approach. New York,
NY: John Wiley & Sons.
83
Heilman, K., Nadeau, S., & Beversdorf, D. (2003). Creative Innovation: Possible Brain
Mechanisms. Neurocase, 9(5), 369-379.
Helms, M., & Goel, A. (2012). Analogical Problem Evolution in Biologically Inspired Design.
Design Computing and Cognition’12, College Station, TX: Springer.
Helms, M., Vattam, S., & Goel, A. (2009). Biologically inspired design: Process and products.
Design Studies, 30(5), 606-622.
Heuer, R. (1999). Psychology of Intelligence Analysis, Ch. 8, Washington, DC: Center for the
Study of Intelligence.
Jansson, D., & Smith, S. (1991). Design Fixation. Design Studies, 12(1), 3-11.
Jin, Y., Kim, D., & Danesh, M. (2006). Value Based Design: An Objective Structuring
Approach to Design Concept Generation. Proceedings of ASME IDETC/CIE (DETC2006-
99497), Philadelphia, PA: ASME.
Jung, R., Gasparovic, C., Chavez, R., Flores, R., Smith, S., Caprihan, A., & Yeo. R. (2009).
Biochemical Support for the “Threshold” Theory of Creativity: A Magnetic Resonance
Spectroscopy Study. The Journal of Neuroscience, 29(16), 5319-5325.
Kahneman, D., & Tversky, A. (1972). Subjective Probability: A Judgement of
Representativeness. Cognitive Psychology, 3, 430-454.
Kahneman, D., Tversky, A. (1984). Choices, Values, and Frames. American Psychologist,
39(4), 341-150.
Kan, J., Gero, J., & Tang, H. (2010). Measuring cognitive design activity changes during an
industry team brainstorming session. Design Computing and Cognition’10, Stuttgart,
Germany: Springer.
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for confidence. Journal of
Experimental Psychology: Human Learning and Memory, 6, 107-118.
84
Kramer, A., Erickson, K., & Colcombe, S. (2006). Exercise, cognition, and the aging brain.
Journal of Applied Physiology, 101, 1237-1242.
Kruger, J., Wirtz, D, Van Boven, L., & Altermatt, W. (2004). The Effort Heuristic. Journal of
Experimental Social Psychology, 40, 91-98.
Kurtoglu, T., Campbell, M., & Linsey, J. (2009). An experimental study on the effects of a
computational design tool on concept generation. Design Studies, 30, 676-703.
Levin, I., Schneider, S., & Gaeth, G. (1998). All Frames Are Not Created Equal: A Typology
and Critical Analysis of Framing Effects. Organizational Behaviour and Human Decision
Processes, 76(2), 149-188.
Linsey, J., Tseng, I., Fu, K., Cagan, J., Wood, K., & Schunn, C. (2010). A study of Design
Fixation, Its Mitigation and Perception in Engineering Design Faculty. Journal of
Mechanical Design, 132(4), 2041003-1-12.
Lippin, R. (2001). Expressive and Creative Arts Therapies, In M. Micozzi, (Ed.), Fundamentals
of Complimentary and Alternative Medicine, pp. 264, Philadelphia, PA: Churchill
Livingstone.
Liu, Y. (2000). Creativity or novelty?: Cognitive-computational versus social-cultural. Design
Studies, 21(3), 261-276.
MacDougall, R. (1905). On the Discrimination of Critical and Creative Attitudes. The Journal
of Philosophy, Psychology and Scientific Methods, 2(11), 287-293.
Malenka, R., & Nicoll, R. (1999). Long-Term Potentiation – A Decade of Progress? Science,
285(5435), 1870-1874.
Marsh, R., Ward, T., & Landau, J. (1999). The inadvertent use of knowledge in a generative
cognitive task. Memory and Cognition, 27(1), 94-105.
Martin, S., Grimwood, P., & Morris, R. (2000). Synaptic Plasticity and Memory: An Evaluation
of the Hypothesis. Annual Review of Neuroscience, 23, 649-711.
85
Martindale, C. (1995). Creativity and Connectionism, In S. Smith, T. Ward & R. Finke (Eds.),
The Creative Cognition Approach, Ch. 11, Cambridge, MA: MIT Press.
Martindale, C. (1999). Biological basis of creativity, In R. Sternberg (Ed.), Handbook of
creativity, Ch. 7, Cambridge, UK: Cambridge University Press.
Martinez, J., & Derrick, B. (1996). Long-Term Potentiation and Learning. Annual Review of
Psychology, 47, 173-203.
Mednick, S. (1962). The Associative Basis of the Creative Process. Psychological Review,
69(3), 220-232.
Mendelsohn, G. (1976). Associative and attentional processes in creative performance. Journal
of Personality, 44, 341-369.
Merriam, S. (2009). Qualitative Research: A Guide to Design and Implementation (3rd ed.), San
Francisco, CA: Josey-Bass.
Meyerowitz, B., & Chaiken, S. (1987). The effect of message framing on breast self-
examination attitudes, intentions, and behavior. Journal of Personality and Social
Psychology, 52(3), 500-510.
Miles, M., & Huberman, A. (1994). Qualitative Data Analysis: An expanded Sourcebook, (2nd
ed.), Thousand Oaks, CA: SAGE Publications Inc.
Mullen, B., & Johnson, C. (1990). Distinctiveness-based illusory correlations and stereotyping:
A meta-analytic integration. British Journal of Social Psychology, 29, 11-28.
National Research Council. (1991). Improving Engineering Design: Designing for Competitive
Advantage. Washington, DC: National Academy Press.
Nickerson, R. (1998). Confirmation Bias: A Ubiquitous Phenomena in Many Guises. Review of
General Psychology, 2(2), 175-220.
86
Nicholl, B., & McLellan, R. (2007). ‘Oh yeah, yeah you got lots of love hearts. The Year 9s are
notorious for love hearts. Everything is love hearts.’ Fixation in Pupils’ Design and
Technology Work (11-16 Years). Design and Technology Education, 12, 34-44.
Osborn, A. (1963). Applied Imagination. New York, NY: Scribner’s.
Perky, C. (1910). An Experimental Study on Imagination. The American Journal of Psychology,
21(3), 422-452.
Perttula, M., & Liikkanen, L. (2006). Structural Tendencies and Exposure Effects in Design
Idea Generation. Proceedings of ASME IDETC/CIE (DETC2006-99123). Philadelphia, PA:
ASME.
Pfenninger, K., & Shubik, V. (2001). Insights into the foundation of creativity: A Synthesis, In
K. Pfenninger & V. Shubik (Eds.), The origins of creativity, pp. 217, Oxford, MA: Oxford
University Press.
Plous, S. (1993). The Psychology of Judgement and Decision Making, New York, NY:
McGraw-Hill.
Purcell, A., & Gero, J. (1996). Design and other types of fixation. Design Studies, 17(4), 363-
383.
Qin, S., van Marle, H., Hermans, E., & Fernandez, G. (2011). Subjective Sense of Memory
Strength and the Objective Amount of Information Accurately Remembered Are related to
Distinct Neural Correlates at Encoding. The Journal of Neuroscience, 31(24), 8920-8927.
Quillian, M. (1962). A revised design for an understanding machine. Mechanical Translation, 7,
17-29.
Reinig, B., Briggs, R., & Nunamaker, J. (2007). On the Measurement of Ideation Quality.
Journal of Management Information Systems, 23(4), 143-161.
Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and Sequential
Though Processes in PDP Models, In J. McClelland & D. Rumelhart (Eds.), Parallel
87
Distributed Processing: Explorations in the Microstructure of Cognition, Ch. 14, Cambridge,
MA: MIT Press.
Shrout, P., & Fleiss, J. (1979). Intraclass Correlations: Uses in Assessing Rater Reliability,
Psychological Bulletin, 86(2), 420-428.
Simon, H. (1966). Scientific Discovery and the Psychology of Problem Solving, In R. Colodny
(Ed.), Mind and Cosmos: Essays in contemporary science and philosophy, pp. 22-40,
Pittsburgh, PA: University of Pittsburgh Press.
Smith, S., & Blankenship, S. (1989). Incubation Effects. Bulletin of the Psychonomic Society,
27(4), 311-314.
Stein, M. (1975). Stimulating Creativity, Vols. 1-2, New York, NY: Academic Press.
Sternberg, R., & Lubart, T. (1999). The concept of creativity: Prospects and paradigms, In R.
Sternberg (Ed.), Handbook of Creativity, pp. 3-15, Cambridge, UK: CUP.
Tang, H., & Lee, Y. (2008). Using Design Paradigms to Evaluate the Collaborative Design
Process of Traditional and Digital Media. Design Computing and Cognition’08, Atlanta, GA:
Springer.
Teyler, T., & DiScenna, P. (1987) Long-Term Potentiation. Annual Review of Neuroscience, 10,
131-161.
Thaler, R. (1980). Toward a positive theory of consumer choice. Journal of Economic
Behaviour and Organization, 1, 39-60.
Thompson, R., Berger, T., & Madden, J. (1983). Cellular processes of learning and memory in
the mammalian CNS. Annual Review of Neuroscience, 6, 447-91.
Tversky, A., & Kahneman, D. (1973). Availability: A Heuristic for Judging Frequency and
Probability. Cognitive Psychology, 5, 207-232.
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice.
Science, 211, 453–458.
88
Tversky, A., & Kahneman, D. (1982). Judgement Under Uncertainty: Heuristics and Biases,
Cambridge, UK: Cambridge University Press.
Van-Praag, H., Christie, B., Sejnowski, T., & Gage, F. (1999). Running enhances neurogenesis,
learning, and long-term potentiation in mice. PNAS, 96(23), 13427-13431.
Vattam, S., Helms, M., & Goel, A. (2008). Compound analogical design: Interaction between
problem decomposition and analogical transfer in biologically inspired design. Design
Computing and Cognition’08, 377-396. Atlanta, GA: Springer.
Viswanathan, V., & Linsey, J. (2011). Design Fixation in Physical Modelling: An Investigation
on the Role of Sunk Cost. Proceedings of ASME IDETC/CIE (DETC2011-47862),
Washington, DC: ASME.
Wagner, U., Gais, S., Haider, H, Verleger, R., & Born, J. (2004). Sleep inspires insight. Nature,
427, 352-254.
Wallas, G. (1926) The Art of Thought, New York, NY: Harcourt, Brace and Company.
Ward, T. (1994). Structured Imagination: The Role of Category Structure in Exemplar
Generation. Cognitive Psychology, 27, 1-40.
Ward, T. (1995). What’s Old About New Ideas? In S. Smith, T. Ward & R. Finke (Eds.), The
Creative Cognition Approach, Ch. 7, Cambridge, MA: MIT Press.
Wason, P. (1968). Reasoning about a rule. Quarterly Journal of Experimental Psychology,
20(3), 273-281.
Wickens, C., Lee, J., Liu, & Gordon Becker, S. (2004). Design and Evaluation Methods. Human
Factors Engineering, Chapter 3, Upper Saddle River, NJ: Pearson Prentice Hall.
Young, J. (2007, March 6). Neuron to Glial Synapse on Axons? Retrieved from
http://scienceblogs.com/purepedantry/2007/03/neuron_to_glia_synapse_on_axon.phyhp.
89
Appendix A: Demographic Questionnaire
Instructions: The purpose of this questionnaire is to assess your general background and gather some information relevant to your ability to perform tasks related to the study. Your personal identity will not be associated with any of your responses and this information will be kept confidential.
1. Gender: Male Female
2. Age: _____________________
3. Specified major (area of study): _________________________________
4. Current level of study: Graduate (or) Undergraduate
5. Year of program: ______
6. Do you have previous design experience? Yes No
a. How many years of design experience do you have? _____
b. Please list any relevant design courses or projects you have been involved in:
i. _____________________________________
ii. _____________________________________
iii. _____________________________________
iv. _____________________________________
v. _____________________________________
7. Is English your first language? Yes No
a. Please rate your English reading fluency on a scale of 1 - 5 (circle one).
1 2 3 4 5
Very Poor Poor Average Good Excellent
Thank you for filling out this questionnaire
90
Appendix B.1: Design Problem Before Defixation Task
Design a system that automatically administers water to a house plant. The system must provide a potted plant with a predetermined amount of water every week. You must consider how the flow of water will be controlled, how the water will be administered to the plant, what the water source will be, and what will power your system. Example Solution
Design Solutions
Atimervalvehasbeenattachedtoahousewatermain.Atpredeterminedintervals,thevalveopensandallowsthedesiredamountofwatertoflowthroughashowerheadontotheplant.
91
Appendix B.2: Design Problem After Defixation Task
Design Problem You will now have 10 additional minutes to continue working on the previously presented design problem. Try and generate as many distinct and feasible ideas as possible. If you have any questions, please ask the researcher and he will be happy to assist you. Please sketch your design solutions on the bottom and back of this page, additional pages have been provided if required. You are free to include descriptions of your design solution with the sketches. Design Solutions
92
Appendix C.1: Defixation Task Instructions TREATMENT GROUP INSTRUCTIONS Design problems like the previous one are typically more analytical in nature and involve activation of the right hemisphere of the brain. To stimulate the left hemisphere of the brain we will be asking you to engage in a language based learning task, as this is typically associated with activation of the left hemisphere of the brain. Physical activity has also been shown to enhance blood flow in the brain, so exercise will be incorporated the task.
On the following page is a story designed to help you learn beginner’s Swedish. Please read through the story and attempt to learn the English equivalent of the Swedish words and phrases used in the story. During the activity you may not write anything down, and may not externally vocalize the words or phrases. Instead simply read and internally verbalize the material to assist your learning. You will be given 10 minutes to complete the language learning portion of the experiment.
In addition, you are being asked to perform a mild aerobic step exercise for the duration
of the language task. Please step up and down from the step block provided while completing the language activity. Keep the pace of stepping brisk and constant as if you were walking up a flight of stairs. If you become fatigued and cannot continue the exercise, you may stop.
Please devote your full attention to the language task as you will be tested on the
material covered in the language exercise. The test will consist of translating the common Swedish words and phrases presented in the story into English. You will also be tested on the content of the story. _____________________________________________________________________________ CONTROL GROUP INSTRUCTIONS Design problems like the previous one are typically more analytical in nature and involve activation of the right hemisphere of the brain. To stimulate the left hemisphere of the brain we will be asking you to engage in a language based task, as this is typically associated with activation of the left hemisphere of the brain.
On the following page is a story designed to help you learn beginner’s Swedish. Please read through the story and attempt to learn the English equivalent of the Swedish words and phrases used in the story. During the activity you may not write anything down, and may not externally vocalize the words or phrases. Instead simply read and internally verbalize the material to assist your learning. You will be given 10 minutes to complete the language learning portion of the experiment.
Please devote your full attention to the language task as you will be tested on the
material covered in the language exercise. The test will consist of translating the common Swedish words and phrases presented in the story into English. You will also be tested on the content of the story.
93
Appendix C.2: Defixation Task Story Material Mike is visiting his friend Greger in Sweden. He is from Toronto, but has learned some basic Swedish over the past couple of days, so he decides to go out for lunch and see how well he can get by on his own. A Simple Greeting Mike is taking a bus to get to a restaurant and decides to practice his Swedish with a stranger next to him at the bus stop. Mike: God morgon! (Good morning!) Jag heter Mike, (My name is Mike,) Vag heter du? (what is your name?) Stranger: Roligt att träffas Mike, (Nice to meet you Mike,) Jag heter Henrik. (my name is Henrik.) Mike: Rolligt att träffas Henrik, (Nice to meet you Henrik,) var kommer du ifrån? (where are you from?) Stranger: Jag är från Stockholm, (I am from Stockholm) Var kommer du ifrån? (Where are you from?) Mike: Jag är från Toronto (I am from Toronto) och jag är ingenjör, (and I am an engineer,) Var arbetar du med? (What is your profession?) Stranger: Jar är lärare. (I am a teacher.) Mike: Talar du engleska? (Do you speak English?) Stranger: Ja, lite. (Yes, a little) Mike’s bus comes Mike: Adjö Henrik. (Goodbye Henrik) At the Restaurant Mike arrives at the restaurant and takes a seat at a table. The waiter approaches him. Waiter: God middag, (Good afternoon,) jag heter Sven. (my name is Sven.) Vad kan jag få för dig? (What can I get for you?) Mike: Kan jag få en koppe kaffe? (May I have a cup of coffee) Waiter: Ja, vill du något att äta? (Yes, would you like anything to eat?) Mike: Jag skulle vilja ha (I would like to have) en kyckling smörgås (a chicken sandwich) och pomme frites, vänligen. (and French fries, please).
94
Waiter: Din mat kommer att vara klar (Your food will be ready) i femton minuter. (in 15 minutes) 20 minutes later Waiter: Hära är din mat, (Here is your food,) ledsen för vänta. (sorry for the wait.) Mike: Det är bra, tack. (That’s fine, thank you.) Mike finishes his lunch. Waiter: Hur van din mat? (How was your food?) Mike: Bra tack. Kan jag få notan? (Good, thank you. May I have the bill?) Waiter: Naturligtvis, (Of course,) här är. (here you are) Getting Lost Mike decides to walk back to his friend’s house from the restaurant, but can’t remember which way to go. After struggling with street signs and a map for a few hours he decides to ask a friendly stranger. Mike: God middag, talar du engelska? (Good afternoon, do you speak English?) Stranger: Nej, tyvärr. (No, sorry.) Kan jag hjälpa dig? (Can I help you?) Mike: Jag är vilse. (I am lost.) Jag letar efter femton Råstavägen gata. (I am looking for fifteen Råstavägen street) Stranger: Det är sex gator norr, (It is six streets north,) jag kan gå med dig. (I can walk with you.) Mike: Tack så mycket! (Thank you very much!) Mike Gets Home Greger: Hur van din dag? (How was your day?) Mike: Det var bra. (It was fine). Jag tog bussen, åt lunch, och gick hem. (I took the bus, had lunch, and walked home.) Greger: Hade du några problem? (Did you have any troubles?) Mike: Naturligtvis inte! (Of course not!)
95
Appendix D: Recall Test for Defixation Material This portion of the experiment is meant to test your recall for the information presented in the language task. Section 1 In this first section, you will be asked to recall the English translation of some Swedish words or phrases presented in the story. Next to each Swedish word or phrase, please write the English translation as accurately as possible. 1) Jag heter Sven. ______________________________________________________________ 2) Jag är ingenjör. ______________________________________________________________ 3) Lärare _____________________________________________________________________ 4) Talar du engleska? ___________________________________________________________ 5) Adjö ______________________________________________________________________ 6) God middag ________________________________________________________________ 7) Kan jag få en koppe kaffe? _____________________________________________________ 8) Kyckling smörgås ____________________________________________________________ 9) Femton ____________________________________________________________________ 10) Hur van din mat? ___________________________________________________________ 11) Kan jag få notan? ___________________________________________________________ 12) Jag är vilse. ________________________________________________________________ 13) Kan jag hjälpa dig? __________________________________________________________ 14) Tack så mycket! ____________________________________________________________ 15) Hur van din dag? __________________________________________________________
96
Section 2 Please answer the following questions (in English) about the content of the story you read previously. 1) What was the name of the stranger Mike was talking to at the bus stop? ________________
2) What was the profession of the stranger at the bus stop? _____________________________
3) Where was the stranger at the bus stop from? _____________________________________
4) What did Mike order for lunch at the restaurant to eat? ______________________________
5) What did Mike order at the restaurant to drink? ____________________________________
6) What was the house number of Mike’s friend’s house? ______________________________
7) In what direction, and how many blocks did the stranger walk Mike to take him back to his
friend’s house? ______________________________________________________________
97
Appendix E: Task Difficulty Questionnaire The purpose of this questionnaire is to assess the subjective level of difficulty you experienced with the tasks in the study. For each question, circle the number (1-7) that corresponds most accurately with your perception of the tasks difficulty. 1) I felt that the level of physical exertion required of me in this study was:
1 ------------- 2 ------------- 3 ------------- 4 ------------- 5 ------------- 6 ------------- 7
Extremely Low Moderate Extremely High
2) I felt that learning and recalling the Swedish story and phrases was:
1 ------------- 2 ------------- 3 ------------- 4 ------------- 5 ------------- 6 ------------- 7 Very Easy Moderate Very Difficult 3) I felt that generating design solutions the first time I worked on the design problem was:
1 ------------- 2 ------------- 3 ------------- 4 ------------- 5 ------------- 6 ------------- 7 Very Easy Moderate Very Difficult
4) I felt that generating design solutions the first time I worked on the design problem was:
1 ------------- 2 ------------- 3 ------------- 4 ------------- 5 ------------- 6 ------------- 7 Very Easy Moderate Very Difficult
98
Appendix F: Fixation Coding Instructions
You have been given a set of design solutions that participants generated in response to a problem requiring the design of a system that automatically waters a houseplant (see attached). Your task is to score each design solution to determine an objective fixation score. The scoring is based on four categories: 1) Water Source, 2) Water regulation, 3) Water Transfer, and 4) Energy Source. On the scoring sheets attached, assign a 1 to the category if it matches the example solution (fixation) or a 0 to the category if it is not the same as the example solution. In cases where an element is not completely distinct from the example solution, or only slightly different you may assign half points (0.5). (e.g. 1 = fixated, 0.5 = somewhat fixated, 0 = no fixation). In the overall column please evaluate the similarity of the solution to the fixation example from 1-7 (1 = very similar, 4 = neither similar or dissimilar, 7 = not at all similar) using a holistic evaluation approach taking into consideration similarity in function, concept, and design. Design solutions have been randomly ordered and numbered. Please ensure the number of the design solution you are scoring matches the design solution number on the scoring sheet. The design solution numbers do not convey any temporal ordering. If you have any comments relevant to your scoring describe them in the space provided or on the back of the sheet, but make it clear what solution the comment corresponds to. Solutions have been grouped by participant, however the ordering (before or after the defixation activity) has been randomized. Therefore you will be viewing one participant’s set of designs before moving onto the next participant’s. If you see a participant repeat an idea make a note of it in comments (i.e. “repeat”). Example Solution for Comparison: In this solution, a timer valve has been attached to a house water line. Once a day the timer opens and the valve and allows exactly 1/70 of a litre of water to be released. The water is administered to the house plant through a spray nozzle. The Four Categories
1) Water Source – Municipal Water Supply (Household) 2) Water regulation – Timer valve 3) Water Transfer – Pressurized release through house pipes and a sprinkler head 4) Energy Source – Municipal Electricity
99
Appendix G: Ratersʼ Raw Scores for Fixation Coding
RATER 1 Concept Participant Source Regulation Transfer Energy Total Overall
1 1 0.5 1 0.5 1 3 2
2 1 0.5 0 0 0.5 1 6
3 1 0 0 0 0 0 7
4 1 0 0 0 0.5 0.5 7
5 1 0.5 0.5 1 0 2 3
6 2 1 1 0.5 1 3.5 1
7 2 1 1 0.5 1 3.5 1
8 3 1 1 1 1 4 1
9 3 0.5 0.5 0.5 0.5 2 3
10 3 1 0 1 0.5 2.5 3
11 3 1 0.5 1 1 3.5 1
12 3 1 0.5 1 1 3.5 1
13 4 0 0 0 0 0 7
14 4 0 0 0 0 0 7
15 4 0.5 0 0 0.5 1 6
16 4 0.5 0 0.5 0.5 1.5 4
17 4 0 0 0 0.5 0.5 7
18 4 0 0 0 0 0 7
19 5 0.5 0 0.5 0.5 1.5 4
20 5 1 1 1 1 4 1
21 5 0.5 1 0.5 1 3 2
22 5 1 1 1 1 4 1
23 5 0.5 1 0.5 1 3 2
24 5 0 0 0 0.5 0.5 7
25 5 1 1 0.5 1 3.5 1
26 6 0 0 0 0 0 0 27 6 0 0 0 0 0 0
28 7 1 0.5 1 1 3.5 1
29 7 0 0.5 0 0.5 1 6
30 7 1 0 0.5 1 2.5 4
31 7 1 0.5 1 1 3.5 1
32 7 0 0.5 0 0.5 1 6
33 7 1 0 0.5 1 2.5 4
34 7 0.5 0 0.5 0.5 1.5 4
35 8 0.5 0 0 0 0.5 6
36 8 1 0.5 1 1 3.5 1
37 8 1 0 0 0.5 1.5 4
38 8 1 0 0 0.5 1.5 4
39 8 0.5 1 0.5 1 3 3
40 9 1 0 0.5 1 2.5 3
41 9 0 0 0 0 0 7
42 9 0 0 0 0 0 7
43 9 0.5 0 0 0.5 1 5
44 9 1 0 0 0.5 1.5 4
100
45 9 0.5 1 0 1 2.5 3
46 9 0.5 0 0 0 0.5 7
47 9 0 0 0 0 0 7
48 10 1 1 0.5 0 2.5 2
49 10 1 1 0.5 1 3.5 1
50 11 1 1 0.5 1 3.5 2
51 11 0.5 1 1 1 3.5 1
52 11 1 1 1 1 4 1
53 11 1 1 1 0.5 3.5 1
54 11 0.5 0 0.5 0 1 7
55 12 1 1 0.5 1 3.5 2
56 12 0 0 0 0 0 7
57 12 0.5 1 0.5 1 3 2
58 12 1 0 0.5 0.5 2 3
59 12 0.5 0.5 0.5 1 2.5 2
60 12 0.5 0 0.5 0 1 7
61 12 0 0 0 0 0 7
62 13 0.5 0 0.5 0 1 7
63 13 1 0 0.5 1 2.5 5
64 13 0 0 0 0 0 0 65 13 1 0 0.5 1 2.5 5
66 13 0 0.5 0 0 0.5 7
67 13 0 0 0 0.5 0.5 7
68 13 0 0 0 0 0 7
69 14 0.5 0 1 0.5 2 3
70 14 0.5 0 0 0.5 1 6
71 14 1 0 0 0.5 1.5 6
72 14 1 1 1 1 4 1
73 14 1 0.5 1 1 3.5 1
74 14 0.5 0.5 1 1 3 2
75 14 1 0.5 1 1 3.5 1
76 15 1 1 0.5 1 3.5 1
77 15 1 0 0.5 1 2.5 5
78 15 1 0 0.5 1 2.5 5
79 15 0 0 0 0 0 7
80 15 1 0.5 0 1 2.5 5
81 16 1 1 1 1 4 1
82 16 1 1 0.5 1 3.5 2
83 16 0.5 1 1 1 3.5 2
84 16 1 1 0.5 1 3.5 1
85 17 0.5 0 0 0 0.5 7
86 17 0 0.5 0 0 0.5 7
87 17 1 0 0.5 1 2.5 4
88 17 1 0.5 0.5 0.5 2.5 3
89 17 0 0 0 0 0 7
90 18 0.5 0.5 0.5 1 2.5 3
91 18 0.5 0.5 1 1 3 3
92 18 0.5 0 0 0 0.5 7
93 18 0 0 0 0 0 7
101
94 18 1 0.5 1 1 3.5 1
95 18 1 1 1 1 4 1 96 19 1 0.5 1 1 3.5 1
97 19 1 0.5 1 1 3.5 1
98 19 1 1 1 1 4 1
99 20 1 0.5 0 1 2.5 3
100 20 0 0 0 0 0 7
101 20 0 0 0.5 0 0.5 6
102 20 0 0 0 0 0 7 103 20 0 0 0 0 0 7
104 20 0.5 0 0 0.5 1 6
105 20 0 0 0 0 0 7
106 20 0 0 0 0 0 7
107 21 1 1 0 1 3 2
108 21 0 0.5 0 1 1.5 5
109 21 0.5 1 0.5 1 3 2
110 21 0 1 1 1 3 3
111 21 0 0 0 0 0 0
112 21 1 0 0 0.5 1.5 7
113 22 0.5 0 0 0 0.5 6
114 22 0 0.5 0 0 0.5 7
115 23 0.5 0 0.5 0 1 5
116 23 0 0 0 0 0 7
117 24 1 0.5 1 1 3.5 1
118 24 0.5 0 0 0.5 1 7
119 24 0.5 0.5 0.5 0.5 2 3
120 24 0 0 0 0 0 7
121 25 0.5 1 0.5 0 2 3
122 25 0.5 1 0 0 1.5 4
123 25 0.5 0.5 0.5 0.5 2 3
102
RATER 2 Concept Participant Source Regulation Transfer Energy Total Overall
1 1 1 1 0 1 3 2
2 1 1 0.5 0 1 2.5 3
3 1 1 1 0 1 3 3
4 1 0 1 0.5 1 2.5 4
5 1 1 1 1 0 3 2
6 2 1 1 0.5 1 3.5 1
7 2 1 1 0.5 1 3.5 1
8 3 1 1 1 1 4 1
9 3 1 0 0.5 0 1.5 5
10 3 1 0 1 0.5 2.5 5
11 3 1 0.5 1 0.5 3 3
12 3 1 0.5 1 1 3.5 1
13 4 0 0 0 0.5 0.5 7
14 4 0 0 0 0.5 0.5 6
15 4 1 0 0.5 0.5 2 5
16 4 1 0 0 1 2 3
17 4 1 0 0 1 2 4
18 4 0 0 0.5 0 0.5 6
19 5 1 0 0.5 0.5 2 5
20 5 1 1 1 1 4 1
21 5 0.5 1 1 1 3.5 2
22 5 1 1 1 1 4 1
23 5 1 1 0.5 1 3.5 1
24 5 0 0 0 1 1 6
25 5 1 1 0.5 1 3.5 2
26 6 0 0 0 0 0 0 27 6 0 0 0 0 0 0 28 7 1 1 1 1 4 1
29 7 0 1 1 1 3 2
30 7 1 0 1 1 3 2
31 7 1 1 1 1 4 1
32 7 0 1 0 1 2 2
33 7 1 0 0.5 0.5 2 2
34 7 1 0 1 0.5 2.5 2
35 8 0.5 0 0.5 0 1 5
36 8 1 0.5 1 1 3.5 6
37 8 1 0 0 1 2 5
38 8 1 0 0.5 0 1.5 6
39 8 0 0 0.5 1 1.5 3
40 9 1 0 0.5 1 2.5 3
41 9 0 0 0 0 0 7
42 9 0 0 0.5 0 0.5 6
43 9 1 1 0.5 1 3.5 2
44 9 1 0 0.5 0.5 2 3
45 9 0 1 1 0 2 2
103
46 9 0 0 0 0 0 7
47 9 0 0 0 0 0 7
48 10 1 1 1 1 4 1
49 10 1 0.5 0.5 1 3 2
50 11 1 1 1 1 4 1
51 11 0.5 1 1 1 3.5 2
52 11 0.5 1 1 1 3.5 1
53 11 1 1 1 1 4 1
54 11 0.5 0 0 0.5 1 7
55 12 1 1 0.5 1 3.5 2
56 12 1 0 0 0 1 6
57 12 0.5 1 0.5 1 3 3
58 12 1 0 0 0 1 6
59 12 0.5 1 1 1 3.5 1
60 12 0.5 0 0 0.5 1 4
61 12 0 0 0 0 0 7
62 13 0 0 0 0 0 7
63 13 1 0 0 0 1 3
64 13 0 0 0 0 0 0 65 13 0.5 0 0 1 1.5 5
66 13 0 0 0 0 0 7
67 13 0 0 0 0 0 7
68 13 0 0 0 0 0 7
69 14 0 0 1 0 1 6
70 14 0 0 0 0 0 7
71 14 1 0 0 0 1 6
72 14 1 0 1 1 3 2
73 14 1 0 1 1 3 2
74 14 1 0 1 1 3 2
75 14 1 0 1 1 3 2
76 15 1 1 1 1 4 1
77 15 1 0 0 0 1 5
78 15 1 0 0 0 1 5
79 15 0 0 0 0 0 7
80 15 1 0.5 0 1 2.5 3
81 16 1 1 1 1 4 1
82 16 1 1 0 1 3 2
83 16 0.5 1 1 1 3.5 1
84 16 1 1 0.5 1 3.5 2
85 17 1 1 0 1 3 2
86 17 0 0.5 0 1 1.5 5
87 17 1 0 1 1 3 2
88 17 1 0 1 1 3 3
89 17 0 0 0 1 1 5
90 18 0 1 1 1 3 3
91 18 1 0 0 1 2 3
92 18 1 0 0.5 0 1.5 3
93 18 0.5 0 0.5 0 1 6
94 18 1 0.5 1 1 3.5 2
104
95 18 0 0 0 0 0 0
96 19 1 0.5 1 1 3.5 2
97 19 1 0.5 1 1 3.5 2
98 19 1 0.5 1 1 3.5 1
99 20 1 1 0 1 3 3
100 20 1 0 0.5 1 2.5 3
101 20 1 0 0.5 1 2.5 3
102 20 0 0 0 0 0 0 103 20 0 0.5 0 0 0.5 6
104 20 1 0 0 1 2 3
105 20 0 0 0.5 1 1.5 3
106 20 1 0 0 1 2 5
107 21 1 1 0 1 3 2
108 21 0 1 0.5 1 2.5 4
109 21 0 0 1 1 2 3
110 21 0 1 1 1 3 2
111 21 0 1 1 1 3 2
112 21 1 0 0 0 1 5
113 22 0.5 0.5 0.5 1 2.5 3
114 22 0 0.5 0 1 1.5 5
115 23 1 0 0.5 1 2.5 5
116 23 0 0 0 0 0 7
117 24 1 1 1 1 4 1
118 24 1 1 0.5 1 3.5 3
119 24 1 1 0.5 1 3.5 3
120 24 0.5 1 0.5 1 3 3
121 25 1 0.5 0.5 1 3 3
122 25 1 1 0.5 1 3.5 2
123 25 1 1 1 1 4 1
105
Appendix H.1: Participant Concepts Ranked Low in Fixation Before Defixation Subject 20 – Concept Numbers: 99, 100, 101 (Average Fixation Score 0.46/1.00)
106
Appendix H.1 – Participant Concepts Ranked Low in Fixation After Defixation Subject 20 – Concept Numbers: 102, 103, 104, 105 (Average Fixation Score 0.22/1.00)
107
Appendix H.2: Participant Concepts Ranked High in Fixation Before Defixation Subject 16 – Concept Numbers: 81, 82 (Average Fixation Score 0.91/1.00)
108
Appendix H.2: Participant Concepts Ranked High in Fixation After Defixation Subject 16 – Concept Numbers: 83, 84 (Average Fixation Score 0.88/1.00)
109
Appendix I: Design Problems and Biological Analogies 1. Promotional Mailing Problem
You are a marketing director for a credit card company. You are looking for an effective
strategy to distribute sign-up promotional mailings within a city. You would like to distribute
promotional mail to selected neighborhoods in the city so that a large proportion of the
promotional mail actually result in people signing up. In other words, you don’t want to waste
resources on sending promotional mail to neighborhoods where people are not likely to sign up.
Assuming that you don’t have any demographic information of the city, how would you
optimize the use of promotional mailings?
Biological Phenomenon (Ant)
An ant colony can identify the shortest path between its nest and food source with the following
strategy. Ants depart the colony to search randomly for food, laying down pheromones on the
trail as they go. When an ant finds food, it follows its pheromone trail back to the nest, laying
down another pheromone trail on the way. Pheromones have more time to dissipate on longer
paths, and less time to dissipate on shorter paths. Shorter paths are also travelled more often
relative to longer paths, so pheromones are laid down more frequently on shorter paths.
Additional ants follow the strongest pheromone trails between the food source and the nest,
further reinforcing the pheromone strength of the shortest path.
2. Authorized Disassembly Problem
Original equipment manufacturers (OEM’s) want easy disassembly of their products to reduce
disassembly cost and increase the net profit from reuse and recycling at product end of life.
However, OEM’s are also concerned with protecting high-value components from theft and
access by competitors. How can you allow disassembly that is easy but only by those
authorized?
110
Biological Phenomenon (Enzymes)
Enzymes are complex proteins that bind to specific substrates (molecules) and form enzyme-
substrate complexes that perform biochemical activities. The specific binding is achieved when
the active site of an enzyme geometrically matches its corresponding substrate. However, an
enzyme changes its shape with environmental factors such as pH and temperature. This shape
change alters the conformation of the enzyme’s active site to the point where substrates can no
longer fit, thereby disabling the function of the enzyme-substrate complex.
3. Wet Scrubber Problem
Wet scrubbers are air pollution control devices that remove pollutants from industrial exhaust
systems. In conventional wet scrubbers, exhaust gas is brought into contact with a liquid
solution that removes pollutants from the gas by dissolving or absorbing them into the liquid.
The removal efficiency of pollutants is often improved by increasing the contact time or the
contact area between the exhaust gas and the scrubber liquid solution. What other strategy could
be used to increase the removal efficiency of wet scrubbers?
Biological Phenomenon (Penguins)
Penguins are warm blooded yet keep their un-insulated feet at a temperature close to freezing to
minimize heat transfer to the environment. The veins that carry cold blood from the feet back to
the body are located closely to the arteries that carry warm blood from the body to the feet. The
warm blood flows in the opposite direction as the cold blood, which allows the penguins to
transfer the most heat to the cold blood. This reduces both the amount the returning blood can
drop the core body temperature, and the amount of heat lost through the feet.
111
Appendix J – Coded Verbal Protocol for Group 2
112
113
114
115
116
117
118
Appendix K: Confirmation and Disconfirmation Coding Scheme
You have been given a set of notes that participants generated while evaluating a design belief. Your task is to review the notes in their entirety and determine: 1) How many pieces of evidence/arguments are evaluated (Cases) with respect to the belief, 2) How many of those cases are evaluated in order to confirm the belief, and 3) How many of those cases are evaluated in order to disconfirm the belief. Please see Problem 2 for a review of the problem as it was presented to participants, the design belief that they were evaluating, and the concepts they evaluated. On the scoring sheets attached, record the participant number, the total number of cases evaluated, the number of confirmatory cases evaluated, the number of disconfirmatory cases evaluated, and the number of cases that were ambiguous or could not be coded. To determine what constitutes a “case” from the notes, determine what sentences, points or arguments, constitute clauses that are independent of each other. If a clause can stand-alone and is used to support or refute the belief in question, it should be considered as an individual case. If the same clause is applied multiple times (X), than it can be considered as X number of cases. For information in the form of a decision matrix, each piece of evidence evaluated should have been included in the left-most column. If that evidence is applied multiple times (X), it too can be considered as X number of cases. The participant numbers have been randomly assigned and do not convey any temporal or categorical ordering. If you have any comments relevant to your rating describe them in the space provided or on the back of the sheet, but make it clear what participant and case the comment corresponds to.
119
Coding Sheet
Participant No.
Total No. of Cases
Confirmatory Cases Disconfirmatory Cases
Ambiguous Cases
Comments
120
Appendix L: Confirmation Bias Problem 1
Problem 1: The Washing Machines A common belief among consumers is that washing machines that are more water efficient (use less water per load) are also more energy efficient (use less energy per load). You would like to know whether this belief is true or false. A local appliance store has said it can send you manufacturer specifications for some of their machines. The store has four different models, but they can only send you the information on two of the four. However, the store was able to tell you a little about each model with respect to relatively how water or energy efficient it is to help you make your decision. Your task is to select two of the four machines that you believe will be the most useful in evaluating the validity of the following belief:
“Washing machines that are highly water efficient are also highly energy efficient.”
Please record your choice, and any relevant considerations that informed your choice, on the blank sheet provided.
121
B
D
A
C
122
Appendix M: Confirmation Bias Problem 2
Problem 2: Fixation in Design Fixation There has been a significant amount of research demonstrating that designers often become fixated by examples of successful design solutions. The research indicates that when designers see an example solution for a design problem they are working on, they often incorporate elements of that example into their own design solutions. This effect has been observed even when designers are instructed not to fixate on examples, and even among experienced designers. An experiment was run to test the hypothesis that designers fixate on examples. The design problem and example solution given to participants is seen at the bottom of the page. Six participants generated solutions for the problem; their concepts can be seen on the next page. Your job is to look at the results of the experiment (the participant concepts) to evaluate the validity of the fixation hypothesis, stated below:
“The presence of an example solution causes designers to fixate and incorporate elements of the example into their own solutions.”
Design Problem Design a system that automatically administers water to a house plant. The system must provide a potted plant with a predetermined amount of water for a predetermined amount of time. Example Solution
A timer valve has been attached to a house water main. At predetermined intervals, the valve opens and allows the desired amount of water to flow through a sprinkler head onto the plant.
123
Concepts to Evaluate
A water wheel sits in a tank filled by a house water line. The rotational speed of the wheel is programmed so that it pours the desired amount of water into the plant.
A drip container is inserted into the soil. The container is filled with enough water, and drips at a rate, so the plant receives the desired amount of water.
The plant is placed on a platform that lowers into and out of a tank of water. The frequency and duration is pre-set to provide adequate water.
Sponges are hydrated and placed on the soil around the plant. The number of sponges is adjusted to provide enough hydration.
Rainwater is collected in a tank. A timer valve is set to release water from the tank onto the plant at predetermined intervals based on the desired water volume.
A water tank is filled from a water line. A ball-float is attached to a valve at the bottom of the tank. The float and valve are set so that the desired amount of water is released from the tank onto the plant.
124
Appendix N.1 – Treatment Group Instructions for Concept Evaluation
Instructions Your task is to evaluate the belief stated in Problem 2. You have been provided with 6-design concepts, which are relevant pieces of evidence you may use to help you evaluate that belief. You have also been provided with procedural instructions to help formalize the process of decision-making. Please use the procedure to help you evaluate the belief provided in Problem 2. Read through the procedure completely before beginning. Procedure
1) Identify all the possible conclusions you could draw, e.g. what are the options you can decide
between with respect to the belief provided. a. Example: If you are evaluating the belief smoking causes cancer, you could decide that the
belief is true, or false, conditionally true, etc. 2) Consider all the evidence available that is relevant in deciding to reject or accept each possible
conclusion. In this case that evidence will come from your evaluation of the 6 design concepts. a. Example: In evaluating the belief from step 1, one piece of evidence could be “results of
studies examining a correlation between smoking and lung cancer”. A different piece of evidence could be “anecdotal cases of individuals who have smoked their whole lives and not been diagnosed with cancer”.
3) Prepare a table (see template below). Place each conclusion (from step 1) in its own cell across the top row. Put each piece of evidence (from step 2) in its own cell in the left column.
4) Work through the table and evaluate each conclusion relative to each piece of evidence. a. Example: Given the evidence “results of studies examining a correlation between smoking
and lung cancer” you would evaluate each of your conclusions regarding the smoking-cancer belief. This evidence would likely verify the conclusion that the belief is true, however it would contradict the conclusion that the same belief is false.
b. Consider the value of each piece of evidence, how strongly or weakly does it verify or contradict a conclusion.
5) Finally select the most likely or favourable conclusion, e.g. what is your conclusion regarding the belief stated in Problem 2 given your evaluation of the evidence.
Table Template
Conclusion1 Conclusion2 Conclusion…
Evidence1 Evaluation of Conclusion1 given Evidence1
… …
Evidence2 … Evaluation of Conclusion2 given Evidence2
…
Evidence… … … Evaluation of Conclusion… given Evidence…
Please use the blank page provided to generate your MACH table. Please use a separate blank sheet for any other decision-making considerations you make that aren’t integrated into the MACH table. We estimate that this task will take approximately 15 minutes, however there is no time limit.
125
Appendix N.2 – Control Group Instructions for Concept Evaluation Instructions Your task is to evaluate the belief stated in Problem 2. You have been provided with 6-design concepts which are relevant pieces of evidence you may use to help you evaluate that belief. Please review those design concepts carefully, and provide a written record that reflects the considerations you made in reaching your conclusion. Please make sure to write down anything that you believe was relevant in helping you evaluate the belief from Problem 2 based on the evidence (6 concepts) that was provided. Record these considerations in point form notes as they occur to you. Once you feel you have performed a thorough analysis please write down your opinion regarding the belief stated in Problem 2. We estimate that this task will take approximately 15 minutes, however there is no time limit.
126
Appendix O.1 – Example of Participant Concept Evaluation Matrix
127
Appendix O.2 – Example of Participant Concept Evaluation Notes
