Addressing Large, Complex, Unstructured Problems

Written by Alexa DiBenedettoAdvised by Roger Hoerl

Senior ThesisMarch 2014

I. Introduction

Abstract

We are all faced with problems throughout our lives, not only in our educations and

careers, but also in our personal lives and relationships. Despite the universal need for

effective means of addressing problems, there is surprisingly little agreement in the

statistical or scientific literature as to how to approach problems in general. The problem

solving frameworks that do exist tend to be designed for narrow, well-defined problems.

Unfortunately, the most significant problems faced by modern society tend to be large,

complex, and unstructured, that is, not well defined. I have identified major themes in the

problem solving literature across disciplines, which I have divided into two macro-

themes: six key phases that any problem solver must pass through and consciously

consider during the problem solving process, and the key considerations that should be

kept in mind during every step of the problem solving process. In addition, I have made

some recommendations to problem solvers who face large, complex unstructured

problems, such as understanding that there is no one best method for solving every

problem, being able to use a variety of tools and techniques, even if we aren’t familiar

with them, and allowing yourself more time than you think is necessary to work through

the problem solving process.

Large, Complex, Unstructured Problems

The term “large, complex, and unstructured problem” itself can be a bit ambiguous. Thus,

I explain what I mean by large, complex, and unstructured in terms that are general

enough they can be applied in any field, not just statistics. One characteristic of these

types of problems is that they are large in scale. For instance, solving a textbook problem

2

usually involves analyzing manageable data sets that we can simply enter into Excel or

statistical software, and apply regression, hypothesis testing, and so on. On the other

hand, real-world problems usually are not neat and tidy in this way, as there are often

extremely large amounts of data and information being gathered over a large span of time

or from many sources.

These problems are also complex. While this term is somewhat ambiguous, in this

instance it can mean that the thing we are trying to “solve” is complex, or the tools

needed to solve it are complex; clearly, there is no “correct” or standard solution. In most

cases, there are multiple stakeholders who each have a different objective within the

problem.  Usually, the relationship between these stakeholders is unclear or undefined,

which causes even more complexity. There also may be legal or ethical issues in these

kinds of problems, in addition to questions about who should be responsible for

conducting and funding the research necessary to solve them. By highlighting all of these

complexities, it becomes clear why we must treat large complex problems differently

than we do regular “textbook” problems.  

Lastly, these problems are unstructured, meaning that a problem is not necessarily

precisely defined, or set up with a clearly defined set of data, background information,

problem, or solution. Sometimes, researchers do not agree on exactly what question they

are trying to answer, or what problem they are solving. In this case, it becomes

impossible to simply plug in equations and regression models etc. Many preliminary

steps must be taken to assess the root of the problem being posed, such as why it is being

asked, what is important about it, what is the significance of the results we are aiming to

obtain, and so on. This aspect of problems makes them most difficult to address in a

3

cookie cutter way. However we can hopefully establish a way to streamline this process

so that it becomes easier and more manageable to attack such problems.

Motivation

While there is not much research being done on methods of solving large,

complex, and unstructured problems, there has been some thought into the challenges

caused by not having a systematic way of attacking such problems. Hoerl and Snee

(Closing the Gap 52) highlighted one of the main issues regarding this type of problem

solving: often, most of the emphasis is placed on the tools and methods used to solve

problems, and there is little done to teach students and researchers to apply these tools in

a more broad setting where the problems are not neatly laid out like they would be in a

textbook. This process, sometimes referred to as “statistical engineering,” might prove to

be the most important aspect in solving large, complex, unstructured problems, as it

focuses on creatively applying the tools we learn from doing “cookie-cutter” examples

and problems in school. This article actually suggests that the statistical problem solving

process only relies 20% on studying the “pure” science of statistics, and that 80% is

dependent on statistical engineering, where we consider how to link and integrate these

tools. Thus, the need to establish a structured way of solving large, complex, and

unstructured problems becomes clear.

The Mathematics Department at Harvard University recognized the need to

develop courses to prepare students to address the kinds of problems they would face in

the “real world” as statisticians or researchers. Thus, a new course was created, titled

“Statistics 399: Problem Solving in Statistics,” with the goal of “helping Statistics PhD

students transition through the qualifying exams and into research”(Harvard Statistics

4

website). The professors who created this class point out that well written exams not only

benefit the students taking them, but also help the professors who are writing them

(Blitzstein, Meng 1-2). They note that there is often no textbook or library to simply copy

and paste exam questions that are unique and also stimulating. Thus, there is a need for

creative problem generation, and even more importantly, creative problem solving. None

of these problems have cookie-cutter solutions. The goal of a course like Stat 399 is to

allow students to develop skills that will not only help them to solve simple, straight-

forward problems like the ones they’ve seen in class, but also recognize the similarities

and differences between various problems in order to allow them to solve problems that

are not as straightforward. These problems, called “nano-research problems,” are meant

to mimic real world examples and research.

Example

For the purposes of this paper, we can study large, complex, and unstructured problems

through the analysis of a particular example, making it easier to understand key concepts

and pieces of advice. An example of a real-life problem that is large, complex, and

unstructured is determining how to provide an affordable yet comprehensive healthcare

system to Americans. The amount of data that is available to study is immense – patient

records, insurance claims, Medicare/Medicaid records, and more, gathered over the last

few decades. Additionally, the scope of the problem in itself is large – it involves all

Americans.

In addition, the problem is unstructured – no one solution is “correct” because

there are many varying viewpoints on what is considered a “good” healthcare system and

what we even mean by the word “good.” Who is the judge of the “goodness” of the

5

healthcare system? The government? The citizens of America? Insurance companies?

Doctors? The list is almost endless. Further, what exactly do we mean by “Americans”?

Do we mean people who were born in America? Anyone who lives in America, including

or not including illegal immigrants? Additionally, how do we define the problem? Are we

searching for a solution that is beneficial to the most people for the least amount of out-

of-pocket expense? Most profitable for the medical profession or private insurance

companies? Least expensive to the US Government? Has the highest rate of surgical

success? Least number of fatalities? Clearly, this list goes on indefinitely.

Our problem is also complex. We also have no metric to measure the success of the

healthcare model we come up with, because our original problem was not well defined.

There are multiple stakeholders, each with different objectives: doctors, patients, the US

government, insurance companies, pharmaceutical companies, hospitals, and more. There

are many legal and ethical issues regarding the healthcare system: is it ever appropriate to

refuse treatment to someone? Should cost be considered as a factor, or is age more

important? How important is the risk of failure or probability of success? Who should

pay for medical research to create this healthcare model? What is the role of individual

responsibility – if a person chooses to participate in inherently dangerous activities,

should they pay a higher premium? Who should pay for medical insurance – an

individual’s employer, the government, or the individual? This list is almost endless. It is

clear from this real life example that large, complex, unstructured problems could benefit

from a systematic approach, and that this would make solving problems in the future

much more consistent and simplified.

6

II. Literature Review and Interviews

Researchers in almost every discipline face large, complex, and unstructured problems.

Thus, an important step in creating guidelines for solving such problems that can be

applied to a variety of fields is researching and analyzing the available literature across

disciplines. While we have not interviewed professors in all disciplines or researched all

fields, we have taken a sampling of relevant disciplines that have something to say about

the process of problem solving, and acknowledge that our list may leave some input out,

and is certainly not exhaustive.

Input from Psychology

There has been some research on the problem solving process in Psychology, which

focuses on brain activity and the personality traits that contribute to the way we solve

problems. One finding has been about the “aha moment,” or the sudden moment of

clarity during which a person realizes the answer to the problem they have been trying to

solve. In their article, called “Gaining Insight into the Aha Experience,” Topolinski and

Reber define and interpret the “aha moment.” The four key components of the “aha”

experience are: suddenness, ease, positive affect, and the feeling of being right (403).

Suddenness refers to the idea that this “aha” moment comes unexpectedly and

immediately. The insight is gained and processed easily, and is gratifying. Lastly, the

problem-solvers have confidence in their solution, before they have formally tested the

“correctness” of it. In other words, “insight is an experience during or subsequent to

problem-solving attempts, in which problem-related content comes to mind with sudden

7

ease and provides a feeling of pleasure, the belief that the solution is true, and confidence

in this belief” (402-3).

Another interesting idea that this article proposes is the fact that when a general-

knowledge question is more easily processed, people feel more confident in their

answers, independent of their actual knowledge. Additionally, the sudden onset of a

solution is sometimes enough to make a person feel more confident in his or her answer

(403-4). In conclusion, the authors state, “when a solution to a problem pops into a

person’s mind, information that has been difficult to process can be processed more

fluently” (404). One negative aspect of this moment of insight is the fact that most of the

time it cannot be artificially induced, and its onset cannot be predicted. Thus, while it can

be a helpful aspect of solving a problem, we cannot rely on experiencing it when we are

solving problems.

In addition to exploring the “aha moment,” researchers in Psychology have

attempted to understand the way people make decisions based on different personality

characteristics. Professor Graham Wilson describes the way our personalities affect our

decision-making styles in a chapter of his manual (Decision-Making and Problem

Solving). The objectives of this chapter are: to identify your personal psychological type

and relate it to your personal preferences, to describe factors and personal styles that have

an impact on decision making, to distinguish between situations requiring individual

decisions and those requiring group decisions, and to identify the attributes of effective

decision makers. This unit is based on the Myers-Briggs Type Indicator (MBTI), and

highlights the different mental processes we use to think, as established by psychiatrist

Carl Jung: taking in information (perceiving), and organizing information and drawing

8

conclusions (judging) (3-1). Jung also differentiated between two different ways people

either perceive or judge: people take in information (perceive) by either using their sense

or their intuition, while people organize information by either thinking or feeling (3-2).

Combining these with one other set of preferences, we have the MBTI, which assesses

preferences on the following scale: extroversion vs. introversion (where energy is derived

and focused), sensing vs. intuition (how information is obtained), thinking vs. feeling

(how decisions are made), and judging vs. perceiving (how a person is oriented toward

the external world). Combining these four scales, there are a total of 16 psychological

types, none of which is “right” or “wrong,” “better” or “worse” (3-3-4).

This manual suggests that our preferences inherently affect the way we make

decisions. The results of sensing vs. intuition (S/N) and of thinking vs. feeling (T/F) have

a lot to do with our personal decision making styles. Sensing favors stability, basing

decisions on past experiences, while intuition favors innovation, basing decisions on

creativity and insight. On the other hand, thinking favors effectiveness, objectivity, and

logic, while feeling favors integrity, and considering people’s values and needs (3-4). The

critical thing to take away from this chapter is that a person cannot effectively make

decisions if they do not combine all of these aspects, using sense and intuition, thinking

and feeling (3-5).

Another important consideration is who should be making the decisions. Four

categories are presented: individual, consultation, group, and delegation (3-6). In

individual decision-making, the leader must make the decision alone, collecting input

from others only when it is relevant and necessary. When decision making through

consultation, the leader shares the issue with one or more people, who all provide input,

9

although the final decision may or may not take this input into consideration. Group

decision-making is similar, however all participants have an equal say in the final

decision. During delegation, the leader chooses someone to make the decision within

certain guidelines set forth by the leader (3-6). One thing to avoid during group decision-

making is a phenomenon called “groupthink,” where members “let their need to agree

with each other interfere with their ability to think about the decision critically” (3-8).

This is where the group leader plays a huge role in the decision making process, as he or

she has the authority to prevent “groupthink” from happening (3-9).

The last aspect of decision making that is addressed in this chapter are the

attributes that make an effective decision maker. These attributes are: knowledge,

initiative, advice seeking, selectivity, comprehensiveness, currency, flexibility, good

judgment, calculated risk-taking, and self-knowledge. Most of these are self-explanatory.

However, some of them need to be explained in greater detail. For instance, selectivity

means that an effective decision maker seeks only pertinent data, and avoids getting

bogged down by extraneous information. Currency implies considering current

conditions and taking advantage of current opportunities. Finally, self-knowledge means

knowing yourself and your strengths and weaknesses. (3-14-5). Two ways of judging

whether someone is a good decision maker are: whether they make competent and

confident decisions, and whether most of their decisions work out right (3-13).

The IESE School of Business’s six-step process to resolving unstructured

problems explains that the benefits of their process is its ability to bring together two

different types of people - those who see the world in terms of numbers, and those who

see the world in terms of words and emotions. Problem solving that uses only one of

10

these approaches is problematic because we may be missing key statistical or quantitative

analysis, or we may be missing key insight into why the problem exists, or how to solve

and implement the results of the problem. When both approaches are recognized and

utilized together, we have the highest chance of sustained success (4-5). Thus, we cannot

rely on one type of person to solve any given problem, and combining the strengths of

different kinds of people will help us achieve the greatest possible results.

Erika Wells, Visiting Assistant Professor of Psychology at Union College, gave

her perspective on the problem solving process in Psychology. She tried to give me an

explanation for why and how people make decisions when it comes to problem solving.

She believes that even when we think a problem is complex and unstructured, that it

actually is not. She explains that with enough work, any problem can be stated in more

simple and easy to understand terms.

Further, Professor Wells argues that most of the time, when we are solving a

complex problem, we reach an “Aha!” moment, or the moment of insight, during which

the solution suddenly becomes clear, which is supported by Topolinski and Reber. One of

the methods she suggests, when we do now know exactly what our problem is, is trial and

error. By trying simple analysis, even if we don’t know the exact problem we are trying

to solve, we can usually at least rule out the things that do not work or are not correct,

which brings us one step closer to solving the problem.

One of the important aspects of problem solving according to Professor Wells is

the size of the problem space. The larger the problem space, the longer it takes for us to

acknowledge all of our variables and alternative approaches and test them. Thus, we

11

should always take steps to try to reduce our problem space as much as we can, which

usually involves a subject matter expert.

Input from Economics

Lewis Davis, Associate Professor of Economics at Union College, suggests that the

easiest way to solve a large, complex, unstructured problem is to try to add structure to it.

He argued that oftentimes, what appears to lack structure actually does have structure, but

it is just hidden. Professor Davis’s focus is on Econometrics (applied statistics) and

Economic Development, which focuses on the policies that attempt to better the lives of

communities. In this field, many of the variables studied rely on another variable, which

relies on another variable, which leads to a never-ending chain of dependence. To avoid

this problem, he tries to find variables that are truly external and provide “exogenous

shocks to the system.” To do this, he notes that economists look for “natural

experiments” which provide us with limited knowledge that we can apply to a wider

question at least to gain a little insight, if nothing else.

Professor Davis admitted that most of the time, Economists and Economic

students do not actually systematize undefined/unstructured variables, but rather agree to

use metrics that have been developed by experts in other areas (i.e. using a political

scientist’s method for measuring how “democratic” a society is). He notes that the

hardest aspect of writing an Economics Senior Thesis is developing a problem that is

appropriate and researchable, and he often helps his students significantly with this step.

Another concern relating to problem solving in Economics is the fact that

creativity cannot be effectively “taught,” but it is often what separates the successful

12

students from those who are not successful. He questions whether this type of creativity

can be taught or if it is ingrained in us from a young age.

Input from Engineering

In addition, there is a lot to learn from the problem solving process in Engineering,

specifically Mechanical Engineering. Bradford Bruno, Associate Professor of Mechanical

Engineering at Union College, explained to me that the problem solving process in

engineering is similar to the process of solving problems in any field. The first step,

which is often overlooked, is defining the problem. The next step is generally

brainstorming and listing various alternative methods for solving the problem. Then,

testing is done to determine which alternative is most likely best. This step can be

difficult, however, because you must define your criteria for judging the “goodness” of

the model, which can be one of the harder steps of the process. For instance, in trying to

design a bridge, would the best bridge be the one that is cheapest to make and supports

the most weight? Or would it be made of the lightest materials and the longest lasting?

Clearly, there is a lot of freedom for the mechanical engineer to decide which criteria are

most important, and which ones can be compromised. The next steps are pretty standard:

building the model, testing the model, making changes to the model, and repeating the

process until an acceptable result has been achieved.

Professors of Mechanical Engineering at Union try to weave creative problem

solving into all of their courses, but it becomes more of a main focus during the senior

capstone project, where students must create and solve a problem all on their own (with

the help of an advisor). During this class, the engineering and mathematical tools are

emphasized along with the problem solving techniques mentioned above. This class is

13

meant to help prepare students to leave the college world where problems are laid out and

well defined, and enter the real world where this is usually not the case.

William Keat, Associate Professor of Mechanical Engineering at Union College,

focuses on engineering design, which is the process of studying a problem or task and

figuring out how to solve it, including the brainstorming, testing and evaluation stages.

The three main points in this process are design, building, and testing. However, within

the context of college-level courses in engineering, most problems students are asked to

solve are not large, complex, or unstructured.

In general, engineers follow a set of guidelines for solving problems. The six

main steps in this process are: recognizing the need to find a solution, defining the

problem, brainstorming possible solutions, evaluating these solutions in order to pick

what we think is best, developing a detailed solution, and testing and evaluating the

solution. For instance, Professor Keat is working with a senior thesis student to develop a

portable piece of equipment that helps senior citizens get out of chairs. The problem

needs to be solved because there is human need for it - there is no such device that we

know of that is affordable and easy to transport. The problem as they have defined it is

inventing a portable and affordable machine to help disabled people or senior citizens get

out of chairs more easily. Then, they had to come up with the requirements that they were

going to use to measure how well this piece of equipment works and how they will

measure these criteria.

Professor Keat says that brainstorming often entails breaking down the problem at

hand into the smallest possible pieces, so that it is manageable to work with. Then, if we

can individually solve each piece, we can attempt to put the pieces together to form a

14

solution of the whole problem. Next comes preliminary testing and evaluation, and then

testing of the final model to check if it passes our original criteria.

One unique difference between engineering and other fields is the emphasis on

teamwork. Most problems require a significant amount of background knowledge in a

variety of fields, which would be unreasonable for one person to know. So, combining

efforts allows us to come up with creative and helpful solutions. At the end of our

discussion, Professor Keat listed the two fundamental design principles for engineers, one

of which is the Information Axiom, which basically states to keep it simple. I think this

principle could also apply for solving large, complex, unstructured problems.

Professor Keat wrote “Engineering Design in General Education,” in which he

explains how he created a sophomore research seminar (SRS) that was targeted towards

engineers and non-engineers alike, which aimed at teaching students how to think as an

engineer. In this course, titled “Impossible Missions Design Teams,” Professor Keat

hoped to teach students with a variety of backgrounds about the fundamental principles

of engineering design. Professor Keat states, “The objective was for students to learn

basic research skills in the context of creative problem solving and engineering design.”

Professor Keat cites a book written by John Dewey, titled “How We Think,”

listing a five-step analysis of the thought process: felt difficulty, its location and

definition, suggestion of possible solutions, development of reasoning, and further

observation and experimentation leading to acceptance or rejection. He claims that these

steps can be directly compared to the steps an engineer takes when attempting to solve a

difficult or complex problem. The goal of this SRS was to use these steps, with the

15

“team” nature of these kinds of problems in mind, and develop a systematic approach that

everyone could use to solve the problems.

Input from Quality Management and Statistics

One field that commonly faces problem solving is quality management, where

researchers are constantly faced with creating solutions and analyzing them to determine

how to optimize their results. In “One Size Does Not Fit All,” Hoerl and Snee propose

attempting to break down large unstructured statistical problems into manageable

categories so that they can be analyzed and solved in a structured way. The first

characterization of a problem comes from two variables: is the solution known, and the

complexity of the problem. The answer to the first question is a simple “yes” or “no” (1).

However, simply knowing the solution to a problem does not make it easy to solve. In the

article, the example is given where the “solution” is that a particular person needs to lose

weight, but the “how,” i.e. ways to keep weight off, becomes the important factor, as

knowing a solution is meaningless if we cannot implement it. On the other hand, when

the solution is unknown, the way we approach the problem changes; instead of figuring

out how to implement the solution we already know, we must shift our attention to

figuring out what the solution even is (i.e. why is a person overweight, vs. how to

maintain weight loss).

We can also categorize a problem based on how complex it is, either low or high

based on the article. As discussed, low complexity problems involve isolated instances of

something going wrong, and they are usually easy to fix once the solution is known. High

complexity problems, on the other hand, do not necessarily have a particular isolated

problem, but rather the problem is on a much larger scale (2). When combined, these two

16

variables with two levels create a two by two matrix, which can be used to organize

problems that are seemingly very different into easy to understand categories. Once a

problem is categorized, it becomes much easier to start the problem solving process.

Hoerl and Snee study this process further in “Closing the Gap.” They state three

main problems in the field of statistics; that many times, statistical projects do not have a

high enough impact, that there is often a disconnect between statistical thinking and the

tools and methods used to implement such thinking, and that statisticians have the

potential to play a much more active role in whatever field they are in, instead of just

playing the passive role of a consultant (52). The term “statistical engineering” is

discussed, meaning in the literal sense the creative application of (statistical) knowledge.

For example, a chemist inventing a new substance is an example of science; while a

chemical engineer figuring out how to mass-produce it and use it in creative ways is the

goal of engineering. This idea can be applied to statistics just as easily. Thus, the role of a

statistical engineer is not to “advance fundamental laws of science,” but rather is to figure

out a way to use these laws of science in the most beneficial way to society as a whole.

The article suggests that the balance needed is actually 80% statistical engineering and

only 20% of statistics as a pure science (52-3).

The article suggests Lean Six Sigma (LSS) as an example of engineering: it did

not create any new tools as in science, but rather integrated existing tools in a more

efficient and beneficial way to generate enhanced results. LSS, as expressed in the article,

has not invented anything new, but rather provides a roadmap to address various types of

business problems (53). The article states that there are many people who are experts in

17

statistics and statistical tools, but lack the ability to apply these tools in a creative,

efficient, and meaningful way. This then, is the role of statistical engineering.

In “Tried and True,” Hoerl and Snee focus on the potential uses of statistical

engineering. The article opens with an example of using statistical engineering to

increase the efficiency of a transactional process. While many people can come up with

many correct methods for solving this problem, statistical engineering allows researchers

to streamline their results and create a methodical approach to problem solving (in this

case, it was designing an experiment). They realized that they could use methods they

already knew and used in other fields and apply them to their problem (in this case,

financial collections) (58).

Another example discussed is in the context of quality control. In this case, the

Product Quality Management System (PQM) was developed in the DuPont Company,

and over $30 million was gained in operating costs over two years. Many varying

statistical tools were woven into PQM, such as DoE, ANOVA, response surface methods,

and more. Again, no new tools were invented, but existing tools were linked and

sequenced in a novel way to achieve significant results (58). Together, statistical

engineers are able to create new and creative solutions utilizing existing methods to

confusing or unique problems.

Statistical engineering can be seen as a more basic example of simply

streamlining the way we approach any statistics problem. In the area of experimental

design, three steps – screening, characterization, and optimization – provide an overall

strategy of experimentation that goes beyond any one design. The screening phase

involves looking at large numbers of variables with the goal of narrowing them down and

18

figuring out which ones were most important to the problem at hand. During the

characterization phase, emphasis is put on measuring linear effects and interactions, and

finding good operating conditions. Lastly, in optimization, we use response surface

designs to make predictive models (59). However, it is important to note that no one step

can be used to solve a large, complex, and unstructured problem, but knowing the three

different stages of a problem may be a good place to start, and this methodology relates

to the generally sequential approach of solving problems.

In “Statistical Thinking: Improving Business Performance,” Hoerl and Snee focus

on adapting the Lean Six Sigma techniques to be more broad and applicable to more

types of problems. The overall strategy, called DMAIC, is an acronym for “Define,

Measure, Analyze, Improve, and Control.” An important aspect of Lean Six Sigma and

the DMAIC process is the fact that two entire phases are devoted to defining the problem

precisely, and quantifying it.

Picking a project and clearly defining the problem characterize the “define” stage.

In the case of large, complex, unstructured problems, this may prove to be the hardest

step, as once the problem is well defined it usually becomes easier to solve. The next

step, measure, is where we pick the output(s) that we believe to be important or need

improvement, and determine whether they are quantifiable. We gather as much data as

we can on these variables (130-32). Next comes the analyze step, which is where we use

our statistical tools to run various tests and create models to represent and explain our

data (132-34). The next step is the improvement step, which is when we propose a way or

ways to improve our problem based on the data we analyzed previously, and then test the

improvements to make sure they worked. This step and the one before it are generally

19

iterative, as one run of analysis and improvement usually do not solve a problem

completely, while they so usually point us in the right direction (135). Lastly, we control

our solution, so that we can sustain it once we are done working on the problem (135-36).

It is crucial to understand that this framework does not work on every kind of

problem, specifically where we don’t have a well-defined problem or we don’t have a

clear way of measuring our variables. This framework should not be used as strict

science, but rather as a suggested set of guidelines to help make the problem solving

process easier to attack.

A workshop presented by the National Science Foundation, titled “Discovery in

Complex or Massive Datasets: Common Statistical Themes,” focused on the application

of statistics in various fields, specifically in the case of large data sets. The areas

discussed were biology, finance, computer science, astrophysics, and more. Two key

points that were brought up in all cases were sparsity and machine learning. Sparsity

reflects the way we pare down information from extremely large data sets into a more

manageable size (5). In the case of the examples in this article, this usually meant

compressing data and information. Machine learning highlights the role of computers and

models in analyzing the data we have, which becomes a much more difficult and

important task when we have big data sets (6).

This workshop takes time to point out that statisticians work in all kinds of fields

and disciplines, and that even some of the examples given in this workshop come from

people who would not consider themselves “statisticians,” but use statistics in their

everyday life enough to understand its role and importance. The author(s) argues that

20

statistics can achieve the most success when it is applied to subjects outside of its origins

(7).

Statistics is meant to model situations that happen in everyday life, in all fields

such as the ones listed above. The contextual knowledge comes from all of these

disciplines, but the statistical tools used to solve these problems are all the same. Some

ways statisticians attack really large problems is by clustering, which is splitting the

problem up into smaller, more manageable groups. In this case, the way the data is split

up relies on the subject matter knowledge of the person running the experiment or

analysis, and the statistician’s role is to use this knowledge to make his job more

manageable.

One example, statistics used in biological data, shows that different kinds of data

are combined, resulting in large sets of often times confusing information. What makes

this type of problem possible to solve is the role of the scientist who has background

knowledge on the problem at hand (8-9). Without that, the statistician can only make

assumptions and guesses, and if he or she is not correct, then his outcome means nothing.

Statisticians also often need the help of a computational expert, and the teamwork among

these three groups of people is what makes large problems solvable.

The article concludes by addressing the fact that most fields deal with large and

complex data sets regularly, whereas they were not as major before the invention of

computers and the type of computing we do today. We also have to be aware of the effect

computers have on randomness in our experiments and data analysis. Some of the other

major questions and concerns brought up in the article are: how do we decide which

variables are important or not important? What role does traditional data analysis play in

21

analysis or large data sets? How can we systematically analyze the outputs of modern

statistical methods (20-1)?

Input from Science

Another field that applies problem-solving techniques in new and interesting ways is

Biology or Biochemistry. Brian Cohen, Lecturer of Biology and Biochemistry at Union

College, explains that in general, biologists have more data than they even know what to

do with (for example, DNA or brain activity data). Thus, he proposes trying to break the

problem down into the smallest possible pieces, if this is possible, i.e. “working from the

bottom up.” He suggests that this approach is more beneficial and efficient than trying to

take all of the available information and developing many pathways to the “solution,” and

then trying to test them all. Professor Cohen believes that one of the most important steps

is defining our target precisely, and then thinking about the ramifications of making the

changes we need.

Professor Cohen suggested that I look into systems biology, which is an emerging

approach to solving complex problems in biology, particularly biomedicine, which

focuses on interactions using a holistic approach. This approach is generally in opposition

to the scientific method, as it emphasizes the holistic approach as opposed to breaking the

problem down into manageable parts and solving them individually in a systematic way.

The student research that Professor Cohen advises is usually fairly well defined,

and the students generally each work on a little piece of the bigger problem that he is

trying to solve. Thus, the problem solving process in this case is defined but still requires

the creativity to come up with various ways of solving it, since the “answer” is not

always straightforward.

22

Another method for solving problems in any scientific field is the scientific

method, a process used in many analytical scientific disciplines to help acquire

knowledge or investigate phenomena. This method for problem solving was first seen in

Parmenides’s writings, where he uses deduction to solve problems. Later, the atomists

Leucippus and Democritus built upon this, and Aristotle formalized the scientific method.

The whole premise of the scientific method is to build upon previous advancements in

science by applying what we already know in order to learn something new. Importantly,

not all of the steps of the scientific method will always be used, and they don’t

necessarily need to be used in the order presented below (Edmund).

The scientific method consists of five steps, each of which requires significant

thought and creativity. The first step is formulating your question. As we have seen

numerous other times, this step generally seems trivial, but usually ends up being one of

the most important steps in the problem solving process. Sometimes, our questions are

straightforward (i.e. how can I bake a moist and delicious cake). Other times, the

questions are open-ended, and this is the case we are more interested in studying. This

stage also requires some research into similar studies or experiments done in the past.

The next step in the scientific method is creating a hypothesis. A hypothesis is a

conjecture about what we think will or will not happen. It is important that we set up our

hypotheses in a way such that we can draw meaningful conclusions from them. In

statistics, this means that our null hypothesis must be some statement of equality, which

is usually what we believe to be false. Thus, if we conclude that our null hypothesis fails,

we can accept our alternate hypothesis, which is what we believed would be true based

23

on our initial research and/or subject matter knowledge. Our hypotheses must also be

mutually exclusive and exhaustive of all the possible outcomes of our question (Harris).

Thirdly we make more predictions after we disprove or fail to disprove our

hypothesis. We want to continuously test our predictions until we have concluded

something worthwhile (or failed to do so…). Once we predict our outcome, we test it.

This is the part of the process where we see if what we predicted would be true is actually

what happens naturally. Our last step is analysis. Once we conduct our experiment or test,

we need to analyze the outcome and determine what our next steps should be. In general,

this process cannot be used only once to solve a problem, but rather will be needed in an

iterative way until we finally reach some sort of a conclusion. If we realize our

hypotheses were wrong, we should make new ones. If we notice flaws in our

experiments, we should design new ones. It is also important to note that whatever

conclusions we draw about our sample cannot be applied to anything outside of this

sample (Harris). We can use these results to help guide us but should not simple apply

our results to our target population if this is not the sample we originally tested. These

steps should be considered suggestions, which can help us in the problem solving

process, rather than strict rules that we must follow exactly.

Wolkenhauer et al. describe the role of systems biology in solving problems,

specifically in the field of medicine, in their article titled “The Road from Systems

Biology to Systems Medicine.” This article emphasizes the ways in which we can use

interdisciplinary techniques to solve biomedical problems, specifically in the context of

big data analytics where we are often trying to dig deeper into massive data sets. The

article claims that an interdisciplinary approach is necessary to “face the challenges of

24

integrating basic research and clinical practice” (502). Further, the article states:

“advances in measurement technologies can generate large-scale, multi level but also

heterogeneous datasets, requiring not only new computational platforms to manage data

but most importantly, requiring new ways of thinking, including the application and

development of methodologies from the mathematical sciences” (502). The importance

of systems biology is the multidisciplinary aspect, as it combines experimental work with

mathematical and computational analyses.

One of the motivations for studying systems biology was the time-sensitive nature

of the problems: cell processes happen quickly and standard statistical and bioinformatics

approaches do not work under these circumstances (503). Further, often times many

varying data sets are combined, which makes it more difficult to apply standard

techniques and necessary to apply new techniques that can handle heterogeneous data

sets.

This article also mentions the fact that mathematical modeling has not reached its

full potential within the field of biomedical research, but one of the positive aspects of

mathematical modeling is the fact that it can be done outside of the experiment being

conducted (504). There is a lot of emphasis placed on the relationship between the

“modeler” and the “experimenter,” as they must work together and utilize

interdisciplinary tools to solve these kinds of problems. Recently, there has been a focus

on “data-driven modeling,” and then “model driven experimentation,” which is a key

feature of big data analytics (504). Thus, not only is the actual experiment important, but

equally important is the step that comes before it, where data is analyzed without

performing actual experiments, to gather background knowledge on the problem at hand.

25

One of the common themes of the article is the need to have an interdisciplinary

approach to solving large and complex problems in biomedicine, as described: “many

scientific questions require a range of expertise from different fields. Therefore,

integration across disciplinary boundaries is crucial. The expertise for particular

technologies and experimental systems is rarely found in a single laboratory, institute, or

country, and this raises the need for standards and ontologies that support the sharing and

integration of data and models” (505).

Input from the Problem Solving Discipline

Graham Wilson, author of “Decision-Making and Problem Solving,” focuses on decision

making and problem solving in emergency situations. Thus, there is emphasis on

determining the problem (and how this is different from the symptoms of the problem),

and knowing how to react in a timely and appropriate manner. The book also has sections

that use personality traits to help determine what kind of decision maker you are and how

this affects the way we should proceed in these kinds of situations. The introduction

chapter highlights the fact that making poor decisions during the early stages of a crisis or

event can create more problems later on and make the decision-maker’s job more

difficult. The six units covered in this text are: introduction, the decision-making process,

identifying decision-making styles and attributes, ethical decision making and problem

solving, decision making in an emergency, and a course summary.

The second chapter focuses on the decision-making process. The objectives of

this chapter are to acknowledge the importance of making decisions before a crisis

happens as opposed to after, defining the steps of the decision-making process, and

distinguishing the causes from the symptoms of a real-life case study. The first thing

26

discussed is the difference between problem solving and decision-making, where

problem solving is “designed to analyze a situation systematically and generate,

implement, and evaluate solutions” (2-2). On the other hand, decision making is “a

mechanism for making choices at each step of the problem-solving process,” and it a

necessary part of the problem solving process (2-2). The five steps that are suggested to

solve problems in this book are: identifying the problem, exploring alternatives, selecting

an alternative, implementing the solution, and evaluating the solution. Wilson emphasizes

the importance of considering all of these steps, while it may not always be necessary to

write down or otherwise document them all.

Wilson breaks down each step into smaller pieces to make it more clear and easy

to understand. Wilson acknowledges, as we have seen many times before, that problem

identification is often the most important and most difficult step in the decision making

process. Every step we take after defining the problem will be based on this step. Wilson

defines a problem as “a situation of condition of people or the organization that will exist

and is considered undesirable” (2-6). Further, many people confuse the problem for its

solutions, by stating the problem in terms of its solution rather than the actual problem at

hand. Another important piece of this step is identifying the parameters of the problem,

which usually involves answering questions like, “what is happening?” “who is

involved?” or “what are the stakes?” (2-7). When considering complex and unstructured

problems, we also have to determine the best metric to define all of these parameters,

which can be surprisingly tricky.

The second step in the problem solving process is exploring alternatives. This step

actually consists of two parts: exploring alternatives, and evaluating alternatives (2-11).

27

Wilson suggests three methods for coming up with alternatives, which include

brainstorming, surveys, and discussion groups. Brainstorming consists of thinking out

loud either individually or in a group, while a survey is meant to poll a large number of

respondents. Discussion groups consist of those people who are directly involved in the

problem solving process and is similar to the brainstorming process but is more formal.

Once the alternatives are created, there must be some way to test them initially in order to

pick the “best” one to pursue. In order to evaluate alternatives, the manual suggests 6

approaches to consider: identifying the constraints, determining the appropriateness of

the solution, verifying the adequacy of the solution, evaluating the effectiveness,

evaluating the efficiency, and determining the side effects of the solution (2-13).  One

other aspect of this part of the process, which hasn’t been mentioned in other literature, is

identifying contingencies, or trying to predict what might go wrong (2-12).

Next, we select an alternative. Whichever alternative we decided was the “best” in

the last step becomes the subject of our main attention during this step. There are many

factors that should be considered when selecting an alternative, such as political, safety,

financial, environmental, and ethical factors (2-15). Also, sometimes it is difficult to

implement this alternative once it is chosen, as there are often unforeseen repercussions

or consequences. This manual provides a worksheet that can help us determine the best

solution by listing the solution and its limiting factors, and considering combining

solutions if one does not stand out as a clear winner (2-16). After we pick the best

alternative, we must implement it, which generally consists of 5 steps: developing an

action plan, determining objectives, identifying needed resources, building a plan, and

finally implementing this plan. Developing the action plan entails creating a series of

28

steps to follow, as well as who will be responsible for each step. Then, we determine

objectives, which are “measurable targets that we use to monitor progress and establish

priorities” (2-17). We identify the resources we will need, considering cost, time, and any

special requirements. Then, we build our plan, which should state “who will do what by

when, where, and how,” (2-17). Finally, we put this plan into action. The manual includes

a checklist that can aid us in following through with the implementation and action plan

process.

The last step is evaluating the solution, which involves two parts: monitoring the

progress, and evaluating the results. To monitor the progress, we must ask whether the

situation has changed, are other resources required, and are there other alternative

solutions required (2-21)? This step is an ongoing process that does not end at the end of

our project, but must be continued to ensure our situation doesn’t change (2-21).

Chapter 4 focuses on the role of ethics in the decision making process,

specifically in emergency situations, and attempts to identify potential ethical issues that

may come up and ways which we can apply the problem-solving model to these ethical

issues. Ethics is defined as a “set of standards that guides our behavior, both as

individuals and as members of organizations; principles of right and wrong, such as being

honest, fair, and treating others with respect (4-2). It is important to note that ethics do

not imply legality. This manual offers a list of ethical “don’ts” that specifically apply to

emergency situations: don’t exceed your authority or make promises, and don’t use your

position to seek personal gain - it even goes as far as to make sure you not only act

ethically, but are aware of the appearance of being unethical as well (4-3). In addition,

some ethical “do’s” are listed: do place the law and ethical principles above personal

29

gain, do act impartially, so protect and conserve agency property, and do put forth honest

effort (4-4).

Next, the manual outlines four components of ethical decision-making:

commitment, consciousness, and competency. Commitment entails “demonstrating a

strong desire to act ethically and to do the right thing, especially when ethics impose

financial, social, or psychological costs” (4-6). Consciousness involves “seeing and

understanding the ethical implications of our behavior and applying our ethical values to

our daily lives.” Finally, competency can be broken down into three parts: evaluation

creativity, and prediction. Evaluation is the ability to “collect and evaluate relevant facts

and [to know] when to stop collecting facts,” creativity is the “capacity to develop

resourceful means of accomplishing goals in ways that avoid or minimize ethical

problems,” and prediction means the “ability to foresee the potential consequences of

conduct and assess the likelihood or risk that persons will be helped or harmed by an act”

(4-7).

The fifth unit focuses specifically on decision making in an emergency, which

changes the way we go about making decisions. Some impediments people face in these

situations are due to stress, such as time pressure, fatigue, lack of information, and

uncertainty. Often, people who must make decisions under stress experience conflict with

other key players, perceive selectively because of sensory overload, or experience

perception distortion and poor judgment (5-2). In addition, these people are usually less

tolerant of ambiguity and are susceptible to making premature decisions, have a

decreased ability to handle difficult tasks or work effectively, and experience a greater

tendency toward aggression and escape behaviors (5-2).

30

The last chapter summarizes the key concepts from the rest of the manual. It

defines problem solving versus decision-making, and outlines a five-step problem solving

model: identify the problem, explore alternatives, select an alternative, implement the

solution, and evaluate the solution (6-2). Next, it lists the factors that affect decision

making, which are political,, safety, financial, environmental, and ethical (6-3). The

manual also lists four decision-making styles, based on the MBTI test: sensing, intuition,

thinking, and feeling (6-4). The four ways to make a decision as outlined by Wilson are

individual, consultation, group, and delegation (6-5). The manual lists many attributes of

a good decision maker: knowledge, initiative, currency, flexibility, self-knowledge,

calculated risk-taking, and good judgment are just a few (6-6). Next, some ethical “do’s”

and “don’ts” are listed: don’t exceed your authority, or use your position to seek personal

gain, but do place the law above personal gain and act impartially, while putting forth an

honest effort (6-8). Finally, the three components of ethical decisions are commitment or

motivation, consciousness or awareness, and competency or skill (6-9).

In their article, titled “The Process of Solving Complex Problems,” Fischer et al.

discusses Complex Problem Solving (CPS) and how it applies to various fields. Five key

points of this process are highlighted: information generation, information reduction,

model building, dynamic decision-making, and evaluation. The article’s first sentence

does a good job of summing up the motivation behind this research: “In times of

increasing globalization and technological advances, many problems humans have to face

in everyday life are quite complex, involving multiple goals as well as many possible

actions that could be considered, each associated with several different and uncertain

consequences, in environments that may change dynamically and independent of the

31

problem solvers’ actions” (20). Thus, the aim of the article is to come up with a

systematic way of solving complex problems that can be applied in any field.

The article tries to more clearly define exactly what is meant by “complex

problem solving” by describing some of the important characteristics of such problems.

For instance, a problem is defined as complex when it has many elements or parts. A

problem exists, as described by the article, when we have goals but do not know how to

reach them (22). Thus, problem solving is trying to find the means of achieving our goals.

One of the most important aspects of problem solving in any field is that the

researcher should have knowledge on the tools needed to actually solve the problem (i.e.

applying equations, models, etc.), but also should have subject matter knowledge (or

should have an expert nearby).

The article continues, outlining the most important aspects of complex problem

solving (CPS), the first of which is human problem solving (23). The most important

aspects of this are creating an internal representation of the problem space. Additionally,

we must try to formulate a method for achieving our goal, given our internal

representation of it. This process is iterative and often requires us to go back to the

beginning if our methods do not work out. Frequently, the problem solver must define

parts of the problem him or herself, which is consistent with what we have to do to solve

unstructured problems.

Another aspect of CPS is expertise. Having expertise in a given field greatly

reduces the amount of things we must filter through to find the most important things. It

allows us to make educated decisions (even guesses sometimes) and helps guide our

technical analyses. In general, experts can solve problems faster than novices in their

32

given fields (25-8). Next, the article outlines decision-making strategies. The article

concluded that the important part of this discussion is not which one strategy is best, but

how to discern when to use which strategy depending on the circumstances. Problems

solvers base their decisions on the known solutions if they are available. If not, they focus

on solving the problem at hand given the circumstances, or trying to find new

information that would help them solve the problem. Information reduction is also

important in this process because it can simplify a problem that at first seems really

complex and large. The article acknowledges that a lot of research needs to be done in the

field of complex problem solving. However, the techniques listed above provide a good

start (37).

Similarly, TRIZ, also commonly referred to as the “theory of inventive problem

solving/TIPS,” is a problem solving technique, which was developed by Soviet inventor

Genrich Altshuller and his colleagues around 1946. The basis of this theory is that we can

use already established principles and insights from other fields that many not necessarily

be related to what we are studying, but if they are similar enough we can apply these

techniques to help us solve our new problems. Specifically, TRIZ is often used where we

have to somehow avoid an inherent conflict within our problem, such as creating a high-

power engine that is also light. These two qualifications seemingly contradict themselves

- the most high-power engine we could make is probably really large and heavy, while

the lightest engine we could make is not high-powered (Technical Innovation Center).

The idea behind TRIZ is to take our problem, find a general problem that has been

solved that can give us insight into our own problem, and apply the solution to this

general problem somehow to our specific problem. Technical contradictions are solved

33

using 39 elimination principles, while physical contradictions are solved using 4 basic

principles, which study super systems, subsystems, and separation of time and space. All

of this information was put together into a matrix, which attempts to systematically

approach the way we attack problems (Technical Innovation Center). Thus, we can use

TRIZ to solve large, complex unstructured problems because we can use previous results

from similar problems and apply them in new and creative ways.

III. Key Themes from Literature

I have found many recurring themes that appear throughout the literature for large,

complex, unstructured problems. I have organized them into two macro-themes – the first

contains six key phases that, regardless of the problem solving method used, a researcher

must pass through and consciously consider. I believe that these phases cannot be skipped

and must be taken into consideration regardless of the type of problem. The second

macro-theme contains the key considerations that a researcher must take into account

during all phases of the problem solving process. Thus, while there is no “best” or

“correct” method of problem solving, these key themes will always be encountered and

must always be taken into consideration.

Within each macro-theme, I have identified additional sub themes. In macro-

theme one, I suggest that there are six phases that a problem solver will necessarily

encounter (in order): identifying the motivation for solving the problem at hand, problem

definition, understanding the context, strategizing, brainstorming, testing, and evaluating

alternatives, and maintaining the solution. Within macro-theme two, I distinguish three

important things that all researchers much consider throughout the problem solving

34

process: gathering subject-matter knowledge, assembling a team of diverse perspectives,

and using diverse methods.

35

Figure 1: Outlining the six key phases of the problem solving process, and the three key considerations that should be applied during all six phases

36

I. While there is no “best” method that can be applied to every problem, there are

certain phases that appear in any problem solving process that cannot be skipped or

ignored.

1. Identifying the motivation for solving the problem at hand.

The first step of solving any problem, which cannot be skipped or ignored,

is understanding the need to solve the problem at hand. As Fischer et al.

explain, “in times of increasing globalization and technological advances,

many problems humans have to face in everyday life are quite complex,

involving multiple goals as well as many possible actions that could be

considered, each associated with several different and uncertain

consequences, in environments that may change dynamically and

independent of the problem solvers’ actions” (20). Clearly, whether we

realize it or not, we are faced with large, complex, and unstructured

problems in our daily lives. Whether it is in our professional lives, our

social lives, or our personal lives, these problems arise constantly.

However, there are some problems that need to be solved

immediately, and some that are not a high priority for us to solve. Further,

we must identify why this is a problem worth or not worth solving, by

considering the consequences of solving or not solving any given problem.

For instance, we might not have enough time to solve a given problem,

thus it is not worth solving. Or, we might only be able to slightly make a

situation better, like in the case of quality management for example. If we

can only reduce mistakes in a certain piece of machinery by such a

37

negligible amount that it wouldn’t make a noticeable difference, perhaps it

is not worth spending the time, money, and manpower to fix the problem.

In other words, given the many challenges of modern life, we must

“choose our battles” carefully.

2. Problem Definition

We cannot attempt to solve a problem that has yet to be defined. As in the

US Healthcare system example, we cannot begin to try to come up with a

“better” healthcare model if we do not know exactly what problem we are

trying to solve.

The first aspect of this stage is defining the actual problem

statement. As Hoerl and Snee state in “One Size Does Not Fit All,” this

step is often the hardest step especially in the case of large, complex, and

unstructured problems. In general, most people are comfortable solving

problems that are well defined, such as textbook problems in a college

course, because once the problem is defined they can easily identify the

appropriate methods to use. However, in the case of unstructured

problems, we have to define the problem before we can continue the

problem solving process. Wilson describes how often people fail to clearly

define the problem, and sometimes confuse the problem for its solutions,

which we should always avoid (2-6).

Another important aspect of this phase is defining the goals of our

research and problem solving process. In the case of the US Healthcare

system, do we want to cover the largest number of Americans? Increase

38

life expectancy? Or achieve some other goal? We must consider the

multiple stakeholders and ensure our problem is defined in terms of

everybody’s interests.

Similarly, we must make sure that we are determining the

necessary resources we will need to solve the problem, so that we can

define the problem and our goals keeping this in mind. As explained by

the IESE, if we define the problem in such a way that it becomes

impossible to solve, then our efforts will be futile.

3. Understanding the context

One important aspect of a large, complex, unstructured problem is that its

solution often requires deep understanding of its context. Big problems

have defied solution for a reason; to address them we need to understand

how they got there, and what sustains them. For instance, in the case of the

US Healthcare system, it is important to consider the economic status of

the US, the political situation, and the social condition of the country, as

well as the history of healthcare in the US. This could significantly

enhance the way we approach the problem and the solutions we come up

with.

In addition, we must know what has already been attempted or

solved in relation to our problem. For instance, using the methodology of

TRIZ, we should look for any information or advancements that relates to

our project, even if it does not directly answer the question we have posed.

Using TRIZ, we could apply similar techniques to a portion of our project

39

that is similar to a problem that has already been solved. In general, we

can apply strategies from various fields and problems that have been

solved in the past that relate to our field, regardless of whether or not it

directly answers our question. Any advancements we make simply by

researching what has been done already will help us achieve our results.

4. Strategizing

First, we must acknowledge the need for a strategy. Fischer et al. point out

that in order to create a strategy, we must first create the problem space

that we will be working within, which is defined as, “a set of possible

states of the problem, given the initial state, the applicable operators, and

certain goal states” (23). In small, structured, or simple problems, i.e.

textbook examples, this step is often assumed and not emphasized.

However, especially for complex problems, this step is crucial, as we

cannot continue with the problem solving process unless we

systematically classify the problem. Frequently, we realize that the current

format of the problem and the problem space do not promote our creativity

and ability to solve the problem, so we must go back to the problem

definition and restructure the problem space.

It is in this way that we utilize a sequential approach to problem

solving. Once we lay out the problem space, we can try to break up the

problem into smaller, sequential “steps” that make resolving the problem

easier. For example, in “Tried and True,” Hoerl and Snee argue that we

can use three steps in a sequential approach to solve experimental design

40

problems: screening, characterization, and optimization. During the

screening phase, our goal is to identify the most important variables, so

that in the characterization phase we can measure linear effects and

interactions. Finally, in the optimization phase, we make predictive

models from the data analysis in the first two steps. Approaching a

problem this way can also be helpful on a larger scale: first we want to

narrow down the factors we are studying, and then measure the effects

they have on the given question. Finally, we want to create some sort of

final model or solution based on this process. Importantly, we cannot just

use one step of this approach to solve a large, complex, unstructured

problem, and they must be used in this arrangement to achieve the desired

results.

5. Brainstorming, testing, and evaluating alternatives

One crucial step in the problem solving process is defining what our

criteria will be for determining how “good” our solution will be.

According to the IESE’s six-step problem solving process, “the key to

successfully solving a problem is choosing which if all of the possibly

important issues will really be the most critical in making a decision” (2).

In our example of the US Healthcare system, this means determining

which criteria will be used to judge the “goodness” of our healthcare

model – will we be most interested in reducing citizen’s out-of-pocket

payments, decreasing overall fatalities, least amount of government

spending, or some combination of these and many other criteria? We must

41

carefully consider this step before we move on, as it impacts the way we

judge the “goodness” of our solutions, which ultimately is the end result of

our problem solving process.

The next logical step in this process is identifying alternatives. The

IESE states that this step is more difficult in a large, complex, unstructured

problem, but it is critical nonetheless (3). This step is rarely mentioned in

the quality management and statistics literature, but is mentioned in the

engineering, biomedical, and psychological literature. I also think it is

important to note, as the IESE article explains, researchers must choose

the criteria before selecting their alternatives, as bias is introduced when

this process works in reverse (3). We want to avoid bias at every stage of

the problem solving process, and it is a quality found in human nature that

we will bias our criteria if we already have the alternatives laid out.

Wilson suggests various methods we can use to produce alternatives –

brainstorming, surveying, and utilizing discussion groups – each of which

having its benefits and unfavorable effects (2-13).

Another important aspect of this phase is the role of creativity. We

cannot teach people how to be creative, yet it is crucial that the people

who are working on the problem at hand are creative enough to think

outside of the box when it comes to alternative development. If we can

only come up with solutions that have been used in the past or that are

similar to things we have seen or done before, we are severely restricting

our ability to solve a large, complex, unstructured problem.

42

Once we pick our alternatives, we must test them to determine

which one or ones are the most likely to be successful, by using our

criteria. This is where researchers have the most freedom to use whatever

tools and methods are appropriate for their particular project, but this step

can also be confusing for those who are studying large, complex

unstructured problems because the correct method might not be apparent.

We continue to narrow down our alternatives by analyzing them and

comparing them to the criteria we selected until we are happy with our

results and our final solution.

Wilson suggests various ways of determining which alternate

solution is “best,” in addition to measuring the criteria we set forth. The

six things he recommends considering are: identifying the constraints,

determining the appropriateness of the solution, verifying the adequacy of

the solution, evaluating the effectiveness, evaluating the efficiency, and

determining the side effects of the solution (2-13). In addition to our

criteria, these guidelines can be a good way for us to determine which

alternative is the best and are a good place to start especially in the case of

large, complex, unstructured problems.

One last step we should always consider during this phase is

whether or not it would be feasible to actually implement our solution.

Wilson constructs a five-step method for implementing a solution once we

have picked the best alternative: developing an action plan, determining

objectives, identifying needed resources, building a plan, and finally

43

implementing this plan (2-17). We should always consider these steps, in

order, to ensure that since we have gone through the trouble of solving our

problem, we can successfully implement it.

6. Maintaining the solution

I believe that ensuring we can maintain whatever solution we decide to

implement is just as important as being able to come up with the solution

in the first place. Hoerl and Snee, Wilson, and the IESE all emphasize the

importance of sustaining our results, stating that we often need to come

back to our problem and reevaluate the solution to ensure it is still

successful after some time has passed and the problem has been closed.

Similar to the problem solving process as a whole, sustaining the

successful result is usually iterative and requires constant maintenance, but

is necessary to guarantee that all of the hard work it took to create the

solution was not in vain.

II. As we saw in Figure 1, the following key considerations should be kept in mind

during all of the stages of the problem solving process.

1. Gathering subject matter knowledge

There are many times during the problem solving process in which it

would be beneficial to have subject matter knowledge. For instance, when

defining the problem and the criteria, it would be helpful to have a subject

matter expert there to advise the researcher on how to appropriately define

the problem. For example, in the US Healthcare problem, it would be

appropriate to gather doctors, insurance companies, and government

44

officials to help the researcher understand exactly how the current

healthcare system works, so that he or she can correctly define the

problem. The NSF workshop mirrors this idea, highlighting the role of the

subject matter expert. Without the expert’s advise, a statistician cannot

make informed decisions. Specifically in the case of large, complex,

unstructured problems, data is often combined from multiple sources and

it is up to the subject matter expert to make sense of it all so the statistician

or researcher can perform his or her analysis.

Fischer et al. highlight another important aspect of consulting

experts during the problem solving process: it allows us to filter through

large amounts of information more quickly with less wasted time (26).

They explain that problems become easier to solve when we have a wealth

of subject matter knowledge, and something that seems complex may

actually be simple when we consult an expert. Further, consulting experts

allows us to make educated guesses instead of random guesses, and can

usually help guide our analysis. Before we even analyze any data, a

subject matter expert can help us make decisions that will directly impact

our analysis. Such expertise is critical at each subsequent phase of the

problem solving process.

2. Assembling a team of diverse perspectives

A significant amount of time should be spent constructing a team of

diverse individuals who will all bring something meaningful to the

problem solving process. Diversity can refer to different personalities as

45

well as disciplines and levels of experience. As the IESE describes, there

are two types of people – those who see the world in terms of numbers and

those who see the world in terms of emotions – and both types of people

should be involved in the problem solving process together (4-5). Further,

consideration should be given towards who should be making the

decisions and solving the problem.

Wilson presents four categories of decision-making: individual,

consultation, group, and delegation (3-6). Based on the context of the

problem, an appropriate method should be used, and this step should not

be overlooked. We should take extra care to ensure we avoid the

“groupthink,” where the desire to conform to the group drives people to

make biased decisions. Groupthink can be extremely detrimental to the

problem solving process, as creative solutions can be stifled by leaders

who are not open to new ideas. Leaders must be careful to avoid only

working with people who think like them – something that unfortunately

happens all too often without leaders and bosses even being aware of it.

By intentionally picking individuals who bring different perspectives to

the process, problem solvers can increase the chances of coming up with a

successful result.

3. Using diverse methods

I believe that one of the most important themes in the literature has been

that regardless of the problem solving technique, researchers must be

willing to be creative and use methods and techniques that are not familiar

46

to them. Sticking to what is comfortable and familiar will not foster

“good” problem solving. Rather, it prevents us from solving problems,

specifically large, complex, unstructured problems.

In addition, problem solvers must understand that only rarely will

one single method or technique solve any problem, regardless of the size,

complexity, or structure or lack thereof. So, it is important to keep a large

toolbox of methods and techniques available to use whenever necessary,

as well as the ability to be flexible and creative. A combination of

technique and creativity is important in problem solving, and no one

method or technique, when used alone, is universally “best,” which is

controversial to much of the literature that attempts to compare various

methods and pick the “best” one.

IV. Recommendations for Problem Solvers

After compiling as many of the problem solving techniques from various disciplines, I

have created my own list of recommendations that I would suggest to people trying to

solve large, complex, unstructured problems. This list is not exhaustive, and is meant to

be supplemental and additive to the key themes I outlined in the previous section.

My first recommendation for problem solvers is also the most important in my

opinion – it is crucial to understand that while I have provided a list of phases and

techniques that should be applied, this methodology is not meant in any way to be the

“correct” or “only” way of solving a problem. It is meant, however, to be a good starting

point for people who do not know what the steps are for solving a large, complex,

47

unstructured problem. One issue too often seen in the problem solving literature is that it

spends too much time defending different methods and techniques as the “best” ones.

However, I believe that this is a waste of time, as no single method can solve every

problem. This brings back the role of statistical engineering, where researchers and

problem solvers should focus on utilizing tools they have in inventive and creative ways,

rather than attempting to figure out which tool or tools are the “best” and work for every

problem.

In addition, I recommend that those responsible for addressing big problems

spend a significant amount of time gathering the right group of people to assist them

during the problem solving process. This step is often overlooked because natural

“teams” can form through existing relationships and problem solving teams in the past.

However, I advise that each problem be thought about individually, so that various

personalities and experts can be assembled to form a cohesive and productive problem

solving team. As I mentioned earlier, the phenomena of “groupthink” should be avoided

at all costs, so that the same mistakes are not made consistently and new approaches can

be utilized. No one person, regardless of their experience or expertise, can solve every

problem, and it is always beneficial to have diverse people on the problem solving team.

This is especially true for large, complex, unstructured problems, where there is often a

diverse amount of information necessary to make informed decisions, and it would be

inefficient to try to learn all of the required background information. In particular, we

need to include those with subject matter knowledge, those with data analysis skills, and

those who can creatively apply the tools and techniques they know.

48

Another important recommendation is being comfortable with using tools and

techniques that you are not familiar with. It is easy to rely on the tools that you know and

have worked in the past, but when dealing with large, complex, unstructured problems, it

is important to look outside of your “toolbox” when thinking of ways to solve the

problem at hand. This is another reason why it is so important to gather a diverse team,

each of whom brings a different viewpoint and skill set to the table. Being able to adapt

and adjust your skill set and use unfamiliar tools will make you more likely to succeed.

Having a diverse toolkit will enable problem solvers to select the method based on the

unique nature of the problem, as opposed to tailoring the problem to fit within a certain

framework so they can use the tools they are familiar with. In other words, it will help

problem solvers avoid viewing every problem as a nail because the only tool they have is

a hammer. Often, this will require researchers to read diverse literature outside of their

own fields.

I suggest reading the manual by Graham Wilson, referenced several times in this

thesis (“Problem Solving and Decision Making”). While this manual is meant to help

students teach themselves the methods of problem solving, it focuses on emergency

situations. It may not seem relevant to the process of solving large, complex, unstructured

problems, but it actually does a great job of breaking down this process into its most

basic definitions (i.e. what do we define as a “problem”?), and then providing various

steps and methods for working our way through the problem solving process. If you are

looking for a comprehensive guide to the basics of problem solving, especially in an

emergency situation, this is a great place to start, and it provides worksheets that a

49

problem solver could easily fill out with the necessary information to be even simpler to

use and understand.

Another recommendation is to give yourself more time than you think you might

need to solve the problem. Problem solvers should take the time to carefully define the

problem, as this is the “make or break” aspect of the problem solving process.

Additionally, researchers and problem solvers could benefit from having time to create

alternatives and analyze them, and then have more time to contemplate these solutions

and reflect on what they have accomplished. As mentioned in the article on the “Aha

moment,” you cannot force or expedite these moments of clarity; often just having more

time to think about the problem will prove beneficial (Topolinski and Reber). Going

along with this, do not get frustrated if your first few solutions do not work out. If

problem solvers could solve any problem in a few simple steps, large, complex,

unstructured problems would not even exist. So, do not let a temporary failure or minor

setback cause you give up on the problem solving process as a whole.

One final recommendation I have for problem solvers is to make sure that once

you have a solution, you can successfully implement and sustain the results. Our efforts

would be futile if we spent our resources on solving a problem, and could not implement

and sustain the results. This implies that throughout the problem solving process,

including deciding which problems to work on, we consciously consider the long term

view of sustainability.

V. Conclusion

In this thesis, I researched available literature and interviewed experts about the problem

solving process in various disciplines, such as psychology, economics, biology,

50

engineering, quality management and statistics, and the problem solving discipline. I

organized the wealth of information into two macro-themes: six key phases that a

problem solver will necessarily encounter during the problem solving process, and key

considerations that must be consciously applied during all stages of the problem solving

process.

The six phases of the problem solving that I have identified are: identifying the

motivation for solving the problem at hand, problem definition, understanding the

context, strategizing, brainstorming, testing, and evaluating alternatives, and maintaining

the solution. Within each of these phases there are other important things to consider.

These steps are not meant to be definitive and I do not claim that this is the “best” process

of problem solving. Rather, I believe that these steps are necessary regardless of the kind

of problem, and that problem solvers should seriously reflect on all of them.

The three key considerations that I believe should be employed during all stages

of the problem solving process are: gathering subject matter knowledge, assembling a

team of diverse perspectives, and using diverse methods. These themes should not be

overlooked, and researchers should make a conscious effort to utilize them during every

stage of the problem solving process.

In addition to the key themes above, I have also come up with a list of

recommendations to problem solvers, especially when attempting to solve large,

complex, unstructured problems. The first one, which I believe to be the most important,

is to take everything I have gathered and organized lightly. I do not intend this to be a

comprehensive method for solving large, complex, unstructured problems. Rather, I

suggest this paper as a good starting place for those who are unsure of how to begin

51

solving a large, complex, unstructured problem. While much of the literature on problem

solving in various disciplines spends time debating which methods and techniques are

best, I argue that there is no one best method. However, I believe that the phases I have

outlined will be present regardless of the type of problem being solved.

In addition, problem solvers must be flexible in all aspects of the problem solving

process. Whether this means working with people who have a variety of subject matter

backgrounds, people with different personalities, or being able to utilize tools and

techniques that they are not comfortable with or used to using, researchers and problem

solvers must be able to step out of the box if they want to solve a large, complex,

unstructured problem.

Finally, researchers should ensure that they have the necessary resources

available. These resources come in many forms, for instance time, money, and

manpower. Problem solvers should always give themselves more time than they

anticipate needing to solve a big problem, to allow them to take the problem definition

stage seriously, as well as all of the subsequent stages. Further, researchers should make

sure that they have access to all of the necessary resources they will need to not only

solve the problem, but also implement and sustain their results. If results cannot be

implemented or sustained, then the problem solving process was ineffective.

While there are many problem solving techniques presented within various

disciplines, my goal was to condense these methods to identify important themes that

could help people solve big problems across disciplines. By using the guidelines and

consciously considering the key themes I have proposed, I believe we can successfully

solve large, complex, unstructured problems.

52

Works Cited

Bruno, Brad. Personal interview. 23 Jan. 2014.

Cohen, Brian. Personal interview. 12 Feb. 2014

Davis, Lewis. Personal interview. 27 Jan. 2014.

Edmund, Norman W. "The Scientific Method Today." The Scientific Method Today.

Web. 16 Mar. 2014.

Fischer, Andreas, Samuel Greiff, and Joachim Funke. "The Process of Solving Complex

Problems." The Journal of Problem Solving 4.1 (2012): n. pag. Print.

Harris, William. "How the Scientific Method Works." HowStuffWorks.

HowStuffWorks.com, 14 Jan. 2008. Web. 16 Mar. 2014.

Hoerl, Roger W., and Ronald D. Snee. "Closing the Gap." Quality Progress (2010): 52-

53. Web.

Hoerl, Roger W., and Ronald D. Snee. "One Size Does Not Fit All." Quality Progress

46.5 (2013): 48. Web.

Hoerl, Roger W., and Ronald D. Snee. "Tried and True." Quality Progress (2010): 58-60.

Web.

Hoerl, Roger, and Ronald Snee. "Statistical Engineering: Tactics to Deploy Statistical

Thinking." Statistical Thinking: Improving Business Performance. 2nd ed.

Hoboken, NJ: Wiley & Sons, 2012. 93-138. Print.

Keat, Bill. Engineering Design in General Education. Symposium on Engineering and

Liberal Education. Web.

Keat, Bill. Personal interview. 30 Jan. 2014.

53

Meng, Xiao-Li, and Joseph Blitzstein. Nano-Project Qualifying Exam Process: An

Intensified Dialogue Between Students and Faculty. Department of Statistics,

Harvard University. Web.

National Science Foundation. Discovery in Complex or Massive Datasets: Common

Statistical Themes. Proc. of Mathematical Association of America (MAA),

Washington, DC. 2007. Print.

Rosenberg, Mike. "IESE’s Six Step Process for Resolving Unstructured Problems." IESE

Business School. Web.

"Technical Innovation Center, Inc." Technical Innovation Center, Inc., Oct. 2012. Web.

16 Feb. 2014.

Topolinski, S., and R. Reber. "Gaining Insight Into the "Aha" Experience." Current

Directions in Psychological Science 19.6 (2010): 402-05. Print.

Wells, Erika. Personal interview. 29 Jan. 2014.

Wilson, Graham. Decision Making and Problem Solving: Student Manual. 2002. Print.

Emergency Management Institute.

Wolkenhauer, Olaf, Charles Auffray, Robert Jaster, Gustav Steinhoff, and Olaf

Dammann. "The Road from Systems Biology to Systems Medicine." Pediatric

Research 73.4-2 (2013): 502-07. Print.

54