IS research strategies Richard T. Watson copyright 2005

IS research strategies

Richard T. Watsonrwatson@terry.uga.edu

copyright © 2005

The fundamental problem• All research strategies and

methods are flawed• A method’s strengths are also the

source of its weaknesses• Researchers must decide which

method best fits their goals

Classes of strategies

Experimentalsimulations

Fieldexperiments

Fieldstudies

Computersimulations

Formaltheory

Surveys

Judgmenttasks

Laboratoryexperiments

Obtrusiveresearchoperations

Unobtrusiveresearchoperations

Universalbehavioralsystems

Particularbehavioral

systems

A

B C

A three-horned dilemma• It is always desirable to maximize:

– Actors• Generalizability with respect to populations

– Behavior• Precision in control and measurement of variables

– Context• Existential realism, for the participants, of the

context within which those behaviors are observed• There is no way to maximize all three goals• The researcher must choose among the

lesser of three evils

Quadrant I strategies• Existentially real for the

participants• Field studies

– Settle on C– Lack precision and generalizability

• Field experiments– Increasing precision

Quadrant II strategies• Deliberately contrived settings• Laboratory experiments

– Settle on B– Lack contextual realism and

generalizability• Experimental simulations

– Increasing realism

Quadrant III strategies• Context should not play a part in

the behavior of concern• Sample surveys

– Settle on A– Lack contextual realism and precision

• Judgment– A few population units construed as

‘judges’ rather than ‘respondents’

Quadrant IV strategies• Not empirical• No actors, no behaviors, and no

context• Formal theory

– Settle on A– Lack contextual realism and precision

• Computer simulations– Increasing realism– Model a particular concrete system

The construct validity dilemma

• Goal is to test the relationship between concept A and concept B

Other factorsaffecting concept B

Concept A Concept B

Operationaldefinition a

Operationaldefinition b

Other factorsaffecting operational

definition of b

The relationships• A-B is conceptual and cannot be

tested empirically• A-a and B-b are definitional and

can only be tested indirectly• a-b is an empirical relation used to

assess the validity of the three other relations

Faulty logic• Since strategy X, the one I am opposing is

bad and has many flaws, then strategy Y, the one I am proposing must be good.

• It is a waste of time arguing which is the right strategy

• Argue about how to combine multiple strategies in multiple studies of a given problem

• Studying a problem by multiple methods means that you do not focus on one strength or weakness