HRM4710 Qualitative Research Methods

Week 23Thinking about ‘triangulation’ & other issues of mixing methods

Dr. Maria Kapsali (m.kapsali@imperial.ac.uk)Imperial College Londonwith Dr. Uracha Chatrakul Na Ayudhya

2

Learning Objectives

By the end of this workshop, you should be able to:

1. Critically discuss the arguments for & against multi-strategy research

2. Appreciate the range of possibilities in engaging in multi-strategy research, including triangulation

3. Understand what is meant by ‘mixing methods in a qualitatively driven way’ as outlined by Mason (2006)

4. Critically reflect on the appropriateness of multi-strategy research for your own research interests

3

Contents

• Part 1 Theory: Qualitative research, Multi-level strategy and triangulation

• Part 2 Types of triangulation + Activity

• Part 3 Practical example

4

Part 1: Theory Types of qualitative research

Case study Attempts to shed light on a phenomena by studying indepth a single case example of the phenomena.  The case can be an individual person, an event, a group, or an institution.

Grounded theory Theory is developed inductively from a corpus of data acquired by a participant-observer.

Phenomenology Describes the structures of experience as they present themselves to consciousness, without recourse to theory, deduction, or assumptions from other disciplines

Ethnography Focuses on the sociology of meaning through close field observation of sociocultural phenomena. Typically, the ethnographer focuses on a community.

Historical Systematic collection and objective evaluation of data related to past occurrences in order to test hypotheses concerning causes, effects, or trends of these events that may help to explain present events and anticipate future events. (Gay, 1996)

5

Multi-strategy research

•Refers to research that combines qualitative & quantitative research

•On the surface, this can be seen as a straightforward way of breaking down the qual-quant divide

•But if we dig deeper, this approach is not without difficulties

6

Make a list:

• What are some of the advantages?

• What are some of the difficulties?

7

The Argument Against Multi-strategy Research (I)

The embedded methods argument: Research methods carry epistemological and ontological commitments

Thus multi-strategy research is not feasible or even desirable

Why? Integrating research strategies could lead to danger of ignoring assumptions underlying research methods

Could result in transforming “qualitative inquiry into a procedural variation of quantitative inquiry”

(Smith & Heshusius, 1986: 8)

8

The Argument Against Multi-strategy Research (II)

• The paradigm argument:

Quantitative and qualitative research are

separate, incommensurable paradigms

Thus even when combined they are

incompatible: the integration is only at a

superficial level and within a single paradigm

9

Your view:

• Do you see data as just data?

• Your reflection on this shapes how

you engage in the multi-strategy research

debate

10

Two Versions of the Debate About Quantitative and Qualitative Research

• An epistemological version: as in the embedded methods argument and the paradigm argument, multi-strategy research is not possible

•A technical version: Emphasises the strengths of data collection and

data analysis techniques associated with quantitative and qualitative research and sees these as capable of being fused

Research methods are perceived as autonomous and compatible

11

Your view:

• Think about ways in which multi-strategy research can take place

• In particular, think about how and why

• The ‘why’ question compels us to provide a rationale for the choices we make if and

when we choose to combine methods

12

Subjectivist Projection of human imagination

Social construction Contextual field of information

Concrete process Concrete structure

Objectivist

Different assumptions about reality from Morgan and Smircich (1980).  Quantitative vs Qualititive Researcher is independent vs Researcher is involved Large samples vs Small samples Testing theories vs Generating theories Experimental design vs Fieldwork methods Verification vs Falsification  The choices of research design (Hassard, 2002).

13

Theory

Empirical

Generalization

Deduction, induction and Retroduction representation of how deduction, induction and Retroduction work through the empirics and theory (Alvesson and Skoldberg, 1994).

Empirical

Deduction Induction Retroduction

Deduction Logical inferences from major and minor premises

Formally correct but sometimes empirically flawed- it depends on the correctness of premises

Induction The ability to form probability statements when all conditions being equal the higher past frequency the higher the probability

Inductive thinking is susceptible to habit and expectation

Abduction The act of seeing something anew to connect a case with an unexpected rule related to vague common sense

Abductive reasoning is Perceptual judgments and their clarity is questionable

Retroduction Aims to specify the necessary and sufficient causes and conditions to be produced or reproduced, for the phenomenon to come into existence.

The four types of scientific abstraction – logical reasoning (as in Bertilsson, 2004).

Ontology and epistemology

14

Triangulation – popular strategy for mixing methods

• It is based on the assumption that “any bias inherent in particular data sources, investigator, and method would be neutralised when used in conjunction with other data sources, investigators and methods”

(Creswell, 1994: 174)

• It assumes that data from different methods will corroborate one another, where the choice of methods is intended to investigate a single social phenomenon from different vantage points

(Denzin,1970; Brannen,2005)

15

Combining methods & data: many possible outcomes

“Data collected from different methods cannot be simply added together to produce a unitary or rounded reality” (Brannen 2005: 176)

Triangulation (see corroboration) is just one of many possible outcomes

1.Corroboration: The ‘same result’ are derived from both qualitative and quantitative methods2.Elaboration: The qualitative data analysis exemplifies how the quantitative findings apply in particular cases.3.Complementarity: The qualitative and quantitative results differ but together they generate insights. 4.Contradiction: Where qualitative data and quantitative findings conflict.

(Morgan, 1998, cited in Bryman, 2001; Hammersley, 1996)

16

Part 2: Types of Triangulation –strategy for mixing methods

• It is based on the assumption that “any bias inherent in particular data sources, investigator, and method would be neutralised when used in conjunction with other data sources, investigators and methods”

(Creswell, 1994: 174)

• It assumes that data from different methods will corroborate one another, where the choice of methods is intended to investigate a single social phenomenon from different vantage points

(Denzin, 1970; Brannen, 2005)

17

Types of Triangulation

• Methods TriangulationOne researcher using two or more research techniques (within and between QUAN-

QUAL techniques);

Two or more researchers using the same research technique;

Two or more researchers using two or more research techniques.

• Theoretical TriangulationLooking at the research situation from different theoretical perspectives.

• Data TriangulationCombining qualitative and quantitative data within the same method. The types of data these three methods generate are field notes, audio (and sometimes video) recordings, and transcripts.

18

Mixed Methods Strategies

• What is the implementation sequence of the QUAN and QUAL data collection in the proposed study?

• What priority will be given to the QUAN and QUAL data collection and analysis?

• At what stage in the research project will the QUAN and QUAL data and findings be integrated?

19

1. Implementation

• SequentialThe researcher collects both the QUAN and QUAL data in phases

• ConcurrentThe researcher collects both the QUAN and QUAL data at the same time

20

2. Priority

• Equal prioritySame weight is given to QUAN and QUAL

• Dominant and less-dominant priorityPriority for either QUAN or QUAL

21

3. Integration

• Integration of QUAN and QUAL data might occur at several stages in the process of research:Data collection

Data analysis

Interpretation

Or combination of places

22

Alternative Strategies and Visual Models

• Sequential Explanatory StrategyUse QUAL results to assist in explaining and interpreting the findings of a primarily

QUAN study.

Especially useful when unexpected results arise from QUAN study.

23

Alternative Strategies and Visual Models (cont.)

• Sequential Explanatory Design

Qual

QUAN

Data Collection

QUAN

Data Analysis

qual

Data Collection

qual

Data Analysis

Interpretation of Entire Analysis

QUAN

Use QUAN data and results in the interpretation of QUAL findings.

Focus of this model is on exploring a phenomenon.

24

Alternative Strategies and Visual Models (cont.)

• Sequential Exploratory Design

QUAL

quan

QUAL

Data Collection

QUAL

Data Analysis

quan

Data Collection

quan

Data Analysis

Interpretation of Entire Analysis

25

Alternative Strategies and Visual Models (cont.)

• Concurrent Triangulation Strategy

QUAN QUAL

QUAN

Data Collection

QUAN

Data Analysis

QUAL

Data Collection

QUAL

Data AnalysisData results compared

Selected as a model when the researcher uses two different methods in an attempt to confirm, cross-validate, or corroborate findings within a single study.

26

Alternative Strategies and Visual Models (cont.)

• Concurrent Nested Strategy

Qual Quan

QUAN QUAL

Analysis of Findings

Analysis of Findings

Often used to gain broader perspectives as a result of using the different methods

May be employed to study different groups or levels

27

Other qualitative strategies • Multiple case studies, quantification of qualitative data

• Within qualitative methods triangulation

1. Participant observation is appropriate for collecting data on naturally occurring behaviours in their usual contexts.

2. In-depth interviews are optimal for collecting data on individuals’ personal histories, perspectives, and experiences, particularly when sensitive topics are being explored.

3. Focus groups are effective in eliciting data on the cultural norms of a group and in generating broad overviews of issues of concern to the cultural groups or subgroups represented.

28

Activity : think of your own research

• What is my ontology

• What is my epistemology

• What are my methods

• What type of data am I going to use

• What kind of answer do I look for: what, how or why

29

Part 3 Example: qualitative multiple case studies

using Critical Realism

•My Research Question: Why and how projects succeed or otherwise

•Why and how needs in depth rich data

•Qualitative rich data usually are collected through qualitative interviews-focus groups etc and analysed within case studies

Problem: Generalizability is low in case studies Problem: I needed to generalize the causes of an extended observed phenomenon in quantitative research

30

Choices

•Wouldn’t it be good to generalize from many case studies like as if from a survey

Finding patterns of behaviour and explanation through many case studies strengthen the argument placed by evidence

Problem: Finding patterns in case studies? Horrific ! Blending ontologies and epistemologies – very tricky indeed !

31

Research design

Research Question: What is the effect of Policy on Project Management?

Method: Multiple, explanatory comparative cross cases

Instrumentation: Human In-depth, Semi-structured interviews

Purpose: Building model Understand social interpretations of a phenomenon and built a model to represent it.

Data: Subjective Data are perceptions of the people in the environment.

Orientation: Causality Find how the issues involved interact.

Focus: Holistic A total or complete picture is sought.

Reality: Dynamic Reality changes with changes in people’s perceptions.

Viewpoint: Insider Reality is what people perceive it to be

Results: Valid The focus is on design and procedures to gain "real," "rich," and "deep" data.

Analysis: Retroduction Deductively collecting inductively analyzing and abductively concluding and theorizing

32

Normative, Descriptive and Systems theories in Project Management

Stakeholderism vs boundary roles

Policy

Project Management

Operational Change

Public Policy Implementation Strategy public sector -Institutions

Managing projects in change – planned vs emergent approaches

Relationality

33

Core Ontological Assumptions Reality as a concrete process

Assumptions About Human Nature Man as an adaptor

Basic Epistemological Stance To study systems, process, change

Favored Metaphors Organism

Table 6.1: The ontological assumptions of CR (Downward, 2008: 314).

Real

Actual

Empirical

Contingent conditions

Triggers

Observation Experience

Intrinsic objects Mechanisms

Events and tendencies Patterns

may or may not fire

may or may not be observable

The ontology of CR (the author from Modell, 2005)

34

Real

Actual

Empirical

Contingent conditions

Instruments

Observation Experience of participants

Policy Mechanisms Programme actions and contexts

Project Management

35

Epistemology Basic ideas Role of theoryWhat is real is not given. The world has structure (there are levels of reality) and emergent structures. People’s involvement with structures is transformational.

Subject matter has to reflect both its meaningfulness to actors and their location in a given network of relationships and structures. Knowledge is dualistic.

Theory is a conjecture about the connectedness of events and the causal sequences produced by generative mechanisms.

Nature of explanation Method of study  Something is explained if it is allocated a place at the end of a causal sequence. There may be multiple causes of a single event, co-variation and feedback

The aim is to produce a good theory which accurately identifies causal mechanisms. The ways these work themselves out in given cases will be complicated. Multiple data is required

 Ackroyd (2004: 150-1) on the characteristics of CR

36

1. 17 multiple case studies 2. Could quantify results but didn’t 3. Instead I made a conceptual model based

on the common causal explanations directly within and across embedded case studies

4. I linked the data to the literatures using the framework of open systems

Mixing methods in a qualitatively driven way…

Jennifer Mason (2006)

38

Mixing methods in a qualitative driven way (Mason, 2006)

• Contributes to the debate of multi-strategy research by arguing “the value of mixed-methods approaches for researching questions about social experiences and lived realities” (Mason, 2006: 9)

• Makes a case for looking at mixed-methods as “multi-dimensional research that transcend or even subvert the so-called qualitative divide” (Mason, 2006: 9)

• “social experience and lived realities and multi-dimensional... our understandings are impoverished and may be inadequate if we view these phenomena only along a single dimension” (Mason, 2006: 10)

39

‘Meshing’ methods? (I)

• Mason (2006) proposes that instead of talking about ‘integrating’ methods & data, perhaps it is more useful to describe mixing methods as ‘linking’ or ‘meshing’

processes

• “But how can this be done without sinking into a relativist mire, where we have many different and fragmented descriptions of social experience, but no real explanation of anything? On the face of it, mixed-methods approaches are trapped between the devil and the deep blue sea”

(Mason, 2006: 20)

40

‘Meshing’ methods? (II)

•“I think the answer lies in how we construct our explanations and what we expect them to do. Explanations do not have to be internally consensual and neatly consistent to have meaning and to have the capacity to explain. Indeed, if the social world is multi-dimensional, then surely our explanations need to be likewise?” (Mason, 2006: 20)

41

‘Meshing’ methods? (III)

Essentially, Mason (2006) argues for going beyond divides, including qual-quant; micro-macro; global-local, socio-cultural-individual in order to acknowledge the multi-dimensionality of contexts & how they intersect to shape social experience & lived realities

42

Qualitative derived principles for mixing methods (Mason, 2006: 21-22) (I)

• Underpinned by a qualitative constructivist approach & the aim of understanding the how and why of social experience & lived realities

1. A questioning, reflexive, and non-accepting approach to research design & practice

2. Recognising the validity (legitimacy) of more than one approach

3. A flexible, creative approach

43

Qualitative derived principles for mixing methods (Mason, 2006: 21-22) (II)

4. Celebrating richness, depth, complexity, and nuance (through embracing a range of data types & sources, including ‘quantitative’ understandings)

5. A reflexive approach to what it is that data represent & how they constitute knowledge• this involves questions about the contexts/situatedness of the

social phenomena / processes being investigated & extent to which the methods used can provide knowledge about them

• E.g. Are they to be found in people’s behaviours, practices, imaginations, in physical or visual environments, in norms or discourses, etc.?

44

Questions?

45

Reading

Bryman and Bell (2007): Chapter 25: Mixed methods research: Combining quantitative and qualitative research

You might also want to look at:

Brannen, J. (2005). Mixing Methods: The Entry of Qualitative and Quantitative Approaches into the Research Process. Int. J. Social Research Methology, 8(3), 173-184.

Bryman, A. (2007). Barriers to Integrating Quantitative and Qualitative Research. Journal of Mixed Methods Research, 1(1), 8-22.

Mason, J. (2006). Mixing methods in a qualitatively driven way. Qualitative Research, 6(1), 9-25.

46

Other references

• Campbell, D.T. and Fiske, D.W. (1959): “Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix”, Psychological Bulletin, iss.2, pp.81-105.

• Cherryholmes, C.H. (1993): “Notes on Pragmatism and Scientific Realism”, Educational Researcher, vol.21, iss.6, pp.13-17.

• Creswell, J.W. (1994): Research Design: Qualitative & Quantitative Approaches, (Sage Publications: U.S.A.).

• Creswell, J.W. (2003): Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 2nd ed., (Sage: U.K.).

• Datta, L. (1994): “Paradigm Wars: A Basis for Peaceful Coexistence and Beyond”, in Reichardt, C.S. and Rallis, S.F. (eds.): The Qualitative-Quantitative Debate: New Perspectives, pp.53-70, (Jossey-Bass: San Francisco).

47

...... references ....... • Denzin, N.K. (1978): The Research Act: A Theoretical Introduction to Sociological

Methods, (McGraw-Hill: New York).

• Fielding, N.G. and Fielding, J.L. (1986): Linking Data, (Sage Publications Inc.: London).

• Flick, U. (1991): “Triangulation”, in Flick, U., Kardoff, E., Keupp, H., Rosenstiel, L., and Wolff, S. (eds.): Handbuch Qualitative Sozialforschung, pp.432-434, (Psychologie Verlags Union: Munich).

• Flick, U. (1992): “Triangulation Revisited: Strategy of Validation or Alternative?”, Journal of Theory of Social Behaviour, iss.2, pp.175-197.

• Flick, U. (1998): An Introduction to Qualitative Research, (Sage: U.S.A.).

• Silverman, D. (2000): Doing Qualitative Research: A Practical Handbook, (Sage Publications Inc.: London).

• Tashakkori, A. and Teddlie, C. (1998): Mixed Methodology: Combining Qualitative and Quantitative Approaches, Applied Social Research Methods Series, vol.46, (Sage Publications, London).

• Webb, E.J., Campbell, D.T., Schwartz, R.D., and Sechrest, L. (1966): Unobtrusive Measures: Nonreactive Research in the Social Sciences, (Rand McNally: Chicago).

48

...... references ..........

• Greene, J.C., Caracelli, V.J., and Graham, W.F. (1989): “Toward a Conceptual Framework for Mixed-Method Evaluation Designs”, Educational Evaluation and Policy Analysis, vol.11, iss.3, pp.255-274.

• Jick, T.D. (1979): “Mixing Qualitative and Quantitative Methods: Triangulation in Action”, Administrative Science Quarterly, vol.24, iss.4, pp.602-611.

• Johnson, R.B. and Onwuegbuzie, A.J. (2004): “Mixed Methods Research: A Research Paradigm Whose Time Has Come”, Educational Researcher, vol.33, iss.7, pp.14-26.

• Lamnek, S. (1995): Qualitative Sozialforschung, Band 1: Methodologie, (Psychologie Verlags Union: Germany).

• Murphy, J.P. (1990): Pragmatism: From Peirce to Davidson, (Westview Press: Oxford).