Mixed Methods Research

Peter Ratcliffe

Lecture outline

• Differences between qualitative and quantitative methods: a review

• Breaking down the quantitative/qualitative divide

• Mixed methods: rationales and practices• Mixed methods: research examples (Peter

Ratcliffe)• Concluding thoughts

Differences between qualitative and quantitative research strategies

Quantitative QualitativePrincipal orientation to the role of theory in relation to research

Deductive; testing of theory

Inductive; generation of theory

Epistemological orientation

Natural science model; in particular positivism

Interpretivism

Ontological orientation

Objectivism Constructionism

Bryman, 2008, Social Research Methods, p. 22

Differences between qualitative and quantitative research methods

Quantitative methods are generally seen as ideally suited to:•Questions to which responses are potentially measurable, i.e. quantifiable e.g. ‘How often do you make use of service x? •Questions where only absolute guarantees of anonymity would persuade respondents to answer honestly e.g. questions about income, illegal/’deviant’ activities, voting intentions, etc. •When you want to be able to say something about ‘the whole population’ (generalisability, statistical inference, etc.)

Qualitative methods are seen as preferable when:•You are interpreting views, opinions, ideas•Context is important•Research is with vulnerable or ‘hard to reach’ groups•Research is into complex or dynamic social processes

Breaking down the quantitative/qualitative divide

• It is important not to exaggerate differences between quantitative and qualitative research.

• Qualitative research sometimes exhibits features that are normally associated with positivism:• e.g. Adler and Adler 1987: qualitative study that explored

whether participation in athletics in higher education in the USA is associated with higher or lower levels of academic achievement (i.e. tested hypothesis; ‘objectivist’ overtones)

• Quantitative research sometimes engages with an interpretivist stance:– e.g. Westergaard et al 1989: quantitative study that explored how

people who were made redundant responded to their experience in terms of their job-search methods, their inclination to find jobs, and their political attitudes (i.e. interpretivist overtones)

Mixing our methods and epistemology

• ‘Mixed methods’ (or ‘multi-method’): generally refers to research strategies that mix qualitative and quantitative methods (although some may use the term more loosely to refer to other ‘mixtures’ of methods).

• Epistemological and methodological debates are routinely set up around the discussion of supposedly mutually exclusive (quantitative and qualitative) approaches.

• The argument against mixed methods tends to be based on one or both of two kinds of argument:– the idea that research methods carry epistemological

commitments, and– the idea that quantitative and qualitative research are separate

paradigms (i.e. incommensurable, or incompatible, in epistemological terms).

Arguments for mixed methods• The connections between epistemology and ontology and research

methods are not deterministic. • Research methods as techniques of data collection or analysis can

be viewed from a pragmatic epistemological point of view, concerned with what is useful for research purposes rather than the nature of reality.

• Strong objections to quantitative research amongst feminist researchers (e.g. the silencing of women’s voices, controlling variables as a ‘masculine’ approach) have softened in recent years (e.g. recognizing that quantitative methods can be useful for highlighting statistics of gender discrimination ).

• BUT: mixed-method research is not intrinsically superior to mono-method research: like any other method it must be appropriate to the research questions, context and design.

Mixed methods: approaches

Different methods can be used to good effect either in parallel or sequentially and either discretely or in an integrated fashion. •Triangulation (also known as convergent findings)

– use quantitative and qualitative methods to address the same research question, to ensure greater certainty of results

•Additional coverage– Assigns different methods to different components of a project

on the basis of their particular strengths/relevance – Example: parallel use of methods for discrete purposes

•Complementary assistance– Link methods together so that one method helps the other– Example: sequential use of methods for pilot studies

Triangulation of data • Triangulation is a process of checking the inferences

drawn from one set of data sources by collecting data from others.

• Within an ethnographic approach you might want to check your own observations of a situation/event against how it is described to you in an interview.

• Alternatively you might use a diary entry or a set of family photos to confirm or contest something said in conversation.

• This process of triangulation in ethnographic work is a way of improving validity [countering the view that the researcher gets a very ‘partial’ understanding of the social world]. But this comes about because it allows you to first capture differences and contradictions and work through them to try and understand the whole.

Additional coverage with parallel use of methods

• Triangulation is an example of the parallel use of methods as a check on inference from those data.

• However, methods can be used discretely in parallel in order to provide a different kind of data, answering a different question, but helpful to the overall research

• It is common for qualitative researchers to employ a small-scale survey, for example, in order to generate descriptive [though not inferential] statistics.

• The dangers in this kind of parallel work is that the data from

different sources may simply present different types of information that cannot be easily integrated and thus produce no discernable analytic gain.

Complementary assistance with mixed methods applied sequentially

• An alternative approach is to use different methods sequentially.

• The most common example of this is the recognition by survey researchers that the formulation of questions for questionnaires is benefited by using qualitative research methods – either interview or focus groups – to understand how respondents are likely to interpret, and respond to, questions. The aim of such mixing of methods is to reduce misreporting and cross-cultural confusion and thus the qualitative element is designed to improve the main technique which is quantitative.

• Although less common, qualitative specialists may also use quantitative work to identify interesting cases for study, or else to test statistically hypotheses generated by ethnographic work.

• Finally, remember the potential gains from pursuing the secondary analysis of existing survey, and official statistical, data

Research Examples

Selected to illustrate:

• Theoretical/methodological issues – what’s the problem? Epistemological issues….

• Relationship between different components – horses for courses…

• Role of sociologist (ethical/political concerns, potential of ‘public sociology’)

CASE 1: ‘Race’ and Housing in Bradford (1996)

• Background (severe housing need, esp. in certain areas/communities, fiscal constraint, investment required making case to govt.)

• Principal method – major survey (Why? Why not ethnographic?)

• Ethical considerations (Pathologisation v. potential gain, role of sociologist)

• Sampling issues (use of NAMPECHAN)

Key concept – housing need

Major issue: ‘housing need’ is multi-dimensional (N.B. process: theory/concepts/indicators)

• current needs (quality, space, social/physical environment....)

• special needs (‘cultural’, health/disability....)• future needs (spatial and social mobility,

fertility/fecundity, household transformation processes, net in-migration.......)

• Different dimensions require different methods…

Mixed methods

• Current/special needs – survey and observational data complemented by official statistics (Census, LA databases, morbidity/mortality data, unemployment levels, deprivation indices…..)

• Future needs – more complex…why?– BHF as focus group – social change processes– Key informant interviews – semi-structured– Demographic data – in-migration, spatial mobility

CASE 2: Black Barrister Study (1993)

Very different research model… highlights the role of sociologist….

• Background (Judicial Review – High Court, backed by Society of Black Lawyers)

• Substantive issues – institutional discrimination on basis of ethnicity, gender (+ class?)

• Sociologist as expert advisor/advocate• Used all available data to make the strongest case

(various documents – records/reports/ quantitative data, qualitative interviews)

Mixed methods: complementarity

• Qualitative interviews with complainants – subjective assessment of discriminatory processes

• Statistical analysis of marks (ethnicity/gender)• Investigation of assessment procedures – discourse

analysis of documents - subjective judgements (reports), single marking, minutes of Exam Boards……

• Report interrogated by Lincoln’s Inn QCs…….. (multidisciplinary - socio-legal - research)

• Outcomes (role of ‘public sociologist’ – ramifications for CLE + led to a public inquiry chaired by the black QC, Dame Jocelyn Barrow)

CASE 3: ‘Community Cohesion’ research (2006)

• Background – urban unrest in northern English towns and cities in summer 2001

• National policy agenda ensued - based on concept of ‘community cohesion’

• Invoked complex theoretical arguments - ‘parallel lives’, self-segregation’, relevance of social capital theory…….

• Aims of research – evaluation of policy agenda (with major theoretical implications)

Mixed methods: complex interplay

Major four town study over five years - multi-method approach with multidisciplinary team

• Key informant interviews (sequential/repeated – qualitative focus)

• Focus groups – based on age/gender/ethnicity• Discourse analysis of policy documents, reports..• Official statistical data – demography, spatial

patterns, deprivation indices

Outcomes: theoretical insights and policy development

• Critique of ‘community cohesion’, ‘parallel lives’ and the ‘self-segregation’ thesis

• Cast light on interpretations of ‘Britishness’, ‘British values’ and of inclusive/exclusionary notions of identity

• Highlighted the lack of focus on class and on the role of the Political Right

• Influenced the agenda of the Commission on Integration and Cohesion (2007)

CASE 4: Promoting Equality through Procurement (1991-)

Background – complex and convoluted process!

• Thatcher’s role in undermining equality agenda – demise of LA ‘race’ and gender units

• Local Government Act 1988 – significance of CCT• Resulting discussions with CRE starting in 1991• Emergence of WMF – significance?• Evaluation research (2001-3)

Mixed Methods Action Research

STAGES:• Participatory action research with WMF• Policy staff as researchers• Generation of pre-qualification questionnaire

– data generation as part of change process• Evaluation Research - submitted pre-

qualification questionnaire complemented by qualitative interview with key informants

Outcomes: theory and policy• Theoretical insights – nature of institutional

discrimination and the dialectical nature of social change processes.

• Policy impacts:–Dissemination of Forum model regionally–CRE review of Procurement Policies – support

for Common Standard for (Racial) Equality – adopted(?) by Government–Dedicated national body (ESPG) promoting idea,

backed by EHRC + now disseminated across EU

Conclusions• Epistemological divides in the social sciences are weakening, but

there remains some scepticism about mixing methods.

• We have suggested that there are no absolute divisions between deductive and inductive approaches or between statistical/survey [quantitative] and interview/observation based [qualitative] methodological techniques.

• These approaches and methods should be viewed as complementary rather than mutually exclusive and we should focus on selecting the most appropriate methods for a particular question and considering how different methods might be integrated (or ‘triangulated’) in the course of investigation.

• We should also pay attention to the different ways in which methods can be used discretely or in an integrated fashion in order to maximise the benefits of a mixed method approach.