Leverhulme 1

Overview of different Overview of different Qualitative Analysis Qualitative Analysis

MethodsMethods

Dr. Anne Adamsanne.adams@open.ac.uk

Leverhulme 2

OVERVIEWOVERVIEW• Paradigm background (research onion)• Analysis Methods

– Content Analysis – Discourse Analysis– Thematic Analysis– Grounded Theory– Phenomenological Analysis– Research Quality and Action Research

Leverhulme 3

RESEARCH PARADIGM

Leverhulme 4

Analysis paradigm

• Crotty (1998) rigorous = identify / defend research strategy decisions

• Adams et al (2008) rigour from appropriate identification and application of methods according to research questions

• Reflexivity essential to maintain research rigour

Qualititative methodsQualititative methods

• In-depth Interviews,

• Focus groups,

• Observational / ethnographic studies,

• Open ended data (system logs)

• Data transcribed

QUANT / QUAL ComparisonQuantitative approaches Qualitative approaches

'Simple' numeric data 'Complex' rich dataMeasurement Meaning

Explanation UnderstandingPrediction Interpretation

Generalisable account Contextual accountRepresentative population sample Purposive/ representative

perspective sampleHypothesis-testing ExploratoryClaims objectivity Accepts subjectivity

Closed system (experimental control)

Open system(ecological validity)

Qualitative / Quantitative DivideQualitative / Quantitative Divide

Research is like fishing

Quantitative methods

• You find the best river for the fish you want, you have one line, a specific bait for a specific type of fish.

Qualitative methods • You may want to catch tuna so you fish in certain parts of the sea BUT on the whole you throw your nets out to sea and catch everything including the things you want and don’t want.

Qualitative Analysis MethodsQualitative Analysis Methods

• Conversational / Discourse Analysis• Thematic Analysis / Grounded Theory • Content Analysis / Critical Incident Analysis

• counting• imposing established frameworks

“Both qualitative and quantitative approaches share a common concern with theory as the goal of research” (Henwood & Pidgeon, 1992 p.101)

Leverhulme 9

Quantitative methodsQuantitative methods

• Experiments

• Questionnaires

• System logs (producing data for statistical analysis)

Leverhulme 10

Research Paradigms:Research Paradigms:QuantitativeQuantitative

• Imposes prior theories (discovery)

• Reliant on hypothetico-deductive method to establish causal relationship (justification)o Operationalised (reductionistic)

o Measured

Leverhulme 11

Research Paradigms:Research Paradigms:Quantitative LimitationsQuantitative Limitations

• Validityo Inappropriate fixing of meaningso Imposing external system of meaning

for internal subjective structures• Complexity of data lost through

reductionistic approach

Leverhulme 12

Qualititative methodsQualititative methods

• In-depth Interviews

• Focus groups

• Observational / studies

• Open ended data

Leverhulme 13

Research Paradigms:Research Paradigms:QualitativeQualitative

• Generates working hypothesis by producing concepts from data

• Represents participants reality in its complex context

Leverhulme 14

Research Paradigms:Research Paradigms:Qualitative LimitationsQualitative Limitations

• Subjectivityo Data collection procedures

o Analysis

• Reliabilityo Across context & researcher

Leverhulme 15

The research onion: TEL / HCI

Leverhulme 16

TEL example : Juxtalearn

• An Educational design project in field of TEL • HCI perspective connects computing and education • ‘in the wild’ practice based perspective to TEL see Adams et al

(2013, 2014) expanded in Fitgerald et al (2015)• Research onion simplified - Four main research epistemologies

& related methods: overlay of TEL & HCI• Juxtalearn inspired by action research and design-based

approaches from a ‘pragmatic’ perspective. • But used both qualitative and quantitative analysis methods to

mixed method data collection. Grounded Theory Analysis.

Leverhulme 17

Content Analysis

• Quantitative analysis of qualitative data

• Counting instances

• Either using pre-existing coding framework or creating one (problems with both).

• Careful to apply this it needs linked to other analysis methods (neither quant nor qual)

Leverhulme 18

Content Analysis Coding

• Take data• Categorise data – often with pre-determined

models, frameworks and coding schemes• Code the data according to category• Count the data – instances of the data

Leverhulme 19

Example of coding Units

Unit ExamplesWord social engagement wordsTheme occasions of social engagementItem whole accounts of SECharacter different characters roles in SETime & Space space & time-logs in blog spent

on SE activities

Leverhulme 20

Discourse Analysis• Very different from other forms of QA• In pure form: based on meaning and language

used – for that ‘discourse’• From a Radical constructivist approach (e.g.

discourse action model)• Memories not facts but socially

reconstructions• Pure: naturally occurring talk• Need to fully understand philosophy to use it

Leverhulme 21

Discourse Analysis Coding

• TEL: specific impact of technology on the discourse e.g. virtual reality impact on learning speech

• Code every utterance, gaps, breaths• Identify language usage: plays, stock

phrases, how language is used• Not understanding of the concepts generally

Leverhulme 22

Sections of code4. [J.H.] you are black or Asian you cannot join (.) this national (.) political party5. (.) slightly different I think from the metropolitan police black6. Officers association wouldn’t you say?7. (.)8. [N.G.] I don’t think it’s different in the slightest because as the simple fact9. is as I say that white police officers couldn’t set up an association10. Of their own er if for instance they feel they’re being Overlooked for11. ap-prom for promotion so every different ethic group er in 12. this (.) multicultural multiethic society that our

Coolican, 2014

Leverhulme 23

Thematic Analysis

• Data gathering and analysis approach• Understanding drawn from the data• Recognise limitations but not based on

specific epistemology (many data types)• Can be used to provide generic

understanding but limited application to theory building

Leverhulme 24

Thematic Analysis Coding

• 1- familiarise with data• 2 – generatie initial codes• 3 – search for themes• 4 – review themes • 5 – define and name themes

Leverhulme 25

Example of Blog code

Coolican, 2014

Leverhulme 26

Grounded Theory

• Analysis grounded in the data• Mixed methods data – quant and qual• 1st stages similar to TA• Systematic merging of mixed data • Development of a ‘theory’ (also models)• Process pushes analysis of rouge / awkward

data to reveal ‘novel findings’

Leverhulme 27

Grounded Theory Coding

• Open coding – line by line (as with TA)• Memo writing – capturing insights for

theory• Axial coding – linkages between categories• Selective Coding – important themes ‘high-

level storyline. • Process effects – conditions, actins, effects

effect relationships in a process

Leverhulme 28

Example process effect model

4 1

3

IS

TRUST privacy secured

based on assumptions

Users

IR

Contexts

IU

Technology & its implementation make assumptions inaccurate

Increased perceived privacy invasions

2

Decreased trust in organisationEmotive reaction Reject technology

Leverhulme 29

Phenomenological AnalysisInterpretive Phenomenological Analysis (IPA)

• The individuals ‘experience’ events• Individuals perspective on and understanding

in the world. • Understands constructivist role of researcher

in interpreting individual experience.• Unlike DA takes what people say as realistic

account (semi-structured interviews)

Leverhulme 30

IPA coding

• Read text ‘eyeball it’ • Comments on transcripts (varied in size and

length)• Interpretations / analysists reactions /

recurrent points / queries about meaning• Identify themes (hierarchical themes)• Compare and contrast themes

Good Quality Research

• Henwood / Pidgeon (1992)– good quality research

• 7 golden rules of good quality research

• Exploratory (discovery) – reductionistic (justification)

• Not Divide but to compliment

• ISSUES OF BIAS

Levels in research

PLAN: Background research & planning

ACT and /or OBSERVE

REFLECT: Implications / Discussion

Initial QUESTION?

Action Research & Levels

PLAN: Background research & planning

ACT and /or OBSERVE

REFLECT: Implications / Discussion

Initial QUESTION?

PLANNING courses, teaching activities

REFLECT on Changes to current practices

CONDUCTING EVERYDAY ACTIONS (E.G. Teaching)

Action Research & Exploratory Research

PLAN

REFLECT ACTACT

OBSERVE

What is my question?

Action Research (2)

Plan

Reflect Act

Observe

Initial Question

Plan

Reflect Act

Observe

Further Question

Leverhulme 36

References

Adams, A., FitzGerald, E. and Priestnall, G. (2013). Of catwalk technologies and boundary creatures. ACM Transactions of Computer-Human Interaction, 20(3), article no. 15. download pre-print from: http://oro.open.ac.uk/35323/ Adams, Anne and Cox, Anna L. (2008). Questionnaires, in-depth interviews and focus groups. In: Cairns, Paul and Cox, Anna L. eds. Research Methods for Human Computer Interaction. Cambridge, UK: Cambridge University Press, pp. 17–34. download from : http://oro.open.ac.uk/11909/Adams, Anne; Lunt, Peter and Cairns, Paul (2008). A qualititative approach to HCI research. In: Cairns, Paul and Cox, Anna eds. Research Methods for Human-Computer Interaction. Cambridge, UK: Cambridge University Press, pp. 138–157. download from : http://oro.open.ac.uk/11911/ Coolican, H (2014) Research methods and statistics in psychology. Psychology press, NY. Crotty, M.J. (1998) The Foundations of Social Research: Meaning and Perspective in the Research Process. Thousand Oaks, CA: Sage.FitzGerald, E. and Adams, A. (2015). Revolutionary and evolutionary technology design processes in location-based interactions. International Journal of Mobile Human Computer Interaction, 7(1) pp. 59–78. download from : http://oro.open.ac.uk/42067/ Henwood, K. L., & Pidgeon, N. F. (1992): “Qualitative research and psychological theorising.” British Journal of Psychology. 83, 1, pp. 97-111.Saunders, M., Lewis, P. and Thornhill, A. (2012) Research Methods for Business Students, 6th edition. Harlow: Pearson Education Ltd