Finding and communicating the story in qualitative information - Lesson 2

Finding and Communica-ng the Story – Lesson 2 of 6 – Qualita-ve Informa-on Ray Poynter, 2016

Finding and Communica-ng the Story

Lesson 2 of 6

Working with Qualita-ve Informa-on

Ray Poynter

April 2016


Series Schedule

•  An Introduc5on and Overview -‐ Feb 23 •  Working with Qualita-ve Informa-on – Apr 5

•  Working with Quan5ta5ve Informa5on -‐ May 26

•  Working with mul5ple streams & big data -‐ July 5

•  U5lizing visualiza5on – Sep 13 •  Presen5ng the story -‐ Nov 8


Agenda

•  Overview of the Frameworks approach

•  Qualita5ve informa5on

•  Qualita5ve analysis •  Finding the story in qualita5ve informa5on

•  Communica5ng qualita5ve messages


The Frameworks Approach

1.  Define and frame the problem 2.  Establish what is already known –  And, what is believed/expected

3.  Organise the data to be analysed 4.  Apply systema5c analysis processes 5.  Extract and create the story


Who is the project for? _________________

What is the business issue/problem that is being addressed? __________________________________________________

What does the business want to do, once it has addressed this issue? ______________________________________________________

What do we already know? Item Held by: Descrip-on

1  ______ ______ ______________ 2  ______ ______ ______________ 3  ______ ______ ______________

Assump-ons and predic-ons Who What

1.  ______ ______ 2.  ______ ______

Simplified


What is Qualita-ve? No single, perfect descrip5on

–  Defini5ons o\en a ma]er of degree

•  Qual includes human judgements as part of the analysis –  Quant is algorithmic, removing or minimising the human role

•  Qual is about meaning and understanding –  Quant is about quan5fica5on

•  Qual deals with all sorts of informa5on, including unstructured –  Quant requires the data to become structured/opera5onalised

•  Qual looks at within case informa5on (≈ lots of informa5on about a few people) –  Quant looks at across cases informa5on (≈ small amount of informa5on about

lots of people)


What is Qualita-ve? Which is the best door for our building?

Focus Group or IDIs Determine A is preferred by le\-‐handed people, and B by right-‐handed people. Perhaps find out that one group is more insistent than the other -‐ Qual

A B Ethnographical approach Watch people tackling a variety of doors, plus other objects. Determine people who tend to favour their le\ prefer A and visa versa -‐ Qual

Usability Professional Assesses the op5ons based on experience and criteria -‐ Qual Or, apply a fixed scoring system -‐ Quant

Survey People Discover 90% prefer B – Quant Or, include le\/right handed variable, find right-‐handed people prefer B – Quant Or, include open-‐ended ques5on on why, some people cite handedness – Quant with some Qual

Picking the best door? Qual


Quant starts as Qual A.  How many drinks did you have today? – What is a drink? 2 sips from a bo]le versus 2 sips from a fountain? 2 separate glasses of wine versus a glass of wine that was topped up?

B.  Agree Strong, Agree, Neither Agree Nor Disagree, Disagree, Disagree Strongly? –  In the mind of the par5cipant there are no numbers, they pick an answer which they believe best reflects their view


Opera-onalizing From Qual to Quant

Qual is analysed by a human*, quant employs an algorithm

If we code qual data and count the codes, we convert from qual to quant, via opera5onalizing –  Brand men5ons –  Likes and Dislikes –  Sen5ment – Marking an essay –  Evalua5ng people for mental health disorders Tendency to treat this quant as ‘hard’ data, and the underlying qual as ‘so\’


Computers ＆ Qualita-ve Analysis •  Scissors & coloured pens è Word, Excel etc

•  CAQDAS – Computer Aided Qualita5ve Data Analysis So\ware, e.g. Nvivo

•  Text analy5cs, from word clouds to Leximancer

•  Social Media analysis, e.g. Brandwatch & Radian 6

•  Coding so\ware, e.g. Ascribe

•  Photos and Video organising, e.g. Google Photos and Living Lens

Your organisa5on’s Framework should specify the tools to be used, storage protocols, and approaches to things like memos, tags, and notes.


AI and Qual

At some point in the future, and maybe somewhere in the world today, it might be possible for qual data to be analysed by AI instead of, or as well as, humans.


Organising Exis-ng Knowledge

•  Include qual and quant knowledge •  Stakeholders summarise what is known and what they think the research will show

•  Make the data* accessible – Transcripts, transla5ons, video libraries, photo galleries

– Consider computer tools like NVivo


Qualita-ve Data? •  Notes created by researchers

when observing, listening, discussing with par5cipants

•  Open-‐ended comments in interviews, focus groups, surveys etc

•  Posts in Social Media

•  Le]ers

•  Videos, recordings, transcripts

•  Art

•  Meals, clothes, trash

•  Theatre, cinema

•  Play, ac5vi5es, interac5ons

•  Objects

•  Photographs & recordings

•  Observa5on & passive data

Many of these can also be called artefacts (ar5facts in North America)


Symbiosis of Collec-on and Analysis

Establish the Ques5on and what is Known, Plan Research

Do Research

Analyse Update plan

Analyse Story


Academic versus Commercial Analysis of Qualita-ve Data

Many techniques are used by both, e.g. conversa5on analysis, grounded theory, etc

But! –  Timelines vary, commercial one day to one week, academic can be months

–  Success can vary, commercial = be]er business decision, academic = advancing knowledge (academic defini5on of knowledge)

–  Purity of methodology, academic more pure, commercial more pragma5c (which o\en means using hybrids)


Common Analy-cal Approaches •  Grounded Theory – created by Glaser & Strauss in the 1960s adopts

a formal approach to coding the data, linking the codes into concepts, linking these into categories, and crea5ng an overarching structure. Tends to require plenty of 5me. Tries to ignore exis5ng theories – increasing sensi5vity to the content of the data. Induc5ve approach, general theories from specific observa5ons.

•  Abduc-ve Analysis – compares the data with the theories and expecta5ons, iden5fy the non-‐expected and leap (abduct) from these observa5ons to a new theory that is sufficient and probably correct/plausible.

•  Content Analysis – is popular both with tradi5onal researchers and those seeking to computerise some or all of qualita5ve analysis. As with other approaches, the data is coded and categorised, but in content analysis the frequency of codes and categories and the frequency of links between them is taken into greater account that with most other methods. The use of ‘’coun5ng’ increases the importance of sampling when using content analysis.


Common Analy-cal Approaches •  Narra-ve Analysis – focuses on the en5re text, not subdivided components.

Enter the text (coding/memoing), interpre5ng, verifying (e.g. alterna5ve explana5ons), represen5ng (write the plot of the story), illustra5ng (e.g. finding quotes, drawing diagrams).

•  Conversa-on Analysis – is one form of Discourse Analysis, CA, Conversa5on Analysis, was developed from the work of Harvey Sacks’ work in the 1960s & 1970s. CA looks at how people speak, the pa]erns they use, how they create meaning, for example: turn-‐taking, repairs, dispreferred responses. Conversa5on analysis pays less a]en5on to what people say than the way they say it.

•  Thema-c Analysis – the focus is to generate themes from the data. In par5cular pa]erns (e.g. codes and categories) are iden5fied in the early data (e.g. the first interviews or focus groups) and then used as tools to analyse subsequent data. One difference between thema5c and grounded theories is that grounded theory seeks to create a broader theory, thema5c analysis tends to be happy to create a narra5ve to explain the data.


Semio-cs Semio-cs was developed from the work of Ferdinand de Saussure from the later 19thCentury onwards. Semio5cs is the study of meaning-‐making by looking at the use of signs and symbols (which can be any form of data, including worlds, brands, images, sounds etc.) Semio5cs does not require the collec5on of data from research par5cipants; semio5cs if frequently conducted with artefacts that exist in the ‘real world’ rather than in an MR created world. However, semio5cs can be applied to MR data, just as it can be applied to any other data.

Sign

Signified

Signifier

Sign

Rose

Sign

Passion

Rose

Sign

Passion


Overarching Structure No uniform

No books

Travel costs

School fees

Worry

Mind elsewhere

Tired in School

Headaches

Lack school materials

Unable to pay school costs

Worry about dependents

Feeling exhausted

Physically & emo5onally stressed

Can’t afford school

These children have tangible problems

Adapted from www.open.edu/openlearnworks/mod/resource/view.php?id=52658


Deciding What to Believe and What to Interpret

Less believable –  Yes, I always give my

children healthy snacks –  Yes, I will buy this new

product –  I always remember to take

my medicine –  I buy on value, not

because of the adver5sing

More believable –  I have two children –  No, I did not like it –  I think men will like this

more than women –  Which of these three is

the odd one out? –  Why is it the odd one out?


Popular Internet meme


Why ‘Just Say No!’ is Not so Easy

Just Say No? The Use of Conversa5on Analysis in Developing a Feminist Perspec5ve on Sexual Refusal, Celia Kitzinger, 1999


Common Analy-cal Elements

•  Saturated analysis – keep going un5l you stop finding new/useful things

•  Structure – find/create an architecture to what you find

•  Make notes of what you find, linking back to the data, highligh5ng examples

•  Look to support AND break hypotheses

Conversa5on Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either preven5ve materials or restora5ve materials; what do you take in account when you decide which one to adopt? Well, that's a good ques5on. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own li]le research about that, because I don't really trust the research that comes with the product and once again what other den5sts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products.

Conversa-on Analysis Pauses/Repairs/Disconnects: Person is portraying that they are not confident. Restructured answer “Well, that’s a good ques5on.” – Indicates the ques5on was not a good ques5on, deals with it by saying ‘Don’t know’ and then proceeds to answer what he/she thinks the ques5oner is hoping to learn.

From an example of Grounded Theory www.biomedcentral.com/imedia/4037816045634649/supp3.doc

Discourse Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either preven5ve materials or restora5ve materials; what do you take in account when you decide which one to adopt? Well, that's a good ques5on. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own li]le research about that, because I don't really trust the research that comes with the product and once again what other den5sts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products.

DA -‐ Foo-ng The role the den5st is filling? Somebody who is not confident, and who is doub}ul about the sources available to him/her.


DA – Repe--on Reliability & “Internet sites” No repe55on of cost. Cost is a ‘preferred response’ – it is used and discarded.


DA – Evalua-ve terms I quite like my own li]le research I don’t really trust the research that comes with the product

Some of those internet forums are actually quite good for new products

DA Thoughts Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either preven5ve materials or restora5ve materials; what do you take in account when you decide which one to adopt? Well, that's a good ques5on. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own li]le research about that, because I don't really trust the research that comes with the product and once again what other den5sts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products.

The story? The den5st lacks confidence, he/she men5ons cost, but comes back to the topic of reliability.

He/she distrusts the research from the manufacturers, so tries to do his/her own research, by connec5ng with people who have used the new products, via internet forums

Sales Recommenda-on Connect this type of den5st with happy users. Encourage reliability tes5monials and SM posts.


Word Clouds?

A weak form of qualita5ve analysis Can be an entry point, some5mes Can be useful in communica5ng the story


Finding the Story •  Use the client’s ques5on as the lens •  Tag, code, memo the material as you analyse

•  Challenge what is known/believed •  Find the main story •  Find the relevant excep5ons/differences •  Create an overall structure, the plot •  Is it good news or bad news?


Finding the Story

•  Use the client’s ques5on as the lens – What does success look like? – What ac5ons are pending on the results? – What do people think is true? – What do people think the results will be?


Good and Bad News •  There are four typical stories – Good news – Good news with caveats –  Bad news with some op5ons –  Bad news

•  The storytelling for these four cases is different •  Good news and bad news is defined by what the client wanted AND what the research finds


Bad News •  5 stages of grief –  Anger, Denial, Bargaining, Depression, Acceptance

•  One presenta5on/report rarely tackles all the stages of bad news

•  ‘Facts’ are rarely enough to persuade –  Emo5ons are the key – a customer video can be more powerful than any amount of analysis

•  Go back to a point where the expecta5ons match the findings and build from there


Conveying Confidence •  Confidence is created by the researcher •  Don’t convey more confidence than you have – Don’t convey less confidence

•  U5lise –  Triangula5on –  Testable predic5ons –  Consistency –  Coherence


Case Study

Calvin Klein, semio5cs study by Semio5cs Analysis

The problem – 1980s success Obsession – 1990s success Eternity – 2000s failure e.g. Truth – Why and what should CK do next?

RW Connect, Greg Rowland, 2014 h]ps://rwconnect.esomar.org/semio5cs-‐the-‐billion-‐dollar-‐case-‐study/


Case Study The story – CK success based on codes of modernism – CK failure linked to using industry codes – Use modernism

Good news? Bad news? – Depends on what CK believed –  If they wanted modernism, simply urge them forward –  If they liked the new codes, take them back to success and build the story from there


Case Study

1980s ✔

1990s ✔

2000s ✗

$Billions ✔


The Big Picture •  Frameworks for reliable / effec5ve stories •  Define the problem

•  Organise the data according to the Framework – everybody using the same tools and approaches

•  Find the main story and build out from there

•  Is it good or bad news, confirming or challenging expecta5ons/beliefs

•  Engaging, memorable, simple story


Schedule

•  An Introduc5on and Overview -‐ Feb 23 •  Working with Qualita-ve Informa-on – Apr 5

•  Working with Quan5ta5ve Informa5on -‐ May 26

•  Working with mul5ple streams & big data -‐ July 5

•  U5lizing visualiza5on – Sep 13 •  Presen5ng the story -‐ Nov 8


Thank You!

Follow me on Twiber @RayPoynter

Or sign-‐up to receive our weekly mailing at hbp://NewMR.org


Q & A

Ray Poynter The Future Place

Business

Finding and communicating the story in qualitative information - Lesson 2