11
Communicating and judging the quality of qualitative research: the need for a new language S. A. Fade Royal Brompton Hospital, Sydney Street, London, UK Introduction Having clarified their research question research- ers need to select a method that will enable them to answer that question. Where there is a hypo- thesis to test, a quantitative approach is required and randomized controlled trials offer the route to the strongest evidence (Bowling, 1997b; DePoy & Gitlin, 1998c). However, where little is known about a subject or where the researcher wants to understand the nature or meaning of human experiences, a qua- litative approach offers the opportunity to gain deeper insights. For example, a randomized controlled trial may prove or disprove the hypothesis that enteral feeding improves clinical outcomes for patients requiring bone marrow transplants. However, we may also want to know about what it is like to be a bone marrow transplant patient on an enteral feed and what patients believe about enteral feeding. We may offer patients enteral feeds on the basis that we feel that there is evidence that it will improve their prognosis, but consent and toler- ance to feeding may be influenced by the patients’ beliefs and experiences. Quantitative and qualitat- ive approaches are both required if we are to get a full understanding of the issues. Similar arguments Ó The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149 139 Correspondence Stephanie Fade, Department of Nutrition and Dietetics, Royal Brompton Hospital, Sydney Street, London SW3 6NP, UK. Tel.: 020 73518079 Fax: 020 7351 8990 E-mail: [email protected] Keywords qualitative research, quality standards, research terminology. Abstract Background Traditionally UK dietitians have tended to take a more quantitative approach to research. Qualitative research which gives an in-depth view of people’s experiences and beliefs is also now being used to help answer some important dietetic research questions. Review A review of the limited number of qualitative research papers in the Journal of Human Nutrition and Dietetics 1990–2002 (nine papers in all), revealed a lack of specific discussion of the quality strategies commonly used in qualitative research. This could indicate a less than robust approach, but might also reflect a different per- spective on quality, or simply the difficulties associated with dis- seminating qualitative research to a profession whose members lack familiarity with the language. The fact that qualitative research seems to be used rarely may also indicate a poor understanding of its role. Purpose of this paper This paper seeks to clarify the potential role of qualitative research and draws on previously published guidelines for demonstrating quality. It is hoped that this will offer dietitians a framework for carrying out qualitative research and a language for reporting it, as well acting as a stimulus for discussion.

Communicating and judging the quality of qualitative research: the need for a new language

Embed Size (px)

Citation preview

Page 1: Communicating and judging the quality of qualitative research: the need for a new language

Communicating and judging the quality ofqualitative research: the need for a new language

S. A. Fade

Royal Brompton Hospital, Sydney Street, London, UK

Introduction

Having clarified their research question research-

ers need to select a method that will enable them

to answer that question. Where there is a hypo-

thesis to test, a quantitative approach is required

and randomized controlled trials offer the route to

the strongest evidence (Bowling, 1997b; DePoy &

Gitlin, 1998c).

However, where little is known about a subject

or where the researcher wants to understand the

nature or meaning of human experiences, a qua-

litative approach offers the opportunity to gain

deeper insights.

For example, a randomized controlled trial may

prove or disprove the hypothesis that enteral

feeding improves clinical outcomes for patients

requiring bone marrow transplants. However, we

may also want to know about what it is like to be a

bone marrow transplant patient on an enteral feed

and what patients believe about enteral feeding.

We may offer patients enteral feeds on the basis

that we feel that there is evidence that it will

improve their prognosis, but consent and toler-

ance to feeding may be influenced by the patients’

beliefs and experiences. Quantitative and qualitat-

ive approaches are both required if we are to get a

full understanding of the issues. Similar arguments

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149 139

CorrespondenceStephanie Fade,

Department of Nutrition and Dietetics,

Royal Brompton Hospital,

Sydney Street,London SW3 6NP,

UK.

Tel.: 020 73518079Fax: 020 7351 8990

E-mail: [email protected]

Keywordsqualitative research, quality standards,

research terminology.

Abstract

Background Traditionally UK dietitians have tended to take a more

quantitative approach to research. Qualitative research which gives

an in-depth view of people’s experiences and beliefs is also now being

used to help answer some important dietetic research questions.

Review A review of the limited number of qualitative research papers

in the Journal of Human Nutrition and Dietetics 1990–2002 (nine

papers in all), revealed a lack of specific discussion of the quality

strategies commonly used in qualitative research. This could indicate

a less than robust approach, but might also reflect a different per-

spective on quality, or simply the difficulties associated with dis-

seminating qualitative research to a profession whose members lack

familiarity with the language. The fact that qualitative research seems

to be used rarely may also indicate a poor understanding of its role.

Purpose of this paper This paper seeks to clarify the potential role of

qualitative research and draws on previously published guidelines

for demonstrating quality. It is hoped that this will offer dietitians a

framework for carrying out qualitative research and a language for

reporting it, as well acting as a stimulus for discussion.

Page 2: Communicating and judging the quality of qualitative research: the need for a new language

could be made about almost any area of dietetic

practice.

It is important to understand that qualitative

research does not offer generalizable proofs in the

statistical sense. Some qualitative researchers

choose not to make any attempt to generalize

beyond the specific context of their research.

However, qualitative research findings are often

applicable to a wide range of settings not just the

specific research context. Mason (1996) refers to

this as theoretical or conceptual generalizability as

distinct from the statistical generalizability seen in

quantitative research. Qualitative research papers

should invite the reader to consider whether their

own experience has any commonality with the

findings. This brings issues to the attention of

practitioners and may help with practice reason-

ing by contributing to the reflective process. This

in turn may highlight issues for further quantita-

tive or further qualitative study. For example

Anderson et al. (2001) used the results of their

qualitative research into the reasons given by

mothers for early weaning to design a more

quantitative structured interview study. Sampling

strategies such as theoretical and/or purposive

sampling discussed later, can help improve gen-

eralizability. A further discussion on obtaining

generalizability in qualitative research can be

found in Silverman (2000).

Assuring the quality of qualitative research

All research whether quantitative or qualitative

should be judged on the quality of the methods

used.

Quantitative research is judged largely on issues

of validity, reliability and reproducibility (Bowl-

ing, 1997b; DePoy & Gitlin, 1998c). There has been

some debate about whether this language is sui-

table for judging qualitative research or whether

different terms are required (Mays & Pope, 2000;

Whittemore et al., 2001; Dodd & Davies, 2002).

Mays & Pope (2000) writing in the British

Medical Journal assert that both qualitative and

quantitative research should be seen as an attempt

to represent reality rather than attain truth. On

this basis they suggest that qualitative and quan-

titative research can be judged by common quality

criteria particularly those of validity and

relevance.

The advantage of a common language is clearly

that all researchers can understand each other.

However Mays & Pope (2000) go on to offer a

variety of indicators of validity and relevance for

qualitative research some of which would be

unfamiliar to quantitative researchers. These are

summarized in Table 1 and will be discussed and

explained later.

Quantitative researchers seek to eliminate bias

and demonstrate statistically that any pheno-

menon reported is likely to be associated with a

particular intervention or event rather than

chance. However, qualitative research does not

seek to show statistical associations or cause and

effect relationships. Instead the emphasis is on

describing or illuminating social phenomena and

human experience. Some approaches to qualitative

research such as those described by Moustakas

(1994) and Colaizzi (1978) seek to �bracket out�potential bias emanating from the researcher’s

beliefs and preconceptions. However, there has

been considerable debate about whether it is

actually possible to achieve this (Finlay, 1999;

Braddock, 2001; Caelli, 2001) and much qualitative

research focuses more on exposing and discussing

possible biases rather than eliminating them.

Consequently, I would suggest that using the same

language risks misinterpretation, which could

distract those seeking to critique qualitative

research from a proper consideration of the phi-

losophical principles underpinning the particular

approach used.

Table 1 Strategies for ensuring validity and relevance afterMays & Pope (2000). With permission from the BMApublishing group

Strategies to ensure validityTriangulationMember checkingClear exposition of methods of data collection andanalysis

ReflexivityAttention to negative casesFair dealing

Strategies to ensure relevanceSampling strategies, e.g. probability sampling ortheoretical sampling

S. A. Fade140

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 3: Communicating and judging the quality of qualitative research: the need for a new language

Numerous researchers working in the social

sciences, psychology and medicine have proposed

terms and strategies for assessing quality in qual-

itative research (Glaser, 1978, 1992; Le Compte &

Goetz, 1982; Lincoln & Guba, 1985; Guba & Lin-

coln, 1989; Marshall, 1990; Strauss & Corbin, 1990,

1998; Eisner, 1991; Greene, 1992; Hammersley,

1992; Altheide & Johnson, 1994; Janesick, 1994;

Leininger, 1994; Lincoln, 1995; Mays & Pope, 1995,

2000, Thorne, 1997; Chapple & Rogers, 1998;

Popay et al., 1998; Elliott et al., 1999; Emden &

Sandelowski, 1999). Whittemore et al. (2001)

offered a synthesis of these views and their terms

have been used as a basis for this paper.

The quality strategies used commonly in qual-

itative research can be considered under four key

headings. These are credibility, criticality,

authenticity and integrity (adapted from Whitte-

more et al., 2001).

Credibility

It is broadly accepted that qualitative researchers

should expose their biases and personal perspec-

tives and demonstrate that these have been taken

into account during analysis (DePoy & Gitlin,

1998e; Hall & Callery, 2001; Finlay, 2002). This is

known as reflexivity and it is essential because

qualitative researchers interpret what study par-

ticipants do and say and often ask further probing

questions based on the information they receive.

They also interpret the data and allocate codes to

phrases or phenomena as part of the analytical

process. Clearly this could lead to bias which some

qualitative research frameworks seek to �bracket

out� (Colaizzi, 1978) and others seek to expose and

discuss (e.g. Hall & Callery, 2001). Either approach

can be justified but researchers should articulate

clearly the stance they are taking.

One way of ensuring reflexivity is for the

researcher to keep a reflective diary throughout

the research process. This could be analysed using

a recognized approach (see Table 2) to reveal any

biases, which the researcher should account for

when reporting the findings. Researchers should

also make their personal stance clear in relation to

the subject being studied, along with any relevant

personal characteristics such as their relationships

with the participants.

Using a process known as triangulation can

also enhance credibility. In other words, the study

design could incorporate the use of more than one

method of data collection (e.g. semistructured

interviews and field observations) or more than

one analyst (e.g. peers or participants). The term

triangulation gives the impression that three dif-

ferent approaches should be used in each case,

however, in practice it is interpreted more loosely

to include the use of more than one approach.

Some qualitative research papers report the level

of agreement between different analysts in statis-

tical terms quoting interrater reliability figures.

The tendency amongst social scientists is often to

discuss coding decisions until an agreement is

reached. Either approach is acceptable but the

exact details should be clear in any paper or

presentation. Examples from the field of nutrition

are discussed in Savoca & Miller’s (2001) paper on

food selection and eating patterns amongst people

with type 2 diabetes (interrater reliability 0.92)

and Edstrom & Devine’s (2001) study looking at

women’s orientations to food and nutrition where

Table 2 Key qualitative research frame-works Framework Purpose

Endogenous research Ensure a true �insider perspective�Action research Generate knowledge that can be applied in a

specific context to liberate theparticipants and enhance their lives

Ethnography Understand culturePhenomenology Reveal the meaning behind the participants

experience in a specific settingHeuristic research Reveal personal experiencesLife history Report biographical experienceGrounded Theory Evolve theory

Communicating and judging the quality of qualitative research 141

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 4: Communicating and judging the quality of qualitative research: the need for a new language

coding decisions were discussed until agreement

was reached.

If time and skills constraints prevent peer or

participant analysis, participants could be given

the opportunity to check the data for accuracy and

to discuss whether they feel the findings include

their own experiences. This is often referred to as

member checking. Equally peers could be invited

to a meeting to review the findings and consider a

summary of the researcher’s path of thinking. This

should enable them to evaluate the researcher’s

logic and the clarity of their decision-making. This

is often described as peer debriefing or consider-

ing an audit trail of the findings.

It is essential that qualitative researchers keep

detailed records of the rationale for their analysis

in order to ensure that peers can trace the audit

trail. This can be achieved by writing memos or

drawing diagrams that reveal the logic behind each

decision made.

Audio or videotapes of interviews with partici-

pants should also be kept along with transcripts,

which should usually be made by the researcher

themselves rather than an administrative col-

league. This is significant because the way that

something is said is often just as important as

what is said. A note should be made of the parti-

cipant’s tone of voice, emotional state or body

language wherever this is relevant. Only the

researcher would be able to do this.

It is also worth stressing that all data must be

kept in a manner that complies with the data

protection act and any requirements of the ethics

committee.

Qualitative research proposals and reports

should give enough methodological detail to

enable another researcher to repeat the study,

although the exact conditions could not of course

be replicated. There are numerous frameworks for

qualitative research design and even more

approaches to analysis. Tesch (1990) reviewed

examples of qualitative analysis and found 26

different approaches. Some key qualitative

research frameworks and analytical approaches

are listed in Tables 2 and 3. It is not practical to

discuss these in full here but references are given

for further reading. Qualitative researchers should

always discuss and reference the framework and

analytical approach used and the reasons for their

choice.

Analytical computer software is available (EG

NUD *IST produced by QSR Europe, Northamp-

ton and Ethnograph produced by Scolari, Sage

Publications, London.) However, many qualitative

researchers find it is more helpful to do all the

analysis by hand as this gives them a deeper

understanding of the data. If software is used it is

essential to ensure that the data is transcribed in

plain text only with no formatting.

Making the rationale behind the sampling

strategy clear may further enhance credibility. In

quantitative research probability sampling is

considered favourable. Power studies and reviews

of previous research are used to indicate the size

of sample that would be required. This helps

ensure that any effect seen is due to the inter-

vention rather than chance. Random sampling is

then used to ensure that the sample is not biased.

Using a probability sample of this kind is often

highly impractical for qualitative research as the

number of participants required might be high

and the data collection and analysis methods are

very time intensive. Where probability sampling is

possible it is a perfectly acceptable approach

although it is not considered essential for quality.

This is because the aim of qualitative research

is not necessarily to show unbiased, generaliza-

ble results. However, the sampling technique

should be clear and details should be given of any

relevant characteristics of the population so that

readers can interpret the findings. Sampling

techniques commonly used in qualitative research

are summarized in Table 4.

Sample size in qualitative research is often

determined by convenience but it is also common

practice to collect data until a point termed sat-

uration (Glaser & Strauss, 1967) or informational

redundancy (Lincoln & Guba, 1985) is reached.

Both terms refer to the point where the resear-

cher finds that new sources of data reveal nothing

new about the analytical categories. Researchers

should ensure that they specify the approach

used.

An interesting review of sample extensiveness in

qualitative nutrition education research can be

found in Sobal (2001).

S. A. Fade142

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 5: Communicating and judging the quality of qualitative research: the need for a new language

Table 4 Some sampling techniques commonly used in qualitative research

Sampling technique Basic features

Purposive/systematic, nonprobabilistic Selects subjects with a particular characteristic(Bowling, 1997a; Mays & Pope, 2000)

Theoretical (Glaser & Strauss, 1967) Data from an initial purposive sample is analysed. Further participantsare then selected to locate specific data that might help developor challenge emerging ideas

Snowball (Bowling, 1997a) An initial group of respondents are asked to recruit others who they knowhave the characteristics that are of interest to the researchers. If even moreparticipants are still required the second group are asked to do the same thing

Convenience (Bowling, 1997a) Sample selected based on convenience, i.e. location, willingness to take part

Quota (Bowling, 1997a) Known parameters of a population and their distribution are used topurposively select a sample that is representative of the population

Deviant case (Neuman, 1999) A special type of purposive sampling. Selects cases that differsubstantially form the dominant pattern.

Table 3 Examples of analytical approaches used in qualitative research

Analytical approach Brief explanation and sample references

Componential analysis Distinguishes items from each other on the basis of binary sets of features, i.e. featuresthat either apply or do not apply. For example if looking at weaning foods rice canbe separated from apple on the basis that rice is a starch but apple is not andapple is a fruit and rice is not. See Spradley (1979)

Taxonomies Capture the hierarchical structures in sets of terms. See Ryan & Bernard (1994)

Mental maps Develop explanations through visual displays of the degree of similaritybetween phenomena. Things that are very similar are placed close to eachother and things that are different further apart. See Ryan & Bernard (1994)

Word counts Looks at the frequency of the use of words and phrases by different groups.See Ryan & Bernard (1994)

Semantic network analysis Looks at the co-occurrence of words and phrases using quantitative matrices,which researchers then interpret qualitatively. See Ryan & Bernard (1994)

Cognitivemaps As above but considers the meanings of the words and phrases in the matrixas well as the meaning of the relationships. See Robson (1993) andRyan & Bernard (1994)

Grounded theory Allows theory to evolve from data as a result of line by line analysis,identification of themes and comparison within and across themes.See Glaser & Strauss (1967), Glaser (1992) and Strauss & Corbin (1998)

Schema analysis Looks at metaphors and tacit assumptions in people’s words andactions and tries to determine people’s cognitive interpretations oftheir worlds. See Ryan & Bernard (1994)

Classical content analysis Reduces text to a unit by variable matrix. See Patton (1990) andRyan & Bernard (1994)

Analytic induction Assigns explanations to phenomena through consecutive analysis ofrepeated examples of the phenomena. See Patton (1990)

Ethnographic decision models Looks at causes of behaviours in specific contexts and uses thisinformation to develop explanatory decision trees. See Robson (1993)and Ryan & Bernard (1994)

Communicating and judging the quality of qualitative research 143

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 6: Communicating and judging the quality of qualitative research: the need for a new language

Criticality

A qualitative study proposal needs to include a

detailed account of how the researchers intend to

critically appraise their own findings. This infor-

mation should also be evident in any report or

academic paper that results from the study.

Again analyst triangulation and methods trian-

gulation have important contributions to make.

Qualitative analysis is not simply a case of

counting up the number of times a view is

expressed and presenting the most frequently

expressed view. Qualitative researchers often look

carefully at more unusual views and ask what the

data tells them about the causes and consequences

of these. This is known as searching for negative

cases. The technique can be seen in Cossrow

et al.’s (2001) paper on weight stigmatization.

Authenticity

This term refers to the extent to which the

research reflects the experiences of the respond-

ents as they lived them and perceived them.

Member checking and respondent analysis,

discussed previously enhance the authenticity of

qualitative research as well as the credibility and

criticality.

Qualitative research papers can further demon-

strate authenticity by quoting significant blocks of

raw narrative from the original data. This narra-

tive is often referred to as being �rich� or �thick�.These terms are used to express the vivid picture

of experience that extensive narrative offers.

Qualitative researchers should ensure that their

research reports include enough raw narrative to

convey a vivid picture and support each of the

points they are making from the analysis. Paisley

et al. (2001) demonstrate the use of this approach

in their paper on the meanings associated with

eating fruits and vegetables.

Another way of helping to ensure strong

authenticity is for the participants themselves to

act as the researchers. This approach is used in

endogenous research and action research (DePoy

& Gitlin, 1998d). Alternatively researchers could

ensure that participants are free to talk about

issues that are important to them rather than

issues that are important to the researchers. This

is why qualitative research questions are often

very broad, e.g. �How do bone marrow transplant

patients perceive their experiences of enteral

feeding?� rather than �Do bone marrow transplant

patients accept the side-effects of enteral feeding?�The latter question assumes that the side-effects

are an issue of importance to the group of bone

marrow transplant patients being studied; this

may not be the case. This approach to research is

known as �emic� (DePoy & Gitlin, 1998b). This

means that the researcher is seeking to present an

insider perspective on the subject being studied.

Consequently questionnaires and highly struc-

tured interviews are rarely appropriate unless

previous qualitative studies have already given a

strong indication of the issues of relevance. Even if

this is the case the questions used should be open.

This raises important ethical issues and ethics

committees may be reluctant to sanction research

that does not have a clear agenda from the outset.

This leads us to the next quality issue in quali-

tative research, that of the integrity of the

researchers. Whilst this is clearly an important

issue for all researchers there are particular issues

to attend to in qualitative research.

Integrity

Dodd & Davies (2002) point out that ethics in

qualitative research needs to be looked at con-

textually and flexibly. It is of some concern that

NHS ethics forms tend to focus heavily on issues

of relevance to quantitative studies. Qualitative

researchers could offer the following strategies as

evidence of their trustworthiness as ethical

researchers:

1 A full and verifiable declaration of their posi-

tion and beliefs about the subject they are inves-

tigating could be made on the ethics form, which

could be backed up by references from someone of

suitable standing in the organization. Alternatively

representatives of the ethics committee could

interview researchers and referees could be asked

to verify the information obtained at interview.

2 Consent forms should ensure that respondents

are clear that they are consenting to an open

S. A. Fade144

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 7: Communicating and judging the quality of qualitative research: the need for a new language

interview and that they can refuse to answer any

question that they feel uncomfortable about.

3 Researchers should ensure that they have

access to ethical advice from an experienced

qualitative researcher during data collection and

analysis should any unforeseen incidents occur.

4 Researchers should also ensure that partici-

pants have access to counselling support should

the interview unearth issues that the respondent

finds distressing.

Qualitative researchers can also demonstrate

their integrity in general terms by ensuring cre-

dibility, authenticity and criticality.

Judging quality

A qualitative study does not have to demonstrate

every one of the quality measures discussed in this

paper to be acceptable. As with quantitative

research what is important is that quality issues

are highlighted and discussed so that readers can

make a fully informed decision about the findings.

Qualitative research in UK dietetics

It is impossible to say exactly the extent to which

UK dietitians use qualitative research. Qualitative

studies may have been published by dietitians in a

wide variety of journals and these cannot be

identified using standard computer searches. Our

UK professional journal might be expected to

reflect the extent to which UK dietitians use

qualitative research and the approach taken. A

hand search of the Journal of Human Nutrition

and Dietetics 1990–2002 identified nine qualitative

studies. It is interesting that so few qualitative

studies were found and this may indicate a poor

understanding of the role of qualitative research.

However, those that were found showed a number

of similar characteristics that raise some import-

ant issues for our consideration as a profession.

The extent to which these nine studies used or

discussed some key quality strategies is summari-

zed in Table 5.

Notably only two studies (Rayner & Ziebland,

1999; Hart et al., 2002) specified the analytical

strategy used. The remaining studies described

identifying themes or categories from the data

without giving details of how this was carried out.

Describing techniques and procedures in detail

can lead to verbose reports that would be difficult

to read. However, using frameworks and strategies

that can be recognized and understood by readers

can help make this process more concise.

The rationale behind the sampling technique

was only discussed explicitly in one paper (Hart

et al., 2002) and none of the studies mentioned

collecting data until all the analytical categories

were saturated. This reduces the credibility of the

studies and is an issue also taken up in Sobal’s

(2001) review of qualitative nutrition education

research.

Many studies identified the characteristics of

the participants (Chapman & Chan, 1995; Fiske &

Zhang, 1999; Williams & Sultan, 1999; Wylie et al.,

1999; Hart et al., 2002) and three of these linked

the characteristics with significant sections of raw

narrative (Williams & Sultan, 1999; Wylie et al.,

1999; Hart et al., 2002). The other studies only

reported raw narrative in a very limited way

reducing the vividness of the information pre-

sented. Interestingly several studies also reported

quantitative information about the narrative such

as the proportion of respondents who said certain

things (Williams & Sahota, 1990; Rayner & Zieb-

land, 1999; Wylie et al., 1999). The last of these

even mentioned statistical findings. It is possible

that the focus on counts of responses reflects a

lack of belief in the value of rich narrative alone

and an underlying reliance on numerical results.

Only four of the studies (Chapman & Chan,

1995; Fiske & Zhang, 1999; Rayner & Ziebland,

1999; Hart et al., 2002) described a process of

triangulation, member checking or peer review to

increase the credibility of their work and demon-

strate criticality. This is disappointing but under-

standable as it may have been difficult to find

people with an adequate understanding of quali-

tative research methods. However, where meticu-

lous notes explaining coding decisions have been

kept and recognized frameworks for analysis used,

peer review and member checking become more

realistic propositions even for peers or respond-

ents with no experience of qualitative research.

None of the studies mentioned searching for or

analysing negative cases. As already discussed the

Communicating and judging the quality of qualitative research 145

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 8: Communicating and judging the quality of qualitative research: the need for a new language

Table 5 Quality strategies described in qualitative research papers published in the Journal of Human Nutrition and Dietetics 1990–2002

Reference andkeywords

Framework specifiedyes/no

Analytic approach specifiedyes/no

Triangulationyes/no

Samplingyes/no

Saturationyes/no

Reflexivityyes/no

Negativecasesyes/no

Hart et al. (2002) – children’s knowledge,nutrition

No Yes. Krueger, (2000) Analyst Quota No No No

Fiske & Zhang (1999) – oral health,diet, elderly mentally ill

No No Methods Not discussed No No No

Mitchell (1999) – audit,nutrition, community hospital

No No No Not discussed No No No

Wylie et al. (1999) – nutritional intake,elderly, mobility problems

No No No Not discussed No No No

Rayner & Ziebland (1999) –research workshop, evaluation

No YesConstantcomparativemethod. Software(NUDIST)

No Not discussed No No No

Roberts & Ashley (1999) – rural generalpractice, weight loss, obesity

No No No Not discussed No No No

Williams & Sultan (1999) – Asian women,healthy eating, exercise

No No No Not discussed No No No

Chapman & Chan (1995) – calcium,education materials

No, butpublished proceduresfor focus groupswere followed

No Analyst Purposive butnot discussed

No No No

Williams & Sahota (1990) –Asian, Muslim mothers,feeding, dental health

Blinkhorn et al.(1989) was referencedbut not discussed

No No Purposivebut notdiscussed

No No No

S.A

.Fad

e146

�The

British

Dietetic

Asso

ciation

Ltd2003

JH

um

Nutr

Die

tet,

16,

pp.

139–1

49

Page 9: Communicating and judging the quality of qualitative research: the need for a new language

studies seem to emphasize the views noted to be

most common. This reflects a rather quantitative

approach to qualitative analysis.

Reflexivity was not discussed in any of the

papers and only one group of researchers gave any

information about themselves (Rayner & Ziebland,

1999). This study reported a qualitative evaluation

of a research workshop and the researchers

pointed out that neither of them had been tutors

at the workshops they were evaluating. This is

important information as respondents might have

been expected to give different responses if their

tutors were interviewing them. This helps the

reader develop a more thorough understanding of

the findings. In qualitative research it is essential

that potential sources of bias are exposed and

discussed.

All the studies used predefined topics as a guide

for their observations or interviews. Most also

used predefined questions (Williams & Sahota,

1990; Fiske & Zhang, 1999; Mitchell, 1999; Rayner

& Ziebland, 1999; Williams & Sultan, 1999; Wylie

et al., 1999; Hart et al., 2002.) However, Fiske &

Zhang (1999) based their questions on their

observations of the participants rather than on

their own opinions about what was important.

Whilst Wylie et al. (1999) encouraged participants

to continue talking freely after the end of the

interview in order to elicit additional views.

Overall the studies gave very limited consideration

to quality strategies relating to authenticity. Only

one of the papers reported involving participants

in the analysis through the process of member

checking (Rayner & Ziebland, 1999). As already

discussed the use of recognized frameworks and

the keeping of meticulous coding notes could help

facilitate this process.

Conclusion

It is not surprising that dietitians who are more

familiar with carrying out and critiquing quanti-

tative research should initially try to underpin

qualitative research with quantitative principles.

Qualitative research is founded on naturalistic

philosophy, which espouses the view that the

knower and knowledge are interrelated and

interdependent (DePoy & Gitlin, 1998a). It is used

to develop in-depth descriptions and to illuminate

social phenomena and human experience. Quan-

titative research is based on logical-positivist

philosophy, which views knowledge as part of a

reality that is separate from individuals and veri-

fiable through scientific method (DePoy & Gitlin,

1998a). It is precisely these philosophical differ-

ences that confirm the need for a new and distinct

language of quality for qualitative research.

In an era of interprofessional learning it is

essential that we seek to understand the language

of quality used by professions with more experi-

ence in the field. In relation to our own research

we need to ensure that we use appropriate lan-

guage to clearly articulate our research methods

and quality strategies and that we engage in debate

with other health care professionals about how

these can be developed.

Acknowledgements

I would like to thank Susan Ryan and Jacqueline Potterat the University of East London for introducing me toqualitative research and for their continued help andencouragement. I would also like to thank Liza Draper,Dr Gary Frost, Shirena Counter, Mary Kelly and theBDA research committee in particular Dr AngelaMadden for their helpful comments and support.

References

Altheide, D.L. & Johnson, J.M. (1994) Criteria for assessing

interpretive validity in qualitative research. In Hand-

book of Qualitative Research eds N. K. Denzin &

Y. S. Lincoln, pp. 485–499. Thousand Oaks, CA: Sage.

Anderson, A.S., Guthrie, C.A., Alder, E.M., Forsyth, S.,

Howie, P.W. & Williams, F.L. (2001) Rattling the plate –

reasons and rationales for early weaning. Health Educ.

Res. 16, 471–479.

Blinkhorn, A.S., Leather, D.S. & Kay, E.J. (1989) An assess-

ment of the value of quantitative and qualitative data

collection techniques. Comm. Dent. Health 6, 147–151.

Bowling, A. (1997a) Sample size and sampling for quan-

titative research, Chapter 7. In Research Methods

in Health. Investigating Health and Health Services.

Buckingham: Open University Press.

Bowling, A. (1997b) Quantitative research: experiments and

other analytic methods of investigation, Chapter 9. In

Research Methods in Health. Investigating Health and

Health Services. Buckingham: Open University Press.

Braddock, G. (2001) Beyond reflection in naturalised

phenomenology. J. Consciousness Studies 8, 1–13.

Communicating and judging the quality of qualitative research 147

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 10: Communicating and judging the quality of qualitative research: the need for a new language

Caelli, K. (2001) Engaging with phenomenology: is it more

of a challenge than it needs to be? Qual. Health Res. 11,

273–281.

Chapman, K.M. & Chan, M.W. (1995) Focus groups: their

role in developing calcium related education materials.

J. Hum. Nutr. Dietet. 8, 363–368.

Chapple, A. & Rogers, A. (1998) Explicit guidelines for

qualitative research. A step in the right direction, a

defence of the soft option or a form of sociological

imperialism? Family Prac 15, 556–561.

Colaizzi, P.F. (1978) Psychological research as the phe-

nomenologist views it. In Existential Phenomenological

Alternatives for Psychology. eds R. Vaile & M. King,

pp. 48–71. New York: Oxford University Press.

Cossrow, N.H.F., Jeffery, R.W. & McGuire, M.T. (2001)

Understanding weight stigmatisation: a focus group

study. J. Nutr. Educ. 33, 208–214.

DePoy, E. & Gitlin, L.N. (1998a) Philosophical foundations,

Chapter 3. In Introduction to Research: Understanding

and Applying Multiple Strategies, 2nd edn. London:

Mosby.

DePoy E. & Gitlin L.N. (1998b) Language and thinking

processes, Chapter 8. In Introduction to Research:

Understanding and Applying Multiple Strategies, 2nd

edn. London: Mosby.

DePoy, E. & Gitlin, L.N. (1998c) Experimental-type designs,

Chapter 9. In Introduction to Research: Understanding

and Applying Multiple Strategies, 2nd edn. London:

Mosby.

DePoy, E. & Gitlin, L.N. (1998d) Naturalistic inquiry,

Chapter 10. In Introduction to Research: Understanding

and Applying Multiple Strategies, 2nd edn. London:

Mosby.

DePoy, E. & Gitlin, L.N. (1998e) Analysis in naturalistic

inquiry, Chapter 20. In Introduction to Research:

Understanding and Applying Multiple Strategies, 2nd

edn. London: Mosby.

Dodd, J. & Davies, D. (2002) Qualitative research and the

question of rigor. Qual. Health Res. 12, 279–289.

Edstrom, K.M. & Devine, C.M. (2001) Consistency in

women’s orientations to food and nutrition in midlife

and older age. A 10 year qualitative follow-up. J. Nutr.

Educ. 33, 215–223.

Eisner, E. (1991) The Enlightened Eye: Qualitative Inquiry

and the Enhancement of Educational Practices.

New York: Macmillan.

Elliott, R., Fischer, C.T. & Rennie, D.L. (1999) Evolving

guidelines for publication of qualitative research studies

in psychology and related fields. Br. J. Clin. Psych. 38,

215–229.

Emden, C. & Sandelowski, M. (1999) The good the bad and

the relative, part two: goodness and the criterion problem

in qualitative research. Int. J. Nursing Prac. 4, 206–212.

Finlay, L. (1999) Applying phenomenology in research:

problems, principles and practice. Br. J. Occ. Ther. 62,

299–306.

Finlay, L. (2002) �Outing� the researcher: the provenance,

process and practice of reflexivity. Qual. Health Res. 12,

531–545.

Fiske, J. & Zhang, W. (1999) Fitting dietary advice for oral

health into the roles of elderly mentally ill people.

J. Hum. Nutr. Dietet. 12, 389–394.

Glaser, B. (1978) Theoretical Sensitivity: Advances in the

Methodology of Grounded Theory. Mill Valley, CA:

Sociology Press.

Glaser, B.G. (1992) Basics of Grounded Theory Analysis:

Emergence Versus Forcing. Palo Alto, CA: Sociology

Press.

Glaser, B.G. & Strauss, A.L. (1967) Discovery of Grounded

Theory. Strategies for Qualitative Research. Chicago, IL:

Aldine.

Greene, J.C. (1992) Objectivity in educational research: the

practitioners’ perspective. Curriculum Inquiry 22, 39–45.

Guba, E.G. & Lincoln, Y.S. (1989) Fourth Generation Eva-

luation. Newbury Park, CA: Sage.

Hall, W.A. & Callery, P. (2001) Enhancing the rigor of

grounded theory: incorporating reflexivity and

relationality. Qual. Health Res. 11, 257–272.

Hammersley, M. (1992) What’s Wrong with Ethnography?

Methodological Exploration. London: Routledge.

Hart, K.A., Bishop, J.A. & Truby, H. (2002) An investigation

into school children’s knowledge and awareness of food

and nutrition. J. Hum. Nutr. Dietet. 15, 129–140.

Janesick, V.J. (1994) The dance of qualitative design:

metaphor, methodolatry and meaning. In Handbook of

Qualitative Research. eds N. K. Denzin & Y. S. Lincoln,

pp. 209–219. Thousand Oaks, CA: Sage.

Krueger, R.A. (2000) Focus Groups: A Practical Guide for

Applied Research, 3rd edn. Thousand Oaks, London:

Sage.

LeCompte, M.D. & Goetz, J.P. (1982) Problems of reliability

and validity in ethnographic research. Rev. Educ. Res.

52, 31–60.

Leininger, M. (1994) Evaluation criteria and critique of

qualitative research studies. In Critical Issues in Quali-

tative Research Methods, ed. J. M. Morse, pp. 95–115.

Thousand Oaks, CA: Sage.

Lincoln, Y.S. (1995) Emerging criteria for quality and

qualitative and interpretive research. Qual. Inquiry 3,

275–289.

Lincoln, Y.S. & Guba, E.G. (1985) Naturalistic Inquiry.

Newbury Park, CA: Sage.

Marshall, C. (1990) Goodness criteria: are they objective or

judgement calls? In The Paradigm Dialogue, ed. E. C.

Guba, pp. 188–197. Newbury Park, CA: Sage.

Mason, J. (1996) Qualitative Researching, Chapters 5–7.

London: Sage.

Mays, N. & Pope, C. (1995) Rigour and qualitative research.

Br. Med. J. 311, 109–112.

Mays, N. & Pope, C. (2000) Qualitative research in health

care. Assessing quality in qualitative research. Br. Med.

J. 320, 50–52.

S. A. Fade148

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149

Page 11: Communicating and judging the quality of qualitative research: the need for a new language

Mitchell, H. (1999) Nutrition audit at a community hospital.

J. Hum. Nutr. Dietet. 12, 425–432.

Moustakas, C. (1994) Phenomenological research methods.

Thousand. Oaks, CA: Sage.

Neuman, W.L. (1999) Social Research Methods: Qualitative

and Quantitative Approaches. Needham Heights, MA:

Allyn and Bacon.

Paisley, J., Sheeshka, J. & Daly, K. (2001) Qualitative

investigation of the meanings of eating fruits and

vegetables for adult couples. J. Nutr. Educ. 33, 199–207.

Patton, M.Q. (1990) Analysing and interpreting qualitative

data. Chapter 6. In Qualitative Evaluation and Research

Methods, 2nd edn. Newbury Park, CA: Sage.

Popay, J., Rogers, A. & Williams, G. (1998) Rationale and

standards for the systematic review of qualitative lit-

erature in health services research. Qual. Health Res. 8,

341–351.

Rayner, M. & Ziebland, S. (1999) Process evaluation of a

research workshop and follow-up support to help prac-

titioners from 13 weight management projects to carry

out evaluations. J. Hum. Nutr. Dietet. 12 (Suppl. 1), 9–19.

Roberts, A. & Ashley, G. (1999) What are the characteristics

of overweight and obese patients who achieve weight

loss and what factors are most helpful? A quantitative

and qualitative study of patients and interventions in a

rural general practice. J. Hum. Nutr. Dietet. 12 (Suppl.

1), 20–27.

Robson, C. (1993) The analysis of qualitative data. Chapter

12. In Real World Research. Oxford: Blackwell.

Ryan, G.W. & Bernard, H.R. (1994) Data management and

analysis methods. pp. 769–802. In Handbook of Quali-

tative Research. ed. N. K. Denzin & Y. S. Lincoln.

Thousand Oaks, CA: Sage.

Savoca, M. & Miller, C. (2001) Food selection and eating

patterns: themes found among people with type two

diabetes. J. Nutr. Educ. 33, 224–233.

Silverman, D. (2000) Selecting a case, Chapter 8. In Doing

Qualitative Research: A Practical Handbook. ed.

D. Silverman. London: Sage.

Sobal, J. (2001) Sample extensiveness in qualitative nutri-

tion education research. J. Nutr. Educ. 33, 184–192.

Spradley, J.P. (1979) The Ethnographic Interview. New

York: Holt, Rinehart and Winston.

Strauss, A. & Corbin, J. (1990) Basics of Qualitative

Research: Grounded Theory Procedures and Techniques.

Newbury Park, CA: Sage.

Strauss, A.L. & Corbin, J. (1998) Basics of Qualitative

Research: Techniques and Procedures for Developing

Grounded Theory, 2nd edn. Thousand Oaks, CA: Sage.

Tesch, R. (1990) Qualitative research: analysis types and

software tools. New York, NY: Falmer.

Thorne, S. (1997) The art and science of critiquing quali-

tative research. In Completing a Qualitative Project:

Details and Dialogue, ed. J. M. Morse, pp. 117–132.

Thousand Oaks, CA: Sage.

Whittemore, R., Chase, S.K. & Mandle, C.L. (2001)

Validity in qualitative research. Qual. Health Res. 11,

522–537.

Williams, S.A. & Sahota, P. (1990) An enquiry into the

attitudes of Muslim Asian mothers regarding infant

feeding practices and dental health. J. Hum. Nutr. Dietet.

3, 393–401.

Williams, J. & Sultan, M. (1999) Evaluation of an Asian

women’s healthy eating and exercise group. J. Hum.

Nutr. Dietet. 12 (Suppl. 1), 91–98.

Wylie, C., Copeman, J. & Kirk, S.F.L. (1999) Health and

social factors affecting the food choice and nutritional

intake of elderly people with restricted mobility. J. Hum.

Nutr. Dietet. 12, 375–380.

Communicating and judging the quality of qualitative research 149

� The British Dietetic Association Ltd 2003 J Hum Nutr Dietet, 16, pp. 139–149