NB Single Case Designs All 2010 6 Slides Pp Handout

8/2/2019 NB Single Case Designs All 2010 6 Slides Pp Handout

1/168

SINGLE-CASERESEARCH DESIGNS:

An Alternative to StandardResearch Designs

PSYC 344 2010

Neville M Blampied

Associate Professor

Department of

Psychology


2/168


3/168

A brain teaser to begin

From the perspective of standard psychological

research there is something radical about thefollowing quote from an astronomer and anastrophysicist

What is it?


4/168

When cherished ideas are in ruins at your feet,nature is challenging you to look at the world

anew: What have you missed? What have younot thought of before? In such situations, it helpsto focus on the most extreme cases, because

they are where the unknown forces may beoperating most distinctly.

Balick, B, & Frank, A (2004). The extra-

ordinary deaths of ordinary stars. Scientific American, 291 (1), 27 35.


5/168

Theme 1 Research design

mattersThe sad and enlightening case

of Ignaz Semmelweis (1818 1865)

Time 1840s

Place Vienna

Profession specialist in

obstetric medicine Job Physician at Vienna

Obstetric Clinic [worldslargest such clinic]

Clients women who couldnot give birth at home(unmarried/poor/

complications)


6/168

Problem

Death from puerperal fever

Death rate up to 25 30% in epidemics

Theories: caused by miasma or the onset oflactation. Endorsed by all leading physicians.

Most regarded it as unpreventableandinevitable.

(+ religious justification)


7/168

Two key observations

1. A friend of Semmelweiss died of a

wound infection inflicted during thepost-mortem of a woman who had

died of puerperal fever.


8/168

Day and night the image of [the friends] illness

pursued me. As we found identical changes inhis body and those of women, it can beconcluded that [he] died of the same disease.


9/168

First Clinic vs Second Clinic

2. 1846 data on fever deaths

First Clinic: 459 of 4,010 women admitted to the First Clinic

died.

mortality of 11.4 per cent for the year; and some months, deaths rose to 18 per cent.

Second Clinic (physically a duplicate of the First)

105 of 3,754 women patients died

mortality of 2.7 per cent.


10/168

Also observed:

Ward 1 was used to teach medical students.Ward 2 was served by midwives.

Only medical students & staff did post-mortems

Otherwise the 2 wards were the same.


11/168

Semmelweis new theory

He suspected that something infectious was

carried from the dead women in the morgueto the newly delivered women in the wards.

[Since only physicians & medical students, notmidwives, did post-mortems].

Note this was before Lister and Pasteur!


12/168

Semmelweis interventionHe insisted that all staff entering the wards wash their

hands in a solution of chlorinated lime.

Chlorinated lime was a cleaning agent known to get rid

of the awful smell of the infection dodd.cmcvellore.ac.in/hom/26 -Semmelweis.jpg


13/168

Results

April, 1847, 57 (18%) died before

handwashingAfter handwashing -

June, 1847, 6 (2.38%)deaths July 1847, 3 (1.2%).


14/168

Professional reactions

Some younger medical staff accepted

Semmelweis evidence Opposed by older physicians, especially the

head of the clinic, because it did not agreewith accepted theory.


15/168

Consequences for Semmelweiss

1849 fired from the Clinic

1849 was refused a post at the Vienna MedicalSchool

1850 found work in Budapest Put in charge when there was an epidemic of

puerperal fever

Introduced the same hand-washing procedure


16/168

Results in Budapest 1850 - 1855

Results following the

second interventionshown in bottom panel

of Fig.

This was a replication ofhis findings in Vienna


17/168

1855 became Professor of Obstetrics at

University of Pest. Government of Hungary accepted his ideas,

and ordered his hand-washing procedure tobe implemented in all districts.


18/168

Reactions in Vienna

Editor of Vienna Medical Journalsaid that

it was time to stop this nonsense about hand-washing.


19/168

Semmelweis further work

1861 published his book

Etiology, Understanding & Prevention of ChildbedFever. [original in German]

When I look back upon the past, I can only dispelthe sadness which falls upon me by gazing into thathappy future when the infection will be banished . . .

The conviction that such a time must inevitablysooner or later arrive will cheer my dying hour.Semmelweis, Etiology, Foreword.


20/168

His book

Rejected by all prominent medical societies

and obstetric specialists, because it wentagainst traditional authorities.

Harshly criticised by reviewers.


21/168

Semmelweis final years

Died 13 August, 1865 in a mental hospital, probably of

injuries. Likely he was suffering early onsetdementia.

But by end of 19thC was accepted as a pioneer of

antisepsis - Lister I think with the greatestadmiration of him.

Listers first trial of antisepsis using phenol on a post-

operative wound was on 12 August, 1865.


22/168

Semmelweis final years

Died 13 August, 1865 in a mental hospital, probably of

injuries. Likely he was suffering early onsetdementia.

But by end of 19thC was accepted as a pioneer of

antisepsis - Lister I think with the greatestadmiration of him.

Listers first trial of antisepsis using phenol on a post-

operative wound was on 12 August, 1865.


23/168

The idea of antisepsis spread very slowly, despite the

evidence of Semmelweis & Lister:

Britain 1898 leading gynaecologist denied the

existence of bacteria and the value of Listers

methods.USA 1880s the majority of obstetricians would still

go from one delivery to the next without any sort of

adequate cleansing of hands or instruments.


24/168

The Moral of the Story1. Research design matters it can be a

matter of life or death!


25/168

The Moral of the Story1. Research design matters it can be a

matter of life or death!

2. Research designs are accepted by socialconsensus within the sciences

Rules of design, conduct & presentation of

research are social constructions. Research is a form of persuasion

Problem for Semmelweis was that no

consensus existed about what evidence wasor that evidence could overcome accepted

theory

No commitment to evidence-based practice


26/168

3. Research-based knowledge has special status

(when gathered according to valid researchdesigns).

I regard scientific knowledge as the best and mostimportant kind of knowledge we have though ofcourse it is not, by any means, the only kind of

knowledge we possess. Sir Karl Popper (1982)


27/168

Incidentally, Popper went on to say

Scientific knowledge starts from problems,

both from theoretical and from practicalproblems. An example of a great practicalproblem is the fight of medical science

against avoidable suffering


28/168

4. Progress may come when a fresh look is

taken at an old problem Semmelweiss rejected the

unpreventable notion.

5. Exploiting the signal of variability in anatural experiment (Ward 1 vs Ward 2)

? What if just consider average mortalityover Ward 1 & Ward 2?


29/168

6. Developing a tentative theory

7. Inventing an intervention based on thetheory.

8. Using data [not opinion/cultural authorities]to evaluate the outcome.

9. Replicating the outcome at another place

and time.


30/168

Theme 2 -The Emergence of the Standard

Model of Research Design in Psychology

How did Psychology develop a sharedconsensus about research design?

The origins lie in the later 19C


31/168

Historical Background

those who are not accustomed to original enquiry entertain ahatred and a horror of statistics. They cannot endure the idea of

submitting their sacred impressions to cold-blooded verification.But it is the triumph of scientific men to rise superior to suchsuperstitions, to devise tests by which the value of beliefs may beascertained, and to feel sufficiently masters of themselves to

discard contemptuously whatever may be found to be untrue.

Sir Frances Galton (late 19th C)


32/168

Origins of Social Statistics

Middle-class philanthropists and social workers used statistics to

learn about kinds of people whom they did not know and did notcare to know, as persons. A method of study that ignoredindividuality seemed right somehow for the lower classes.

Porter, T.M. (1995). Trust in Numbers: the Pursuit of Objectivity in

Science and Public Life. Princeton, NJ: Princeton UniversityPress.


33/168

The man who fulfilled Galtons prophecy

Sir Ronald Aylmer Fisher, FRS(17 February 189029 July1962) was a British statistician,evolutionary biologist, andgeneticist. He was describedby Anders Hald as "a genius

who almost single-handedlycreated the foundations formodern statistical science" andRichard Dawkins described

him as "the greatest ofDarwin's successors".

Source: Wikipedia& Yates & Mather(1963) Biographical memoirs of theFellows of the Royal Society ofLondon.
http://en.wikipedia.org/wiki/Fellow_of_the_Royal_Societyhttp://en.wikipedia.org/wiki/February_17http://en.wikipedia.org/wiki/1890http://en.wikipedia.org/wiki/July_29http://en.wikipedia.org/wiki/1962http://en.wikipedia.org/wiki/UKhttp://en.wikipedia.org/wiki/Statisticianhttp://en.wikipedia.org/wiki/Evolutionary_biologisthttp://en.wikipedia.org/wiki/Geneticshttp://en.wikipedia.org/wiki/Anders_Haldhttp://en.wikipedia.org/wiki/Richard_Dawkinshttp://en.wikipedia.org/wiki/Charles_Darwinhttp://en.wikipedia.org/wiki/Charles_Darwinhttp://en.wikipedia.org/wiki/Richard_Dawkinshttp://en.wikipedia.org/wiki/Anders_Haldhttp://en.wikipedia.org/wiki/Geneticshttp://en.wikipedia.org/wiki/Evolutionary_biologisthttp://en.wikipedia.org/wiki/Statisticianhttp://en.wikipedia.org/wiki/UKhttp://en.wikipedia.org/wiki/1962http://en.wikipedia.org/wiki/July_29http://en.wikipedia.org/wiki/1890http://en.wikipedia.org/wiki/February_17http://en.wikipedia.org/wiki/Fellow_of_the_Royal_Society


34/168

Fishers contributions to statistics

The analysis of variance 1918 (genetics)

Statistical Methods for Research Workers(1925). The Design of Experiments(1935)

Devised (or improved on)

Control groups Randomization

Factorial designs, &

Analysis of Variance & Null Hypothesis Tests

Every experiment may be said to exist only in orderto give the facts a chance of disproving the nullhypothesis. (Fisher, 1935).


35/168

Fishers statistics & psychology

Fishers ideas about research and statistical

testing began to influence research inpsychology in the 1930s, initially byeducational psychologists, later by

experimental and clinical psychologists.Inferential statistics made their way from the

applied fields into the laboratory of theexperimenter.

[Gigerenzer et al., 1989]


36/168

The inference revolution in psychology

Although their initial understanding of these techniques was often defective,psychologists embraced them with considerable enthusiasm. (Danziger, 1987, p35).


37/168

The inference revolution in psychology

Although their initial understanding of these techniques was often defective,psychologists embraced them with considerable enthusiasm. (Danziger, 1987, p35).


38/168

The Standard Model of Research

Based on Statistical Sampling Theory

Scientific interest is in attributes of a population -A complete set of individuals sharing a commoncharacteristic.

Scientific hypotheses are about populations, notindividuals or samples, but ...

It is impractical to measure every individual, so we

take a sample.


39/168

Statistical sampling ...

Measurement of the sample yields our

Dependent variable(s) (DV) Sample statistics (means, standard

deviations, etc) are estimates of population

parameters calculated from the DVs.

Null hypothesis assumes samples are from

same population. Experimental hypothesis assumes they are

from different populations.


40/168

The Null Hypothesis Statistical Test

NHST1. Assume that the experimental and control

samples are drawn from the samepopulation - Ho.

(any differences will be sampling error)

2. Given this assumption, compute thelikelihood that the obtained sample meanvalues would be as different as observed.

3. If probability < some value (e.g.,


41/168

Key Attributes of the Standard Model

Recruit as large a sample as possible.

Randomly allocate participants to treatmentconditions.

Aggregate individual data into groupaverages (sample statistics).

Draw inferences about the population from

the sample statistics.


42/168

Key attributes ...

Use inferential statistics to separate out

experimental effects (experimental variance)from error variance, by comparing means andvariances about the means.

Use Null-hypothesis tests to determine thescientific legitimacy of the results (typically,

requiring p < 0.05)


43/168

The hybrid model of science in

Psychologys standard modelClassical statistical inference as it is generally

portrayed is in face an anonymous hybridconsisting of the union of the ideasdeveloped by Ronald Fisher, on the one

hand, and Jerzy Neyman and Egon Perrson,on the other It is a union that neither sidewould have agreed to, given the pronounced

philosophical and methodological differencesbetween them.

(Hubbard, 2004, p 296)


44/168

Theme 3 -Criticisms of the Standard

ModelCriticism began early in 1940s

Criticism has focussed on use of NHST as theonly decision rule.


45/168

Criticisms of NHST

NHST has been called:-

a religion(Cohen, 1990) an addiction(Schmidt & Hunter, 1997)

an enduring tyranny(Loftus, 1991)

ajustification for betraying the evidence of the rawdata(Faverau, 1993)

the most bone-headedly misguided procedure everinstitutionalized in the rote training of sciencestudents(Rozenbaum, 1997)


46/168

Further, NHST

forces researchers to play science by therules of the 20-questions game(Johnston

& Pennypacker, 1993) does not tell us what we want to know

(Cohen, 1994) is among the most questionable things we

do(Trerweiler & Stricker, 1998), and


47/168

Meehl on NHST

Sir Ronald has befuddled us, mesmerized us, and led us down

the primrose path. I believe that the almost universal reliance onmerely refuting the null hypothesis as the standard method forcorroborating substantive theories [in psychology] is a terriblemistake, is basically unsound, poor scientific strategy, andone of

the worst things that every happened in the history ofpsychology.

[Meehl, P.E. (1978). Theoretical risks and tabular asterisks: SirKarl, Sir Ronald, and the slow progress of soft psychology.

Journal of Consulting and Clinical Psychology, 46, p 817]


48/168

More Meehl criticisms of NHST

Other productive sciences do not use it, so

why is it so dominant in psychology? It is the handmaiden of a discredited

hypothetical-deductive model of science.

It exposes theories to only a weak risk ofrefutation (Sir Karl Popper).


49/168

Meehls paradox

As N (the sample size) increases towards

infinity it approximates the population, a goodthing, because it reduces sampling error. But

As N increases, the difference between means

required to reject the null hypothesisapproaches zero. So

The better your experiment, the more trivialyour test of theory!


50/168

Paul Meehl on NHST

a potent but sterileintellectual rake wholeaves in his merry path

a long train of ravishedmaidens but no viablescientific offspring.

Meehl, P.E. (1967). Theory testing inpsychology and physics: Amethodological paradox. Philosophy of

Science, 34, 103-115.


51/168

The mismatch of research methods and

applied researchThe standard model does not suit the needs of

much applied research, e.g.,there is a growing disaffection fromtraditional experimental designs and

statistical procedures which are heldinappropriate to the subject

matter[particularly] in the area oftherapeutic changeBergin, A, & Strupp, H. (1972). Changing frontiers in the science of

psychotherapy. Chicago, Il: Aldine [p 440].


52/168

The Mismatch

Is both a matter of

Principle&

Practice

? Why was the mismatch at the level ofprinciple not understood by the earlyadopters of Fishers methods?


53/168

Mismatch at the level of principle

Hypotheses are about populations

Not about

- The sample

- The individual cases in the sample

Yet, applied psychology needs to apply

knowledge at the individual level[Exceptions actuarial or public health

applications]


54/168

In the standard model of psychological

research the individual disappears we investigate 150 freshman from Michigan for 20 minutes of

their lifetimes, and think of them as interchangeable physical objectsthat do not change over time which allows us to present oursignificant result as if it were about all mankind and all time.

Gigerenzer (1987). Probabilistic thinking and the fight against subjectivity. In

The probabilistic revolution. Vol 2. p 13 - 33.

h d b d d h


55/168

The double standard in Psychology

Throughout its history as a science, psychology hasben plagued by a double standard in its treatment ofthe individual subject

In psychological discourse (both scientific and

applied) the individual is constantly given highrelevance. In contrast, theindividual case is usuallyforgotten in the practice of psychological researchbecause it is replaced by samples of subjectsthatare assumed to represent some general population.

S h d f h l i


56/168

So, how do we get from the population to

the individual?Well we cant, legitimately

So

We infer from the sample to the population(legitimately except that psychologists rarelyspecify the population they have sampled from)

We attribute generic, prototypic, ideal status to thepopulation average (a step of inductivegeneralization) e.g., gender generalizations

characteristics of that abstracted individual may

easily become attributed to particular concreteindividuals with whom psychologists work(adeduction - Valsiner).

Th b d f


57/168

This is bad for our science

The inductive inference from samples to the

abstract individual, and from it now (alreadydeductively) back to the multitude of concretehuman beings, is guided by a number of

implicit assumptions. cognitive blinders thatobscure insight into the science and hamperits applications.

Valsiner, J. (1986). The individual subject and scientific psychology.New York: Plenum.

Th Mi h I P i d d


58/168

The Mismatch In Practice standard

model research has distorted clinicalresearch

I P i d d d l h h


59/168

In Practice standard model research has

distorted clinical researchWe found that p values received an absurdly centralposition in many studies often at the expense of

descriptive statistics. In a study of weight loss, forexample, authors chose to mention the imposingp of .001 rather than any measure of weight loss.

Finally, one article contained only p values; nomeans, standard deviations; correlationscoefficients; or any other statistics were mentioned.

Dar, Serlin, & Omer (1994). Misuse of statistical tests in three decades ofpsychotherapy research. Journal of Consulting & Clinical Psychology, 62,p77

A d h h h d i


60/168

And not much has changed since

A d h h h d i


61/168

And not much has changed since

Dar et al (1994) found that 30% of articles reportedclinical significance our coding in Period 4 [1993 2001] shows only 40% of articles made anyattempt to discuss clinical significance. This isserious. In a major journal dedicated to the researchof psychotherapy and other interventions, clinicalsignificance should be relevant to more than 40%...

Fidler, Cumming,Thompson et al (2005). Towards improved statistical reportingin the Journal of Consulting & Clinical Psychology. JCCP, 73, 136 -143.

Al i NHST


62/168

Alternatives to NHST

There is no magic alternative or other objective

mechanical ritualto replace NHST (Cohen,1994), but critics recommend:


63/168

Exploratory data analysis (Tukey)

Graphic analysis (Tufte, Loftus)

Inspection of distributions (Faverau)


64/168


Graphic analysis (Tufte, Loftus) Inspection of distributions (Faverau)

Calculation of point or range estimates of

parameters (based on theoreticalpredictions) (Meehl)

Use of confidence intervals (Schmidt)


65/168


Graphic analysis (Tufte, Loftus) Inspection of distributions (Faverau)

Calculation of point or range estimates of parameters (based ontheoretical predictions) (Meehl)

Use of confidence intervals (Schmidt)

Calculation of effect sizes (Cohen)

Meta-analyses (Smith & Glass)

R i i i


66/168

Responses to criticism

Four broad forms of response to criticism of the

standard model can be noted1. Ignore it - the most common response.

2. Reform from within (e.g., the APA Taskforceon Statistical Inference, 1999).

3. Qualitative research methods

4. Single-case research

R f f i hi


67/168

Reform from within

Has a long history, beginning almost as soon

as Fisherian statistics were adopted. Mostly ignored, especially by gatekeepers

such as journal editors.

And by academic teachers of statistics!

Became more strident in the 1990s

Led to the APA Taskforce on StatisticalInference (1999)

Th APA T kf


68/168

The APA Taskforce

As an important step in Reform from withinthe Taskforce

Kept most features of the Standard Model

Focussed on NHST

Recommended that NHST be supplementedby other forms of statistical inference andquantitative analysis.

Wilkinson & the Taskforce on Statistical Inference. (1999). StatisticalMethods in Psychology Journals. American Psychologist, 54(8),594-604.

Q lit ti R h


69/168

Qualitative Research

During recent decades has become very

influential in all the social sciences, includingpsychology

Committed to the social construction of

knowledge

Rejects most aspects of quantitative,

inferential research. Employs detailed descriptions of individual or

group experience and thematic or other formsof verbal analysis.

Th b i t i


70/168

The brain teaser again

From the perspective of standard psychological

research there is something radical about thefollowing quote from an astronomer and anastrophysicist

What is it?

Balick B & Frank A (2004). The extra-


71/168

Balick, B, & Frank, A (2004). The extraordinary deaths of ordinary stars. Scientific American, 291 (1), 27

35.

When cherished ideas are in ruins at your

feet, nature is challenging you to look at theworld anew: What have you missed? Whathave you not thought of before? In such

situations, it helps to focus on the mostextreme cases, because they are where theunknown forces may be operating mostdistinctly.

Ans er


72/168

Answer

Psychologists would say Try to increase

your sample size because only data aboutthe average nebula, galaxy etc isscientifically valid.

Psychologists have a horror of the extremecase, and try to avoid considering extreme

cases - outliers.

Theme 4 Single case Research


73/168

Theme 4 -Single-case Research

Has a long, but largely ignored history in

psychology. Many founding figures in history of

psychology studied individuals Names?

The Single case in History


74/168

The Single-case in History

The intensive study of a single case was used by Broca - 1861, neuropsychology

Bernard - 1865, medicine/physiology

Ebbinghaus - 1885, memory

Straton - 1897, perception

Thorndike - 1898, learning Piaget - 1923, cognitive development

Kohler - 1925, primate cognition

Pavlov 1927, learning Skinner - 1936. Operant behavior

Theme 4 Single case Research


75/168

Theme 4 -Single-case Research

Rejects sampling theory.

Does not average over participants. Does not use NHST-based inference.

BUT

Maintains commitment to quantification.

Uses rigorous (but different) experimental

designs Uses Replication as its key justificatory

procedure

Single case research not just for applied


76/168

Single-case research not just for applied

psychologyYou have two

boats to

experimentwith to winthe Americas

Cup

OR

Evaluating vaccines for cancer


77/168

Evaluating vaccines for cancer

attempted to solve thelong standing problem of

quantifying the results ofcustom-made [cancer]treatments. The courseof the study design was

the first innovation eachpatient was his or her owncontrol

The disease is the remedy Time, 23

April, 2007, p 54

Or you want to evaluateindividually tailoredcancer treatments

Single case Nomenclature


78/168

Single-case Nomenclature

Past

N = 1; own-control, time-series,single-subject, etc

Present

Single-case research designs

Recognizes that the case may be an

individual, a dyad, social group,organizational unit, or some other entity.

Skinner reacted against the adoption of


79/168

Skinner reacted against the adoption of

group statistical designs from thebeginning

It is obvious the kind of science proposed here[in hisbook] belongs on the non-statistical side. inplacing itself in that position it gains the advantage

of a kind of prediction concerning the individual thatis necessarily lacking in a statistical science. Individual prediction is of tremendous importance so

long as the organism is to be treated scientifically.Skinner, B.F. (1936). The behavior of organisms. New

York: Appleton Century [emphasis added].

Formal exposition of single case


80/168

Formal exposition of single-case

research designs

Sidman (1960) Developed experimental laboratory procedures

Based on Skinners work

Hersen & Barlow (1976)

Developed designs for applied settings

Issues about averaging


81/168

Issues about averaging

Is it

The average, or

Variability

that is important?

I believe that the fallacy of reified variation or failureto consider the full-house of all cases plunges usinto serious error again and again. (Gould, 1997).

What does psychology vote for?

Psychology the individual and the


82/168

Psychology, the individual and the

problem of averagingthe individual case is usually forgotten in the

practice of psychological research because itis being replaced by samples of subjects thatare assumed to represent some generalpopulation. Overwhelmingly, psychologistsstudy samples of subjects and proceed togeneralize their findings to the ideal (i.e.,average ) abstract individual.

Valsiner (1986)


83/168

Behavior is defined as a part of the

interaction between organism andenvironment. As such, its occurrence isalways peculiar to individual organisms, and

it is only those peculiarities which are ofscientific interest if our subject matter isbehaviour.


84/168

The problem lies in the generality of group

comparisons to the individual case. Becausebehaviour is a phenomenon that occurs onlyat the individual level, the science of

behaviour must have as its goal theunderstanding of the individual organismsinteraction with its environment.

Johnston & Pennypacker (1980.)

Key precepts of single-case research


85/168

Key precepts of single case research

Sidman (1960) All behavioural processes,

as they occur in nature,

must be evident in thebehaviour of someindividual.

A group function may haveno counterpart in thebehaviour of the individual.

[see Tactics, Chapter 2]

Variability can be explainedwhen it is eliminated (orreduced) by experimentalmanipulation

An example of the problem of averaging


86/168


i.e., a group functionFrom a study of cravings

following quitting

cigarettes.

Top panel shows dataaveraged over

individuals.

Suggests neatness &

order in the groupfunctions



87/168


From a study of cravings

following quitting

cigarettes.

Top panel groupaverages

Bottom shows actual

individual variation.

Averaging


88/168

Averaging

Averaging between cases/subjects poses risks: Average may not represent any actual individual

E.g. average two non-overlapping distributions

Average may create a synthetic rather than a realphenomenon

E.g., averaging over step-like developmental transitions

Averages may create spurious order

E.g., experiences following quitting (above)

Average may misrepresent actual change. E.g., relapse after treatment


89/168

Single-case research designs

Rarely average between participants/subjects May average to condense or reduce data

from an single individual

BUT

Always be cautious that the average is

representative and not synthetic

The logic of Single-case research design


90/168

The logic of Single case research design

Key tasks of any experimental design

1. Detect change: Must permit the detectionof any change in the dependent variable

coincident with the application of theindependent variable.2. Permit drawing valid causal inferences. This

involves defending the claim that the application ofthe independent variable caused the change inthe dependent variable,

Detecting change


91/168

Detecting change

Baseline logic

Single-case, time series data Considers changes in

Trend

Level

Variability


92/168

Baseline

Baseline phase provides for Descriptive function pattern and extent of

target behaviour/problem.

Predictive function project or extrapolatefrom baseline into the immediate future,

assuming that conditions remain the same.

From Bailey & Burch (2002). Research methods in applied behaviour analysis.


93/168

Stability of performance


94/168

Stability of performance

Stability is shown by absence of

Trend (slope) & Variability

in the data.

Trend


95/168

Trend

Stable level trend is ideal

Same direction trends are

problem

Increasing trend

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9

Decreasing trend

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9 10

Trend


96/168

Trend

Trend opposite toexpected intervention is

not problematic

Increasing trend

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9

Decreasing trend

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9 10


97/168

If intervention clearlyincreases rate of change

in trend, this may be OKfor interpretation.

Sudden change in trend

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 1 2 13 1 4 15 1 6 17 1 8

Series1


98/168

Variability

Fluctuation in performance over time. Excessive variability in any phase can mask the effects of any intervention

and make interpretation difficult.

Range of values is large No clear trend

Excessive Variabilty

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9

Sessions

%

BUT


99/168

Variability is a fact of life remember S.J. Gould.

Variability is an aspect of the phenomenon studied

Sometimes variability is what changes with treatment

We should strive to control variability experimentally,not lament it or conceal by averaging

Better experimentation

Careful functional analysis

Level


100/168

Level refers to the

location of the overalldata path relative to thepossible maximum or

minimum values. Baseline too close to

maximum ceiling

effect Baseline too close to

minimum floor effect

Both ceiling and flooreffects may mask anytreatment effect bymaking them hard todetect.

Ceiling effect

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7

Sessions

% Series1

Floor Effect

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7

Sessions

Number

Single-case research design


101/168

g g

Reminder -Key tasks of any experimentaldesign

1. Detect change: Must permit the detectionof any change in the dependent variable

coincident with the application of theindependent variable.

Change detection process


102/168

g p

Occurs via baseline intervention phasecomparisons

All single-case designs are made up ofcombinations of baseline intervention

phases Compare this with standard research designs

minimum is 1 experimental & 1 control group

Apply baseline logic


103/168

pp y g

Use the baseline to

Describe the baselinebehaviour

Consider Level

Trend

Variability.

Apply baseline logic


104/168

pp y g

Use the baseline to

Describe

& then

Predict

what will happen if nointervention

Then, apply intervention

Change detection


105/168

g

? Can we detect anychange between

baseline actual & predicted

& treatment phase?

Change detection


106/168

g

Difference between

phases clearer if highly

variable pointsexcluded.

Notice variability is

decreasing.

Pattern is repeating cause?

Again


107/168

g

Key tasks of any experimental design1. Detect change: Must permit the detection of any

change in the dependent variable coincident withthe application of the independent variable.

2. Permit drawing valid causal inferences.This involves defending the claim that theapplication of the independent variablecaused the change in the dependentvariable,

Making causal inferences


108/168

g

How do we know that the change detectedbetween baseline and intervention phase wascaused by the intervention?

Could just be a coincidence, or caused by

some unknown variable. We build the case for drawing causal

inferences by REPLICATION

Replication


109/168

p

Replication draws for its force on the principleof unlikely successive coincidences(Barlow

et al, 1984). Replication means doing it again and

seeing if the same effect is observed. Replication must be systematic

Different systematic replication procedures

yield different forms of research design

Replication


110/168

p

Replication is of central importance to sciencebut neglected by psychology

A scientific fact should be regarded as experimentally establishedonly if a properly designed experimentrarely failsto give thislevel of significance

[Fisher, 1926, p 504].

given the problems of statistical induction, we [psychologists]

must finally rely, as have the older sciences, on replication.[Cohen, 1994, p 1002]

Visual analysis is not an inferior form of


111/168

analysisVisual analysis can simultaneously detectcurvilinear trends, repeating patterns orcycles in data, delayed or lagged responsesfollowing intervention onset, and within-phase

changes in variability. In addition it can detectchanges in mean level and trend slopeacross phases That breath is not equalled

by any other analytic technique.Parker, R.I., & Hagen-Burke, S. (2007). Useful effect sizes for single case

research. Behavior Therapy, 38, 95-105.

Visual analysis


112/168

Visual analysis vs statistical analysis

Graphs CalculationsFormal Formal

conventions requirements

Visual analysis


113/168

GraphsAxes and phase changelines

Y- axis (dependentvariable) Axis scale

Axis label

X- axis Axis scale

Axis label

Phase (condition)change lines

Phase (condition) labels

Baseline Intervention Baseline

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 1 3 1 4 1 5 16 17 1 8 19 2 0 21 22 23 24 25 2 6

Sessions

%

Series1

Visual analysis


114/168

GraphsData & data paths

Data points Data path(s)

Key (data path label)

NB: 1. data paths do not crossphase change lines

2: Axes show zero

Baseline Intervention Baseline

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 1 3 1 4 1 5 16 17 1 8 19 2 0 21 22 23 24 25 2 6

Sessions

%

Series1

Data point

Designs


115/168

Reversal (aka Withdrawal)

Multiple-baseline Changing criterion

Alternating treatments

Multi-element functional analysis

Designs


116/168

Reversal (aka Withdrawal)

Multiple-baseline Changing criterion

Alternating treatments }

Multi-element functional analysis}

Reversal


117/168

Reversal designs are within- participantdesigns

Put same case through repeated sequencesof baseline and intervention phases

Replication by Withdrawal/Reversal


118/168

Step 1: AB phases give

Baseline -> Intervention

Permits the detection of

change



119/168

The reversal by

withdrawing treatment-

shows a

REPLICATION of thebaseline performance



120/168

Treatment is then

REPLICATED

A complete AB AB

reversal design.

Look for the pattern to be

replicated in each AB

pair.



121/168

Follow-up is not part ofthe replication logic itprovides a demonstrationof the durability of thechange.

? Why is follow-up not just

another reversal generally because ofMaintenance activeaspects of treatment arecontinued in the post-treatment phase.

Within-participant replication can be extended by between-participant replications


122/168

p p p

Again, we look to see if

the pattern replicates

across each AB pair,and across eachparticipant.

Consistency in patternobserved is important:

Consistent successfulreplications buildconfidence that it is thetreatment that caused the

effect


123/168

Variations

ABA

ABAB

AB1AB2AC etc

Problem of order effects

ABAB designs


124/168

Strengths & limitations

Clear demonstration of effect Target behaviour must be able to be reversed

Clinically desirable that treatment end withimprovement

Ethical issues of behaviour worsening.

Practical limits to getting individuals to doreversals but accidental reversals.

Multiple Baseline designs


125/168

Change is evaluated across different baselines

May be across

Different behaviours

Different settings or time periods (contexts/stimuli)

Different individuals

Intervention introduced to baselines at different

points in time

Causality shown by a when and only when pattern

Multiple-baseline across Behaviours


126/168

Three aspects of assertiveverbal behaviour recorded

during role-playsTraining introduced to one

(excuses)

Others remain on baseline

Successive introduction oftraining

Looking for a when and only

when pattern.

Multiple-baseline across Behaviours


127/168

This is then replicated with asecond participant

.

And then with a third

Each replication buildsconfidence in the claim thattreatment produces effect

Multiple-baseline acrossparticipants


128/168

This is the mostcommonly used

design in currentapplied behaviouranalysis

Example babies who

cry persistently atnight, treated byplanned ignoring

Again look for awhen and onlywhen pattern



129/168



Example babies who





130/168



Example babies who





131/168

Note

Post-treatment

response bursts

Floor effect for one

subject

Multiple-baseline across settings or


132/168

contextsOne individual

fearful across a

set of differentsituations(arranged in fear

hierarchy)Each treated with

exposure

therapy



133/168

contextsTreated context

show change

Untreated contexts

show no change

Change happens

when and onlywhen treatment

introduced



134/168

contextsTreated context

show change

Untreated contexts

show no change

Change happens

when and onlywhen treatment

introduced



135/168

contexts

Some

generalizationshows


136/168

Confirmed

byreplication

with new

child

Multiple baselines


137/168

Strengths and limitations

No withdrawal or reversal needed

Number of baselines needed (2 minimum)

Independence of data in baselines & problem ofgeneralization & transfer

Threats from uncontrolled variables (e.g., therapistcontact)

Clinically versatile & friendly designs

Can combine with reversals or other combinations

Changing criterion designs


138/168

Following baseline phase, a criterion for either

reinforcement or punishment is set.

When behaviour level reaches the criterion level, anew criterion is introduced.

Causality is demonstrated by the behaviourconsistently tracking the changing criterion.

Sometimes the direction of the change in criterion is

reversed. May be combined with reversals to baseline

Changing criterion design


139/168

Shows use in studyof reducing

caffeine intakeSolid bar in each

phase shows the

criterion for thatphase

Data path shouldbe at or belowthat criterion

Foxx & Rubionoff (1979)

Another example of changing-criterion


140/168

design Study involved

increasing work

productivity forhandicappedworkers

Number of units of

work per hour wasmeasured.

Combines a final

reversal phase

Changing criterion


141/168


Good for situations involving shaping orwhere behaviour change is gradual

Target behaviour may persistently over- orunder-shoot the criterion and resist furtherchange.

Temporary reduction in criterion is not asproblematic as full reversal.

Alternating treatments


142/168

Some times called multi-element designs

Or simultaneous treatment designs

2 or more treatments are rapidly, andalternatively presented to the participant, inthe same context, and the effects on the

same target behaviour is measured(= multiple schedule in basic operant research)


143/168

Relies on each treatment being signalled bydistinctive discriminative stimuli

Usually preceded by a common baseline

If one treatment emerges as superior, then

effects can be replicated by introducing thattreatment in place of the other treatments

Then ends in a final best treatment phase.


144/168

Note

Middle phase

compares twotreatments toeach other and

to baselineLast phaseswitches to thebest treatment,and compares tobaseline



145/168

Minimises order effects

No treatment withdrawal

Quick comparison of treatments Reduces problem of non-reversability of behaviour

Can assess generalization or transfer effects

Does not require initial baseline Does require that the alternatives be clearly

discriminated.

Functional analysis


146/168

Concerned with the purpose of behaviour E.g., eating to get nutrition when hungry

but emotional eating not related to nutrition or

hunger, so has a different function.More specifically

Describes a relationship between an

[environmental] variable and a behaviour,such that the behaviour varies as a functionof the specified variable E.g rate of behaviour changes as rate of

reinforcement changes

Behaviour rate = f(reinforcement rate)


147/168

Functional analysis has provided a means todetermine in advance which treatmentsshould and should not work, as well as why.

Hanley et al. (2003). Functional analysis of problem behaviour: A review. Journal ofApplied Behavior Analysis, 36, 147 185.

Functional analysis techniques


148/168

Functional analysis interviews

Where, when, who, what questions.

Diaries/Antecendent-Behavior-Consequencelogs

Analogue experimental functional analysis Often uses multi-element single case designs

Rapid alternation of 2+ conditions (mixed order) May use rather standard protocols

For problem/SIB: Standard protocols


149/168

Alone condition Play condition

Tangible condition (e.g., access to toy afterbehaviour)

Social reinforcement/Attention condition (e.g.,

positive social attention given afterbehaviour)

Escape condition (e.g., social attention and/or

task demands removed after condition) Conditions are presented briefly (5-15mins)

and repeated in different orders several times

Examples of Analogue experimental


150/168

functional analysis5 yr old with ADHD who ran

away from recreation

activities Attention intense social

attention from teacherwhen retrieved

Escape allowed to sitand face away fromactivity for 30 sec

Control non-contingentsocial attention

Kodak et al (2003)

Examples of Analogue experimental


151/168

functional analysisConditions were

Ignore

Social attention Escape demands

Control (alone)

Problem disruptions in work groups.

From Wallace & Knights (2003) JABA 36.

Problem: self injury,

aggression & property


152/168

destruction by inpatientadults withdevelopmental

disabilities.

Escape

Social Attention

Tangible

Free play} Alone }

The merits of single-case research


153/168

Gives a scientific basis for understanding andhelping individuals

gets round the difficulty of drawing inferences

from samples to populations, and then somehow

making a valid deduction to an individual(Valsiner, 1986).

Reveals actual variance in response of individuals

to treatment


154/168

resolves the difficulty of reconciling statistical andclinical significance

visual analysis generally needs clinically

significant changes before change is detected.


155/168

Avoids the problems of averaging betweenindividuals

avoids creating synthetic rather than real

psychological processes (because psychological

processes must occur within individuals)

accurately describes varying trajectories of

change


156/168

Brings the rare and unusual within the scopeof science

uniqueness of each individual is recognised

rare cases can be studied scientifically, which is

difficult if one has to recruit large samples.


157/168

Gives meaning to the ideal of the scientist-practitioner and to evidence-based practice

almost every case studied can be the basis for

making valid causal inferences


158/168

May establish both efficacy and effectiveness

Efficacy treatment works in high-qualityresearch trials (RCT or single-case Chambless

& Hollon criteria) Treatment manual

Intensive training & supervision of therapists

Involvement of highly qualified researchers/clinicians

Sited in research clinics/hospitals

Selected participants/pure diagnoses


159/168

Effectiveness works in actual clinical practice

Most therapists can learn to use it

Withstands some variation in use (Manualadapted/discarded)

Routine supervision/clinical support

Wide range of clinical settings Wide range of participants with multi-diagnoses

Single-case designs can be used in bothefficacy and effectiveness contexts.

Third component is Acceptability and SocialValidity of the treatment


160/168

y Social validity refers to

The social significance of the goals

The social appropriateness of the procedures The social importance of the effects

Acceptability refers to

Fairness Reasonableness

Intrusiveness

Perceptions of appropriateness/ethics, etc

Measure Kazdins Treatment Acceptability Index


161/168

Facilitates ethical innovation & accountability

new treatments can be investigated withoutexposing large numbers to unknown risks

no need to keep large numbers of suffers waiting

in the control group.

Can map the range of normal environments and

contexts in which the problem occurs and the

intervention is used move from efficacy toeffectiveness research


162/168

Is an efficient way to map the generality oftreatments

Systematic replications over the various

dimensions of experimental variability can

discover where the intervention fails

Does not require recruitment of large numbers of

participants, nor expose them to risk of failure

Is much less expensive and time-consuming thatdoing multiple randomised clinical trials


163/168

Ethical Accountability - Makes it easier forresearchers & practitioners to be accountablefor their work

results are directly available for each treated individual no knowledge of complex statistics required to

understand outcomes.

Outliers and failures are not hidden in groupaverages

Conclusions


164/168

1. Research design and data analysismatters!

Conclusions


165/168

2. Psychology needs multiple methodsmatched to the questions being asked

Description of abstract types at population level

use Fisherian stats

Exploration of predictive relationships atpopulation level use regression/correlation

techniques

Actuarial, public health, some public policy asabove

BUT clearly state the population sampled from!


166/168

3. For intervention science

Use single-case research designs to develop

innovations

Use single-case designs to map and establish

generality and limits

Use single-case designs in routine practice to be

ethical and accountable

Use RCTs sparingly, to establish comparativeeffectiveness of several treatments


167/168

4. In all research

Be aware of variance and distributions

Focus on effect sizes

Focus on clinical significance

Focus on durability and generality of effects

Dont be fooled by p < 0.0XX!

And for rich description, use qualitative methods

to supplement quantitative experimental methods

Darwinism implies an intense awareness that


168/168

all categorical or essentialist claims about livingthings are overdrawn Repetition is the habit ofnature, but variation is the rule of life.

belief in the primacy of the single case isnot an illusion nurtured by fancy but a hopequietly underscored by science. The general

case is the tentative abstract hypothesis; thecase right there is the real thing.

Gopnik, A. (2009). Angels and Ages: A Short Book about Darwin, Lincoln, and Modern Life.

London, UK: Quercus. (p. 197-198).

Documents

NB Single Case Designs All 2010 6 Slides Pp Handout