Why use mixed methods? Webinar with Dr. Schutt

Los Angeles | London | New Delhi

Singapore | Washington DC

Dr. David Fetterman

November 20 2014 #SAGEtalks

Please be sure to check your settings

Make sure your volume is set appropriately

Make sure you have followed the instructions on your keypad properly

Make sure everything is plugged in properly to assure your devices are working correctly

If you continue to have audio or

visual difficulties, please let us know

via the chat box and one of our

SAGE employees will be happy to

assist you.



Why Use Mixed Methods?

Content and Presentation byRussell K. Schutt



Dr. David Fetterman


Before we get started…

Let’s take a

moment to answer

2 quick questions



Dr. David Fetterman


Dr. Russell K. Schutt

University of Massachusetts, Boston

Erica DeLuca

Executive Marketing Manager, SAGE



Dr. David Fetterman


While we do our best to answer as many questions as we can, time constraints

may not allow us to answer every question. Thank you for understanding.

Send us your questions!

Using Twitter? Use the hashtag #SAGEtalks.

Send in your questions via the Chat Box on your

screen. →

Russell K. Schutt, Ph.D.

University of Massachusetts Boston

Why Use Mixed

Methods?

The home of Mr. and Mrs. Henry Adams Breckenridge…three stories

topped by a captain’s walk…. Large trees and a tall thick hedge…garden

stretches one hundred yards…many old rose bushes. …The life and

surroundings, old-family and upper-upper,… Her [I.S.C.] ratings give her

a final score of 12, or perfect…. (Warner 1960. Social Class in America.)

Long exploratory interviews with key informants, …the actual political life

of the union, attending union meetings…. … At this point it seemed that

crucial aspects of the internal political process could best be studied

through survey research methods, 500 interviews …. (Lipset, Trow,

Coleman 1956. Union Democacy.)

Such complexity and interdependency requires agile research strategies

…assess causal factors at multiple levels, flexibly incorporate new

information as it arises. Enabling creative and productive conversation:

qualitative, quantitative measurement; analytic modalities. (Brown 2013)

Mixed Methods Past & Present

http://www.hup.harvard.edu/

Outline

1. The Research Question

2. Mixed Methods

3. Findings

a. Consumer and clinician preferences

b. Housing type

c. Social processes

d. Interaction effects

4. Conclusions

The Research Question

1. Originality

2. Complexity

3. Ambiguity

4. Authenticity

Hypotheses & Question

Client outcomes will be more favorable in

group than in independent housing.

Client outcomes will be more favorable if

client and clinician housing choice match.

By what process do group homes evolve

to consumer-operated households?

GROUP HOME: A traditional community

residence for a group of individuals with

chronic mental illness. 24 hour supervision

with awake overnight staff.

INDEPENDENT APARTMENT: A

supported housing program serving

individuals who require mental health and

community services.

Originality: Housing Comparison

Social integration protective for suicide(Durkheim).

Loneliness: depressive symptoms, chronic health conditions, elevated blood pressure, stress, helplessness, social problems(Cacioppo & Patrick 2008)

Social stimulation & neurogenesis (Kempermann, Brandon & Gage 1998)

Social interaction & rehabilitation (Kern et al. 2009)

67% - 90% homeless singles choose living alone (Neubauer 1993; Owen et al., 1996; Tanzman 1993).

Practicality: Policy Relevance

Consumer preference is a key theme of Council innovations. (Interagency Council, Homeless 2008)

Housing First: “Service plans are not based on clinician assessments of consumers’ needs but driven by consumers’ own treatment goals.” (Tsemberis 2010)

Mainstream housing where persons live alone and manage in their own apartments by themselves is beyond the capability of the great majority. (Lamb, 1990)

Authenticity: A Mechanism

6-25% lose independent housing within one year.

Up to 50% lose housing after five years.

Very intensive services lower the 5-year risk to 25%.

Long-term housing loss higher for dually diagnosed.

No clear advantage of a specific housing type.

(Kasprow et al., 2000; Kertesz et al. 2009; Leff et al. 2009; Lipton et al., 2000; Lipton, Nutt and Sabitini, 1988:43; O’Connell et al. 2008; Padgett, Gulcur and Tsemberis, 2006; Shern et al., 1997; Siegel et al. 2006; Stefanic and Tsemberis 2007)

Mixed Methods

a. Design type

b. Measurement

c. Case selection

d. Experimental design

e. Process analysis

f. Contextual analysis

Mixed Method DesignsPriority

Prioritized Equal

Sequencing

Sequential Staged Method

QualQUAN

QuanQUAL

QUALquan

QUANqual

Research Program

QUALQUAN

QUANQUAL

Concurrent Embedded Method

QUAL(quan)

QUAN(qual)

Integrated Method

QUAL+QUAN

Schutt 2015: 545

Preferences (α = .72)If you now had a choice of living with others in a shared

residence or alone in your own apartment, which would you prefer? 1 = Group living 2 = Apartment a. How strongly?

How would you feel about having staff come in just during the day and help with cooking, cleaning and shopping? (1-5)

Ethnographic Observation; Clinician observations

Recommendations (α = .84)Overall, taking into account all of your sources of information, do

you believe that this person will do better clinically living in an evolving consumer household or in an IL? (1-5)

Clinician comments (inter-rater r = .66 - .91)Behavioral risk; Needs support,Needs structure); Social withdrawal); Poor insight; Substance abuse

Case Selection: The Sample

0

20

40

60

80

100

Research (Experimental) Design

Group Group

Apt.

Group

Apt.

Baseline 6 Mos. 12 Mos. 18 Mos.

PSC

PSW

BVI

DMH

Shelters

Apt.

3 yrs,

20 yrs.

Neuropsych testing Ethnographic Observation Neuropsych

Clinician Recommendation, comments Life Skills Profiles

Process Analysis: Evolution

Staff Resident

Group Traditional

Group

Consumer-

Run

Single Supported

Living

Independent Apartments

ControlTen

an

ts

Measures of Context

Measure Description Baseline Value

Lifetime Substance

Abuse (38)

SCID-based, scored as no use, some

use, abuse or dependence

61.3% abuse or

dependence

Clinician Housing

Recommendation

(39)

Average of answers to nine

questions by two independent raters,

scored 1-5

Mean=3.18,

s.d.=.46.

Cronbach’s alpha = .84

Findings

1. Consumer & Clinician Preferences

2. Housing type

3. Social processes

4. Interaction effects

CONSUMER PREFERENCES,

CLINICIAN RECOMMENDATIONS

Clinician Recommendations,

Consumer Residential Preferences

0

10

20

30

40

50

60

70

80

90

100

FT Staff Indep Apt

Clinician A

Clinician B

Consumers

Consumer Preferences by Clinician Recommendations

“Ability to organize thoughts good; can successfully live either setting; history of independence.”

“Inability to manage money, no insight; anger, hostility, limited skills; polysubstance abuse; high risk”

Clinicians Recommended Independent Living

Participates in meetings, school, active outside of

house; No meds, self-medicating; Got own apartment;

High functioning; Sociable, active, talkative; Motivated.

Clinicians Recommended Group Living

Low self-esteem, paranoid; Drug abuse, in and out of

detox; Isolated, angry, alcoholic, antisocial, abusive;

Cocaine use in house (so expelled); Difficult, into

pornography and drugs.

HOUSING TYPE

(CAUSAL EFFECT)

0

10

20

30

40

50

60

70

McKinney 18Mos.

McK in MetroDB

Metro DB McK & Metro

Group

Indep

Any Homelessness by Housing

Type & Followup

SOCIAL PROCESSES

(MECHANISM)

Staff engagement planned outings

expressive art activities; basement recreation center

simple birthday celebrations ; Thanksgiving dinner

modeling behavior

Tenant activities group shopping trips, group meals, chore days

talking and laughing together; parties

Meetings divergent opinions ; friendly and supportive.

shared responsibilities; voting for new staff member

planning group meals and shopping

“Things have really come together, … we're

working together as a group more.”

“Do you know how much help I asked for

today [making dinner]? I never did that

before!”

“People are really hanging out together—

talking, helping each other out.”

“People still grumble, but things get resolved

now. I've even heard people apologize...”

A Case Study of Improvement

She did not seem to have close relations to

anyone in the house, just sitting in a chair…didn’t

get out of the house, apprehensive toward doing

things independently. She attended all meetings

but rarely participated. High functioning in self-

care. After a while, started to become slightly

more involved, cooked a group meal, participated

in a homelessness demo. She engaged in

weekly outings with female staff and residents.

Then she became more social, joking more

frequently and participating more in meetings.

Finally she was more independent outside the

house and felt comfortable reducing house staff.

Anti-Social Experiences Substance abuse

tension and emotional outbursts in meetings

theft to support drug use; dealers in house

Psychiatric symptoms expressions of bizarre ideas

loud, abrupt, screaming in your face

Reactions to staff complaints about staff “telling me when to play the

stereo and how loud”

Disputes over medication, rep payee status, guests

Interpersonal tension rudeness; harassing women, incessant swearing

loud music; TV control; not contributing to house kitty

Negative Social Experiences

“She finds it difficult to know when

someone is going to lose their temper with

her all of a sudden.”

“These people just don't know how to have

normal human relations.”

A Case Study of Deterioration

The resident was causing conflict…tenants

complaining she didn’t do her share of

housework, played loud music late at night, drank

in the house, and got into lots of arguments. She

missed many meetings and got defensive when

people brought up disruptive things she does, but

other tenants were afraid to confront her. She did

not respond to a staff “ultimatum” or to a tenant

letter asking her to change and blamed her

problems on her traumatized past. Conflict

continued over her drinking and enforcing house

rules. Finally, she was asked to leave.

INTERACTION EFFECTS

(CONTEXT)

% Days Homeless by Substance

Abuse, Race, & Housing Type

0

10

20

30

40

50

60

70

80

None Some Abuse None Some Abuse

Group

Indep

Minority White

% Days Homeless by Housing

Type & Preference/Rec.

0

5

10

15

20

25

30

35

Both Prefer

Apartment

Consumers

Prefer Apt,

Clinicians Say

Group

Consumers

Prefer Group,

Clinicians Say

Apt

Both Prefer

Group

Group

Indep

Change in Executive Functioning by

Housing & Clinician Recommendation

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Rec=Group Rec=Indep

Group

Indep

Change in Executive Functioning

by Housing & Substance Abuse

0

0.5

1

1.5

2

2.5

3

3.5

4

Time 1 Time 2 Time 3

ECH-NoSubs

IL-NoSubs

ECH-Subs

IL-Subs

Conclusions

Substantive conclusions

Methodological lessons

Substantive Conclusions

Group Housing Maximizes Housing Retention, Cognition

Consumer Preferences Do Not Predict Optimal Placement

Clinicians Can Predict Need for Support

Rejection of Needed Support Predicts Housing Loss

Social Interaction Helps Some Regain Stability

Rejection of Needed Support and Substance Abuse Interfere with Cognitive Benefits from Social Process

Individual Orientations May Challenge Social Process

Methodological Lessons

Research questions must correspond in complexity to the social world

Research vision constrained by limited methods

Mixed methods transform and enrich understanding of measures & causal process

Interactions reveal context with mixed methods

Mixing methods can be an iterative process, in design or analysis, thus allowing exploration and confirmation of emerging patterns

Mixed methods improve authenticity and theory

Boston McKinney ProjectInvestigators

Stephen M. Goldfinger, MD (PI); Russell K. Schutt*, PhD; Larry J. Seidman, PhD; Barbara Dickey, PhD; Walter E. Penk, PhD; Norma Ware, PhD; Sondra Hellman, RN, MS, Martha O’Bryan, RN

Research Staff

Brina Caplan**, EdD, PhD; Win Turner, PhD,

George Tolomiczenko, PhD; Mark Abelman, MSW

Funding

National Institute of Mental Health, HUD

*UMass Boston

**NARSAD

Blog

Investigatingthesocialworld.com



Michael Quinn Patton

December 2014 #SAGEtalks

While we do our best to answer as many questions as we can, time constraints

may not allow us to answer every question. Thank you for understanding.

Send us your questions!

Using Twitter? Use the hashtag #SAGEtalks.

Send in your questions via the Chat Box on your

screen. →



Michael Quinn Patton

December 2014 #SAGEtalks

Webinar recording available on www.sagepub.com/sagetalks.

Why Use Mixed Methods?

Russell K. Schutt

• Blog

• Website

Be sure to check our website for updates on our 2015 Spring webinar series!

http://www.sagepub.com/sagetalks

http://investigatingthesocialworld.com/

http://rschutt.wikispaces.umb.edu/

Education

Why use mixed methods? Webinar with Dr. Schutt