Research Methods for the Social Sciences: An Introductory Course January 19 th, 2010 – Class Introduction Division on Addictions, Cambridge Health Alliance

Research Methods for Research Methods for the Social Sciences:the Social Sciences:

An Introductory CourseAn Introductory Course

January 19January 19thth, 2010 – Class , 2010 – Class IntroductionIntroduction

Division on Addictions, Cambridge Health AllianceDivision on Addictions, Cambridge Health AllianceHarvard Medical SchoolHarvard Medical School

Course ObjectivesCourse Objectives

Learn to critically evaluate empirical Learn to critically evaluate empirical researchresearch

Learn to identify critical components Learn to identify critical components and potential pitfalls of research and potential pitfalls of research designsdesigns

Learn to determine the appropriate Learn to determine the appropriate research design for a given research research design for a given research questionquestion

Gain the foundation to begin to design Gain the foundation to begin to design own researchown research

Course ScheduleCourse Schedule 1/19: Course Introduction 1/19: Course Introduction (Sarah Nelson, PhD)(Sarah Nelson, PhD) 1/26: Research Design Overview 1/26: Research Design Overview (Heather Gray, (Heather Gray,

PhD)PhD) 2/2: Research Design Part II – Data Collection 2/2: Research Design Part II – Data Collection

and Procedures and Procedures (John Kleschinsky, MPH)(John Kleschinsky, MPH) 2/9: Measurement and Validity 2/9: Measurement and Validity (Julia Braverman, (Julia Braverman,

PhD)PhD) 2/16: Crash Course in Statistics 2/16: Crash Course in Statistics (Evelyn Chao, PhD)(Evelyn Chao, PhD) 2/23: More on Design – Causality 2/23: More on Design – Causality (Nelson)(Nelson) 3/2: The Context of Research 3/2: The Context of Research (Debi LaPlante, PhD)(Debi LaPlante, PhD) 3/9: Ethics 3/9: Ethics (Ryan Martin, PhD)(Ryan Martin, PhD) 3/16: Practical Notes 3/16: Practical Notes (Course Faculty)(Course Faculty) 3/23: Conclusion 3/23: Conclusion (Nelson)(Nelson)

Course WebpageCourse Webpage

www.divisiononaddictions.org/methodswww.divisiononaddictions.org/methodscourse.htmcourse.htm (also available through the education (also available through the education section of the Division’s website)section of the Division’s website)

Powerpoints and suggested readings Powerpoints and suggested readings will be posted. will be posted.

You will need a password to access the You will need a password to access the material. Password is “methods”.material. Password is “methods”.

Website will be functional by the end Website will be functional by the end of this week.of this week.

AssignmentAssignment

By the end of this week, please select By the end of this week, please select one scientific article from your area of one scientific article from your area of specialty that you would be interested specialty that you would be interested in reading critically as part of this in reading critically as part of this class. Email the reference to class. Email the reference to [email protected]@hms.harvard.edu. .

““It ain't what you don't knowIt ain't what you don't knowthat gets you in trouble;that gets you in trouble;it's what you know for sureit's what you know for surethat ain't so.”that ain't so.”

Mark TwainMark Twain

Witt, P. N. (1956). Die wirkung von substanzen auf den netzbau der spinne als biologischer test. Berline-Gottinger-Heidelberg: Springer-Verlag.

NAME THAT DRUG!

1) Hashish

2) LSD

3) Caffeine

4) Mescaline

B.

C. D.

A.

NAME THAT DRUG!

1) Hashish

2) LSD

3) Caffeine

4) Mescaline

B. LSD

C. Caffeine D. Mescaline

A. Hashish

Baseball & ScienceBaseball & Science

“Baseball—of all things—was an

example of how an unscientific culture responds, or fails to respond, to the scientific method”

Michael Lewis, Moneyball, 2003, p.xiv

How Do Baseball Scouts How Do Baseball Scouts Typically Identify “Talent”?Typically Identify “Talent”?

“…“…there was a bias toward what there was a bias toward what people saw with their own eyes, or people saw with their own eyes, or thought they had seen. The human thought they had seen. The human mind played tricks on itself when it mind played tricks on itself when it relied exclusively on what it saw… relied exclusively on what it saw… There was a lot you couldn’t see There was a lot you couldn’t see when you watched a baseball when you watched a baseball game”game”

(Lewis, 2003, p.18)(Lewis, 2003, p.18)

Revised Models of What Revised Models of What Predicts Winning in BaseballPredicts Winning in Baseball

Runs scored is best predictor of Runs scored is best predictor of winningwinning

Best predictor of runs Best predictor of runs

scored is on-base %, scored is on-base %,

not batting average, not batting average,

ERA, defense, etc.ERA, defense, etc.

If our eyes can deceive us to If our eyes can deceive us to thinking that certain key thinking that certain key “baseball” factors are important, “baseball” factors are important, when often they are not, then… when often they are not, then…

Our eyes can deceive us similarly Our eyes can deceive us similarly to thinking that certain key factors to thinking that certain key factors are integral to other behaviors we are integral to other behaviors we observe or experiences we study, observe or experiences we study, when often they are notwhen often they are not

What Does Baseball Have to What Does Baseball Have to Do with Research Methods?Do with Research Methods?

Often our Often our definitions of definitions of concepts rely on concepts rely on non-integralnon-integral aspects of those aspects of those concepts.concepts.

A Bald Girl?A Bald Girl?

Marlo Thomas and Friends, 1974. Free to Be, You and Me

Flawed ResearchFlawed Research ““Cute feet, small, dainty, yup, I’m a Cute feet, small, dainty, yup, I’m a

girl”girl”– [No comparison group] [No comparison group]

““Bald – you’re bald, fella, bald as a Bald – you’re bald, fella, bald as a ping pong ball” ping pong ball” – [Predictive at one age, not all][Predictive at one age, not all]

““You’re bald, too!” “Yech, a bald girl You’re bald, too!” “Yech, a bald girl – disgusting.” – disgusting.” – [Dropping initial assumption in order to [Dropping initial assumption in order to

support theory]support theory] ““You just shaved, right?” “Wrong!” You just shaved, right?” “Wrong!”

“Exactly!” “Exactly!” – [Could have used evidence either way][Could have used evidence either way]

The Dangers of CHCThe Dangers of CHC

1.1. More than 98 percent of convicted felons have tested More than 98 percent of convicted felons have tested positive for CHC. positive for CHC.

2.2. CHC is associated with all the major diseases of the CHC is associated with all the major diseases of the body. For example, nearly all sick people have ingested body. For example, nearly all sick people have ingested CHC. The effects are obviously cumulative: 99.9% of CHC. The effects are obviously cumulative: 99.9% of all people who die from cancer have ingested CHC.all people who die from cancer have ingested CHC.

3.3. 99.7% of the people involved in auto accidents used a 99.7% of the people involved in auto accidents used a substance containing CHC within 6 months preceding substance containing CHC within 6 months preceding the accident.the accident.

4.4. 93.1% of juvenile delinquents come from 93.1% of juvenile delinquents come from neighborhoods where CHC is used frequently.neighborhoods where CHC is used frequently.

From Fraley, C. (http://www.uic.edu/classes/psych/psych242f/spring2004)

The Dangers of BreadThe Dangers of Bread

1.1. More than 98 percent of convicted felons are bread More than 98 percent of convicted felons are bread users. users.

2.2. Bread is associated with all the major diseases of the Bread is associated with all the major diseases of the body. For example, nearly all sick people have eaten body. For example, nearly all sick people have eaten bread. The effects are obviously cumulative: 99.9% of bread. The effects are obviously cumulative: 99.9% of all people who die from cancer have eaten bread.all people who die from cancer have eaten bread.

3.3. 99.7% of the people involved in auto accidents ate 99.7% of the people involved in auto accidents ate bread within 6 months preceding the accident.bread within 6 months preceding the accident.

4.4. 93.1% of juvenile delinquents came from homes where 93.1% of juvenile delinquents came from homes where bread is served frequently.bread is served frequently.

From Fraley, C. (http://www.uic.edu/classes/psych/psych242f/spring2004)

Konrad Lorenz Konrad Lorenz ((Austrian ethologist)Austrian ethologist)

“It is a good morning exercise for a research scientist to discard a pet hypothesis every day before breakfast. It keeps him young.”On Aggression, ch. 2 (1963; tr. 1966).

Problems with Personal Problems with Personal ExperienceExperience

We tend to seek information that is consistent We tend to seek information that is consistent with our beliefs and ignore inconsistent with our beliefs and ignore inconsistent information (bald girl)information (bald girl)

We tend to base our inferences on little We tend to base our inferences on little information (anecdote)information (anecdote)

Our expectations tend to influence what we see Our expectations tend to influence what we see (bald girl, CHC)(bald girl, CHC)

We’re not attentive to base rates and We’re not attentive to base rates and comparisonscomparisons– We overlook the fact that inferences about We overlook the fact that inferences about

contingencies require some kind of comparison or contingencies require some kind of comparison or standard. (CHC)standard. (CHC)

Adapted from Fraley, C. (http://www.uic.edu/classes/psych/psych242f/spring2004)

Problems with Personal Problems with Personal ExperienceExperience

The confirmation bias helps to The confirmation bias helps to ensure that, once an expectation or ensure that, once an expectation or theory has been developed, the theory has been developed, the belief will be self-perpetuating.belief will be self-perpetuating.


The Scientific MethodThe Scientific Method

The scientific method is a way of The scientific method is a way of dealing with these concerns.dealing with these concerns.

Science is theScience is the process of process of constructing, testing, and refining constructing, testing, and refining theories about natural phenomena theories about natural phenomena though the use of systematic though the use of systematic empirical observationempirical observation..


Scientific Method, DefinedScientific Method, Defined

The collection of data through The collection of data through observation and experimentation and observation and experimentation and the formulation and testing of the formulation and testing of hypotheses. hypotheses. (Merriam-Webster’s Dictionary)(Merriam-Webster’s Dictionary)

Scientific Method as Self-Scientific Method as Self-CorrectingCorrecting

theory about how something

works

systematic empirical

observations

testing: comparing the observations

with the theory

generate predictions

(what would the theory lead you to

observe?)




works


observations


with the theory



observe?)


ensures that inconsistent

observations will be recorded



works


observations


with the theory



observe?)


ensures that inconsistent

observations will be counted against the

theory



works


observations


with the theory



observe?)


theory is revised in light of the tests, and, hopefully, becomes more

accurate

Scientific MethodScientific Method

TheoryTheory– An organized set of principles that An organized set of principles that

describes, predicts, and explains some describes, predicts, and explains some phenomenon.phenomenon.

HypothesisHypothesis– A specific testable prediction, often A specific testable prediction, often

derived from a theoryderived from a theory

from Kassin, 1998, Psychology (2nd Edition).


Science is a Science is a processprocess for for understanding the world.understanding the world.

It is It is notnot a topic or subject matter. a topic or subject matter. Some people believe that chemistry, Some people believe that chemistry, for example, is inherently scientific. for example, is inherently scientific. However, one can study chemical However, one can study chemical phenomena in scientific ways or in phenomena in scientific ways or in non-scientific ways.non-scientific ways.



Step 1. Observe behavior or other phenomenaStep 1. Observe behavior or other phenomena

InductionInduction– Involves using a relatively Involves using a relatively smallsmall set of specific set of specific

observations as the basis for forming a observations as the basis for forming a general general statement about a larger set of possible statement about a larger set of possible observations. observations.

People at Cambridge Health Alliance are not People at Cambridge Health Alliance are not very interested in learning about research very interested in learning about research methodsmethods

Adapted from research methods lecture by Julia Braverman


Step 2. Form a Tentative answer or Step 2. Form a Tentative answer or explanationexplanation

Initial HypothesisInitial Hypothesis

People at Cambridge Health Alliance do People at Cambridge Health Alliance do not sign up for research methods because not sign up for research methods because they do not value sciencethey do not value science



Step 3. Use hypothesis to generate a Step 3. Use hypothesis to generate a testable predictiontestable prediction

Deduction (From big to small)Deduction (From big to small)– Uses a general statement as the basis for Uses a general statement as the basis for

reaching a conclusion about specific examples. reaching a conclusion about specific examples.

If people at CHA do not sign up for If people at CHA do not sign up for research methods because they do not research methods because they do not value science, then changing the date, value science, then changing the date, time, or location of the class should not time, or location of the class should not alter enrollment.alter enrollment. Adapted from research methods lecture by Julia Braverman

Scientific MethodScientific Method Step 4. Evaluate the prediction by making Step 4. Evaluate the prediction by making

systematic planned observations systematic planned observations – Compare the predictions with observations. Compare the predictions with observations.

Changing the date, time, and location Changing the date, time, and location increased enrollment significantlyincreased enrollment significantly

Lack of agreement:Lack of agreement:– Revise the hypothesisRevise the hypothesis

People at CHA do not sign up for research People at CHA do not sign up for research methods if it is not located at Cambridge methods if it is not located at Cambridge HospitalHospital


ObservationHypothesis

Prediction

Planned observations

New hypothesis

New prediction

Planned observations

Scientific Theory


Evaluating ResearchEvaluating Research

Is the evidence being forced to fit the Is the evidence being forced to fit the hypothesis?hypothesis?

Is the interpretation appropriate, given Is the interpretation appropriate, given the results?the results?

What can the study tell us about the What can the study tell us about the relationships it measures?relationships it measures?

What unmeasured variables might affect What unmeasured variables might affect the results?the results?

Exercise: Jumping to Exercise: Jumping to Conclusions Conclusions

In each of the following situations, the conclusion may be erroneous or In each of the following situations, the conclusion may be erroneous or is is

not justified by the facts. In the space provided, describe the error or not justified by the facts. In the space provided, describe the error or errors in thinking or methodology that invalidate the conclusion and errors in thinking or methodology that invalidate the conclusion and suggest changes that could be made in the study that might allow for suggest changes that could be made in the study that might allow for

the the conclusion given. conclusion given.

A clinic sponsoring a Stop Smoking treatment A clinic sponsoring a Stop Smoking treatment program surveys all the people who completed program surveys all the people who completed the program during its first year. The survey the program during its first year. The survey revealed that 74% of the respondents were still revealed that 74% of the respondents were still not smoking three months after completing the not smoking three months after completing the program. In their next advertising campaign, they program. In their next advertising campaign, they claim a 75% success rate for those who enroll in claim a 75% success rate for those who enroll in their program. their program. Copyright © 1996 Andrew Geoghegan

Exercise: Jumping to Exercise: Jumping to Conclusions Conclusions

In each of the following situations, the conclusion may be erroneous In each of the following situations, the conclusion may be erroneous or is or is

not justified by the facts. In the space provided, describe the error not justified by the facts. In the space provided, describe the error or or

errors in thinking or methodology that invalidate the conclusion and errors in thinking or methodology that invalidate the conclusion and suggest changes that could be made in the study that might allow suggest changes that could be made in the study that might allow

for the for the conclusion given. conclusion given.

An investigator gave a personality test to a An investigator gave a personality test to a large number of crack cocaine addicts. The large number of crack cocaine addicts. The addicts scores on the trait of "emotionality" addicts scores on the trait of "emotionality" were significantly lower than the scores on were significantly lower than the scores on that trait obtained from the general that trait obtained from the general population. This caused the investigator to population. This caused the investigator to believe that highly unemotional people are believe that highly unemotional people are more susceptible to crack cocaine addiction. more susceptible to crack cocaine addiction.

Copyright © 1996 Andrew Geoghegan

Media Headlines Media Headlines 1.1. Soda consumption could lead to Soda consumption could lead to

medical problems (wvec.com)medical problems (wvec.com)

2.2. Kids' Urge To Drink Boosted By Kids' Urge To Drink Boosted By Outdoor Alcohol Ads (Medical News Outdoor Alcohol Ads (Medical News Today)Today)

3.3. Teen Girls Using Pills, Smoking More Teen Girls Using Pills, Smoking More Than Boys (Washington Post)Than Boys (Washington Post)

Soda consumption could lead Soda consumption could lead to medical problems to medical problems

An all-American drink, soda, is now raising some real medical concerns according to a new An all-American drink, soda, is now raising some real medical concerns according to a new study. A Framingham Heart study of more than 6,000 people was released Monday study. A Framingham Heart study of more than 6,000 people was released Monday

afternoon. afternoon. According to it, if you’re middle-aged and drink more than one soda a day, you may have a According to it, if you’re middle-aged and drink more than one soda a day, you may have a more than 40% greater rate of either having or developing metabolic syndrome; a cluster of more than 40% greater rate of either having or developing metabolic syndrome; a cluster of conditions that increase the risk of heart disease. Researchers discovered that people who conditions that increase the risk of heart disease. Researchers discovered that people who drank one or more sodas each day had a 30% increased risk for obesity, a 25% greater risk drank one or more sodas each day had a 30% increased risk for obesity, a 25% greater risk for abnormal blood sugar levels and a 32% increase in the odds of having low levels of good for abnormal blood sugar levels and a 32% increase in the odds of having low levels of good cholesterol. All three contribute to heart disease. cholesterol. All three contribute to heart disease.

Former Ghent resident Tim Minger is already cutting back, which is why he and his wife Former Ghent resident Tim Minger is already cutting back, which is why he and his wife didn’t didn’t

drink soda at their Monday lunch. His wife, Agnes, say Tim's soft drink addiction was really drink soda at their Monday lunch. His wife, Agnes, say Tim's soft drink addiction was really bad. "Like a twelve pack, ok, every other day a twelve pack of Mountain Dew,” said Agnes bad. "Like a twelve pack, ok, every other day a twelve pack of Mountain Dew,” said Agnes Aldana. “I'm sort of addicted,” said Tim. “I drink at least everyday. I've recently switched to Aldana. “I'm sort of addicted,” said Tim. “I drink at least everyday. I've recently switched to diet. That's my concession. diet. That's my concession.

Here's the clincher; researchers say even drinking diet soda may increase the risk of heart Here's the clincher; researchers say even drinking diet soda may increase the risk of heart disease. They can't explain why. disease. They can't explain why.

The researchers say it's worth noting that people who drink both regular and diet soft drinks The researchers say it's worth noting that people who drink both regular and diet soft drinks have a tendency to eat the same kind of food. This study on soda is published in the on-line have a tendency to eat the same kind of food. This study on soda is published in the on-line edition of the Journal of Circulation. edition of the Journal of Circulation.

Kids’ urge to drink boosted by Kids’ urge to drink boosted by outdoor ads outdoor ads

In the world depicted in an alcohol billboard, bikini-clad babes clutch icy bottles, frothy beer In the world depicted in an alcohol billboard, bikini-clad babes clutch icy bottles, frothy beer flows over frosty mugs and the slogan reads, 'Life is good.' Ads like these may target adults, flows over frosty mugs and the slogan reads, 'Life is good.' Ads like these may target adults, but children are getting the message too, a University of Florida and University of Minnesota but children are getting the message too, a University of Florida and University of Minnesota study shows. study shows.

Adolescents attending schools in neighborhoods where alcohol ads litter the landscape tend Adolescents attending schools in neighborhoods where alcohol ads litter the landscape tend to to

want to drink more and, compared with other children, have more positive views of alcohol, want to drink more and, compared with other children, have more positive views of alcohol, researchers report in the researchers report in the Journal of Studies on Alcohol and Drugs.Journal of Studies on Alcohol and Drugs.

UF and UM researchers counted the number of alcohol ads within a two-block radius of 63 UF and UM researchers counted the number of alcohol ads within a two-block radius of 63 Chicago schools and compared students' opinions on drinking when they were in sixth grade Chicago schools and compared students' opinions on drinking when they were in sixth grade and again two years later. The result - The more ads for alcohol there were in a and again two years later. The result - The more ads for alcohol there were in a neighborhood, the more students were interested in drinking alcohol, the findings show…. neighborhood, the more students were interested in drinking alcohol, the findings show….

…….About half of all teens sample their first alcoholic drink by the time they are 15, according .About half of all teens sample their first alcoholic drink by the time they are 15, according to the U.S. Surgeon General, which released a report on teen drinking earlier this year. Teens to the U.S. Surgeon General, which released a report on teen drinking earlier this year. Teens who start drinking this early are more likely to have trouble in school, become addicted to who start drinking this early are more likely to have trouble in school, become addicted to alcohol, smoke cigarettes and use drugs than adolescents who don't drink, the report alcohol, smoke cigarettes and use drugs than adolescents who don't drink, the report

states…. states….

……."I think results from this study and studies like this study clearly indicate that there should ."I think results from this study and studies like this study clearly indicate that there should be policies to ban alcohol advertising near the schools," Komro said. "It clearly shows that be policies to ban alcohol advertising near the schools," Komro said. "It clearly shows that exposure is dangerous for our children."exposure is dangerous for our children."

Teen girls using pills, smoking Teen girls using pills, smoking more than boys more than boys

Teenage girls, having caught up to their male counterparts in Teenage girls, having caught up to their male counterparts in illegal drug use and alcohol consumption, now have the illegal drug use and alcohol consumption, now have the dubious distinction of surpassing boys in smoking and dubious distinction of surpassing boys in smoking and prescription drug abuse. In the past two years, in fact, more prescription drug abuse. In the past two years, in fact, more young women than men started using marijuana, alcohol and young women than men started using marijuana, alcohol and cigarettes, according to government findings being released cigarettes, according to government findings being released today.today.

The results are doubly disturbing, researchers said, because The results are doubly disturbing, researchers said, because they run counter to trends indicating an overall decline in they run counter to trends indicating an overall decline in teenage drug use and because young women appear to suffer teenage drug use and because young women appear to suffer more serious health consequences as a result.more serious health consequences as a result.

Critique found at stats.org

For further information:For further information:

• [email protected]@hms.harvard.edu• www.divisiononaddictions.orgwww.divisiononaddictions.org

•A copy of this presentation will be available A copy of this presentation will be available online (through the “Education” link) by online (through the “Education” link) by January 22ndJanuary 22nd

• www.basisonline.orgwww.basisonline.org•Online review of current research in Online review of current research in

addictions (alcohol, tobacco, illicit drugs, addictions (alcohol, tobacco, illicit drugs, gambling, and humanities)gambling, and humanities)

•Addiction resourcesAddiction resources

Documents

Research Methods for the Social Sciences: An Introductory Course January 19 th, 2010 – Class Introduction Division on Addictions, Cambridge Health Alliance