Research MethodsResearch Methods

DefinitionsDefinitions• TheoryTheory

o Systematic way of organizing/explaining Systematic way of organizing/explaining observationsobservations

• HypothesisHypothesiso Tentative belief/prediction about the relationship Tentative belief/prediction about the relationship

between two or more variablesbetween two or more variables

• VariableVariableo Something that can change or vary from one Something that can change or vary from one

situation or person to another (IQ score, age, sex, situation or person to another (IQ score, age, sex, education, height)education, height)

• Operational definitionOperational definition• Clear “label” of how a variable is measuredClear “label” of how a variable is measured

DefinitionsDefinitions

• 2 types of variables2 types of variableso Independent variableIndependent variable

• Potential influences on behavior under Potential influences on behavior under studystudy

• Age, gender, advertisement, SES, stress Age, gender, advertisement, SES, stress

o Dependent variableDependent variable• ““Outcome variable” or “outcome factor”Outcome variable” or “outcome factor”• A behavior, action, attitude, A behavior, action, attitude,

performance, score on tests/surveysperformance, score on tests/surveys

DefinitionsDefinitions

• Variable measurementVariable measuremento ContinuousContinuous (degree of optimism, rate of (degree of optimism, rate of

recovery, shyness)recovery, shyness)

vs.vs.o CategoricalCategorical (gender, species, history of (gender, species, history of

cancer)cancer)

SamplingSampling

• PopulationPopulation = a larger group from which to draw = a larger group from which to draw a a sample sample

• SampleSample = a subgroup representative of the = a subgroup representative of the populationpopulation

• ParticipantsParticipants = the individuals participating in = the individuals participating in the studythe study

• GeneralizabilityGeneralizability = the applicability of findings to = the applicability of findings to the the entire population of entire population of interestinterest

• Random SamplingRandom Sampling = the sample should be = the sample should be similar to similar to the larger group as the larger group as every every person in the study person in the study has an has an equal chance of equal chance of being placed being placed into the into the treatment condition treatment condition

MeasurementMeasurement

• ReliabilityReliabilityo A measure’s ability to produce consistent A measure’s ability to produce consistent

resultsresultso ““Internal consistency”Internal consistency”

• Degree participants’ scores on a Degree participants’ scores on a measure resemble their scores on the measure resemble their scores on the same measure when administered later same measure when administered later

MeasurementMeasurement

• ValidityValidityo A measure’s ability to assess the variable A measure’s ability to assess the variable

it is supposed to assessit is supposed to assess

o Are you measuring what you Are you measuring what you thinkthink you you are measuring?are measuring?

o Note: Prior validity is not an indication of Note: Prior validity is not an indication of present validity. WHY?present validity. WHY?

MeasurementMeasurement

• General Sources of ErrorGeneral Sources of Erroro RandomRandom = due to chance events (i.e. = due to chance events (i.e.

misrecordings)misrecordings)o CoverageCoverage = failure to include relevant = failure to include relevant

people in apeople in a sample sample (i.e. random digit dialing)(i.e. random digit dialing)

o NonresponseNonresponse = refusal to respond or = refusal to respond or absentabsent

Research MethodsResearch Methods

• Descriptive (Case) studiesDescriptive (Case) studies• Poor sample size (N = 1)Poor sample size (N = 1)• Selection bias (nonrandom sample)Selection bias (nonrandom sample)• Observation bias (is the therapist seeing what he/she wants Observation bias (is the therapist seeing what he/she wants

to see?)to see?)• Causal factors unidentifiedCausal factors unidentified

• Correlational StudiesCorrelational Studies• Do not specify cause/effect relationshipsDo not specify cause/effect relationships• Can help identify naturally occurring relationshipsCan help identify naturally occurring relationships

• Experimentation and SurveysExperimentation and Surveys• Accuracy of self-report?Accuracy of self-report?• Low response ratesLow response rates• Social desirability bias = try to make yourself look better Social desirability bias = try to make yourself look better

than you arethan you are• Demand characteristics = say what you think the survey Demand characteristics = say what you think the survey

wants to hearwants to hear• Issues of generalizability outside the lab?Issues of generalizability outside the lab?

SurveysSurveys

• What do surveys measure?What do surveys measure?o AttitudesAttitudeso BehaviorsBehaviorso BeliefsBeliefso OpinionsOpinionso PreferencesPreferenceso SatisfactionSatisfactiono DemographicsDemographicso Orientations (religious, sexual, political, Orientations (religious, sexual, political,

etc.)etc.)

SurveysSurveys

• Principles for writing survey questions:Principles for writing survey questions: Choose simple over specialized words (“apartheid?”)Choose simple over specialized words (“apartheid?”) Use complete sentences to ask questionsUse complete sentences to ask questions Avoid vague language and/or cultural slangAvoid vague language and/or cultural slang Provide appropriate time referentsProvide appropriate time referents Limit the number of personal questions Limit the number of personal questions Keep instructions brief!Keep instructions brief! Ask only one question at a timeAsk only one question at a time Start with the most salient questions in case of drop-Start with the most salient questions in case of drop-

outout When possible, group similar response sets togetherWhen possible, group similar response sets together Use dark print for questions + light print for answer Use dark print for questions + light print for answer

choiceschoices

SurveysSurveys

List answers vertically instead of horizontallyList answers vertically instead of horizontally Avoid double-barreled questionsAvoid double-barreled questions Keep font simple and “clean”Keep font simple and “clean” Limit or eliminate skip patternsLimit or eliminate skip patterns Do not use emotionally charged words (“hate”, Do not use emotionally charged words (“hate”,

“fantastic”) “fantastic”) Do not use leading questionsDo not use leading questions Use a scale that covers the full range of response Use a scale that covers the full range of response

possibilitiespossibilities Conduct focus groups in pre-testing phase Conduct focus groups in pre-testing phase

(qualitative output)(qualitative output)

ExperimentsExperiments

• There are two types of information we There are two types of information we want:want:

1)1) Description/Prediction (“what?)Description/Prediction (“what?) What are people doing? Thinking?What are people doing? Thinking? What is happening?What is happening? What do people do in a certain situation?What do people do in a certain situation?

2)2) Explanation/Understanding/Control (“why?”)Explanation/Understanding/Control (“why?”) What causes people to do or think like they do?What causes people to do or think like they do? Why is it happening?Why is it happening? How can manipulating something (attitudes, How can manipulating something (attitudes,

situations) change people’s behavior?situations) change people’s behavior?

ExperimentsExperiments

• Correlational relationshipsCorrelational relationshipso Can indicate if two variables are related but Can indicate if two variables are related but

do not indicate which variable influence do not indicate which variable influence whichwhich

o If variable A and variable B If variable A and variable B areare correlated, correlated, we can't know which variable influences we can't know which variable influences which. Why not? which. Why not?

• does A cause B?does A cause B?• does B cause A?does B cause A?• does C (some other factor) cause both A and does C (some other factor) cause both A and

B?B?

ExperimentsExperiments

• Correlation coefficients range from – 1.00 to + 1.00Correlation coefficients range from – 1.00 to + 1.00o Positive correlation: 0 to +1.00Positive correlation: 0 to +1.00o Negative correlation: 0 to –1.00Negative correlation: 0 to –1.00

• The sign of the correlation (+/–) indicates the kind of The sign of the correlation (+/–) indicates the kind of relationship (but relationship (but notnot the the strengthstrength of the relationship) of the relationship)

o Positive correlationPositive correlation: the more cars on the road, the : the more cars on the road, the more more

smog there is in the airsmog there is in the airo Negative correlationNegative correlation: the fewer days you miss in my : the fewer days you miss in my

class, class,

the higher your final gradethe higher your final grade

Evaluation of ResearchEvaluation of Research

o Does the theoretical framework make Does the theoretical framework make sense?sense?

o Is the sample appropriate?Is the sample appropriate?o Are the measures and procedures Are the measures and procedures

adequate?adequate?o Are data conclusive?Are data conclusive?o Are data interpreted correctly?Are data interpreted correctly?o Does the study add anything meaningful?Does the study add anything meaningful?o Is the study ethical?Is the study ethical?

Ethical GuidelinesEthical Guidelines

• Human subjects researchHuman subjects research Confidentiality and anonymity of Confidentiality and anonymity of

participantsparticipants Informed consent Informed consent Be clear participation is voluntaryBe clear participation is voluntary Limited to no deception allowed Limited to no deception allowed Lack of coercionLack of coercion Minimize harmful consequencesMinimize harmful consequences