Quantitative Research

Overview

Non-experimental Qualitative Case studyPhenomenologyEthnographyHistoricalLiterature Review

Quantitative ObservationalSurveyArchivalMeta-analysis

Experimental ExperimentQuasi-experiment

Observational

• Definition- Directly observing naturally occurring behavior

unobtrusively, typically in the field, but can also take place in laboratory settings

• Pros- Goal is unobtrusive observation so that your

presence does not alter the participant’s affect, behavior, cognition

- Allows for continuous measurement- Avoid participant self report error

• Other- Construct Validity – depends on usage- External Validity – Excellent b/c naturally

occurring- Bias – coding bias- Error – coding errors

Observational

• Choose “Observational” if…- interested in naturally occurring

behavior- interested in studying a phenomenon as

it is naturally occurring- interested in gathering information

about naturally occurring boundary conditions or moderators

- interested in depth/breadth of continuous measurement

Survey

• Definition- Using self-report measures in any type of

collection method ---► in-person, online, telephone, and mail

• Pros- Relatively easy to collect data- Multiple collection methods for larger and

representative sample sizes

• Other- Construct – depends on usage- External – excellent because large sample

sizes and representative samples- Bias – social desirability bias (except online)- Error – participant self-report error

Survey

• Choose “Survey” if…- your topic can be analyzed using self-

report- interested in collecting a lot of data –

many variables and questions- interesting in collecting a lot of data –

many subjects, power- interested in representative sample and

generalizability - have little money, time, resources

(except for mail/telephone if using Survey Company)

Archival

• Definition- Using previously collected materials to analyze

new research questions by using quantitative (numbers) analysis

• Pros- No longer restricted only to present-day people

and events so access to larger sets of data- Unobtrusive so reduced chances of

experimenter error• Other

- Construct – Insensitive measures since not collected for purpose of your study, No control over how information collected so possibly flawed

- External – depends on inclusion criteria- Bias – coding bias- Error – coding error

Archival

• Choose “Archival” if…- Same as with “Historical”, such as

• interested in origins and growth• interested in particular historical events• no current data on point so look to past data• want to “generalize” from past events to

current or future events• research question can only be answered by

previously collected data

- Plus…• have resources like coders, time• want to minimize experimenter bias • want to synthesize and compare data

quantitatively

Meta-analysis• Definition

- A meta-analysis statistically combines the results of several studies that address a shared research hypotheses

• Pros- Central tendency - whether X affects Y, is the

effect significant, and how strong is that effect? - Variability - If there is heterogeneity, then look for

moderating variables that explain the variability. Does the effect of X on Y differ with moderator?

• Other- Construct – Depends on CV of included works- External – Depends on EV of included works- Bias – Inclusion/Exclusion bias, Interpretation bias- Errors – Inclusion/Exclusion error

Meta-analysis

• Choose “Meta-analysis” if…- Same as with “Literature Review”, such as

• have an argument that can be supported by published research

• interested in “summarizing” the literature for a variety of reasons

• interested in in “interpreting” the literature, for a variety of reasons

• interested in communicating the “quality” of the literature, for a variety of reasons

- Plus…• have resources like coders, time• want to minimize experimenter bias • want to synthesize and compare data

quantitatively

Meta-analysis• What are those “reasons”?

- Same as “Literature Review”, such as• No one has previously summarized and/or

interpreted the primary articles• The literature has grown to the point that it

necessitates guidance or direction• There is a new topic that cross-cuts many previous

literatures so new Literature Review needed to synthesize disparate literature relevant to new topic

• There are controversies or disagreements that need resolution or support from summarizing/interpreting the literature

- Plus…• Interested in “overall effect” of literature• There are new/old “moderators” that you want to

test and/or can be tested across studies

Experiment• Definition

- Testing cause-and-effect relationships by:(1) random assignment of Ss (2) to two or more conditions (3) which differ in terms of (only) IVs

• Pros- Can prove causation- Tight controls

• Other- Construct – depends on usage- External – experiments are artificial; alternative is

conduct field study but then problem is loss of control and influence of extraneous variables

- Internal – see the information from the previous PowerPoint slides about internal validity

- Bias – experimenter bias- Error – experimenter error

Experiment

• Choose “Experiment” if…- want to prove causation

Quasi-Experiment

• Definition- Contains aspects of both experiments and non-

experiments because deficient in at least one of the three aspects of experimental designs

• Pros- Depends on which aspects of experiment and

which aspects of non-experiment are involved in the quasi-experiment

• Other- Construct – depends on…- External – depends on…- Bias – depends on…- Error – depends on…

Quasi-Experiment

• Choose “Quasi-Experiment” if…- want scientific rigor of experiments but

can’t satisfy all three requirements for variety of reasons (see next PowerPoint presentation about all the types of quasi-experiments such as hybrid, matched-pairs, within-subjects, mixed-designs, and single-n studies

Advanced Sources

• Observational Participant Observation: A Methodology for Human

Studies, by Jorgensen, Sage Publications• Survey

Chapter 9 (Survey Research) of The Handbook of Research Methods in Social and Personality Psychology, Edited by Reis and Judd

• Archival Archival Strategies and Techniques, by Hill

• Meta-analysis Practical Meta-analysis, by Lipsey and Wilson

• Experiment and Quasi-experiment Experimental And Quasi-experimental Designs For

Research, by Campbell Experimental and Quasi-Experimental Designs for

Generalized Causal Inference, by Shadish et al