View
218
Download
0
Category
Preview:
Citation preview
Experimental design
Experimental design
ITS classDecember 2, 2004
ITS classDecember 2, 2004
Quantitative researchQuantitative research
Quantify relationships between variables
Measured on a sample of subjects
Relationships expressed through effect statistics Correlations, relative
frequencies, diff. between means
Quantify relationships between variables
Measured on a sample of subjects
Relationships expressed through effect statistics Correlations, relative
frequencies, diff. between means
Types of StudyTypes of Study
Descriptive Measure things as they are
Experimental Take measurements Try an intervention
(treatment) Take measurements
Descriptive Measure things as they are
Experimental Take measurements Try an intervention
(treatment) Take measurements
Descriptive studiesDescriptive studies
Case Case series Cross-sectional Cohort or prospective or
longitudinal Case-control or
retrospective
Case Case series Cross-sectional Cohort or prospective or
longitudinal Case-control or
retrospective
Experimental studiesExperimental studies
AKA longitudinal, repeated-measures, interventions
Without a control group Time series Crossover
With a control group
AKA longitudinal, repeated-measures, interventions
Without a control group Time series Crossover
With a control group
Control group studiesControl group studies
Necessary if treatment effect cannot be “washed out”
Random assignment: min. chance that either group is not typical of population
Single-blind: subjects Double-blind: experimenter
too Best studies: data analyzed
blind
Necessary if treatment effect cannot be “washed out”
Random assignment: min. chance that either group is not typical of population
Single-blind: subjects Double-blind: experimenter
too Best studies: data analyzed
blind
Ethical issuesEthical issues
Randomized controlled study may not be ethical
E.g.: Heart disease study over 10 years
Access to treatment Inform subjects of
randomization
Randomized controlled study may not be ethical
E.g.: Heart disease study over 10 years
Access to treatment Inform subjects of
randomization
Quality of designsQuality of designs
Quality of evidence for causal relationships
Least for case, case-series Cross-sectional, case-
control Prospective Experimental best
evidence
Quality of evidence for causal relationships
Least for case, case-series Cross-sectional, case-
control Prospective Experimental best
evidence
ConfoundingConfounding
Two variables may be related for other reasons
Control for potential confounding factors
Two variables may be related for other reasons
Control for potential confounding factors
SamplesSamples
Generalizability Representative Safest: Random sample
Balance: Stratified random (%s) Balance on pretest!
Selection bias: Not representative Sources: age, socioeconomic
status Self-selection into groups High compliance
Generalizability Representative Safest: Random sample
Balance: Stratified random (%s) Balance on pretest!
Selection bias: Not representative Sources: age, socioeconomic
status Self-selection into groups High compliance
Sample sizeSample size
Statistical significance Big enough to be sure you will
detect the smallest worthwhile effect To be sure =
detect 80% of time Detect =
95% of time, expect a smaller value if there is no effect, p<0.05
Smallest worthwhile effect = Smallest to make a diff. to subjects’
lives
Statistical significance Big enough to be sure you will
detect the smallest worthwhile effect To be sure =
detect 80% of time Detect =
95% of time, expect a smaller value if there is no effect, p<0.05
Smallest worthwhile effect = Smallest to make a diff. to subjects’
lives
Statistical significanceStatistical significance
P-value: the probability of getting
something more extreme than your result, when there is no effect in the population
P-value: the probability of getting
something more extreme than your result, when there is no effect in the population
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
SourcesSources
http://www.sportsci.org/jour/0001/wghdesign.html: Good resource for basic stats
http://www.softwareevaluation.de/ : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines
http://www.sportsci.org/jour/0001/wghdesign.html: Good resource for basic stats
http://www.softwareevaluation.de/ : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines
Recommended