Upload
coleen-lewis
View
213
Download
0
Embed Size (px)
DESCRIPTION
Table of Contents Variables Types of Data Hypothesis Testing Types of Errors Methods Middle Range Analysis Testing Differences Relationships between variables When to use each method Correlation Sample size Sampling Methods Bias Reliability Validity Philosophy of Four World Views Alternate Strategies of Inquiry Approaches Described A Framework for Research Design Pre Experimental Designs Quasi Experimental Designs True Experimental Designs Qualitative Methodology Aspects To Consider When Planning A Mixed Methods Design Criteria For Choosing and Selecting Statistical Tests 3
Citation preview
DATA ANALYSIS FOR RESEARCH AND APPLICATION
John J. BennettFellow in Healthcare Systems Engineering and all around Swell Guy.
Define MeasureAnalyze
Improve Control
Visio
nAn
alys
is
Team Aim
Map
Mea
sure Change
Sustain/Spread
PLAN
DOCHECK
ACT
CustomerSatisfaction
PLAN
DOCHECK
ACT
CustomerSatisfaction
Lean Six Sigma VA~TAMMCS Crosswalk
Table of Contents Variables Types of Data Hypothesis Testing Types of Errors Methods Middle Range Analysis Testing Differences Relationships between variables When to use each method Correlation Sample size Sampling Methods Bias Reliability Validity
Philosophy of Four World Views
Alternate Strategies of Inquiry Approaches Described A Framework for Research
Design Pre Experimental Designs Quasi Experimental Designs True Experimental Designs Qualitative Methodology Aspects To Consider When
Planning A Mixed Methods Design
Criteria For Choosing and Selecting Statistical Tests
3
Philosophy of Four World Views
FOUR WORLDVIEWS
Post positivism Constructivism
Determination Reductionism Empirical observation and
measurement Theory verification
Understanding Multiple participant meanings Social and historical Construction Theory generation
Advocacy/Participatory Pragmatism
Political Empowerment Issue-oriented Collaborative Change-oriented
Consequences of actions Problem-centered Pluralistic Real-world practice oriented
4
Alternate Strategies of Inquiry
Alternative Strategies of InquiryQuantitative Mixed Methods Qualitative
Experimental designs Non-experimental designs,
such as surveys
Sequential Concurrent Transformative
Narrative Research Phenomenology Ethnographies Grounded Theory Case Study
5
Quantitative Methods Mixed Methods Qualitative Methods Pre-determined Instrument based questions Performance data, attitude
data, observational data, and census data
Statistical analysis Statistical interpretation
Both Pre-determinedAnd emerging methods
Both open and closed ended questions
Multiple forms of data drawing on all possibilities
Statistical and text analysis Across databases
interpretation
Emerging methods Open-ended questions Interview data, observation
data, document data, and audio-visual data
Text and image analysis Themes, patterns
interpretation
Approaches Described Qualitative, Quantitative, and Mixed Methods Approaches
Tend to or Typically… Qualitative Approaches Quantitative Approaches Mixed Methods Approaches Use these philosophical assumptions
Constructivist/Advocacy/Participatory knowledgeClaims
Post-positivist knowledge claims Pragmatic knowledge claims
Employ these strategies of inquiry
PhenomenologyGrounded TheoryEthnographyCase Study and narrative
Surveys and experiments Sequential, concurrent, and transformative
Employ these methods Open ended questions, emerging approaches, text or image data
Close-ended questions, predetermined approaches, numeric data
Both open and closed ended questions, both emerging and predetermined approaches and both quantitative and qualitative data and analysis
Use these practices of research as the researcher
Positions him/herselfCollects participant meanings Focuses on a single concept or phenomenon Brings personal values into the studyStudies the context or settings of participants Validates the accuracy of findingsMakes interpretations of the dataCreates an agenda for change or reform Collaborates with the participants
Tests or verifies theories or explanationsIdentifies variables to study Relates variables in questions or hypotheses Uses standards of validity and reliabilityObserves and measures information numerically Uses unbiased approaches Employs statistical procedures
Collects both quantitative and qualitative dataDevelops a rationale for mixing Integrates the data at different stages of inquiry Presents visual pictures of the procedures in the study Employs the practices of both qualitative and quantitative research
6
7
Philosophical WorldviewsPostpositive
Social constructionAdvocacy/participatory
Pragmatic
Research MethodsQuestions
Data CollectionData AnalysisInterpretation
Write-upValidation
Selected Strategies of Inquiry
Qualitative Strategies Quantitative Strategies
Mixed Methods Strategies
Research Designs
A Framework For Research Design
Researcher tests or verifies a theory
Researcher tests hypotheses or
research questions from the theory
Researcher defines and
operationalizes variables derived from the theory
Researcher measures or
observes variables using an instrument
to obtain scores
Deductive Approach Used in Quantitative
Research
Inductive Logic of Research in a
Qualitative StudyResearchers poses generalizations or theories from past experiences and
literature
Researcher looks for broad patterns,
generalizations, or theories from themes or
categories
Researcher analyzes data to form themes or
categories
Researcher asks open ended questions of
participants or records fieldnotes
Researcher gathers
information (e.g. interviews,
observations)
Pre Experimental Designs One Shot Case Study
This design involves an exposure of a group to a treatment followed by a measure.
Group AX-----------0 One Group Pre Test Post Test Design
This design includes a pre-test measure followed by a treatment and a post-test for a single group.
Group A01----------X------------02 Static Group Comparison or Post Test Only with Nonequivalent Groups
Experimenters use this design after implementing a treatment. After the treatment, the researcher selects a comparison group and provides a post-test to both the experimental group(s) and the comparison group(s).
Group AX-------------------------0
---------------------------------------
Group B---------------------------0 Alternative Treatment Post Test Only with Nonequivalent Groups Design
This design uses the same procedure as the Static Group Comparison, with the exception that the nonequivalent comparison group received a different treatment.
Group A X1---------------------0
------------------------------------
Group BX2----------------------0
8
Quasi Experimental Designs Nonequivalent (Pre-Test and Post-Test) Control-Group Design
In this design, a popular approach to quasi experiments, the experimental group A and the control group B are selected without random assignment. Both groups take a pre-test and post-test. Only the experimental group receives the treatment.
Group A 0-------------X----------0
-------------------------------------
Group B 0 ------------------------0 Single-Group Interrupted Time-Series Design
In this design, the researcher records measures for a single group both before and after a treatment.
Group A 0---0---0---0---X---0---0---0---0 Control-Group Interrupted Time-Series Design
A modification of the Single-Group Interrupted Time-Series design in which two groups of participants, not randomly assigned, are observed over time. A treatment is administered to only one of the group (i.e. Group A).
Group A 0---0---0---0---X---0---0---0---0
---------------------------------------------------
Group B 0---0---0---0---0---0---0---0---0
9
True Experimental Designs Pre-Test-Post-Test Control Group Design
A traditional, classical design, this procedure involves random assignment of participants to two groups. Both groups are administered both a pre-test and a post-test, but the treatment is provided only to experimental Group A.
Group A R-------0-------X-------0
Group B R-------0----------------0 Post-Test Only Control Group Design
This design controls for any confounding effects of a pre-test and is a popular experimental design. The participants are randomly assigned to groups, a treatment is given only to the experimental group, and both groups are measured on the post-test.
Group A R-----X------0
Group B R-------------0 Solomon Four-Group Design
A special case of 2 x 2 factorial design, this procedure involves the random assignment of participants to four groups. Pre-tests and treatments are varied for the four groups. All groups receive a post-test.
Group A R----0----X-----0 Group C R----------X-----0
Group B R----0-----------0 Group D R-----------------0 A-B-A Single Subject Design
This design involves multiple observations of a single individual. The target behavior of a single individual is established over time and is referred to as a baseline behavior. The baseline behavior is assessed, the treatment provided, and then the treatment is withdrawn.
Baseline A -Treatment B---Baseline A
0-0-0-0-0-0-X-X-X-X-X-X-X-0-0-0-0-0-0-0
10
Aspects To Consider When Planning A Mixed Methods Design
Aspects to Consider in Planning a Mixed Methods Design
Timing Weighting Mixing Theorizing
No Sequence Concurrent
Equal Integrating Explicit
Sequential-Qualitative first
Qualitative Connecting Explicit/Implicit
Sequential- Quantitative first
Quantitative Embedding Implicit
11
Sequential Mixed Method Design
Sequential Explanatory Design
QUAN QUAN qual qual
Data Collection Data Analysis Data CollectionData AnalysisInterpretation of Entire Analysis Sequential Exploratory Design
QUAL QUAL quan quan
Data Collection Data Analysis Data CollectionData AnalysisInterpretation of Entire Analysis Sequential Transformative Design
12
QUAN qual
QUAL quan
QUAL quan
Social science theory, qualitative theory, advocacy worldview
QUAN qual
Social science theory, qualitative theory, advocacy worldview
Concurrent Mixed Method Designs Concurrent Triangulation Design Concurrent Embedded Design
+
QUAN QUAL
Data collection Data Collection
QUAN Data Results Compared QUAL
Data Analysis Data Analysis Analysis of Findings Analysis of Findings
Concurrent Transformative Design
13
QUAN QUAL
QUAN
qual
QUAL
quan
QUAN + QUAL
Social science theory, qualitative theory, advocacy worldview
QUAN
Social science theory, qualitative theory, advocacy worldview
Variables14
Independent variable = (antecedent) the presumed cause of the dependent variable
Dependent variable = the presumed effect The independent variable influences the dependent
variable Dichotomies = 2-valued variables (such as male-
female, correct-incorrect, young-old) Polytomies = multi-valued variables (such as
Catholic, Muslim, Jew, Buddhist, etc) Active variable = experimental or manipulated
variable Attribute variable = a measured variable
Moderator Variables15
A variable is considered a moderator if it explains when a relationship between an independent variable and a dependent variable is larger or smaller.
Independent
Dependent
Moderator
Mediator Variable16
A variable is considered a mediator if it explains how a relationship between an independent variable and a dependent variable occurs.
Independent
Dependent
Mediator
Types of Variable Data17
Nominal = the numbers or symbols have no number meaning beyond presence or absence of the property or attribute being measured (i.e., 0=male, 1=female)
Ordinal = data that can be ordered in terms of importance, quantity or similar hierarchical attributes
Interval = possess the characteristics of nominal and ordinal, numerically equal distances on interval scales representing equal distances in the property being measured
Ratio = in addition to possessing the characteristics of nominal, ordinal, and interval scales, has an absolute or natural zero that has empirical meaning.
Hypothesis Testing18
Research hypothesis (H1) and null hypothesis (H0): H1 states that there is a difference (or relationship) between two variables; H0 states that there is no difference (or relationship) between two variables
H0 is the negation of H1 Note that H1 (H0) refers to populations A sample is used to test H1; results are interpreted
probabilistically. The idea behind the scene here is that one sample is enough to determine if H1, which states differences or relationships in the population, can be supported or not statistically (this is based on the central limit theorem)
In the sample, a statistic is calculated: t for differences in the means of a variable between two groups (t-test), F for differences in the means of a variable among three or more groups (one-way ANOVA and MANOVA) , r for relationships between two variables (Pearson correlation coefficient)
Hypothesis Testing - contd19
The question is, based on the results in the sample, can H1 be supported? The answer is probabilistic; a p value is determined based on the calculated statistic (t, F, or r), sample size (N), and degrees of freedom* (df) - SPSS provides all the information you need; tables are not needed when using SPSS
A level of significance is selected for the study; usually, this level is .05 (alpha level or the probability of committing the type I error** or the level of significance)
The calculated p value is compared with the alpha level (let's assume that is .05)
If p < .05 H0 can be rejected; therefore, H1 is accepted If p >= .05 H0 cannot be rejected; therefore, H1 cannot
be supported
Type I and II Errors20
The type I error (alpha; ) is rejecting the H0 when it should have been accepted
type II error (beta; ) is accepting the H0 when it should have been rejected. (Power = 1 - = the probability of correctly rejecting H0).
Here is another way of remembering these errors. If, in your research, you make the type I error, you incorrectly
reject the H0. Therefore, you incorrectly support the H1, which you then publish. What a shame! (Shme, that is!).
On the other hand, if you make the type II error, you incorrectly fail to reject H0. Therefore, you incorrectly reject the H1. Therefore, you drop the research incorrectly thinking that there was nothing worth publishing. All that work for nothing - What a pity!
Methods21
Quantitative – used for description of trends, attitudes or opinions of a population by studying a sample of that population (Creswell, 2003) Common tool is survey. From sample results, the researcher generalizes
or makes claims about the population. Experiment would test the impact of a treatment or intervention on an
outcome where all other factors are controlled to isolate the variable in question.
Qualitative – a type of field study where quantitative approaches cannot adequately capture the appropriate information (Kerlinger & Lee, 2000). Provides a deeper understanding of a process or experience. More descriptive – addresses the ‘why’ behind observations Uses direct observation and semi-structured interviews in the natural
environment Researcher may even develop new hypotheses during the research
process, flexible. Creswell, 2003
22
Validating the Accuracy of the
Information
Interpreting the Meaning of Themes/
Descriptions
Interrelating Themes/Description
(e.g. grounded theory, case study, Phenomenology, ext.)
Themes Description
Coding the Data(hand or
computer)
Reading Through all Data
Organizing and Preparing Data for
Analysis
Raw Data (transcripts,
fieldnotes, images, etc.)
Data Analysis in Qualitative Research
Criteria for Choosing Select Statistical Tests
Nature of Question Number of Independent
Variables
Number of Dependent Variables
Number of Control Variables
(covariates)
Type of Score Independent/
Dependent Variables
Distribution of Scores
Statistical Test
Group Comparison 1 1 0 Categorical/Continuous
Normal T-Test
GroupComparison
1 or more 1 0 Categorical/Continuous
Normal Analysis of Variance
Group Comparison
1 or more 1 1 Categorical/Continuous
Normal Analysis of Covariance
Group Comparison
1 1 0 Categorical/Continuous
Non-normal Mann-Whitney U Test
Association BetweenGroups
1 1 0 Categorical/Categorical
Non-normal Chi-square
Relate Variables
1 1 0 Continuous/Continuous
Normal Pearson product moment correlation
Relate Variables
2 or more 1 0 Continuous/Continuous
Normal Multiple Regression
Relate Variables
1 1 or more 0 Categorical/Categorical
Non-normal Spearman rank-order
correlation
23
Two Kinds of Operational Definitions24
Measured = describes how a variable will be measured
Experimental = spells out the details of the investigators manipulation of the variable
Middle-Range Analysis25
There are three general ways of relating theory to data:
Grand Theory: Theory built from abstract concepts using deductive logic; mathematical theories are a good example. [theory theory]
Middle-Range Analysis: [theory data theory] Grounded Theory: Theory inductively derived
from systematic data collection and analysis; see Glaser & Strauss (1967), Strauss & Corbin (1990) for more details on this type of qualitative research. [data theory]
Middle Range Analysis Illustration
26
Theoretic Hypothesis:
Innovativeness is positively related to
Cosmopoliteness
Concept Concept
Empirical Hypothesis:
Early adoption of hybrid corn
is positively related to
Number of trips to Des Moines
Theoretical Level
Empirical Level
(Epistemic (Epistemic
Relationship) Relationship)
Operation Operation
Testing Differences Between Independent Groups
27
T-Test for comparing two parametric samples that we want to compare concerning their mean value for some variable of interest
Nonparametric alternatives for this test are the Wald-Wolfowitz runs test, the Mann-Whitney U test, and the Kolmogorov-Smirnov two-sample test.
If we have multiple groups, we would use analysis of variance - ANOVA/MANOVA
Nonparametric equivalents to this method are the Kruskal-Wallis analysis of ranks and the Median test.
Testing Differences Between Dependent Groups
28
If we want to compare two variables measured in the same sample we would customarily use the t-test for dependent samples
Nonparametric alternatives to this test are the Sign test and Wilcoxon's matched pairs test. If the variables of interest are dichotomous in nature (i.e., "pass" vs. "no pass") then McNemar's Chi-square test is appropriate.
If there are more than two variables that were measured in the same sample, then we would customarily use repeated measures ANOVA.
Nonparametric alternatives to this method are Friedman's two-way analysis of variance and Cochran Q test (if the variable was measured in terms of categories, e.g., "passed" vs. "failed"). Cochran Q is particularly useful for measuring changes in frequencies (proportions) across time.
Relationships Between Variables
29
To express a relationship between two variables one usually computes the correlation coefficient.
Nonparametric equivalents to the standard correlation coefficient are Spearman R, Kendall Tau, and coefficient Gamma (see Nonparametric correlations).
If the two variables of interest are categorical in nature (e.g., "passed" vs. "failed" by "male" vs. "female") appropriate nonparametric statistics for testing the relationship between the two variables are the Chi-square test, the Phi coefficient, and the Fisher exact test.
In addition, a simultaneous test for relationships between multiple cases is available: Kendall coefficient of concordance. This test is often used for expressing inter-rater agreement among independent judges who are rating (ranking) the same stimuli.
When to Use Which Method30
Nonparametric methods are most appropriate when the sample sizes are small. When the data set is large (e.g., n > 100) it often makes little sense to use
nonparametric statistics at all. Each nonparametric procedure has its peculiar sensitivities and blind spots. For example, the Kolmogorov-Smirnov two-sample test is not only sensitive to
differences in the location of distributions (for example, differences in means) but is also greatly affected by differences in their shapes.
The Wilcoxon matched pairs test assumes that one can rank order the magnitude of differences in matched observations in a meaningful manner.
If this is not the case, one should rather use the Sign test. In general, if the result of a study is important (e.g., does a very expensive and
painful drug therapy help people get better?), then it is always advisable to run different nonparametric tests; should discrepancies in the results occur contingent upon which test is used, one should try to understand why some tests give different results.
On the other hand, nonparametric statistics are less statistically powerful (sensitive) than their parametric counterparts, and if it is important to detect even small effects one should be very careful in the choice of a test statistic.
Correlation / Cause and Effect
31
The mathematical correlation between two variables is represented by an italicized, lower case r.
The correlation coefficient r measures the strength of relationship between any two variables. Values of r range between -1 and 1. The sign (+ or –) gives the direction of the relationship.
A value of zero means there is no relationship. +1 indicates a perfect positive relationship (when one increases, the other
increases), and -1 indicates a perfect negative relationship (one decreases as the other
increases). When two variables are correlated it is tempting to assume a cause-and-
effect relationship. This cannot be concluded from r alone but must include consideration of supporting research and/or a sound theoretical argument.
It is possible to find what are known as spurious correlations due to “freaks of nature,” coincidence, or “accidents.”
Kerlinger & Lee (2000) argued that when an experimental design is properly executed, the researcher can determine a cause and effect relationship between the independent and dependent variables.
Sample Size32
Creswell (2003) suggested that the sample size for groups should be calculated by using: the level of significance; the amount of power desired for the study; and the effect size.
Hair, et. al. (2006) noted that a researcher must appreciate sample size considerations for multiple regression testing.
Based upon their recommendations an effective sample size can be as low as 20 while maintaining a desired statistical power of .80 for simple regression testing.
Hair et. al. recommend a minimum sample size of 50 for a multiple regression testing but noted that 100 is the preferred sample size.
Sampling Methods33
Sampling refers to taking a portion of the population that is representative of the total population (Kerlinger & Lee, 2000)
Random – each member of the population sampled has an equal chance of being selected (Kerlinger & Lee, 2000)
Stratified-random - Stratified random sampling occurs when the population is divided into smaller groups, or strata, based on some similar characteristic within each stratum. A random sample is taken from each stratum. The size of the sample is proportional to the stratum’s size as compared to the overall population. These random samples are then combined to create a random population. Stratified random sampling provides that the sample will accurately reflect the population (Zikmund, 2003).
Cluster – Successive random sampling of units, or sets (held together by some common characteristic) (Kerlinger & Lee, 2000)
Convenience - Because the individuals were asked to participate this was not a random sample. Creswell (2003) describes this as a non-probability sample and not as desirable.
Research Bias34
Unknown or unacknowledged error created during the design, measurement, sampling, procedure, or choice of problem studied
There are two types of error associated with most forms of research: random and systematic. Random errors, i.e., those due to sampling variability or measurement precision,
occur in essentially all quantitative studies and can be minimized but not avoided.
Systematic errors, or biases, are reproducible inaccuracies that produce a consistently false pattern of differences between observed and true values.
Both random and systematic errors can threaten the validity of any research study.
However, random errors can be easily determined and addressed using statistical analysis; most systematic errors or biases cannot. This is because biases can arise from innumerable sources, including complex human factors. For this reason, avoidance of systematic errors or biases is the task of proper research design
key difference between qualitative and quantitative research is attempts to eliminate bias by quantitative researcher explicit acknowledgement of bias by qualitative researchers
Major Categories of Bias35
Selection biases, which may result in the subjects in the sample being unrepresentative of the population of interest
Measurement biases, which include issues related to how the outcome of interest was measured
Intervention (exposure) biases, which involve differences in how the treatment or intervention was carried out, or how subjects were exposed to the factor of interest
Hartman et al, 2002
Selection Bias36
Selection biases occur when the groups to be compared are different. These differences may influence the outcome. Common types of sample (subject selection) biases include volunteer or referral bias, and non-respondent bias. By definition, nonequivalent group designs also introduce selection bias.
Volunteer or referral bias. Volunteer or referral bias occurs because people who volunteer to participate in a study (or who are referred to it) are often different than non-volunteers/non-referrals. This bias usually, but not always, favors the treatment group, as volunteers tend to be more motivated and concerned about their health.
Non-respondent bias. Non-respondent bias occurs when those who do not respond to a survey differ in important ways from those who respond or participate. This bias can work in either direction.
Measurement Bias37
Measurement biases involve systematic error that can occur in collecting relevant data. Common measurement biases include instrument bias, insensitive measure bias, expectation bias, recall or memory bias, attention bias, and verification or work-up bias.
Instrument bias. Instrument bias occurs when calibration errors lead to inaccurate measurements being recorded, e.g., an unbalanced weight scale.
Insensitive measure bias. Insensitive measure bias occurs when the measurement tool(s) used are not sensitive enough to detect what might be important differences in the variable of interest.
Expectation bias. Expectation bias occurs in the absence of masking or blinding, when observers may err in measuring data toward the expected outcome. This bias usually favors the treatment group
Recall or memory bias. Recall or memory bias can be a problem if outcomes being measured require that subjects recall past events. Often a person recalls positive events more than negative ones. Alternatively, certain subjects may be questioned more vigorously than others, thereby improving their recollections.
Attention bias. Attention bias occurs because people who are part of a study are usually aware of their involvement, and as a result of the attention received may give more favorable responses or perform better than people who are unaware of the study’s intent.
Verification or work-up bias. Verification or work-up bias is associated mainly with test validation studies. In these cases, if the sample used to assess a measurement tool (e.g., diagnostic test) is restricted only to who have the condition of factor being measured, the sensitivity of the measure can be overestimated.
Intervention Bias38
Intervention or exposure biases generally are associated with research that compares groups. Common intervention biases include: contamination bias, co-intervention bias, timing bias(es), compliance bias, withdrawal bias, and proficiency bias.
Contamination bias. Contamination bias occurs when members of the 'control' group inadvertently receive the treatment or are exposed to the intervention, thus potentially minimizing the difference in outcomes between the two groups.
Co-intervention bias. Co-intervention bias occurs when some subjects are receiving other (unaccounted for) interventions at the same time as the study treatment.
Timing bias(es). Different issues related to the timing of intervention can bias. If an intervention is provided over a long period of time, maturation alone could be the cause for improvement. If treatment is very short in duration, there may not have been sufficient time for a noticeable effect in the outcomes of interest.
Compliance bias. Compliance bias occurs when differences in subject adherence to the planned treatment regimen or intervention affect the study outcomes.
Withdrawal bias. Withdrawal bias occurs when subjects who leave the study (drop-outs) differ significantly from those that remain.
Proficiency bias. Proficiency bias occurs when the interventions or treatments are not applied equally to subjects. This may be due to skill or training differences among personnel and/or differences in resources or procedures used at different sites.
Design bias39
research design bias is introduced NOT when the study fails to control for threats to internal and external validity BUT RATHER when the study fails to identify the
validity problems OR when publicity about the research fails to
incorporate the researchers cautions
Measurement bias40
measurement bias exists when researcher fails to control for the effects of data collection and measurement e.g. tendency of people to give socially
desirable answers using self report is often biased by social
desirability
Sampling bias41
sampling bias exists (beyond regression) when the sampling procedure introduces bias Key Sampling Problem #1: omission of
women, Hispanics or other minorities from samples OR studying only minorities
Key Sampling Problem #2: targeting the most desirable or most accessible sample
Procedural bias42
procedural bias exists most often when we administer the research interview or questionnaire under adverse conditions Using students Paying subjects
“Type III " error or problem bias
43
Not acknowledging a Type I or Type II error Influences researcher to see a result that is
not there
Other biases44
Contamination bias. Contamination bias occurs when members of the 'control' group inadvertently receive the treatment or are exposed to the intervention, thus potentially minimizing the difference in outcomes between the two groups.
Co-intervention bias. Co-intervention bias occurs when some subjects are receiving other (unaccounted for) interventions at the same time as the study treatment.
Timing bias(es). Different issues related to the timing of intervention can bias. If an intervention is provided over a long period of time, maturation alone could be the cause for improvement. If treatment is very short in duration, there may not have been sufficient time for a noticeable effect in the outcomes of interest.
Compliance bias. Compliance bias occurs when differences in subject adherence to the planned treatment regimen or intervention affect the study outcomes.
Withdrawal bias. Withdrawal bias occurs when subjects who leave the study (drop-outs) differ significantly from those that remain.
Proficiency bias. Proficiency bias occurs when the interventions or treatments are not applied equally to subjects. This may be due to skill or training differences among personnel and/or differences in resources or procedures used at different sites.
Reliability45
Opposite of random error Refers to the systematic or consistent
portion of scores (Schwab, 1999) 3 contexts of reliability:
Internal consistency – applies to multiple items in a measure
Interrater reliability – applies to multiple observers or raters
Stability – applies to multiple time periods
Validity46
Are you measuring what you think you are measuring?
3 types are important: Content validity = sampling adequacy of the
content (is the substance or content of this measure representative of the content being measured?)
Criterion-related validity = comparison of test or scale scores with one or more external variables known or believed to measure the attribute under study
Construct validity = meaning of the tests (what factors or constructs account for the variance in test performance?)
Kerlinger & Lee, 2000
Selected References:47
Creswell (2003) Glaser, B., & Strauss, A. (1967). The discovery of
grounded theory. Chicago: Aldine. Hair, (2006). Multivariate Data Analysis Hartman, J.M., Forsen, J.W., Wallace, M.S., Neely, J.G.
(2002). Tutorials in clinical research: Part IV: Recognizing and controlling bias. Laryngoscope, 112, 23-31.
Kerlinger & Lee (2000) Patton () Rogers, E. M. (1995). The diffusion of innovations (4th
ed.). New York, NY: Free Press. Strauss, A., & Corbin, J. (1990). Basics of qualitative
research: Grounded theory procedures and techniques. Newbury Park, CA: SAGE.