Upload
neil-atwill
View
220
Download
1
Tags:
Embed Size (px)
Citation preview
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
First Steps in Designing Quantitative Research: Asking the Right QuestionsSusan R. Hutchinson, Ph.D.Department of Applied Statistics & Research Methods
1
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Workshop Outline
• Overview of purposes for conducting quantitative research• Role(s) served by research questions• Attributes of useful RQs• Group activity on evaluating RQs• Exercise on constructing RQs
2
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Pretest
3
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
All quantitative studies include one or more research questions.
4
False
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Research questions and research hypotheses are the same thing.
5
False
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Research questions should “foreshadow” the data analyses.
6
True
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Research questions should be posed after data are collected.
7
False
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Research questions should be written using non-specific language to allow flexibility in designing the study.
8
False
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
All of the following are types of variables that might be included in research questions:dependent, moderator, mediator, extraneous, independent
9
True
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Writing quantitative research questions requires knowledge of statistics.
10
True
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What are the major purposes for conducting quantitative research?
11
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Quantitative research is intended to examine:• Differences between groups• Statistical significance, magnitude, and direction of
differences
• Change over time• Direction, magnitude, and between-group differences
• Relationships between 2 or more variables• Direction, magnitude, type/form, between-group
differences• Descriptions of populations and phenomena
12
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
As you’re constructing your research questions, keep in
mind the overriding purpose of quantitative research is to
understand differences
13
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What role do research questions play in designing quantitative research?
14
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What information should research questions include?• Indicate the target population• Identify the variables/constructs and what we
want to know about them• Foreshadow the type of data analysis (and
sometimes the research design)
15
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What characterizes a “good” research question?• Clearly conveys the variables/constructs and the nature of
their relationships• Includes independent, dependent, moderator, and mediator
variables (where applicable)• Must be answerable based on data from the study• RQs should not be rhetorical or require a value judgment to
answer• RQs should match the research design
• Should be worth investigating• A good RQ does not address a trivial issue; “so what?”• Answers to RQs should further the researcher’s understanding of
the phenomenon16
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
“It is better to have an approximate answer to the right question than an exact answer to the wrong one.”
John Tukey
17
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Types of variables that might appear in research questions• Dependent• Independent • Extraneous• Moderator• Mediator
18
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
19
Dependent variable(s)• The dependent variable is a phenomenon one is
attempting to explain or predict• Also referred to as the outcome or criterion variable
• In a survey study, often this is the variable or construct of primary interest to the researcher• It is something which the researcher wishes to
understand more fully• For many researchers conducting nonexperimental
research this is the starting point in planning a study
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
20
Independent variable(s)• In an experimental or quasi-experimental design,
an independent variable would be the manipulated or treatment variable in your study• In a survey or other type of nonexperimental
study, where no variables are manipulated, independent variables are the factors used to explain or predict the DV
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
21
Extraneous variables
• Extraneous variables are also called nuisance variables• These are variables which are not usually of
primary interest but are believed to be related to the independent and/or dependent variables• Their effects need to be controlled in order to
obtain meaningful results• Extraneous variables are of two broad types:• dispositional and situational
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
22
Illustration of controlling for extraneous variables
salaryjobsatisfaction
A researcher is interested in examining the relationshipbetween salary and job satisfaction. However, the researcher suspects that factors other than salary (suchas job autonomy) are also responsible for anemployee’s level of job satisfaction.
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
23
salaryjobsatisfaction
jobautonomy
Relationship with “job autonomy" takeninto account or controlled
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
24
Moderator variable(s)• A moderator variable influences the strength of
the relationship between the primary independent variable and the dependent variable• Usually a moderator variable is of secondary interest
• A moderator variable would be the variable that interacts with the independent variable
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
25
Illustration of moderator effect
Computer skills/comfort
Sat
isfa
cti o
n w
i th
onl i
ne c
ours
e
low aptitude
high aptitude
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
26
Mediator variable
• Also referred to as an intervening variable• A mediator (or mediating) variable is some
construct through which the independent variable indirectly affects the dependent variable
SR
M 6
27 S
urve
y R
esea
rch
Met
hods
, Hut
chin
son
27
Illustration of mediator variable
Parents’educationallevel
Achieve-mentmotivation
Performanceon standard-ized tests
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Examples of less than ideal RQs • What are psychologists’ perceptions of professional
development?• Are university faculty depressed?• What is the relationship between self-esteem and happiness?• How many undergraduate students dropped out of college in
2011?• What factors predict job satisfaction?• How are exercise and blood pressure related?• Is there a difference between gender and depression?• What is the impact of test anxiety on test performance?• Is religion good for society?
28
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What are psychologists’ perceptions of professional development?
• What is the researcher hoping to learn?• What are the variables/constructs in the RQ?• What is the target population?• How might this RQ be improved?
• Which aspects of professional development do psychologists rate most versus least positively?
29
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Or, instead of posing a descriptive question about a particular construct, consider rewording so that you can explain differences on the construct.
30
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
• To what extent are private practice psychologists’ perceptions of the need for professional development explained by their years of experience, area of specialization, and professional degree?
31
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Are university faculty depressed?
• What are the independent and dependent variables?• What assumption about university faculty is
implied by the research question? • What are possible revisions?
• Does level of depression among university faculty differ based on their gender, departmental affiliation, and tenure status?
32
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Are university faculty depressed?
• What are the independent and dependent variables?• What assumption about university faculty is
implied by the research question?• What are possible revisions?
• Does level of depression among university faculty differ based on their gender, departmental affiliation, and tenure status, after controlling for marital status and life satisfaction? 33
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Hypothetical depression scores
34
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
For complex constructs that are likely to be multiply determined, you should
attempt to control for potentially extraneous
variables.
35
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What is the relationship between self-esteem and happiness?
• What do you think the researcher wants to know?• How might the question be revised to more
clearly convey the type of information sought?
• What is the strength of relationship between self-esteem and happiness?• Is self-esteem linearly related to happiness
among college-educated adults? 36
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
How many undergraduate students dropped out of college in 2011?
• What is wrong with this question?• How might it be improved in order to provide
more useful information?
• Which of the following variables predicted undergraduate college student dropout in 2011? (grade point average, perceived social support, ….)
37
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What factors predict job satisfaction?
• What are the independent and dependent variables?• What is the target population?• How might you reword this RQ so that it provides
a clearer focus the researcher?
• To what extent do salary, autonomy, …. predict job satisfaction among full-time employees?
38
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
How are exercise and blood pressure related?
• What is the independent and dependent variable?• Target population?• How would you reword this RQ if you had reason
to believe gender could be a moderator variable?
• Does the strength of relation between amount of daily aerobic exercise and blood pressure differ for male and female adults?
39
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Is there a difference between gender and depression?
• What flaw do you notice in this RQ?• What do you think the researcher actually wants
to know?• What is a possible revision?
40
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
What is the impact of test anxiety on test performance?
• Assuming the researcher plans to collect data via survey, what part of this RQ needs to be reworded and why?• How might you revise this item if you wanted to
determine if study strategies was a mediator variable?
• Do study strategies mediate the relationship between test anxiety and test performance among doctoral level social science majors? 41
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Is religion good for society?
• Why is this not a suitable research question?• How could you revise the question so that it is
suitable?
• Are rates of violent crimes lower in communities with higher attendance at formal religious services?
42
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Other considerations when writing RQs• Be careful about your choice of words • Verbs such as improve, impact, affect, and predict
imply certain types of research designs• Think about how you plan to answer your
research questions as you write them• Consider the research design, measurement, and
data analysis
43
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Research question templates
• Do [groups of interest] differ on [name of construct]?
• Do male and female undergraduate students differ in their level of statistical test anxiety?
44
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Does [name of construct] differ by [variable]?
• Does level of participation in Greeley Parks & Recreation Center classes differ by age, educational level, gender, or ethnic group?
45
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• To what extent does a set of variables explain (or predict) a given outcome variable?
• To what extent do salary, working conditions, and types of job tasks predict job satisfaction?
46
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Is there a relationship between [variable] and [variable]?
• Is there a relationship between teachers’ job satisfaction and perceived level of administrative support?
47
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Is the relationship between [variable] and [variable] the same for [names of groups]?
• Is the relationship between teachers’ job satisfaction and perceived level of administrative support the same for novice versus experienced teachers?
48
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Does a given explanatory variable (or set of variables) explain a criterion, after taking into account one or more extraneous variables?
• Does child’s achievement motivation explain performance on statewide standardized tests, after controlling for parents’ SES and academic self-concept?
49
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Does [dependent variable] change [or improve or decline] across time among [population of interest]?
• Does reading level improve during a two-month summer intensive reading program for elementary school English language learners?
50
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Templates – cont’d
• Is the pattern of change [or improvement or decline] on [name of dependent variable] the same for [names of groups]?
• Is the rate of improvement in reading level the same for males and females?
51
CETL
Wor
ksho
p, Ju
ly 1
0, 2
012
Question Writing Exercise
• Consider a quantitative study you might conduct• Think about the overall purpose and how the
research questions will help support the purpose• You are to compose at least two research
questions, each of which should include a dependent variable and one or more independent variables.• One of your RQs will include one or more
extraneous variables and the other will include a moderator variable. 52