Upload
daniel-mcgee
View
228
Download
3
Tags:
Embed Size (px)
Citation preview
Educational Research: Educational Research: Descriptive ResearchDescriptive Research
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Research...Research...
The systematic application of a family of methods employed to provide trustworthy information about problems…an ongoing process based on many accumulated
understandings and explanations that, when taken together, lead to generalizations about problems and the development of theories
Quantitative research methods...Quantitative research methods...
…collect and analyze numerical datadata obtained from formal instrumentsinstruments
Quantitative methods...Quantitative methods...
descriptive research (“survey research”)correlational researchcausal-comparative research (“ex post facto research”)
experimental research
descriptive researchdescriptive research (“survey research”)…collects data in order to answer
questions about the current status of the subject or topic of study
…uses formal instruments to study preferences, attitudes, practices, concerns, or interests of a samplesample
The basic steps of descriptive The basic steps of descriptive research...research...
An orderly scientific and disciplined process, involving:
selecting an appropriate sample of participants
collecting valid and reliable data reporting conclusions
recognizing and identifying a topic to be studied
Classifications of descriptive Classifications of descriptive research studies...research studies...
cross-sectional surveycross-sectional survey……involves collecting data from
selected individuals in a single time period however long it takes to collect data from participants
longitudinal surveylongitudinal survey……involves collecting data at two or more
instances in order to measure changes over time
self-report surveyself-report survey……requires individuals to respond to a
series of statements or questions about themselves
observation studyobservation study……the researcher obtains data by
watching participants in a situation
Types of longitudinal surveys...Types of longitudinal surveys...
trend surveycohort surveypanel surveyfollow-up survey
trend surveytrend survey…a study where a sample is taken from
the general population in order to collect data over time
…involves different groups and different samples over time
cohort surveycohort survey…a study where a specific population is
examined by sampling different groups within the population over time
…involves the same group but different samples from that group over time
panel surveypanel survey…a study where the same group and the
same sample are examined over time
follow-up surveyfollow-up survey…a study undertaken after a panel
survey and seeks to examine subsequent development or change
Conducting a questionnaire study…Conducting a questionnaire study…
1. state the problem2. select participants3. construct the questionnaire4. prepare cover letter5. pretest questionnaire6. follow up activities
7. analyze/report results
1. state the problem……topic must be of sufficient significance to
motivate potential respondents and justify the research effort in the first place
…define topic in terms of specific objectives indicating the kind of information needed
2. select participants……use an appropriate sampling
technique…participants must be able to provide
the desired information sought and willing to provide it to the researcher
3. construct the instrument……should be attractive, brief, and easy
to fill out…there are a variety of data collection methods
that can be used, including: questionnaires, interviews, observations, and telephone calls
types of self-report items…scaled…ranked…checklist…free response
…include a brief statement describing the study and its purpose at the top of the instrument
…provide standardized directions …organize items leaving sufficient
white space on instrument
Designing an instrument…Designing an instrument…
…place similar items together
…ask general, non-threatening items first, moving into more specific, self-revealing items
some do’s and don’t’s……construct items according to a set of
predetermined guidelines…include only items relating directly to
the purpose of the study…avoid jargon or any term or concept
that might mean different things to different people
…each item should deal with a single concept and be worded as clearly as possible
…items should indicate a point of reference beyond the self in order to judge the self
…avoid ambiguous terms like several or usually
…avoid touchy matters in items which respondents might not respond to honestly or at all
…avoid leading questions which suggest that one response may be more appropriate than another
…each item must be able to stand on its own
…don’t jam items together…don’t put the most important
questions at the end of the instrument
4. prepare the cover letter……a brief, neat, explanation of the
significance of the study and what is being asked of the respondent and why
…addressed to the specific, potential respondent
…an endorsement adds credibility…guarantee anonymity anonymity or
confidentialityconfidentiality…include a specific deadline date by
which the completed instrument (“questionnaire”) is to be returned
…individually sign each letter…include a self-addressed, stamped,
return envelope
5. pretest the instrument……conduct a pilot studypilot study to gather
information about deficiencies and suggestions for improving the instrument
…provides greater content validity
6. follow-up activities……because first mailing results tend to be low
(30% - 50% response rate), researchers need strategies to increase the response rate
initial follow-up strategies to increase response rate up to 20%…
…send out reminder postcard…mail a second packet with a new,
positively worded cover letter and another self-addressed, stamped, return envelope
secondary follow-up strategies to increase response rate up to 10%…
…telephone nonrespondents to encourage them to respond
dealing with nonrespondents…study whether nonrespondents differ from
respondents in some systematic way by selecting a sample from the nonrespondents
…telephone sample, aggregate data, include in report
dealing with nonresponses…study the items to determine the
problem with the item…include description of the
nonresponses and the determination of the reason in the report
7. analyze/report results……discriminant item analysisitem analysis includes giving the
response rate for each item as well as the total sample size and the overall percentage of returns, since not all respondents will answer questions
…group items into clusters that address the same issue and develop total scores across an item cluster in order to avoid “information overload”
Conducting an interview study…Conducting an interview study…
1. state the problem2. select participants3. construct the interview guide4. communicate professionally and record responses accurately
5. pretest interview procedure
6. analyze/report results
1. state the problem……topic must be of sufficient significance to
motivate individuals to participate and justify the research effort in the first place
…define topic in terms of specific objectives indicating the kind of information needed
2. select participants……use an appropriate sampling
technique…participants must be able to provide
the desired information sought and willing to provide it to the researcher
3. construct the interview instrument (“instrument guideinstrument guide”)…
…indicates the question to be asked, the order, and how much additional prompting or probing is permitted
…the goal is to obtain standardized, comparable data from each interviewee
4. communicate professionally and record responses accurately…
…effective interviewing requires training and experience to avoid interviewer bias and interviewer error
…record responses manually on the interview guide
…(requires interviewee permission) use a tape recorder or VCR to verify accuracy of responses
5. pretest interview procedure……use a small group from the same population or a
similar population to the one being studied to validate the interview instrument and the interviewer’s communication and recording skills
6. analyze/report results……item analysis includes giving the response rate for
each item, as well as the total sample size and the overall percentage of returns, since not every respondent will answer all questions
…also include in-depth data to open-ended interview items to portray responses in more accurate and honest terms
Conducting an observational study…Conducting an observational study…
1. state the problem2. select participants3. define the observational variables4. record observations5. assess observer reliability6. reduce observer bias
7. analyze/report results
1. state the problem……topic must be of sufficient significance to
motivate potential respondents and justify the research effort in the first place
…define topic in terms of specific objectives indicating the kind of information needed
2. select participants……use an appropriate sampling
technique…participants must be able to provide
the desired information sought and willing to provide it to the researcher
3. define the observational variables……operationalize the variables to be
observed in terms of specific behaviors that can be quantified
…ensure that the observations can be quantified in a way so that all observers will count the behavioral activities in exactly the same way
…simplify procedure for recording observations by developing a coding procedurecoding procedure
4. record observations……record behavior at the time it occurs…alternate observation periods and
recording periods
5. assess observer reliability……use at least two observers who make
independent observations…compute interobserver reliabilityinterobserver reliability
6. reduce observer bias……train observers until a satisfactory
level of agreement is achieved (at least 80%)
…monitor observers
types of observer bias…response set…halo effect…knowledge of participants
response setresponse set…the tendency of an observer to rate the majority of
observees as above average, average, or below average regardless of the observees’ actual behavior
halo effecthalo effect…the tendency of an observer’s initial positive
or negative impressions of the observee to affect subsequent observations
self-fulfilling prophecyself-fulfilling prophecy…the tendency of an observer’s knowledge
of the observees or the purposes of the study to affect the observation(s)
Meta-Analysis...Meta-Analysis...
a statistical approach to summarizing the results of many studies that have investigated basically the same problem
…provides a numerical coefficient expressing the “average” result of the studies
…requiring the researcher to find, describe, classify, and code the research studies to be included meta-analytic review, and for measuring and analyzing the study findings
…each study’s results are translated into an effect effect sizesize (ES) which is a numerical expression of the strength or magnitude of a reported relationship, be it causal or not
Xe – Xc
ES = SDc
Where Xe = the mean score of the experimental group
Xc = the mean score of the control group
SDc = the standard deviation of the control group
Strength or magnitude of an effect Strength or magnitude of an effect size (size ()…)…
-.80 +.80
strong negative strong positive
control group performed better
than experimental group
experimental group performed
better than control group
0.00
both groups performed similarly
Mini-Quiz…Mini-Quiz…
True or false……in a descriptive research study, the
researcher predetermines what variables will be surveyed before selecting or observing the research participants
True
…one of the most difficult activities on the part of questionnaire researchers is to write or select questions that are clear and unambiguous
True
…longitudinal surveys are useful for studying the dynamics of a topic or issue over time
True
…one of the problems with longitudinal studies is that the samples tend to shrink as time goes by
True
…one of the problems with cross-sectional studies is selecting samples that truly represent the population at a particular level or ability
True
…an external review of an instrument provides the researcher greater assurance of it its content validity
True
…if the responses from respondents and nonrespondents are essentially the same, the researcher may assume that the response group is representative of the whole sample and that the survey results are generalizable
True
…if the responses from respondents and nonrespondents are different, the generalizability across both the respondent and nonrespondent groups is not present and must be discussed in the report
True
…analyzing clusters of instrument items related to the same issue improves the reliability of the scores
True
…bias can affect the validity of the interpretations in observational studies
True
…although there are no hard and fast rules, it is generally agreed than an effect size in the twenties indicates a treatment that produces a relatively small effect, whereas an effect size in the eighties indicates a powerful treatment
True
Fill in the blank…
…a study requiring individuals to respond to a series of statements or questions about themselves
self-report study
Fill in the blank…
…a study in which individuals are not directly asked for information but data is obtained as the researcher watches participants engage in a situation
observation study
Fill in the blank…
…items on a survey that are answered by circling a letter, checking a list, or numbering preferences
close-ended items
Fill in the blank…
…the researcher guarantees that no one, including the researcher, knows who completed each questionnaire
anonymity
Fill in the blank…
…the researcher guarantees that no one, except the researcher, knows who completed each questionnaire and promises not to divulge that information
confidentiality
Fill in the blank…
…the oral, in person administration of an instrument to each member of a sample
interview
Fill in the blank…
…determining the current status of a phenomenon not through an instrument but as the researcher watches the participants engage in the activity being studied
observational study
Fill in the blank…
…an observation in which the researcher watches behavior as it normally unfolds
naturalistic observation
Fill in the blank…
…an observation in which the researcher creates a situation to be observed and tells participants what activities to engage in
simulation observation
Fill in the blank…
…a coefficient found by dividing the total number of agreed observations by the total number of agreed and disagreed observations
inter-observer reliability
Fill in the blank…
…a means by which observers record observations at the time it occurs by simplifying the recording process
coding
Fill in the blank…
…a statistical approach that summarizes the results of many studies having investigate the same problem
meta-analysis
This module has focused on...This module has focused on...
…which involves collecting data in order to test hypotheses or to answer questions about the opinions of people about some topic or issue
descriptive research
The next module will focus on...The next module will focus on...
...which involves collecting data in order to determine whether, and to what degree, a relationship exists between two or more quantifiable variables
correlational research