Upload
willis-mccoy
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Research Designs
By: Fatma and Wenshan
Outline
Introduction Short Break Research on instructional strategies Activity Trends and Development
Introduction: A Quick Review on Research Methods
Get A Research Method Quantitative Research
› Is the collection and analysis of numerical data to describe, explain, predict, or control phenomena of interest.
Qualitative Research is the collection, analysis, and interpretation of
comprehensive narrative and visual (i.e., non-numerical) data to gain insights into a particular phenomena of interest.
Mixed-methods research› Studies that combine qualitative and quantitative data
collection methods.
Overview of qualitative an quantitative research characteristics
Quantitative Research
Qualitative Research
Type of Data collected Numerical data Nonnumerical data
Research Problem Hypothesis and research procedures stated before beginning the study
Research problems and methods evolve as understanding of topic deeper
Manipulation of context Yes No
Sample size Large Smaller
Research Procedures Relies on statistical procedures
Relies on categorizing organizing data into patterns to produce a descriptive synthesis.
Participant interaction Little interaction Extensive interaction
Underlying belief We live in a stable and predictable world that we can measure, understand, and generalize about.
Meaning is situated in a particular perspective or context that is different for people and groups, therefore, the world has many meanings.
An example› What are the most significant differences
in the ways Grade 3 students at K.G school interact with inert media such as books, and with active media such as computer software for learning Mandarin vocabulary?
Break…
Research on Instructional Strategies
The rise, fall, and likely resurgence
Types of Experimental Design
Factors impacting the rigorous experiments on instructional strategies
Are media instructional strategies?
The Rise, Fall, and Likely Resurgence of Experimentation on Instructional Strategies
The Rise: during the first half of the 20th century
› “Although the full gamut of research methodologies, ranging from formative evaluation and design-based research to qualitative and case studies to true(randomized) experiments, has been used to investigate strategy effects, the predominant approach in the research literature has been experimental designs to increase objectivity and rigor. ”
Richey, Klein, & Wang, (2005)
What gave experimental designs initial impetus for researching instructional strategies?
› The influence of the field’s behavioral roots
› Experimentation has been traditionally viewed as the definition of acceptable research in the field
› Promotion and tenure criteria at colleges and universities have been strongly biased toward experimental studies
› The research journals have published proportionately more experimental studies than alternative types
Hannafin (1986)
The fall: 1983-2004
Hsieh et al. (2005)
1983
•47%
1995
•34%
2004
•26%
Percentage of randomized intervention studies published:
Factors attributed to the fall
› Resources and time
› Increase interest in and acceptance of qualitative studies over the years
The likely resurgence
Rigorous Studies
Randomized Experiments
Rigorous Regression Discontinuity Designs
Rigorous quasi-experiments
Ross and Morrison
Types of experimental designs
› Randomized experiment
› Rigorous quasi-experiment
Key Words (1)
› Internal validity
› External validity
Internal validity› The purpose of an experiment is to determine the
degree to which outcomes of the study are caused by the treatment (intervention), but not by extraneous factors. This degree to which one can draw valid conclusions about the causal effects of one variable on another is called internal validity.
“The most powerful design for maximizing internal validity is the true experiment . (Slavin, 2002)
Key words (2)
› Experiment A design that involves the comparison of one treatment to another,
using two or more different groups.
› Randomized Experiment A design that involves the comparison of one treatment to another,
using two or more different groups. In randomized experiment design, participants are randomly
assigned to treatment
› Quasi – experiment A design that involves the comparison of one treatment to another,
using two or more different groups. In Quasi-experimental design, participants are not randomly assigned
to treatment.
Research Methods
Quantitative Approach
Descriptive Study
Experimental Study
Randomized Experimental Design
Quasi-Experimental Design
Qualitative Approach
“The most powerful design for maximizing internal validity is the true experiment (Slavin, 2002).
An example of randomized experiment
› Does the “Bio-Sym”, a software package that allows simulated experiments , improve the grade 11 students’ learning in Biology 20?
(adopted from Using Experimental Methods in Higher Education Research Ross and Marrison, 2005)
An example of quasi-experiment
› Does “Bio-Sym”, a software package that allows simulated experiments , improve Grade 11 students’ learning in Biology 20?
(adopted from Using Experimental Methods in Higher Education Research Ross and Marrison, 2005)
Randomized experimental ? Quasi-experimental?
› Rummel et al. (2003) compared usage of a visual mnemonic to free study of a list of theorists’ names and accomplishments. Participants were randomly assigned to either a visual mnemonic or a free study treatment. The visual mnemonic showed an image of one or more individuals with objects to provide the mnemonic device and delayed post-test and a measure of attitude toward the effectiveness of the method. Result showed that participants in the mnemonic group remembered more of the targeted information than the free study group. For designers, this suggests that visual mnemonics are beneficial in helping learners recall important information.
Randomized experiment? Quasi-experiment?
› The study by Cavalier et. Al. (1995) of cooperative learning in a technical training course was conducted. Intact groups were assigned to either a cooperative learning treatment or control (conventional instruction) treatment, both of which were taught by the same instructor. Assessments included a post-test, attitude survey, and group interaction behaviors. Findings showed that practice in a cooperative group environment produced superior achievement as well higher levels of social and cognitive interactions.
Activity
In a small group, › Pick a study topic.
› Discuss whether you would choose true experimental design or quasi-experimental design to your study, and Why?
› Briefly identify what your research process would be.
Factors Impacting the rigor of Experiments on Instructional Strategies
› Types of validity threats› Balancing Internal and External Validity› Basic-applied design replications
Type of validity threats› Group› Time› Teacher› Historical events› And…
“Quasi-experimental designs have the advantage of convenience and practicality due to the random assignment of intact classes rather than individual students to treatments. A disadvantage is reduced internal validity due to the possibility of certain treatment groups, by chance, having more capable students or more favourable learning conditions.” (Ross & Morrison 205)
Other possible validity threats
› Does “Bio-Sym”, a software package that allows simulated experiments , improve Grade 11 students’ learning in Biology 20?
Balancing internal and external validity
Basic-applied design replications
Are media instructional strategies?
Trends and Future of Research
Trends in Design & development Research .
Trends in Product & Tool Research.Trends in Model Research.
The Future of Design & Development Research.
Instructional Technologies:A Brief History
Instructional Technology: Looking Backward, Thinking Forward
A Summary of Reading
Research on Instructional Strategies Research on Educational Technologies
› Does it work? Educational Technology: What Works
Research on instructional design models
Research on instructional design and development
Research Designs
Research on Instructional Strategies
Strategies
Research on Educational Technologies
Technologies
Research on Models for
Instructional Design
Models
Research on Design and
Development
Design and Development
Thank You !
References:› Van
Richey, R. C., Klein, J. D., and Nelson, W. A. (2004). Developmental research: studies of instructional design and development. In Handbook of Research on Educational Communications and Technology, 2nd ed., edited by D. H. Jonassen, pp. 1099–1130. Mahwah, NJ: Lawrence Erlbaum Associates.
Ross, S. M., Morrison, G. R., and Lowther, D. (2005). Using experimental methods in higher education Research. J. Comput. Higher Educ., 16(2), 39–64.*
The Rise of Behaviorism--The Psychology of Pavlov, Watson, and Skinner By Kendra Van Wagner, About.com
Hsieh, P., Acee, T., Chung, W., Hsieh, Y., Kim, H., Thomas, G., Levin, J. R., and Robinson, D. H. (2005). Is educational intervention research on the decline? J. Educ. Psychol., 97(4), 523–529.
Levin, J. R. (2004). Random thoughts on the (in)credibility of educational–psychological intervention research. Educ. Psychol., 39(3), 173–184.* http://bobhall.tamu.edu/epsy602/Topics/Articles/Levin.pdf