Download ppt - UKM Quantitative and Qualitative Research Design

Quantitative and Qualitative Research Design

Prepared By:Dr. Nik Rahimah Yacob

Positivist Paradigm

Quantitative Methods

Verification

Interpretivist Paradigm

Qualitative Methods

Discovery

The Research Process by Cavana et al. (2001)Catalyst for business research

• Opportunity

• Problem

Preliminary information gathering and literature survey

Problem definition

Framework development

• Conceptual

• Theoretical

Research objectives

• Research questions

• Hypotheses

Research design Data collection

• Qualitative

• Quantitative

Data analysis

• Qualitative

• Quantitatve

Interpretation of findings

Report preparation and presentation

Management action

• Plan

•Implement

• Monitor

Research Design by Cavana et al. (2001)

Problem Statem

ent

Purpose of the study

• Exploration

• Description

• Hypotheses testing

• Case Study

Types of investigation

• Clarification

• Causal

• Correlational

• Experimental

Extent of Researcher Interference

• Minimal

• Manipulation

Measurement and measures

• Operational definition

• Items (measure)

• Scaling

Qualitative data collection

• Interviews

• Focus groups

• Observation

Data analysis

Unit of Analysis

• Individuals

• Dyads

• Groups

• Organisations

• Machines, etc.

Study setting

• Contrived

• Non-contrived

Time Horizon

• One-shot (cross-sectional)

• Longitudinal

Sampling design

• Probability/ non-probability

• Sample size

Quantitative data collection

• Questionnaires

• Experimental designs

A Classification Of Research Data

RESEARCH DATA

SECONDARY DATA PRIMARY DATA

QUALITATIVE DATA

QUANTITATIVE DATA

DESCRIPTIVE CAUSAL

EXPERIMENTAL DATA

OBSERVATIONAL & OTHER DATASURVEY DATA

QUANTITATIVE DATA

QUALITATIVE DATA

A Classification of the Qualitative Research Methods

QUALITATIVE RESEARCH PROCEDURES

DIRECT (NONDISGUISED)

INDIRECT (DISGUISED)

FOCUS GROUPS

PROJECTIVE TECHNIQUES

EXPRESSIVE TECHNIQUES

CONSTRUCTION TECHNIQUES

COMPLETION TECHNIQUES

DEPTH INTERV

ASSOCIATION TECHNIQUES

SECON-DARY DATA

CASE STUDY

Quantitative Research Approaches/Methods

A Classification Of Survey Methods

SURVEY METHODS

TELEPHONE PERSONAL MAIL

IN-HOME MALL INTERCEPT

COMP-ASSISTED PERSONAL

INTERVIEWING

MAIL INTERVIEW

MAIL PANELTRADITIONAL

TELEPHONECOMPUTER-ASSISTED

TELEPHONE INTERVIEWING

A Classification Of Observation Methods

OBSERVATION METHODS

PERSONAL OBSERVATIO

N

MECHANICAL OBSERVATIO

NAUDIT

CONTENT ANALYSIS

TRACE ANALYSIS

A Classification Of Experimental Designs

EXPERIMENTAL DESIGNS

PREEXPERIMENTAL TRUE EXPERIMENTAL

QUASI EXPERIMENTAL STATISTICAL

One-Shot Case StudyOne Group Pretest-PosttestStatic Group

Pretest-Posttest Control GroupPosttest Only Control GroupSolomon Four Group

Time SeriesMultiple Time Series

Randomized BlocksLatin SquareFactorial Design

Qualitative Research Approaches/Methods

The continuum model for interviews

Structured interviews Semi-structured interviews Unstructured interviews

• Standardised interviews* Survey interviews

* In-depth interviews* Survey interviews* Group interviews

* In-depth interviews* Group interviews• Oral or life-history interviews

The Pattern of an Interview (Source: Delahaye (2000:166))

Level of defence barriers

Time elapsed

Intimacy

Entrance time investment

Exit time investment

Rapport zone

Activity no.1

Activity no. 2 – a series of question sequences

Preview

Rules

Reason

Pass time

Ritual

Future action

Final questions

Pass time

Ritual

Focus Groups

Depth interviewing of a group of 5 to 12 people; researcher serves as a moderator

• Logistics• Group Composition• Homogeneity• Representation• Strangers vs acquaintances• Size of group

Case StudyBy understanding a single system a researcher canbetter understand similar instances and address the problems and issues identified in one case.

As bounded systems of time and space (context), casesrely on multiple sources of information to providean in-depth picture of an organisation or situation (phenomenon) under study.

Via observation, interviews, documents, or surveys.

Survey Methods

Survey as a Research Approach• It is a quantitative method• Capitalizes on the communication approach (respondents are required

to communicate their responses to the researcher through a structured or unstructured questionnaire)

• Involves the creation and selection of the measurement questions• Sampling issues which drive contact and call-back procedures• Instrument design which incorporates attempts to reduce error and

create respondent-screening procedures• Data collection processes which create the need for follow-up

procedures and possible interviewer training

A Classification of Survey Methods

Survey Methods

Cross-sectional

design

Longitudinal design

On Sampling…..• Capitalizes on a relatively large sample size• The sample can either be drawn on a probability

sampling procedure or a nonprobability sampling procedure, depending on the purpose of the study

On Instrument Design….• The instrument for a survey is a questionnaire. • The questionnaire can either be highly structured (close-ended

questions) or highly unstructured (open-ended questions) • The norm is to utilize a structured questionnaire for ease of

data coding and analysis. • The questionnaire design has to take into consideration the

data collection method (personal interview, telephone interview, mail survey, Internet survey, mall intercept or self-administered)

Data Collection Method

SURVEY METHODS

TELEPHONE PERSONAL MAIL

IN-HOME MALL INTERCEPT

COMP-ASSISTED PERSONAL

INTERVIEWING

MAIL INTERVIEW

MAIL PANELTRADITIONAL

TELEPHONECOMPUTER-ASSISTED

TELEPHONE INTERVIEWING

Examine the pros and cons of each method prior to selecting a particular method for your study.

Observation Methods

Observation as a Research Approach

• It can either be a quantitative method or a qualitative method depending on whether the purpose of your study is to verify or to discover

• Capitalizes on visual data collection. It also involves listening, reading, smelling and touching

• Monitors a full range of behavioural (nonverbal, lingusitic, extralinguistic and spatial analysis) and nonbehavioural (record, physical condition and physical process analysis) activities and conditions

• Sampling issues • Instrument design• Data collection process

A Classification of the Observation Methods

Observation methods

Direct observation

Indirect observation

On Sampling…..• Capitalizes on a relatively large sample size for a

quantitative study and a small sample size for a qualitative study

• The sample can either be drawn on a probability sampling procedure or a nonprobability sampling procedure for a quantitative study

• The sample is almost always drawn on a nonprobability sampling procedure for a qualitative study

On Instrument Design….• The instrument for an observation study is known as an

observation checklist or an observation form • The observation checklist or form can either be highly

structured (clear indications of what to observe and how to tally the observation) or highly unstructured (vague ideas on the scope of observation and what to observe)

• Observation checklist or form for quantitative research would tend to be more structured than that of a qualitative research

• The observation checklist or form has to take into consideration the data collection method (personal observation, mechanical observation, audit, content analysis or trace analysis)

Data Collection Methods for Observation Studies

OBSERVATION METHODS

PERSONAL OBSERVATIO

N

MECHANICAL OBSERVATIO

NAUDIT

CONTENT ANALYSIS

TRACE ANALYSIS

Examine the pros and cons of each method before selecting a particular method for your study

Experimentation

“Ex post facto research designs, where a researcher interviews respondents or

observes what is or what has been, also have the potential for discovering causality. The

distinction between these methods and experimentation is that the researcher is

required to accept the world as it is found, whereas an experiment allows the researcher to alter systematically the variables of interest

and observe what changes follow.”Source: Cooper & Schindler (2003, pp.424-425)

Beaumind Soft sdn bhd

What is experimentation?

• Involves at least one independent variable (IV) and one dependent variable (DV) in a causal relationship.

• The IV constitutes the intervention or manipulation and its effect on the DV is measured.

• Three requirements for drawing a causal conclusion are:

Concomittant variationTime occurrence of variablesControl over extraneous factors


Advantages & Disadvantages of Experimentation


ADVANTAGES:

• Ability to manipulate the IV

• Increases the probability that changes in the DV are a function of the changes in the IV

• Use of control group strengthens the causality finding

• Convenient and cost effective

• Ease of replication

DISADVANTAGES:

• Artificiality of the setting

• Generalizability of the study findings

• For some experiments, they can be costly

• Limited to issues of present and immediate future not past

• Due to ethical considerations, there is a limitation on manipulation on people

IV and Experimental Treatments

• Experimental treatments are the various levels in the manipulation of the IV

• Normally, one level of the IV constitutes the control• For the following RQ of an experiment:

Does the new ad generate more sales than the existing ad?

IV = Advertisement DV = SalesLevels of IV = Treatments = 1. New ad (experimental

treatment) 2. Existing ad (control)


Sampling and Randomization• With two treatments in the experiment, there must be two

experimental groups• Randomization can happen at three stages: Stage 1 - Sample selection

Stage 2 – Group division Stage 3 - Assignment of treatment to group • Stage 1 randomization is not as important as the other stages. • Stage 2 randomization is important to equate the characteristics

of the two groups. When this is not possible then use a matching technique.

• Stage 3 randomization defines the experimental method. Without this randomization the experimental design is flawed.


Experimental Research Designs

• Pre-Experimental DesignsOne-Shot Case Study X OOne-Group Pretest-Post-Test O X O

Static Group Comparison X O1

O2

• True Experimental Designs Pretest-Post-Test Control Group R O1 X O2

R O3 O4

Post-Test-Only Control Group R X O1

R O2


Internal Validity of Experiments

• Maturation• Testing• Instrumentation• Selection• Statistical regression• Experiment mortality


Depth Interview

Accuracy and Replicability1. Trustworthiness

2. Verification

3. Acknowledging subjectivity and bias

4. Process and sequence

5. Interpretation

6. Referential adequacy

7. Paint the path

It takes two people to speak the truth-one tospeak, and the other to listen (Henry Thoreau 1849)

Field Interview Techniques

• Interviewing in the field requires a researcher to develop a rapport with interviewees or informants.• Researchers often apprehensive about engaging strangers in conversation.• The objective is to keep the informant talking .

Field InterviewsBuilding Rapport / Relationships

Gain confidence through purposeful conversation.

By explaining what you want to know.Repeating key phrases used by interviewee.Restating what they say in your own words.

Don’t have questions asking for meanings or motives:“what do you mean?” or “why would you?”

These contain hidden judgments, interviewees may thinkthey have not clearly explained things or answered your questions!

Improving Interviewee Responses

The basic probe: repeat the initial question, is useful when interviewee wanders off track. Explanatory probe: building onto incomplete or vague statements;

asks: what did you mean by that? Focused probe: to obtain specific information;

asks: what sort of X? The silent probe: pause and let interviewee break silence drawing out: when interviewee has halted or slows simply repeat the last few words from the interviewee then look expectant or say tell me more Giving ideas or suggestions: offering an idea or suggestion to think about; have you thought about? Did you know? Mirroring or reflection: express in your (interviewer) own words what interviewee just said; what you seem to be saying is X?

Focus Group Method

Design Issues

• Logistics• Group Composition• Homogeneity• Representation• Strangers vs acquaintances• Size of group

Conducting the Focus Group

1. Use pattern of interview as guide2. Specific considerations

– Facilitator team– Recording– Use of visual aids– Thinking time– Group dynamics

Procedure for Planning and Conducting Focus Groups

Determine the objectives of the Marketing Research Project and Define the Problem

Specify the objectives of Qualitative Research

State the objectives/questions to be answered by Focus Group

Write a Focus Group Protocol

Develop a moderator’s outline

Conduct the Focus Group interviews

Review tapes and analyze the data

Summarize the findings and plan follow-up research or action

Case Study Method

Case studies

Case studies can be single or multiple

Within case analysis or across cases analysis

Case studies are: where a researcher explores a singleentity or phenomenon (the case) bounded by time and activity(a program, event, institution, process or group) and collects

detailed information by using a variety of data collection procedures (Yin 2001).

Selecting a Research Method Research Strategy Type of research

Question Requires Control over Events

Focus on Contemporary Events

Experiment HOW , WHY YES YES Survey WHO, WHAT, WHERE,

HOW MANY, HOW MUCH

NO YES

Archival Analysis WHO, WHAT WHERE, HOW MANY, HOW MUCH

NO YES/NO

History HOW, WHY NO NO Case Study HOW, WHY NO YES

Source: Yin (1989:17)

Case Study for four design tactics. Tests Case Study Tactic Phase of Research Construct Validity Use multiple sources of evidence.

Establish chain of evidence. Have key informants review draft report.

Data collection Data collection

Composition

Internal Validity Do pattern matching

Do explanation building Do time-series analysis

Data analysis Data analysis Data analysis

External validity Use replication logic in multiple case studies

Research design

Reliability Use case study Protocol Develop case study data base

Data collection

Data collection

Source: Yin(1989:p41)

Case study protocol components Case Study Protocol Components Component Requirements Overview Objectives & Auspices

Study Issues Relevant Readings

Field Procedures Credentials Access to Site General sources of Information Procedural Reminders

Case Study Instrument Questions Specific Questions Potential sources for Answers

Case Report Guide Outline Format Additional Documentation

Source: Yin(1989:p70)

Quality Case Study Designs(design & collection issues)

Construct Validity:Establish & follow appropriate (correct) operational measures for theConcepts (phenomenon) being studied.

External Validity:Establish the domain to which the findings can be generalised.

Reliability:Demonstrate that the operations of the study (data collection methods)Can be repeated, achieving similar results.

• All aspects must be adhered to throughout the case study life and in effect there is a strong link between design and collection.


Establishing the Chain of Evidence (construct validity)

Case study report

Case database

Reference to specific sources in the database

Case study protocol

Case study questions

Source: Yin, 2003


Generalisation (external validity)

Generalisation to THEORY not populations

Criticisms: cases cannot be generalised (or poor generalisation)

Generalisation to samples – universe is not correct as survey designs rely on statistical generalisation and

cases use theoretical generalisation through the results.

Generalisation requires replication via multiple cases of the phenomenon (this is replication logic)


Reliability

Emphasis on replication

The key is doing the same case again, not replicating the findings, i.e., results

The objective of reliability is to minimise error and bias

This is achieve through correctly detailed plans of actionsundertaken (a-priori and post gathering) through protocols,Questions, histories etc

Basic Types of Case DesignSingle V Embedded Designs

Context

Case

SINGLE MULTIPLE

Case 1

Context

Case 2

HOLISTICSINGLE

EMBEDDEDMULTIPLE

Context

Case

Context

Case

Context

Case

Context

Case

Case 1Context

Case 2Case 1

Context

Case 2

Case 1Context

Case 2Case 1

Context

Case 2

Source: Yin, 2003

Basic Types of Case Design

Multiple Case Studies

Literal Replication V Theoretical Replication

Literal sees the purpose of the cases being to predictsimilar results (findings).

Theoretical sees the purpose of the cases being topredict different results (outcomes) for theoreticallypredictable reasons.

Basic Types of Case DesignSingle Case Study

Critical case: tests well established theory is correct or alternative explanations are found and acceptable.Extreme case: rare situation is found and studied to establish theory.Representative case: study situation identified to be common and theory explains how situation occurs or functionsRevelatory case: study a situation not previously able to be examined.Longitudinal case: same situation studied at multiple points of time to see how things change over the stages (time).