BY TEMTIM ASSEFAOCTOBER, 2013

Quantitative Research Method (Research Design)

Research Design It is the entire design of the research projectIt is said as the research proposal It involves deciding on all aspects of the research

process That includes

Philosophical assumption Research method Data collection techniques Data analysis Publication outlet if possible

It matters to get your proposal accepted or rejected

Overview of Quantitative Research

Uses deductive method of knowledge acquisition

Intends to falsifies an existing theory Tries to generate generalizable knowledge

Accepts objectivity of knowledge Uses a standard measurement instruments The researcher is independent in the process

of knowledge constructionQuantifies the phenomena in terms of numbers Intends with prediction and controlling

Types of Design

Research methods used in Quantitative research

method include Survey /field study field experiment, laboratory experiment, simulation, and case study

Survey Design

Webster’s Dictionary defines a survey as “a general study or inspection”

A survey is a means of "gathering information about the characteristics, actions, or opinions of a large group of people, referred to as a population" [Pinsonneault and Kramaemer]

Epistemologically surveys provide one way of obtaining and validating knowledge

Survey Research

Survey for research is distinctly different Focus on surveys that are conducted to advance

scientific knowledge, which we refer to as survey research.

Distinctiveness of survey research

The purpose of the survey is to produce quantitative descriptions of some aspects of the studied population.. requires standardized information from and/or about the subjects being studied relationships between variables projecting findings descriptively to a predefined

population The main way of collecting information is by asking

people structured and predefined questions. Their answers constitute the data to be analyzed.

Distinctive ….

Information is generally collected about a fraction of the study population--a sample--but it is collected in such a way as to be able to generalize the findings to the population-such as service or manufacturing organizations, line or staff work groups, MIS departments, or various users of information systems.

Techniques

Survey research combines three techniquesCollection of answers using standardized

questionnairesRandom sampling from a known

populationStatistical analysis of a quantified

representation of the survey answers

Purposes of Survey

Exploratory Little is known about a population Further information is desired about research variables Prelude to a costlier, larger research endeavors

Descriptive Focus on who, what, when, how Existence of opinions and attitudes

Explanatory Cause and effect, focus on why Reasons for existence of facts and opinion is of

interest

Explanatory Requires Conceptual Framework

Participation

Conflict Conflict resolution

Influence

Conceptual Framework of User Involvement in system development

Survey stages

1. Clarify the research problem2. Develop conceptual framework if explanatory design 3. Decide what type of survey design is appropriate4. Decide what type of sample design is most appropriate5. Decide on the size of sample6. Plan sampling procedures7. What form of data collection8. Design the questions and test your instruments 9. Undertake the fieldwork and data collection10. Process the data 11. Analyze the data and interpret your results12. Disseminate your findings

Clarify the research problem

Move from general idea to focused topicsDegree to which study meets its aims will be

determined by relevance and completeness of research questions addressing the research problem

Empirical to theoretical mapping

Decide type of survey

Cross-sectional or one-offLongitudinal or time series ( collection of data at two

or more points in time) Before and after studies Panel studies

Combination of Cross-sectional and longitudinal

Sample design

Decide which type of sample design is most appropriate Probability samples

Simple random samples Cluster samples

Non-Probability samples Quota samples Volunteer samples Convenience samples Snowball samples Purposive sampling

Errors

Sampling errors Difference between the results you will get from a full census

and the results you will get from a sampleNon-Sampling error

Systemic Random

Sample Size

Inevitably a compromise between methodological imperative and practical constraints

Determined by level of sampling errorHow about Non-response?Formula [Calder Judith, 1998]

n = (desired confidence level)2 * (s.d) 2 /desired level of precision) 2

Decide form of data collection

QuestionnairesInterviewsObservation

Data Collection

Some steps Organize in advance Agree access to your respondents Collect the data Monitor progress Chasing up Quality of the process of fieldwork and data collection

Report honest and reflect on fieldwork in research reports[Calder Judith]

Finally – Interpret Data

Analyze the data and interpret your results Does the data make sense? Beware of spurious patterns

Present and disseminate your findings

Benefits

Theoretical propositions can be tested in an objective fashionEnable generalizationsEasy to use (although may be an illusion)Established and accepted research method across multiple fieldsAided by statistical toolsImmensely helped by advancement computingStrong in external validity if sampling errors are minimized.

Weaknesses

Difficulty in obtaining truly random samplingLow response rates. Business and IT staff are inundated

with surveys Weak linkages between units of analysis and respondents,

and Over-reliance on cross-sectional surveys where longitudinal

surveys are really neededsingle-method designs where multiple methods are needed

Inappropriateness Different results with behavioral observation and self report

Field Experiment

Field Experiment

A field experiment is an experimental research method which is performed outside the laboratory in the natural settings

It follows all steps of scientific process1) selection and determination of a problem,

2) selection of participants and measuring instruments,

3) selection of a research plan,

4) execution of the plan,

5) analysis of data and

6) formation of conclusions

Features of FE

It has three unique features1. the research takes place in

natural setting

2. the experimenter manipulates one or more independent variables while exerting much control over confounding variables; and

3. the effect of the manipulations on one or more dependent variables is systematically observed.

FE Design

There three type of designs

Controlling Nuisance in Field Experiment

Solomon Four Groups in True ExperimentExperimental group 1: R O X O Control group 1: R O O Experimental group 2: R X O Control group 2: R O

Removing statistically the effect of suspected confounding variables

Example – Interface Design and Computer use

System use is low among employees. One reason is poor interface design. You divide the employees into two groups (Group 1 & 2)Redesign the Interface and let Group 1 work on the new

system - Treatment GroupLet Group 2 work on the old Interface– Controlled

GroupAfter six or some months after Interface design,

measure, the computer use.If computer use is increased for Group 1, You can

conclude that Interface design and computer use has association

Data collection

The following methods are used to collect data

Questionnaire

Structured interview

Computer log files

Data Analysis

Statistical Data analysis methods are usedFor example

T-test is used to compare the mean between treatment and controlled groups if they significantly differ

Does Group 1 mean is different from Group 2 that can be generalized for the entire population

ANOVA is used for groups more than two Correlation is used to observe association

Doe increase one variable follows the same pattern in other variable

Advantages of FE

Conducted in natural settings and has higher experimental validity

Help to clearly identify antecedents of observed effects in casual relationships

Used for the development of theory as well as for the solution of applied problems;

The logic of FE can be applied in the analysis of many naturally occurring changes

Help for testing of broad hypotheses dealing with complex social process in lifelike situations.

Disadvantage

The methods used to control confounding variables in FE are not sufficient

Manipulation of variables cause legal and ethical problemsDifficult to control dynamically changing environment

during the course of the experimentDifficult to precisely measure dependant variables in the

field settingsExpensive to conduct compared to lab experimentNeeds highly skilled person to design and conduct

Role of FE in computer Science

It has many applications in IS To develop theory To test hypothesis To evaluate IS tools and techniques

Practical ExamplesFukada et al (n.d.) used field experiment to study the effect

of Road Facility Management Support System on asset

management of the public infrastructure.

Mayur et al (2000) made study to compare and suggest the

best training method for companies between instructor

based training and computer based training methods.

Chen et al (2007) made a field experiment study to identify

effects of different types of social information on

contributions to an online communities.

Instrument Design

Some Notes on Questionnaires design

Questionnaires make use of lists and rating scalesBehaviors and attitudes are complex and cannot

be easily evaluated and quantifiedCheck list is a list of behaviors, characteristics

or other entities that a researcher is investigatingEither the researcher or participants simply

check(s) items from the listWhat are features of user friendly software

1. Graphical interface2. Clear navigation direction3. Immediate feedback4. Other specify ____________

Questionnaire …

A rating scale is more useful when a behavior, attitude, or other phenomena of interest needs to be evaluated on a continuum scale

It is designed with the following scales 1. “inadequate” to “excellent”, 2. “never” to “always” or 3. “strongly disapprove” to “strongly

approve”

Guideline for Questionnaire

1. Keep it short• What do I do with the information?• Is it absolutely essential to have this information

to solve part of the research problem2. Use simple, clear, unambiguous language3. Check for unwarranted assumptions implicit in

your questions How many cigarette do you smoke each day?

Good to add a choice 25 ___ 25-16 ____ 15-5 ___ <5 ____ None ____

4. Word your questions in ways that don’t give clues about preferred or more desirable responses

What strategies have you used to try to quit smoking?

Leads him to list strategies he did not try

Guide line …

Check consistency – that leads to give contradictory answer for two questions

Determine in advance how you will code the response

Keep the respondents task simpleProvide clear instructionsMake the questionnaire attractive and professional

looking Conduct a pilot test

Give for half a dozen friends to see they have difficulty understanding any items

Scrutinize the almost final product carefully to make sure it address your needs

Review Questions

Explain the different quantitative research design methods

What is the strength and weakness of each method

What are the different survey research methods

What is the main data analysis methodswhat problem can you address with

quantitative research methods

Review questions

What are the procedures in quantitative Research

How do you use Quantitative research in computer science?