Research Process, Research Design and Questionnaires

Research Process, Research Design Research Process, Research Design

and Questionnairesand Questionnaires

RESEARCH PROCESSRESEARCH PROCESS

Identify and Define Research ProblemIdentify and Define Research Problem

Theory / PracticeTheory / Practice

Hypotheses / ConceptualizationHypotheses / Conceptualization

Research DesignResearch Design

Data collectionData collection

Data AnalysisData Analysis FindingsFindings

In this workshop we talk about all of the steps in the research process except Data Analysis and Findings.

What is a problem?What is a problem?

. . . any situation where a gap exists . . . any situation where a gap exists between the actual and the desired state.between the actual and the desired state.

A problem does not necessarily mean that A problem does not necessarily mean that something is seriously wrong. It could simply something is seriously wrong. It could simply indicate an interest in improving an existing indicate an interest in improving an existing situation. Thus, problem definitions can include situation. Thus, problem definitions can include both existing problems in the current situation both existing problems in the current situation as well as the quest for idealistic states in the as well as the quest for idealistic states in the future.future.

RESEARCH PROCESS – Research ProblemRESEARCH PROCESS – Research Problem

How are problems identified?How are problems identified?

1.1. ObservationObservation – manager/researcher senses that – manager/researcher senses that changes are occurring, or that some new changes are occurring, or that some new behaviors, attitudes, feelings, communication behaviors, attitudes, feelings, communication patterns, etc., are surfacing in one’s environment. patterns, etc., are surfacing in one’s environment. The manager may not understand exactly what is The manager may not understand exactly what is happening, but can definitely sense that things happening, but can definitely sense that things are not what they should be.are not what they should be.

2.2. Preliminary Data CollectionPreliminary Data Collection – use of interviews, – use of interviews, both unstructured and structured, to get an idea both unstructured and structured, to get an idea or “feel” for what is happening in the situation.or “feel” for what is happening in the situation.

3.3. Literature SurveyLiterature Survey – a comprehensive review of – a comprehensive review of the published and unpublished work from the published and unpublished work from secondary sources of data in the areas related to secondary sources of data in the areas related to the problem.the problem.

RESEARCH PROCESS – Problem IdentificationRESEARCH PROCESS – Problem Identification

A literature survey ensures that:A literature survey ensures that:

1.1. Important variables likely to influence the problem Important variables likely to influence the problem are not left out of the study.are not left out of the study.

2.2. A clearer idea emerges regarding what variables A clearer idea emerges regarding what variables are most important to consider, why they are are most important to consider, why they are important, and how they should be investigated.important, and how they should be investigated.

3.3. The problem is more accurately and precisely The problem is more accurately and precisely defined.defined.

4.4. The interviews cover all important topics.The interviews cover all important topics.5.5. The research hypotheses are testable.The research hypotheses are testable.6.6. The research can be replicated.The research can be replicated.7.7. One does not “reinvent the wheel”; that is, time One does not “reinvent the wheel”; that is, time

is not wasted trying to rediscover something that is not wasted trying to rediscover something that is already known.is already known.

8.8. The problem to be investigated is perceived by the The problem to be investigated is perceived by the scientific community as relevant and significant.scientific community as relevant and significant.


Typical Business Research ProblemsTypical Business Research Problems::

1.1. Training programs are not as effective as Training programs are not as effective as anticipated.anticipated.

2.2. Sales volume of products/services is not increasing.Sales volume of products/services is not increasing.3.3. Balancing of accounting ledgers is becoming Balancing of accounting ledgers is becoming

increasingly difficult.increasingly difficult.4.4. The newly installed information system is not being The newly installed information system is not being

used by the employees for whom it was designed.used by the employees for whom it was designed.5.5. Introduction of flexible work hours has created more Introduction of flexible work hours has created more

problems than it has solved.problems than it has solved.6.6. Anticipated results of a recent merger/acquisition Anticipated results of a recent merger/acquisition

have not been realized.have not been realized.7.7. Inventory control systems are not effective.Inventory control systems are not effective.8.8. Frequent interruptions in production.Frequent interruptions in production.9.9. Low employee morale.Low employee morale.10.10. Frequent customer complaints.Frequent customer complaints.11.11. Installation of an MIS keeps getting delayed.Installation of an MIS keeps getting delayed.12.12. Ad campaign is not generating new sales prospects.Ad campaign is not generating new sales prospects.

RESEARCH PROCESS – Problem RESEARCH PROCESS – Problem IdentificationIdentification

What are some business What are some business problems you are aware of or problems you are aware of or have confronted?have confronted?


Problem Definition StepsProblem Definition Steps::• Understand and define the complete problem. If Understand and define the complete problem. If more than one problem is identified, separate and more than one problem is identified, separate and prioritize them in terms of who and when they will prioritize them in terms of who and when they will be dealt with.be dealt with.

• Identify and separate out measurable symptoms to Identify and separate out measurable symptoms to determine root problem versus easily observable determine root problem versus easily observable symptoms. For example, a manager may identify symptoms. For example, a manager may identify declining sales or lost market share as the problem, declining sales or lost market share as the problem, but the real problem may be bad advertising, low but the real problem may be bad advertising, low salesperson morale, or ineffective distribution. salesperson morale, or ineffective distribution. Similarly, low productivity may be a symptom of Similarly, low productivity may be a symptom of employee morale or motivation problems, or employee morale or motivation problems, or supervisor issues.supervisor issues.

• Determine the unit of analysis = individuals, Determine the unit of analysis = individuals, households, businesses, objects (e.g., products, households, businesses, objects (e.g., products, stores), geographic areas, etc., or some stores), geographic areas, etc., or some combination.combination.

• Determine the relevant variables, including Determine the relevant variables, including specifying independent and dependent specifying independent and dependent relationships, constructs, etc.relationships, constructs, etc.

RESEARCH PROCESS – Problem RESEARCH PROCESS – Problem DefinitionDefinition

““A problem A problem

well defined is well defined is

a problem half a problem half

solved!”solved!”

Examples of Well-Defined problems:Examples of Well-Defined problems:

1.1. Has the new packaging affected the sales of the product?Has the new packaging affected the sales of the product?2.2. How do price and quality rate on consumers’ evaluation of How do price and quality rate on consumers’ evaluation of

products?products?3.3. Is the effect of participative budgeting on performance Is the effect of participative budgeting on performance

moderated by control systems?moderated by control systems?4.4. Does better automation lead to greater asset investment per Does better automation lead to greater asset investment per

dollar of output?dollar of output?5.5. Has the new advertising message resulted in higher recall?Has the new advertising message resulted in higher recall?6.6. To what extent do the organizational structure and type of To what extent do the organizational structure and type of

information systems account for the variance in the perceived information systems account for the variance in the perceived effectiveness of managerial decision-making?effectiveness of managerial decision-making?

7.7. Will expansion of international operations result in an Will expansion of international operations result in an improvement in the firm’s image and value?improvement in the firm’s image and value?

8.8. What are the effects of downsizing on the long-range growth What are the effects of downsizing on the long-range growth patterns of companies?patterns of companies?

9.9. What are the components of “quality of life”?What are the components of “quality of life”?10.10.What are the specific factors to be considered in creating a What are the specific factors to be considered in creating a

data warehouse for a manufacturing company?data warehouse for a manufacturing company?

RESEARCH PROCESS – Problem RESEARCH PROCESS – Problem DefinitionDefinition

RESEARCH PROCESS – DefinitionsRESEARCH PROCESS – Definitions

Variable = the observable and measurable characteristics/attributes the Variable = the observable and measurable characteristics/attributes the researcher specifies, studies, and draws conclusions about.researcher specifies, studies, and draws conclusions about.

Types of VariablesTypes of Variables::

• Independent variable =Independent variable = also called a predictor variable, it is a also called a predictor variable, it is a variable or construct that influences or explains the dependent variable or construct that influences or explains the dependent variable either in a positive or negative way.variable either in a positive or negative way.

• Dependent variable =Dependent variable = also known as a criterion variable, it is a also known as a criterion variable, it is a variable or construct the researcher hopes to understand, explain variable or construct the researcher hopes to understand, explain and/or predict.and/or predict.

• Moderator variable =Moderator variable = a variable that has an effect on the a variable that has an effect on the independent – dependent variable relationship. The presence of a independent – dependent variable relationship. The presence of a moderator variable modifies the original relationship between the moderator variable modifies the original relationship between the independent and dependent variables by interacting with the independent and dependent variables by interacting with the independent variable to influence the strength of the relationship with independent variable to influence the strength of the relationship with the dependent variable.the dependent variable.

• Mediating variable =Mediating variable = also known as an intervening variable, it is a also known as an intervening variable, it is a variable that surfaces as a function of the independent variable and variable that surfaces as a function of the independent variable and explains the relationship between the dependent and independent explains the relationship between the dependent and independent variables. Moderator variables specify when certain effects will occur variables. Moderator variables specify when certain effects will occur whereas mediators speak to how or why such effects occur. whereas mediators speak to how or why such effects occur. Moreover, mediators explain how external events take on internal Moreover, mediators explain how external events take on internal psychological significance.psychological significance.

RESEARCH PROCESS – Definitions continued . . .RESEARCH PROCESS – Definitions continued . . .

Measurement Measurement = = is the process of determining the direction and is the process of determining the direction and intensity of feelings about persons, events, concepts, ideas, and/or intensity of feelings about persons, events, concepts, ideas, and/or objects of interest that are defined as being part of the business objects of interest that are defined as being part of the business problem. As part of measurement, researchers use predetermined problem. As part of measurement, researchers use predetermined rules to assign numbers or labels to: (1) individuals’ attitudes, rules to assign numbers or labels to: (1) individuals’ attitudes, behaviors, characteristics, etc.; (2) objects’ features or attributes; behaviors, characteristics, etc.; (2) objects’ features or attributes; and (3) any other phenomenon or event being investigated. and (3) any other phenomenon or event being investigated. RulesRules tell researchers how to assign numbers or labels; e.g., assign the tell researchers how to assign numbers or labels; e.g., assign the numbers 1 to 7 to responses based on the intensity of an numbers 1 to 7 to responses based on the intensity of an individual’s feelings, beliefs, etc.individual’s feelings, beliefs, etc.

Measurement involves two processesMeasurement involves two processes:: (1) identification/development (1) identification/development of constructs; and (2) scale measurement. The first process of constructs; and (2) scale measurement. The first process involves identifying and defining what is to be measured, while the involves identifying and defining what is to be measured, while the second process involves selecting the scale to measure the second process involves selecting the scale to measure the construct(s).construct(s).

ConstructConstruct = = also referred to as a concept, it is a abstract idea also referred to as a concept, it is a abstract idea formed in the mind based on a set of facts or observations. The formed in the mind based on a set of facts or observations. The idea is a combination of a number of similar characteristics of the idea is a combination of a number of similar characteristics of the construct. Examples of constructs include: brand awareness, brand construct. Examples of constructs include: brand awareness, brand familiarity, purchase intentions, satisfaction, importance, trust, familiarity, purchase intentions, satisfaction, importance, trust, service quality, role ambiguity, etc.service quality, role ambiguity, etc.

Scale measurementScale measurement = = using a set of symbols or numbers to using a set of symbols or numbers to represent the range of possible responses to a research question.represent the range of possible responses to a research question.

Examples of Constructs Investigated in MarketingExamples of Constructs Investigated in Marketing::

ConstructsConstructs Operational Operational DescriptionDescription

Brand AwarenessBrand Awareness Percentage of respondents that have heard of a Percentage of respondents that have heard of a designated brand; awareness could be either designated brand; awareness could be either unaided or aided.unaided or aided.

Brand AttitudesBrand Attitudes The number of respondents and their intensity of The number of respondents and their intensity of feeling positive or negative toward a specific brand.feeling positive or negative toward a specific brand.

Purchase IntentionsPurchase Intentions The number of people planning to buy theThe number of people planning to buy thespecified object (e.g., product or service) withinspecified object (e.g., product or service) withina designated time period.a designated time period.

Importance of FactorsImportance of Factors To what extent do specific factors influenceTo what extent do specific factors influencea person's purchase choice.a person's purchase choice.

PsychographicsPsychographics The attitudes, opinions, interests and The attitudes, opinions, interests and lifestyle characteristics of individuals lifestyle characteristics of individuals providing the information.providing the information.

SatisfactionSatisfaction How people evaluate their post-purchase How people evaluate their post-purchase consumption experience with a particular consumption experience with a particular product, service or company.product, service or company.

RESEARCH PROCESS – ConstructsRESEARCH PROCESS – Constructs

““Role Ambiguity” ConstructRole Ambiguity” Construct

Conceptual/theoretical definitionConceptual/theoretical definition = the difference between the information = the difference between the information available to the person (actual knowledge) and that which is required for available to the person (actual knowledge) and that which is required for adequate performance of a role.adequate performance of a role.

Operational definitionOperational definition = the amount of uncertainty an individual feels = the amount of uncertainty an individual feels regarding job role responsibilities and expectations from supervisors, regarding job role responsibilities and expectations from supervisors, other employees and customers.other employees and customers.

Measurement scaleMeasurement scale = consists of 45 items assessed using a 5-point scale, = consists of 45 items assessed using a 5-point scale, with category labels 1 = very certain, 2 = certain, 3 = neutral, 4 = with category labels 1 = very certain, 2 = certain, 3 = neutral, 4 = uncertain, and 5 = very uncertain.uncertain, and 5 = very uncertain.

Examples of itemsExamples of items::

• How much freedom of action I am expected to have.How much freedom of action I am expected to have.

• How I am expected to handle non-routine activities on the job.How I am expected to handle non-routine activities on the job.

• The sheer amount of work I am expected to do.The sheer amount of work I am expected to do.

• To what extent my boss is open to hearing my point of view.To what extent my boss is open to hearing my point of view.

• How satisfied my boss is with me.How satisfied my boss is with me.

• How I am expected to interact with my customers.How I am expected to interact with my customers.

Source: Singh & Rhoads, JMR, August 1991, p. 328.

““Service Quality” ConstructService Quality” Construct

Conceptual/theoretical definitionConceptual/theoretical definition = the difference between an individual’s = the difference between an individual’s expectations of service and their actual experiences.expectations of service and their actual experiences.

Operational definitionOperational definition = how individuals react to their actual service = how individuals react to their actual service experience with a company relative to their expectations that a experience with a company relative to their expectations that a company will possess certain service characteristics.company will possess certain service characteristics.

Measurement scaleMeasurement scale = consists of 82 items assessed using a 7-point scale, = consists of 82 items assessed using a 7-point scale, with category labels 1 = not at all essential to 7 = absolutely essential.with category labels 1 = not at all essential to 7 = absolutely essential.

Examples of itemsExamples of items::

• Employees of excellent companies will give prompt service to customers.Employees of excellent companies will give prompt service to customers.

• Excellent companies will have the customers’ best interests at heart.Excellent companies will have the customers’ best interests at heart.

• Excellent companies will perform services right the first time.Excellent companies will perform services right the first time.

• Employees of excellent companies will never be too busy to respond toEmployees of excellent companies will never be too busy to respond to customer requests.customer requests.

• Excellent companies will give customers individual attention.Excellent companies will give customers individual attention.

• Materials associated with products and services of excellent companiesMaterials associated with products and services of excellent companies (such as pamphlets or statements) will be visually appealing .(such as pamphlets or statements) will be visually appealing .

Source: Parasuraman, Zeithaml & Berry, JM, Fall 1985, p. 44.








What is What is theory ??theory ??

RESEARCH PROCESS – Theory/PracticeRESEARCH PROCESS – Theory/Practice

Theory = Theory = a systematic set of relationships a systematic set of relationships providing a consistent and comprehensive providing a consistent and comprehensive explanation of a phenomenon. In practice, a explanation of a phenomenon. In practice, a theory is a researcher’s attempt to specify the theory is a researcher’s attempt to specify the entire set of dependence relationships entire set of dependence relationships explaining a particular set of outcomes. explaining a particular set of outcomes.

Theory is based on prior empirical research, Theory is based on prior empirical research, past experiences and observations of behavior, past experiences and observations of behavior, attitudes, or other phenomena, and other attitudes, or other phenomena, and other theories that provide a perspective for theories that provide a perspective for developing possible relationships.developing possible relationships.

Theory is used to prepare a Theory is used to prepare a theoretical theoretical frameworkframework for the research. for the research.









Hypotheses Hypotheses = preconceptions the = preconceptions the researcher develops regarding the researcher develops regarding the relationships represented in the data, relationships represented in the data, typically based on theory, practice or typically based on theory, practice or previous research.previous research.

ExamplesExamples::

““The average number of cups of coffee students The average number of cups of coffee students drink during finals will be greater than the average drink during finals will be greater than the average they consume at other times.” they consume at other times.”

““Younger, part-time employees of Samouel’s Younger, part-time employees of Samouel’s restaurant are more likely to search for a new job.” restaurant are more likely to search for a new job.”

RESEARCH PROCESS – HypothesesRESEARCH PROCESS – Hypotheses

Theoretical Framework = Theoretical Framework = a written a written description that includes a conceptual model. description that includes a conceptual model. It integrates all the information about the It integrates all the information about the problem in a logical manner, describes the problem in a logical manner, describes the relationships among the variables, explains relationships among the variables, explains the theory underlying these relationships, and the theory underlying these relationships, and indicates the nature and direction of the indicates the nature and direction of the relationships.relationships.

The process of developing a theoretical The process of developing a theoretical framework involves conceptualization – which framework involves conceptualization – which is a visual specification (conceptual model) of is a visual specification (conceptual model) of the theoretical basis of the relationships you the theoretical basis of the relationships you would like to examine.would like to examine.

RESEARCH PROCESS – Theoretical RESEARCH PROCESS – Theoretical FrameworkFramework

Basic Features of a Good Theoretical Framework:Basic Features of a Good Theoretical Framework:

1.1. The variables/constructs considered relevant to the The variables/constructs considered relevant to the study are clearly identified and labeled.study are clearly identified and labeled.

2.2. The discussion states how the variables/constructs are The discussion states how the variables/constructs are related to each other, e.g., dependent, independent, related to each other, e.g., dependent, independent, moderator, etc.moderator, etc.

3.3. If possible, the nature (positive or negative) of the If possible, the nature (positive or negative) of the relationships as well as the direction is hypothesized relationships as well as the direction is hypothesized on the basis of theory, previous research or on the basis of theory, previous research or researcher judgment.researcher judgment.

4.4. There is a clear explanation of why you expect these There is a clear explanation of why you expect these relationships to exist.relationships to exist.

5.5. A visual (schematic) diagram of the theoretical A visual (schematic) diagram of the theoretical framework is prepared to clearly illustrate the framework is prepared to clearly illustrate the hypothesized relationships.hypothesized relationships.


RESEARCH PROCESS – Conceptual ModelsRESEARCH PROCESS – Conceptual Models

Price Purchase Likelihood

Price Purchase Likelihood

Independent Dependent

Variable Variable


Variable Variable

• Discount Level• Restrictions

Moderator Variable

RESEARCH PROCESS – Conceptual ModelsRESEARCH PROCESS – Conceptual Models

PricePurchase Likelihood


Variable Variable

Perceived Value

Mediator Variable (full mediation)

Price

Perceived Value

Purchase Likelihood

Mediator Variable (partial mediation)

Group ExerciseGroup Exercise: Use the Samouel’s and : Use the Samouel’s and Gino’s restaurant database variables to Gino’s restaurant database variables to develop a theoretical develop a theoretical framework/conceptual model of the framework/conceptual model of the relationships that could be examined. relationships that could be examined. Consider and evaluate several models, but Consider and evaluate several models, but be prepared to report your most be prepared to report your most interesting or thought provoking model.interesting or thought provoking model.

Theoretical Framework – ConceptualizationTheoretical Framework – Conceptualization

Conceptual Models – Samouel’s Employee Conceptual Models – Samouel’s Employee DatabaseDatabase

Potential Hypotheses: Commitment is positively related to supervision, work groups and compensation. Intention to Search is negatively related to supervision, work groups & compensation.

Employee CommitmentWork Groups

Supervision

Compensation

Intention to Search

Compensation

Work Groups

Supervision

Variable Description Variable Type

Restaurant PerceptionsX1 Excellent Food Quality MetricX2 Attractive Interior MetricX3 Generous Portions MetricX4 Excellent Food Taste MetricX5 Good Value for the Money MetricX6 Friendly Employees MetricX7 Appears Clean & Neat MetricX8 Fun Place to Go MetricX9 Wide Variety of menu Items MetricX10 Reasonable Prices MetricX11 Courteous Employees MetricX12 Competent Employees MetricSelection Factor RankingsX13 Food Quality NonmetricX14 Atmosphere NonmetricX15 Prices NonmetricX16 Employees NonmetricRelationship VariablesX17 Satisfaction MetricX18 Likely to Return in Future MetricX19 Recommend to Friend MetricX20 Frequency of Patronage NonmetricX21 Length of Time a Customer NonmetricClassification VariablesX22 Gender NonmetricX23 Age NonmetricX24 Income NonmetricX25 Competitor NonmetricX26 Which AD Viewed (#1, 2 or 3) NonmetricX27 AD Rating MetricX28 Respondents that Viewed Ads Nonmetric

Description of Customer Survey VariablesDescription of Customer Survey VariablesVS.VS.

Variable Description Variable Type

Work Environment MeasuresX1 I am paid fairly for the work I do. MetricX2 I am doing the kind of work I want. MetricX3 My supervisor gives credit an praise for work well done. MetricX4 There is a lot of cooperation among the members of my work group. MetricX5 My job allows me to learn new skills. MetricX6 My supervisor recognizes my potential. MetricX7 My work gives me a sense of accomplishment. MetricX8 My immediate work group functions as a team. MetricX9 My pay reflects the effort I put into doing my work. MetricX10 My supervisor is friendly and helpful. MetricX11 The members of my work group have the skills and/or training

to do their job well. MetricX12 The benefits I receive are reasonable. MetricRelationship MeasuresX13 Loyalty – I have a sense of loyalty to Samouel’s restaurant. MetricX14 Effort – I am willing to put in a great deal of effort beyond that

expected to help Samouel’s restaurant to be successful. MetricX15 Proud – I am proud to tell others that I work for Samouel’s restaurant. MetricClassification VariablesX16 Intention to Search MetricX17 Length of Time an Employee NonmetricX18 Work Type = Part-Time vs. Full-Time NonmetricX19 Gender NonmetricX20 Age NonmetricX21 Performance Metric

Description of Employee Survey VariablesDescription of Employee Survey Variables








RESEARCH DESIGN – TypesRESEARCH DESIGN – Types

Research Design Alternatives – PurposeResearch Design Alternatives – Purpose::

(1)(1) Exploratory –Exploratory – to formulate the problem, to formulate the problem, develop develop hypotheses, identify constructs, hypotheses, identify constructs, establish priorities establish priorities for research, refine ideas, for research, refine ideas, clarify concepts, etc.clarify concepts, etc.

(2)(2) Descriptive –Descriptive – to describe characteristics of to describe characteristics of certain certain groups, estimate proportion of people in a groups, estimate proportion of people in a population population who behave in a given way, and who behave in a given way, and to to make directional make directional predictions.predictions.

(3)(3) Causal –Causal – to provide evidence of the to provide evidence of the relationships relationships between variables, the sequence in between variables, the sequence in which events which events occur, and/or to eliminate other occur, and/or to eliminate other possible explanations.possible explanations.

Two Broad ApproachesTwo Broad Approaches::

1.1. Qualitative.Qualitative.

2.2. Quantitative.Quantitative.

Research Design – Approaches Research Design – Approaches

Role of Qualitative ResearchRole of Qualitative Research::

Search of academic, trade and professional Search of academic, trade and professional literature (both traditional & Internet).literature (both traditional & Internet). Use of interviews, brainstorming, focus groups.Use of interviews, brainstorming, focus groups. Internalization of how others have undertaken Internalization of how others have undertaken both qualitative and quantitative research.both qualitative and quantitative research. Use of existing questionnaires/constructs.Use of existing questionnaires/constructs.

Outcome of Qualitative ResearchOutcome of Qualitative Research:: Improve conceptualization.Improve conceptualization. Clarify research design, including data Clarify research design, including data collectioncollection approach.approach. Draft questionnaire.Draft questionnaire.

RESEARCH RESEARCH DESIGNDESIGN

Role of Quantitative ResearchRole of Quantitative Research::

Quantify data and generalize results fromQuantify data and generalize results from sample to population.sample to population. Facilitates examination of large number ofFacilitates examination of large number of representative cases.representative cases. Structured approach to data collection.Structured approach to data collection. Enables extensive statistical analysis. Enables extensive statistical analysis.

Outcome of Quantitative Outcome of Quantitative ResearchResearch::

Validation of qualitative research findings.Validation of qualitative research findings. Confirmation of hypotheses, theories, etc.Confirmation of hypotheses, theories, etc. Recommend final course of action.Recommend final course of action.

RESEARCH RESEARCH DESIGNDESIGN








DATA COLLECTIONDATA COLLECTION

ApproachesApproaches::

ObservationObservation• Human Human • Mechanical/Electronic DevicesMechanical/Electronic Devices

SurveysSurveys• Self-Completion Self-Completion

• Mail/Overnight Delivery/FaxMail/Overnight Delivery/Fax• ElectronicElectronic

• Interviewer-AdministeredInterviewer-Administered• Face-to-Face – Home, Work, Mall, Focus Face-to-Face – Home, Work, Mall, Focus

GroupsGroups• TelephoneTelephone

DATA COLLECTIONDATA COLLECTION

Selection of data collection approachSelection of data collection approach??

• BudgetBudget• Knowledge of issues – qualitative vs. Knowledge of issues – qualitative vs. quantitativequantitative• Respondent ParticipationRespondent Participation

Taste Test; Ad TestTaste Test; Ad Test Card Sorts; Visual ScalingCard Sorts; Visual Scaling

• Time AvailableTime Available

DATA DATA COLLECTIONCOLLECTION

Types of Types of DataData::

• PrimaryPrimary

• SecondarySecondary

PRIMARY DATAPRIMARY DATA

Primary Data SourcesPrimary Data Sources::

• Informal discussions; Informal discussions; brainstormingbrainstorming• Focus groupsFocus groups• Observational MethodsObservational Methods• Structured & Unstructured Structured & Unstructured SurveysSurveys• ExperimentsExperiments

Primary Data – Focus GroupsPrimary Data – Focus Groups

Focus Groups = bring a small group of people (10-Focus Groups = bring a small group of people (10-12) together for an interactive, spontaneous 12) together for an interactive, spontaneous discussion of a particular topic or concept. discussion of a particular topic or concept. Discussion is led by a trained moderator and Discussion is led by a trained moderator and usually lasts 1 ½ hours.usually lasts 1 ½ hours.

Typical ObjectivesTypical Objectives::

• To identify and define problems.To identify and define problems.• To generate new ideas about products, services, delivery To generate new ideas about products, services, delivery

methods, etc.methods, etc.• To test advertising themes, positioning statements, To test advertising themes, positioning statements,

company and product names, etc.company and product names, etc.• To discover new constructs and measurement methods.To discover new constructs and measurement methods.• To understand customer needs, wants, attitudes, behaviors, To understand customer needs, wants, attitudes, behaviors,

preferences and motives.preferences and motives.

Primary DataPrimary Data

Factors Influencing Overall Mobile Phone Factors Influencing Overall Mobile Phone SatisfactionSatisfaction

20032003 20022002• FeaturesFeatures 27% 27% 21% 21%• DurabilityDurability 23% 23% 16% 16%• Physical DesignPhysical Design 19% 19% 28% 28%• Battery FunctionBattery Function 16% 16% 16% 16%• OperationOperation 15% 15% 19% 19%

2004 Wireless Retail Sales Satisfaction Study2004 Wireless Retail Sales Satisfaction Study• Sales StaffSales Staff 44%44%• Price/PromotionPrice/Promotion 28%28%• Store DisplayStore Display 14%14%• Store FacilityStore Facility 14%14%

Source: J.D. Power and Associates, 2002, 2003 & 2004.Source: J.D. Power and Associates, 2002, 2003 & 2004.

These These factors factors typically are typically are identified in identified in qualitative qualitative focus groups focus groups (exploratory (exploratory research).research).

These percentages These percentages typically are typically are determined in determined in quantitative surveys quantitative surveys (descriptive (descriptive research).research).

Hotel Selection FactorsHotel Selection Factors:: 1. Location1. Location 2. Past Experience 2. Past Experience 3. Recommendations or Friends and Family 3. Recommendations or Friends and Family 4. Brand Reputation 4. Brand Reputation

Guest Satisfaction FactorsGuest Satisfaction Factors:: 1. Guest Room1. Guest Room 2. Departure Process 2. Departure Process 3. Pre-Arrival/Arrival Experiences 3. Pre-Arrival/Arrival Experiences 4. Hotel Services 4. Hotel Services 5. Food & Beverage services 5. Food & Beverage services

Note: the first three factors account for more than 70 Note: the first three factors account for more than 70 percentpercent of guest satisfaction ratings. of guest satisfaction ratings.

Source: J.D. Power & Associates, August 21, 2001.Source: J.D. Power & Associates, August 21, 2001.

Primary Primary DataData

Original Equipment Tire Satisfaction StudyOriginal Equipment Tire Satisfaction Study::

1. Product Quality1. Product Quality 39%39%- Number of tires with a problem- Number of tires with a problem- Number of problems experienced- Number of problems experienced- Number of original tires replaced- Number of original tires replaced

2. Long-Term Performance2. Long-Term Performance 22%22%- Wear ability- Wear ability- Length of warranty- Length of warranty- Overall reliability & dependability- Overall reliability & dependability- Freedom from pull to left or right- Freedom from pull to left or right

3. Situational Performance3. Situational Performance 19%19%- Traction on wet roads- Traction on wet roads- Traction at fast starts- Traction at fast starts- Holds road well in emergencies- Holds road well in emergencies- Lack of vibration at highway speeds- Lack of vibration at highway speeds- Overall safety- Overall safety- Overall ride at highway speeds- Overall ride at highway speeds

4. Design4. Design 14%14%- Road quietness- Road quietness- Style & appearance of sidewalls- Style & appearance of sidewalls- Tread design- Tread design- Size of tire matches size of vehicle - Size of tire matches size of vehicle

5. Winter Traction5. Winter Traction 5%5%

Source: J.D. Power & Associates, August 27, 2001.Source: J.D. Power & Associates, August 27, 2001.

Primary Primary DataData

What is the What is the construct in this construct in this study?study?

PRIMARY DATA – Focus PRIMARY DATA – Focus GroupsGroups

Focus GroupsFocus Groups::

Some of my Some of my “best”“best” experiences? experiences?

Some of my Some of my “worst” “worst” experiences?experiences?

PRIMARY DATA – PRIMARY DATA –

ObservationsObservations

CONSIDERATIONSCONSIDERATIONS::

• Methods – human/mechanical/electronic.Methods – human/mechanical/electronic.

• Useful where respondent cannot or will Useful where respondent cannot or will not not articulate the answer.articulate the answer.

• Cannot be used to measure thoughts, Cannot be used to measure thoughts, feelings, feelings, attitudes, opinions, etc.attitudes, opinions, etc.

Purpose of Purpose of QuestionnairesQuestionnaires::

• To obtain information that cannot be To obtain information that cannot be easily easily observed or is not already observed or is not already available inavailable in

written or electronic form.written or electronic form.

• Questionnaires enable researchers to Questionnaires enable researchers to measure measure concepts/constructs.concepts/constructs.

PRIMARY DATA – PRIMARY DATA – QUESTIONNAIRESQUESTIONNAIRES

Steps in Questionnaire DesignSteps in Questionnaire Design::

1.1. Initial Considerations – problem, objectives, Initial Considerations – problem, objectives, target population, sampling, etc. target population, sampling, etc.

2.2. Clarification of Concepts – select variables, Clarification of Concepts – select variables, constructs, measurement approach, etc.constructs, measurement approach, etc.

3.3. Developing the QuestionnaireDeveloping the Questionnaire Length and sequence.Length and sequence. Types of questions.Types of questions. Sources of questions.Sources of questions. Wording, coding, layout and instructions.Wording, coding, layout and instructions.

4.4. Pre-testing the Questionnaire.Pre-testing the Questionnaire.

5.5. Questionnaire Administration Planning.Questionnaire Administration Planning.

QUESTIONNAIRE DESIGNQUESTIONNAIRE DESIGN

Open-ended Questions = place no constraints on respondents; i.e., they are free to answer in their own words and to give whatever thoughts come to mind.

Closed-ended Questions = respondent is given the option of choosing from a number of predetermined answers.

Two Types of Two Types of QuestionsQuestions::1.1. Open-endedOpen-ended2.2. Closed-endedClosed-ended


Examples of Open-ended QuestionsExamples of Open-ended Questions::

How do you typically decide which restaurant you will eat at?

Which mutual funds have you been investing in for the past year?

How are your investment funds performing?

Do you think airport security is better now than it

was six months ago?


Open-ended QuestionsOpen-ended Questions

• Typically used in exploratory/qualitative studies.Typically used in exploratory/qualitative studies.• Typically used in personal interview surveys Typically used in personal interview surveys

involving small samples.involving small samples.• Allows respondent freedom of response.Allows respondent freedom of response.• Respondent must be articulate Respondent must be articulate andand willing to spend willing to spend

time giving a full answer.time giving a full answer.• Data is in narrative form which can be time Data is in narrative form which can be time

consuming and difficult to code and analyze.consuming and difficult to code and analyze.• Possible researcher bias in interpretation.Possible researcher bias in interpretation.• Narrative is analyzed using of content analysis. Narrative is analyzed using of content analysis.

Software is available (e.g., NUD*IST).Software is available (e.g., NUD*IST).

QUESTIONNAIRE QUESTIONNAIRE DESIGNDESIGN

Content Analysis Content Analysis Software:Software:

TextSmart is a software package that enables users to TextSmart is a software package that enables users to view, manipulate and automate the coding or categorization of view, manipulate and automate the coding or categorization of responses to narative data. The ability to automate the responses to narative data. The ability to automate the examination and organization of narrative data is particularly examination and organization of narrative data is particularly helpful when a ‘large scale’ survey is undertaken. It can be helpful when a ‘large scale’ survey is undertaken. It can be used to analyze any textual data, and its output can be used to analyze any textual data, and its output can be exported to SPSS for further analysis. For example, you can do exported to SPSS for further analysis. For example, you can do correspondence analysiscorrespondence analysis** on a contingency table from a on a contingency table from a TextSmart analysis. For more information about TextSmart and TextSmart analysis. For more information about TextSmart and related SPSS products visit the WWW site related SPSS products visit the WWW site www.spss.com. .

QSR NUD*IST stands for Non-Numerical Unstructured QSR NUD*IST stands for Non-Numerical Unstructured

Data Indexing and Theorizing. It is a popular computer Data Indexing and Theorizing. It is a popular computer software package used by researchers to analyze text from software package used by researchers to analyze text from focus group or interview transcripts, literary documents and so focus group or interview transcripts, literary documents and so on. It examines non-textual data such as photographs, tape on. It examines non-textual data such as photographs, tape recordings, films and so on. Users can us it to index and link recordings, films and so on. Users can us it to index and link several documents in a structured way to produce categorical several documents in a structured way to produce categorical data in a form amenable to further analysis. NUD*IST output data in a form amenable to further analysis. NUD*IST output can be exported to software programs such as SPSS and Excel. can be exported to software programs such as SPSS and Excel. For more information about QSR NUD*IST and its related For more information about QSR NUD*IST and its related product NVIVO visit their websiteproduct NVIVO visit their website

((http://www.scolari.co.uk/qsr/qsr_n4.htm).).

Closed-end QuestionsClosed-end Questions::

Single AnswerSingle Answer Multiple AnswerMultiple Answer Rank OrderRank Order NumericNumeric Likert-Type ScalesLikert-Type Scales Semantic DifferentialSemantic Differential


1.1. Did you check your email this morning? __ Yes __ NoDid you check your email this morning? __ Yes __ No

2.2. Do you believe Enron senior executives should be put in jail? __ Yes __ Do you believe Enron senior executives should be put in jail? __ Yes __ NoNo

3.3. Should the U.K. adopt the Euro or keep the Pound?Should the U.K. adopt the Euro or keep the Pound?__ Adopt the Euro __ Adopt the Euro __ Keep the Pound __ Keep the Pound

4.4. Which countries in Europe have you traveled to in the last six months?Which countries in Europe have you traveled to in the last six months?__ Belgium __ Belgium __ Germany__ Germany__ France __ France __ Holland __ Holland __ Italy __ Italy __ Switzerland __ Switzerland __ Spain __ Spain __ __ Other (please specify) _____________Other (please specify) _____________

5.5. How often do you eat at Samouel’s Greek Cuisine restaurant?How often do you eat at Samouel’s Greek Cuisine restaurant?__ Never __ Never __ 1 – 4 times per year __ 1 – 4 times per year __ 5 – 8 times per year __ 5 – 8 times per year __ 9 – 12 times per year __ 9 – 12 times per year __ More than 12 times per year __ More than 12 times per year

Examples of Closed-end Examples of Closed-end Questions:Questions:

Closed-end QuestionsClosed-end Questions

• Typically used in quantitative studies.Typically used in quantitative studies.• Assumption is researcher has knowledge to pre-Assumption is researcher has knowledge to pre-

specify response categories.specify response categories.• Data can be pre-coded and therefore in a form Data can be pre-coded and therefore in a form

amenable for use with statistical packages (e.g., amenable for use with statistical packages (e.g., SPSS, SAS) – data capture therefore easier.SPSS, SAS) – data capture therefore easier.

• More difficult to design More difficult to design butbut simplifies analysis. simplifies analysis.• Used in studies involving large samples.Used in studies involving large samples.• Limited range of response options.Limited range of response options.


Broad ConsiderationsBroad Considerations

• Sequencing of questions.Sequencing of questions.• Identification of concepts.Identification of concepts.• How many questions are required to How many questions are required to

capture each concept.capture each concept.• Question wording.Question wording.• Overall length of questionnaire.Overall length of questionnaire.• Placing of sensitive questions.Placing of sensitive questions.• Ability of respondents.Ability of respondents.• Level of measurement.Level of measurement.• Open-ended versus closed-end questions.Open-ended versus closed-end questions.


Questionnaire SequenceQuestionnaire Sequence

• Opening QuestionsOpening Questions

• Research Topic Research Topic QuestionsQuestions

• Classification QuestionsClassification Questions


Screening or Filter Screening or Filter QuestionsQuestions::

. . . are used to ensure respondents included in . . . are used to ensure respondents included in the study are those that meet the pre-the study are those that meet the pre-determined criteria of the target population.determined criteria of the target population.

““Tonight we are talking with individuals who are Tonight we are talking with individuals who are 18 years of age or older and have 50 percent or 18 years of age or older and have 50 percent or more of the responsibility for banking decisions more of the responsibility for banking decisions in your household. Are you that person?” __ in your household. Are you that person?” __ Yes __ No Yes __ No

QUESTIONNAIRE DESIGN – Opening QuestionsQUESTIONNAIRE DESIGN – Opening Questions

Rapport QuestionsRapport Questions::

. . . are used to establish rapport with the . . . are used to establish rapport with the respondent by gaining their attention and respondent by gaining their attention and stimulating their interest in the topic.stimulating their interest in the topic.

• ““Have you seen any good movies in the last Have you seen any good movies in the last month?” month?”

__ Yes __ No __ Yes __ No

• “ “What is your favorite seafood restaurant?”What is your favorite seafood restaurant?”

QUESTIONNAIRE DESIGN – Opening QuestionsQUESTIONNAIRE DESIGN – Opening Questions

Concept/construct Concept/construct = an abstract idea formed in the mind. = an abstract idea formed in the mind. The idea is a combination of a number of similar The idea is a combination of a number of similar characteristics/variables that collectively define the concept characteristics/variables that collectively define the concept and are used to measure it. Constructs are and are used to measure it. Constructs are abstract/intangible and cannot be directly observed or abstract/intangible and cannot be directly observed or measured because they are the mental images a person measured because they are the mental images a person attaches to an object, such as attitudes, feelings, perceptions, attaches to an object, such as attitudes, feelings, perceptions, expectations, or expressions of future actions (e.g., purchase expectations, or expressions of future actions (e.g., purchase intentions).intentions).

Example ConceptExample Concept:: “Customer Service” issues “Customer Service” issues for for a B-to-B a B-to-B situationsituation

Reliable deliveryReliable delivery Technical sales SupportTechnical sales Support Inside sales representativesInside sales representatives Field sales representativesField sales representatives Complaint resolutionComplaint resolution Ordering/InvoicingOrdering/Invoicing Website designWebsite design

QUESTIONNAIRE DESIGN – QUESTIONNAIRE DESIGN – Research Topic QuestionsResearch Topic Questions

ConceptsConceptsConcept IdentificationConcept Identification Conceptual definition – e.g., Service Conceptual definition – e.g., Service

Quality.Quality.

As perceived by customers, it is the As perceived by customers, it is the difference between customers’ difference between customers’ expectations or desires of a vendor and expectations or desires of a vendor and their perceptions of the actual situation their perceptions of the actual situation (their experiences).(their experiences).

Working Definition for ConceptWorking Definition for Concept Decompose definition into components.Decompose definition into components. Search for items that are measurableSearch for items that are measurable.


Service Quality Construct:Service Quality Construct:

Research has shown the service quality construct can be indirectly Research has shown the service quality construct can be indirectly represented by the following measurable components:represented by the following measurable components:

“ “ The service provider’s ability to . . . . “The service provider’s ability to . . . . “

•communicate and listen to consumers;communicate and listen to consumers;

•sincerely empathize with customers in interpreting their sincerely empathize with customers in interpreting their needs needs and wants;and wants;

•be tactful in responding to customers’ questions, objections, be tactful in responding to customers’ questions, objections, and and problems;problems;

•create an impression of reliability in performing services;create an impression of reliability in performing services;

•create an image of credibility by keeping promises; create an image of credibility by keeping promises;

•demonstrate sufficient technical knowledge and competence;demonstrate sufficient technical knowledge and competence;

•exhibit strong interpersonal skills in dealing with customers.exhibit strong interpersonal skills in dealing with customers.


Concept Development Concept Development ExerciseExercise::

Concept Concept = “Restaurant Service Quality” = “Restaurant Service Quality”

1.1. What are the components of service What are the components of service quality as they relate to a restaurant?quality as they relate to a restaurant?

2.2. How do you measure these components?How do you measure these components?


Preparing Good Questions:Preparing Good Questions:

• Use Simple Words.Use Simple Words.

• Be brief.Be brief.

• Avoid Ambiguity.Avoid Ambiguity.

• Avoid Leading Questions.Avoid Leading Questions.

• Avoid Double-Barreled Avoid Double-Barreled Questions.Questions.

• Check Questionnaire Layout.Check Questionnaire Layout.

• Prepare Clear Instructions.Prepare Clear Instructions.

• Watch Question Sequence.Watch Question Sequence.



Recently a survey was conducted by the United Nations Recently a survey was conducted by the United Nations using a sample from several different countries. The using a sample from several different countries. The question asked was:question asked was: " Would you please give your opinion about the food " Would you please give your opinion about the food shortage in the rest of the world?"shortage in the rest of the world?"

The survey was a huge failureThe survey was a huge failure. . WhyWhy??

• In In AfricaAfrica they did not know what 'food' meant. they did not know what 'food' meant.• In In Western EuropeWestern Europe, they did not know what 'shortage' , they did not know what 'shortage' meant.meant.• In In Eastern EuropeEastern Europe they did not know what 'opinion' they did not know what 'opinion' meant.meant.• In In South AmericaSouth America they did not know what 'please' they did not know what 'please' meant.meant.• And in the And in the U.S.U.S., they did not know what 'the rest of the , they did not know what 'the rest of the world' meant.world' meant.

Avoid Position BiasAvoid Position Bias::

Position BiasPosition Bias: :

• ““How important are flexible hours in evaluating How important are flexible hours in evaluating

job alternatives?”job alternatives?”

• “ “What factors are important in evaluating What factors are important in evaluating


No Position BiasNo Position Bias::

• “ “What factors are important in evaluating What factors are important in evaluating


• “ “How important are flexible hours in evaluating How important are flexible hours in evaluating




To what extent do you agree or disagree with the To what extent do you agree or disagree with the following statements? following statements?

• “ “Harrod’s employees are friendly and helpful.”Harrod’s employees are friendly and helpful.”

• “ “Harrod’s employees are courteous and Harrod’s employees are courteous and knowledgeable.”knowledgeable.”

Double-Barreled Double-Barreled QuestionsQuestions::


. . . are used to direct respondents to answer the . . . are used to direct respondents to answer the right right questions as well as questions in the proper sequence.questions as well as questions in the proper sequence.

• “ “Have you seen or heard any advertisements for Have you seen or heard any advertisements for wirelesswireless telephone service in the past 30 days?”telephone service in the past 30 days?”• “ “If ‘No’, go to question #10.If ‘No’, go to question #10.• “ “If ‘Yes’ , were the advertisements on radio or TV or If ‘Yes’ , were the advertisements on radio or TV or both?”both?”• “ “If the advertisements were on TV or on both radio If the advertisements were on TV or on both radio andand TV, then go to question #6?TV, then go to question #6?• “ “If the advertisements were on radio, then go toIf the advertisements were on radio, then go to question #8.”question #8.”

Following questions #6 and #8 the next question Following questions #6 and #8 the next question would be:would be:

• “ “Were any of the advertisements for ‘Sprint PCS’?”Were any of the advertisements for ‘Sprint PCS’?”

Branching QuestionsBranching Questions::


• Introducing and explaining how to answer a series of Introducing and explaining how to answer a series of questions on a particular topic.questions on a particular topic.• Transition statements from one section (topic) of theTransition statements from one section (topic) of the questionnaire to another.questionnaire to another.• Which question to go to next (branching or skipping).Which question to go to next (branching or skipping).• How many answers are acceptable, e.g., “Check onlyHow many answers are acceptable, e.g., “Check only one response” one response” oror “Check as many as apply.” “Check as many as apply.” • Whether respondents are supposed to answer the Whether respondents are supposed to answer the question by themselves, or can consult anotherquestion by themselves, or can consult another person or reference materials.person or reference materials. • What to do when the questionnaire is completed, e.g.,What to do when the questionnaire is completed, e.g., “ “When finished, place this in the postage paid When finished, place this in the postage paid envelope and mail it.”envelope and mail it.”

Issues – Self-Completion Issues – Self-Completion InstructionsInstructions::


• How to increase respondent participation.How to increase respondent participation.• How to screen out respondents that are not wanted andHow to screen out respondents that are not wanted and still keep them happy.still keep them happy.• What to say when respondents ask how to answer a What to say when respondents ask how to answer a particular question.particular question.• When concepts may not be easily understood, how toWhen concepts may not be easily understood, how to define them.define them.• When answer alternatives are to be read to respondentsWhen answer alternatives are to be read to respondents (aided response) or not to be read (unaided response).(aided response) or not to be read (unaided response).• How to follow branching or skip patterns.How to follow branching or skip patterns.• When and how to probe.When and how to probe.• How to end the interview.How to end the interview.

Issues – Interviewer-Assisted Issues – Interviewer-Assisted InstructionsInstructions::

Identify response bias for below questions:Identify response bias for below questions:

1.1. ““Do you advocate a lower speed limit to save human lives?”Do you advocate a lower speed limit to save human lives?”

2.2. ““When you visited the museum, how many times did you read the When you visited the museum, how many times did you read the plaques that explain what the exhibit contained?”plaques that explain what the exhibit contained?”

3.3. ““About what time do you ordinarily eat dinner?”About what time do you ordinarily eat dinner?”

4.4. ““How important is it for stores to carry a large variety of different How important is it for stores to carry a large variety of different brands of this product?”brands of this product?”

5.5. ““Would you favor increasing taxes to cope with the current fiscal Would you favor increasing taxes to cope with the current fiscal crisis?”crisis?”

6.6. ““Don’t you see some danger in the new policy?”Don’t you see some danger in the new policy?”

7.7. ““What small appliance, such as countertop appliances, have you What small appliance, such as countertop appliances, have you purchased in the past month?”purchased in the past month?”

8.8. ““When you buy ‘fast food,’ what percentage of the time do you When you buy ‘fast food,’ what percentage of the time do you order each of the following types of food?”order each of the following types of food?”

9.9. ““Do you like orange juice?”Do you like orange juice?”


Comments on Questions:Comments on Questions:

1.1. A loaded question because everyone wants to save lives. A loaded question because everyone wants to save lives. Also, it presumes that lower speed limits saves lives.Also, it presumes that lower speed limits saves lives.

2.2. Too specific because respondents likely cannot remember Too specific because respondents likely cannot remember the exact number of times.the exact number of times.

3.3. Ambiguous because don’t know if dinner is lunch or Ambiguous because don’t know if dinner is lunch or evening.evening.

4.4. Not specific enough about types of stores.Not specific enough about types of stores.

5.5. Overemphasis because refers to crisis.Overemphasis because refers to crisis.

6.6. Leading question because uses “danger” in sentence.Leading question because uses “danger” in sentence.

7.7. Answers likely to relate only to countertop appliances and Answers likely to relate only to countertop appliances and not all small appliances.not all small appliances.

8.8. Over generalization because does not specify time period.Over generalization because does not specify time period.

9.9. Ambiguous because may like orange juice for themselves, Ambiguous because may like orange juice for themselves, or for their kids, but really do not know.or for their kids, but really do not know.


• ObjectiveObjective: to identify possible shortcomings of : to identify possible shortcomings of questionnaire.questionnaire.

• ApproachesApproaches – informal or formal. – informal or formal.• Can assessCan assess::

• No hard and fast rules.No hard and fast rules.

• ability to perform ability to perform meaningful analysesmeaningful analyses

• time to complete the time to complete the questionnaire questionnaire

• cost of data collection cost of data collection • which questions are which questions are

relevant relevant • whether key questions whether key questions

have been overlooked have been overlooked • sources of biassources of bias

• clarity of instructionsclarity of instructions

• cover letter cover letter

• clarity of questions clarity of questions

• adequacy of codes adequacy of codes and categories for and categories for pre-coded questionspre-coded questions

• quality of responses quality of responses

• likely response rate likely response rate

Pre-testing QuestionnairesPre-testing Questionnaires::


Scales = the approach used to Scales = the approach used to measure concepts (constructs).measure concepts (constructs).

Two OptionsTwo Options::

1.1. Use published scales.Use published scales.

2.2. Develop original Develop original

scales.scales.

Scale Scale DevelopmentDevelopment

Sources of Published Sources of Published ScalesScales

Organizational Behavior and ManagementOrganizational Behavior and ManagementPrice, James L., Price, James L., Handbook of Organizational MeasurementHandbook of Organizational Measurement, International , International Journal of Manpower, Vol. 18, Number 4/5/6, 1997, ISSN 0143-7720, Journal of Manpower, Vol. 18, Number 4/5/6, 1997, ISSN 0143-7720, www.mcb.co.uk Has 28 chapters with constructs measuring organizational behavior.Has 28 chapters with constructs measuring organizational behavior.

Management Information Systems (MIS)Management Information Systems (MIS) www.ucalgary.ca/~newsted/surveys.html. . www.misq.org/archivist/home.html. . MarketingMarketingBearden, William O. and Richard Netemeyer, Bearden, William O. and Richard Netemeyer, Handbook of Marketing Handbook of Marketing ScalesScales, Sage Publications, 2, Sage Publications, 2ndnd ed., 1998. Summarizes over 130 marketing ed., 1998. Summarizes over 130 marketing related scales.related scales.

Bruner, Gordon Paul Hensel, Bruner, Gordon Paul Hensel, Marketing Scales HandbookMarketing Scales Handbook, Chicago, Ill., , Chicago, Ill., American Marketing Association, 1992. Includes almost 600 scales.American Marketing Association, 1992. Includes almost 600 scales.

GeneralGeneralRobinson, John P., Phillip R. Shaver and Lawrence S. Wrightsman, Robinson, John P., Phillip R. Shaver and Lawrence S. Wrightsman, Measures of Personal and Social Psychological AttitudesMeasures of Personal and Social Psychological Attitudes, San Diego, CA: , San Diego, CA: Academic Press, 1991. Contains over 150 published scales in 11 different Academic Press, 1991. Contains over 150 published scales in 11 different areas.areas.

Buros Institute of Mental Measurement’s website – has reviews of Buros Institute of Mental Measurement’s website – has reviews of published tests and measurements. published tests and measurements. www.unl.edu/buros

Decision AnalystDecision Analystwww.decisionanalyst.comwww.decisionanalyst.com

Decisive TechnologyDecisive Technologywww.decisive.comwww.decisive.com

Perseus DevelopmentPerseus Developmentwww.perseusdevelopment.www.perseusdevelopment.comcom Socratic TechnologiesSocratic Technologieswww.sotech.comwww.sotech.com

SPSSSPSSwww.spss.comwww.spss.com

Online Questionnaire Online Questionnaire DesignDesign

Survey BuilderSurvey Builderwww.surveybuilder.com SurveyProSurveyProwww.surveypro.com SurveySezSurveySezwww.surveysez.com WebSurveyorWebSurveyorwww.websurveyor.com

Types of ScalesTypes of Scales::• MetricMetric (interval & ratio)(interval & ratio)

• Likert-typeLikert-type• Summated-Ratings Summated-Ratings

(Likert)(Likert)• NumericalNumerical• Semantic Differential Semantic Differential • Graphic-RatingsGraphic-Ratings

• NonmetricNonmetric (nominal & (nominal & ordinal)ordinal)• CategoricalCategorical• Constant Sum MethodConstant Sum Method• Paired ComparisonsPaired Comparisons• Rank OrderRank Order• SortingSorting

MEASUREMENT SCALESMEASUREMENT SCALES

Examples of Likert-Type ScalesExamples of Likert-Type Scales::““When I hear about a new restaurant , I eat there to see When I hear about a new restaurant , I eat there to see what it is like.”what it is like.”

Strongly Agree Neither Agree Disagree Strongly Agree Neither Agree Disagree StronglyStrongly

Agree SomewhatAgree Somewhat or Disagree Somewhat or Disagree Somewhat DisagreeDisagree

11 2 2 3 3 44 5 5

MEASUREMENT SCALES – MetricMEASUREMENT SCALES – Metric

““When I hear about a new restaurant , I eat there to see When I hear about a new restaurant , I eat there to see what it is like.”what it is like.”

Strongly Strongly Strongly Agree Strongly Agree

Disagree Disagree 11 2 2 3 3 4 4

5 5

Summated Ratings ScalesSummated Ratings Scales::

A scaling technique in which respondents are asked A scaling technique in which respondents are asked to indicate their degree of agreement or to indicate their degree of agreement or disagreement with each of a number of statements. disagreement with each of a number of statements. A subject’s attitude score (summated rating) is the A subject’s attitude score (summated rating) is the total obtained by summing over the items in the total obtained by summing over the items in the scale and dividing by the number of items to get the scale and dividing by the number of items to get the average.average.

ExampleExample::

““My sales representative is . . . . “My sales representative is . . . . “ SDSD D D N N A A SA SA

CourteousCourteous ______ ______ ______ ______ ______FriendlyFriendly ______ ______ ______ ______ ______HelpfulHelpful ______ ______ ______ ______ ______KnowledgeableKnowledgeable ______ ______ ______ ______ ______

MEASUREMENT SCALES – Metric MEASUREMENT SCALES – Metric

Alternative Approach to Summated Ratings scalesAlternative Approach to Summated Ratings scales::

““When I hear about a new restaurant , I eat there to see what it is like.”When I hear about a new restaurant , I eat there to see what it is like.”

Strongly AgreeStrongly Agree Neither Agree Neither Agree Disagree Strongly Disagree Strongly

Agree SomewhatAgree Somewhat or Disagree or Disagree Somewhat Disagree Somewhat Disagree

11 22 3 3 44 5 5

““I always eat at new restaurants when someone tells me they are good.”I always eat at new restaurants when someone tells me they are good.”

Strongly AgreeStrongly Agree Neither Agree Neither Agree Disagree Strongly Disagree Strongly

Agree SomewhatAgree Somewhat or Disagree or Disagree Somewhat Disagree Somewhat Disagree

11 22 3 3 44 5 5


This approach includes a separate labeled Likert scale with each This approach includes a separate labeled Likert scale with each item (statement). The summated rating is a total of the item (statement). The summated rating is a total of the responses for all the items divided by the number of items.responses for all the items divided by the number of items.

Numerical ScalesNumerical Scales::

ExampleExample::

““Using a 10-point scale, where ‘1’ is ‘not at all Using a 10-point scale, where ‘1’ is ‘not at all important’ and ’10’ is ‘very important,’ how important’ and ’10’ is ‘very important,’ how important is ______ in your decision to do important is ______ in your decision to do business with a particular vendor.”business with a particular vendor.”

Note: you fill in the blank with an attribute, such as Note: you fill in the blank with an attribute, such as reliable delivery, product quality, complaint resolution, reliable delivery, product quality, complaint resolution, and so forth.and so forth.


Semantic Differential ScalesSemantic Differential Scales::

A scaling technique in which respondents are asked A scaling technique in which respondents are asked to check which space between a set of bipolar to check which space between a set of bipolar adjectives or phrases best describes their feelings adjectives or phrases best describes their feelings toward the stimulus object.toward the stimulus object.

ExampleExample::

““My sales representative is . . . . “My sales representative is . . . . “

CourteousCourteous ______ ______ ______ ______ ______DiscourteousDiscourteous

FriendlyFriendly ______ ______ ______ ______ ______ UnfriendlyUnfriendly

HelpfulHelpful ______ ______ ______ ______ ______ UnhelpfulUnhelpful

HonestHonest ______ ______ ______ ______ ______ DishonestDishonest


Graphic-Ratings ScalesGraphic-Ratings Scales::

A scaling technique in which respondents are asked to indicate A scaling technique in which respondents are asked to indicate theirtheir

ratings of an attribute by placing a check at the appropriate point ratings of an attribute by placing a check at the appropriate point on a line that runs from one extreme of the attribute to the other.on a line that runs from one extreme of the attribute to the other.

““Please evaluate each attribute in terms of how important the Please evaluate each attribute in terms of how important the attribute is to you personally (your company) by placing an “X” attribute is to you personally (your company) by placing an “X”

at the position on the horizontal line that most reflects your at the position on the horizontal line that most reflects your feelings.”feelings.”

Not ImportantNot Important Very Important Very ImportantCourteousnessCourteousness __________________________________________________________________________FriendlinessFriendliness __________________________________________________________________________HelpfulnessHelpfulness __________________________________________________________________________KnowledgeableKnowledgeable __________________________________________________________________________


Categorical scaleCategorical scale::

Categorical scales are nominally measured Categorical scales are nominally measured opinion scales that have two or more response opinion scales that have two or more response categories.categories.

““How satisfied are you with your current job?”How satisfied are you with your current job?”

[ ][ ] Very Satisfied Very Satisfied

[ ] [ ] Somewhat SatisfiedSomewhat Satisfied

[ ] [ ] Neither Satisfied nor DissatisfiedNeither Satisfied nor Dissatisfied

[ ] [ ] Somewhat DissatisfiedSomewhat Dissatisfied

[ ] [ ] Very DissatisfiedVery Dissatisfied

Note: Some researchers consider this a metric scale when Note: Some researchers consider this a metric scale when coded 1 – 5 .coded 1 – 5 .

MEASUREMENT SCALES – NonmetricMEASUREMENT SCALES – Nonmetric

Constant-Sum MethodConstant-Sum Method::

A scaling technique in which respondents are asked to divideA scaling technique in which respondents are asked to divide

some given sum among two or more attributes on the basis ofsome given sum among two or more attributes on the basis of

their importance to them.their importance to them.

““Please divide 100 points among the following attributes in Please divide 100 points among the following attributes in

terms of the relative importance of each attribute to you.”terms of the relative importance of each attribute to you.”

Courteous ServiceCourteous Service ________

Friendly ServiceFriendly Service ________

Helpful ServiceHelpful Service ________

Knowledgeable ServiceKnowledgeable Service ________

TotalTotal 100 100


Paired Comparison Paired Comparison MethodMethod::

A scaling technique in which respondents are A scaling technique in which respondents are given pairs of stimulus objects and asked which given pairs of stimulus objects and asked which object in a pair they prefer most.object in a pair they prefer most.

““Please circle the attribute describing a sales Please circle the attribute describing a sales representative which you consider most representative which you consider most desirable.”desirable.”

CourteousCourteous versusversus Knowledgeable Knowledgeable

FriendlyFriendly versusversus HelpfulHelpful

HelpfulHelpful versusversus Courteous Courteous


SortingSorting::

A scaling technique in which respondents A scaling technique in which respondents areare

asked to indicate their beliefs or opinions byasked to indicate their beliefs or opinions byarranging objects (items) on the basis of arranging objects (items) on the basis of perceived importance, similarity, preferenceperceived importance, similarity, preferenceor some other attribute.or some other attribute.


Rank Order MethodRank Order Method::

A scaling technique in which respondents are presented A scaling technique in which respondents are presented with several stimulus objects simultaneously and asked with several stimulus objects simultaneously and asked to order or rank them with respect to a specificto order or rank them with respect to a specificcharacteristic.characteristic.

““Please rank the following attributes on how important Please rank the following attributes on how important each is to you in relation to a sales representative. Place a each is to you in relation to a sales representative. Place a “1” beside the attribute which is most important, a “2” “1” beside the attribute which is most important, a “2” next to the attribute that is second in importance, and so next to the attribute that is second in importance, and so on.”on.”

Courteous ServiceCourteous Service ______Friendly ServiceFriendly Service ______Helpful ServiceHelpful Service ______Knowledgeable ServiceKnowledgeable Service ______

MEASUREMENT SCALES – Nonmetric MEASUREMENT SCALES – Nonmetric

Practical Decisions When Developing Practical Decisions When Developing ScalesScales::

• Number of items (indicators) to measure a Number of items (indicators) to measure a concept? concept?

• Number of scale categories? Number of scale categories?

• Odd or even number of categories? Odd or even number of categories?

(Include neutral point ?) (Include neutral point ?)

• Balanced or unbalanced scales? Balanced or unbalanced scales?

• Forced or non-forced choice? Forced or non-forced choice?

(Include Don’t Know ?) (Include Don’t Know ?)

• Category labels for scales? Category labels for scales?

• Scale reliability and validity? Scale reliability and validity?

Scale DevelopmentScale Development

Balanced vs. Unbalanced Balanced vs. Unbalanced ScalesScales??

BalancedBalanced: : • “ “To what extent do you consider TV shows with sex To what extent do you consider TV shows with sex

andandviolence to be acceptable for teenagers to violence to be acceptable for teenagers to

view?” view?” __ Very Acceptable__ Very Acceptable

__ Somewhat Acceptable__ Somewhat Acceptable__ Neither Acceptable or Unacceptable__ Neither Acceptable or Unacceptable__ Somewhat Unacceptable__ Somewhat Unacceptable__ Very Unacceptable__ Very Unacceptable

Unbalanced:Unbalanced:

__ Very Acceptable __ Very Acceptable

__ Somewhat Acceptable __ Somewhat Acceptable

__ Unacceptable__ Unacceptable


Forced or Non-ForcedForced or Non-Forced??

• ““How likely are you to purchase a laptop PC in the next six months?”How likely are you to purchase a laptop PC in the next six months?”

Very Very Very Very

UnlikelyUnlikely Likely Likely

11 22 3 3 4 4 5 5 6 6 __ No Opinion __ No Opinion


Category Labels for ScalesCategory Labels for Scales??

Verbal LabelVerbal Label: : • ““How important is the size of the hard drive in selecting a laptop PC to purchase?”How important is the size of the hard drive in selecting a laptop PC to purchase?” Very SomewhatVery Somewhat Neither Important Somewhat Neither Important Somewhat Very Very

Unimportant UnimportantUnimportant Unimportant or Unimportant Important Important or Unimportant Important Important

11 2 2 3 3 4 4 5 5

Numerical LabelNumerical Label::

• ““How likely are you to purchase a laptop PC in the next six months?”How likely are you to purchase a laptop PC in the next six months?”

Very Very Very Very

UnlikelyUnlikely Likely Likely

11 2 2 3 3 4 4 5 5

UnlabeledUnlabeled::

• ““How important is the weight of the laptop PC in deciding which brand How important is the weight of the laptop PC in deciding which brand

to purchase?”to purchase?”

VeryVery Very Very

UnimportantUnimportant Important Important

______ ___ ___ ___ ___ ______ ___ ___


Choosing a Measurement Choosing a Measurement ScaleScale::• Capabilities of Capabilities of

Respondents.Respondents.• Context of Scale Context of Scale

Application.Application.• Data Analysis Approach.Data Analysis Approach.• Validity and Reliability.Validity and Reliability.


Assessing Measurement Assessing Measurement ScalesScales::

• ValidityValidity

• ReliabilityReliability


Measurement ErrorMeasurement Error = occurs when the = occurs when the values obtained in a survey (observed values obtained in a survey (observed values) are not the same as the true values) are not the same as the true values (population values).values (population values).

RESEARCH DESIGNRESEARCH DESIGN

Types of ErrorsTypes of Errors::

• Nonresponse = problem definition, refusal, Nonresponse = problem definition, refusal, sampling, etc.sampling, etc.• Response = respondent or interviewer.Response = respondent or interviewer.• Data Collection Instrument:Data Collection Instrument:

Construct Development.Construct Development. Scaling Measurement.Scaling Measurement. Questionnaire Design/Sequence, etc.Questionnaire Design/Sequence, etc.

• Data Analysis.Data Analysis.• Interpretation.Interpretation.

SECONDARY DATASECONDARY DATA

Data that has been Data that has been gathered gathered previously for previously for other purposes.other purposes.

SECONDARY DATASECONDARY DATA

Secondary Data Secondary Data IssuesIssues::

• AvailabilityAvailability

• RelevanceRelevance

• AccuracyAccuracy

• SufficiencySufficiency








MethodsMethods::

DependenceDependence• Multiple RegressionMultiple Regression• Discriminant Discriminant

AnalysisAnalysis• ANOVA/MANOVAANOVA/MANOVA

InterdependenceInterdependence• Factor AnalysisFactor Analysis• Cluster AnalysisCluster Analysis

Data AnalysisData Analysis

Learning CheckpointLearning Checkpoint::

• Define a research problem to be studied.Define a research problem to be studied.• Identify the topics /concepts that will be coveredIdentify the topics /concepts that will be covered

to answer research questions.to answer research questions.• Identify the types of questions and/or scaling Identify the types of questions and/or scaling

you will use.you will use.• How will you evaluate the questions/scales you How will you evaluate the questions/scales you

use?use?• Determine the best way to collect the data.Determine the best way to collect the data.• Present group suggestions; defend.Present group suggestions; defend.

Research Design & Data Research Design & Data CollectionCollection

Documents

Research Process, Research Design and Questionnaires