04b Sampling, Questionnaire & Scaling Techniques

7/31/2019 04b Sampling, Questionnaire & Scaling Techniques

1/66

Sampling, Questionnaire &

Scaling Techniques

ANAND PRAKASH


2/66

Questionnaire


3/66


4/66

Flowchart for Instrument Design

Phase 1


5/66


6/66

Flowchart for Instrument Design

Phase 3


7/66

Guidelines for Question Sequencing

Interesting topics early

Classification questions later

Sensitive questions later

Simple items early

Transition between topics


8/66

Overcoming Instrument Problems

Build rapport

Redesign question process

Explore alternatives

Use other methods

Pretest


9/66

A Good Questionnaire

Appears as easy to compose as a good poem

But, usually the result of long, painstaking

work


10/66

Developing a Questionnaire

No hard and fast rules

Only guidelines


11/66

Major Decisions in Questionnaire

Design What should be asked?

How should each question be phrased?

In what sequence should the questions bearranged?

What questionnaire layout will best serve

the research objectives?

How should the questionnaire be pre-tested?

Should it be revised?


12/66

Questionnaire Organization

Logical flow

Usually go from general to specific

Ask sensitive questions later


13/66


14/66

Some Questionnaires


15/66

Customer Service Satisfaction Survey

PURPOSE: Our goal is to provide all of the computer users in ourorganization with the highest quality of service and responsivesupport. In order to do this, we need your help.Please take a moment to evaluate your most recent experience withour Help Desk Support Engineers. Please rate your satisfaction withthe Help Desk Service form Very Satisfied to Very Dissatisfied.

1. Ease of reaching the Help Desk.2. Time required to restore or repair the equipment or software.3. Prompt response by Help Desk Support Engineer.4. Technical understanding of your problem.5. Ability to resolve problem on initial visit.

6. Do you have additional equipment or reinstallations planned for yoursite within the next six months? ____________________________

7. Please provide any additional comments that could help us improveour Help Desk service and support in the future: _______________

______________________________________________________


16/66

EMPLOYEE SURVEY

SURVEY OVERVIEW

Every organization has a unique and individual culture.This culture affects every action of the organization andpermeates every level. If an organization wants to

function at its most effective level, it must pay closeattention to its culture. Successful organizations developa culture over time to achieve their vision, mission, andobjectives, which move them closer to their vision.

The following survey is designed to evaluate CornerRockManufacturing's culture and pinpoint possible areas forimprovement.


17/66


18/66

Pyramid Corporation 360 DegreeLeadership Assessment

The purpose of this 360 Degree Assessment is to identify improvement opportunities for our leadership personnel.This information will be used to measure the effectiveness of our leadership today and where they might be in thefuture. All information given in this assessment and subsequent interviews will be held confidential by ourorganization.

Mark on 5 point scale 1 = Definite Strength 2 = Moderately Effective 3 = Average Performance 4 = RarelyEffective 5 = Definite Weakness

LEADERSHIP EFFECTIVENESS

1. This individual is sensitive to the influence his/her actions have on co-workers.2. This individual leads co-workers by example.3. This individual continually encourages co-workers to express their ideas and opinions.4. When conflict occurs, this individual resolves them in a constructive win/win manner.5. This individual continually develops the spirit of teamwork among co-workers.6. This individual always looks for new and creative methods to motivate co-workers.7. This individual clearly understands co-worker's roles in our organization.

PLANNING

8. This Individual understands his/her job responsibility, accountability, and authority clearly.9. This individual makes realistic plans and schedules and puts them in writing.10. This individual uses his/her resources (workforce, time, money, etc.) productively.11. This individual sees that co-workers have the necessary resources to do their jobs productively.


19/66

Health Insurance Satisfaction Survey

Please take a few minutes and tell us how we are doing. We appreciate your valuabletime and comments. All your responses will be held confidential.

1. Date:2. Time:3. How would you rate our agency on type of service provided:

4. Purpose for insurance policy:5. Number of people covered by your insurance policy:6. How much time did it take our staff to collect the information and complete your

claim? 5 minutes or less 6-15 minutes 16-30 minutes 31-45 minutes

46 minutes to 1 hour Over 1 hour

7. Would you recommend our health insurance services to your relatives and friends?Yes Maybe No


20/66

How would you rate our agency on a 5 point scalemeaning 1 = Great 2 = Very Good 3 = Good 4 = Fair 5= Poor.

8. Promptness

9. Friendliness10. Patience11. Responsiveness12. Clear consistent communication

13. How claims were handled14. How would you rate the value received for money

spent for our health insurance services?

15. Is there anything we could do differently to improve our

insurance coverage and/or service?

http://www.surveytracker.com/surveys/samples.htm#360
http://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htmhttp://www.surveytracker.com/surveys/samples.htm


21/66

KINGFISHER AIRLINES[The Good Times Monitor]

Wed like to know a little more about your flight1. Your flight number: IT________________________________2. Your Seat No.:______________________________________3. Takeoff city:________________ Landing city:_____________4. Flight date (dd)/(mm)/(yyyy): __________________________

5. King Club No.:______________________________________

Wed like to know a little more about youName (Mr./Mrs.)________________________________________Address:______________________________________________

_____________________________________________________City:______________________________ Pin Code:___________Tel. No.:______________________________________________Email ID:______________________________________________


22/66


How did you make your reservation?

[ ] Through our website?

[ ] Through a travel agent?

[ ] Through the Kingfisher Airlines ticketing office?

[ ] Through home delivery of ticket?

In which city did you make your reservation? ________________

Had a good time?This section comprises of asking opinions for three dimensions ona four pointer scale as Wow, Good, Not bad, and Could bebetter. Various statements under this heading are:


23/66


1. Reaching Outa) Accessibility of our call center members

b) How were we during the reservation

process?i. Staff efficiency

ii. Staff courtesy and friendliness

c) Overall experience

2. At the Airport

3. On board


24/66


1. Reaching Out2. At the Airport

a) How did we handle your baggage?i. Time taken for security screeningii. Assistance provided

b) How did we check you in?

i. Waiting time at Check-in counterii. Time taken in to Check-iniii. Staff greeting, helpfulness & warmthiv. Staff presentabilityv. Staff efficiencyvi. Overall check-in process

c) Did we get you on board alright?i. Clarity of boarding announcement

ii. Process at the boarding halliii. Boarding the aircraft

d) Overall airport experience

3. On board


25/66


1. Reaching Out2. At the Airport3. On board

a) How did you like your crew?i. Welcome while boardingii. Assistance on boardiii. Courtesy & professionalism

iv. Responsiveness to your needsv. Presentabilityvi. Overall cabin crew rating

b) Had an entertaining time?i. Audio selection Kingfisher Radioii. Video selection Fun TViii. Magazine Hi-Blitz

c) Did we make ourselves heard?i. Clarity of captain's announcements

ii. Flight info in captain's announcementd) Did we make you comfortable?

i. Cleanliness of cabinii. Cleanliness of washroomiii. Temperature before takeoff Too warm [ ] Just right [ ] Too cold [ ]iv. Temperature after takeoff Too warm [ ] Just right [ ] Too cold [ ]v. Did we make you feel at home


26/66

Lets Talk About FoodCould you tell us what kind of meal you had?[ ] Vegetarian [ ] Non-vegetarian[ ] Other (please specify) __________________________________

Rate on the scale of 14, where 1 = Wow, , 4 = Could be bettera) Appropriateness of the meal for the time of flightb) Visual appealc) The tasted) Meal portione) Overall food rating

How was the Kingfisher Class Experience, overall?Wow [ ] Good [ ] Not bad [ ] Could be better [ ]

How can we make the Kingfisher Class Experience even better for you?[ ] Suggestion [ ] Query [ ] Compliment [ ] Concern______________________________________________________________________________________________________________________________________________________________________________________________________



27/66

Check Your Own StudentFeedback Sheet

1. Knowledge Gain

2. Guidance & Advice

3. Style of Teaching4. Atmosphere in Class

5. Adaptability & Responsiveness

6. General Impression


28/66


1. Knowledge Gain Could have learnt by reading text book

Copies/Notes provided to supplement text book

Explanation and examples given in the class

A very all-round, practicable learning


3. Style of Teaching

4. Atmosphere in Class5. Adaptability & Responsiveness



29/66


1. Knowledge Gain

2. Guidance & Advice Questions not always answered

Questions usually answered but partially Care taken to explain specifics

Care taken to clarify satisfactory & wholistically





30/66


1. Knowledge Gain



Mainly syllabus covering lectures (Speech) Limited use of OHP/PC for presentation

Use of OHP/PC + cases/examples for teaching

Balanced use of lectures, cases, and other acts




31/66


1. Knowledge Gain



4. Atmosphere in Class Not teaching, one way communication by lectures

Too light and open; not much learning

Occasionally lively

Usually very lively5. Adaptability & Responsiveness



32/66


1. Knowledge Gain2. Guidance & Advice3. Style of Teaching4. Atmosphere in Class5. Adaptability & Responsiveness

Generally, a feeling of unapproachability Generally approachable, but time severely limited Generally approachable and usually tries to

understand students view point Very approachable, responsive and flexible



33/66


1. Knowledge Gain




6. General Impression Attend classes mainly to avoid TNG

Would attend since helpful for examinations

Would attend mostly since a learning experience

Would positively attend all classes


34/66


Are you willing to discuss your assessmentwith the concerned faculty?

Yes / No

Do you wish to keep this confidential? Yes / No

Please use the reverse side for generalcomments.

Surveys are all around us They can be used to:


35/66

Surveys are all around us. They can be used to:1. Measure and improve customers satisfaction levels.2. Discover worker attitudes about issues affecting the work environment, quality, and

productivity.3. Quickly evaluate opinions and attitudes.4. Provide data for long-range strategic planning.

5. Enhance customer relations.6. Determine specific training needs for an organization.7. Determine the effectiveness of a Help Desk.8. Evaluate internal and external supplier quality9. Determine readiness for industry standards such as ISO 9000 and Malcolm Baldrige.10. Increase employee commitment through involvement and implementation of a

common mission.

11. Evaluate an employee's leadership effectiveness.12. Measure the quality of an education system.13. Evaluate and track Total Quality Team effectiveness.14. Ensure communication among all levels of an organization.15. Determine public perception of political candidates and issues.16. Determine how many and what types of people watch television programs.

17. Determine the effectiveness and attractiveness of a web page.18. Evaluate customer satisfaction.19. Gather feedback about a restaurants service and aesthetics.20. Evaluate the effectiveness of care and facilities of a hospital.21. Measure satisfaction, cooperation, and readiness of the military.22. Discover new market possibilities.23. Gauge a lodger's enjoyment of the facilities, room cleanliness, etc.

24. Discover opportunities for improvement.


36/66


37/66

Reliability Reliability can be defined as the extent to which

measures are free from random error, XR. If XR = 0,the measure is perfectly reliable.

In test-retest reliability, respondents areadministered identical sets of scale items at two

different times and the degree of similarity betweenthe two measurements is determined.

In alternative-forms reliability, two equivalent formsof the scale are constructed and the same

respondents are measured at two different times, witha different form being used each time.


38/66

Reliability

Internal consistency reliability determines the extentto which different parts of a summated scale areconsistent in what they indicate about the characteristicbeing measured.

In split-half reliability, the items on the scale aredivided into two halves and the resulting half scores arecorrelated.

The coefficient alpha, or Cronbach's alpha, is the

average of all possible split-half coefficients resultingfrom different ways of splitting the scale items. Thiscoefficient varies from 0 to 1, and a value of 0.6 or lessgenerally indicates unsatisfactory internal consistency

reliability.


39/66

Validity

The validity of a scale may be defined as the extent towhich differences in observed scale scores reflect truedifferences among objects on the characteristic beingmeasured, rather than systematic or random error.

Perfect validity requires that there be no measurementerror (XO = XT, XR = 0, XS = 0).

Content validity is a subjective but systematicevaluation of how well the content of a scale represents

the measurement task at hand. Criterion validity reflects whether a scale performs as

expected in relation to other variables selected (criterionvariables) as meaningful criteria.


40/66

Validity

Construct validity addresses the question of whatconstruct or characteristic the scale is, in fact,measuring. Construct validity includes convergent,discriminant, and nomological validity.

Convergent validity is the extent to which the scalecorrelates positively with other measures of the sameconstruct.

Discriminant validity is the extent to which a measuredoes not correlate with other constructs from which it is

supposed to differ. Nomological validity is the extent to which the scale

correlates in theoretically predicted ways with measuresof different but related constructs.


41/66

Reliability is a necessary, but notsufficient, condition for validity.

BACK PAIN AND DISABILITY QUESTIONNAIRE (Revised Oswestry)


42/66

Name Date

This questionnaire has been designed to give your health care provider information as to how your back pain has affected your ability to manage everyday life. Please answer every section and mark in each

section only the ONE box which applies to you. I realize you may consider that two of the statements in any one section relate to you, but please just mark the box which most closely describes your problem

today.

Section 1 Pain Intensity Section 6 Standing

The pain comes and goes and is very mild.

The pain is mild and does no t vary much.

The pain comes and goes and is moderate.

The pain is moderate and does not vary much.The pain comes and goes and is severe.

The pain is severe and does not vary much.

I can stand as long as I want without pain.

I have some pain on standing but it does not increase with time.

I cannot stand for longer than one hour without increasing pain.

I cannot stand for longer than hour without increasing pain. I cannot stand for longer than 10 minutes without increasing pain.

I avoid standing because it increases the pain straight away.

Section 2 Personal Care Section 7 Sleeping

I would not have to change my way of washing or dressing in order to avoid pain.

I do not normally change my way of washing and dressing even though it causes some pain.

Washing and dressing increase the pain but I manage not to change my way of doing it.

Washing and dressing increase the pain and I find it necessary to change my way of doing i t.

Because of the pain I am unable to do some washing and dressing.

Because of the pain I am unable to do any washing and dressing without help.

I get no pain in bed.

I get pain in bed but it does not prevent me from sleeping well.

Because of pain my normal nights sleep is reduced by less than .



Pain prevents me from sleeping at all.

Section 3 Lifting Section 8 Social Life

I can lift heavy weights without extra pain.

I can lift heavy weights but it causes extra pain.

Pain prevents me from lifting heavy weights off the floor.

Pain prevents me from lifting heavy weights off the floor, but I manage if they are conveniently

positioned (e.g. on a table).

Pain prevents me from lifting heavy weights but I can manage light to medium weights if they are

conveniently positioned.

I can only lift very light weights at the most.

My social life is normal and g ives me no pain.

My social life is normal but increases the degree of pain.

Pain has no significant effect on my social life apart from limiting my more energetic

interests, e.g. dancing, etc.

Pain has restricted my social life and I do not go out very often.

Pain has restricted my social life to my home.

I have hardly any social life because o f the pain.

Section 4 Walking Section 9 Traveling

I have no pain on walking.

I have some pain on walking but it does not increase with distance.I cannot walk more than 1 mile without increasing pain.

I cannot walk more than mile without increasing pain.

I cannot walk more than mile without increasing pain.

I cannot walk at all without increasing pain.

I have no pain while traveling.

I have some pain while traveling but none o f my usual forms of travel make it anyworse.

I have extra pain while traveling but it does not compel me to seek alternate forms of

travel.

I have extra pain while traveling that compels me to seek alternative forms of travel.

Pain restricts all forms of travel.

Pain prevents all forms of travel except that done lying down.

Section 5 Sitting Section 10 Changing Degree of Pain

I can sit in any chair as long as I like.

I can only sit in my favorite chair as long as I like.

Pain prevents me from sitting more than 1 hour.

Pain prevents me from sitting more than hour.

Pain prevents me from sitting more than 10 minutes.I avoid sitting because it increases my pain right away.

My pain is rapidly getting better.

My pain fluctuates but overall is definitely getting better.

My pain seems to be getting better but improvement is slow at present.

My pain is neither getting better nor worse.

My pain is gradually worsening.My pain is rapidly worsening.


43/66

Oswestry Disability Questionnaire (revised)

Purpose: * Acute and Chronic population of low back pain sufferers* Discriminate between chronic and acute low back pain

* An evaluative measure in clinical trials

* Used to predict different rates of improvement

Face Validity: + Measured 0 5 by degree of difficulty+ Very specific questions

+ No specific frame of reference

+ Capacity related questions

+ Score by summing all items = percentage score

Content Validity: Pain intensity Personal care Lifting

Walking Sitting Standing

Sleeping Sex / social life Travelling


44/66

Oswestry Disability Questionnaire (revised)

Content Validity: Omits: bending kneelingtwisting turning

emotional state sudden movement

Sex life reduced response rates

(Hudson-Cook et al. 1989)

Scoring issues: 11% is a cut off score (Erhard et al. 1994)

00 - 20% Minimal Disability20 - 40% Moderate Disability

40 - 60% Severe Disability

60 - 80% Crippled

80 - 100% Bed Bound or Exaggerating

Stratford et al. 1988


45/66

Modified Oswestry Low Back Pain Disability Questionnaire

This questionnaire has been designed to give your therapist information as to how your back pain has affected your ability to manage ineveryday life. Please answer every question by placing a mark in the one box that best describes your condition today. We realize you mayfeel that 2 of the statements may describe your condition, but please mark only the box that most closely describes your current condition.

Pain Intensity I can tolerate the pain I have without having to use pain medication. The pain is bad, but I can manage without having to take pain medication. Pain medication provides me with complete relief from pain. Pain medication provides me with moderate relief from pain. Pain medication provides me with little relief from pain. Pain medication has no effect on my pain.

Personal Care (e.g., Washing, Dressing) I can take care of myself normally without causing increased pain. I can take care of myself normally, but it increases my pain. It is painful to take care of myself, and I am slow and careful. I need help, but I am able to manage most of my personal care. I need help every day in most aspects of my care. I do not get dressed, I wash with difficulty, and I stay in bed.

Lifting I can lift heavy weights without increased pain. I can lift heavy weights, but it causes increased pain. Pain prevents me from lifting heavy weights off the floor, but I can manage if the weights are conveniently positioned (e.g., on a table). Pain prevents me from lifting heavy weights, but I can manage light to medium weights if they are conveniently positioned.

I can lift only very light weights. I cannot lift or carry anything at all.

Walking Pain does not prevent me from walking any distance. Pain prevents me from walking more than 1 mile. (1 mile = 1.6 km). Pain prevents me from walking more than 1/2 mile. Pain prevents me from walking more than 1/4 mile. I can walk only with crutches or a cane. I am in bed most of the time and have to crawl to the toilet.

Sitting I can sit in any chair as long as I like


46/66

I can sit in any chair as long as I like. I can only sit in my favorite chair as long as I like. Pain prevents me from sitting for more than 1 hour. Pain prevents me from sitting for more than 1/2 hour. Pain prevents me from sitting for more than 10 minutes. Pain prevents me from sitting at all.

Standing I can stand as long as I want without increased pain. I can stand as long as I want, but it increases my pain. Pain prevents me from standing for more than 1 hour. Pain prevents me from standing for more than 1/2 hour. Pain prevents me from standing for more than 10 minutes. Pain prevents me from standing at all.

Sleeping Pain does not prevent me from sleeping well. I can sleep well only by using pain medication. Even when I take medication, I sleep less than 6 hours. Even when I take medication, I sleep less than 4 hours. Even when I take medication, I sleep less than 2 hours.

Pain prevents me from sleeping at all.

Social Life My social life is normal and does not increase my pain. My social life is normal, but it increases my level of pain. Pain prevents me from participating in more energetic activities (e.g., sports, dancing). Pain prevents me form going out very often. Pain has restricted my social life to my home. I have hardly any social life because of my pain.

Traveling I can travel anywhere without increased pain.

I can travel anywhere, but it increases my pain. My pain restricts my travel over 2 hours. My pain restricts my travel over 1 hour. My pain restricts my travel to short necessary journeys under 1/2 hour. My pain prevents all travel except for visits to the physician / therapist or hospital.

Employment / Homemaking My normal homemaking / job activities do not cause pain. My normal homemaking / job activities increase my pain, but I can still perform all that is required of me. I can perform most of my homemaking / job duties, but pain prevents me from performing more physically stressful activities (e.g., lifting, vacuuming). Pain prevents me from doing anything but light duties. Pain prevents me from doing even light duties.

Pain prevents me from performing any job or homemaking chores.


47/66


48/66

Quebec Back Pain Disability Scale

Content Validity: Omits: twisting turning

emotional state sudden movement

sex life


49/66

Characteristics of Good Measurement

1. Utility / Practicality Is it useful?

2. Reliability Is it dependable?

3. Internal Consistency Split Half

4. Validity Does it do what it is supposed to?

5. Sensitivity Can it identify patients with a condition?

6. Specificity Can it identify those that do not have the condition?

7. Responsiveness Can it measure differences over time?


50/66

Consider the dart boards shown

below.


51/66

The marks you see on the targets

were left by darts thrown at them.


52/66

Ifthe goal was to hit the Bullseyewith each dart

Then the

resultswereconsistent

but off-target


53/66

Then the

resultswere bothconsistent

andaccurate

If the goal was to hit the Bullseyewith each dart


54/66

Then theresults

were

neitherconsistent

noraccurate



55/66

Reliability and Validity

Reliability has to do with consistency

Validity has to do with accuracy


56/66


We have

reliabilitybut notvalidity


57/66

We have

bothreliabilityand

validity



58/66

We have

neitherreliabilitynor validity



59/66

Reliability and Validity

To have validity we must first have

reliability

i.e. reliability is a prerequisite for validity

Reliability is a necessary but not

sufficient condition for validity


60/66

Reliability is a necessary but notsufficient condition for validity

We canhave

reliabilitywithouthavingvalidity


61/66

We cannothave

validity

without

first having

reliability

In order to be valid themeasurement must be reliable


62/66

Without

reliability,we cannot

have

validity

Reliability is a prerequisite forvalidity

M t li bl b t t


63/66

Measurement reliable but notvalid

M t li bl d


64/66

Measurement reliable andvalid


65/66

Measurementneither reliable nor valid


66/66

Reliable

Not Valid

Reliable

Valid

Not Reliable

Not Valid

Documents

04b Sampling, Questionnaire & Scaling Techniques