VARIABILITY OF LIKERT SCALES EFFECTS ON DATA COLLECTION RESULTS CLASS 7 – 2011/2012 MIMED - FMUP

VARIA

BILITY

OF

LIKERT

SCALE

S

EFFECTS

ON D

ATA C

OLLECTI

ON RESULT

S

CL

AS

S 7

– 2

01

1/ 2

01

2

MI M

ED

- F

MU

P

• Questionnaires

• Likert scale: what is it?

• Different Likert scales

• Study objective

• Methods

- Study participants

- Study design

• Results

• Conclusions

Questionnaries

Introduction Objective Methods

Results Conclusion

Currently, people’s opinion about an issue or service is a valuable way for researchers to acquire information in diverse areas of knowledge.

• A questionnaire is an essential instrument that aims to collect this information based on the inquisition of the population in study;

• It consists in a certain amount of questions covering a topic of interest to investigators;

• Is useful in almost every kind of job.

Likert Scale and Its Variables - A Comparative Study


Having in mind the need to measure and classify this information, one reaches the conclusion of the pressing necessity for scales, tools that permits measuring people’s attitudes and opinions[1].

Likert scale

1. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple treatment groups. Stat Med, 2010. 29(24): p. 2486-500.


Results Conclusion

LIKERT SCALE

A scale that consists in asking people to respond to a series of

statements about a topic, in terms of the extent to which they

agree or disagree with them.

What is it?

Measure pain, quality of life

Applications


2. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple treatment groups. Stat Med, 2010. 29(24): p. 2486-500. 3.Huber JF, H.J., Zumstein MD, Ruflin G, Lüscher M., Visual circle scale (VCS)-a patient-friendly scale to measure pain compared to VAS and Likert scale. Z Orthop Unfall, 2007. 145(6): p. 795-7.

[2]

[3]


Results Conclusion

DIFFERENT LIKERT SCALES

Different scales produce

different resultsThen, different

Likert scales may produce different

results

Search: some articles were found confirming our

idea


4 Dawes, J., Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 2008. 50(1): p. 61-77.

[4]


Results Conclusion

INFLUENCE OF A CENTRAL ANSWER

Different Likert scales

produce different results?

Does a neutral answer influence

the results?

Does both "I don't want to answer"

and neutral options influence the

results?

Apparently not studied yet.

Apparently not studied yet.


5. Kulas JT. , S.A., Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings. Journal of Research in Personality, 2009. 43: p. 489–49

Already studied

[5]

Does a "I don't want to answer" option

influence the results?


Results Conclusion

OBJECTIVES

Discover if the addition of a neutral element, a “refusal to answer” option or both

combined has repercussions in the results obtained.

Evaluate the reproducibility of the scales used in the questionnaire.


6. Hansson LM, G.M., Bergström R, Factors affecting reproducibility of dietary reports using food frequency questionnaires. Eur J Clin Nutr, 2000. 54(8).

[6]


Results Conclusion


Study Design:

One questionnaire with two versions

Questionnaire 1 Questionnaire 2Number of

questions in each

questionnaire: 12

Same questions in both questionnaires

Methods


Results Conclusion

Type of study

Our study is included in a Longitudinal Study type.

BECAUSE :

-There is a follow-up period of the individuals in study, which can be longer or shorter;

- Data is gathered at least in two points in time which allows us to observe changes in the population.



Results Conclusion

Students from classes 6 and 7

Study Participants:

All students that belong to the first year of the Integrated Master’s Degree in

Medicine of Faculdade de Medicina da Universidade do Porto, distributed by

classes 1 to 22

Exc

ept


Methods


Results Conclusion


Monday Tuesday Wednesday

Thursday Friday

8:30-11 h 1 4 6 9

14-16:30h 2 x 7 10

16:30-19h 3 5 8

These ten blocks where randomly divided in two

groups with SPSS.

Questionnaire 1

Questionnaire 2

Study Participants:

Methods


Results Conclusion

About the questionnaires:

• Theme of the questionnaire: Evaluation systems and study methods

Theme with less opinion change

probability


Why?

Methods


Results Conclusion

• Half of the participants answer to version 1 and the other half to version 2

• One month later, the first half answer to version 2 and the other half to version 1

Why?


To avoid certain biases.


Methods


Results Conclusion


• The questionnaires ought to be confidential.

How?The participants wrote their

names on the questionnaires and we

ensured confidentiality .

Why?In order to receive from

the participants their truly opinion


Methods


Results Conclusion



Comparison scale:I totally

agree I agree I disagreeI totally disagree

… plus a neutral answer:Neither agree nor disagree

… plus refusal to answer:I don’t want to answer

Methods


Results Conclusion



Three types of comparison

Comparison scaleVs

Comparison scale plus neutral answer

Comparison scale:Likert scale with “I totally

agree”, “I agree”, “I disagree” and “I totally disagree”

Comparison scaleVs

Comparison scale plus refusal to answer option

Comparison scaleVs

Comparison scale plus neutral and refusal to

answer option

Methods


Results Conclusion

About the questionaires:


To evaluate the reproducibility

Comparison scaleVs

Comparison scale

Comparison scale plus refusal to answer

optionVs

Comparison scale plus refusal to answer

option


answer optionVs


answer option

Comparison scale:Likert scale with “I totally

agree”, “I agree”, “I disagree” and “I totally disagree” Methods


Results Conclusion

Questionnaire 1I1. Comparison Scale2. Comparison Scale3. Comparison Scale + neutral answer4. Comparison Scale5. Comparison Scale + refusal to answer6. Comparison Scale + neutral answer + refusal to answer7. Comparison Scale + refusal answer

II1. Comparison Scale + refusal to answer2. Comparison Scale + neutral answer + refusal to answer3. Comparison Scale4. Comparison Scale + neutral answer + refusal to answer5. Comparison Scale

Questionnaire 2I1. Comparison Scale + neutral answer2. Comparison Scale3. Comparison Scale 4. Comparison Scale + neutral answer5. Comparison Scale 6. Comparison Scale + neutral answer + refusal to answer7. Comparison Scale

II1. Comparison Scale + refusal to answer2. Comparison Scale

3. Comparison Scale + refusal to answer4. Comparison Scale 5. Comparison Scale + neutral answer + refusal to answer

Results

Introduction

Objective

Methods

Results

Conclusion

Study Participants:


250

234

158

REPRODUCIBILITY

Introduction

Objective

Methods

Results

Conclusion

Valid

(n)

Question

Scale

Proportion of

Agreement

[Confidence interval]

Kappa


Weighted Kappa


157 I.2O sistema de avaliação por

exames é justo.

Standard 0,76[0,68;0,84]

0,54[0,42;0,67]

0,60[0,49;0,71]

156 II.1O meu estudo

rende mais quando trabalho

em grupo.

Standard+NR 0,64[0,56;0,72]

0,47[0,36;0,58]

N.A.

157 I.6O número de

alunos por turma no 1º ano da

FMUP é excessivo.

Standard+NR+Neutral

0,47[0,39;0,55]

0,30[0,20;0,41]

N.A.

STANDARD SCALE VS. NEUTRAL OPTION SCALE

Introduction

Objective

Methods

Results

Conclusion

Valid n

QuestionAnswers’ Options

n(%)Standart

Scale

n(%)Standard Scale

+ Neutral Option

Proportion of Agreement[Confidence

Interval]

155

I.4. É justo ser

necessário ter

frequência a uma

disciplina (frequentar

pelo menos 2/3) das

aulas para ser admitido

a exame teórico.

Discordo Totalmente

15 (9,7) 12 (7,7) 0.29 [0,12;0,52]

Discordo 37 (23,9) 33 (21,3) 0.37 [0,24;0,52]Não Concordo nem Discordo

------- 33 (21,3) -------

Concordo 79 (51,0) 57 (36,8) 0,37 [0,28;0,48]

Concordo Totalmente

24 (15,5) 20 (12,9) 0,26 [0,13;0,44]

156

I.1

O sistema de avaliação

contínua é vantajoso

para os alunos.

Discordo Totalmente

3 (1,9) 2 (1,3) 0 [0;0,54]

Discordo 25 (16,0) 30 (19,2) 0,41 [0,26; 0,58]Não Concordo nem Discordo

------- 31 (19,9) -------

Concordo 104 (66,7) 73 (46,8) 0,57 [0,47;0,66]Concordo

Totalmente24 (15,4)

20 (12,8) 0,38 [0,22;0,56]

155

I.3 Penso que o grau de

exigência da FMUP é

superior ao de outras.

Discordo Totalmente

0 (0) 1 (0,6) 0 [0;0,94]

Discordo 7 (4,5) 3 (1,9) 0,43 [0,12;0,80]Não Concordo nem Discordo

-------- 12 (7,7) -------


Totalmente102 (65,8) 82 (52,9) 0,64 [0,55;0,73]

STANDARD SCALE VS. REFUSAL TO ANSWER OPTION SCALE

Introduction

Objective

Methods

Results

Conclusion

Validn


n(%)Standard

Scale

n(%)Standard Scale

+ Refusal to Answer Option

Proportion of Agreement [Confidence

Interval]

158

I.5 Os professores do

1º ano da FMUP estão

sempre disponíveis

para tirar dúvidas

Discordo Totalmente

1 (0,6) 0 (0,0) 0 [0;0]

Discordo 28 (18,1) 30 (19,4) 0,35 [0,21;0,51]

Concordo 113 (72,9) 87 (56,1) 0,54 [0,45;0,63]

Concordo Totalmente

13 (8,4) 29 (18,7) 0,14 [0,05;0,30]

Não respondo ------- 9 (5,8) -------

158

I.7 Quando há

avaliação contínua,

acho antipedagógico

os alunos não irem

tendo ao longo do

semestre feedback do

seu desempenho

Discordo Totalmente

0 (0,0) 3 (1,9) 0 [0;0]

Discordo 12 (7,6) 11 (7,0) 0,15 [0,04;0,39]


Totalmente55 (35,0) 64 (40,8) 0,49 [0,38;0,60]

Não respondo ------- 9 (5,7) -------

158

II.3 Tenho necessidade

de recorrer a bebidas

energéticas e/ou

vitaminas para

acompanhar o ritmo

de estudo exigido

pelo curso.

Discordo Totalmente

70 (44,3) 71 (44,9) 0,52 [0,41;0,62]

Discordo 57 (37,1) 50 (31,6) 0,35 [0,25;0,47]

Concordo 25 (15,8) 22 (13,9) 0,31 [0,17;0,48]

Concordo Totalmente

6 (3,8) 11 (7,0) 0,31 [0,10;0,61]

Não respondo ------- 4 (2,5)

-------

STANDARD SCALE VS. BOTH OPTIONS COMBINED SCALE

Introduction

Objective

Methods

Results

Conclusion

Validn


N(%)Standard Scale

N(%)Standard Scale + Both Combined Options Scale

Proportion of Agreement [Confidence

Interval]

158

II.2 O estudo é mais

proveitoso quando

trabalho num local

silencioso.

Discordo Totalmente

0 (0,0) 0 (0,0) 0,00 [0,00;0,00]


------- 11 (7,0) -------


Totalmente89 (56,3) 98 (62,0) 0,67 [0,57;0,75]

Não Respondo ------- 0 (0,0) -------

158

II.4 Frequentemente acho mais rentável

estudar pelos apontamentos de

colegas mais velhos do que pela bibliografia

recomendada.

Discordo Totalmente

0 (0,0) 0 (0,0) 0,00 [0,00;0,00]

Discordo 13 (8,2) 8 (5,1) 0,31[0,12;0,59]Não Concordo nem Discordo

------- 11 (7,0) -------

Concordo 81 (51,3) 56 (65,4) 0.44 [0,34;0,55]Concordo

Totalmente64 (40,5) 73 (46,2) 0,57 [0,46;0,68]

Não Respondo ------- 2 (1,3) -------

158

II.5 É difícil conciliar as actividades extra-

curriculares (por exemplo praticar um desporto, aprender

um instrumento musical, etc…) com o trabalho exigido na

FMUP

Discordo Totalmente

8 (5,1) 3 (1,9)0,10 [0,01;0,46]


------- 22 (13,9) -------


Totalmente37 (23,4) 47 (29,7) 0,33 [0,22;0,46]

Não Respondo ------- 2 (1,3) -------

Conclusion

Introduction

Objective

Methods

Results

Conclusion

Freedom of choice

It depends…

VS Reproducibility


F A C U L D A D E D E M E D I C I N A D A U N I V E R S I D A D E D O P O R T O

2 º S E M E S T R E M A R Ç O D E 2 0 1 2

Abel Trzan

Ana Fonseca

Ana Filipa Ferreira

Ana Isabel Maduro

Catarina Tinoco

Guilherme Amorim

Joana Antunes

João Dias

José Carneiro

Maria João Bastos

Maria João Costa

Pedro Barros

Pedro Santos


Documents

VARIABILITY OF LIKERT SCALES EFFECTS ON DATA COLLECTION RESULTS CLASS 7 – 2011/2012 MIMED - FMUP