Surveys Dr Carol Taylor CPD and Post-Graduate Studies Division School of Health, Psychology and...

Surveys

Dr Carol TaylorCPD and Post-Graduate Studies DivisionSchool of Health, Psychology and Social

CareManchester Metropolitan University

November 2007

Introduction… What are surveys and what are they

not? How is survey data collected? What kinds of data do they generate? How do we analyse it? Why do we use them? What kinds of information do they

provide?

What is a survey? Dictionary definition:

‘Survey’ – view or examination of land or population to examine its features.

Who or what may be surveyed?

Population may be human or non-human trees, animals, Buildings documents, records

Human surveys…

Whole populations National Census

Subgroups… for example:

Children Unemployed Users of a particular product or service Groups of professionals Particular ethnic groups Working mothers

What kind of information do surveys provide?

Descriptive Comparative Explanatory

For knowledge, attitudes and opinions, or behaviour

Studies themselves may be either descriptive or experimental

Demographics Factual personal information about the

individual

For example: questions asking for age, gender, ethnicity, postcode, job title, etc

Behaviours

Self –reports of behaviours Smoking habits Spending habits Leisure pursuits

May be observable, may or may not be “accurate”

Often asked for a scale of frequency of behaviour E.g Frequently, sometimes, rarely, never

Attitudes, beliefs, perceptions

Use of attitude scales E.g: “Strongly agree, partially agree, no view, partially

disagree, strongly disagree”

May be asked to rank list of attitude statements

Use of ‘open – ended’ questions “Why do you feel that way?” “In your view, how might the service be improved”?

Types of studies…

Descriptive studies –

Cross-sectional

Cohort

Case control

Types of studies… Experimental – testing hypotheses

Examine associations

e.g. between lifestyle and health; education and achievement

Examine change - before and after an intervention

e.g. attitudes to back pain after attending a back rehab programme

What are they for? Primarily for information gathering:

Planning (town, workforce, energy)

Examining behaviours (spending, lifestyle)

Attitudes, beliefs, perceptions, opinions (opinion poles, satisfaction surveys)

Screening (Sampling, Identifying a niche market)

Who carries out surveys?For example:Local and National Authorities

TV networks

Commercial companies

Service providers: e.g. - transport authorities, health professionals, educators

What surveys don’t do… Give precise objective measurements.

Establish causal relationships between variables, though may suggest links.

Provide rich qualitative data

Where do surveys sit in the world of research methods?

Research paradigms

Qualitative research e.g. ethnography

Quantitative research e.g. Double blind RCTs

X

Surveys

How is the information (data) collected?

Questionnaire

Interviews Examination Observation

Postal

Telephone

Internet

Email

How is the information (data) collected?

Interviews

Examination – eg documents

Observation - behaviour

individual

focus groups

Face to face

Telephone

What can we do with the data?

Survey data is quantitative.

Descriptive statistics – univariate analysis

Inferential statistics – bivariate / multivariate analysis - Suggest relationships in the data

Statistical significance – e.g. Chi square test

Its data is descriptive but may be examined for differences which have statistical significance.

What next…?

Data generation….

…and analysis

Sampling? Representative sample

Purposive sample

Sample size

Types of questions Closed ended

Factual data Fixed choice Likert scales

Open ended

Closed-ended: Factual data….

Demographics

e.g. Age, occupation, type of dwelling, income

Contributes to examining populations’ living and working habits for example.

Fixed choice…Which of the following newspapers do you

read?:The TimesThe DailyTelegraphThe IndependentThe Daily MailThe SunManchester Evening NewsOtherIf other, please state………………………………..

Likert Scales The doctor listened to you attentively

Strongly disagree Partially disagreeNeither agree nor disagreePartially agreeStrongly agree

How often do you question the doctor’s advice?NeverRarelySometimesFrequentlyAlways

Types of questions Closed ended

Factual data Fixed choice Likert scales

Open ended No predetermined choices or responses Usually asking for opinions, beliefs, attitudes etc. Respondent free to provide their own responses to

the question. This does not include “any other comments?”

Open ended questions…

What are your views on the Government’s ban on smoking in pubs?

Response rates Relevance

Timeliness

Convenience

Analysing the data Closed ended questions

Open ended questions

Closed - ended questions

Levels of measurement Nominal Ordinal Interval (Ratio)

Most commonly nominal and ordinal data

Analysing the dataNominal data Univariate analysis - the proportion of

responses in each category Methods: charts (e.g. bar chart, pie chart) or frequency table.

Presented as descriptive statistics.

May use multiway tables and tests of statistical significance (e.g.Chi Square) to measure associations between variables

Analysing the data

Multiway tables influenced by: The number of variables being examined

Bivariate = 2 variables (e.g. gender + purchase of computers) Methods: cross-tabulations (2 x 2 tables) scatterplots; regression; comparison of means

Support for going into Iraq?

% Strongly approve 159 3.8 Approve 125030.1 33.9 positive

Disapprove 141133.9 Strongly disapprove 124429.9 63.8negative

DK 93 2.2 Total 4157100 (99.9)

Is there a gender effect? approve disapprove

total Men 765 (42%)1045 (58%) 100%

1810 Women 642 (29%)1608 (71%)

100% 2250

13 -13

Analysing the data Multivariate = 3 or more variables

e.g. income influenced by both education level and gender.

Therefore assuming we have a representative sample to begin with, we can infer something about the population as a whole.

Analysing the data Ordinal data e.g.Likert Scales

Tend to convert responses to numerical codes e.g. Completely disagree = 1; Completely agree = 5

These are codes only but some order can be assumed.

Modes and central tendencies may be useful but means are not.

Data analysis

0

5

10

15

20

25

30

35

40

S/ D D N A S/ A

Qu1Qu2Qu3

Responses to 3 five-point Likert scale questions

Open ended questions

Qualitative information which is examined for themes.

Usually do content analysis presenting themes as frequency tables.

What kind of data do we get?

Data is presented numerically from both open and closed questions.

Questionnaires are therefore quantitative in nature even though they may ask for qualitative information.

Ethical considerations Survey participants deserve all the

same rights as participants in other types of study. Research governance applies!

Ethical approval from a REC not necessary unless the participants are NHS patients and /or from a recognised vulnerable group

Advantages of questionnaires

Access to large numbers in wide geographical area

Relatively easy / quick to analyse

Large volume of descriptive information

Relatively cheap to administer

Disadvantages of questionnaires Difficult to construct well

Information may lack depth

Information may not be accurate (memory, perceptions etc)

Commonly poor response rates

Delphi technique Uses questionnaires to gain a consensus on

a particular issue

Useful where there is little or no convincing research evidence on a specific topic

Barker K and Burns M (2001) Using consensus techniques to produce clinical guidelines for patients treated with the Ilizarov fixator. Physiotherapy Vol 87 No 6 pp 289-297

Key Points…. Surveys examine characteristics of

populations Surveys are a quantitative method

of research Results are not precise objective

measures but can suggest links between variables