Questionnaire

Questions Structure

Introduction

Satisfaction of end-users about the

quality of water and wastewater services

A methodology

J. Rosa*, D. Alves**, Á. Carvalho***

* The Water and Waste Services Regulation Authority, rosa.joaofilipe@gmail.com

**The Water and Waste Services Regulation Authority, david.alves@ersar.pt

*** The Water and Waste Services Regulation Authority, alvaro.carvalho@ersar.pt

Since 2004, The Water and Waste Services Regulation Authority (ERSAR) has been regulating the quality of service through the implementation of a system of performance indicators, by comparing

and publicly disclosing the results achieved by the operators, thus producing an objective measure of benchmarking that pushes operators to improve efficiency and the quality of service. But what

are the views of the end-users? Do they have the same perception?

Analysis of user satisfaction and perception of the services provided by water and waste utilities is an important tool for an accurate regulatory assessment of these services. ERSAR has the legal

obligation of “measuring, assessing and publicly disclosing the satisfaction levels of the users of (…) water and waste services”. Therefore, ERSAR decided to develop a pilot study that, when

implemented, should allow for an understanding of the global satisfaction and perception levels of end-users in water and waste services and to bring forth latent problems in this sector.

This paper presents a methodology developed in that pilot study. It discusses the main problems faced in the process and stresses the solutions found for a sampling plan and for a questionnaire to

be addressed at the users of water services.

Study’s scope

inspiring change

Conclusions

www.iwahq.org

This pilot study is a thorough methodological starting point for future assessments. It allows for a nationwide analysis of the satisfaction of end-users of water services while avoiding expensive and

time consuming analysis of all municipalities in the country. It is also a useful tool for small scope comparative analysis of a few municipalities. In the future it is expectable that the analysis is spread

over to more municipalities in order to improve the benchmarking exercise at regional level.

Target population

▪ Over 18 years old;

▪ Drinking water, sewerage and waste management customers;

▪ Being customer for more than 1 year of service.

Collecting responses

▪ Computer Aided Telephone Interview (CAPI).

Territorial’s scope

▪ National approach [Fig. 1, (a)] - 1068 interviews

▪ Local approach [Fig.1, (b)] - 1176

Figure 1 – Sampling plan (national and local approaches)

(a) National approach (b) Local approach

The sampling plan shows the best method for the collection of a representative sample of the population in the most efficient way. It is necessarily different on nationwide and municipal approaches.

The national analysis is based on a sample of 1068 interviews in total, regionally stratified and thus accounting for the specific weight of the population in each region. The municipal analysis is

based on a random sampling of 1176 interviews, for the municipalities of Lisbon, Oporto, Coimbra, Moura and Portimão. This way, the study will be covering a multiple range of operator profiles in

what concerns geographical position and dimension, population served and seasonal demographic variations, urban/rural typologies and management models (direct, delegation and concession

management models).

Finally, this sampling methodology sets out the baseline for future analysis of water and waste users satisfaction and perception, in which all 278 municipalities of mainland Portugal will be included.

Single measurement scale

with 7 evaluation levels

Length 12 minutes

3 blocks of questions

3 blocks for each service, allowing the

interviewer to choose randomly the order of

the services that start the questionnaires and

thus mitigating the possible length effect on

the last block if the structure were a fixed

one.

Less than 15 minutes,

which is the maximum

length recommended in

this type of studies.

Using a mixed measurement scale could be

confusing for interviewees.

7 levels of choice isn’t hard enough to take a clear

response and to appraise a positive or negative

evaluation (odd scale).

General satisfaction

Perceived price and perceived

value

Awareness about the operator

Customer support

Complaints

Billing

Specific topics like satisfaction with

drinking water quality, water pressure,

cutting of water supply and time spent

by the operator solving cuts/breakdowns

in water supply are taken into account.

Common topics address by this

questionnaire: the perceived price and the

perceived value of the services,

awareness about the operator, perceived

quality of the customer service, handling

of complaints and billing.

The questions address general and specific topics and are based on an analysis of other

questionnaires, consumer complaints and regulatory information about the quality of service.

The questionnaire ends with the characterization of the interviewees in terms of gender, age,

education, working situation and income level.