Quality Assurance at the European Monitoring Centre for Drugs and Drug Addiction

Sandrine SleimanEuropean Conference on Quality in Official Statistics, Rome8-11 July 2008

Contents

• Scope of EMCDDA activities

• Quality assurance: operating framework

• Challenges and results

Who we are

• A decentralised EU agency

• Formally established in 1993

• Based in Lisbon, Portugal (operating since 1995)

• The hub of drug-related information in the EU

Guiding principles and priorities: 2007-2009

Principles

• Scientific excellence• Partnership • Good governance and

efficiency

Priorities

• Consolidating monitoring and reporting activities

• Enhancing data analysis• Communicating more

effectively

Collecting and analysing existing data

Areas:

• Epidemiology — drug situation, monitoring new trends, five key indicators

• Interventions — health, social and criminal justice responses, best practice in demand and supply reduction

• Legislation — national and Community strategies and policies

• Policy — political and institutional framework, coordination

Five Key indicators • Drug use in the general population (students and young adults):

prevalence and patterns of drug use in the general population

• Drug users accessing treatment (TDI)

• Drug related infectious diseases (HIV, HCV, HBV)

• Drug related deaths and mortality of drug users

• Problem drug use: prevalence and patterns of problem drug use

Requires commitment from MS to invest in data collection on a routine basis

Type of data collected

• Annual registry and epidemiological data

• Descriptive and qualitative data (non-annual)

• Data from research studies

• Institutional data

• Database cases

• Coverage and availability of prevention, treatment and harm reduction

• Statistics from law enforcement sources

Collecting and analysing existing data

• Reitox national focal points (NFPs), national experts

• Annual reporting to EMCDDA (national reports, standard tables, questionnaires)

• Expert groups for developing indicators (nominated by NFPs)

• Ad hoc working groups

Reporting system: standard templates

• National reports and selected issues (based on detailed guidelines)

• Standard Tables (ST) for quantitative information

• Sturctured Questionnaires (SQ) for qualitative information supplementing ST and NR

Quality Management at the EMCDDA

Data collectioninstruments

Quality overviews

Quality Control

by Scientific

Units

Quality assurance at EMCDDA level

Data Validation

Data collection instruments, protocols and guidelines

National Focal Points

National Focal Points

&

National data providers

Data submission in FONTE, including quality control at

national level

National data

providers

Quality Assurance

by QA team

Main quality related outputs & reports

Annual Quality reports

• Implementation Needs Profiles on the 5 key indicators

• EU experts groups ad-hoc documents

• Regular reporting to EC on implementation of 5 KI and on quality of reporting

Annual Quality reports

Quality reports includes feedback on :

Quality criteria :

Completeness, insight, reliability, usefulness,

internal consistency

National reports

1. Global evaluation

2. Method. Aspects

3. By section : level of details, clarity, strong/ weak points, suggestions for improvements, discrepencies

4. Compliance with formal r equirements

5. Layout presentation incl. deadline

Standard tables C1 C2 C3 C4 M1 M2 M3 D1 + Number of tables uploaded; Date of 1 st upload; Number of uploads.

Structured questionnaires - Level of details (insuf., suff., good) - Conciseness (insuf., suff. , good) - Discrepencies with NR

Criteria for National reports• Completeness: the report contains all the necessary and existent

information in order to provide an overview of the situation

• Insight: the report includes complete and significant information, giving an interpretation to the reported information, according to social and political contexts.

• Reliability: the extent to which the information in the report allows comparisons (between different time periods).

• Usefulness: information oriented to the targets; acceptable and pertinent to the report objectives; no redundant information is presented.

• Internal consistency: the extent to have coherent information in the report or to describe the reasons for a lack of internal consistency.

Criteria for data checking in STs

CNDA: Not clear whether all data in the ST are new or not

CQUA: Quantitative information needed clarification

CQUAL: Qualitative information needed clarification

CREF: References needed clarification

DNR: Discrepancy between NR and ST

MDA: Quantitative information missing

MME: Methodological information missing

MRE: References missing

OTH: Others

Criteria for SQ

Level of details

Insufficient, sufficient, good

Conciseness

Insufficient, sufficient, good

Discrepencies with NR

Yes - No

General comment

Any other important feedback

Implementation Needs Profiles

• Concerns 5 key epidemiological indicators

• Focuses on activities related to indicators and partially on content

• Is a general analysis, helps to identify problematic areas, status quo or improvement in data collection

Criteria used

• Most recent data

• Reference

• Description

• Weaknesses

• Activities implemented or ongoing/planned

• Actions needed

Quality assurance: work in progress

Quality Assurance…

• is mentioned as a priority in the EU action plan• is mentioned in the external evaluation report• is part of the EMCDDA’s 3-year work programme

Quality documents should be …

• considered as real tools for the EMCDDA and the REITOX network (as well as for external actors)

• aimed at scientific excellence and not administrative purposes • an overview of strengths and weaknesses of data or collection

processes • …

Quality assurance: work in progress

• Production of 5 KI methodological packages • Definition of precise assessment criteria for 5 KI

based on :

• minimum implementation standards• specific to each indicator• sensitive to national contexts• somewhere between pragmatism and perfection

• Redefinition of templates and guidelines with adaptation of quality criteria

Quality assurance : work in progressUnderstand needs and capacity to

deliver from the Reitox network

Priorities for value delivery: revision of reporting guidelines

Promise what can be delivered

Change capability to match promise

Deliveries matching

standards and priorities

Change capability to match future

needs:

- Quality feedbacks

- Capacity development

activities

Case study: national reportsSuccess

• Recommendations for improvement are generally applied as regards adherence to guidelines:

- Document layout - References to standard tables and structured questionnaires- Sources and use of Harvard for bibliography- Methodological information is more precise- Efforts in writing the National report

Case study: national reports Gaps

• Deadlines missed• Conciseness• Lack of insight on trends and interpretation of results • In some countries, it is still too early to have

information on trends• Work on quality assurance depends on external

factors, such as Member States’ involvement in quality issues

THANK YOU FOR YOUR ATTENTION

Sandrine.Sleiman@emcdda.europa.eu

http://www.emcdda.europa.eu