"Quality Issues: Collection, Coding,

Metadata- some NHS

experiences"Philip Johnston

NHS Scotland, Information Services Division

Quality Issues: Collection, Coding, Metadata- some NHS experiences

• about Information Services Division (ISD)

• about ISD data

• ISD processes, and some examples

• some statistics on quality

about ISD

• part of National Services Scotland, and NHS

• 40+ years old

• around 650 staff across Scotland

• Scotland's national organisation for health information, statistics and IT services

• 17 work programmes – eg waiting times, mental health

about ISD data

• around 60 data sets, evolving

• variety of data providers – eg: hospitals, NHS boards, GP practices, voluntary sector, private sector

• per year: ~60 publications, ~3000 information

requests, ~500 Parliamentary Questions

• Part of National Statistics

• “Scotland has some of the best health service data in the world”

ISD data : some context

• some uses: clinical governance, planning, epidemiology, performance management, setting policy

• some stakeholders: NHSScotland, Scottish Executive, Scottish Parliament, local authorities, Audit Scotland, GRO, researchers, media, public, political parties

• some influences: health policy, devolution, National Statistics, Freedom of Information, data protection, patient involvement, IT developments (eg web), media awareness

ISD data : SMR01, an example dataset

• patient/episode -based

• acute inpatients / day cases

• ~100 data items, including clinical codes (ICD, OPCS)

• ~1.2m records/year

SMR01 data collection

• electronic, secure (encrypted)

• varying timetables for submission

• data for ~1000 locations, from ~17 IT systems

• ~100 validation files, with ~3m records

• ~1500 error/query checks

SMR01 quality : accuracy

Main condition accuracy at 3-digit level

0 10 20 30 40 50 60 70 80 90 100

Royal Aberdeen Children's Hospital

Royal Hospital for Sick Children, Yorkhill

Royal Hospital for Sick Children, Edinburgh

CHILDREN'S HOSPITALS

Lorn & Islands District General Hospital

Dr Gray's Hospital, Elgin

Vale of Leven District General Hospital

Stracathro Hospital

SMALL GENERAL HOSPITALS

Dumfries & Galloway Royal Infirmary

Royal Alexandra Hospital

Southern General Hospital

Monklands Hospital

Hairmyres Hospital

Stirling Royal Infirmary

Wishaw General Hospital

Falkirk and District Royal Infirmary

The Ayr Hospital

Perth Royal Infirmary

Stobhill Hospital

Borders General Hospital

Raigmore, Caithness & Belford Hospitals

Inverclyde Royal Hospital

Queen Margaret Hospital, Fife

Victoria Infirmary, Glasgow

Victoria Hospital, Fife

St J ohn's Hospital at Howden

Crosshouse Hospital

LARGE GENERAL HOSPITALS

Ninewells Hospital

Western General Hospital, Edinburgh

Aberdeen Royal Infirmary & Woodend Hospital

Glasgow Royal Infirmary & Canniesburn Hospital

Royal Infirmary of Edinburgh, City & PMR Ortho. Hospitals

Western Infirmary/Gartnavel Gen & Beatson Hospitals

TEACHING HOSPITALS

Scotland

Percentage accuracy (target 90%)

• Accuracy of coding of main condition, 2000-02, by hospital

SMR01 quality : timeliness

Timeliness of SMR01 SubmissionsScotland

1997 - 2005

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Month of Discharge

% submitted in 6 mths % submitted in 3 mths % submitted in 2 mths % submitted in 1 mth 6 Month Moving Average**

SMR01 quality : other measures

• validity• completeness• fitness for purpose• relevance• coherence• comparability• data ‘sign off’

SMR01 quality – some factors

• conflicting local/national priorities

• IT system issues (eg purpose / specification / delivery)

• timeliness vs accuracy

• quality of source data (eg case notes)

• training and definitions

• changes in service delivery

SMR01 : example of accuracy versus timeliness

SMR01 : some metadata

• definition?

• Data Dictionary (web-based; ~1500 items), including definitions and recording manuals

• statistics on quality of data

• data audit controls• Change Control process

• ‘health warnings’ on outputs• statistical imputation / suppression

other data collection : some examples

• census of Allied Health Professionals (non-NHSnet; developmental)

• Scottish Birth Record (national repository)

• the future, eg: clinical datasets, electronic health record, SNOMED, data warehouses

Quality Issues: Collection, Coding, Metadata- some NHS experiences – in summary

• range of complex, interdependent factors affect data quality

• ISD applies considerable resource to data quality

• ISD ability to measure data quality, and to understand influences, is essential and is much valued by users