LGI NCHDA Report Sept 2016

The National Congenital Heart Disease Audit

Data Quality Audit for CONGENITAL HEART DISEASE PROCEDURES

The Leeds Teaching Hospitals NHS Trust

4 August 2016

performed by Lin Denne and Dr R Rehman,

1

LGI NCHDA Report Sept 2016

SummaryThe data return to the NCHDA made by the Congenital Cardiac Department of the Leeds

Teaching Hospitals NHS Trust (LGI) and analysed 4 weeks prior to this visit, indicated that

some 1273 procedures had been undertaken during the data collection year of 2015/2016.

This is the first validation of the NCHDA Dataset v5.01 at LGI.

Since June 2013 a dedicated congenital Database Manager (DBM) has been in post.

Connection to the NCHDA database for this role was established in December 2014. There

is a nominated clinician with responsibility for this data and another who also had access to

the NCHDA database.

As previously reported, the Congenital Cardiac Department at LGI uses a bespoke database

(OSCAR 4D) and this is available at secretaries’ and clinicians’ desks within the Department

and in the operating theatre where most congenital surgery is performed. There is an

interface between OSCAR and the Trust PAS.

Actions undertaken or changes to processes since the September 2015 validation visit.LGI Report the following actions:

1. The Perfusionists’ records are now routinely used as a reference point for

appropriate data and these were seen in the case notes at this visit.

2. The DBM has observed external visits to other congenital centres with the NCHDA

clinical auditor.

3. Monthly validation (IG) meetings take place with the clinical team and the DBM so

that all of data are checked and signed off. This not only improves the quality of the

data but also allows each individual to take ownership of their data. Training is

available and discussion of data is now built into departmental clinical governance

meetings

4. LGI report that here has been a greater engagement of the clinical staff to become

more responsible for their own data.

5. The system used for collecting the LGI data (4D) was promptly updated for the start

of the new NCHDA data set (v5.01) in April 2015.

6. It is reported that there have been some slight concerns around implementation of

modifications required of the database to allow improved data quality - for example

2

LGI NCHDA Report Sept 2016ability to delete duplicate records at a local level. (This has now been resolved and

the DBM has full editing rights.)

7. The appropriate people now have access to Lotus notes.

Consent for External Validation of Notes.Patients have been required to give their consent to allow external validation of their hospital

notes since 1 April 2007. This process was established in 2000 at Leeds Teaching

Hospitals NHS Trust using a separate paper form. As previously reported, immediately prior

to the November 2013 validation, LGI designed and produced their own A5 size sticky label

to indicate on the inside of the patients notes whether or not consent for external validation

has been gathered.

LGI are moving towards an electronic patient record (ePR) and methods of capturing this

piece of information electronically is currently being discussed and reviewed.

Data Quality Indicator ScoreThe overall DQI for the Trust (with the previous years in parentheses) is calculated to be

97.75% (97 96.75, 94.75) with domain scores Demographics .99 (1.0 1.0, .88) Pre .98

Procedure (.92 .93 .97) Procedure .96 (.99 .97 .99) and Outcome .98 (.97 .97 .95 .88).

This is a further 0.75% increase since the previous NCHDA visit which is an excellent

achievement in a year when it was anticipated that DQI would fall slightly as centres

adjusted to a 30% increased dataset. There were just 26 errors in 1001 variables. 12 of

the errors related to some of the newer fields for implantable devices details such as device

name, manufacturer and serial numbers, (part of the Procedure Performed domain).

3

LGI NCHDA Report Sept 2016Individual DQI for Surgery and for Catheters

Data Year Validated

Surgery DQI Catheter DQI

2009 07/08 88.25% 96.25%

2010(i) 08/09 84.0% 96.0%

2010(ii) 09/10 88% 98.75%

2012 10/11 95% 96.25%

2013 11/12 92% 95.75%

2013(ii) 12/13 94.25% 96%

2014 13/14 95.25% 99%

2015 14/15 97.25% 96%

2016 15/16 98.5% 97.25%

The body of this report is drawn from answers given on the NCHDA pre visit Questionnaire,

and from discussions on the day of the visit.

IntroductionPrior to this validation visit, the NCHDA Data Return from the Paediatric Cardiac Department

of the Leeds Teaching Hospitals NHS Trust indicated that 1273 (496 surgery, 466 catheters,

311 others, 21 deaths) procedures had been undertaken during the data collection year of

2015/2016 of which 20 cases were selected for review.

The NCHDA Data Auditor and one external Senior Trainee (ST6) in congenital cardiology

undertook the site audit. Also in attendance and observing the validation visit was a

Database Manager from another congenital centre.

The Congenital Database Manager from LGI in collaboration with the nominated clinician for

responsibility of these data and others completed the previsit questionnaire. Information on

actions undertaken or changes to the local processes are mentioned above.

As previously reported and as stated above, the Department uses its own database to

collect data (the Orion Software for Cardiology – OSCAR 4D). This database is connected

to the hospital PAS. Access to this database is available throughout the department

including the catheter labs and operating theatre where most congenital cardiac surgical

4

LGI NCHDA Report Sept 2016procedures are undertaken. The consultants and their secretaries have access at their

desks and input data. From the data that are input, a discharge summary is generated at

time of discharge.

There is a detailed process (Standard Operating Protocol) for auditing data internally and

reverse validating it once submitted to the NCHDA.

The Validation Team are extremely grateful to the Database Manager and the clinical team

who organised and itemised many of the items in the case notes that the Review Team

would need look at. This has been meticulously prepared.

A sample of 20 records with a reserve list of a further 10 was supplied prior to this validation.

On the day 18 records were made available from the sample and 2 records were used from

the reserve list. All 20 records either had a signed consent form, consent label or a verbal

consent noted for external validation of the related hospital notes. The Reviewers are also

extremely grateful to the Chief Medical Offers for Leeds Teaching Hospitals NHS Foundation

Trust for giving permission to view any hospital notes where it was unclear if there was

informed patient/parent/guardian consent and it had not been possible (or inappropriate in

the case of a deceased patient) to contact the family.

The accuracy of the NCHDA data return was checked against each set of notes. This was

then recorded to enable the Data Quality Indicator (DQI) to be scored.

Review of notes As described above, each set of notes was meticulously prepared with sticky post-it type

labels identifying many of the pages the Validation Team needed to review.

1. As previously reported, the smaller slimmer case notes were very tidy and the filing

was in very good order.

2. The pink operation note, when seen was very helpful in establishing exactly what

procedure had been performed.

3. Also as previously documented, perfusion sheets were seen in the case notes of by-

pass patients.

4. Documentation of exactly when (date and time) a patient was extubated was at times

difficult to find.

5

LGI NCHDA Report Sept 20165. In some case notes there appeared to be a discrepancy between the electronic

record and the paper records as to the exact date a patient was discharged from

hospital.

Review of the Log BooksCardiac Operating TheatresThe bespoke bound operating theatre ledgers for Theatres M, 8, and 9  were made

available.   The location of the log book for theatre 10 was unknown until the very end of the

day and Reviewers were unable to review this book due to time pressures.  Each entry of

the log books seen is hand written.  As previously noted it is not always clear whether or not

a procedure is for congenital heart disease.

1. 2 submitted surgical records appear to have errors in them

2. 4  records were identified from the log book that may have been missed from the

congenital submission

3. 1 surgical procedure had been submitted incorrectly in the category Other rather than

Bypass or Non Bypass

4. 94 submitted records were not validated in the log books that were seen.  This may

be because it was not possible to discern from the handwritten entries if the

operation was for congenital heart disease, or that the procedure took place in

another operating theatre that the reviewers did not see the log book for.

Cardiac Catheter Lab Log Book ReviewThere are 6 cath labs at this Centre.  The Validation Team were informed that most

congenital procedures are performed in Lab 1 and Lab 5.  The individual log books for each

of these two cath labs were reviewed.   These books are A4 lined and ruled books.  The

remaining books were made available but were unable to review them due to time

pressures.  As previously reported, it was quite difficult to identify whether or not a procedure

is for congenital heart disease.  The findings are;

1. 3 submitted catheter records appear to have errors in them

2. 5 procedures were identified in the cath lab log books which may have been missed

from the data submission.

3. 52 submitted catheter procedures were not validated in the log books.  This may be

because it was not possible to discern if the procedure was for congenital heart

6

LGI NCHDA Report Sept 2016disease, or that the procedure took place in another cathlab that the reviewers did

not see the log book for.

4. 2 records appear to be for non-congenital heart disease and these should be

removed from the NCHDA

5. 1 submitted record appears to be a duplicate

7

LGI NCHDA Report Sept 2016

Validation of Deceased Patients Diagnostic and Procedure Coding

Commencing with the validation of the 2013/14 data in 2014, the National Congenital Heart

Disease Audit wish to verify any dates of death of deceased patients included in the year

under review. The diagnosis and procedure coding will also be validated. The requirement

for consent to validate these hospital data are the same as for the congenital procedures.

Where there is no evidence that consent has been given the Medical Director is asked to

give permission for the case note examination. The Validation Team are grateful to Dr

Yvette Oade (Chief Medical Officer for LGI) for facilitating this.

21 patients who had had procedures during  the 2015/16 data collection year were noted to

have died.   The procedural and outcome documentation was made available to the

Reviewers. 

2 records appeared to have an incorrect date of death on the local PAS when

compared to the Death Certificate

In 2 further records it was not possible to verify the dates of death against any other

source than the local PAS.  These were out of hospital deaths and these data are

manually input from a report that is printed from the Summary Care Record.

1 record appears to have an incorrect previous procedure submitted to the NCHDA

1 record appears to have an incorrect weight submitted

5 records appear to have incomplete comorbidities recorded in the data submitted to

the NCHDA

2 records appear to require more specific procedure coding

8

LGI NCHDA Report Sept 2016Security and ConfidentialityAs at all the previous Validation Visits, the NCHDA system has been registered with the

Trusts Data Protection Officer and there are tested procedures to ensure data backup and

disaster recovery within the Trust. The NCHDA system is fully compliant with the Trust’s

policies on security and confidentiality.

There are written and printed Powerpoint presentations available to staff in all the areas

where staff collect and manage data. There are processes in place to audit data collection

activities. The Centre are confident that all patient data is consistently collected in almost all

instances and rely on internal and NCHDA external validation visits to feedback on data

completeness; in particular completeness pertaining to surgery

Validation and Quality AssuranceFormal validation routines are built into the OSCAR system which includes checking for

invalid entries and completeness of data items. These procedures are used when data are

input. There are checks for duplicates and NHS Numbers. The process for auditing data

collection was set up in early 2006 and was last reviewed and modified in April 2015.

There is ongoing review of data entry – cases are discussed weekly and also monthly for

NHSE CQINS dashboard returns including PRAiS software risk analysis. The DBM meets

with clinicians at the scheduling meetings and IG meetings. Due to the sensitivity of

congenital cardiac data generally, any records can only be deleted centrally on request to

NICOR on the NCHDA database. This is unchanged in 2016.

The Data Manager routinely checks the cath lab and theatre log books. As previously

reported, since April 2013 processes are in place to check TMS, PAS weekly. Cath log

books are checked monthly and missing cases input.

TrainingThere is someone in the Trust with central responsibility for the identification of training

needs. Within the cardiac department the congenital DBM undertakes some of the provision

of training in data collection pertinent to that specialty. Within the Trust there an annual

mandatory programme for each individual employee on the importance of patient

confidentiality and data security. A documented training programme has been developed

and covers all aspects of clinical activity on NCHDA and is being continually updated to meet

new demands. This is available to all staff both permanent and temporary relative to their

job role. This is unchanged in 2016. An SOP has been created for the administrative staff

9

LGI NCHDA Report Sept 2016which covers validation and errors. NCHDA classification tables are available in the training

package.

CommunicationsAs at all the previous visits, there are processes in place for responding to queries arising

from the NCHDA data and an established procedure for reissuing amended information

following changes to the data. There is a procedure to ensure timely collection and

dissemination of activity data within the organisation and to NCHDA. Variable life adjusted

displays (VLADs) are now prepared for monthly reviews and discussed as necessary.

AccountabilityAs previously reported, there is a nominated clinician and a dedicated Data Manager and

they are responsible for data quality and have influence over other staff whose actions

affect data quality.

Health Records ManagementAs previously reported, most but not all of the information required can be found in the

hospital notes. There is no log of septostomies or other procedures that may occur outside

the catheter laboratory.

TimelinessThe returns to NCHDA have mostly met the agreed deadlines and the Centre are now able

to generate internal reports. Submission to NCHDA is dependant on a single operator and a

nominated clinician. Due to workload there can be a delay. There has also been some

delays arising from the editing rights on the local OSCAR database not allowing deletion of

records that accidentally become duplicated. It is reported on the pre visit questionnaire for

the 2016 validation visit that the DBM can now delete duplicate records from the local

OSCSR database if required. At the time of this visit very little data has been received for

Q1 (Apr-Jun) 2016.

Completeness and ValidityAs at the 2009 visit there appeared to be internal targets for completeness of data and the

answers given on the Previsit Questionnaire 2016 indicate that these are being met.

Accuracy

10

LGI NCHDA Report Sept 2016There is a data quality and audit programme and checking of data items against source

documentation. Since 2007 there has been a monthly feedback of this activity to clinicians

that is now part of the Trust policy. Clinical staff are therefore involved in the validation of

diagnostic and procedure codes on a routine basis but the timeliness of this exercise was

variable. Since the autumn of 2013, a internal data quality report is issued to all stakeholders

on a weekly basis. This is unchanged in 2016.

11

LGI NCHDA Report Sept 2016

Case note Audit 2015/16 Data.20 patients underwent 28 procedures (15 caths, 13 operations)

Parameter Total Score

Total No

Comments Scores for Cardiology & Surgery

C S

1 Hospital Number 20 20 11 9

2 NHS Number 20 20 11 9

3 Surname 20 20 11 9

4 First Name 20 20 11 9

5 Sex 20 20 11 9

6 DOB 20 20 11 9

7 Ethnicity 18 20 2 incorrect 9/11 9

8 Patient Status 20 20 11 9

9 Postcode 20 20 11 9

10 Pre Procedure

Diagnosis

28 28 1 incomplete 15 13

11 Previous Procedures 31 34 3 absent 16/18 15/16

12 Patients Weight at

Operation

28 28 15 13

13 Height 26 26 13 13

14 Ante Natal Diagnosis 4 4 2 2

15 Pre Proc Seizures 28 28 15 13

16 Pre Proc NYHA 4 4 - 4

17 Pre Proc Smoker 4 4 - 4

18 Pre Proc Diabetes 4 4 - 4

19 Hx Pulmonary Dis 4 4 - 4

20 Pre Proc IHD 4 4 - 4

21 Comorbidity Present 12 12 5 12

22 Comorbid Conditions 18 21 3 absent 5/7 13/14

23 Pre Proc Systemic

Ventricular EF

28 28 15 13

24 Pre Proc Sub Pul

Ventricular EF

26 26 15 13

25 Pre-proc valve/septal

defect/ vessel size

- - 15 10

26 Consultant 28 28 15 13

12

LGI NCHDA Report Sept 2016

Parameter Total Score

Total No

Comments Scores for Cardiology & Surgery

C S

27 Date of Procedure 28 28 15 13

28 Time Start 28 28 15 13

29 Proc Urgency 28 28 15 13

30 Unplanned Proc 28 28 15 13

31 Single Operator 5 5 5 -

32 Operator 1 28 28 15 13

33 Operator 1 Grade 28 28 15 13

34 Operator 2 23 23 10 13

35 Operator 2 Grade 23 23 10 13

36 Procedure Type 28 28 15 13

37 Sternotomy

Sequence

13 13 - 13

38 Operation

Performed

28 28 15 13

39 Sizing balloon used

for septal defect

0 0 0 -

40 No of stents or coils 2 6 4 absent 2/6 -

41 Device

Manufacturer

13 15 2 absent 10/12 3

42 Device Model 15 17 2 absent 12/14 3

43 Device Ser No 13 17 4 absent 10/14 3

44 Device Size 11 17 3 absent, 1 incorrect 8/11 3

45 Total Bypass Time 13 13 - 13

46 XClamp Time, 10 10 - 10

47 Total Arrest 1 1 - 1

48 Cath Proc Time, 15 15 15 -

49 Cath Fluro Time, 13 13 13 -

50 Cath Fluro Dose, 13 13 13 -

13

LGI NCHDA Report Sept 2016Parameter Total

ScoreTotal No

Comments Scores for Cardiology & Surgery

C S

51 Duration of Post Op

Intubation

13 13 - 13

52 Post Procedure

Seizures

28 28 15 13

54 Post Proc

Complications

0 1 1 incorrect - 0/1

55 Date of Discharge 26 28 15 11/13

56 Date of Death 1 1 1

57 Status at Discharge 28 28 15 13

58 Discharge

Destination

28 28 15 13

14

LGI NCHDA Report Sept 2016Data Quality Indicator Assessment:

The Overall Trust DQI = 97.75% Cardiology DQI = 97.25% Surgery DQI = 98.5%

This DQI is based upon the domain scoring below. The methodology for this DQI is provided in the paper The CCAD Audit –

An Introduction to the Process.

DOMAIN DOMAINScore

Demographics

Hospital Number, NHS Number, Surname, First Name, DOB, Sex, Ethnicity, Postcode, Patient

Status,

Overall .99

Card.98

Surg1.0

Pre Procedure

Pre procedure Diagnosis, Selected Previous Procedures, Patient Weight at Operation,

Consultant, Antenatal Diagnosis, Pre Procedure Seizures, Comorbid Conditions,

Height, Pre Procedure NYHA, Pre Procedure Smoker, Pre Procedure Diabetes, Previous Pulmonary Disease, Pre Procedure Ischaemic Heart Disease, Comorbidity Present, Pre Procedure Systemic Ventricular Ejection Fraction, Pre Procedure Sub Pulmonary Ejection Fraction, Pre Procedure valve/septal defect/vessel size,

Note, the scores for his domain are affected by the selected previous procedure and pre procedure

diagnosis

Overall .98

Card

.97

Surg

.99

Procedure

Date of procedure, Operator 1, Operator 2 Cardiopulmonary Bypass used, Operator 1 grade,

Operator 2 grade, Operation performed, Sternotomy sequence, Bypass Time, CircArrest,

XClamp Time, Cath Proc Time, Cath Fluro Time, Cath Fluro Dose,

Time Start, Procedure Urgency, Unplanned Procedure, Single Operator, Sizing Balloon

Used, No of Stents/Coils, Device Mfr, Device Model, Device Ser No, Device Size,

Overall .96

Card

.94

Surg

1.0

Outcome

Duration of Post Op Intubation, Post Procedure Seizures, Date of Discharge, Date of Death,

Status at Discharge, Discharge Destination.

Post Procedure Complications.

Overall .98

Card1.0

Surg.95

15

LGI NCHDA Report Sept 2016

The Trust DQI = 97.75% (97, 96, 94.75)This DQI is based upon the domain scoring below. The methodology for this DQI is

provided in the paper The NCHDA Audit – An Introduction to the Process.

DOMAIN Score 2013 (ii)

2014 2015 2016

Demographics .99 1.0 1.0 .99

Pre Procedure .88 .93 .92 .98

Procedure .97 .97 .99 .96

Outcome .95 .97 .97 .98

16

LGI NCHDA Report Sept 2016ConclusionsOn the whole the submitted NCHDA data were accurate, well documented, good quality and

were appropriately recorded in the Theatre and Congenital Cath lab log books that were

seen. This is the first validation of the NCHDA v5.01 dataset which is considerably enlarged

(30%) on the previous one and as such it is fully anticipated that the DQI for all congenital

cardiac centres may drop in the initial 1-2 years of review.

There has been a further 0.75% increase in the DQI to 97.75% which is an excellent

achievement. In total there were just 26 errors or omissions in 1001 data variables. The

Trust has not only maintained an excellent DQI but has increased it in a year when generally

data quality was expected to drop. This demonstrates a strong commitment to good quality

verified clinical data. There appears to be a very robust culture of clinical audit embedded

within the Trust and the DBM has invested may hours overtime to achieve timely data

submissions of a high quality. There have also been some extreme technical challenges with

the NCHDA database itself that has further impeded timely data submission during the year

2015/16 that have affected almost every congenital centre.

Again, the Validation Team are particularly grateful to the Congenital Data Manager for

meticulously detailing the documents needed at this review. The Reviewers would also like

to thank the Clinical Lead for Congenital Cardiology and other clinicians for making time to

spend with the audit team throughout the day.

The Reviewers are concerned however that as the NCHDA Dataset has considerably

expanded to include 30% more data fields and all diagnostic cardiac catheter procedures;

there must be sufficient personnel with adequate clinical and IT skills to facilitate this and

maintain data quality at LGI. There is also the addition of the Fetal Dataset to NCHDA due

in 2016. The New Congenital Heart Disease Review (NHSE July 2015) recommendation

B32(L1) states that each Specialist Surgical Centre must have a minimum of 1.0 WTE

dedicated paediatric cardiac surgery/cardiology data collection manager, with at least 1.0

WTE assistant, responsible for audit and database submissions in accordance with

necessary timescales. This has recently been further underpinned by The Report of the

Independent Review of Childrens Cardiac Services in Bristol (June 2016 Grey, Kennedy

1.22(2) and Ch17 ).

Handwritten entries into log books will always be challenging to decipher and the Reviewers

are aware that the Galaxy Theatre Information System is available in this Centre. This has

been successfully used to replace the handwritten log books in at least one other large

17

LGI NCHDA Report Sept 2016congenital cardiac centre as it is possible to record procedures using the OPCS codes that

can be cross mapped to the Association of European Paediatric and Congenital Heart

Disease (AEPC) coding that the NCHDA uses.

DeathsAs reported above 21 patients who had had procedures during  the 2015/16 data collection

year were noted to have died.  

2 records appeared to have an incorrect date of death on the local PAS when

compared to the Death Certificate

In 2 further records it was not possible to verify the dates of death against any other

source than the local PAS.  These were out of hospital deaths and these data are

manually input from a report that is printed from the Summary Care Record.

1 record appears to have an incorrect previous procedure submitted to the NCHDA

1 record appears to have an incorrect weight submitted

5 records appear to have incomplete comorbidities recorded in the data submitted to

the NCHDA

2 records appear to require more specific procedure coding

18

LGI NCHDA Report Sept 2016Recommendations

1. It is recommended that in line with the New Congenital Heart Disease Review

(NHSE July 2015) recommendation B32(L1) that each Specialist Surgical Centre

must have a minimum of 1.0 WTE dedicated paediatric cardiac surgery/cardiology

data collection manager, with at least 1.0 WTE assistant, responsible for audit and

database submissions in accordance with necessary timescales. This is further

underpinned by The Report of the Independent Review of Childrens Cardiac

Services in Bristol (June 2016 Grey, Kennedy 1.22(2) and Ch17 ). The additional

fetal data fields may require a further individual to manage these data.

2. It is recommended that the April – June 2016 NCHDA data (Q1) are submitted as

soon as possible.

3. It is recommended that greater attention to detail is used when recording procedures

performed on patients with congenital heart disease in the operating theatre and cath

lab log books. Until a fully electronic system is available the use of a self inking

stamp with the word Congenital on it may help identify cases.

4. It is recommended that consideration be given to developing the GALAXY

information system used in the operating theatres to include the accurate recording

of the exactly which congenital operation was performed on each patient.

5. As previously, it is recommended that the local Standard Operating Protocols (SOPs)

already devised for the congenital data collection, continue to be reviewed at regular

intervals to ensure their fitness for the purpose they are required to address.

6. To keep a log of all procedures such as septostomies that occur outside the cardiac

catheter laboratory.

7. In conjunction with the person responsible for training, it is suggested that regular

Quality Assurance and Governance training should be available to the DBM. Visits

to other centres who are involved in NCHDA data collection and submission are

encouraged at least once, preferably twice annually.

8. Regular training updates should be provided for all staff who may be involved with

data collection and input

19