Assessing verbal autopsy as a complement to vital registration

Assessing verbal autopsy as a complement to vital registrationA data-driven simulation study

Andrea Stewart, Post-Bachelor FellowJune 18, 2013

2

Presentation outline

• Uses of verbal autopsy (VA)

• Potential uses of new VA methods

• Creation of a simulation of incomplete vital registration (VR) complemented with VA estimates

3

Presentation outline

• Uses of verbal autopsy (VA)

• Potential uses of new VA methods

• Creation of a simulation of incomplete vital registration (VR) complemented with VA estimates

4

Uses of verbal autopsy

Photo by Peter Biro/The IRC. ©2011 International Rescue Committee.

5

Presentation outline

• Uses of Verbal Autopsy (VA)

• Potential uses of new VA methods

• Creation of a simulation of incomplete vital registration (VR) complemented with VA estimates

6

Advancements in VA science

• Automated methods for collection and analysis

• Less time

• Fewer resources

• More accurate

Could we use VA in routine national data

collection?

7

Presentation outline

• Uses of Verbal Autopsy (VA)

• Potential uses of new VA methods

• Creation of a simulation of incomplete vital registration (VR) complemented with VA estimates

8

Simulation setting – the data

• Need data that has:

o VR estimates (death certificates)

o VA estimates (analyzed by Tariff)

o Known cause of death (based on stringent diagnostic criteria)

9

Population Health Metrics Research Consortium gold standards

VA Tariff method results

Death certificate

Gold standard diagnosis

1,288 VAs from Morelos, Mexico

x 500 test data sets of varying cause composition

10

Metrics

• Cause-specific mortality fractions (CSMFs)

• Measure of accuracy

• Change in that metric across situations

%  Accuracy change=( 𝐴𝑐𝑐 h𝑤𝑖𝑡 𝑉𝐴−𝐴𝑐𝑐 𝑖𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒

𝐴𝑐𝑐𝑖𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒)×100%

𝐶𝑆𝑀𝐹𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦=1−∑𝑗=1

𝑘

¿𝐶𝑆𝑀𝐹 𝑗𝑡𝑟𝑢𝑒−𝐶𝑆𝑀𝐹 𝑗

𝑝𝑟𝑒𝑑∨¿

2(1−𝑀𝑖𝑛𝑖𝑚𝑢𝑚(𝐶𝑆𝑀𝐹 𝑗𝑡𝑟𝑢𝑒))

¿

11

Data – mortality registries from Morelos

• Mortality registries provide the proportion of deaths by age, sex, and cause of death that occur in health facilities

• %

12

Simulation setting – the data

Using Morelos probabilities -

Deaths in facility Deaths out of facility

622 observations with: Death certificateVerbal autopsy

Gold-standard diagnosis

666 observations with: Death certificateVerbal autopsy

Gold-standard diagnosis

13

Simulation setting – incomplete vital registration

Deaths in facility Deaths out of facility

Use death certificate data for

60%

Use death certificate data for 10%

No death certificate

No death certificate

Deaths in facility Deaths out of facility

Gold-standard diagnosis

Gold-standard diagnosis

vs.

𝑜𝑢𝑡𝑝𝑢𝑡 : 𝐴𝑐𝑐𝑖𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒

14

Simulation setting – add VA estimates

vs.

Deaths in facility Deaths out of facility

Use death certificate data for

60%

Use death certificate data for 10%

Use verbal autopsy data for remaining

90%Use verbal autopsy data for remaining

40%

Deaths in facility Deaths out of facility

Gold-standard diagnosis

Gold-standard diagnosis

𝑜𝑢𝑡𝑝𝑢𝑡 : 𝐴𝑐𝑐 h𝑤𝑖𝑡 𝑉𝐴

15

Simulation setting

• Do this for each of the simulated scenarioso 0%-100% deaths with DC in facility

o 0%-100% deaths with DC out of facility

• Calculate change in CSMF accuracy for each scenario

Accuracy change=( 𝐴𝑐𝑐 h𝑤𝑖𝑡 𝑉𝐴− 𝐴𝑐𝑐𝑖𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒

𝐴𝑐𝑐𝑖𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒)×100

16

ResultsPercent change in CSMF accuracy when adding VA

% of out-of-facility deaths with death certificates 0 10 20 30 40 50 60 70 80 90 100

% of in-facility deaths with

death certificates

0 27.5 24.3 20.1 17.4 14.5 11.4 10.4 8.9 7.2 6.010 29.1 24.7 21.7 18.0 15.4 12.3 9.6 9.0 7.9 5.7 5.320 26.3 21.8 19.1 15.9 13.7 10.9 8.4 7.8 6.8 5.1 4.330 23.1 19.5 16.6 13.8 11.8 9.5 6.9 6.1 5.3 3.8 3.640 20.2 17.1 14.3 11.7 9.8 7.5 5.2 4.7 4.0 2.6 2.350 17.5 14.9 12.4 10.0 8.0 6.0 4.1 3.9 2.9 1.9 1.760 15.5 12.7 10.3 7.9 6.7 5.0 3.3 3.1 2.3 1.0 0.870 13.3 10.5 8.3 6.2 4.9 3.4 2.1 2.0 1.3 0.6 0.380 11.2 8.8 7.4 5.1 4.0 2.6 1.2 1.2 0.8 -0.1 0.290 9.6 7.9 6.2 3.9 2.7 1.8 0.6 0.6 0.2 -0.2 -0.1

100 8.5 7.0 5.3 3.2 2.2 0.8 0.3 0.2 0.3 -0.2 0.0

17

ResultsPercent change in CSMF accuracy when adding VA

% of out-of-facility deaths with death certificates 0 10 20 30 40 50 60 70 80 90 100

% of in-facility deaths with

death certificates

0 27.5 24.3 20.1 17.4 14.5 11.4 10.4 8.9 7.2 6.010 29.1 24.7 21.7 18.0 15.4 12.3 9.6 9.0 7.9 5.7 5.320 26.3 21.8 19.1 15.9 13.7 10.9 8.4 7.8 6.8 5.1 4.330 23.1 19.5 16.6 13.8 11.8 9.5 6.9 6.1 5.3 3.8 3.640 20.2 17.1 14.3 11.7 9.8 7.5 5.2 4.7 4.0 2.6 2.350 17.5 14.9 12.4 10.0 8.0 6.0 4.1 3.9 2.9 1.9 1.760 15.5 12.7 10.3 7.9 6.7 5.0 3.3 3.1 2.3 1.0 0.870 13.3 10.5 8.3 6.2 4.9 3.4 2.1 2.0 1.3 0.6 0.380 11.2 8.8 7.4 5.1 4.0 2.6 1.2 1.2 0.8 -0.1 0.290 9.6 7.9 6.2 3.9 2.7 1.8 0.6 0.6 0.2 -0.2 -0.1

100 8.5 7.0 5.3 3.2 2.2 0.8 0.3 0.2 0.3 -0.2 0.0

18

ResultsPercent change in CSMF accuracy when adding VA

% of out-of-facility deaths with death certificates 0 10 20 30 40 50 60 70 80 90 100

% of in-facility deaths with

death certificates

0 27.5 24.3 20.1 17.4 14.5 11.4 10.4 8.9 7.2 6.010 29.1 24.7 21.7 18.0 15.4 12.3 9.6 9.0 7.9 5.7 5.320 26.3 21.8 19.1 15.9 13.7 10.9 8.4 7.8 6.8 5.1 4.330 23.1 19.5 16.6 13.8 11.8 9.5 6.9 6.1 5.3 3.8 3.640 20.2 17.1 14.3 11.7 9.8 7.5 5.2 4.7 4.0 2.6 2.350 17.5 14.9 12.4 10.0 8.0 6.0 4.1 3.9 2.9 1.9 1.760 15.5 12.7 10.3 7.9 6.7 5.0 3.3 3.1 2.3 1.0 0.870 13.3 10.5 8.3 6.2 4.9 3.4 2.1 2.0 1.3 0.6 0.380 11.2 8.8 7.4 5.1 4.0 2.6 1.2 1.2 0.8 -0.1 0.290 9.6 7.9 6.2 3.9 2.7 1.8 0.6 0.6 0.2 -0.2 -0.1

100 8.5 7.0 5.3 3.2 2.2 0.8 0.3 0.2 0.3 -0.2 0.0

19

ResultsPercent change in CSMF accuracy when adding VA

% of out-of-facility deaths with death certificates 0 10 20 30 40 50 60 70 80 90 100

% of in-facility deaths with

death certificates

0 27.5 24.3 20.1 17.4 14.5 11.4 10.4 8.9 7.2 6.010 29.1 24.7 21.7 18.0 15.4 12.3 9.6 9.0 7.9 5.7 5.320 26.3 21.8 19.1 15.9 13.7 10.9 8.4 7.8 6.8 5.1 4.330 23.1 19.5 16.6 13.8 11.8 9.5 6.9 6.1 5.3 3.8 3.640 20.2 17.1 14.3 11.7 9.8 7.5 5.2 4.7 4.0 2.6 2.350 17.5 14.9 12.4 10.0 8.0 6.0 4.1 3.9 2.9 1.9 1.760 15.5 12.7 10.3 7.9 6.7 5.0 3.3 3.1 2.3 1.0 0.870 13.3 10.5 8.3 6.2 4.9 3.4 2.1 2.0 1.3 0.6 0.380 11.2 8.8 7.4 5.1 4.0 2.6 1.2 1.2 0.8 -0.1 0.290 9.6 7.9 6.2 3.9 2.7 1.8 0.6 0.6 0.2 -0.2 -0.1

100 8.5 7.0 5.3 3.2 2.2 0.8 0.3 0.2 0.3 -0.2 0.0

20

ResultsPercent change in CSMF accuracy when adding VA

% of out-of-facility deaths with death certificates 0 10 20 30 40 50 60 70 80 90 100

% of in-facility deaths with

death certificates

0 27.5 24.3 20.1 17.4 14.5 11.4 10.4 8.9 7.2 6.010 29.1 24.7 21.7 18.0 15.4 12.3 9.6 9.0 7.9 5.7 5.320 26.3 21.8 19.1 15.9 13.7 10.9 8.4 7.8 6.8 5.1 4.330 23.1 19.5 16.6 13.8 11.8 9.5 6.9 6.1 5.3 3.8 3.640 20.2 17.1 14.3 11.7 9.8 7.5 5.2 4.7 4.0 2.6 2.350 17.5 14.9 12.4 10.0 8.0 6.0 4.1 3.9 2.9 1.9 1.760 15.5 12.7 10.3 7.9 6.7 5.0 3.3 3.1 2.3 1.0 0.870 13.3 10.5 8.3 6.2 4.9 3.4 2.1 2.0 1.3 0.6 0.380 11.2 8.8 7.4 5.1 4.0 2.6 1.2 1.2 0.8 -0.1 0.290 9.6 7.9 6.2 3.9 2.7 1.8 0.6 0.6 0.2 -0.2 -0.1

100 8.5 7.0 5.3 3.2 2.2 0.8 0.3 0.2 0.3 -0.2 0.0

21

Conclusion

In this preliminary simulation environment, adding VA estimates to incomplete VR scenarios improved CSMF accuracy much of the time.

22

Future directions

• Update simulation environment to better reflect specific VR coverage scenarios

• Re-analyze the VAs assigned to “out of facility,” dropping questions determined to be related to “health care experience”

Thank you

Andrea Stewart

Post-Bachelor Fellow

als39@uw.edu