6
Correspondence: M. Benatar, Department of Neurology, Emory University School of Medicine, Woodruff Memorial Research Building, Suite 6000, 101 Woodruff Circle, Atlanta, Georgia 30322, USA. Fax: 404 727 3157. E-mail: [email protected] (Received 27 May 2010; accepted 9 August 2010) Amyotrophic Lateral Sclerosis, 2011; 12: 130–135 ISSN 1748-2968 print/ISSN 1471-180X online © 2011 Informa Healthcare DOI: 10.3109/17482968.2010.515224 established collaborative network of European regis- tries (EURALS) draws on a combined population of approximately 25 million (9) and has greatly enhanced the utility of the data (10). The development and success of ALS registries in regions and countries such as Lombardy, Italy (4), Scotland (11), and Ireland (6) has been greatly facil- itated by the social, geopolitical, and demographic characteristics of the local population. In Ireland, for example, few people seek medical care outside of the free or heavily subsidized public sector; there is a single designated ALS outpatient clinic that serves the entire country and medical services are largely hospital based (6). Circumstances in the United States are much less favorable for establishment of a registry insofar as the population is much larger and more widely distributed while health care delivery is significantly more fragmented. In anticipation of the signing of the ALS Registry Act and the development of a national ALS registry Introduction The Amyotrophic Lateral Sclerosis (ALS) Registry Act (H.R. 2295), passed by the United States House of Representatives in October 2008, calls for the establishment of a national registry of patients with ALS or Lou Gehrig's disease (1). The goal of a pop- ulation based registry such as this is to collect infor- mation about all diagnoses of ALS and thereby permit reliable estimation of disease incidence and preva- lence, definition of the demographic characteristics of individuals with ALS, effective monitoring of the temporal and geographic trends in disease distribution, and an investigation of the environmental risk factors for the disease. A number of national and regional ALS registries have existed in Europe for over a decade (2–7), but with the exception of the Veterans registry (8), there have been no robust efforts to develop a national registry in the United States. While these European registries have been hindered in the past by their relatively small population base, the recently ORIGINAL ARTICLE Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia MICHAEL BENATAR 1 , JOANNE WUU 2 , SHARON USHER 1 & KEVIN WARD 3 1 Department of Neurology, School of Medicine, Emory University, 2 Section of Neurostatistics, Department of Neurology, School of Medicine, Emory University, and 3 Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, USA Abstract Our objective was to investigate the utility of existing data sources for identifying cases of amyotrophic lateral sclerosis (ALS) and related motor neuron diseases (MND) in the State of Georgia. Data were acquired from Medicare, Medicaid, Veterans Administration, Emory Healthcare, community neurologists, the ALS Association, and mortality records for ALS/ MND patients residing in Georgia during 2001–2005. A neurologist used abstracted medical records to verify the diag- nosis of ALS/MND. The positive predictive value (PPV) of an ICD code for a verified diagnosis of ALS was estimated. Simple ‘rules’ were developed to improve PPV. Results showed that a total of 2413 unique potential cases were identified in existing data sources. Medical records of 579 cases were available for review; the diagnosis of ALS (or a related MND) was confirmed in 486 (PPV 84%) cases. Predictive rules, which permitted classification of 80% of the chart-reviewed population, improved PPV to 96–98%. In conclusion, existing data sources are useful for identifying cases of ALS/MND; most data sources contribute a substantial number of unique cases. Predictive algorithms may permit correct classification of a large proportion of cases without the need for verification based on medical record review. Key words: Epidemiology, neuroepidemiology, ALS Registry, ICD code, positive predictive value Amyotroph Lateral Scler Downloaded from informahealthcare.com by Nyu Medical Center on 11/05/14 For personal use only.

Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

  • Upload
    kevin

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

Amyotrophic Lateral Sclerosis, 2011; 12: 130–135

ORIGINAL ARTICLE

Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

MICHAEL BENATAR 1 , JOANNE WUU 2 , SHARON USHER 1 & KEVIN WARD 3

1 Department of Neurology, School of Medicine, Emory University, 2 Section of Neurostatistics, Department of Neurology, School of Medicine, Emory University, and 3 Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, USA

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

Abstract Our objective was to investigate the utility of existing data sources for identifying cases of amyotrophic lateral sclerosis (ALS) and related motor neuron diseases (MND) in the State of Georgia. Data were acquired from Medicare, Medicaid, Veterans Administration, Emory Healthcare, community neurologists, the ALS Association, and mortality records for ALS/MND patients residing in Georgia during 2001 – 2005. A neurologist used abstracted medical records to verify the diag-nosis of ALS/MND. The positive predictive value (PPV) of an ICD code for a verifi ed diagnosis of ALS was estimated. Simple ‘ rules ’ were developed to improve PPV. Results showed that a total of 2413 unique potential cases were identifi ed in existing data sources. Medical records of 579 cases were available for review; the diagnosis of ALS (or a related MND) was confi rmed in 486 (PPV � 84%) cases. Predictive rules, which permitted classifi cation of ∼ 80% of the chart-reviewed population, improved PPV to 96 – 98%. In conclusion, existing data sources are useful for identifying cases of ALS/MND; most data sources contribute a substantial number of unique cases. Predictive algorithms may permit correct classifi cation of a large proportion of cases without the need for verifi cation based on medical record review.

Key words: Epidemiology , neuroepidemiology , ALS Registry , ICD code , positive predictive value

Introduction

The Amyotrophic Lateral Sclerosis (ALS) Registry Act (H.R. 2295), passed by the United States House of Representatives in October 2008, calls for the establishment of a national registry of patients with ALS or Lou Gehrig ' s disease (1). The goal of a pop-ulation based registry such as this is to collect infor-mation about all diagnoses of ALS and thereby permit reliable estimation of disease incidence and preva-lence, defi nition of the demographic characteristics of individuals with ALS, effective monitoring of the temporal and geographic trends in disease distribution, and an investigation of the environmental risk factors for the disease. A number of national and regional ALS registries have existed in Europe for over a decade (2 – 7), but with the exception of the Veterans registry (8), there have been no robust efforts to develop a national registry in the United States. While these European registries have been hindered in the past by their relatively small population base, the recently

Correspondence: M. Benatar, Department of Neurology, Emory University 101 Woodruff Circle, Atlanta, Georgia 30322, USA. Fax: 404 727 3157. E-mail:

(Received 27 May 2010; accepted 9 August 2010)

ISSN 1748-2968 print/ISSN 1471-180X online © 2011 Informa HealthcareDOI: 10.3109/17482968.2010.515224

established collaborative network of European regis-tries (EURALS) draws on a combined population of approximately 25 million (9) and has greatly enhanced the utility of the data (10).

The development and success of ALS registries in regions and countries such as Lombardy, Italy (4), Scotland (11), and Ireland (6) has been greatly facil-itated by the social, geopolitical, and demographic characteristics of the local population. In Ireland, for example, few people seek medical care outside of the free or heavily subsidized public sector; there is a single designated ALS outpatient clinic that serves the entire country and medical services are largely hospital based (6). Circumstances in the United States are much less favorable for establishment of a registry insofar as the population is much larger and more widely distributed while health care delivery is signifi cantly more fragmented.

In anticipation of the signing of the ALS Registry Act and the development of a national ALS registry

School of Medicine, Woodruff Memorial Research Building, Suite 6000, [email protected]

Page 2: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

Pilot ALS Registry in USA 131

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

in the United States, in late 2006 the Agency for Toxic Substances and Disease Registries (ATSDR) and the Centers for Disease Control and Prevention (CDC) funded three (and later a fourth) ALS registry pilot projects. One of these pilot projects was based in the State of Georgia with the primary goal of exploring the utility of existing data sources for seeding a national ALS registry within the United States. Importantly, this pilot project was not designed to estimate the completeness of case ascertainment or the preva-lence of ALS. Instead, the aims of this pilot project were 1) to investigate whether multiple data sources would enhance case identifi cation; 2) to determine the positive predictive value of a search of existing data sources using ICD9 and ICD10 codes for true cases of ALS; and 3) to explore whether it might be possible to develop predictive rules or algorithms to identify true cases of ALS without the need for diag-nosis confi rmation by medical record review.

Methods

Disease defi nitions

In the United States, the term ‘ ALS ’ is used in two ways: 1) in a more restrictive fashion to describe a specifi c form of motor neuron disease in which there is degeneration of both upper and lower motor neu-rons (i.e. amyotrophic lateral sclerosis); and 2) in a more general manner to describe a group of motor neuron diseases that includes not only amyotrophic lateral sclerosis, but also progressive muscular atro-phy (PMA), primary lateral sclerosis (PLS) and pro-gressive bulbar palsy (PBP). Since the intended scope of the National ALS Registry in the United States includes these related forms of motor neuron disease and in order to circumvent the potential confusion that might result from the dual use of the term ‘ ALS ’ , we have employed the term ‘ ALS � ’ to refer to ALS and the related motor neuron diseases of PMA, PLS and PBP. On the other hand, we use the broader term ‘ motor neuron disease (MND) ’ to refer to any disorder of the motor neuron including, but not limited to, ALS � .

Data sources

Data for this pilot project were acquired for patients residing in the State of Georgia during 2001 – 2005 and from multiple sources including Medicare, Medicaid, the Veterans Administration (VA), Emory Healthcare, two community based neurologists, the ALS Associa-tion (ALSA) and mortality records. The Medicare, Medicaid and VA administrative databases as well as the VA Benefi ts database were searched using ICD9 335.2 (MND), 335.20 (ALS), 335.21 (PMA), 335.22 (PBP), 335.23 (pseudobulbar palsy), 335.24 (PLS) and 335.29 (other motor neuron disease). The Emory Healthcare administrative database was queried using the same search strategy. Data from Emory were also acquired from informal databases maintained in the

multidisciplinary ALS clinic and the electromyogra-phy (EMG) laboratory. These three data sources were combined to refl ect all of the data available from Emory, the dominant tertiary referral center in the state for patients with ALS. Data were also acquired from a patient register maintained by the Georgia chapter of the ALS Association. The Offi ce of Vital Statistics in the Georgia Division of Public Health provided electronic death certifi cate data for patients with MND. Since this data source included only the fi rst four digits of the ICD10 code for MND (G12.2), it could not be used in isolation to identify the more specifi c diagnosis of ALS � . If, however, a case was identifi ed as ALS � in another data source then the designation of MND in the death certifi cate data was taken to also imply a diagnosis of ALS � . Although the scope of the pilot project did not permit acquisi-tion of data from all community neurologists practic-ing in Georgia, requests were made to three community neurologists to search their practice administration databases using ICD9 codes in order to examine the feasibility of acquiring such data.

Efforts to acquire data from the Medical College of Georgia (the other major tertiary referral center in the state), the Mayo Clinic in Jacksonville, Florida (which provides coverage to patients residing in south-ern Georgia), and Grady Memorial Hospital (an inner-city hospital that is affi liated with Emory University) were unsuccessful.

Data linkage

Data from these diverse sources were integrated into a single Microsoft Access database with linkage per-formed using probabilistic algorithms that included multiple data elements such as last name, fi rst name, social security number and date of birth. A single record was created for each unique case identifi ed from the various data sources. Each record included demographic information, an indication of the data source(s) from which the case was identifi ed, the number of unique medical encounters in 2001 – 2005 within each data source (where a unique medical encounter was defi ned as a non-contiguous date period), the ICD9 or ICD10 code(s) via which the case was identifi ed, and the speci alty of the treating physician or healthcare provider.

Medical record abstraction

To confi rm the diagnosis of MND/ALS � , medical records were acquired from the electronic medical record system maintained for all patients receiving care at Emory Healthcare as well as the paper records of the community neurologists. Access to the VA medical records could not be obtained. All medical records were abstracted using a standardized case report form (CRF). This CRF was developed by the CDC in conjunction with the principal investigators of the three original ALS registry pilot projects

Page 3: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

132 M. Benatar et al.

(Emory, Mayo Clinic in Rochester, Minnesota, and the South Carolina State Health Department; Kaiser Permanente of California was later added as the fourth project). A registered nurse either performed chart abstraction or supervised abstraction by trained personnel.

The ‘ true ’ diagnosis of ALS � or a related MND was made by a neurologist (MB) with expertise in the management of patients with neuromuscular dis-ease. The diagnosis was based primarily on review of a summary of the abstracted chart and a review of the available EMG data. The medical record sum-mary included, but was not limited to, information about the nature and temporal evolution of symp-toms and the distribution of examination fi ndings. Based on this review, the neurologist determined the presence or absence of MND/ALS � according to the revised El Escorial criteria. The neurologist reviewed the full medical record when the diagnosis of MND/ALS � was uncertain based on the chart summary. In the few ( n � 25) cases in which the diagnosis remained unclear despite full medical record review, we relied upon the diagnosis of the treating physician that was evident from the medical record.

Predictive value and predictive rules

The strategy for data collection permits estimation of the positive predictive value (PPV) of an existing data source, based on the presence of an ICD9 code (or other means of case identifi cation), for a verifi ed diagnosis of ALS � based on medical record review by an experienced neurologist. Because the majority of records reviewed were obtained from Emory, the estimated PPV primarily refl ects the predictive value of a diagnosis of ALS � based on Emory (i.e. tertiary referral center) data. Estimates of PPV based on other data sources such as Medicare really refl ect the PPV of a Medicare ICD9 ALS � diagnosis for patients also identifi ed as ALS � based on the Emory data source. Furthermore, this strategy did not permit the identifi cation of a population of individuals without an ICD9 code of ALS � in existing data sources; it was therefore not possible to estimate the negative predictive value. As such, this pilot project provides no information about the rate of false negatives (i.e. the number of true cases of ALS or a related syndrome that were not captured by the strategy used to identify potential cases).

Having determined the crude PPV of an ALS � ICD9 code in an existing data source, we sought to determine whether the PPV might be improved through the use of predictive rules. The goal of devel-oping these rules was: 1) to capture as many cases as possible; and 2) to maintain as high a PPV as possible. To accomplish this goal, the data sources were fi rst studied one at a time. Data sources with larger sample sizes (e.g. Emory Healthcare and Medicare), for example, suggested that rules based on the inclusion of these data sources would permit

the capture of a large number of subjects. To improve PPV while preserving the ability to capture as large a number of cases as possible, we considered second-ary factors such as the number of calendar years in which the subject appeared in the data source and whether the provider for the encounter(s) was a neu-rologist. Simple inspection of the data also revealed that various data sources, notably death certifi cates, ALSA and the VA, yielded very high PPV. Demo-graphic characteristics were also considered during the development of these rules. Once the rules had been constructed they were ordered based on two prin-ciples: 1) the preference to capture as many cases as possible with each sequential rule; and 2) the gen-eration of a hierarchy of rules such that the trade-off between capturing an increasing number of subjects on the one hand, and a declining PPV on the other, would be evident.

Results

Total study population

A total of 2413 unique potential cases of MND were identifi ed as residents in the State of Georgia at some stage during the fi ve-year period of this study. Within each data source, the number of potential MND cases identifi ed in that source alone ranged from 13 (ALSA, a smaller database) to 1008 (Medicare, a large database), underscoring the importance of using multiple data sources for case identifi cation (Table I). The average age of MND patients at the time of fi rst appearance in a data source within our study period of 2001 – 2005 was 65 ( � 15) years, with a male: female ratio of 1.6: 1. Racial classifi cation showed 70% of cases were white, 19% black, 1.2% other, and 9% unknown.

Chart reviewed population

Efforts were made to retrieve as many medical records as possible for review. Nevertheless, the overwhelm-ing majority of records available for review were from

Table I. The number of potential MND and ALS � cases iden tifi ed in each data source (and exclusively in a data source). a

MND ALS �

Emory 601 (174) 590 (166)Medicare 1634 (1008) 1275 (690)Medicaid 228 (91) 187 (70)Veterans Administration 171 (107) 145 (82)ALS Association 98 (13) 98 (14)Community neurologists 135 (45) 115 (33)Death certifi cates 677 (198) 451 (0) b

a The number of MND cases listed in the fi rst column exceeds the total of 2413 cases identifi ed in the State of Georgia because many cases were identifi ed in more than one data source. bDeath certifi cates list only the fi rst four digits of ICD10 codes, which did not permit the more specifi c ascertainment of ALS � cases. ALS � cases therefore could not be identifi ed in death certifi cate records alone.

Page 4: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

Pilot ALS Registry in USA 133

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

Emory, with a small minority from other sources such as the offi ces of community neurologists. A total of 599 medical records (25% of the total 2413 cases ini-tially identifi ed) were successfully obtained and reviewed. Of these, 20 had been identifi ed using a broader defi nition of ALS and related disorders during the early phase of the study; these cases were excluded once the fi nal defi nition of our target population, ALS � , was established among the pilot projects. Among the remaining 579 cases, 558 were identifi ed in the Emory databases, 307 in Medicare, 40 in Medi-caid, 27 in the VA databases, 68 in the ALS Association records, 65 in the offi ces of the two community neu-rologists, and 226 in the death certifi cate records. Based on our chart review, the diagnosis of ALS � was con-fi rmed in 481 (83.1%), refuted in 90 (15.5%), and remained uncertain despite review of all available medical records in eight (1.4%). Among the 90 sub-jects with a diagnosis other than ALS � , the most common alternate diagnoses were spinal spondylosis ( n � 15), other MND ( n � 11), neuropathy ( n � 10), movement disorders ( n � 10), and other neu-rological disorders ( n � 20) (Table II).

Demographics

The average ages among patients with ALS, PLS and PMA were comparable (58 – 60 years), with a male pre-ponderance (53 – 55% among patients with ALS or PLS and 86% among patients with PMA) (Table III). Overall, the demographic characteristics of the 481 confi rmed ALS � cases and the 98 confi rmed non-ALS � or undetermined cases were comparable. Cases with diagnoses other than ALS � or an unknown diagnosis were slightly younger than the ALS � cases (52.1 � 19.5 vs. 58.7 � 13.1 years) (Table III).

Predictive rules and positive predictive values

The overall positive predictive value (PPV) among the 571 cases (after excluding the n � 8 with unknown diagnosis) was 84% (95% exact binomial confi dence interval (CI) � 81 – 87%). Within the individual data sources the PPV ranged from 84% (Emory) to 100%

Table II. Alternate diagnoses of 98 cases whose diagnoses were determined to be non-ALS� based on chart review.

n

Spinal stenosis (including radiculopathy and myelopathy)

15

Other MND (including SMA, polio, and Hopkin’s disease)

11

Neuropathy 10Movement disorders 10Dementia 6Autoimmune 4Cerebrovascular disease 3Epilepsy 3Other neurological disorders 20Other non-neurological disorders 8Unknown 8

Tab

le I

II.

Dem

ogra

phic

ch

arac

teri

stic

s of

ch

art-

revi

ewed

pop

ula

tion

.

AL

S �

AL

S (

n �

440

)P

LS

( n

� 1

9)P

MA

( n

� 2

2)A

ll A

LS �

( n

� 4

81)

Oth

er/u

nkn

own

§ ( n

� 9

8)

Tot

al (

n �

579

)

Age

, ye

ars ∗

M

ean

� S

D (

ran

ge)

58.7

� 1

2.7

(20

-89)

58.3

� 1

7.1

(26

–80

)60

.0 �

14.

4 (3

5–8

4)58

.7 �

13.

1 (2

0–8

9)52

.1 �

19.

5 (2

–87

)57

.6 �

14.

5 (2

–89)

Gen

der

, n

(%)

M

ale

240

(55%

)10

(53

%)

19 (

86%

)26

9 (5

6%)

54 (

55%

)32

3 (5

6%)

F

emal

e18

9 (4

3%)

7 (3

7%)

2 (9

%)

198

(41%

)42

(43

%)

240

(41%

)

Un

know

n11

(2

%)

2 (1

0%

)1

(5%

)14

(3%

)2

(2%

)16

(3%

)R

ace,

n (

%)

W

hit

e32

5 (7

4%)

10 (

53%

)9

(41%

)34

4 (7

1%)

54 (

55%

)39

8 (6

9%)

B

lack

59 (

13%

)1

(5%

)8

(36%

)68

(14

%)

18 (

18%

)86

(15

%)

A

sian

3 (1

%)

00

3 (1

%)

03

(1%

)

Oth

er6

(1%

)0

1 (5

%)

9 (2

%)

09

(1%

)

Un

know

n47

(11

%)

6 (3

2%

)4

(18

%)

57 (

12%

)26

(26

%)

83 (

14%

)

∗ Age

at

fi rs

t ap

pear

ance

in

dat

a so

urc

es b

etw

een

2001

an

d 20

05:

14 A

LS

, 2

PL

S,

1 P

MA

, an

d 4

Oth

er/U

nkn

own

case

s d

id n

ot h

ave

age

info

rmat

ion

avai

labl

e.§ I

ncl

ud

es n

= 9

0 w

hose

dia

gnos

is w

as c

onfi

rmed

to

be

non

-AL

S+

an

d n

= 8

who

se d

iagn

osis

was

un

know

n.

The

eig

ht c

ases

wit

h u

nkn

own

dia

gnos

is w

ere

excl

ud

ed f

rom

su

bse

quen

t an

alys

is.

Page 5: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

134 M. Benatar et al.

(VA) (Table IV). Seven predictive rules were identifi ed that permitted classifi cation of almost 80% of the study population with a PPV in the range of 96 – 98% (Table V). We did not identify any further rules that would permit the capture of additional cases without dramatically reducing the PPV.

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

Discussion

Efforts to develop a national ALS registry in the United States, despite being greatly facilitated by the maturity and experience of the European ALS reg-istries, still face many challenges due to the larger and more geographically dispersed population as well as the healthcare delivery and insurance systems in this country. Construction of a national ALS reg-istry requires both accurate and complete ascertain-ment of cases. Our pilot project was not designed to determine the completeness of case ascertainment using only existing data sources, but rather to evalu-ate the accuracy with which cases of ALS might be identifi ed from existing data sources.

We observed that: 1) Existing data sources are useful for identifying cases of ALS and related motor neuron diseases for inclusion in a national registry. 2) Most data sources contribute a substantial num-ber of unique cases of ALS � (and MND); this pro-vides a strong a priori rationale for searching multiple data sources as part of future efforts to develop a

national ALS registry. 3) It may be possible to accept a fair proportion of cases as ALS � using existing data sources without the need for verifi cation based on a neurologist ' s review of the medical records. The need for medical record review has typically been assumed in the process of establishing European ALS registries, but the possibility that medical record review may not always be necessary has not been pre-viously explored. Our data suggest that as many as 80% of ALS � cases preliminarily ascertained from existing data sources may be further identifi ed as a subset of cases with a very high likelihood ( � 96%) of being ‘ true ’ (i.e. confi rmed) ALS � cases based on specifi c criteria. Examples of such criteria include the selection of cases from particular data sources, iden-tifi cation of cases in multiple data sources, diagnosis by a neurologist, and a death certifi cate code of motor neuron disease (G12.2) in conjunction with ascertainment in another data source. However, even within this 80% of ALS � cases, PPV is not 100%; it is therefore important to recognize that construc-tion of a registry without individual medical record review will inevitably lead to inclusion of a small number of false positive cases of ALS � .

We recognize that our pilot project had a number of weaknesses and limitations. First, our methods for case ascertainment, by design, were not exhaustive and therefore did not permit the identifi cation of all cases of ALS � or MND. Notably missing from the current study are data from other tertiary referral cen-ters that serve the population of the State of Georgia, as well as data from the offi ces of the large number of community neurologists who work outside of ter-tiary referral centers. Related to this is the non-random fashion in which the small number of community neurologists who participated in this study was selected. Secondly, the design of our study did not permit quan tifi cation of the number of ALS � or MND cases that would be missed by relying solely on existing data sources for case identifi cation. A third limitation was that our access to medical records for review was largely limited to Emory Healthcare. We have therefore almost certainly over-estimated the PPV of data sources other than Emory.

Table IV. Positive predictive value.

Confi rmed ALS �

Data sources Yes No Total PPV (95% CI) ∗

Emory 464 87 551 84% (81–87%)Medicare 268 34 302 89% (85–92%)Medicaid 36 4 40 90% (76–97%)VA 27 0 27 100% –ALSA 64 3 67 96% (87–99%)Community neurologists

59 5 64 92% (83–97%)

Death certifi cate 218 6 224 97% (94–99%)Any 481 90 571 84% (81–87%)

∗ 95% exact binomial confi dence interval.

Table V. Predictive rules, applied cumulatively, for confi rmed cases of ALS � .

After applying: Rule #1 Rule #2 Rule #3 Rule #4 Rule #5 Rule #6 Rule #7 ... No Rule

# cases captured: 343 414 418 424 428 438 452 ... 571% captured: 60% 73% 73% 74% 75% 77% 79% ... 100%PPV: 98.0% 97.1% 97.1% 97.2% 97.0% 98.8% 96.2% ... 84.2%

Rule 1. Appearance in the Emory data source AND either 1) appearance in the Emory data source for �2 years between 2001 and 2005, OR 2) Emory provider type � neurologist. Rule 2. Death certifi cate ICD10 code of G12.2 (given the method of case ascertainment, all cases identifi ed through death certifi cates also appeared in a second data source as ALS�).Rule 3. Appearance in the VA data source.Rule 4. Appearance in the ALS Association data source.Rule 5. Appearance in �3 data sources.Rule 6. Appearance in the Community neurologists data source.Rule 7. Appearance in the Medicare data source at least twice AND Medicare provider type � neurologist.

Page 6: Preparing for a U.S. National ALS Registry: Lessons from a pilot project in the State of Georgia

Pilot ALS Registry in USA 135

Am

yotr

oph

Lat

eral

Scl

er D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y N

yu M

edic

al C

ente

r on

11/

05/1

4Fo

r pe

rson

al u

se o

nly.

For example, PPV of the Medicare data was high in part because the Medicare cases for which we could obtain medical records for review were, by design, cases also identifi ed in a second data source (e.g. Emory), thus increasing the likelihood of a con-fi rmed defi nitive diagnosis of ALS � . In other words, since the great majority of the chart reviewed cases were seen at Emory, the PPV of any other source would likely be at least as high as the Emory PPV. As a consequence, our predictive rules may be most relevant to ALS � cases ascertained through other tertiary referral centers. Finally, the limited number of charts available to us for review precluded our using part of the data to develop predictive rules and the remaining data to validate these rules.

The establishment of a national ALS registry in the United States will be a complex process, likely requiring multiple approaches to case ascertainment. It is estimated that approximately 44% of ALS patients in the United States receive care through a multidisciplinary ALS clinic at a tertiary referral center (12); the results of our pilot study are most relevant to the ascertainment of these cases. We have shown that it may be possible to correctly classify as much as ∼ 80% of this population (with 96% accu-racy) as having ALS � or a related motor neuron disease using existing data sources and predictive rules. Based on these estimates, approximately ∼ 33% (44% � 80% � 96%) of the ALS population could be identifi ed using the methods presented in this manuscript. More complete case ascertainment, however, will require the use of additional approaches, including patient self-identifi cation and registration, as well as active case ascertainment through com-munity neurologists and relevant patient advocacy groups. Furthermore, the active engagement of ALS patients, their treating physicians and other health-care providers, and organizations such as the ALS Association and Muscular Dystrophy Association will foster a partnership that will be essential to the success of the National ALS Registry.

Acknowledgements

We gratefully acknowledge the many people who assisted our acquisition of data from various sources. These include Wendy Kaye, Oleg Muravov, Jenny Wu and Judy Smith at ATSDR/CDC; Keith Sanders at Atlanta Neurology; David Williams at Peachtree Neurology; Nancy Carter at Emory Healthcare, Meraida Polak and members of the Emory ALS Clinic; Rana Bayakly at the Division of Public Health

in the Georgia Department of Community Health; Linda Wojno at the Georgia Chapter of the ALS Association; Vicki Cuneen at the Georgia Chapter of the Muscular Dystrophy Association (MDA); James Knowles and Karen Fisher at the MDA clinic in Roosevelt Warm Springs; Kevin Boylan at the Mayo Clinic in Jacksonville, Florida; and Michael Rivner and Demetric Hillman at the Medical College of Georgia. We also thank Michael Wittie and Masha Goodman for their assistance with data abstraction. Finally, we thank John Young at the Georgia Cancer Registry for his advice and guidance. This research was funded by a contract from ATSDR/CDC.

Declaration of interest: The authors report no confl icts of interest.

References

ALS Registry Act. 2008 [cited 2009 October 4]. Available from: 1. http://www.govtrack.us/congress/bil ltext.xpd?bill � s110-1382. Incidence of ALS in Italy: evidence for a uniform frequency 2. in Western countries. Neurology. 2001;56:239 – 44. Abhinav K, Stanton B, Johnston C, Hardstaff J, Orrell RW, 3. Howard R, et al. Amyotrophic lateral sclerosis in south-east England: a population based study. The South-East England register for Amyotrophic Lateral Sclerosis (SEALS Registry). Neuroepidemiology. 2007;29:44 – 8. Beghi E, Millul A, Micheli A, Vitelli E, Logroscino G. Incidence 4. of ALS in Lombardy, Italy. Neurology. 2007;68:141 – 5. Forbes RB, Colville S, Parratt J, Swingler RJ. The incidence of 5. motor nueron disease in Scotland. J Neurol. 2007;254:866 – 9. Traynor BJ, Codd MB, Corr B, Forde C, Frost E, Hardiman O. 6. Incidence and prevalence of ALS in Ireland, 1995 – 1997: a population-based study. Neurology. 1999;52:504 – 9. Logroscino G, Beghi E, Zoccolella S, Palagano R, Fraddosio A, 7. Simone IL, et al. Incidence of amyotrophic lateral sclerosis in southern Italy: a population based study. J Neurol Neurosurg Psychiatry. 2005;76:1094 – 8. Allen KD, Kasarskis EJ, Bedlack RS, Rozear MP, Morgen-8. lander JC, Sabet A, et al. The National Registry of Veterans with amyotrophic lateral sclerosis. Neuroepidemiology. 2008;30:180 – 90. Beghi E. 127th ENMC International Workshop: implementation 9. of a European registry of ALS. Naarden, The Netherlands, 8 – 10 October 2004. Neuromuscul Disord. 2006;16:46 – 53. Logroscino G, Traynor BJ, Hardiman O, Chio A, Mitchell D, 10. Swingler RJ, et al. Incidence of amyotrophic lateral sclerosis in Europe. J Neurol Neurosurg Psychiatry. 2010;81:385 – 90. The Scottish Motor Neuron Disease Register: a prospective 11. study of adult onset motor neuron disease in Scotland. Metho dology, demography and clinical features of incident cases in 1989. J Neurol Neurosurg Psychiatry. 1992;55:536 – 41. Khishchenko N, Allen KD, Coffman CJ, Kasarskis EJ, 12. Lindquist JH, Morgenlander JC, et al. Time to diagnosis in the National Registry of Veterans with Amyotrophic Lateral Scler. Amyotroph Lateral Scler. 2010;11:125 – 32.