Thrombosis Research 131 (2013) 31–36

Contents lists available at SciVerse ScienceDirect

Thrombosis Research

j ourna l homepage: www.e lsev ie r .com/ locate / thromres

Regular Article

Identifying Major Hemorrhage with Automated Data: Results of the Veterans AffairsStudy to Improve Anticoagulation (VARIA)

Guneet K. Jasuja a,b,⁎, Joel I. Reisman a, Donald R. Miller a,b, Dan R. Berlowitz a,b,c, Elaine M. Hylek a,c,Arlene S. Ash a,d, Al Ozonoff a,e, Shibei Zhao a, Adam J. Rose a,c

a Center for Health Quality, Outcomes, and Economic Research, Bedford VA Medical Center, Bedford, MAb Department of Health Policy and Management, Boston University School of Public Health, Boston, MAc Department of Medicine, Section of General Internal Medicine, Boston University School of Medicine, Boston, MAd Department of Quantitative Health Sciences, Division of Biostatistics and Health Services Research, University of Massachusetts School of Medicine, Worcester, MAe Clinical Research Center, Boston Children's Hospital, Boston, MA

Abbreviations: ARISTOTLE, Apixaban for Reduction iembolic Events in Atrial Fibrillation Trial; CMS, CenteServices; CPT, Current Procedural Terminology; EHICD-9-CM, International Classification of Diseases, Clinictional Society of Thrombosis and Haemostasis; PPV, PoRandomized Evaluation of Long-Term Anticoagulation TOnce daily oral direct Factor Xa inhibition Comparedthe prevention of stroke and Embolism Trial inXimelagatran Versus Warfarin for Stroke Prevention in Pal Fibrillation Trial; VA, Veterans Health Administration⁎ Corresponding author at: Center for Health Quality,

search, Bedford VA Medical Center, 200 Springs Road, Be687 2555; fax: +1 917 591 3104.

E-mail address: guneetk@bu.edu (G.K. Jasuja).

0049-3848/$ – see front matter © 2012 Elsevier Ltd. Allhttp://dx.doi.org/10.1016/j.thromres.2012.10.010

a b s t r a c t

a r t i c l e i n f o

Article history:

Received 12 July 2012Received in revised form 29 September 2012Accepted 22 October 2012Available online 13 November 2012

Keywords:anticoagulantshemorrhageoutcome assessmentmethods

Introduction: Identifying major bleeding is fundamental to assessing the outcomes of anticoagulation therapy.This drives the need for a credible implementation in automated data for the International Society of Throm-bosis and Haemostasis (ISTH) definition of major bleeding.Materials and Methods: We studied 102,395 patients who received 158,511 person-years of warfarin treat-ment from the Veterans Health Administration (VA) between 10/1/06-9/30/08. We constructed a list ofICD-9-CM codes of “candidate” bleeding events. Each candidate event was identified as a major hemorrhageif it fulfilled one of four criteria: 1) associated with death within 30 days; 2) bleeding in a critical anatomicsite; 3) associated with a transfusion; or 4) was coded as the event that precipitated or was responsible forthe majority of an inpatient hospitalization.Results: This definition classified 11,240 (15.8%) of 71, 338 candidate events as major hemorrhage. Typically,

events more likely to be severe were retained at higher rates than those less likely to be severe. For example,Diverticula of Colon with Hemorrhage (562.12) and Hematuria (599.7) were retained 46% and 4% of the time,respectively. Major, intracranial, and fatal hemorrhage were identified at rates comparable to those found inrandomized clinical trials however, higher than those reported in observational studies: 4.73, 1.29, and 0.41per 100 patient years, respectively.Conclusions: We describe here a workable definition for identifying major hemorrhagic events from large au-tomated datasets. This method of identifying major bleeding may have applications for quality measurement,quality improvement, and comparative effectiveness research.

© 2012 Elsevier Ltd. All rights reserved.

Introduction

Hemorrhage is a frequent complication of anticoagulant therapy[1]. Therefore, understanding rates of bleeding in patients receivinganticoagulation is an essential ingredient in assessing the risks and

n Stroke and Other Thrombo-rs for Medicare and MedicaidR, Electronic Health Record;al Modification; ISTH, Interna-sitive Predictive Value; RELY,rial; ROCKET-AF, Rivaroxabanwith vitamin K antagonist forAtrial Fibrillation; SPORTIF,atients With Nonvalvular Atri-.Outcomes, and Economic Re-dford, MA 01730. Tel.: +1 781

rights reserved.

benefits of such therapy. The ability to assess rates of bleeding is im-portant for diverse applications, including comparative effectivenessstudies of different anticoagulants and efforts to measure and im-prove quality of anticoagulation care.

A common approach to detecting any diagnosis, especially in ad-ministrative datasets, involves using International Classification ofDisease (ICD-9-CM) codes. In examining the accuracy of ICD-9-CMcodes to identify complications of anticoagulation therapy, Arnasonand colleagues reported a PPV of 87% for major bleeding, which in-creased to 96% when the bleeding code was listed as the ‘mostresponsible diagnosis’ or the ‘admitting diagnosis’ [2]. This studysuggests that automated data alone can be sufficient to identifytrue-positive episodes of major hemorrhage rivaling that of chart re-view, particularly when additional strategies are employed to boostPPV.

The majority of previous studies assessing the accuracy of ICD-9-CM codes for identifying major hemorrhage have used chart reviewas a gold standard [2–6]. Additionally, the most prominent definition

32 G.K. Jasuja et al. / Thrombosis Research 131 (2013) 31–36

of major bleeding provided by the International Society of Thrombo-sis and Haemostasis (ISTH) also presupposes chart review [7]. Whilechart review is highly accurate, it is resource intensive. A national da-tabase of Medicare beneficiaries, for example, would require chart re-views at every hospital in the United States, which is clearly notpractical. To fully harness the power of large databases to informpractice, we cannot always rely upon chart review. To our knowledge,there have been no previous attempts to adapt the ISTH definition ofmajor hemorrhage for use with automated data.

Therefore, the objective of our study was to develop a method foridentifying major hemorrhagic events in a linked Veterans Health Ad-ministration (VA)-Medicare dataset [8]. We explored the impact ofdifferent definitions and strategies upon the apparent incidence ofmajor hemorrhage in this automated dataset and compared therates found in our study with those reported in previous randomizedand observational studies. We expected that the results of this effortwould pave the way to unlocking the power of large automateddatasets to track major hemorrhage for comparative effectiveness re-search and quality assurance/quality improvement.

Methods

Data

The data for this study included all 122,159 patients who receivedwarfarin therapy from the VA between 10/1/06 and 9/30/08, includingpatients new to warfarin and those who were already experiencedusers. Details regarding how we built this database appear elsewhere[9]. This included demographics, ICD-9-CM diagnosis codes, and datesof service (from both sources), as well as laboratory data and pharmacyrecords (VA data only). This studywas approved by the Institutional Re-view Board of the Bedford VA Medical Center.

Since we used a merged VA-Center for Medicare and Medicaid Ser-vices (CMS) dataset, we can be assured of nearly complete capture ofall relevant care received by this group [8]. Out of these 122,159,19,764 patients enrolled in Medicare Advantage programwere exclud-ed, because unlike patients participating in this capitated programwould not produce itemized claims data, leading to undercounting ofevents. After excluding patients enrolled in Medicare Advantage, ourfinal sample consisted of 102,395 patients who received 158,511person-years of warfarin treatment from the VA.

For this study, we defined a period when each patient was consid-ered to be “on warfarin” and therefore eligible to record a bleedingevent. The date of warfarin inception is taken to be the first time war-farin is dispensed by the VA pharmacy or the first INR test, whichevercomes first. The period begins with the latter of date of warfarin in-ception and 10/1/06; it ends at the latest date of a pharmacy fill plus30-day grace period (to account for the duration of use) or an INR test,up to a maximum of 9/30/08. These 2 dates define a “window” foreach patient, and only bleeding events that occur during this windowcan be included.

Overview of Strategy to Identify Major Hemorrhage

Our strategy was to approximate the ISTH definition for major hem-orrhage as closely as possible [8], given the nature of our data. The ISTHdefinition uses the following criteria to define major bleeding: 1. Fatalbleeding, and/or, 2. Symptomatic bleeding in a critical area or organsuch as intracranial, and/or, 3. Bleeding causing a fall in hemoglobinlevel of 20 g L-1 or more, or leading to transfusion of two or moreunits of whole blood or red cells [7]. For the current study, candidatebleeding codes were identified from the inpatient and outpatientICD-9-CM codes recorded in VA and Medicare datasets. We requiredthese codes to fulfill at least one of four criteria to be considered amajor hemorrhagic event: 1) bleeding associated with death within30 days; 2) bleeding into a critical anatomic site which necessarily

would threaten life or limb; 3) bleeding associated with a transfusionof packed red blood cells or whole blood; or 4) a bleeding event charac-terized in our datasets as either the primary reason for a hospital admis-sion (VA data) or the main condition (commonly known as the“principal diagnosis”) for which the services are provided during a hos-pital stay (Medicare data).

ICD-9-CM Codes for Major Hemorrhage

We began by examining the lists of ICD-9-CM codes used by severalprevious studies to identify major hemorrhage, including Schalekampet al. [5], Boulanger [6], and Arnason [2]. Thus, we beganwith a compre-hensive list of candidate codes building on prior research. Several codeswere subsequently excluded as they were felt to not be representativeof major hemorrhage, e.g. 593.81, Vascular Disorders of the Kidney.

Deletion of Duplicate Events

To avoid counting multiple mentions of a single event, we devel-oped decision rules for selecting a single record out of multiple re-cords close in time. For example, this might occur if the course ofcare for a bleeding event involved 2 locations, such as a transferfrom a non-VA hospital to a VA hospital. We considered records to de-note a single event if they occurred on the same day or within 7 daysin either direction and used the same 5-digit ICD-9-CM code. We alsodeveloped an algorithm to determine which event to retain when twoconflicted; details can be found in Appendix A.

Finally, we developed an algorithm to assign a “primary” type ofbleeding when codes for different types of bleeding occurred on thesame date. We created five main categories of bleeding, namely intra-cranial bleeding, gastrointestinal bleeding, hemarthrosis, urinarybleeding, and bleeding from the throat. It was uncommon to havebleeding codes frommore than one category on the same day. For pa-tients with multiple codes within a category, we created a ranked hi-erarchy of codes within category to aid in selecting a single code foreach episode. Details for this algorithm are found in Appendix B.

Definition: Fatal Bleeding

For the outcome of death and its date, we used the VA Vital StatusMini-File. This dataset combines multiple sources of data, includingthe national death index, Medicare data, and VA data to determine asingle best date of death for each VA patient. The Vital StatusMini-File is considered a reliable and authoritative source for datesof death among VA patients [10].

We defined fatal hemorrhage as a bleeding event followed bydeath within 30 days. However, we excluded certain categories ofbleeding that were implausible causes of death, including epistaxis,hematuria, bladder wall hemorrhage, hemarthrosis, and any bleedingassociated with internal or external hemorrhoids. Patients who diedwithin 30 days of such events were not considered to have died be-cause of the bleeding.

Definition: Critical Anatomic Site

Bleeding into a critical anatomic site was defined as major bleeding,because it would necessarily threaten life or limb. This included anytype of intracranial hemorrhage, hemopericardium, hemoperitoneum,and any type of hemarthrosis (which would threaten limb function).We could not include retroperitoneal hemorrhage, intraspinal, intraoc-ular, and intramuscular bleeding with compartment syndrome, whichare equally serious, because no ICD-9-CM code uniquely identifiesthese conditions. Additionally, to a large extent we have likely capturedthese relatively uncommon intraspinal and intramuscular bleedingepisodes using the all-purpose ICD-9-CM code “Hemorrhage NOS”(459.0).We also separately tabulated rates for intracranial hemorrhage,

Table 1Patient characteristics (n=102,395).

Variable Number (%), orMedian (IQR)

Female Gender 2,291 (2.2%)Median Age (IQR) 70 (60,78)Race/Ethnicity

Non-Hispanic White 85,987 (84.0%)Non-Hispanic Black 10,248 (10.0%)Hispanic 2,821 (2.8%)Asian 385 (0.4%)Native American 466 (0.5%)Other/Unknown 2,488 (2.4%)

Median Percent Poverty in Zip Code of Residence (IQR) 10.8 ( 6.7,16.1)Median Distance from Nearest VA Facility in Miles (IQR) 8.0 ( 3.7,17.3)Primary Indication for Warfarin

Atrial Fibrillation 61,967 (60.5%)Venous Thromboembolism 30,978 (30.3%)All Others Combined 9,450 (9.2%)

Date When First Began Warfarin Therapy0-1 Years Prior to 10/1/06 75,622 (73.8%)Between 10/1/06 and 9/30/07 16,576 (16.2%)Between 10/1/07 and 9/30/08 10,197 (10.0%)

Physical Comorbid ConditionsCancer (newly diagnosed) 7,472 (7.3%)Chronic Kidney Disease 13,968 (13.6%)Chronic Liver Disease 1,350 (1.3%)Chronic Lung Disease 29,867 (29.2%)Diabetes 39,771 (38.8%)Epilepsy 2,949 (2.9%)Congestive Heart Failure 31,738 (31.0%)

Mental Health ConditionsAlcohol Abuse 11,193 (10.9%)Bipolar Disorder 2,723 (2.7%)Dementia 5,110 (5.0%)Major Depression 23,300 (22.8%)Substance Abuse (non-Alcohol) 5,259 (5.1%)

Number of Non-Warfarin Medications0-7 41,946 (41.0%)8-11 32,522 (31.8%)12-15 17,844 (17.4%)16+ 10,083 (9.8%)

Hospitalized at least once during study 28,458 (27.8%)

33G.K. Jasuja et al. / Thrombosis Research 131 (2013) 31–36

a subset of critical-site bleeding defined by the ICD-9-CM codes 430,431, 432.0, 432.1, and 432.9.

Definition: Transfusion

We identified all transfusions of packed red blood cells or wholeblood that occurred within 30 days of a bleeding event. The numberof units transfused was not available. Transfusions were identifiedfrom surgical codes 99.0, 99.03, or 99.04 (Blood Transfusion, WholeBlood Transfusion, and Packed Cell Transfusion) or CPT codes36430, 36431, or 36440 (Blood Transfusion Service).

Definition: Primary/Principal Inpatient Diagnosis Code

In VA data, the “primary” diagnosis for each inpatient admission isthe ICD-9-CM code for the condition that is most responsible for thepatients' length of stay. In the CMS dataset, a “principal” code wasidentified as the diagnosis, condition, problem or other reason forthe admission/encounter/visit shown in the medical record to bechiefly responsible for the services provided. We considered anyICD-9-CM code that occupied the primary or the principal positionfor an inpatient stay to represent a major hemorrhage.

Statistical Analyses

We considered any candidate event which fulfilled at least one ofthe four criteria discussed above (fatal event, critical anatomic site,transfusion, primary reason for a hospitalization) to represent a majorhemorrhage. We also performed sensitivity analyses using an alterna-tive, more restrictive definition that omitted the criterion “primary di-agnosis code”. We compared, in tabular form, the proportion of eventsretained under each definition, stratified by ICD-9-CM code. For theseanalyses, we allowed multiple events and did not censor.

For age-stratified rates of bleeding, all events were classified bypatient's age as of Oct. 1, 2006, not age at time of event. Confidence in-tervals were obtained assuming events were Poisson-distributed. Allanalyses were performed using SAS version 9.2 (SAS Institute, Cary,NC). Drs. Rose and Jasuja and Mr. Reisman had full access to the studydata and guarantee the accuracy and completeness of the results.

Results

Patients

Descriptive characteristics of the patients are reported in Table 1.The study sample primarily consisted of White males, with a medianage of 70 years. Approximately 74% of the sample had been usingwarfarin for at least one year at the time of study inception. Patientsin this study had a high burden of comorbid illness; for example,31% had heart failure, 39% had diabetes, and 14% had chronic kidneydisease. The burden of mental health conditions was also consider-able: 23% of patients carried a diagnosis of depression, and 11% a di-agnosis of alcohol abuse or dependence. Twenty-eight percent ofpatients were hospitalized at least once during the two-year study.

Inclusion of Candidate Events

Table 2 lists the bleeding ICD-9-CM codes for only those codeswith at least 100 candidate events, and shows the number and ratesfor each of these bleeding event retained and the reason for their in-clusion. A full version of this table, which also includes rarer event types,is presented in Supplementary Table 1 (Online Appendix). Among 71,338unique events, 11,240 events (15.8%) were retained. Events that wouldseem on their face to be relatively severe were more often retained. Forexample, Diverticula of Colon with Hemorrhage (562.12), EsophagealHemorrhage (530.82) and Hematemesis (578.0) were retained 46%,

43% and 42% of the time, respectively. Many candidate events, however,were contributed by event types that would only rarely have truly severemanifestations; these events were retained at much lower rates. For ex-ample, there were 30,100 episodes of Hematuria (representing 42% ofcandidate events), but only 4% of them qualified as major hemorrhageby our definition. Similarly, there were 5,927 episodes of Epistaxis, butonly 6% were retained. Comparison of rates of inclusion and retentionfor our main definition and the more restrictive definition are presentedin Supplementary Table 2 (Online Appendix).

Population Rates of Major Hemorrhage

We computed event rates by age strata formajor hemorrhage, intra-cranial hemorrhage, and fatal hemorrhage for the entire population of102,395 patients who received 158,511 patient-years of warfarin treat-ment from the VA (Table 3). Major hemorrhage occurred with anoverall rate of 4.73 per 100 patient-years. The rate of intracranial hem-orrhage, 1.29, represented 27% of all major hemorrhages, and the rate offatal hemorrhage, 0.41, represented 9% of all major hemorrhages. Therewas an increasing rate of all 3 kinds of hemorrhage (anymajor, intracra-nial, and fatal) with increasing age. In particular, the rate of fatal hemor-rhage increased sharply above age 70.

Comparison of the Present Study to other Major Studies

Table 4 presents a comparison of the rates of major bleedingreported in previous observational and randomized studies with thepresent study. The rates for major bleeding reported in observational

Table 2Inclusion of candidate events by ICD-9-CM code. The total number of candidate events was 71,338, of which 11,240 were retained (15.8%). In this abbreviated version of the table,only event types with≥100 candidate events are shown; the full version of the table, including rarer event types, can be found in the online appendix (Supplementary Table 1).

ICD-9 – CM code Label Candidate Events Retained Primary Diagnosis⁎ Transfusion⁎ Fatal Event⁎†

Bleeding into Critical Anatomic Site430 Subarachnoid Hemorrhage 292 ALL ‡ 103 (35.3%) 6 (2.1%) 31 (9.7%)431 Intracerebral Hemorrhage 1,317 ALL ‡ 441 (33.5%) 31 (2.3%) 236 (16.1%)432.1 Subdural Hemorrhage 1,289 ALL ‡ 330 (25.6%) 29 (2.3%) 108 (7.4%)432.9 Intracranial Hemorrhage NOS 440 ALL ‡ 146 (33.2%) 7 (1.6%) 52 (11.1%)568.81 Hemoperitoneum 221 ALL ‡ 75 (33.9%) 31 (14%) 12 (5.0%)719.16 Hemarthrosis, Lower Leg 258 ALL ‡ 35 (13.6%) 8 (3.1%) N/A

Bleeding into Non-Critical Anatomic Site455.2 Internal Hemorrhoids Without Complications NEC 735 68 (9.2%) 52 (76.5%) 19 (27.9%) N/A455.5 External Hemorrhoids With Complications NEC 166 10 (6.0%) 5 (50.0%) 5 (50.0%) N/A455.8 Hemorrhoids NOS With Complications NEC 232 14 (6.0%) 10 (71.4%) 4 (28.6%) N/A459.0 Hemorrhage NOS 1,512 231 (15.3%) 130 (56.3%) 78 (33.8%) 52 (22.5%)530.7 Mallory-Weiss Syndrome 182 63 (34.6%) 33 (52.4%) 32 (50.8%) 4 (6.3%)535.01 Acute Gastritis With Hemorrhage 111 42 (37.8%) 25 (59.5%) 21 (50.0%) 6 (14.3%)535.41 Gastritis NEC With Hemorrhage 204 81 (39.7%) 39 (48.2%) 42 (51.9%) 9 (11.1%)535.51 Gastritis/Duodenitis NOS With Hemorrhage 244 66 (27.0%) 33 (50.0%) 32 (48.5%) 4 (6.1%)537.83 Angiodysplasia Stomach/Duodenum With Hemorrhage 178 82 (46.1%) 33 (40.2%) 54 (65.9%) 2 (2.4%)562.12 Diverticula of Colon With Hemorrhage 552 253 (45.8%) 163 (64.4%) 99 (39.1%) 8 (3.2%)562.13 Diverticulitis of Colon With Hemorrhage 155 62 (40.0%) 48 (77.4%) 15 (24.2%) 3 (4.8%)569.3 Rectal And Anal Hemorrhage 5,911 616 (10.4%) 485 (78.7%) 175 (28.4%) 65 (10.6%)569.85 Angiodysplasia with Hemorrhage NEC 197 79 (40.1%) 41 (51.9%) 41 (51.9%) 3 (3.8%)578.0 Hematemesis 739 313 (42.3%) 220 (70.3%) 86 (27.5%) 65 (20.8%)578.1 Blood In Stool 7,866 975 (12.4%) 725 (74.4%) 280 (28.7%) 106 (10.9%)578.9 Hemorrhage of Gastrointestinal Tract NOS 6,837 1,898 (27.8%) 1,390 (73.2%) 583 (30.7%) 308 (16.2%)596.7 Bladder Wall Hemorrhage 105 18 (17.1%) 9 (50.0%) 10 (55.6%) N/A599.7 Hematuria 30,100 1229 (4.1%) 1070 (78.3%) 366 (30.0%) N/A782.7 Spontaneous Ecchymoses 509 28 (5.5%) 18 (64.3%) 6 (21.4%) 5 (17.9%)784.7 Epistaxis 5,927 346 (5.8%) 264 (76.3%) 105 (30.3%) N/A786.3 Hemoptysis 4,370 532 (12.2%) 326 (61.3%) 106 (19.9%) 150 (28.2%)

⁎ These reasons for inclusion are not mutually exclusive.† N/A for some diagnoses that were not considered a plausible direct cause of death.‡ Bleeding into a critical anatomic site was always included as a major hemorrhage.

Table 3Population rates of major hemorrhage and subtypes of major hemorrhage by age category,among patients receiving warfarin therapy from the VA between 10/1/06-9/30/08.

Age atStart ofWindow

#Patients

AggregateTime on Therapy(patient-years)

Rate of Hemorrhagic Events(per 100 patient-years with 95% C.I.)

Any Major Intracranial⁎ Fatal

Total 102,395 158,511 4.73(4.62 - 4.83)

1.29(1.23 - 1.35)

0.41(0.38 - 0.45)

Under 55 9,865 13,958 2.29(2.04 - 2.55)

0.63(0.51 - 0.78)

0.10(0.05 - 0.17)

55 – 59 14,185 21,169 2.96(2.73 - 3.20)

1.01(0.88 - 1.15)

0.14(0.09 - 0.20)

60 – 64 14,436 21,882 2.94(2.72 - 3.18)

0.85(0.74 - 0.99)

0.19(0.13 - 0.25)

65 – 69 11,120 17,644 3.89(3.60 - 4.19)

1.00(0.86 - 1.16)

0.27(0.20 - 0.35)

70 – 74 15,001 24,105 5.26(4.97 - 5.55)

1.49(1.34 - 1.65)

0.42(0.34 - 0.51)

75 – 79 15,689 25,206 6.12(5.82 - 6.43)

1.60(1.45 - 1.77)

0.54(0.46 - 0.64)

80 – 84 15,095 23,895 6.70(6.37 - 7.03)

1.80(1.63 - 1.98)

0.72(0.62 - 0.84)

85+ 7,004 10,652 7.56(7.04 - 8.10)

1.76(1.51 - 2.03)

1.09(0.90 - 1.31)

Patients may experience >1 major hemorrhage during the study.⁎ ICD-9-CM codes 430, 431, 432.0, 432.1, 432.9.

34 G.K. Jasuja et al. / Thrombosis Research 131 (2013) 31–36

cohorts such as ATRIA (0.91 per 100 person-years) and ACTION (1.90per 100 person-years) were much lower than what we found in ourstudy. The rates for major bleeding found in randomized clinical trialsranged from 2.2 per 100 person-years (ACTIVE W, 2006) to 3.4 per-cent per year (ROCKET-AF, 2011). Rates of bleeding from earlier stud-ies (as reflected in the Linkins meta-analysis) tended to be higher(the cumulative rate of all included studies was 7.2 percent per year).

Discussion

The ability to perform surveillance for adverse events is a key foun-dation for any program of comparative effectiveness research or qualitymeasurement and improvement. Major hemorrhage is an adverseevent common to all types of anticoagulant therapy, and therefore theability to identify major hemorrhage in real-time, using an automatedapproach,would be extremely attractive. In this study, we sought to de-velop a comprehensive definition for identifying major hemorrhagicevents amongwarfarin patients in a large, automated database, in a set-ting where chart review would not be possible.

In our population of patients receiving warfarin therapy for variedindications from the VA, we found an overall rate of major hemor-rhage of 4.73 events/100 patient-years. The rates of intracranial andfatal hemorrhage were 1.29 and 0.41 events/100 patient-years, re-spectively. These rates of warfarin-related bleeding are similar inmagnitude to the rates observed in recent large randomized trials,such as ROCKET-AF, ARISTOTLE, SPORTIF-V and RE-LY, as well as thepooled results of earlier randomized and observational studies sum-marized in a key meta-analysis by Linkins, et al. [1]. However, ourrates for major bleeding were much higher than those reported in ob-servational cohorts such as ACTION, ATRIA and Euro Heart Survey;this is likely attributable to a much higher illness burden among VApatients. Taken together, these results suggest that our approach to

identifying major hemorrhage in warfarin patients may achieve similarresults to other methods that have been employed. In addition, we notethat the risk of any major hemorrhage, intracranial hemorrhage, andfatal hemorrhage among this at-risk population all increased in ourstudy with increasing age, with a generally monotonic trend. This alsoechoes the findings of earlier studies [20–22] and further suggests

Table 4Comparison of the present study to other major studies regarding rates of major hemorrhage.

Studies Type of study N (Patient yrs) Major⁎ Intracranial⁎ Fatal⁎

ATRIA, 2003 [11] Observational 11536 (25341) 0.91 0.46 –

ACTION, 2008 [12] Observational 3396 (2892.1) 1.90 – –

Euro Heart Survey, 2010 [13] Observational 3978 1.50‡ – –

Linkins 2003 [1] Meta-analysis† 10757 (4374) 7.22 1.15 1.31SPORTIF III, 2003 [14] Randomized 3410 (4941) 1.8 – –

SPORTIF-V, 2005 [15] Randomized 3922 (6405) 3.1 – –

ACTIVE W, 2006 [16] Randomized 6706 2.2 – 0.26RE-LY, 2009 [17] Randomized 18,113 3.36 0.74 1.80ROCKET-AF, 2011 [18] Randomized 14,264 3.4‡ 0.7‡ –

ARISTOTLE, 2011 [19] Randomized 18,201 3.09‡ 0.80‡ –

Present Study Automated Database 102,359 (158,511) 4.73 1.29 0.41

⁎ Rates are expressed as events per 100 person-years.† The Linkins meta-analysis combined 29 randomized trials and 4 cohort studies, all published between 1990-2001.‡ Rates are expressed as percent per year.

35G.K. Jasuja et al. / Thrombosis Research 131 (2013) 31–36

that our algorithm to identify warfarin-related major hemorrhage isworking as intended.

The main limitation of this study is that we were not able to performchart review to confirm cases of major hemorrhage, in large part becausethe merged VA-Medicare dataset we used would have required us to ob-tain data from every hospital in the United States. However, as discussedabove, this study represents an important effort to find a rigorous andthoughtful approach to identifying warfarin-related major hemorrhagewhen a chart review is simply not possible. Our efforts borrowed heavilyfromprevious studies that hadused chart review as a gold standard [2–6].However, no study involving chart review could have ever included a da-tabase of this size, because of the expense involved in reviewing somanycharts. This study, therefore, represents an important effort to apply theresults of previous (smaller) studies, which did involve chart review, toa new, larger platform, onewhich is ultimately necessary if automatedda-tabases are to be used for such important purposes as comparative effec-tiveness and quality assurance. Because Medicare data, which largelyformed the basis of our study, is also the basis for so many other studies,our approach and our resultsmay havewide applicability. However, sinceour at-risk population comprised of only patients on warfarin, findingsfrom this study are only generalizable to patients on oral anticoagulanttherapy such as warfarin and may not apply, for example, to patients re-ceiving aspirin alone.

Our definition for major bleeding was actually fairly conservative,and it seems likely that we undercounted major bleeding if anything.The ISTH definition [7] includes a drop in hemoglobin level of 20 g L-1

to qualify an event as amajor hemorrhage.Wewere unable to preciselycharacterize the time course of the hemoglobin level, particularly whenusingMedicare data (which does not include lab results). Therefore, he-moglobin levels were not used as a criterion for major hemorrhages inour study, and this likely resulted in considerable undercounting ofevents that otherwise would have qualified.

On the other hand, it is possible that our definition overcounted cer-tain events; indeed, we cannot have avoided this entirely. In particular,our addition of a primary inpatient diagnosis code to the criteria formajor hemorrhage represents an innovation, and is not containedwith-in the original ISTH definition [7]. We considered this necessary as away to compensate partially for our inability to establish major hemor-rhage based on hemoglobin levels. In addition, this decision wassupported by the finding of Arnason, et al. [2] that ICD-9 codes for hem-orrhage have a 96% PPVwhen they are in the primary position for an in-patient stay. To us, this choice seemed like a logical way to capture atleast some of the events that would have been included had we beenable to track hemoglobin values. In addition, it can of course be arguedthat any bleeding that causes a hospitalization (or prolongs one) is, bydefinition, “major”. Thosewhoprefer to employ a definitionmore close-ly aligned to the original ISTH definition may wish to use our variantdefinition, which omitted this criterion (see Online Appendix for acomparison of the main and variant definitions). However, our main

definition filtered more severe types of events to a lesser extent andfiltered less severe sorts of events much more heavily. For example,"Diverticulosis of Colon with Hemorrhage" (562.13) was retained 47%of the time, while "Hematuria" (599.7) was retained only 4% of thetime. In this respect, we consider our approach a success in differentiat-ing between major and minor hemorrhagic events.

In conclusion, a definition of “warfarin-related major bleeding”,modeled on similar parameters as the ISTH definition and applied toan automated dataset, allowed for computation ofmeaningful and com-parable rates of hemorrhage. This method of extracting major hemor-rhage may also prove useful with other automated datasets similar toour VA-Medicare merged dataset, for applications including compara-tive effectiveness, quality measurement, and quality improvement.

Conflict of Interest Statement

None.

Acknowledgement of Funding Sources

This project is supported by VA-HSR&D-IIR-10-374. Dr. Rose issupported by a VA Career Development Award (HSR&D-CDA-2-08-017).The sponsor had no role in the design and conduct of the study; the col-lection, management, analysis, and interpretation of the data; and thepreparation, review, and approval of the manuscript.

Disclaimer: The opinions expressed in this manuscript do not nec-essarily represent the official views of the Department of VeteransAffairs.

Disclosures: Dr. Hylek has served on advisory boards for Bayer,Boehringer-Ingelheim, Bristol Myers Squibb, Daiichi Sankyo, Johnsonand Johnson, Merck, and Pfizer. None of the other authors report anypotential conflicts of interest.

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.thromres.2012.10.010.

References

[1] Linkins L, Choi PT, Douketis JD. Clinical impact of bleeding in patients taking oralanticoagulant therapy for venous thromboembolism. Ann Intern Med 2003;139:893-900.

[2] Arnason T, Wells PS, van Walraven C, Forster AJ. Accuracy of coding for possiblewarfarin complications in hospital discharge abstracts. Thromb Res 2006;118:253-62.

[3] Andrade SE, Gurwitz JH, Chan KA, Donahue JG, Beck A, BolesM, et al. Validation of diag-noses of peptic ulcers and bleeding from administrative databases: a multi-healthmaintenance organization study. J Clin Epidemiol Mar 2002;55(3):310-3.

[4] Cattaruzzi C, Troncon MG, Agostinis L, García Rodríguez LA. Positive predictivevalue of ICD-9th codes for upper gastrointestinal bleeding and perforation in

36 G.K. Jasuja et al. / Thrombosis Research 131 (2013) 31–36

the Sistema Informativo Sanitario Regionale database. J Clin Epidemiol Jun1999;52(6):499-502.

[5] Schalekamp T, Klungel OH, Souverein PC, Boer A. Effect of oral antiplatelet agentson major bleeding in users of coumarins. Thromb Haemost 2008;100:1076-83.

[6] Boulanger L, Hauch O, Friedman M, Foster T, Dixon D, Wygant G, et al. Warfarinexposure and the risk of thromboembolic and major bleeding events among Med-icaid patients with atrial fibrillation. Ann Pharmacother 2006;40:1024-9.

[7] Schulman S, Kearon C. Definition of major bleeding in clinical investigations ofantihemostatic medicinal products in non-surgical patients. J Thromb Haemost2005;3:692-4.

[8] HynesDM,KoellingK, Stroupe K, ArnoldN,Mallin K,Min-Woong Sohn, et al. Veterans’Access to andUse ofMedicare andVeterans Affairs Health Care.Med Care 2007;45(3):214-23.

[9] Rose AJ, Hylek EM, Ozonoff A, Ash AS, Reisman JI, Berlowitz DR. Patient characteristicsassociated with oral anticoagulation control: Results of the Veterans AffaiRs Study toImprove Anticoagulation (VARIA). J Thromb Haemost Oct 2010;8(10):2182-91.

[10] Arnold N, Min-Woong Sohn, Maynard C, Hynes DM. VA-NDI Mortality Data MergeProject. VIReC Technical Report 2. Illinois: VA Information Resource Center; 2006.

[11] Go AS, Hylek EM, Chang Y, Phillips KA, Henault LE, Capra AM, et al. Anticoagulationtherapy for stroke prevention in atrial fibrillation: how well do randomized trialstranslate into clinical practice? JAMA Nov 26 2003;290(20):2685-92.

[12] Rose AJ, Ozonoff A, Henault LE, Hylek EM. Warfarin for atrial fibrillation incommunity-based practise. J Thromb Haemost Oct 2008;6(10):1647-54.

[13] Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJ, Lip GY. A novel user-friendlyscore (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrialfibrillation: the Euro Heart Survey. Chest Nov 2010;138(5):1093-100.

[14] SPORTIF-III Investigators. Stroke prevention with the oral direct thrombin inhibi-tor ximelagatran compared with warfarin in patients with non-valvular atrial

fibrillation (SPORTIF III): randomised controlled trial. Lancet 2003;362(9397):1691-8.

[15] Albers GW, Diener HC, Frison L, Grind M, Nevinson M, Partridge S, et al.Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrialfibrillation: a randomized trial. JAMA Feb 9 2005;293(6):690-8.

[16] ACTIVE Writing Group of the ACTIVE InvestigatorsConnolly S, Pogue J, Hart R,Pfeffer M, Hohnloser S, et al. Clopidogrel plus aspirin versus oral anticoagulationfor atrial fibrillation in the Atrial fibrillation Clopidogrel Trial with Irbesartan forprevention of Vascular Events (ACTIVE W): a randomised controlled trial. LancetJun 10 2006;367(9526):1903-12.

[17] Connolly SJ, Ezekowitz MD, Yusuf S, Eikelboom J, Oldgren J, Parekh A, et al.Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med Sep17 2009;361(12):1139-51.

[18] Patel MR, Mahaffey KW, Garg J, Pan G, Singer DE, Hacke W, et al. Rivaroxaban ver-sus warfarin in nonvalvular atrial fibrillation. N Engl J Med Sep 8 2011;365(10):883-91 [Epub 2011 Aug 10].

[19] Granger CB, Alexander JH, McMurray JJ, Lopes RD, Hylek EM, Hanna M, et al.Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med Sep 152011;365(11):981-92.

[20] Hylek EM, Singer DE. Risk factors for intracranial hemorrhage in outpatients tak-ing warfarin. Ann Intern Med 1994;120:897-902.

[21] Fang MC, Chang Y, Hylek EM, Rosand J, Greenberg SM, Go AS, et al. Advanced age,anticoagulation intensity, and risk for intracranial hemorrhage among patientstaking warfarin for atrial fibrillation. Ann Intern Med Nov 16 2004;141(10):745-52.

[22] Fang MC, Go AS, Hylek EM, Chang Y, Henault LE, Jensvold NG, et al. Age and therisk of warfarin-associated hemorrhage: the anticoagulation and risk factors inatrial fibrillation study. J Am Geriatr Soc Aug 2006;54(8):1231-6.