A paradigm of diagnostic criteria for polyarteritis nodosa: Analysis of a series of 949 patients with vasculitides

ARTHRITIS & RHEUMATISMVol. 58, No. 5, May 2008, pp 1528–1538DOI 10.1002/art.23470© 2008, American College of Rheumatology

A Paradigm of Diagnostic Criteria for Polyarteritis Nodosa

Analysis of a Series of 949 Patients With Vasculitides

Corneliu Henegar,1 Christian Pagnoux,2 Xavier Puechal,2 Jean-Daniel Zucker,3

Avner Bar-Hen,4 Veronique Le Guern,2 Mona Saba,2 Denis Bagneres,2 Olivier Meyer,5

and Loıc Guillevin,2 for the French Vasculitis Study Group

Objective. To establish a set of clinical andparaclinical criteria potentially useful as a diagnosticscreening tool for polyarteritis nodosa (PAN).

Methods. The abilities of individual descriptiveitems to predict a diagnosis of PAN were evaluatedby screening available data from 949 patients from theFrench Vasculitis Study Group database, including262 with PAN and 687 with control vasculitides. Se-lected items were tested in a logistic regression modelto establish a minimal set of nonredundant PAN-predictive criteria. The discriminative accuracy of theseitems and of the American College of Rheumatology(ACR) 1990 criteria were assessed by reapplying themto the initial patient sample and a subgroup restrictedto PAN and microscopic polyangiitis (MPA) patients. Acomputer simulation procedure was conducted on arti-ficially generated patient data to evaluate the usefulnessof these criteria in predicting a diagnosis of PAN.

Results. The analysis resulted in the retention of

3 positive predictive parameters (hepatitis B virus an-tigen and/or DNA in serum, arteriographic anomalies,and mononeuropathy or polyneuropathy) and 5 negativepredictive parameters (indirect immunofluorescencedetection of antineutrophil cytoplasmic antibody; asth-ma; ear, nose, and throat signs; glomerulopathy; andcryoglobulinemia) for the criteria set. These criteriayielded 70.6% sensitivity for all control vasculitides and89.7% for MPA controls, with 92.3% specificity for allcontrols and 83.1% for MPA controls. The discriminantabilities of this set of items outperformed the ACR 1990criteria in all analytical situations, showing better ro-bustness to variations in the prevalence of individualvasculitides.

Conclusion. The use of positive and negativediscriminant criteria could constitute a sound basis fordeveloping a diagnostic tool for PAN to be used byclinicians. Further prospective analyses and validationsin different populations are needed to confirm theseitems as satisfactory diagnostic criteria.

The systemic vasculitides are a heterogeneousgroup of diseases that have blood vessel inflammation asa common trait. In 1990, relying on many previousefforts, the American College of Rheumatology (ACR)proposed a set of classification criteria that were se-lected by a panel of experts based on an analysis ofmulticenter patient data (1). Seven distinct entities werethus characterized, including polyarteritis nodosa (PAN)(2). Although the primary purpose of the ACR 1990criteria was to standardize the classification of vasculitispatients to facilitate communication among researchers(3), their good discriminative accuracy indicated by theinitial assessments, with sensitivities of 71–94% andspecificities of 87–92% for different types of vasculitides,

1Corneliu Henegar, MD, PhD: Hopital Cochin, Paris 5-ReneDescartes University, Assistance Publique Hopitaux de Paris, andINSERM, UMR-S 872, Les Cordeliers, Eq. 7 Nutriomique, Paris,France; 2Christian Pagnoux, MD, Xavier Puechal, MD, PhD,Veronique Le Guern, MD, Mona Saba, MD, Denis Bagneres, MD,Loıc Guillevin, MD: Hopital Cochin, Paris 5-Rene Descartes Univer-sity, Assistance Publique Hopitaux de Paris, Paris, France; 3Jean-Daniel Zucker, PhD: INSERM, UMR-S872, Les Cordeliers, Eq. 7Nutriomique, Paris, and IRD, Unite de Recherche Geodes, CentreIRD de l’Ile de France, Bondy, France; 4Avner Bar-Hen, PhD: Paris5-Rene Descartes University, MAP5, UMR CNRS 8145, Paris,France; 5Olivier Meyer, MD: Centre Hospitalier Universitaire Bichat,Paris 7 University, Assistance Publique Hopitaux de Paris, Paris,France.

Address correspondence and reprint requests to Loıc Guil-levin, MD, Department of Internal Medicine, Hopital Cochin, Paris5-Rene Descartes University, Assistance Publique Hopitaux de Paris,27 Rue du Faubourg St. Jacques, Paris F-75014, France. E-mail:[email protected].

Submitted for publication May 11, 2007; accepted in revisedform February 11, 2008.

1528

suggested a potential usefulness for diagnostic predic-tion (4).

Subsequent evaluations of the ACR 1990 criteriasets used for diagnostic screening under routine clinicalconditions yielded inconsistent and unsatisfactory re-sults, with positive predictive values ranging from 17%to 75% for different types of vasculitides and highpercentages of false-positive diagnoses (4). Further as-sessments, which extended to other classification sys-tems, such as the Chapel Hill Nomenclature (5,6), wereequally disappointing, highlighting the necessity of de-veloping separate criteria for classification and diagnos-tic purposes (7).

We conducted a prospective analysis of theFrench Vasculitis Study Group (FVSG) patient data-base with the intention of evaluating the feasibility ofestablishing a minimal set of predictive clinical andparaclinical features that could serve as a screening toolfor the diagnosis of PAN in situations in which theclinical picture is suggestive of systemic vasculitis. Be-cause the ACR 1990 criteria do not distinguish betweenPAN and microscopic polyangiitis (MPA), which wasformally defined by the Chapel Hill Consensus Confer-ence Nomenclature of systemic vasculitides in 1994 (8),a secondary objective of our analysis was to achievebetter discrimination between PAN and MPA.

PATIENTS AND METHODS

Analytical approach. The analytical design comprised2 distinct stages. The first stage was to select a minimum set oflow-redundant items of positive and/or negative predictivevalue for PAN from among those exhibiting the highestindividual accuracy in distinguishing PAN from other systemicvasculitides. The selection of this set relied on clinical judg-ment supported by a combination of univariate and multi-variate statistical analyses of clinical and paraclinical itemsused to describe characteristics of patients in the FVSGdatabase. During the second stage, the PAN-predictive abili-ties of the selected set of criteria were evaluated through anunsupervised computer simulation procedure designed to re-produce the case-based aspect of clinical diagnostic reasoning.During both analytical stages, the selected items were com-pared with the ACR 1990 criteria for the classification of PAN(2), which is considered to be the most reliable referencecriteria to date.

Analysis of the capabilities of the available clinical andparaclinical items to predict PAN. The FVSG patient databasecontains a large set of clinical and paraclinical items designedto describe the characteristics of the systemic vasculitides witha good level of detail. Our analysis relied on patient informa-tion extracted from the FVSG database, after filtering formissing items and secondary vasculitides associated with othersystemic diseases, such as rheumatoid arthritis and systemiclupus erythematosus. This procedure selected 949 patients

with primary systemic vasculitides for which definitive diagnos-tic evidence was available. In all cases, the diagnoses werebased on compatible clinical manifestations, biochemical para-meters, including results of antineutrophil cytoplasm antibody(ANCA) testing, histologic analysis, and when available, angio-graphy. Histologic confirmation of the clinical diagnosis reliedon the same elements as used in the ACR 1990 analysis (1,2)and was a mandatory selection criterion. This criterion wasused only as diagnostic reference to ensure a reliable assess-ment of the available clinical and paraclinical items.

The selected sample of 949 patients had the followingdistribution of vasculitis types: 262 (27.6%) had PAN, amongwhom 108 had hepatitis B virus (HBV)–related PAN (41.2% ofall PAN), 256 (27%) had Wegener’s granulomatosis (WG),207 (21.8%) had MPA, 150 (15.8%) had Churg-Strauss syn-drome (CSS), 18 (1.9%) had cryoglobulin-associated vasculitis,and 56 (5.9%) had other primary systemic vasculitides.

Analysis of these data in our search for a minimal setof low-redundant PAN-predictive items, was conducted in 2steps. During the first step, the entire list of over 100 clinicaland paraclinical items used to describe the characteristics ofthe patients in the FVSG database, including all of the ACR1990 criteria, was subjected to univariate analysis to assess theindividual discriminative value of each available feature.Among these items, the presence of HBV surface antigen, oneof the ACR 1990 criteria, was replaced by markers reflectingactive HBV replication, such as the detection of hepatitis Benvelope antigen and/or DNA of �105 copies/ml in serum (9).An indirect immunofluorescence assay was used to test forANCA according to the recommendations of the EuropeanVasculitis Study Group (10). Because the ANCA specificity(myeloperoxidase or proteinase 3) was not systematically de-termined, it was ignored.

The univariate analysis assessed the strength of indi-vidual associations between the diagnosis of PAN and theavailable clinical or paraclinical features by relying on a nor-malized, pairwise, mutual information measure, which is awell-established entropic approach for quantifying mutualdependence (e.g., positive or negative) between variables (11).A short presentation of the mutual information measure,together with other formal aspects pertaining to the analyticalstrategy of the study, is provided in Supplementary materialsposted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm. The univariate analysis en-abled quantification of the usefulness of the information pro-vided by each clinical and paraclinical item for diagnosing PANand allowed the ranking of available items by the decreasingorder of their PAN-predictive value.

To establish a minimal set of low-redundant PAN-predictive criteria, we conducted an exploratory multivariateanalysis that relied on available parameters to build a logisticregression model through a forward inclusion approach basedon the R2 criterion. The inclusion procedure was directed byclinical judgment and by the PAN-predictive value of individ-ual items, as determined by the univariate analysis, and wasreiterated until the logistic regression model could no longerbe significantly improved by further inclusion of additionalitems. This analysis was conducted in 2 distinct situations, onein which all non-PAN vasculitides were considered as controlsand the other in which controls were restricted to MPA, inorder to favor the selection of discriminant items capable of

DIAGNOSTIC CRITERIA FOR PAN 1529

distinguishing between PAN and MPA, which was not differ-entiated by the ACR 1990 analysis. Also, taking into consid-eration the current trend toward a reduction in the incidenceof HBV infection as a result of systematic vaccination pro-grams as well as its diminishing association with PAN, thediscriminant performance of these criteria was evaluated sep-arately in a subgroup of vasculitides that included only HBV-negative PAN.

The resulting FVSG set of criteria, which were selectedby multivariate analysis, was further used to derive a set ofrelevant positive and negative association rules based on theimplementation (12) of a decision-tree inference algorithm(13) designed to optimize the ratio between the accuracy ofdiagnostic prediction and its cost (e.g., the number of requireditems). This analysis was undertaken to devise a tool for futureclinical use of this set of criteria by exposing the most relevantassociation rules between selected items. Finally, the PAN-predictive accuracy of the FVSG set of criteria and the ACR1990 classification criteria were comparatively assessed interms of sensitivity and specificity by receiver operating char-acteristic (ROC) curve analysis (14) in each of the above-mentioned analytical situations.

Computer simulation of the PAN-predictive abilities ofthe FVSG criteria and the ACR 1990 criteria. During thesecond analytical stage, computer simulations were run toevaluate the PAN-predictive abilities of the 2 sets of criteriaunder various conditions, which were simulated through arti-ficially generated vasculitis patient data. The computer simu-lation procedure was designed to reproduce the case-basedaspect of medical diagnostic reasoning, which attempts toascribe unambiguous labels (e.g., corresponding to distinctpathologic entities) to clusters of cases with similar clinical andparaclinical features. The aim of this simulation was to test thedependence of the PAN-predictive performances of thesecriteria sets on the prevalence of individual vasculitides in the

patient samples analyzed. The Boolean aspect of the presenceor absence of clinical and paraclinical features in vasculitispatients suggested the possibility of relying on a model ofaggregated dependent Bernoulli trials to represent the realjoint distributions of the clinical and paraclinical parametersspecific for each form of vasculitis. To ensure good reproduc-ibility of the computer-simulation results, we considered 2distinct approaches to quantifying and expressing marginaldistributions and dependencies between individual para-meters.

The first approach relied on the Bahadur-Lazarsfeldtheoretical framework, which computes a complete represen-tation of the joint distribution of a set of n correlated Bernoullitrials (e.g., corresponding to n clinical or paraclinical items)through an expansion of a binomial law (15). A short formalpresentation of the Bahadur-Lazarsfeld framework is providedin Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm.Despite its good theoretical accuracy, a major drawback of theBahadur-Lazarsfeld expansion is its requirement of a highnumber of dependency parameters (e.g., correlation para-meters expressing dependencies between Bernoulli trials fromthe second order to the nth order), which challenges thecomputational tractability of the model, even for a moderatenumber of trials. These considerations suggested the useful-ness of a recently proposed theoretical solution, which relieson the maximum entropy principle to optimize the inference ofmissing parameters of a truncated Bahadur-Lazarsfeld expan-sion (e.g., which considers as input parameters only marginalprobabilities and second-order correlations) (16), thereby al-lowing for a highly precise reproduction of the clinical andparaclinical characteristics of real vasculitis patients in artifi-cially generated data. The required marginal probabilities andthe second-order correlations between clinical and paraclinical

Table 1. The FVSG minimal set of low-redundant PAN-predictive criteria derived from analysis of the FVSG patientdatabase*

Criterion PAN association† Definition

1. HBV infection Positive Markers reflecting active HBV replication, such as the presenceof HBeAg in serum and/or the detection of HBV DNA at�105 copies/ml

2. ANCA positivity Negative Presence of ANCA in serum, as determined by indirectimmunofluorescence

3. Asthma Negative Personal antecedents of asthma4. ENT signs Negative Signs of maxillary sinusitis or otitis media5. Cryoglobulin positivity Negative Detection of cryoglobulins in serum6. Glomerulopathy Negative Signs of glomerulopathy, such as proteinuria and/or hematuria,

with or without renal insufficiency, not due to urinary tractinfection, urolithiasis, or hematologic or other nonglomerularcauses

7. Arteriographic anomalies Positive Arteriogram showing aneurysms or occlusions of the visceralarteries, not due to arteriosclerosis, fibromuscular dysplasia,or other noninflammatory causes

8. Mono-/polyneuropathy Positive Development of mononeuropathy, multiple mononeuropathies,or polyneuropathy

* HBV � hepatitis B virus; HBeAg � hepatitis B envelope antigen; ANCA � antineutrophil cytoplasmic antibody; ENT � ear,nose, and throat.† Significant positive or negative association with a diagnosis of polyarteritis nodosa (PAN) among patients in the FrenchVasculitis Study Group (FVSG) database.

1530 HENEGAR ET AL

items were computed from the patient sample extracted fromthe FVSG database.

The second approach used to generate artificial patientdata relied on a maximum-spanning tree (MST) dependence-modeling technique, which approximates dependencies be-tween individual parameters by arbitrarily limiting them tothose expected to have the most impact on the results (17,18).The principle of the MST dependence-modeling techniqueresides in estimating the joint distribution of item presenceby relying on an MST representation of their strongest inter-dependencies. Further details are provided in Supplementarymaterials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm. The marginal probabili-ties of clinical and paraclinical items and the pairwise mutualinformation coefficients, which are required by the MSTapproach, were computed from the FVSG patient sample.

The generation of artificial vasculitis patient dataaimed to reflect 2 types of situations. The first was a referencesituation, in which patient data were artificially generated byusing either the maximum entropy correction of a truncatedBahadur-Lazarsfeld expansion or the MST method to simulateintricate interdependencies between various clinical and para-

clinical parameters seen in the FVSG patient database. In thissituation, the relative frequencies of the 4 main types ofvasculitides represented within artificially generated patientsamples were chosen to reflect the prevalence reported in theFrench population: 34.03% PAN, 27.83% MPA, 26.27% WG,and 11.86% CSS (19). In the second situation, the relativefrequencies of PAN cases were modified to test the effect ofvariations in vasculitis prevalence on the predictive perfor-mances of the 2 sets of criteria. To this end, the relativefrequency of PAN patients was arbitrarily reduced to 10% ofall generated cases, while the relative frequencies of the other3 vasculitides were increased to 30% each.

The artificial patient data thus generated were furtherused in a computer simulation to evaluate the usefulness of the2 sets of criteria in screening potential vasculitis patients for apositive diagnosis of PAN. To achieve this goal, we relied on acombination of an unsupervised hierarchical clustering ap-proach, used to group artificially generated cases based on thesimilarities of their clinical and paraclinical profiles, and asupervised labeling procedure, which assigns to each resultingcluster of similar cases the true label of the cases that form themajority of its content.

Table 2. Discriminant performance of a minimal set of low-redundant predictive items of the FVSGcriteria for distinguishing PAN from other systemic vasculitides or MPA among patients in the FVSGdatabase, considering all PAN cases and considering only HBV-negative PAN cases*

FVSG discriminant item Odds ratio (95% CI)† R2‡

All PAN casesPAN versus other vasculitides

1. (�) HBV infection (active replication) 16.41 (7.34–36.68) 0.3232. (–) ANCA positivity 0.04 (0.02–0.08) 0.5393. (–) Asthma 0.10 (0.04–0.21) 0.5984. (–) ENT signs§ 0.09 (0.03–0.25) 0.6295. (–) Cryoglobulin positivity 0.13 (0.04–0.40) 0.6446. (–) Glomerulopathy 0.36 (0.22–0.58) 0.6567. (�) Arteriographic anomalies 3.48 (1.68–7.22) 0.6688. (�) Mono-/polyneuropathy 1.87 (1.19–2.94) 0.674

PAN versus MPA1. (–) ANCA positivity 0.03 (0.01–0.08) 0.5082. (�) HBV infection (active replication) 18.40 (6.10–55.51) 0.6073. (–) Glomerulopathy 0.19 (0.10–0.33) 0.6484. (�) Arteriographic anomalies 6.13 (2.13–17.65) 0.669

Only HBV-negative PAN casesHBV-negative PAN versus other vasculitides

1. (–) ANCA positivity 0.02 (0.01–0.07) 0.3362. (–) Asthma 0.10 (0.04–0.23) 0.4273. (–) ENT signs§ 0.10 (0.03–0.29) 0.4684. (–) Glomerulopathy 0.37 (0.23–0.61) 0.4855. (–) Cryoglobulin positivity 0.19 (0.06–0.56) 0.5036. (�) Arteriographic anomalies 3.18 (1.49–6.77) 0.515

HBV-negative PAN versus MPA1. (–) ANCA positivity 0.02 (0.01–0.07) 0.4952. (–) Glomerulopathy 0.18 (0.10–0.33) 0.5563. (�) Arteriographic anomalies 5.61 (1.86–16.89) 0.581

* Each discriminant item showed a significant positive (�) or negative (–) association with polyarteritisnodosa (PAN). FVSG � French Vasculitis Study Group; MPA � microscopic polyangiitis; HBV �hepatitis B virus; 95% CI � 95% confidence interval; ANCA � antineutrophil cytoplasmic antibody;ENT � ear, nose, and throat.† Stepwise estimation of the odds ratio (OR).‡ Incremental R2 was computed at each step of the logistic regression model.§ Signs such as maxillary sinusitis or otitis media.


The clustering approach starts by grouping the twomost similar cases together to form a first cluster and thenreiterating the agglomerative procedure until all cases arecollected in a single cluster, thereby generating a new partitionof cases into clusters at each iterative step. The choice of theoptimal partition of clusters (e.g., reflecting the actual distri-bution of distinct pathologic entities in the analyzed patientdata) is a fundamental issue in unsupervised learning. Apopular solution to this problem is to simplify it by finding thepartition that provides the best tradeoff between the homoge-neity of the clusters and their isolation on the partition (20).Although there is no best approach to fit all situations, thecomputation of the Silhouette index, a well-understood parti-tion quality indicator, was shown to be a simple yet robuststrategy for predicting optimal clustering partitions (ref. 21,and Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm).

After identifying the optimal partition and labeling itsclusters, the predictive abilities of the 2 sets of criteria wereevaluated by computing estimations of sensitivity, specificity,positive predictive value, and negative predictive value fromthe contingency table, reflecting the attribution of cases to

each type of vasculitis. The differences between the estimatedperformances of the ACR and the FVSG criteria sets, whichwere computed from 30 independent iterations of the simula-tion procedure, were assessed for statistical significance bychi-square test, comparing the distributions of discrete vari-ables, and by Student’s paired t-test, evaluating mean valuesfrom continuous distributions. All statistical analyses andsimulations were conducted with the use of SPSS softwareversion 13.0 (SPSS, Chicago, IL) and the R software environ-ment for statistical computing (22).

RESULTS

The logistic regression analysis, which consideredall non-PAN vasculitides as controls, retained a set of 8minimally redundant PAN-predictive items (Table 1),including 3 parameters that were positively associatedand 5 parameters that were negatively associated withPAN (Table 2). When restricted to the subgroup ofHBV-negative PAN patients, this analysis confirmed the

Table 3. Discriminant performance of significant and nonredundant items of the ACR 1990 criteria fordistinguishing PAN from other systemic vasculitides or MPA among patients in the FVSG database,considering all PAN cases and considering only HBV-negative PAN cases*

ACR 1990 criteria Odds ratio (95% CI)† R2‡

All PAN casesPAN versus other vasculitides

1. (�) HBV infection 25.53 (13.67–47.67) 0.3232. (�) Arteriographic anomalies 4.40 (2.43–7.95) 0.3563. (–) Renal insufficiency 0.31 (0.18–0.56) 0.3774. (�) Livedo reticularis 2.11 (1.24–3.58) 0.3875. (�) Mono-/polyneuropathy 1.69 (1.16–2.46) 0.3976. (�) Testicular pain or tenderness 3.12 (1.19–8.20) 0.4027. (�) Diastolic BP �90 mm Hg 1.66 (1.06–2.61) 0.4088. (0) Weight loss �4 kg 0.75 (0.52–1.08) –9. (0) Myalgias 1.40 (0.98–2.00) –

PAN versus MPA1. (�) HBV infection 21.21 (8.17–55.09) 0.2932. (–) Renal insufficiency 0.13 (0.07–0.25) 0.3803. (�) Arteriographic anomalies 7.45 (2.94–18.86) 0.427

Only HBV-negative PAN casesHBV-negative PAN versus other vasculitides

1. (�) Arteriographic anomalies 4.42 (2.38–8.21) 0.0432. (–) Renal insufficiency 0.27 (0.15–0.51) 0.0803. (�) Livedo reticularis 2.07 (1.21–3.55) 0.0954. (�) Myalgias 1.55 (1.07–2.25) 0.1085. (�) Diastolic BP �90 mm Hg 1.69 (1.06–2.71) 0.1166. (0) Testicular pain or tenderness 2.69 (0.98–7.41) –7. (0) Weight loss �4 kg 0.72 (0.50–1.05) –8. (0) Mono-/polyneuropathy 1.45 (0.99–2.15) –

HBV-negative PAN versus MPA1. (–) Renal insufficiency 0.13 (0.07–0.26) 0.1472. (�) Arteriographic anomalies 6.99 (2.68–18.20) 0.209

* Each discriminant item showed a significant positive (�) or negative (–) association with polyarteritisnodosa (PAN) or showed no significant usefulness (0) for distinguishing PAN. ACR � American Collegeof Rheumatology; MPA � microscopic polyangiitis; FVSG � French Vasculitis Study Group; HBV �hepatitis B virus; 95% CI � 95% confidence interval; BP � blood pressure.† Stepwise estimation of the odds ratio (OR).‡ Incremental R2 was computed at each step of the logistic regression model.

1532 HENEGAR ET AL

PAN-predictive abilities of 1 positively associated para-meter and 5 negatively associated parameters (Table 2).Combining these items yielded sensitivities of 70.6% for

all PAN cases and 76.6% for HBV-negative PAN, withspecificities of 92.3% and 88.9%, respectively. Whencontrols were restricted to MPA alone, the nonredun-

Figure 1. Receiver operating characteristic curve analyses comparing the discriminant perfor-mance of the French Vasculitis Study Group (FVSG) discriminant set of criteria for polyarteritisnodosa (PAN) with the American College of Rheumatology (ACR) 1990 classification criteria forPAN in 2 situations: A, considering all cases of systemic vasculitides as controls and B, restrictingcontrols to patients with microscopic polyangiitis.

Table 4. Two sets of positive-association and negative-association rules illustrating the potential use of the FVSG set of discriminant items todistinguish PAN from other systemic vasculitides or from MPA under clinical conditions*

Rule†Confidence,

%‡No. ofcases§

PAN versus other vasculitidesPositive PAN association

1. If HBV � 1 and arteriographic anomalies � 1 then PAN � 1 100 312. If HBV � 1 and ANCA � 0 and neuropathy � 1 then PAN � 1 97 963. If HBV � 1 and cryoglobulins � 0 and glomerulopathy � 0 and ENT signs � 0 then PAN � 1 96 804. If ANCA � 0 and arteriographic anomalies � 1 and ENT signs � 0 then PAN � 1 79 66

Negative PAN association1. If HBV � 0 and ANCA � 1 then PAN � 0 99 4012. If ANCA � 1 and glomerulopathy � 1 and arteriographic anomalies � 0 then PAN � 0 99 2703. If ENT signs � 1 then PAN � 0 98 2534. If neuropathy � 0 and arteriographic anomalies � 0 then PAN � 0 86 3775. If HBV � 0 and arteriographic anomalies � 0 then PAN � 0 83 779

PAN versus MPAPositive PAN association

1. If HBV � 1 and ANCA � 0 then PAN � 1 96 1082. If ANCA � 0 and arteriographic anomalies � 1 then PAN � 1 93 573. If ANCA � 0 and glomerulopathy � 0 then PAN � 1 86 221

Negative PAN association1. If ANCA � 1 then PAN � 0 94 1382. If HBV � 0 and glomerulopathy � 1 and arteriographic anomalies � 0 then PAN � 0 85 161

* FVSG � French Vasculitis Study Group; PAN � polyarteritis nodosa; MPA � microscopic polyangiitis; HBV � hepatitis B virus; ANCA �antineutrophil cytoplasmic antibody; ENT � ear, nose, and throat.† 1 indicates the presence of the item; 0 indicates its absence.‡ Precision of the rule, as evaluated using FVSG data.§ Number of cases in which the rule was applicable.


Figure 2. Results of computer simulations evaluating the potential usefulness of the FrenchVasculitis Study Group (FVSG) discriminant set of criteria (blue boxes) compared with theAmerican College of Rheumatology (ACR) 1990 criteria (red boxes) for a diagnosis of polyarteritisnodosa (PAN). Artificial patient data were generated by using the maximum entropy correction ofa truncated Bahadur-Lazarsfeld expansion, either respecting the reported French prevalence of the4 main types of vasculitides (PAN, Wegener’s granulomatosis, Churg-Strauss syndrome, andmicroscopic polyangiitis [MPA]) (hatched boxes) or after arbitrarily modifying them to low PANprevalence (10%) (open boxes). Data were computed from 30 consecutive iterations of thesimulation procedure (see Patients and Methods for details). Shown are the optimal case partitions,represented as both the number of clusters (A) and the silhouette width (B), the sensitivity (C), thespecificity (D), the positive predictive value (E), and the negative predictive value (F) of the 2criteria sets. Data are shown as box plots. Each box represents the first and third quartiles (upperand lower limits, respectively). Whiskers represent the minimum and maximum range. Lines insidethe boxes represent the median.

1534 HENEGAR ET AL

dant PAN-predictive abilities were confirmed for only 4of the previously identified items, 2 were positivelyassociated with a diagnosis of PAN, and 2 were morefrequent in MPA patients (Table 2). One positivelyassociated parameter and 2 negatively associated para-meters (Table 2) showed significant nonredundantPAN-predictive abilities when considering only the sub-group of HBV-negative PAN with controls restricted toMPA alone. The combined sensitivity of these items was89.7% for all PAN cases and 83.1% for HBV-negativePAN, with specificities of 83.1% and 83.6%, respectively.

The logistic regression analysis confirmed theabilities of 7 items from the ACR 1990 classificationcriteria to identify PAN (Table 3). However, the positiveassociation with a diagnosis of PAN indicated by theanalysis of the ACR 1990 criteria could not be confirmedfor 1 item (renal insufficiency), which occurred morefrequently in non-PAN vasculitides. These 7 criteriayielded a combined sensitivity of 48.9%, with 95.6%specificity when all other vasculitides served as controls.When considering only the subgroup of HBV-negativePAN cases, with all other non-PAN vasculitides ascontrols, the nonredundant PAN-predictive abilities ofthe ACR 1990 criteria were confirmed for only 4 posi-tively associated parameters and 1 negatively associatedparameter (Table 3), resulting in a major decrease insensitivity to 8.4%, with 98% specificity in this situation.When controls were restricted to MPA patients, thediscriminant abilities of nonredundant PAN items wereconfirmed for only 3 of them (Table 3), 2 of which werepositively associated with a diagnosis of PAN, while 1(renal insufficiency) occurred more frequently in MPA.In the latter settings, the 3 discriminant items yielded50.8% sensitivity for all forms of PAN, with 96.1%specificity. One positively associated parameter and 1negatively associated parameter (Table 3) showed sig-nificant nonredundant PAN-predictive abilities whenconsidering only the subgroup of HBV-negative PAN,yielding 12.3% sensitivity, with 99% specificity.

The 2 sets of association rules established fromthe selected FVSG criteria are presented in Table 4.When all other vasculitides served as controls, 4 positiveand 5 negative association rules could be established,based on 7 of the 8 previously selected features (exclud-ing asthma). This model had a combined sensitivity of70.2%, with 88.2% specificity. When the controls wererestricted to MPA, 3 positive and 2 negative associationrules were established, which included all 4 of thefeatures identified by the logistic regression analysis(Table 4).

ROC curve analyses were performed on the

selected FVSG items and the ACR 1990 criteria, con-sidering all other vasculitides as controls (Figure 1A).This yielded areas under the curves of 0.916 (95%confidence interval [95% CI] 0.898–0.934) for the FVSGselected items and 0.711 (95% CI 0.671–0.750) for theACR 1990 criteria, confirming the significantly betterdiscriminant accuracy of the FVSG criteria (P � 0.05).The asymptotic significance for these curves was accept-able for both sets of criteria (P � 0.001). When controlswere restricted to MPA (Figure 1B), ROC curve analy-ses yielded areas under the curves of 0.906 (95% CI0.878–0.934) for the selected FVSG criteria and 0.588(95% CI 0.537–0.639) for the ACR 1990 criteria, therebyconfirming the significantly better predictive accuracy ofthe FVSG criteria set in this situation as well (P � 0.05).Again, the asymptotic significance for these ROC curveswas acceptable for both sets of criteria (P � 0.001).

The results of the computer simulation are sum-marized in Figure 2. No significant differences wereobserved between the data sets obtained with the 2distinct approaches used to generate artificial patientdata, thus indicating the good robustness and reproduc-ibility of our simulation procedures (data not shown).Application of the 2 sets of criteria to the artificiallygenerated data suggested a significant relationship be-tween the sparseness and the quality (i.e., in terms ofintracluster homogeneity) of the resulting cluster parti-tions and the positive predictive value of the criteria setsused to generate these partitions. Indeed, case partitionsobtained with selected FVSG items displayed signifi-cantly lower sparseness and higher intracluster homoge-neity than did those generated with the ACR 1990criteria (Figures 2A and B), particularly when used todistinguish PAN from MPA (P � 0.05). Most notably,the FVSG set of items yielded higher PAN-predictiveperformances in terms of positive predictive value andsensitivity (Figures 2C and E) than did the ACR 1990criteria (P � 0.05). This better performance was con-served even when the prevalence of PAN cases ingenerated patient populations was artificially lowered.The gain in PAN-predictive ability achieved with theFVSG criteria was not accompanied by any significantdecrease in the specificity or the negative predictivevalue (Figures 2D and F) as compared with the ACR1990 criteria.

DISCUSSIONValidated diagnostic criteria for systemic diseases

could be a useful tool for clinicians who cannot obtain ahistologic diagnosis or who want to avoid performing


biopsies because of the risk of side effects. Despite theirinconsistent performance as reported in the literature(4,7), many clinicians still inappropriately use the ACR1990 classification criteria for diagnostic purposes. Assome analysts suggested, a potential explanation for theinconsistent results obtained with those criteria whenused for diagnostic predictions could be the dependenceof their discriminant abilities on the prevalence ofindividual vasculitides within the populations examined(23,24). Moreover, similar studies related to other vas-culitis classification systems, such as the Chapel HillNomenclature (8), showed similar unsatisfactory perfor-mance when used for diagnostic purposes (5,6) andhighlighted the need to develop separate criteria forclassification and diagnosis of the various systemic vas-culitides (7,25).

Herein, we report our analysis of patient dataaccumulated in the FVSG database in which we targeted2 main objectives. The first was to establish a minimal setof nonredundant criteria that were positively or nega-tively predictive of PAN and were potentially useful notonly for classification, but also for diagnostic purposes.The discriminant accuracy of the selected set of FVSGitems (Table 1) was compared with that of the ACR1990 criteria for PAN in a sample of patients withhistologically proven systemic vasculitides selected fromthe FVSG database. The second objective was to eval-uate the PAN-predictive abilities of each of these sets ofcriteria in a computer simulation procedure that reliedon an unsupervised hierarchical classification algorithmapplied to artificially generated patient data.

The results of our analysis showed that when theACR 1990 criteria were applied to the FVSG databasepatients, their initially reported PAN discriminant abil-ities were not confirmed (2). In contrast, the selectedset of FVSG items established by our analysis signifi-cantly outperformed the ACR 1990 criteria, regardlessof the analytical situation being considered (Tables 2and 3 and Figure 1). The difference between the dis-criminant performance of the 2 sets of criteria was evenmore significant when the analysis was restricted to thesubgroup of HBV-negative PAN patients, which theACR 1990 criteria could not distinguish from the groupwith non-PAN vasculitides, thus demonstrating the ex-tremely low sensitivity of these criteria in this particularsituation.

Furthermore, although 7 of the 9 original itemsof the ACR 1990 criteria (except for histology) werefound to have significantly nonredundant usefulness fordistinguishing PAN from other systemic vasculitides inthe FVSG database patients, a significant positive asso-

ciation with a diagnosis of PAN could not be confirmedfor 3 of the items (Table 3). Moreover, renal insuffi-ciency, which yielded significant discriminant ability todistinguish PAN in our analysis, showed a lower fre-quency in PAN patients than in other systemic vasculit-ides or in MPA (Table 3). These findings are consistentwith those in previous studies (4) that indicated a lowpositive predictive value of the ACR 1990 criteria forPAN.

As previously suggested, in addition to someepidemiologic differences between patient samples, an-other potential explanation for this phenomenon couldbe the lack of distinction between PAN and MPA that isinherent in the ACR 1990 analysis. Indeed, a strongargument in support of this hypothesis is the poorlydiscriminant performance of the ACR 1990 criteria inour analyses when used to distinguish between histolog-ically confirmed PAN and MPA (Table 3), which wasalready shown to be a major drawback of these criteria(24,26,27).

A third possible explanation of the observeddifferences in discriminant abilities of the ACR 1990criteria and the FVSG item set could lie in the method-ologic differences between the 2 analyses. While theACR 1990 analysis focused on the selection of positivediscriminant criteria for classification purposes, our ana-lysis sought to maximize the combined predictive rele-vance of the selected items by considering both posi-tively associated and negatively associated parameters.Indeed, this strategic difference could potentially ex-plain why significant negatively discriminant items, forexample, ANCA positivity by indirect immunofluores-cence, could have been missed by the ACR 1990 analy-sis. The importance of ANCA positivity in the diagnosisof vasculitis was first recognized by the Chapel HillNomenclature group (8). In our analysis, ANCA posi-tivity yielded the strongest discriminant ability, covering22% of the variance in the logistic regression modelwhen used to distinguish PAN from other vasculitidesand 50% of the variance when used to distinguish PANfrom MPA (Table 2). It is widely acknowledged that thedetection of ANCA by indirect immunofluorescenceprovides a lower specificity than detection by antigen-specific enzyme-linked immunosorbent assay techniques(10), and it should therefore be expected that thecombination of these 2 detection methods might furtherimprove the discriminant performance of the FVSGitem set.

Our assessment of the relevance of the 2 criteriasets in the diagnosis of PAN was complemented by acomputer simulation procedure that relied on both sets

1536 HENEGAR ET AL

of items to perform an unsupervised classification taskon artificially generated patient data. Although thenumbers of case clusters contained by the optimalpartitions were not the same as the real number ofvasculitis entities (e.g., 4 in our case) represented in theartificial patient samples under either of the analyticalconditions considered, the FVSG item set yielded 2–5times fewer clusters (i.e., closer to the real number ofvasculitis entities) than did the ACR 1990 criteria. Thislower sparseness of the partitions obtained with theFVSG items was associated with significantly betterintracluster homogeneity of the resulting case clustersthan that obtained with the ACR 1990 criteria, therebyconfirming the superior PAN-predictive abilities of theFVSG item set.

The FVSG item set also exhibited significantlystronger robustness when the prevalence of vasculitideswas artificially varied in generated patient samples, thussuggesting better adaptability of this set of criteria tovarious epidemiologic conditions. In addition, the lowsensitivities and positive predictive values obtained withthe ACR 1990 criteria during the computer simulationprocedure are consistent with the high percentages offalse-positive diagnoses of PAN previously reported withthis criteria set (4), indicating good reliability of com-puter simulation analyses.

In conclusion, the results of the analyses de-scribed herein suggest that the combined use of positiveand negative criteria could significantly improve dis-criminant performance while providing a more appro-priate support for analytical medical reasoning as itexamines, with equal importance, both positive andnegative rationales for considering a diagnosis. Indeed,as the results of our computer simulation suggest, thecombination of positively and negatively discriminantcriteria may prove beneficial for the establishment of adiagnostic screening tool for vasculitis patients. Furtherprospective validation of this set of criteria in a multi-center international study of different populations anddifferent epidemiologic settings is needed in order toconfirm that these items together provide a satisfactorydiagnostic tool for PAN.

AUTHOR CONTRIBUTIONS

Drs. Henegar and Guillevin had full access to all of the datain the study and take responsibility for the integrity of the data and theaccuracy of the data analysis.Study design. Henegar, Pagnoux, Puechal, Guillevin.Acquisition of data. Pagnoux, Saba, Bagneres, Meyer, Guillevin.

Analysis and interpretation of data. Henegar, Pagnoux, Puechal, Saba,Guillevin.Manuscript preparation. Henegar, Pagnoux, Zucker, Le Guern,Guillevin.Statistical analysis. Henegar, Zucker, Bar-Hen.

REFERENCES

1. Bloch DA, Michel BA, Hunder GG, McShane DJ, Arend WP,Calabrese LH, et al. The American College of Rheumatology 1990criteria for the classification of vasculitis: patients and methods.Arthritis Rheum 1990;33:1068–73.

2. Lightfoot RW Jr, Michel BA, Bloch DA, Hunder GG, Zvaifler NJ,McShane DJ, et al. The American College of Rheumatology 1990criteria for the classification of polyarteritis nodosa. ArthritisRheum 1990;33:1088–93.

3. Fries JF, Hochberg MC, Medsger TA Jr, Hunder GG, BombardierC, and the American College of Rheumatology Diagnostic andTherapeutic Criteria Committee. Criteria for rheumatic disease:different types and different functions. Arthritis Rheum 1994;37:454–62.

4. Rao JK, Allen NB, Pincus T. Limitations of the 1990 AmericanCollege of Rheumatology classification criteria in the diagnosis ofvasculitis. Ann Intern Med 1998;129:345–52.

5. Lane SE, Watts RA, Barker TH, Scott DG. Evaluation of theSorensen diagnostic criteria in the classification of systemic vascu-litis. Rheumatology (Oxford) 2002;41:1138–41.

6. Sorensen SF, Slot O, Tvede N, Petersen J. A prospective studyof vasculitis patients collected in a five year period: evaluationof the Chapel Hill nomenclature. Ann Rheum Dis 2000;59:478–82.

7. Hunder GG. The use and misuse of classification and diag-nostic criteria for complex diseases. Ann Intern Med 1998;129:417–8.

8. Jennette JC, Falk RJ, Andrassy K, Bacon PA, Churg J, Gross WL,et al. Nomenclature of systemic vasculitides: proposal of aninternational consensus conference. Arthritis Rheum 1994;37:187–92.

9. Servoss JC, Friedman LS. Serologic and molecular diagnosis ofhepatitis B virus. Infect Dis Clin North Am 2006;20:47–61.

10. Hagen EC, Daha MR, Hermans J, Andrassy K, Csernok E, GaskinG, et al, for the EC/BCR Project for ANCA Assay Standardiza-tion. Diagnostic value of standardized assays for anti-neutrophilcytoplasmic antibodies in idiopathic systemic vasculitis. Kidney Int1998;53:743–53.

11. Yao YY. Information-theoretic measures for knowledge discoveryand data mining. In: Karmeshu, editor. Entropy measures, maxi-mum entropy principle and emerging applications. 1st ed. Berlin:Springer; 2003. p. 115–36.

12. Ripley BD, editor. Pattern recognition and neural networks. 1sted. Cambridge (UK): Cambridge University Press; 1996.

13. Breiman L, Friedman JH, Olshen RA, Stone CJ, editors. Classi-fication and regression trees. 1st ed. Belmont (CA): Wadsworth;1984.

14. Hanley JA, McNeil BJ. A method of comparing the areas underreceiver operating characteristic curves derived from the samecases. Radiology 1983;148:839–43.

15. Bahadur RR. A representation of the joint distribution of re-sponses to n dichotomous items. In: Solomon H, editor. Studies initem analysis and prediction. 1st ed. Stanford: Stanford UniversityPress; 1961. p. 158–68.

16. Van Der Geest PA. The binomial distribution with dependentBernoulli trials. J Stat Comput Simulat 2005;75:141–54.

17. Yu CT, Buckley C, Lam K, Salton G. A generalized termdependence model in information retrieval. Inf Technol Res Dev1983;2:129–54.

18. Chow CK, Liu CN. Approximating discrete probability distribu-


tions with dependence trees. IEEE Trans Inf Theory 1968;14:462–7.

19. Mahr A, Guillevin L, Poissonnet M, Ayme S. Prevalences ofpolyarteritis nodosa, microscopic polyangiitis, Wegener’s granulo-matosis, and Churg-Strauss syndrome in a French urban multieth-nic population in 2000: a capture–recapture estimate. ArthritisRheum 2004;51:92–9.

20. Xu R, Wunsch DN. Survey of clustering algorithms. IEEE TransNeural Netw 2005;16:645–78.

21. Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation andvalidation of cluster analysis. J Computat Appl Math 1987;20:53–65.

22. R Development Core Team. R: a language and environment forstatistical computing, version: 2.4.0. Vienna: R Foundation forStatistical Computing; 2006. Online at: http://www.R-project.org.

23. Heller I, Isakov A, Topilsky M. American College of Rheumatol-

ogy Criteria for the diagnosis of vasculitis [letter]. Ann Intern Med1999;130:861.

24. Watts RA, Jolliffe VA, Carruthers DM, Lockwood M, Scott DG.Effect of classification on the incidence of polyarteritis nodosa andmicroscopic polyangiitis. Arthritis Rheum 1996;39:1208–12.

25. Watts R, Lane S, Hanslik T, Hauser T, Hellmich B, KoldingsnesW, et al. Development and validation of a consensus methodologyfor the classification of the ANCA-associated vasculitides andpolyarteritis nodosa for epidemiological studies. Ann Rheum Dis2007;66:222–7.

26. Bruce IN, Bell AL. Effect of classification on the incidence ofpolyarteritis nodosa and microscopic polyangiitis: comment on thearticle by Watts et al [letter]. Arthritis Rheum 1997;40:1183.

27. Bruce IN, Bell AL. A comparison of two nomenclature systems forprimary systemic vasculitis. Br J Rheumatol 1997;36:453–8.

1538 HENEGAR ET AL

Documents

A paradigm of diagnostic criteria for polyarteritis nodosa: Analysis of a series of 949 patients with vasculitides