Original article | Published 15 December 2018 | doi104414smw201814691Cite this as Swiss Med Wkly 2018148w14691

Diagnostic diversity ndash an indicator of institutionaland regional healthcare qualityBrutsche Martina Rassouli Franka Gallion Haraldb Kalra Sanjayc Roger Veronique Ld Baty Florenta

a Lung Centre Cantonal Hospital St Gallen Switzerlandb Department of Medical Coding Cantonal Hospital St Gallen Switzerlandc Department of Pulmonary and Critical Care Medicine Mayo Clinic Rochester Minnesota USAd Department of Internal Medicine Division of Cardiovascular Diseases Mayo Clinic Rochester Minnesota USA

Summary

AIM Our aim was to estimate the diagnostic performanceof institutions and healthcare regions from a nationwidehospitalisation database

METHODS The Shannon diversity index was used asan indicator of diagnostic performance based on the In-ternational Classification of Disease 10th revision Ger-man Modification (ICD-10-GM codes) The dataset includ-ed a total of 9325326 hospitalisation cases from 2009to 2015 and was provided by the Swiss Federal Officefor Statistics A total of 16435 diagnostic items from theICD-10-GM codes were taken as the basis for the calcula-tion of the diagnostic diversity index (DDI) Numerical sim-ulations were performed to evaluate the effect of misdi-agnoses in the DDI We arbitrarily defined the minimumclinically important difference (MCID) as 10 misdiag-noses The R statistical software was used for all analy-ses

RESULTS Diagnostic performance of institutions andhealthcare regions as measured by the DDI were stronglyassociated with caseload and number of inhabitants re-spectively A caseload of gt7217 hospitalisations per yearfor institutions and a population size gt363522 for health-care regions were indicators of an acceptable diagnosticperformance Among hospitals there was notable hetero-geneity of diagnostic diversity which was strongly asso-ciated with caseload Application of misdiagnosis-thresh-olds within each ICD-10-GM category allowedclassification of hospitals in four distinct groups high-vol-ume hospitals with an all-over comprehensive diagnosticperformance high- to mid-volume hospitals with extensiveto relevant basic diagnostic performance in most cate-gories low-volume specialised hospitals with a high di-agnostic performance in a single category and low-vol-ume hospitals with inadequate diagnostic performance inall categories The diagnostic diversity observed in the 26Swiss healthcare regions showed relevant heterogeneityan association with ICD-10-GM code utilisation and wasstrongly associated with the size of the healthcare regionThe limited diagnostic performance in small healthcare re-gions was partially but not fully compensated for by con-

sumption of health services outside of their own health-care region

CONCLUSION Calculation of the DDI from ICD-10 codesis easy and complements the information derived fromother quality indicators as it sheds a light on the fitnessof the institutionalised interplay between primary and spe-cialised medical inpatient care

Keywords Shannon diversity index International Classi-fication of Diseases healthcare quality

Introduction

Measuring healthcare quality is essential to optimise theeffectiveness of healthcare delivery in a rapidly changinghealthcare environment but is challenging and at timescontroversial There is a plethora of quality indicators [1]from self-reported health status [2] to mortality rates in dif-ferent age groups and conditions that are applied to eval-uate care across regions [3] and institutions [4] Thesemeasures are affected by a complex interplay betweenavailability of resources and socioeconomic and health-care factors [5] Measuring and aligning the contribution ofdifferent healthcare determinants is important albeit chal-lenging [6] The United States-based Aligning Forces forQuality (AF4Q)-program the largest of its kind worldwidereaching 125 of the US population at a cost of 300 MioUS dollars analysed the effect of institutional alignmentand networking on 144 quality outcomes [7] The authorsconcluded that the AF4Q initiative had less impact than ex-pected The multitude of unweighted outcomes to be opti-mised and the overweight of reimbursement effects on thedevelopment of a healthcare landscape [8] may have dilut-ed the impact of the intervention The publication of annualinstitutional mortality rates and other quality indicators isimposed on institutions in different industrial countries [9]Such publications seem to stimulate quality improvementactivities at the hospital level However the effect of pub-lic reporting on effectiveness safety and patient-centered-ness remains uncertain [10] Such publications rather be-come public relations and market factors of a nonvalidatednature in the hands of nonprofessionals [11]

CorrespondenceMartin Brutsche MD PhDDepartment of Pulmonaryand Sleep Medicine Can-tonal Hospital St GallenRorschacher Strasse 95CH-9007 St Gallen mar-tinbrutsche[at]kssgch

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 1 of 9

We propose an alternative indicator of healthcare qualitybased on diagnostic diversity ndash the diversity of diagnosticcodes attributed to inpatient medical care Diagnostic di-versity and its demographic implications have been inves-tigated in a recent study from Schuster and colleagues [12]It should be noticed that the term ldquodiagnostic diversityrdquohas also been used independently in sociological scienceIn this unrelated context the term diversity refers to differ-ences in the perception and interpretation of diagnoses invarious sociodemographic groups of the population [13]

Adequate disease management strategies and treatmentsare dependent on the timely identification of an exact di-agnosis beforehand [14] Diagnostic imprecision or errorson the other hand lead to suboptimal disease managementand treatment ndash especially in rare diseases [15] Diagnosticerrors are increasingly recognised as important contribu-tors to preventable morbidity and mortality [16] In theUnited States they are estimated to occur in up to 15of all clinical encounters affect 12 million adults annuallyand lead to permanent damage or death in nearly 160000patients each year [17] Thus the earlier and the more pre-cisely the diagnosis is refined the greater the chance for abetter outcome as well as a better allocation of healthcareresources

The variety of diagnosed diseases established by an insti-tution or accumulated in a healthcare region can be mea-sured with a diversity index In a hypothetical ideal sit-uation where all diseases would be diagnosed correctly ahigh degree of diagnostic diversity can be expected (fig

1) The degree of diagnostic diversity that can be achievedby modern medicine is dependent on the diagnostic effortsand quality and is increasing with new diagnostic develop-ments Thus the diagnostic diversity index (DDI) can alsobe seen as an indicator of diagnostic precision ndash the higherthe degree of diagnostic diversity the better the diagnosticprecision provided

We compared the DDI of the International Classificationof Disease 10th revision German Modification(ICD-10-GM) codes in healthcare regions and hospitalsfrom a nationwide database of all hospitalised cases inSwitzerland from 2009 to 2015

Materials and methods

Health service landscape in SwitzerlandSwitzerland has one of the best European health servicesaccording to the European Health Consumer Index 2015[18] It is divided in 26 cantons of different surface andpopulation size which represent independent healthcareregions (HCRs) These HCRs are serviced by 292 acutecare in-patient institutions An overview of the Swisshealthcare system is provided in table 1 A total of five uni-versity hospitals and several larger cantonal hospitals canbe considered as quaternary and tertiary healthcare institu-tions respectively As restrictions to extracantonal health-care apply patients are mostly treated within a HCR Iftriggered either by a referring physician or patient prefer-ence patients can also be treated outside of their HCR at

Figure 1 Diagnostic diversity as a measure of overall diagnostic performance ndash study hypothesis It can be expected that the diseasesoccurring in a population reflect the biological diversity of humans with their diverse genetic make-up and different exposures Thus it can beassumed that a high diversity of diagnoses established in a health institution or region is an indicator of good overall diagnostic performance

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 2 of 9

increased costs in specific well-defined situations (basichealth insurance) or more broadly with private insurance(approximately 15 of the population)

Hospitalisation databaseThe hospitalisation dataset used in this analysis was pro-vided by the Swiss Federal Office for Statistics It includesmedical information about all hospitalisations in Switzer-land since 1998 In this database the patient informationis fully anonymised No written informed consent wasgiven by patients who were unidentifiable owing to theanonymisation The data belong to the Swiss Federal Of-fice for Statistics (Bundesamt fuumlr Statistik NeuchacirctelSwitzerland) which provides regulated access to the datafor research purposes [19 20] The database included ge-ographical and temporal information (patientrsquos area of res-idence HCR [canton] of institution year and month ofhospitalisation length of hospital stay) as well as age atadmission and reasontype of discharge (including death)Hospitalisations within and outside of the area of residencywere considered Thus we could quantify the percentage ofhospitalisations taking place outside of the HCR of resi-dency Unique anonymised institution numbers for the cal-culation of annual caseload were available The patientsrsquolist of diagnoses included one main diagnosis as well asup to 50 additional diagnoses coded using ICD-10-GMcodes [21] The ICD-10-GM coding was uniformly usedthroughout the study period For the current publicationthe timespan of the analysis was restricted to 2009ndash2015Data prior 2009 were not included in order to keep theanalysed dataset as homogenous as possible In additiononly Swiss residents were considered [22]

Statistical analysisThe diagnostic diversity was measured using the Shannondiversity index [23] defined as follows

H = minus sumi = 1D piln (pi)

with D the number of diagnoses (ICD-10-GM codes) lnthe natural logarithm and pi the proportional abundance

of the ith diagnosis The Shannon diversity index origi-nally developed in information theory is one of the mostcommonly used diversity indices It is widely used in eco-logical science but has also been previously reported as apromising measurement in health service research [12] Inour analysis the Shannon diversity index accounts for boththe abundance and the evenness of the ICD-10-GM at theHCR or institution level It assumes that all ICD-10-GMcodes are equal and therefore treats equally the most abun-dant as well as the rarest codes This index has a lowerbound of 0 but no upper bound It increases logarithmical-ly proportionally to the diversity of codes employed Nu-merical simulations were performed to evaluate the effect

of misdiagnoses in the DDI We used nonparametric boot-strapping to resample the ICD-10-GM codes by removingan increasing proportion of ICD-10-GM codes in a preva-lence-dependent fashion More specifically we used thisprocedure to randomly remove rare ICD-10-GM codes re-placing them by randomly chosen more common codesThis resulted in an artificial decrease of the diagnostic di-versity and an increase in the rate of misdiagnoses We ar-bitrarily defined the minimum clinically important differ-ence (MCID) as a 10-misdiagnosis ie 1 in 10 patientsseen on the ward leaves the hospital with an impreciseor misdiagnosis This cut-off was chosen due to its clini-cal relevance The diagnostic performance was consideredsufficient when an institutionHCR has a DDI above thiscut-off A second cut-off was set at 20 misdiagnosis andused for illustration purposes in some analyses MCID islevel-specific (eg HCR institution) and is different withineach ICD-10-GM code category In a sub-analysis chap-ters I to XIV (A-N) of the ICD-10-GM were analysed sep-arately (codes O-Z being used for specific purposes includ-ing pregnancy or various injuries) All analyses were doneusing the R statistical software [24] including the follow-ing extension packages vegan [25] ade4 [26]

Results

ICD-10-GM utilisation and DDI benchmarkBetween 2009 and 2015 9325326 hospitalisation caseswere reported A total of 16435 ICD-10-GM codes wereused in the study period Each hospitalisation case wascoded with on average 42 ICD-10-GM codes The 10most frequently used ICD10-GM codes (1644 codes) wereused in 87 (8094114 cases) of all hospitalisation casesThe Swiss-wide DDI (ICD-10-GM chapters AndashN) was718 The Swiss-wide DDI for individual chapters A to Nas well the MCID-threshold are given in table 2 Chap-ters A B (infections causes bacteria viruses) and E (en-docrine diseases) had a relatively low diagnostic diversity(DDI below 4) whereas diseases of the musculoskeletalsystem and connective tissue had the highest diagnostic di-versity (DDI above 5) The effect of misdiagnoses on theDDI was assessed by numerical simulation (fig 2) The re-lationship between the rates of misdiagnosis on the DDIshows a linear decrease in the lower misclassification ratesand a more abrupt decrease in the higher misclassificationrates The inclusion of 10 of misdiagnoses resulted in adecrease of 012 point in the DDI According to this sim-ulation a DDI of lt706 would represent the occurrence ofgt10 misdiagnoses in the observed case sample (MCID)A second MCID cut-off set at 20 misdiagnoses was alsoconsidered A further drop of 014 points in the DDI wasobserved when applying a MCID 20 threshold

Table 1 Categorisation of inpatient institutions in Switzerland in 2015

Type of hospital Number of institutions

Primary hospitals (university hospitals) 5

Secondary hospitals (level 2) 35

Tertiaryquaternary hospitals (levels 3 4 5) 66

Psychiatric clinics 49

Rehabilitation clinics 50

Specialised clinics 83

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 3 of 9

Characterisation of acute in-patient medical care insti-tutions in SwitzerlandThere was relevant DDI heterogeneity among hospitals inSwitzerland where the DDI was strongly associated withcaseload (fig 3) Simulated 10 and 20 misdiagnosisthresholds applied to each of the 14 ICD-10-GM chapters(AndashN) allowed Swiss hospitals to be categorised into fourdistinct groups (fig 4)

High-volume hospitals with an all-over comprehensive di-agnostic performance (n = 10 university large cantonalhospitals with an average caseload of 36000 hospitalisa-tions per year)

High- to mid-volume hospitals with a gradient of extensiveto relevant basic diagnostic performance in most chapters(n = 88 most cantonal regional hospitals with an averagecaseload of 8330 hospitalisations per year)

Low-volume specialised hospitals with a high diagnosticperformance in a single chapter (n = 48 with an averagecaseload of 2481 hospitalisations per year) and

Low-volume hospitals with inadequate diagnostic perfor-mance in all chapters (n = 146 with an average caseloadof 689 hospitalisations per year) The 146 low-volume hos-pitals had an estimated misdiagnosis rate of gt20 in allchapters and contributed to 8 (710188 out of 9325326)of all hospitalisations in Switzerland

Residential health care regions and health regionsrsquo in-stitutionsThe DDI observed in the 26 Swiss HCRs (fig 5 upperright panel) also showed relevant heterogeneity and strongassociations with ICD-10-GM code utilisation and case-load and hence size of the HCR (fig 5 left panels) Therewas a strong association of DDI and ICD-10-GM-codeutilisation (fig 5 inlet within upper-left panel) In smallerHCRs lt40 of possible ICD-10-GM codes were utilised

Table 2 Description of the ICD-10-GM categories

ICD-10-GM cate-gory

Description Usage(total = 39410015)

Percentage DDI MCID (10) MCID (20)

A Bacterial infections viral infections of thecentral nervous system and arthropod-borneviral fevers

372010 (094) 331 297 271

B Other viral infections and infections causedby fungi protozoans worms infestations andsequelae

715146 (181) 331 310 270

C Malignant neoplasms 1358151 (345) 458 419 412

D In situ benign neoplasms and neoplasms ofuncertainunknown behaviour

1328001 (337) 410 371 345

E Endocrine nutritional and metabolic diseases 3147705 (799) 372 341 323

F Mental and behavioural disorders 2238711 (568) 437 391 394

G Diseases of the nervous system 1240174 (315) 469 419 411

H Diseases of the eye and adnexa diseases ofthe ear and mastoid process

441895 (112) 479 375 331

I Diseases of the circulatory system 5956121 (1511) 418 390 385

J Diseases of the respiratory system 1570592 (399) 436 405 395

K Diseases of the digestive system 1949259 (495) 480 449 435

L Diseases of the skin and subcutaneous tis-sue

387950 (098) 462 409 405

M Diseases of the musculoskeletal system andconnective tissue

2778975 (705) 565 507 502

N Diseases of the genitourinary system 2159369 (548) 400 377 343

O Pregnancy childbirth and the puerperium 2177981 (553) NA NA NA

P Certain conditions originating in the perinatalperiod

415294 (105) NA NA NA

Q Congenital malformations deformations andchromosomal abnormalities

188454 (048) NA NA NA

R Symptoms signs and abnormal clinical andlaboratory findings not elsewhere classified

1946955 (494) NA NA NA

S Injury involving certain part of the body 1847718 (469) NA NA NA

T Injury involving multipleunspecified part ofthe body and poisoning

726275 (184) NA NA NA

U Codes for special purposes 295714 (075) NA NA NA

V Unintentional vehicletraffic injuries 107434 (027) NA NA NA

W Unintentional hitstruckbittendrowning suffo-cation exposure to electric current or radia-tion

116729 (03) NA NA NA

X Unintentional fireflamenatureenvironmentalpoisoning overexertionself-harm injuries

731188 (186) NA NA NA

Y Intentional assaulthomicide complication ofmedical and surgical care

819456 (208) NA NA NA

Z Factors influencing health status and contactwith health services

4392758 (1115) NA NA NA

The Swiss-wide code usage ie the number of times ICD-10-GM codes have been used within each category is provided as absolute number and percentage as well as thediagnostic diversity index (DDI) and the minimum clinically important difference (MCID) (10 and 20 misdiagnosis rate) are reported NA = not applicable

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 4 of 9

by HCR institutions in the observation period Institutionsof larger HCRs with a higher DDI utilised gt60 of theICD-10-GM codes All but one HCR with a populationgt363522 had a DDI above the 706 MCID threshold Insmaller HCRs (population lt363522) the DDI was most-ly below the MCID threshold and there was an increasingdifference between the DDI observed in the general popu-lation and the DDI provided by the HCR institution Thepopulation of such HCRs more often sought healthcare ser-

Figure 2 Diagnostic diversity index (DDI) benchmark ndash nu-merically simulated introduction of an increasing number ofmisdiagnoses and its effect observed on the DDI The overallSwiss-wide DDI level (before simulation) is represented by a greydashed line The simulation of 10 of misdiagnoses ndash defined asMCID 10 ndash resulted in a DDI decrease of 012 points or 706(blacked dashed line) A second cut-off set at 20 misdiagnosis(MCID 20) is also represented (lower horizontal blacked dashedline)

Figure 3 Association between the diagnostic diversity indexand the institutionsrsquo caseload The 10 misdiagnosis threshold(minimum clinically important difference MCID) is represented bya horizontal dashed lineLocal regression (loess) representing asmooth curve through the set of data points is depicted In generalinstitutions with an annual caseload gt7217 hospitalisation cases(intersection point where the loess regression crosses the 10misdiagnosis diagnostic diversity index [DDI] threshold) were morelikely to have an adequate DDI

vices in another HCR The lower diagnostic performancein small HCRs could partially be compensated for by con-sumption of health services outside the HCR (fig 5 low-er right panel) In smaller HCRs up to 62 of residentswere treated outside their HCR whereas around 90 ofresidents of larger HCRs (among those including universi-ty hospitals) were exclusively treated within their HCR of

Figure 4 Heat map of the diagnostic fitness (DDI) of acutemedical care hospitals within 14 ICD-10-GM chapters (AndashN)For each chapter the 10 (minimum clinically important differenceMCID) and 20 misdiagnosis thresholds were established by sim-ulation In the case of a DDI gt10 and gt20 threshold the re-spective bar is shown in dark and light grey for each chapter foreach of the hospitals The number of hospitalisation cases per in-stitution is coded in different bullet sizes on the right side (biggerindicating a larger caseload) This view allows four groups of acutemedical care hospitals to be distinguished (see text)

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 5 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

We propose an alternative indicator of healthcare qualitybased on diagnostic diversity ndash the diversity of diagnosticcodes attributed to inpatient medical care Diagnostic di-versity and its demographic implications have been inves-tigated in a recent study from Schuster and colleagues [12]It should be noticed that the term ldquodiagnostic diversityrdquohas also been used independently in sociological scienceIn this unrelated context the term diversity refers to differ-ences in the perception and interpretation of diagnoses invarious sociodemographic groups of the population [13]

Adequate disease management strategies and treatmentsare dependent on the timely identification of an exact di-agnosis beforehand [14] Diagnostic imprecision or errorson the other hand lead to suboptimal disease managementand treatment ndash especially in rare diseases [15] Diagnosticerrors are increasingly recognised as important contribu-tors to preventable morbidity and mortality [16] In theUnited States they are estimated to occur in up to 15of all clinical encounters affect 12 million adults annuallyand lead to permanent damage or death in nearly 160000patients each year [17] Thus the earlier and the more pre-cisely the diagnosis is refined the greater the chance for abetter outcome as well as a better allocation of healthcareresources

The variety of diagnosed diseases established by an insti-tution or accumulated in a healthcare region can be mea-sured with a diversity index In a hypothetical ideal sit-uation where all diseases would be diagnosed correctly ahigh degree of diagnostic diversity can be expected (fig

1) The degree of diagnostic diversity that can be achievedby modern medicine is dependent on the diagnostic effortsand quality and is increasing with new diagnostic develop-ments Thus the diagnostic diversity index (DDI) can alsobe seen as an indicator of diagnostic precision ndash the higherthe degree of diagnostic diversity the better the diagnosticprecision provided

We compared the DDI of the International Classificationof Disease 10th revision German Modification(ICD-10-GM) codes in healthcare regions and hospitalsfrom a nationwide database of all hospitalised cases inSwitzerland from 2009 to 2015

Materials and methods

Health service landscape in SwitzerlandSwitzerland has one of the best European health servicesaccording to the European Health Consumer Index 2015[18] It is divided in 26 cantons of different surface andpopulation size which represent independent healthcareregions (HCRs) These HCRs are serviced by 292 acutecare in-patient institutions An overview of the Swisshealthcare system is provided in table 1 A total of five uni-versity hospitals and several larger cantonal hospitals canbe considered as quaternary and tertiary healthcare institu-tions respectively As restrictions to extracantonal health-care apply patients are mostly treated within a HCR Iftriggered either by a referring physician or patient prefer-ence patients can also be treated outside of their HCR at

Figure 1 Diagnostic diversity as a measure of overall diagnostic performance ndash study hypothesis It can be expected that the diseasesoccurring in a population reflect the biological diversity of humans with their diverse genetic make-up and different exposures Thus it can beassumed that a high diversity of diagnoses established in a health institution or region is an indicator of good overall diagnostic performance

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 2 of 9

increased costs in specific well-defined situations (basichealth insurance) or more broadly with private insurance(approximately 15 of the population)

Hospitalisation databaseThe hospitalisation dataset used in this analysis was pro-vided by the Swiss Federal Office for Statistics It includesmedical information about all hospitalisations in Switzer-land since 1998 In this database the patient informationis fully anonymised No written informed consent wasgiven by patients who were unidentifiable owing to theanonymisation The data belong to the Swiss Federal Of-fice for Statistics (Bundesamt fuumlr Statistik NeuchacirctelSwitzerland) which provides regulated access to the datafor research purposes [19 20] The database included ge-ographical and temporal information (patientrsquos area of res-idence HCR [canton] of institution year and month ofhospitalisation length of hospital stay) as well as age atadmission and reasontype of discharge (including death)Hospitalisations within and outside of the area of residencywere considered Thus we could quantify the percentage ofhospitalisations taking place outside of the HCR of resi-dency Unique anonymised institution numbers for the cal-culation of annual caseload were available The patientsrsquolist of diagnoses included one main diagnosis as well asup to 50 additional diagnoses coded using ICD-10-GMcodes [21] The ICD-10-GM coding was uniformly usedthroughout the study period For the current publicationthe timespan of the analysis was restricted to 2009ndash2015Data prior 2009 were not included in order to keep theanalysed dataset as homogenous as possible In additiononly Swiss residents were considered [22]

Statistical analysisThe diagnostic diversity was measured using the Shannondiversity index [23] defined as follows

H = minus sumi = 1D piln (pi)

with D the number of diagnoses (ICD-10-GM codes) lnthe natural logarithm and pi the proportional abundance

of the ith diagnosis The Shannon diversity index origi-nally developed in information theory is one of the mostcommonly used diversity indices It is widely used in eco-logical science but has also been previously reported as apromising measurement in health service research [12] Inour analysis the Shannon diversity index accounts for boththe abundance and the evenness of the ICD-10-GM at theHCR or institution level It assumes that all ICD-10-GMcodes are equal and therefore treats equally the most abun-dant as well as the rarest codes This index has a lowerbound of 0 but no upper bound It increases logarithmical-ly proportionally to the diversity of codes employed Nu-merical simulations were performed to evaluate the effect

of misdiagnoses in the DDI We used nonparametric boot-strapping to resample the ICD-10-GM codes by removingan increasing proportion of ICD-10-GM codes in a preva-lence-dependent fashion More specifically we used thisprocedure to randomly remove rare ICD-10-GM codes re-placing them by randomly chosen more common codesThis resulted in an artificial decrease of the diagnostic di-versity and an increase in the rate of misdiagnoses We ar-bitrarily defined the minimum clinically important differ-ence (MCID) as a 10-misdiagnosis ie 1 in 10 patientsseen on the ward leaves the hospital with an impreciseor misdiagnosis This cut-off was chosen due to its clini-cal relevance The diagnostic performance was consideredsufficient when an institutionHCR has a DDI above thiscut-off A second cut-off was set at 20 misdiagnosis andused for illustration purposes in some analyses MCID islevel-specific (eg HCR institution) and is different withineach ICD-10-GM code category In a sub-analysis chap-ters I to XIV (A-N) of the ICD-10-GM were analysed sep-arately (codes O-Z being used for specific purposes includ-ing pregnancy or various injuries) All analyses were doneusing the R statistical software [24] including the follow-ing extension packages vegan [25] ade4 [26]

Results

ICD-10-GM utilisation and DDI benchmarkBetween 2009 and 2015 9325326 hospitalisation caseswere reported A total of 16435 ICD-10-GM codes wereused in the study period Each hospitalisation case wascoded with on average 42 ICD-10-GM codes The 10most frequently used ICD10-GM codes (1644 codes) wereused in 87 (8094114 cases) of all hospitalisation casesThe Swiss-wide DDI (ICD-10-GM chapters AndashN) was718 The Swiss-wide DDI for individual chapters A to Nas well the MCID-threshold are given in table 2 Chap-ters A B (infections causes bacteria viruses) and E (en-docrine diseases) had a relatively low diagnostic diversity(DDI below 4) whereas diseases of the musculoskeletalsystem and connective tissue had the highest diagnostic di-versity (DDI above 5) The effect of misdiagnoses on theDDI was assessed by numerical simulation (fig 2) The re-lationship between the rates of misdiagnosis on the DDIshows a linear decrease in the lower misclassification ratesand a more abrupt decrease in the higher misclassificationrates The inclusion of 10 of misdiagnoses resulted in adecrease of 012 point in the DDI According to this sim-ulation a DDI of lt706 would represent the occurrence ofgt10 misdiagnoses in the observed case sample (MCID)A second MCID cut-off set at 20 misdiagnoses was alsoconsidered A further drop of 014 points in the DDI wasobserved when applying a MCID 20 threshold

Table 1 Categorisation of inpatient institutions in Switzerland in 2015

Type of hospital Number of institutions

Primary hospitals (university hospitals) 5

Secondary hospitals (level 2) 35

Tertiaryquaternary hospitals (levels 3 4 5) 66

Psychiatric clinics 49

Rehabilitation clinics 50

Specialised clinics 83

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 3 of 9

Characterisation of acute in-patient medical care insti-tutions in SwitzerlandThere was relevant DDI heterogeneity among hospitals inSwitzerland where the DDI was strongly associated withcaseload (fig 3) Simulated 10 and 20 misdiagnosisthresholds applied to each of the 14 ICD-10-GM chapters(AndashN) allowed Swiss hospitals to be categorised into fourdistinct groups (fig 4)

High-volume hospitals with an all-over comprehensive di-agnostic performance (n = 10 university large cantonalhospitals with an average caseload of 36000 hospitalisa-tions per year)

High- to mid-volume hospitals with a gradient of extensiveto relevant basic diagnostic performance in most chapters(n = 88 most cantonal regional hospitals with an averagecaseload of 8330 hospitalisations per year)

Low-volume specialised hospitals with a high diagnosticperformance in a single chapter (n = 48 with an averagecaseload of 2481 hospitalisations per year) and

Low-volume hospitals with inadequate diagnostic perfor-mance in all chapters (n = 146 with an average caseloadof 689 hospitalisations per year) The 146 low-volume hos-pitals had an estimated misdiagnosis rate of gt20 in allchapters and contributed to 8 (710188 out of 9325326)of all hospitalisations in Switzerland

Residential health care regions and health regionsrsquo in-stitutionsThe DDI observed in the 26 Swiss HCRs (fig 5 upperright panel) also showed relevant heterogeneity and strongassociations with ICD-10-GM code utilisation and case-load and hence size of the HCR (fig 5 left panels) Therewas a strong association of DDI and ICD-10-GM-codeutilisation (fig 5 inlet within upper-left panel) In smallerHCRs lt40 of possible ICD-10-GM codes were utilised

Table 2 Description of the ICD-10-GM categories

ICD-10-GM cate-gory

Description Usage(total = 39410015)

Percentage DDI MCID (10) MCID (20)

A Bacterial infections viral infections of thecentral nervous system and arthropod-borneviral fevers

372010 (094) 331 297 271

B Other viral infections and infections causedby fungi protozoans worms infestations andsequelae

715146 (181) 331 310 270

C Malignant neoplasms 1358151 (345) 458 419 412

D In situ benign neoplasms and neoplasms ofuncertainunknown behaviour

1328001 (337) 410 371 345

E Endocrine nutritional and metabolic diseases 3147705 (799) 372 341 323

F Mental and behavioural disorders 2238711 (568) 437 391 394

G Diseases of the nervous system 1240174 (315) 469 419 411

H Diseases of the eye and adnexa diseases ofthe ear and mastoid process

441895 (112) 479 375 331

I Diseases of the circulatory system 5956121 (1511) 418 390 385

J Diseases of the respiratory system 1570592 (399) 436 405 395

K Diseases of the digestive system 1949259 (495) 480 449 435

L Diseases of the skin and subcutaneous tis-sue

387950 (098) 462 409 405

M Diseases of the musculoskeletal system andconnective tissue

2778975 (705) 565 507 502

N Diseases of the genitourinary system 2159369 (548) 400 377 343

O Pregnancy childbirth and the puerperium 2177981 (553) NA NA NA

P Certain conditions originating in the perinatalperiod

415294 (105) NA NA NA

Q Congenital malformations deformations andchromosomal abnormalities

188454 (048) NA NA NA

R Symptoms signs and abnormal clinical andlaboratory findings not elsewhere classified

1946955 (494) NA NA NA

S Injury involving certain part of the body 1847718 (469) NA NA NA

T Injury involving multipleunspecified part ofthe body and poisoning

726275 (184) NA NA NA

U Codes for special purposes 295714 (075) NA NA NA

V Unintentional vehicletraffic injuries 107434 (027) NA NA NA

W Unintentional hitstruckbittendrowning suffo-cation exposure to electric current or radia-tion

116729 (03) NA NA NA

X Unintentional fireflamenatureenvironmentalpoisoning overexertionself-harm injuries

731188 (186) NA NA NA

Y Intentional assaulthomicide complication ofmedical and surgical care

819456 (208) NA NA NA

Z Factors influencing health status and contactwith health services

4392758 (1115) NA NA NA

The Swiss-wide code usage ie the number of times ICD-10-GM codes have been used within each category is provided as absolute number and percentage as well as thediagnostic diversity index (DDI) and the minimum clinically important difference (MCID) (10 and 20 misdiagnosis rate) are reported NA = not applicable

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 4 of 9

by HCR institutions in the observation period Institutionsof larger HCRs with a higher DDI utilised gt60 of theICD-10-GM codes All but one HCR with a populationgt363522 had a DDI above the 706 MCID threshold Insmaller HCRs (population lt363522) the DDI was most-ly below the MCID threshold and there was an increasingdifference between the DDI observed in the general popu-lation and the DDI provided by the HCR institution Thepopulation of such HCRs more often sought healthcare ser-

Figure 2 Diagnostic diversity index (DDI) benchmark ndash nu-merically simulated introduction of an increasing number ofmisdiagnoses and its effect observed on the DDI The overallSwiss-wide DDI level (before simulation) is represented by a greydashed line The simulation of 10 of misdiagnoses ndash defined asMCID 10 ndash resulted in a DDI decrease of 012 points or 706(blacked dashed line) A second cut-off set at 20 misdiagnosis(MCID 20) is also represented (lower horizontal blacked dashedline)

Figure 3 Association between the diagnostic diversity indexand the institutionsrsquo caseload The 10 misdiagnosis threshold(minimum clinically important difference MCID) is represented bya horizontal dashed lineLocal regression (loess) representing asmooth curve through the set of data points is depicted In generalinstitutions with an annual caseload gt7217 hospitalisation cases(intersection point where the loess regression crosses the 10misdiagnosis diagnostic diversity index [DDI] threshold) were morelikely to have an adequate DDI

vices in another HCR The lower diagnostic performancein small HCRs could partially be compensated for by con-sumption of health services outside the HCR (fig 5 low-er right panel) In smaller HCRs up to 62 of residentswere treated outside their HCR whereas around 90 ofresidents of larger HCRs (among those including universi-ty hospitals) were exclusively treated within their HCR of

Figure 4 Heat map of the diagnostic fitness (DDI) of acutemedical care hospitals within 14 ICD-10-GM chapters (AndashN)For each chapter the 10 (minimum clinically important differenceMCID) and 20 misdiagnosis thresholds were established by sim-ulation In the case of a DDI gt10 and gt20 threshold the re-spective bar is shown in dark and light grey for each chapter foreach of the hospitals The number of hospitalisation cases per in-stitution is coded in different bullet sizes on the right side (biggerindicating a larger caseload) This view allows four groups of acutemedical care hospitals to be distinguished (see text)

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 5 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

increased costs in specific well-defined situations (basichealth insurance) or more broadly with private insurance(approximately 15 of the population)

Hospitalisation databaseThe hospitalisation dataset used in this analysis was pro-vided by the Swiss Federal Office for Statistics It includesmedical information about all hospitalisations in Switzer-land since 1998 In this database the patient informationis fully anonymised No written informed consent wasgiven by patients who were unidentifiable owing to theanonymisation The data belong to the Swiss Federal Of-fice for Statistics (Bundesamt fuumlr Statistik NeuchacirctelSwitzerland) which provides regulated access to the datafor research purposes [19 20] The database included ge-ographical and temporal information (patientrsquos area of res-idence HCR [canton] of institution year and month ofhospitalisation length of hospital stay) as well as age atadmission and reasontype of discharge (including death)Hospitalisations within and outside of the area of residencywere considered Thus we could quantify the percentage ofhospitalisations taking place outside of the HCR of resi-dency Unique anonymised institution numbers for the cal-culation of annual caseload were available The patientsrsquolist of diagnoses included one main diagnosis as well asup to 50 additional diagnoses coded using ICD-10-GMcodes [21] The ICD-10-GM coding was uniformly usedthroughout the study period For the current publicationthe timespan of the analysis was restricted to 2009ndash2015Data prior 2009 were not included in order to keep theanalysed dataset as homogenous as possible In additiononly Swiss residents were considered [22]

Statistical analysisThe diagnostic diversity was measured using the Shannondiversity index [23] defined as follows

H = minus sumi = 1D piln (pi)

with D the number of diagnoses (ICD-10-GM codes) lnthe natural logarithm and pi the proportional abundance

of the ith diagnosis The Shannon diversity index origi-nally developed in information theory is one of the mostcommonly used diversity indices It is widely used in eco-logical science but has also been previously reported as apromising measurement in health service research [12] Inour analysis the Shannon diversity index accounts for boththe abundance and the evenness of the ICD-10-GM at theHCR or institution level It assumes that all ICD-10-GMcodes are equal and therefore treats equally the most abun-dant as well as the rarest codes This index has a lowerbound of 0 but no upper bound It increases logarithmical-ly proportionally to the diversity of codes employed Nu-merical simulations were performed to evaluate the effect

of misdiagnoses in the DDI We used nonparametric boot-strapping to resample the ICD-10-GM codes by removingan increasing proportion of ICD-10-GM codes in a preva-lence-dependent fashion More specifically we used thisprocedure to randomly remove rare ICD-10-GM codes re-placing them by randomly chosen more common codesThis resulted in an artificial decrease of the diagnostic di-versity and an increase in the rate of misdiagnoses We ar-bitrarily defined the minimum clinically important differ-ence (MCID) as a 10-misdiagnosis ie 1 in 10 patientsseen on the ward leaves the hospital with an impreciseor misdiagnosis This cut-off was chosen due to its clini-cal relevance The diagnostic performance was consideredsufficient when an institutionHCR has a DDI above thiscut-off A second cut-off was set at 20 misdiagnosis andused for illustration purposes in some analyses MCID islevel-specific (eg HCR institution) and is different withineach ICD-10-GM code category In a sub-analysis chap-ters I to XIV (A-N) of the ICD-10-GM were analysed sep-arately (codes O-Z being used for specific purposes includ-ing pregnancy or various injuries) All analyses were doneusing the R statistical software [24] including the follow-ing extension packages vegan [25] ade4 [26]

Results

ICD-10-GM utilisation and DDI benchmarkBetween 2009 and 2015 9325326 hospitalisation caseswere reported A total of 16435 ICD-10-GM codes wereused in the study period Each hospitalisation case wascoded with on average 42 ICD-10-GM codes The 10most frequently used ICD10-GM codes (1644 codes) wereused in 87 (8094114 cases) of all hospitalisation casesThe Swiss-wide DDI (ICD-10-GM chapters AndashN) was718 The Swiss-wide DDI for individual chapters A to Nas well the MCID-threshold are given in table 2 Chap-ters A B (infections causes bacteria viruses) and E (en-docrine diseases) had a relatively low diagnostic diversity(DDI below 4) whereas diseases of the musculoskeletalsystem and connective tissue had the highest diagnostic di-versity (DDI above 5) The effect of misdiagnoses on theDDI was assessed by numerical simulation (fig 2) The re-lationship between the rates of misdiagnosis on the DDIshows a linear decrease in the lower misclassification ratesand a more abrupt decrease in the higher misclassificationrates The inclusion of 10 of misdiagnoses resulted in adecrease of 012 point in the DDI According to this sim-ulation a DDI of lt706 would represent the occurrence ofgt10 misdiagnoses in the observed case sample (MCID)A second MCID cut-off set at 20 misdiagnoses was alsoconsidered A further drop of 014 points in the DDI wasobserved when applying a MCID 20 threshold

Table 1 Categorisation of inpatient institutions in Switzerland in 2015

Type of hospital Number of institutions

Primary hospitals (university hospitals) 5

Secondary hospitals (level 2) 35

Tertiaryquaternary hospitals (levels 3 4 5) 66

Psychiatric clinics 49

Rehabilitation clinics 50

Specialised clinics 83

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 3 of 9

Characterisation of acute in-patient medical care insti-tutions in SwitzerlandThere was relevant DDI heterogeneity among hospitals inSwitzerland where the DDI was strongly associated withcaseload (fig 3) Simulated 10 and 20 misdiagnosisthresholds applied to each of the 14 ICD-10-GM chapters(AndashN) allowed Swiss hospitals to be categorised into fourdistinct groups (fig 4)

High-volume hospitals with an all-over comprehensive di-agnostic performance (n = 10 university large cantonalhospitals with an average caseload of 36000 hospitalisa-tions per year)

High- to mid-volume hospitals with a gradient of extensiveto relevant basic diagnostic performance in most chapters(n = 88 most cantonal regional hospitals with an averagecaseload of 8330 hospitalisations per year)

Low-volume specialised hospitals with a high diagnosticperformance in a single chapter (n = 48 with an averagecaseload of 2481 hospitalisations per year) and

Low-volume hospitals with inadequate diagnostic perfor-mance in all chapters (n = 146 with an average caseloadof 689 hospitalisations per year) The 146 low-volume hos-pitals had an estimated misdiagnosis rate of gt20 in allchapters and contributed to 8 (710188 out of 9325326)of all hospitalisations in Switzerland

Residential health care regions and health regionsrsquo in-stitutionsThe DDI observed in the 26 Swiss HCRs (fig 5 upperright panel) also showed relevant heterogeneity and strongassociations with ICD-10-GM code utilisation and case-load and hence size of the HCR (fig 5 left panels) Therewas a strong association of DDI and ICD-10-GM-codeutilisation (fig 5 inlet within upper-left panel) In smallerHCRs lt40 of possible ICD-10-GM codes were utilised

Table 2 Description of the ICD-10-GM categories

ICD-10-GM cate-gory

Description Usage(total = 39410015)

Percentage DDI MCID (10) MCID (20)

A Bacterial infections viral infections of thecentral nervous system and arthropod-borneviral fevers

372010 (094) 331 297 271

B Other viral infections and infections causedby fungi protozoans worms infestations andsequelae

715146 (181) 331 310 270

C Malignant neoplasms 1358151 (345) 458 419 412

D In situ benign neoplasms and neoplasms ofuncertainunknown behaviour

1328001 (337) 410 371 345

E Endocrine nutritional and metabolic diseases 3147705 (799) 372 341 323

F Mental and behavioural disorders 2238711 (568) 437 391 394

G Diseases of the nervous system 1240174 (315) 469 419 411

H Diseases of the eye and adnexa diseases ofthe ear and mastoid process

441895 (112) 479 375 331

I Diseases of the circulatory system 5956121 (1511) 418 390 385

J Diseases of the respiratory system 1570592 (399) 436 405 395

K Diseases of the digestive system 1949259 (495) 480 449 435

L Diseases of the skin and subcutaneous tis-sue

387950 (098) 462 409 405

M Diseases of the musculoskeletal system andconnective tissue

2778975 (705) 565 507 502

N Diseases of the genitourinary system 2159369 (548) 400 377 343

O Pregnancy childbirth and the puerperium 2177981 (553) NA NA NA

P Certain conditions originating in the perinatalperiod

415294 (105) NA NA NA

Q Congenital malformations deformations andchromosomal abnormalities

188454 (048) NA NA NA

R Symptoms signs and abnormal clinical andlaboratory findings not elsewhere classified

1946955 (494) NA NA NA

S Injury involving certain part of the body 1847718 (469) NA NA NA

T Injury involving multipleunspecified part ofthe body and poisoning

726275 (184) NA NA NA

U Codes for special purposes 295714 (075) NA NA NA

V Unintentional vehicletraffic injuries 107434 (027) NA NA NA

W Unintentional hitstruckbittendrowning suffo-cation exposure to electric current or radia-tion

116729 (03) NA NA NA

X Unintentional fireflamenatureenvironmentalpoisoning overexertionself-harm injuries

731188 (186) NA NA NA

Y Intentional assaulthomicide complication ofmedical and surgical care

819456 (208) NA NA NA

Z Factors influencing health status and contactwith health services

4392758 (1115) NA NA NA

The Swiss-wide code usage ie the number of times ICD-10-GM codes have been used within each category is provided as absolute number and percentage as well as thediagnostic diversity index (DDI) and the minimum clinically important difference (MCID) (10 and 20 misdiagnosis rate) are reported NA = not applicable

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 4 of 9

by HCR institutions in the observation period Institutionsof larger HCRs with a higher DDI utilised gt60 of theICD-10-GM codes All but one HCR with a populationgt363522 had a DDI above the 706 MCID threshold Insmaller HCRs (population lt363522) the DDI was most-ly below the MCID threshold and there was an increasingdifference between the DDI observed in the general popu-lation and the DDI provided by the HCR institution Thepopulation of such HCRs more often sought healthcare ser-

Figure 2 Diagnostic diversity index (DDI) benchmark ndash nu-merically simulated introduction of an increasing number ofmisdiagnoses and its effect observed on the DDI The overallSwiss-wide DDI level (before simulation) is represented by a greydashed line The simulation of 10 of misdiagnoses ndash defined asMCID 10 ndash resulted in a DDI decrease of 012 points or 706(blacked dashed line) A second cut-off set at 20 misdiagnosis(MCID 20) is also represented (lower horizontal blacked dashedline)

Figure 3 Association between the diagnostic diversity indexand the institutionsrsquo caseload The 10 misdiagnosis threshold(minimum clinically important difference MCID) is represented bya horizontal dashed lineLocal regression (loess) representing asmooth curve through the set of data points is depicted In generalinstitutions with an annual caseload gt7217 hospitalisation cases(intersection point where the loess regression crosses the 10misdiagnosis diagnostic diversity index [DDI] threshold) were morelikely to have an adequate DDI

vices in another HCR The lower diagnostic performancein small HCRs could partially be compensated for by con-sumption of health services outside the HCR (fig 5 low-er right panel) In smaller HCRs up to 62 of residentswere treated outside their HCR whereas around 90 ofresidents of larger HCRs (among those including universi-ty hospitals) were exclusively treated within their HCR of

Figure 4 Heat map of the diagnostic fitness (DDI) of acutemedical care hospitals within 14 ICD-10-GM chapters (AndashN)For each chapter the 10 (minimum clinically important differenceMCID) and 20 misdiagnosis thresholds were established by sim-ulation In the case of a DDI gt10 and gt20 threshold the re-spective bar is shown in dark and light grey for each chapter foreach of the hospitals The number of hospitalisation cases per in-stitution is coded in different bullet sizes on the right side (biggerindicating a larger caseload) This view allows four groups of acutemedical care hospitals to be distinguished (see text)

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 5 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

Characterisation of acute in-patient medical care insti-tutions in SwitzerlandThere was relevant DDI heterogeneity among hospitals inSwitzerland where the DDI was strongly associated withcaseload (fig 3) Simulated 10 and 20 misdiagnosisthresholds applied to each of the 14 ICD-10-GM chapters(AndashN) allowed Swiss hospitals to be categorised into fourdistinct groups (fig 4)

High-volume hospitals with an all-over comprehensive di-agnostic performance (n = 10 university large cantonalhospitals with an average caseload of 36000 hospitalisa-tions per year)

High- to mid-volume hospitals with a gradient of extensiveto relevant basic diagnostic performance in most chapters(n = 88 most cantonal regional hospitals with an averagecaseload of 8330 hospitalisations per year)

Low-volume specialised hospitals with a high diagnosticperformance in a single chapter (n = 48 with an averagecaseload of 2481 hospitalisations per year) and

Low-volume hospitals with inadequate diagnostic perfor-mance in all chapters (n = 146 with an average caseloadof 689 hospitalisations per year) The 146 low-volume hos-pitals had an estimated misdiagnosis rate of gt20 in allchapters and contributed to 8 (710188 out of 9325326)of all hospitalisations in Switzerland

Residential health care regions and health regionsrsquo in-stitutionsThe DDI observed in the 26 Swiss HCRs (fig 5 upperright panel) also showed relevant heterogeneity and strongassociations with ICD-10-GM code utilisation and case-load and hence size of the HCR (fig 5 left panels) Therewas a strong association of DDI and ICD-10-GM-codeutilisation (fig 5 inlet within upper-left panel) In smallerHCRs lt40 of possible ICD-10-GM codes were utilised

Table 2 Description of the ICD-10-GM categories

ICD-10-GM cate-gory

Description Usage(total = 39410015)

Percentage DDI MCID (10) MCID (20)

A Bacterial infections viral infections of thecentral nervous system and arthropod-borneviral fevers

372010 (094) 331 297 271

B Other viral infections and infections causedby fungi protozoans worms infestations andsequelae

715146 (181) 331 310 270

C Malignant neoplasms 1358151 (345) 458 419 412

D In situ benign neoplasms and neoplasms ofuncertainunknown behaviour

1328001 (337) 410 371 345

E Endocrine nutritional and metabolic diseases 3147705 (799) 372 341 323

F Mental and behavioural disorders 2238711 (568) 437 391 394

G Diseases of the nervous system 1240174 (315) 469 419 411

H Diseases of the eye and adnexa diseases ofthe ear and mastoid process

441895 (112) 479 375 331

I Diseases of the circulatory system 5956121 (1511) 418 390 385

J Diseases of the respiratory system 1570592 (399) 436 405 395

K Diseases of the digestive system 1949259 (495) 480 449 435

L Diseases of the skin and subcutaneous tis-sue

387950 (098) 462 409 405

M Diseases of the musculoskeletal system andconnective tissue

2778975 (705) 565 507 502

N Diseases of the genitourinary system 2159369 (548) 400 377 343

O Pregnancy childbirth and the puerperium 2177981 (553) NA NA NA

P Certain conditions originating in the perinatalperiod

415294 (105) NA NA NA

Q Congenital malformations deformations andchromosomal abnormalities

188454 (048) NA NA NA

R Symptoms signs and abnormal clinical andlaboratory findings not elsewhere classified

1946955 (494) NA NA NA

S Injury involving certain part of the body 1847718 (469) NA NA NA

T Injury involving multipleunspecified part ofthe body and poisoning

726275 (184) NA NA NA

U Codes for special purposes 295714 (075) NA NA NA

V Unintentional vehicletraffic injuries 107434 (027) NA NA NA

W Unintentional hitstruckbittendrowning suffo-cation exposure to electric current or radia-tion

116729 (03) NA NA NA

X Unintentional fireflamenatureenvironmentalpoisoning overexertionself-harm injuries

731188 (186) NA NA NA

Y Intentional assaulthomicide complication ofmedical and surgical care

819456 (208) NA NA NA

Z Factors influencing health status and contactwith health services

4392758 (1115) NA NA NA

The Swiss-wide code usage ie the number of times ICD-10-GM codes have been used within each category is provided as absolute number and percentage as well as thediagnostic diversity index (DDI) and the minimum clinically important difference (MCID) (10 and 20 misdiagnosis rate) are reported NA = not applicable

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 4 of 9

by HCR institutions in the observation period Institutionsof larger HCRs with a higher DDI utilised gt60 of theICD-10-GM codes All but one HCR with a populationgt363522 had a DDI above the 706 MCID threshold Insmaller HCRs (population lt363522) the DDI was most-ly below the MCID threshold and there was an increasingdifference between the DDI observed in the general popu-lation and the DDI provided by the HCR institution Thepopulation of such HCRs more often sought healthcare ser-

Figure 2 Diagnostic diversity index (DDI) benchmark ndash nu-merically simulated introduction of an increasing number ofmisdiagnoses and its effect observed on the DDI The overallSwiss-wide DDI level (before simulation) is represented by a greydashed line The simulation of 10 of misdiagnoses ndash defined asMCID 10 ndash resulted in a DDI decrease of 012 points or 706(blacked dashed line) A second cut-off set at 20 misdiagnosis(MCID 20) is also represented (lower horizontal blacked dashedline)

Figure 3 Association between the diagnostic diversity indexand the institutionsrsquo caseload The 10 misdiagnosis threshold(minimum clinically important difference MCID) is represented bya horizontal dashed lineLocal regression (loess) representing asmooth curve through the set of data points is depicted In generalinstitutions with an annual caseload gt7217 hospitalisation cases(intersection point where the loess regression crosses the 10misdiagnosis diagnostic diversity index [DDI] threshold) were morelikely to have an adequate DDI

vices in another HCR The lower diagnostic performancein small HCRs could partially be compensated for by con-sumption of health services outside the HCR (fig 5 low-er right panel) In smaller HCRs up to 62 of residentswere treated outside their HCR whereas around 90 ofresidents of larger HCRs (among those including universi-ty hospitals) were exclusively treated within their HCR of

Figure 4 Heat map of the diagnostic fitness (DDI) of acutemedical care hospitals within 14 ICD-10-GM chapters (AndashN)For each chapter the 10 (minimum clinically important differenceMCID) and 20 misdiagnosis thresholds were established by sim-ulation In the case of a DDI gt10 and gt20 threshold the re-spective bar is shown in dark and light grey for each chapter foreach of the hospitals The number of hospitalisation cases per in-stitution is coded in different bullet sizes on the right side (biggerindicating a larger caseload) This view allows four groups of acutemedical care hospitals to be distinguished (see text)

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 5 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

by HCR institutions in the observation period Institutionsof larger HCRs with a higher DDI utilised gt60 of theICD-10-GM codes All but one HCR with a populationgt363522 had a DDI above the 706 MCID threshold Insmaller HCRs (population lt363522) the DDI was most-ly below the MCID threshold and there was an increasingdifference between the DDI observed in the general popu-lation and the DDI provided by the HCR institution Thepopulation of such HCRs more often sought healthcare ser-

Figure 2 Diagnostic diversity index (DDI) benchmark ndash nu-merically simulated introduction of an increasing number ofmisdiagnoses and its effect observed on the DDI The overallSwiss-wide DDI level (before simulation) is represented by a greydashed line The simulation of 10 of misdiagnoses ndash defined asMCID 10 ndash resulted in a DDI decrease of 012 points or 706(blacked dashed line) A second cut-off set at 20 misdiagnosis(MCID 20) is also represented (lower horizontal blacked dashedline)

Figure 3 Association between the diagnostic diversity indexand the institutionsrsquo caseload The 10 misdiagnosis threshold(minimum clinically important difference MCID) is represented bya horizontal dashed lineLocal regression (loess) representing asmooth curve through the set of data points is depicted In generalinstitutions with an annual caseload gt7217 hospitalisation cases(intersection point where the loess regression crosses the 10misdiagnosis diagnostic diversity index [DDI] threshold) were morelikely to have an adequate DDI

vices in another HCR The lower diagnostic performancein small HCRs could partially be compensated for by con-sumption of health services outside the HCR (fig 5 low-er right panel) In smaller HCRs up to 62 of residentswere treated outside their HCR whereas around 90 ofresidents of larger HCRs (among those including universi-ty hospitals) were exclusively treated within their HCR of

Figure 4 Heat map of the diagnostic fitness (DDI) of acutemedical care hospitals within 14 ICD-10-GM chapters (AndashN)For each chapter the 10 (minimum clinically important differenceMCID) and 20 misdiagnosis thresholds were established by sim-ulation In the case of a DDI gt10 and gt20 threshold the re-spective bar is shown in dark and light grey for each chapter foreach of the hospitals The number of hospitalisation cases per in-stitution is coded in different bullet sizes on the right side (biggerindicating a larger caseload) This view allows four groups of acutemedical care hospitals to be distinguished (see text)

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 5 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

residency This additional diagnostic performance of exter-nal healthcare institutions could increase but in most casesnot entirely correct a reduced diagnostic performance ofHCRs concerned (fig 5 lower left panel)

Discussion

Statement of principal findingsWe found significant spatial and institutional heterogeneityof the DDI calculated from ICD-10-GM codes for acute in-patient medical care in Switzerland Because of the naturaldiversity of human beings a higher DDI is more likely to

approach the true biological diagnostic diversity of a pop-ulation (fig 1 ndash working hypothesis) Although it is im-practicable to verify the correctness of diagnoses of eachof the hospital files [16] the fact that some institutionsrarely coded low-frequency conditions and thus had a lowDDI points to diagnostic limitations Thus regions and in-stitutions with less diagnostic diversity are more likely tounder- or misdiagnose a significant proportion of patientsmissing the chance of an optimal alignment between healthcondition and treatment ndash with the potential for worse out-come

Figure 5 Diagnostic diversity index (DDI) in Swiss health care regions The upper left panel displays the relationship between the size ofthe healthcare region (population) and the diagnostic diversity index generated by healthcare regionsrsquo institutions (inset including the represen-tation of the diagnostic diversity as a function of the percentage of ICD-10-GM codes used) The minimum clinically important difference(MCID) of 10 misdiagnosis is represented using a dashed line The intersection point where the MCID 10 misdiagnosis and the loess re-gression curve cross defines the cut-off of sufficient diagnostic diversity In general health care regions with a population size gt363522 werelikely to have an adequate DDI The upper right panel provides choropleth representation of the DDI generated by the institutions of eachhealthcare region The lower left panel shows the relationship between the size of the healthcare regions (population) and the DDI of the re-gionrsquos residents The lower right panel depicts the relationship between the DDI generated by healthcare regionrsquos institutions and the percent-age of hospitalisation outside of health care regions The dashed line represents the DDI threshold of 10 misclassification The health careregions (Swiss cantons) are coded as follows AG = Aargau AI = Appenzell Innerrhoden AR = Appenzell Ausserrhoden BE = Bern BL =Basel-Landschaft BS = Basel-Stadt FR = Fribourg GE = Geneva GL = Glarus GR = Graubuumlnden JU = Jura LU = Luzern NE = NeuchatelNW = Nidwalden OW = Obwalden SG = St Gallen SH = Schaffhausen SO = Solothurn SZ = Schwyz TG = Turgau TI = Ticino UR = UriVD = Vaud VS = Valais ZG = Zug ZH = Zuumlrich

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 6 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

Implications for clinicians and policy makersThere was a significant interaction between caseload andDDI Acute medical care institutions with an annual case-load of gt7217 cases per year had a good chance to have anacceptable DDI and thus deliver full diagnostic qualityIn Switzerland however 50 of 292 hospitals deliver in-adequate diagnostic precision (ie did not reach MCID inany AndashN category) These mostly small institutions weretreating only 8 of all hospital cases Extrapolated fromthe DDI as many as 6 out of 10 patients are under- or mis-diagnosed in certain Swiss low-volume institutions (lt10thpercentile) The effect of institutional caseload and out-come has already been shown for different diseases andsurgical interventions [4] Our findings allow an intuitivealignment between higher diagnostic precision ndash proba-bly associated with improved outcome ndash and caseloadIn Switzerland the cantons represent fairly closed HCRslimiting reimbursement for healthcare consumption out-side of the residential HCR Although varying significant-ly between HCRs in 2015 only 19 of patients profitedfrom cross-cantonal border healthcare [27] HCRs with apopulation of gt363522 were likely to deliver a sufficientDDI and thus a good diagnostic performance for their res-idents We could not identify a diagnostic diversity-drivenupper limit for the population size of a healthcare regionIn line with the paradigm of the importance of caseloadsmaller HCRs ie cantons in Switzerland may do betterto contract with other HCRs rather than invest in their ownlow-volume higher-care institutions

It is mandatory for Swiss residents to have basic health in-surance coverage giving them access to all levels of health-care Approximately 32 of the Swiss population receivessocial support from the state slightly more in urban ag-glomerations [28] There are practically no relevant dispar-ities between communities and regions in socioeconom-ic factors that impact on healthcare utilisation and thuson our results In Switzerland the ICD-10-GM codes havebeen the nationwide basis for drug-related group (DRG)reimbursement since 2009 assuring a level of attention toguarantee data quality On the other hand the associationwith reimbursement might impact on the ICD-10-GM cod-ing process [29] As the financial pressure applies to allcoding institutions it is likely that reimbursement-drivencoding trends related to specific ICD-10-GM-codes wouldbe mutually balanced out [8] Extensive up-coding wouldif it ever occurs negatively impact diagnostic diversityThe calculation of DDI using the Shannon diversity indexproved to be unaffected by socio-culturo-linguistic factorsand the numerical caseload in the range observed in thismanuscript

Patients with a severe health condition requiring hospitali-sation expect and deserve high-level care competence [30]Institutions with sufficient caseload and as a consequencedelivering primary as well as higher-level healthcare areshown to reach sufficient upfront diagnostic precision In-stitutional caseload and size of HCRs should thus moreconsequently be the basis for a rational healthcare plan-ning The calculation of the DDI from ICD-10-codes issimple and complements the information derived from oth-er quality indicators as it sheds a light on the fitness ofthe institutional interplay between primary and specialisedmedical in-patient care

Furthermore in some specific conditions such as hospi-talisations due to pneumonia we found that different sub-groups of ICD-10 codes (implying different treatments)were more frequently used in larger hospitals than smallerones (appendix 1) This in turn might impact on the pa-tient management

LimitationsLimitations in the use of the DDI include the fact that notall institutions have the same professionalism in terms ofa coding unit On the other hand since the introductionof a new reimbursement system in the Swiss health carestarting on 1 January 2012 and based on DRGs the levelof coding in all Swiss institutions has been uniformly im-proved

Another limitation comes from the fact that the institutionsin the hospitalisation database were anonymously codedwhich made detailed institution-oriented analyses difficultAlso there might be some more-or-less systematic referralof particular patient groups to larger institutions

No gold standard for correct diagnosis exists in the hos-pitalisation database Misdiagnosis and its link with DDIcould only be assessed empirically using numerical simu-lations

Finally in Switzerland only hospitalised cases are system-atically recorded in a nation-wide fashion Therefore ouranalysis is limited to hospitalised cases

Financial disclosureThe study was supported by an unconditional research grant by theLungenliga St Gallen and an institutional grant by the KantonsspitalSt Gallen The funders had no role in study design data collection andanalysis decision to publish or preparation of the manuscript

Potential competing interestsThe authors declare that no conflict of interests exists

References1 WHO ndash Global Reference List of 100 Core Health Indicators

httpwwwwhointhealthinfoindicators2015en 20152 Ng N Hakimi M Santosa A Byass P Wilopo SA Wall S Is self-rated

health an independent index for mortality among older people in Indone-sia PLoS One 20127(4) doi httpdxdoiorg101371jour-nalpone0035308 PubMed

3 WHO - World Health Statistics 2016 httpwwwwhointghopublica-tionsworld_health_statisticsen 2016

4 Amato L Colais P Davoli M Ferroni E Fusco D Minozzi S et al[Volume and health outcomes evidence from systematic reviews andfrom evaluation of Italian hospital data] Epidemiol Prev 201337(2-3Suppl 2)1ndash100 Article in Italian PubMed

5 Keygnaert IGA Ivanova O What is the Evidence on the Reduction ofInequalities in Accessibility and Quality of Maternal Health Care Deliv-ery for Migrants A Review of the Existing Evidence in the WHO Euro-pean Region Copenhagen WHO Regional Office for Europe 2016

6 Christianson JB Shaw BW Greene J Scanlon DP Reporting providerperformance what can be learned from the experience of multi-stake-holder community coalitions Am J Manag Care 201622(12 Sup-pl)s382ndash92 PubMed

7 Shi Y Scanlon DP Kang R McHugh M Greene J Christianson JB etal The longitudinal impact of Aligning Forces for Quality on measuresof population health quality and experience of care and cost of careAm J Manag Care 201622(12 Suppl)s373ndash81 PubMed

8 McNutt R Johnson TJ Odwazny R Remmich Z Skarupski K MeurerS et al Change in MS-DRG assignment and hospital reimbursement asa result of Centers for Medicare amp Medicaid changes in payment forhospital-acquired conditions is it coding or quality Qual Manag HealthCare 201019(1)17ndash24 doi httpdxdoiorg101097QMH0b013e3181ccbd07 PubMed

9 Ketelaar NA Faber MJ Flottorp S Rygh LH Deane KH Eccles MPPublic release of performance data in changing the behaviour of health-

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 7 of 9

care consumers professionals or organisations Cochrane Database SystRev 2011(11) PubMed

10 Faber M Bosch M Wollersheim H Leatherman S Grol R Public re-porting in health care how do consumers use quality-of-care informa-tion A systematic review Med Care 200947(1)1ndash8 doihttpdxdoiorg101097MLR0b013e3181808bb5 PubMed

11 Totten AM Wagner J Tiwari A OrsquoHaire C Griffin J Walker M Clos-ing the quality gap revisiting the state of the science (vol 5 public re-porting as a quality improvement strategy) Evid Rep Technol Assess(Full Rep) 2012(2085)1ndash645 PubMed

12 Schuster F Ostermann T Emcke T Schuster R Analysis of Age andGender Structures for ICD-10 Diagnoses in Outpatient Treatment UsingShannonrsquos Entropy Stud Health Technol Inform 201724352ndash6PubMed

13 Bell AV Diagnostic diversity the role of social class in diagnostic expe-riences of infertility Sociol Health Illn 201436(4)516ndash30 doihttpdxdoiorg1011111467-956612083 PubMed

14 Murray CJ Lopez AD Measuring the global burden of disease N EnglJ Med 2013369(5)448ndash57 doi httpdxdoiorg101056NEJM-ra1201534 PubMed

15 Khullar D Jena AB Reducing prognostic errors a new imperative inquality healthcare BMJ 2016352i1417 doi httpdxdoiorg101136bmji1417 PubMed

16 OrsquoMalley KJ Cook KF Price MD Wildes KR Hurdle JF Ashton CMMeasuring diagnoses ICD code accuracy Health Serv Res 200540(5Pt 2)1620ndash39 doi httpdxdoiorg101111j1475-6773200500444x PubMed

17 Singh H Meyer AN Thomas EJ The frequency of diagnostic errors inoutpatient care estimations from three large observational studies in-volving US adult populations BMJ Qual Saf 201423(9)727ndash31 doihttpdxdoiorg101136bmjqs-2013-002627 PubMed

18 Euro Health Consumer index 2015 httpwwwhealthpowerhousecompublicationseuro-health-consumer-index-2015 2016

19 Baty F Putora PM Isenring B Blum T Brutsche M Comorbidities andburden of COPD a population based case-control study PLoS One

20138(5) doi httpdxdoiorg101371journalpone0063285PubMed

20 Pohle S Baty F Brutsche M In-Hospital Disease Burden of Sarcoidosisin Switzerland from 2002 to 2012 PLoS One 201611(3) doihttpdxdoiorg101371journalpone0151940 PubMed

21 ICD-10-GM httpwwwwhointclassificationsicden 201722 Swiss-DRG Kodierleitfaden 2017 httpswwwbfsadminchbfsde

homestatistikengesundheitnomenklaturenmedkkinstrumente-medi-zinische-kodierungassetdetail1000136html 2017

23 Shannon CE A mathematical theory of communication Bell Syst TechJ 194827(3)379ndash423 623ndash56 doi httpdxdoiorg101002j1538-73051948tb01338x

24 R Core Team R A language and environment for statistical computing[Internet] Vienna Austria R Foundation for Statistical Computing2016 Available httpswwwR-projectorg

25 Dray S Dufour AB The ade4 package implementing the duality dia-gram for ecologists J Stat Softw 200722(4)1ndash20 doihttpdxdoiorg1018637jssv022i04 PubMed

26 Jari Oksanen F Blanchet G Friendly M Kindt R Legendre P McGlinnD et al vegan Community Ecology Package R package version 25-22018 httpsCRANR-projectorgpackage=vegan

27 BFS-Spitalstatistik 2015 httpswwwadminchgovdestartdokumen-tationmedienmitteilungenmsg-id-64676html 2015

28 BFS-Soziale Sicherheit 2016 httpswwwadminchgovdestartdoku-mentationmedienmitteilungenmsg-id-64985html 2016

29 Hoffman GE Schenker M Wilke MH Impact of laboratory testing onDRG coding and reimbursement--results of a data base research ClinLab 200248(5-6)327ndash33 PubMed

30 Bjertnaes OA Sjetne IS Iversen HH Overall patient satisfaction withhospitals effects of patient-reported experiences and fulfilment of ex-pectations BMJ Qual Saf 201221(1)39ndash46 doi httpdxdoiorg101136bmjqs-2011-000137 PubMed

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 8 of 9

Appendix 1 Relationship between DDI and medical man-agement

The appendix is available as a separate file for download-ing at httpssmwchenarticledoismw201814691

Original article Swiss Med Wkly 2018148w14691

Swiss Medical Weekly middot PDF of the online version middot wwwsmwch

Published under the copyright license ldquoAttribution ndash Non-Commercial ndash No Derivatives 40rdquoNo commercial reuse without permission See httpemhchenservicespermissionshtml

Page 9 of 9