Academic impact and rankings of neuroendovascular ... · Vascular Research Center, University at Buffalo, State University of New York; 4Department of Neurosurgery, Gates Vascular

CLINICAL ARTICLEJ Neurosurg 127:1181–1189, 2017

Publication has become a major criterion of success in the competitive academic environment of neuro-surgery. Research productivity has been linked with

the ability to rise to the upper echelons in the realm of academics. In fact, tenure-track appointment is directly

related to the number of scientific publications of an in-dividual.8,10,12 Citation analysis is a central tool of biblio-metric analysis, which is a statistical method used to quan-titatively analyze scholarly publications. An individual is the usual subject of a bibliometric analysis. However, the

ABBREVIATIONS ACGME = Accreditation Council for Graduate Medical Education; CAST = Committee on Advanced Subspecialty Training; DNV = Det Norske Veritas; h10 index = h index for publications in the last 10 years; h index-dept = departmental h index; h index-neuroendovascular = h index for the neuroendovascular fellowship; JC = Joint Commission; UCLA = University of California, Los Angeles; UCSF = University of California, San Francisco.SUBMITTED July 16, 2016. ACCEPTED September 15, 2016.INCLUDE WHEN CITING Published online January 13, 2017; DOI: 10.3171/2016.9.JNS161857.

Academic impact and rankings of neuroendovascular fellowship programs across the United StatesAshish Sonig, MD, MS, MCh,1,4 Hussain Shallwani, MD,1,4 Bennett R. Levy,4 Hakeem J. Shakir, MD,1,4 and Adnan H. Siddiqui, MD, PhD1–5

Departments of 1Neurosurgery and 2Radiology, Jacobs School of Medicine and Biomedical Sciences, and 3Toshiba Stroke and Vascular Research Center, University at Buffalo, State University of New York; 4Department of Neurosurgery, Gates Vascular Institute at Kaleida Health; and 5Jacobs Institute, Buffalo, New York

OBJECTIVE Publication has become a major criterion of success in the competitive academic environment of neuro-surgery. This is the first study that has used departmental h index– and e index–based matrices to assess the academic output of neuroendovascular, neurointerventional, and interventional radiology fellowship programs across the continen-tal US.METHODS Fellowship program listings were identified from academic and organization websites. Details for 37 pro-grams were available. Bibliometric data for these programs were gathered from the Thomson Reuters Web of Science database. Citations for each publication from the fellowship’s parent department were screened, and the h and e indices were calculated from non–open-surgical, central nervous system vascular publications. Variables including “high-pro-ductivity” centers, fellowship–comprehensive stroke center affiliation, fellowship accreditation status, neuroendovascular h index, e index (h index supplement), h10 index (publications during the last 10 years), and departmental faculty-based h indices were created and analyzed.RESULTS A positive correlation was seen between the neuroendovascular fellowship h index and corresponding h10 index (R = 0.885; p < 0.0001). The mean, median, and highest faculty-based h indices exhibited positive correlations with the neuroendovascular fellowship h index (R = 0.662, p < 0.0001; R = 0.617, p < 0.0001; and R = 0.649, p < 0.0001, respectively). There was no significant difference (p = 0.824) in the median values for the fellowship h index based on comprehensive stroke center affiliation (30 of 37 programs had such affiliations) or accreditation (18 of 37 programs had accreditation) (p = 0.223). Based on the quartile analysis of the fellowship h index, 10 of 37 departments had an neuro-endovascular h index of ≥ 54 (“high-productivity” centers); these centers had significantly more faculty (p = 0.013) and a significantly higher mean faculty h index (p = 0.0001).CONCLUSIONS The departmental h index and analysis of its publication topics can be used to calculate the h index of an associated subspecialty. The analysis was focused on the neuroendovascular specialty, and this methodology can be extended to other neurosurgical subspecialties. Individual faculty research interest is directly reflected in the research productivity of a department. High-productivity centers had significantly more faculty with significantly higher individual h indices. The current systems for neuroendovascular fellowship program accreditation do not have a meaningful impact on academic productivity.https://thejns.org/doi/abs/10.3171/2016.9.JNS161857KEY WORDS bibliometrics; cerebrovascular; citation analysis; e index; endovascular neurosurgery; endovascular surgical neuroradiology; fellowship program; h index; interventional neuroradiology; neuroendovascular surgery; neurointerventional; neurosurgery department; stroke; subspecialty

©AANS, 2017 J Neurosurg Volume 127 • November 2017 1181

Unauthenticated | Downloaded 06/22/20 12:44 PM UTC

A. Sonig et al.

J Neurosurg Volume 127 • November 20171182

analysis can be extended to include a department, group, or journal. Several methods have been employed to evalu-ate the productivity of an individual or a department, the commonest of which is the h index.8,10,12 Proposed by a physicist from the University of California, San Diego, Jorge E. Hirsh, the h index is “the number of papers with citation number ≥ h as a useful index to characterize the scientific output of a researcher.”6 Another index that sup-plements the h index is the e index, which was proposed by Zhang.13 The e index accounts for not only all publica-tions that are part of the h index but also the cumulative ci-tations of these publication. Thus, it provides information on ignored citations. Other indices that have been used are the g index3 and the m quotient.8

The number of neuroendovascular and neurointer-ventional fellowship training programs in the US has in-creased in recent years.7 In this scenario, in which more than 50 neurosurgical programs offer these fellowships, it is of paramount importance to assess the academic pro-ductivity of such training programs and to know if ac-creditation with an authority or board has an impact on academic productivity. In the present study, our primary objective was to assess the academic output of neuroen-dovascular and neurointerventional programs across the continental US using h and e index–based matrices. To the best of our knowledge, this is the first study in which departmental academic productivity for this subspecialty has been assessed. An earlier study of pediatric neurosur-gery focused on the h indices of individual faculty mem-bers.9 This method may not be reliable, as faculty can move from one program to another, and thus the results of such an analysis may not reflect the true productivity of the fellowship program. Currently, few neurosurgical fel-lowships are accredited by a formal standardized system. Our secondary aim was to determine if a difference exists in terms of the academic productivity of fellowships with and without accreditation and with and without affiliation with a comprehensive stroke center.

MethodsListings for neuroendovascular, endovascular surgi-

cal neuroradiology, neuroendovascular surgery, and in-terventional neuroradiology fellowship programs were obtained from the websites of the fellowship directories of the American Association of Neurological Surgeons (http://www.aans.org/grants%20and%20fellowships/ fellowship%20directory.aspx), Accreditation Council for Graduate Medical Education (ACGME) (https://apps.acgme.org/ads/public/reports/report/1), and the Commit-tee on Advanced Subspecialty Training (CAST) of the Society of Neurological Surgeons (www.societyns.org/fellowships/NeuroEndovascularNeurosurgery.asp). Only those programs for which a description of a neuroendo-vascular, endovascular surgical neuroradiology, neuro-endovascular surgery, or interventional neuroradiology fellowship was available on the website of the parent de-partment or institution were included in the final analysis (Appendix 2). Thus, institutions offering only a noninter-ventional stroke fellowship were excluded. Additionally, information was gathered on the CAST or ACGME ac-

creditation status of the fellowship program and the af-filiation of the parent department or institution with com-prehensive stroke center certification (Det Norske Veri-tas [DNV; http://dnvglhealthcare.com/hospitals?search_ type=and&q=&c=20806&c=19200&prSubmit=Search] or the Joint Commission [JC; [http://www.strokecenter.org/trials/centers?utf8=L&search=14209]).

Bibliometric data for all years through May of 2016 were gathered from the Thomson Reuters Web of Science citation database (http://login.webofknowledge.com). Because neu-roendovascular training closely involves other departments and neurointerventionists can have a training background in radiology or neurology and neurosurgery, we used the fol-lowing search strings: neurol*surg*(neurological surgery), neurosurg*(neurosurgery), neurolog*(neurology), neuroendovasc*(neuroendovascular), neuroradiol*(neuroradiology), cerebrovasc*(cerebrovascular), and stroke*(stroke). These search strings were attached to the institute’s address strings (Appendix 1).10 Thus, a cumulative h index was generated, which is referred to as the departmental h index (h index-dept) (Table 1).

Calculation of the Neuroendovascular h and e IndicesEach publication citation included in h index-dept was

screened, and the h index for the neuroendovascular fel-lowship (referred to as h index-neuroendovascular) was calculated from citations in the CNS vascular domain, ex-cluding those that were limited to microneurosurgical nu-ances or microneurosurgical surgical anatomy. Similarly, the h index-neuroendovascular for citations during the last 10 years (referred to as h10 index) and the h index of indi-vidual neuroendovascular-trained faculty members at each parent department were calculated. Multiinstitutional data were equally distributed among all included institutes. For example, all institutions that were attributed to the authors of a publication received equal credit for that publication in the calculation of the h index for the department.

Next, h index-neuroendovascular was subjected to quartile analysis. Neuroendovascular fellowship programs falling in the ≥ 75th percentile were labeled as “high-pro-ductivity” centers in terms of academic works. Correlation analysis was done for these high-productivity centers with other parameters, including affiliation with a comprehen-sive stroke center, number of faculty members, accredita-tion status, and “highest” h index of neuroendovascular-trained faculty members from each department.

To calculate the e index, we used the formula13 below:

where citj are the citations received by the jth paper (j re-fers to the serial number when the citations are arranged in descending order of the number of times cited) and denotes excess citations within the h core (the h core comprises all articles within the h index).13 Figure 1 illustrates the calcu-lation of the neuroendovascular h, h10, and e indices.

Calculation of Faculty h IndexThe faculty h index was calculated from the Thomson

Reuters Web of Science citation database (http://login.


https://thejns.org/doi/suppl/10.3171/2016.9.JNS161857


Academic impact and rankings of US neuroendovascular fellowship programs

J Neurosurg Volume 127 • November 2017 1183

TABL

E 1.

Neur

oend

ovas

cula

r fell

owsh

ip ra

nkin

gs b

ased

on

the n

euro

endo

vasc

ular

fello

wshi

p h

inde

x

Institu

tion

Loca

tion

CAST

/AC

GME

Accr

edita

tion

Affili

ation

w/

Comp

rehe

nsive

St

roke

Cen

ter

(JC/D

NV)

Depa

rtmen

tal

h Ind

ex

Neur

oend

ovas

cular

Fe

llows

hip

h Ind

exh1

0 In

dex

e In

dex

No. o

f Ne

uroe

ndov

ascu

lar-

Train

ed

Facu

lty

High

est

h Ind

ex

of Fa

culty

Mea

n h

Inde

x of

Depa

rtmen

t Fa

culty

Med

ian

h Ind

ex

of Fa

culty

Stan

ford

Univ

ersit

ySt

anfo

rd, C

ANo

Yes

187

103

3912

1.35

453

27.5

024

UCSF

San F

ranc

isco,

CANo

No17

582

4011

5.24

674

37.5

036

UCLA

Los A

ngele

s, CA

NoYe

s14

277

4885

.715

5428

.60

22Un

ivers

ity at

Buf

falo

Buffa

lo, N

YYe

sYe

s72

7144

69.14

467

40.2

539

.5Jo

hns H

opkin

s Univ

ersit

yBa

ltimor

e, M

DYe

sYe

s17

764

3374

.215

3013

.80

14W

ashin

gton U

niver

sity

St Lo

uis, M

OYe

sNo

192

6323

89.0

33

4438

.00

40St

. Jos

eph’s

Hos

pital

(Bar

row)

Phoe

nix, A

ZYe

sNo

115

6234

80.01

238

38.0

038

Unive

rsity

of P

ittsbu

rgh

Pitts

burg

h, PA

NoYe

s11

159

3586

.26

427

19.0

017

.5Ne

wYor

k-Pr

esby

terian

/Cor

nell

Unive

rsity

New

York

, NY

NoYe

s14

857

2482

.46

344

21.0

011

Clev

eland

Clin

icCl

evela

nd, O

HYe

sYe

s11

554

2780

.973

2115

.6720

Duke

Univ

ersit

y Hos

pital

Durh

am, N

CNo

Yes

116

4720

49.31

226

17.0

017

Unive

rsity

of Te

xas–

Hous

tonHo

uston

, TX

NoYe

s12

347

2257

.713

2311

.33

6Em

ory U

niver

sity

Atlan

ta, G

AYe

sYe

s13

547

2561

.53

310

9.00

9Un

ivers

ity of

Minn

esot

aMi

nnea

polis

, MN

Yes

Yes

8543

2683

.45

311

9.00

9Un

ivers

ity of

Alab

ama

Birm

ingha

m, A

LNo

Yes

7338

2246

.762

2218

.50

18.5

Unive

rsity

of Ill

inois

at Ch

icago

Chica

go, I

LNo

Yes

5737

1340

.975

2717

.40

13Br

igham

& W

omen

’s Ho

spita

lBo

ston,

MA

NoNo

206

3420

43.0

62

2518

.00

18Ya

le Un

ivers

ityNe

w Ha

ven,

CTNo

Yes

169

328

40.3

23

8438

.33

20Th

omas

Jeffe

rson

Univ

ersit

yPh

ilade

lphia,

PA

Yes

Yes

8231

2652

.63

330

24.3

325

Roos

evelt

Hos

pital

(Mou

nt Si

nai)

New

York

, NY

NoYe

s10

731

1540

.00

436

18.5

017

New

York

Univ

ersit

yNe

w Yo

rk, N

YNo

Yes

109

305

49.0

02

1518

.00

18Un

ivers

ity of

Cali

forn

ia, S

an D

iego

San D

iego,

CANo

Yes

6128

1857

.65

419

9.00

7Un

ivers

ity of

Wisc

onsin

–Mad

ison

Mad

ison,

WI

Yes

Yes

5127

1734

.743

1511

.33

15In

diana

Univ

ersit

yIn

diana

polis

, IN

NoYe

s68

2712

41.51

315

12.3

314

Rush

Univ

ersit

yCh

icago

, IL

Yes

Yes

108

2613

36.19

321

16.3

320

Rutge

rs N

ew Je

rsey

Med

ical

Scho

olNe

wark

, NJ

Yes

Yes

7726

1932

.28

217

13.5

013

.5

Med

ical U

niver

sity o

f Sou

th Ca

rolin

aCh

arles

ton, S

CNo

Yes

5426

2036

.58

418

11.75

11

Unive

rsity

of Te

nnes

see/

Semm

es-

Mur

phey

Clin

icM

emph

is, T

NYe

sYe

s68

228

30.13

39

6.33

9

May

o Clin

ic Ja

ckso

nville

Jack

sonv

ille, F

LNo

Yes

4122

1522

.49

311

9.67

9Un

ivers

ity of

Flor

ida–G

aines

ville

Gaine

sville

, FL

Yes

Yes

3919

1324

.65

333

159

Ceda

rs-S

inai M

edica

l Cen

terLo

s Ang

eles,

CAYe

sYe

s63

1814

33.41

424

14.5

013

CONT

INUE

D ON

PAG

E 11

84 »


A. Sonig et al.


webofknowledge.com). This index was calculated using the author’s name, irrespective of his or her academic af-filiation.

Statistical AnalysisThe Shapiro-Wilk test was performed to determine

if each of the continuous variables in our study was nor-mally distributed. The continuous variables were h index-neuroendovascular, h10 index-neuroendovascular, number of faculty members, mean h index of faculty, median h index of faculty, and highest h index of faculty. Correla-tions between these variables were evaluated using the Spearman’s rank correlation coefficient. The Wilcoxon-Mann-Whitney test was used to compare the distributions of the continuous variables by group. The chi-square test or Fisher’s exact test was used to evaluate the relationship between categorical variables (i.e., affiliation with a com-prehensive stroke center, accreditation status, and higher productivity center). When the p value was < 0.05, the 2 variables under evaluation were considered to be signifi-cantly related. SAS (version 9.4, SAS Institute) was used for the statistical analysis.

ResultsRankings of Neuroendovascular Fellowships Based on the h Indices

The search revealed a total of 54 fellowship programs, of which details for 37 fellowships were available on the parent institute or department websites. These 37 pro-grams were included in our analysis (Table 1).

The median h index-neuroendovascular was 31 (mean 38.86 ± 22.04; range 12–103). The median of the h10 in-dex-neuroendovascular was 19 (mean 20.32 ± 10.92; range 4–48). The mean number of neuroendovascular faculty members in each department was 3.27 ± 0.962 (range 2–6). Stanford University, University of California, San Francisco (UCSF), University of California, Los Angeles (UCLA), University at Buffalo, and Johns Hopkins Uni-versity had the top 5 rankings based on h index-neuro-endovascular. The h index-neuroendovascular indices for these institutions were 103, 82, 77, 71, and 64, respectively.

Correlation Analysis of Neuroendovascular Fellowship h Indices With Faculty h Indices

Correlation analysis (Table 2) showed a strong positive correlation between the neuroendovascular fellowship h index and its h10 index (rs = 0.885; p < 0.0001). Similarly, the mean, median, and highest h indices of the neuroen-dovascular departmental faculty were positively corre-lated with the neuroendovascular fellowship h index (rs = 0.662, p < 0.0001; rs = 0.617, p < 0.0001; and rs = 0.649, p < 0.0001 respectively). There was a significant correlation between the number of faculty members and the neuroen-dovascular h index (rs = 0.408; p = 0.006) and h10 index (rs = 0.441; p = 0.003). The mean, median, and highest h indices of neuroendovascular departmental faculty were also correlated with the h10 index of the fellowship (rs = 0.568, p < 0.0001; rs = 0.569, p < 0.0001; and rs = 0.541, p < 0.0001, respectively).TA

BLE

1. Ne

uroe

ndov

ascu

lar f

ellow

ship

rank

ings

bas

ed o

n th

e neu

roen

dova

scul

ar fe

llows

hip

h in

dex

Institu

tion

Loca

tion

CAST

/AC

GME

Accr

edita

tion

Affili

ation

w/

Comp

rehe

nsive

St

roke

Cen

ter

(JC/D

NV)

Depa

rtmen

tal

h Ind

ex

Neur

oend

ovas

cular

Fe

llows

hip

h Ind

exh1

0 In

dex

e In

dex

No. o

f Ne

uroe

ndov

ascu

lar-

Train

ed

Facu

lty

High

est

h Ind

ex

of Fa

culty

Mea

n h

Inde

x of

Depa

rtmen

t Fa

culty

Med

ian

h Ind

ex

of Fa

culty

Unive

rsity

of M

iami

Miam

i, FL

Yes

No51

169

32.9

82

1211

.00

11Ha

rtfor

d Hos

pital

Hartf

ord,

CTNo

Yes

2416

1523

.123

75.

336

Alba

ny M

edica

l Cen

terAl

bany

, NY

NoNo

6015

923

.973

2517

.6723

Unive

rsity

of U

tah

Salt L

ake C

ity, U

TYe

sYe

s64

1510

19.18

48

3.25

2.5

The O

hio S

tate

Unive

rsity

Colum

bus,

OHYe

sYe

s73

147

14.97

314

11.0

011

Penn

Sta

te He

rshe

y Med

ical

Cente

rHe

rshe

y, PA

Yes

No57

124

24.91

39

4.33

3

» CON

TINU

ED F

ROM

PAGE

1183




Comparison of Neuroendovascular Fellowship h Indices With CAST/ACGME Accreditation and With Affiliation to a JC/DNV Stroke Center

Of the 37 fellowships, 18 (48.6%) were accredited by CAST or ACGME, and 30 (81.1%) were affiliated with a JC- or DNV-accredited stroke center. There was no sig-nificant association between the neuroendovascular fel-lowship h indices and CAST/ACGME accreditation or JC/DNV-accredited stroke center affiliation (Table 3).

On the basis of accreditation status, there was no sta-tistically significant difference in the mean or median of the neuroendovascular fellowship h index and h10 index (Table 3). Similarly, on the basis of an affiliation with a JC- or DNV-accredited comprehensive stroke center, there was no statistically significant difference in the mean or median values of those indices.

Comparison of Neuroendovascular Fellowships Based on Productivity

On the basis of the quartile analysis, the 75th percen-

tile of the neuroendovascular fellowship h index was 54. Departments with an h index-neuroendovascular of ≥ 54 were labeled as “high-productivity” centers. Ten (27.0%) centers formed this cohort: Stanford University, UCSF, UCLA, University at Buffalo, Johns Hopkins University, Washington University, St. Joseph’s Hospital (Barrow), University of Pittsburgh, NewYork-Presbyterian/Cornell University, and Cleveland Clinic. In addition, Stanford University had the highest e index (121.35) (Table 1).

The mean number of neuroendovascular faculty mem-bers in the department was significantly higher in neuroen-dovascular fellowship programs with higher productivity (p = 0.013) (Table 4). Moreover, higher productivity fellow-ship programs had significantly higher mean and median neuroendovascular faculty h indices (p ≤ 0.0001 and p ≤ 0.0001). Also, significantly higher productivity was seen in the last 10 years (p < 0.0001). However, higher productivity was not associated with the CAST/ACGME accreditation of the fellowship or its affiliation with a JC/DNV-accred-ited stroke center (p = 0.7140 and p = 0.295, respectively).

FIG. 1. Steps involved in the computation of the departmental h index, neuroendovascular fellowship h index, and e index.

TABLE 2. Correlation coefficients between neuroendovascular fellowship program–related bibliometric variables

VariableFellowship

h Indexh Index From Previous

10 Yrs (p value)No. of Faculty

Members (p value)Highest h Index of Faculty (p value)

Mean h Index of Faculty (p value)

Median h Index of Faculty (p value)

Neuroendovascular fellowship h index

1.00000 0.885 (<0.0001)* 0.408 (0.006)* 0.649 (<0.0001)* 0.662 (<0.0001)* 0.617 (<0.0001)*

Neuroendovascular h index from last 10 yrs

1.00000 0.441 (0.003)* 0.541 (<0.0001)* 0.568 (<0.0001)* 0.569 (<0.0001)*

No. of faculty 1.00000 0.409 (0.006)* 0.188 (0.132) 0.117 (0.246)Highest h index of faculty 1.00000 0.894 (<0.0001)* 0.685 (<0.0001)*Mean h index of faculty 1.000 0.910 (<0.0001)*Median h index of faculty 1.000

* Significant p value.


A. Sonig et al.


DiscussionTo the best of our knowledge, we have performed the

most detailed bibliometric evaluation to date on the neu-roendovascular subspecialty. Our study is unique for sev-eral reasons. First and foremost, department productivity was analyzed by specifically looking at the subspecialty of interest and using address and subject matter search strings (Appendix 1) to extract publication citations. By screening the citations for publications (Fig. 1) that com-prised the parent department’s h index, we created a new h index (i.e., h index-neuroendovascular) that was based on relevant neuroendovascular publications. This method-ology can be used to study and analyze most fellowship programs in neurosurgery or other specialties across the US. Ponce and Lozano10 compared the academic produc-tivity of American and Canadian neurosurgical training programs using departmental address search strings and concluded that h indices can be useful for comparing aca-demic output across neurosurgery departments. However, their study was limited to neurosurgery and did not in-clude any subspecialty.

Klimo et al.9 used the Scopus (https://www.scopus.com/) and Google Scholar (http://scholar.google.com/) bibliometric databases to profile the academic productiv-ity of individual pediatric neurosurgeons. Their study was mainly an individual authorship–based study in which the h index of a pediatric neurosurgeon was calculated according to his or her name. Those authors also evalu-ated pediatric department productivity for a period of 5 years. The study did not focus on publications pertaining to pediatric neurosurgery alone. For example, a pediatric neurosurgeon might have coauthored several manuscripts

on different topics of interest, but those citations would still be part of his or her h index. Each department was ranked based on the cumulative h and e indices of its cur-rent faculty members; thus, the method may not reflect the department’s productivity. Additionally, faculty members may change from one department to another, which is an-other aspect that was not into taken into account during the ranking of the program.

Accreditation and ProductivityThe accreditation of neurosurgical fellowships is a

“hot” topic in view of a recent increase in the number of programs that offer neuroendovascular or neurointerven-tional fellowships.4 An argument was made to halt fel-lowship training because of a decrease in the volume of procedures and the saturation of larger centers with neu-rointerventionists.4 It was feared that graduating fellows would accept appointments at community hospitals with lower procedure volumes and untrained support staff.

Unlike other specialties (such as obstetrics and gyne-cology, cardiology, vascular surgery, and surgical oncol-ogy) where almost all fellowships are accredited by the ACGME, fellowships in neurosurgery mostly are either unaccredited or accredited by the Accreditation Council for Pediatric Neurosurgical Fellowships (for pediatric neu-rosurgery fellowships) or CAST, which offers accreditation of fellowship programs in the subspecialties of neurocriti-cal care, cerebrovascular neurosurgery, neuroendovascular surgery, spinal neurosurgery, neurosurgical oncology, pe-diatric neurosurgery, peripheral nerve neurosurgery, and stereotactic and functional neurosurgery. Few fellowships in neurosurgery are ACGME accredited.

TABLE 3. Neuroendovascular fellowship and faculty h indices compared by accreditation and stroke center affiliation status and p values (Wilcoxon-Mann-Whitney test)

Affiliation Status No. of Neuroendovascular Fellowship Programs Mean SD Median Min Max p Value

CAST/ACGME Accreditation Status Fellowship h index No 19 42.53 23.64 34.00 15.00 103.00

0.2238 Yes 18 35.00 20.16 26.50 12.00 71.00 h index from last 10 yrs No 19 21.05 11.68 20.00 5.00 48.00

0.7726 Yes 18 19.56 10.92 18.00 4.00 44.00 No. of faculty No 19 3.42 1.12 3.00 2.00 6.00

0.3340 Yes 18 3.11 0.76 3.00 2.00 5.00JC/DNV Stroke Center Affiliation Status Fellowship h index No 7 40.57 28.27 34.00 12.00 82.00

0.824 Yes 30 38.47 20.89 31.00 14.00 103.00 h index from last 10 yrs No 7 19.85 13.55 20.00 4.00 40.00

0.904 Yes 30 20.43 10.82 18.50 5.00 48.00 No. of faculty No 7 3.00 1.41 3.00 2.00 6.00 0.417 Yes 30 3.33 0.84 3.00 2.00 5.00





In the present study on neurosurgical departments with a neuroendovascular fellowship program, we analyzed if academic productivity correlates with accreditation status (ACGME/CAST). Fourteen fellowships were accredited by CAST, 4 by ACGME, and 2 by both. There was no significant difference in the academic productivity of un-accredited versus accredited programs (p = 0.70). In fact, only 5 of 10 programs in the higher productivity cohort are accredited (Table 1). At present, the accreditation system for endovascular training is fragmented, in part because it is a field populated by 3 different specialties: neurosurgery, neurology, and neuroradiology. Further-more, there is no mandate for academic productivity by any of the participating societies, which stands in contrast to the standard for residency training programs that have research criteria that need to be met prior to the gradua-tion of their residents. To meet the needs of this rapidly evolving field, including the evaluation of indications, techniques, and devices, in addition to expanding hori-zons to new paradigms, research should be embraced as a foundational element of accredited programs. This would serve both to advance and justify the field, as well as to further raise the bar for those who wish to enter it. Our study is a step in that direction. The methods can be used by accreditation bodies to extract the research productiv-ity of endovascular fellowship programs and monitor pre-established goals.

Affiliation With a Stroke CenterThe year 2015 saw the publication of 5 randomized

controlled trials that favored mechanical thrombectomy plus intravenous tissue plasminogen activator over medi-cal management with intravenous tissue plasminogen ac-tivator alone for large-vessel occlusion in the setting of anterior circulation stroke.1,2,5–7 These trials endorsed the superiority of endovascular treatment over medical man-agement alone. As a result, endorsement by the DNV or JC

for comprehensive stroke center status became important for an institution. A recent study has shown that the direct admission of patients to a comprehensive stroke center (as opposed to the transfer of patients from another facility to such a center) is associated with better outcomes and lower hospitalization costs.11

The present study showed that departments affiliated with a JC- or DNV-certified stroke center did not have a significantly higher number of neuroendovascular faculty members (p = 0.417). The small number of nonaffiliated centers (7 of 37 centers) may explain the lack of signifi-cance. Moreover, there is a requirement for a minimum of 2 trained neurointerventionists to staff a certified com-prehensive stroke center, which was met by all affiliated and nonaffiliated centers (Table 1). However, association with a JC- or DNV-certified stroke center did not increase the academic productivity of the department. Even though certification as a primary stroke center started in the year 2003 (https://www.jointcommission.org/certification_for_comprehensive_stroke_centers/), comprehensive stroke center certification is a recent phenomenon. The DNV an-nounced its first comprehensive stroke center in the year 2012 (http://dnvglhealthcare.com/releases/dnv-healthcare-introduces-comprehensive-stroke-center-certification). Because the h index depends on the number of times a paper is cited, it will take some time before the impact of faculty volume is reflected in academic productivity at these comprehensive stroke centers.

h Index-Neuroendovascular and its h10 and e IndicesIn our study, we were able to extract publications spe-

cific to the subspecialty of interest and compute the h10 index. The h index-neuroendovascular correlated sig-nificantly with the number of neuroendovascular faculty members (p = 0.04), the highest h index of the faculty (p < 0.0001), and its mean (p < 0.0001).

TABLE 4. Neuroendovascular fellowship and faculty h indices compared by productivity group (Wilcoxon-Mann-Whitney test)

High Productivity No. of Neuroendovascular Fellowship Programs Mean SD Median Min Max p Value

h index from previous 10 yrs No 27 15.00 6.34 15.00 4.00 26.00

<0.0001* Yes 10 34.70 8.33 34.50 23.00 48.00No. of faculty No 27 3.04 0.76 3.00 2.00 5.00

0.013* Yes 10 3.90 1.20 4.00 2.00 6.00Highest h index of faculty No 27 20.96 14.89 18.00 7.00 84.00

<0.0001* Yes 10 45.20 17.15 44.00 21.00 74.00Mean h index of faculty No 27 13.76 7.03 12.33 3.25 38.33

<0.0001* Yes 10 27.93 10.13 28.05 13.80 40.25Median h index of faculty No 27 12.87 5.86 13.00 2.50 25.00

<0.0001* Yes 10 26.20 11.15 23.00 11.00 40.00

* Significant p value.


A. Sonig et al.


If one looks at the subgroup analysis of high-produc-tivity centers (Table 4), these centers had a significantly higher h10 index and number of neuroendovascular facul-ty members. The highest and mean faculty h indices were significantly greater at these centers, suggesting more re-sources for research are provided at these centers or that these centers attract faculty members with better research profiles. Further, larger programs have more faculty mem-bers and can spread call and case responsibilities. This might facilitate more “protected time” for research and academic endeavors.

The h10 index depicts academic productivity regardless of the longevity of the program, and thus it negates the natural advantage of older fellowship programs. Our study showed that h10 correlated significantly with h index-neuro endovascular (p < 0.0001) and the mean, median, and highest h index of neuroendovascular faculty mem-bers (Table 2).

The h index bibliometric has been criticized for its in-ability to truly reflect the citation prowess of an individual or an institution. For example, 2 institutions can have the same h index, yet one of these may have manuscripts that have been cited much more often than its h index indicates. These unaccounted for or excess citations can be calcu-lated in the e index. Thus, if 2 institutions have the same h index and one of them has a higher e index, it would mean that the institution with the higher e index had more citations.

The e index needs to be interpreted along with the h in-dex; hence, we did not perform a separate statistical analy-sis of the e index. In our study, Stanford University had the highest neuroendovascular h and e indices, and UCLA had the highest h10 index.

Study LimitationsThis is the first attempt to rank fellowship programs on

the basis of the h indices that specifically pertained to spe-cialty of interest: in this instance, the neuroendovascular subspecialty. Such an analysis is prone to errors, as the de-nominator was based on the address strings of the parent institution or department. It is possible that a completely different abbreviated address could be used by the depart-ment for publication purposes. It is also possible that the Thomson Reuters Web of Science portal may not harbor all of the articles published by the department.

The h index considers overall citations rather than the importance of single contributions, neglects the quality of the content of the publications, and does not consider the context of the citation. It is also influenced by the accuracy of the citation database used for its calculation.

Future PerspectivesThis study included neuroendovascular fellowship pro-

grams in the continental US. The pattern of research pro-ductivity at centers across the globe is different because each country has its own process for accreditation and af-filiation. Moreover, information for fellowships outside the scope of our study is lacking at present. Once more infor-mation is available on websites, the methods enumerated in this manuscript can be used to analyze neuroendovas-cular fellowship programs across the globe.

In addition, the methods applied here can be used as a benchmark for future academic productivity by deter-mining if a correlation exists between the individual pro-ductivity of the graduates of the fellowship program and that of the parent department. The data can be extrapo-lated into various subsets of publication types: clinically oriented (patient outcomes and technology advancement) and basic science oriented (bench research and preclinical in vivo and in vitro work).

ConclusionsOn the basis of the institution’s address and subject

matter strings, a departmental h index can be calculated. Further analysis of the publications that make up the h in-dex could reveal the inclination of a department toward one particular subspecialty and can be used to accurately calculate the h index of a particular specialty. Our analysis was restricted to the neuroendovascular subspecialty, but the methodology can be extended to other neurosurgical subspecialties. The research interests of individual faculty members are directly reflected in the research productivity of a department. The current systems for the accreditation of neuroendovascular fellowship programs do not have a meaningful impact on academic productivity. Publication prowess based on the above-mentioned methods could be given consideration during the process of the accreditation of neuroendovascular fellowships in the future.

AcknowledgmentsThe authors thank Tingting Zhuang, MA, from the University

at Buffalo Department of Biostatistics for statistical analysis, Adri-enne R. Doepp, BA, MLS, from the A. H. Aaron Health Sciences Library at Buffalo General Medical Center for assistance with the search strategy, and Debra J. Zimmer from the University at Buf-falo Department of Neurosurgery for editorial assistance.

References 1. Berkhemer OA, Fransen PS, Beumer D, van den Berg LA,

Lingsma HF, Yoo AJ, et al: A randomized trial of intraar-terial treatment for acute ischemic stroke. N Engl J Med 372:11–20, 2015

2. Campbell BC, Mitchell PJ, Kleinig TJ, Dewey HM, Churilov L, Yassi N, et al: Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 372:1009–1018, 2015

3. Egghe L: Theory and practice of the g-index. Scientometrics 69:131–152, 2006

4. Fiorella D, Hirsch JA, Woo HH, Rasmussen PA, Shazam Hussain M, Hui FK, et al: Should neurointerventional fellow-ship training be suspended indefinitely? J Neurointerv Surg 4:315–318, 2012

5. Goyal M, Demchuk AM, Menon BK, Eesa M, Rempel JL, Thornton J, et al: Randomized assessment of rapid endovas-cular treatment of ischemic stroke. N Engl J Med 372:1019–1030, 2015

6. Hirsch JE: An index to quantify an individual’s scientific re-search output. Proc Natl Acad Sci U S A 102:16569–16572, 2005

7. Jovin TG, Chamorro A, Cobo E, de Miquel MA, Molina CA, Rovira A, et al: Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med 372:2296–2306, 2015

8. Khan NR, Thompson CJ, Taylor DR, Venable GT, Wham




RM, Michael LM II, et al: An analysis of publication produc-tivity for 1225 academic neurosurgeons and 99 departments in the United States. J Neurosurg 120:746–755, 2014

9. Klimo P Jr, Venable GT, Khan NR, Taylor DR, Shepherd BA, Thompson CJ, et al: Bibliometric evaluation of pediat-ric neurosurgery in North America. J Neurosurg Pediatr 14:695–703, 2014

10. Ponce FA, Lozano AM: Academic impact and rankings of American and Canadian neurosurgical departments as as-sessed using the h index. J Neurosurg 113:447–457, 2010

11. Sonig A, Lin N, Krishna C, Natarajan SK, Mokin M, Hop-kins LN, et al: Impact of transfer status on hospitalization cost and discharge disposition for acute ischemic stroke across the US. J Neurosurg 124:1228–1237, 2016

12. Spearman CM, Quigley MJ, Quigley MR, Wilberger JE: Sur-vey of the h index for all of academic neurosurgery: another power-law phenomenon? J Neurosurg 113:929–933, 2010

13. Zhang CT: The e-index, complementing the h-index for ex-cess citations. PLoS One 4:e5429, 2009

DisclosuresThe authors report the following. Dr. Siddiqui has financial inter-ests in Buffalo Technology Partners Inc., Cardinal Health, Inter-national Medical Distribution Partners, Medina Medical Systems, Neuro Technology Investors, StimSox, and Valor Medical. He serves as a consultant to Amnis Therapeutics Ltd., Cerebrotech Medical Systems Inc., CereVasc LLC, Codman & Shurtleff Inc., Corindus Inc., Covidien (acquired by Medtronic), GuidePoint Global Consulting, Lazarus (acquired by Medtronic), Medina

Medical (acquired by Medtronic), Medtronic, MicroVention, Neuravi, Penumbra, Pulsar Vascular, Rapid Medical, Rebound Medical, Reverse Medical (acquired by Medtronic), Silk Road Medical Inc., Stryker, The Stroke Project Inc., Three Rivers Med-ical Inc., Cerebrotech Medical Systems Inc., and W. L. Gore & Associates. He is a principal investigator or serves on the National Steering Committee for the following trials: Covidien SWIFT PRIME, LARGE, Medtronic SWIFT DIRECT, MicroVention CONFIDENCE trial, MicroVention FRED trial, Penumbra 3D Separator, Penumbra COMPASS, Penumbra INVEST, and POSI-TIVE Trial. He is a member of the board of the Intersocietal Accreditation Committee.

Author ContributionsConception and design: Sonig. Acquisition of data: Siddiqui, Sonig, Shallwani. Analysis and interpretation of data: all authors. Drafting the article: Sonig. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors.

Supplemental Information Online-Only ContentSupplemental material is available with the online version of the article.

Appendices 1 and 2. https://thejns.org/doi/suppl/10.3171/2016. 9.JNS161857.

CorrespondenceAdnan H. Siddqui, Department of Neurosurgery, University at Buffalo, 100 High St., Ste. B4, Buffalo, NY 14203. email: [email protected].




Documents

Academic impact and rankings of neuroendovascular ... · Vascular Research Center, University at Buffalo, State University of New York; 4Department of Neurosurgery, Gates Vascular