Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Analysis of Peer Review data from WoSpart 1
Vladimir Batagelj and Anuska Ferligoj
University of Ljubljana, IMFM Ljubljana and IAM UP Koper
PEERESplit – June 16-18, 2015
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Outline
1 Data2 Distributions3 Citations
Vladimir Batagelj:[email protected]
Anuska Ferligoj:[email protected]
Journal of Irreproducible Results, 32(1988)2, 24
Current version of slides (November 16, 2015 / 20 : 08):http://vlado.fmf.uni-lj.si/pub/slides/peere.pdf
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Goals
The goal of this presentation is to study the publications on ’peerreview’ included in Web of Science till May 2015. The questions tobe answered are:
• which publications and which authors are the most cited;
• which are the main journals publishing papers on ’peer review’;
• which are the most influential publications in the field of ’peerreview’;
• which are the groups of researchers that collaborate the most,what are their topics.
For answering these questions several social network analysisapproaches are applied on large citation and collaboration networksobtained from WoS. The most useful ones are the ’main path’analysis and the ’islands’ procedure.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Collecting the data
To the Web of Science (WoS) we put the query "peer review*".We got 17053 hits, and additional 2867 hits for the queryrefereeing (from Web of Science Core Collection, in May and June2015).
The first analysis revealed many papers without WoS descriptionshaving large indegrees in the citation network. We manually searchedfor each of them (with indegree larger or equal to 20) and if found weadded it into the data set.
After some iterations, we finally constructed the data set used in thisanalysis. Using the program WoS2Pajek we transformed it into acollection of networks: cite, two-mode networks: WA, WJ, WK;partitions DC (DC= a work w has (1) / has not (0) a WoSdescription), year (publication year); and CSV file titles with basicdata about works with DC=1.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Record from Web of Science
PT JAU Dipple, H
Evans, BTI The Leicestershire Huntington’s disease support group: a social network
analysisSO HEALTH & SOCIAL CARE IN THE COMMUNITYLA EnglishDT ArticleC1 Rehabil Serv, Troon Way Business Ctr, Leicester LE4 9HA, Leics, England.RP Dipple, H, Rehabil Serv, Troon Way Business Ctr, Sandringham
Suite,Humberstone Lane, Leicester LE4 9HA, Leics, England.CR BORGATTI SP, 1992, UCINET 4 VERSION 1 0
FOLSTEIN S, 1989, HUNTINGTONS DIS DISOSCOTT J, 1991, SOCIAL NETWORK ANAL
NR 3TC 3PU BLACKWELL SCIENCE LTDPI OXFORDPA P O BOX 88, OSNEY MEAD, OXFORD OX2 0NE, OXON, ENGLANDSN 0966-0410J9 HEALTH SOC CARE COMMUNITYJI Health Soc. Care CommunityPD JULPY 1998VL 6IS 4BP 286EP 289PG 4SC Public, Environmental & Occupational Health; Social WorkGA 105UPUT ISI:000075092200008ER
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Citation network structure
1
2
3
4
5
6
7
8
9
10
11 12
13
14
15
16
Pajek
1
2
3
4
5
6
7
8
9
10
11 12
13
14
15
16
1718 19 20
21
Pajek
yellow – hits for the query, DC = 1red – manually added, DC = 1green, blue – referenced only, DC = 0
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Transforming WoS data into networks
Identification problem: different names for the same unit (synonyms); samename for different units (homonyms). → loops and multiple links
The usual ISI name of a work (field CR), for example
LEFKOVITCH LP, 1985, THEOR APPL GENET, V70, P585
has the following structure (all its elements are in upper case):
AU + ’, ’ + PY + ’, ’ + SO[:20] + ’, V’ + VL + ’, P’ + BP
In WoS the same work can have different ISI names:
GRANOVET.MS, 1973, AM J SOCIOL, V78, P1360GRANOVETTER M, 1983, SOCIOLOGICAL THEORY, V1, P203BORGATTI SP, 2002, UGINET WINDOWS SOFTWBORGATTI S, 1999, UCINET V USERS GUIDECANTANZARO M, 2005, PHYS REV E, V71, UNSP 027103CANTAZARO M, 2005, PHYS REV E, V71, UNSP 056104CATANZARO M, 2005, PHYS REV E 2, V71, ARTN 056104
To improve the precission the program WoS2Pajek supports also shortnames. They have the format:
LastNm[:8] + ’ ’ + FirstNm[0] + ’(’ + PY + ’)’ + VL + ’:’ + BP
For example: LEFKOVIT L(1985)70:585
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
WoS2Pajek report.
>>> End of processing of WoS file
number of works = 639407
number of authors = 268258
number of journals = 36646
number of keywords = 33688
number of records = 20131
number of duplicates = 256
works + titles : titles.csv
works index file: vtxIndex.txt
*** FILES:
year of publication partition: C:/Users/batagelj/work/Python/WoS/peere1\Year.clu
described / cited only partition: C:/Users/batagelj/work/Python/WoS/peere1\DC.clu
number of pages vector: C:/Users/batagelj/work/Python/WoS/peere1\NP.vec
citation network: C:/Users/batagelj/work/Python/WoS/peere1\Cite.net
works X journals network: C:/Users/batagelj/work/Python/WoS/peere1\WJ.net
works X keywords network: C:/Users/batagelj/work/Python/WoS/peere1\WK.net
works X authors network: C:/Users/batagelj/work/Python/WoS/peere1\WA.net
finished: Thu Jun 11 04:09:09 2015
We removed multiple links and loops from networks. The cleaned citationnetwork has n = 639407 nodes and m = 764234 arcs.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Degree distributions in citation network
●
●
●
●
●
●
●
●
●●
●●
●●●
●●
●●●●●●●●●
●
●
●
●
●●●
●●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●●●
●●●
●●●
●●
●●●●
●
●●●●●●●● ●●
1 2 5 10 20 50 100
13
720
66input degree distribution
deg
freq ●
●● ● ● ● ● ●
●●
●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●
●●
●●●●●●●●●●
●●●●●●●●●●●
●●●
●
●●
●
●●
●
●
●●
●●●●●●●●●
●●●●●
●
●
●
●●●●●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●●●
●
●●
●●
●●●
●●●●●●
●●
●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●
●
●
●●●●
●
●
●●
●
●●
●●●
●●●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●●●●●●
●
●
●
●
●
●●●●●●●●
●
●●●●
●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●●
1 5 10 50 100 500
13
823
7629
921
27
output degree distribution
deg
freq
There are 15107 nodes with indeg = 0, 621998 nodes with outdeg = 0,
and 2466 nodes with DC = 1 and outdeg = 0.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most cited worksCiteAll indeg best
name indeg title journal
PETERS_D(1982)5:187 156 PEER-REVIEW PRACTICES OF PSYCHOLOGICAL JOURNALS - THE FATE OF ACCEPTED, PUBLISHED ARTICLES, SUBMITTED AGAINBEHAV BRAIN SCI
COHEN_J(1988): 149 Statistical Power Analysis for the Behavioral Sciences book
STROUP_D(2000)283:2008 132 Meta-analysis of observational studies in epidemiology - A proposal for reporting JAMA-J AM MED ASSOC
ZUCKERMA_H(1971)9:66 125 PATTERNS OF EVALUATION IN SCIENCE - INSTITUTIONALISATION, STRUCTURE AND FUNCTIONS OF REFEREE SYSTEM MINERVA
DERSIMON_R(1986)7:177 122 METAANALYSIS IN CLINICAL-TRIALS CONTROL CLIN TRIALS
EGGER_M(1997)315:629 119 Bias in meta-analysis detected by a simple, graphical test BRIT MED J
CICCHETT_D(1991)14:119 116 THE RELIABILITY OF PEER-REVIEW FOR MANUSCRIPT AND GRANT SUBMISSIONS - A CROSS-DISCIPLINARY INVESTIGATION BEHAV BRAIN SCI
EASTERBR_P(1991)337:867 111 PUBLICATION BIAS IN CLINICAL RESEARCH LANCET
MAHONEY_M(1977)1:161 105 Publication prejudices: An experimental study of confirmatory bias in the peer review system Cognitive Therapy and Research
HORROBIN_D(1990)263:1438 105 THE PHILOSOPHICAL BASIS OF PEER-REVIEW AND THE SUPPRESSION OF INNOVATION JAMA-J AM MED ASSOC
HIRSCH_J(2005)102:16569 105 An index to quantify an individual's scientific research output P NATL ACAD SCI USA
VANROOYE_S(1999)318:23 104 Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial BRIT MED J
MCNUTT_R(1990)263:1371 99 THE EFFECTS OF BLINDING ON THE QUALITY OF PEER-REVIEW - A RANDOMIZED TRIAL JAMA-J AM MED ASSOC
GODLEE_F(1998)280:237 98 Effect on the quality of peer review of blinding reviewers and asking them to sign their reports - A randomized controlled trialJAMA-J AM MED ASSOC
HIGGINS_J(2003)327:557 97 Measuring inconsistency in meta-analyses BRIT MED J
COLE_S(1981)214:881 97 CHANCE AND CONSENSUS IN PEER-REVIEW SCIENCE
JADAD_A(1996)17:1 97 Assessing the quality of reports of randomized clinical trials: Is blinding necessary? CONTROL CLIN TRIALS
LOCK_S(1985): 96 A Difficult Balance: Editorial Peer Review in Medicine book
LANDIS_J(1977)33:159 96 MEASUREMENT OF OBSERVER AGREEMENT FOR CATEGORICAL DATA BIOMETRICS
MOHER_D(1999)354:1896 95 Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement LANCET
JUSTICE_A(1998)280:240 94 Does masking author identity improve peer review quality? - A randomized controlled trial JAMA-J AM MED ASSOC
VANROOYE_S(1998)280:234 88 Effect of blinding and unmasking on the quality of peer review - A randomized trial JAMA-J AM MED ASSOC
BLACK_N(1998)280:231 85 What makes a good reviewer and a good review for a general medical journal? JAMA-J AM MED ASSOC
SCHERER_R(1994)272:158 85 FULL PUBLICATION OF RESULTS INITIALLY PRESENTED IN ABSTRACTS - A METAANALYSIS JAMA-J AM MED ASSOC
HIGGINS_J(2011): 84 Cochrane Handbook for Systematic Reviews of Interventions book
GOODMAN_S(1994)121:11 84 MANUSCRIPT QUALITY BEFORE AND AFTER PEER-REVIEW AND EDITING AT ANNALS OF INTERNAL-MEDICINE ANN INTERN MED
CHUBIN_D(1990): 83 Peerless Science: Peer Review and U. S. Science Policy book
MOHER_D(2009)6:1000097 83 Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement PLOS MED
KRONICK_D(1990)263:1321 78 PEER-REVIEW IN 18TH-CENTURY SCIENTIFIC JOURNALISM JAMA-J AM MED ASSOC
JEFFERSO_T(2002)287:2784 75 Effects of editorial peer review - A systematic review JAMA-J AM MED ASSOC
SEGLEN_P(1997)314:498 74 Why the impact factor of journals should not be used for evaluating research BRIT MED J
ROTHWELL_P(2000)123:1964 74 Reproducibility of peer review in clinical neuroscience - Is agreement between reviewers any greater than would be expected by chance alone?BRAIN
BEGG_C(1996)276:637 73 Improving the quality of reporting of randomized controlled trials - The CONSORT statement JAMA-J AM MED ASSOC
BEGG_C(1994)50:1088 72 OPERATING CHARACTERISTICS OF A BANK CORRELATION TEST FOR PUBLICATION BIAS BIOMETRICS
WENNERAS_C(1997)387:341 72 Nepotism and sexism in peer-review NATURE
KASSIRER_J(1994)272:96 71 PEER-REVIEW - CRUDE AND UNDERSTUDIED, BUT INDISPENSABLE JAMA-J AM MED ASSOC
SMITH_R(2006)99:178 71 Peer review: a flawed process at the heart of science and journals J ROY SOC MED
HELSEN_W(2004)22:179 70 Physical and perceptual-cognitive demands of top-class refereeing in association football J SPORT SCI
DOWNS_S(1998)52:377 68 The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventionsJ EPIDEMIOL COMMUN H
HEDGES_L(1985): 67 Statistical Methods for Meta-Analysis book
BURNHAM_J(1990)263:1323 67 THE EVOLUTION OF EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC
DICKERSI_K(1992)267:374 67 FACTORS INFLUENCING PUBLICATION OF RESEARCH RESULTS - FOLLOW-UP OF APPLICATIONS SUBMITTED TO 2 INSTITUTIONAL REVIEW BOARDSJAMA-J AM MED ASSOC
DICKERSI_K(1990)263:1385 66 THE EXISTENCE OF PUBLICATION BIAS AND RISK-FACTORS FOR ITS OCCURRENCE JAMA-J AM MED ASSOC
HIGGINS_J(2002)21:1539 66 Quantifying heterogeneity in a meta-analysis STAT MED
ALTMAN_D(2001)134:663 65 The revised CONSORT statement for reporting randomized trials: Explanation and elaboration ANN INTERN MED
MERTON_R(1968)159:56 64 MATTHEW EFFECT IN SCIENCE SCIENCE
NEVILL_A(2002)3:261 63 The influence of crowd noise and experience upon refereeing decisions in football PSYCHOL SPORT EXERC
KRUSTRUP_P(2001)19:881 63 Physiological demands of top-class soccer refereeing in relation to physical capacity: effect of intense intermittent exercise trainingJ SPORT SCI
SCHULZ_K(1995)273:408 60 EMPIRICAL-EVIDENCE OF BIAS - DIMENSIONS OF METHODOLOGICAL QUALITY ASSOCIATED WITH ESTIMATES OF TREATMENT EFFECTS IN CONTROLLED TRIALSJAMA-J AM MED ASSOC
MERTON_R(1973): 59 The Sociology of Science book
GOLD_M(1996): 59 Cost-Effectiveness in Health and Medicine book
SACKETT_D(1996)312:71 58 Evidence based medicine: What it is and what it isn't - It's about integrating individual clinical expertise and the best external evidenceBRIT MED J
BOYER_E(1990): 57 Scholarship Reconsidered: Priorities of the Professoriate book
HIGGINS_J(2008): 57 Cochrane Handbook for Systematic Reviews of Interventions book
WEBER_E(1998)280:257 57 Unpublished research from a medical specialty meeting - Why investigators fail to publish JAMA-J AM MED ASSOC
EVANS_A(1993)8:422 55 THE CHARACTERISTICS OF PEER REVIEWERS WHO PRODUCE GOOD-QUALITY REVIEWS J GEN INTERN MED
JEFFERSO_T(2002)287:2786 55 Measuring the quality of editorial peer review JAMA-J AM MED ASSOC
GODLEE_F(2002)287:2762 55 Making reviewers visible - Openness, accountability, and credit JAMA-J AM MED ASSOC
BAILAR_J(1985)312:654 55 JOURNAL PEER-REVIEW - THE NEED FOR A RESEARCH AGENDA NEW ENGL J MED
SPRAGUE_S(2003)85A:158 55 Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings J BONE JOINT SURG AM
WELLER_A(2001): 54 Editorial Peer Review: Its Strengths and Weaknesses book
LIBERATI_A(2009)6:1000100 54 The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and ElaborationPLOS MED
MOHER_D(2009)151:264 54 Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement ANN INTERN MED
SCHROTER_S(2004)328:673 54 Effects of training on quality of peer review: randomised controlled trial BRIT MED J
HARRIS_R(2001)20:21 53 Current methods of the US Preventive Services Task Force - A review of the process AM J PREV MED
FISHER_M(1994)272:143 52 THE EFFECTS OF BLINDING ON ACCEPTANCE OF RESEARCH PAPERS BY PEER-REVIEW JAMA-J AM MED ASSOC
VANROOYE_S(1999)52:625 52 Development of the Review Quality Instrument (RQI) for assessing peer reviews of manuscripts J CLIN EPIDEMIOL
DANIEL_H(1993): 51 Guardians of Science - Fairness and Reliability of Peer Review book
ROSENTHA_R(1979)86:638 51 THE FILE DRAWER PROBLEM AND TOLERANCE FOR NULL RESULTS PSYCHOL BULL
CAMPANAR_J(1998)19:181 51 Peer review for journals as it stands today - Part 1 SCI COMMUN
NYLENNA_M(1994)272:149 51 MULTIPLE BLINDED REVIEWS OF THE 2 MANUSCRIPTS - EFFECTS OF REFEREE CHARACTERISTICS AND PUBLICATION LANGUAGEJAMA-J AM MED ASSOC
COHEN_J(1992)112:155 51 A POWER PRIMER PSYCHOL BULL
INGELFIN_F(1974)56:686 50 PEER REVIEW IN BIOMEDICAL PUBLICATION AM J MED
Works with names [ANONYMO(2011):, . . . , WORLD H(2009):, . . . ,
U S(2011):, . . . , WHO(2008):, . . . , were removed from the network.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most cited works / ordered by pub-yearCiteAll indeg best
name author title journal year indeg
KUHN_T(1962): Kuhn, T The Structure of Scientific Revolutions book 1962 49
GLASER_B(1967): Glaser, BG The Discovery of Grounded Theory: Strategies for Qualitative Research book 1967 49
CRANE_D(1967)2:195 CRANE, D GATEKEEPERS OF SCIENCE - SOME FACTORS AFFECTING SELECTION OF ARTICLES FOR SCIENTIFIC JOURNALS AM SOCIOL 1967 49
MERTON_R(1968)159:56 MERTON, RK MATTHEW EFFECT IN SCIENCE SCIENCE 1968 64
ZUCKERMA_H(1971)9:66 ZUCKERMA.H PATTERNS OF EVALUATION IN SCIENCE - INSTITUTIONALISATION, STRUCTURE AND FUNCTIONS OF REFEREE SYSTEM MINERVA 1971 125
GARFIELD_E(1972)178:471 GARFIELD, E CITATION ANALYSIS AS A TOOL IN JOURNAL EVALUATION - JOURNALS CAN BE RANKED BY FREQUENCY AND IMPACT OF CITATIONS FOR SCIENCE POLICY STUDIESSCIENCE 1972 47
MERTON_R(1973): Merton, RK The Sociology of Science book 1973 59
INGELFIN_F(1974)56:686 INGELFIN.FJ PEER REVIEW IN BIOMEDICAL PUBLICATION AM J MED 1974 50
SCOTT_W(1974)29:698 SCOTT, WA INTER-REFEREE AGREEMENT ON SOME CHARACTERISTICS OF MANUSCRIPTS SUBMITTED TO JOURNAL OF PERSONALITY AND SOCIAL-PSYCHOLOGY AM PSYCHOL 1974 43
MAHONEY_M(1977)1:161 Mahoney, MJ Publication prejudices: An experimental study of confirmatory bias in the peer review system Cognitive Therapy and Research1977 105
LANDIS_J(1977)33:159 LANDIS, JR MEASUREMENT OF OBSERVER AGREEMENT FOR CATEGORICAL DATA BIOMETRICS 1977 96
WILSON_J(1978)61:1697 WILSON, JD 70TH ANNUAL-MEETING OF AMERICAN-SOCIETY-FOR-CLINICAL-INVESTIGATION, SAN-FRANCISCO, CALIFORNIA, 30 APRIL 1978 - PRESIDENTIAL-ADDRESS J CLIN INVEST 1978 45
COLE_S(1978): COLE, S Peer Review in the National Science Foundation book 1978 44
ROSENTHA_R(1979)86:638 ROSENTHAL, R THE FILE DRAWER PROBLEM AND TOLERANCE FOR NULL RESULTS PSYCHOL BULL 1979 51
GOLDMAN_L(1980)303:255 GOLDMAN, L FATE OF CARDIOLOGY RESEARCH ORIGINALLY PUBLISHED IN ABSTRACT FORM NEW ENGL J MED 1980 47
COLE_S(1981)214:881 COLE, S CHANCE AND CONSENSUS IN PEER-REVIEW SCIENCE 1981 97
PETERS_D(1982)5:187 PETERS, DP PEER-REVIEW PRACTICES OF PSYCHOLOGICAL JOURNALS - THE FATE OF ACCEPTED, PUBLISHED ARTICLES, SUBMITTED AGAIN BEHAV BRAIN SCI 1982 156
LOCK_S(1985): Lock, S A Difficult Balance: Editorial Peer Review in Medicine book 1985 96
HEDGES_L(1985): Hedges, LV Statistical Methods for Meta-Analysis book 1985 67
BAILAR_J(1985)312:654 BAILAR, JC JOURNAL PEER-REVIEW - THE NEED FOR A RESEARCH AGENDA NEW ENGL J MED 1985 55
DERSIMON_R(1986)7:177 DERSIMONIAN, R METAANALYSIS IN CLINICAL-TRIALS CONTROL CLIN TRIALS 1986 122
DICKERSI_K(1987)8:343 DICKERSIN, K PUBLICATION BIAS AND CLINICAL-TRIALS CONTROL CLIN TRIALS 1987 45
COHEN_J(1988): Cohen, J Statistical Power Analysis for the Behavioral Sciences book 1988 149
HORROBIN_D(1990)263:1438 HORROBIN, DF THE PHILOSOPHICAL BASIS OF PEER-REVIEW AND THE SUPPRESSION OF INNOVATION JAMA-J AM MED ASSOC 1990 105
MCNUTT_R(1990)263:1371 MCNUTT, RA THE EFFECTS OF BLINDING ON THE QUALITY OF PEER-REVIEW - A RANDOMIZED TRIAL JAMA-J AM MED ASSOC 1990 99
CHUBIN_D(1990): Chubin, DE Peerless Science: Peer Review and U. S. Science Policy book 1990 83
KRONICK_D(1990)263:1321 KRONICK, DA PEER-REVIEW IN 18TH-CENTURY SCIENTIFIC JOURNALISM JAMA-J AM MED ASSOC 1990 78
BURNHAM_J(1990)263:1323 BURNHAM, JC THE EVOLUTION OF EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC 1990 67
DICKERSI_K(1990)263:1385 DICKERSIN, K THE EXISTENCE OF PUBLICATION BIAS AND RISK-FACTORS FOR ITS OCCURRENCE JAMA-J AM MED ASSOC 1990 66
BOYER_E(1990): Boyer, EL Scholarship Reconsidered: Priorities of the Professoriate book 1990 57
CICCHETT_D(1991)14:119 CICCHETTI, DV THE RELIABILITY OF PEER-REVIEW FOR MANUSCRIPT AND GRANT SUBMISSIONS - A CROSS-DISCIPLINARY INVESTIGATION BEHAV BRAIN SCI 1991 116
EASTERBR_P(1991)337:867 EASTERBROOK, PJ PUBLICATION BIAS IN CLINICAL RESEARCH LANCET 1991 111
BRENNAN_T(1991)324:370 BRENNAN, TA INCIDENCE OF ADVERSE EVENTS AND NEGLIGENCE IN HOSPITALIZED-PATIENTS - RESULTS OF THE HARVARD MEDICAL-PRACTICE STUDY-I NEW ENGL J MED 1991 46
BLANK_R(1991)81:1041 BLANK, RM THE EFFECTS OF DOUBLE-BLIND VERSUS SINGLE-BLIND REVIEWING - EXPERIMENTAL-EVIDENCE FROM THE AMERICAN-ECONOMIC-REVIEW AM ECON REV 1991 43
DICKERSI_K(1992)267:374 DICKERSIN, K FACTORS INFLUENCING PUBLICATION OF RESEARCH RESULTS - FOLLOW-UP OF APPLICATIONS SUBMITTED TO 2 INSTITUTIONAL REVIEW BOARDS JAMA-J AM MED ASSOC 1992 67
COHEN_J(1992)112:155 COHEN, J A POWER PRIMER PSYCHOL BULL 1992 51
GOLDMAN_R(1992)267:958 GOLDMAN, RL THE RELIABILITY OF PEER ASSESSMENTS OF QUALITY OF CARE JAMA-J AM MED ASSOC 1992 49
EVANS_A(1993)8:422 EVANS, AT THE CHARACTERISTICS OF PEER REVIEWERS WHO PRODUCE GOOD-QUALITY REVIEWS J GEN INTERN MED 1993 55
DANIEL_H(1993): Daniel, HD Guardians of Science - Fairness and Reliability of Peer Review book 1993 51
SCHERER_R(1994)272:158 SCHERER, RW FULL PUBLICATION OF RESULTS INITIALLY PRESENTED IN ABSTRACTS - A METAANALYSIS JAMA-J AM MED ASSOC 1994 85
GOODMAN_S(1994)121:11 GOODMAN, SN MANUSCRIPT QUALITY BEFORE AND AFTER PEER-REVIEW AND EDITING AT ANNALS OF INTERNAL-MEDICINE ANN INTERN MED 1994 84
BEGG_C(1994)50:1088 BEGG, CB OPERATING CHARACTERISTICS OF A BANK CORRELATION TEST FOR PUBLICATION BIAS BIOMETRICS 1994 72
KASSIRER_J(1994)272:96 KASSIRER, JP PEER-REVIEW - CRUDE AND UNDERSTUDIED, BUT INDISPENSABLE JAMA-J AM MED ASSOC 1994 71
FISHER_M(1994)272:143 FISHER, M THE EFFECTS OF BLINDING ON ACCEPTANCE OF RESEARCH PAPERS BY PEER-REVIEW JAMA-J AM MED ASSOC 1994 52
NYLENNA_M(1994)272:149 NYLENNA, M MULTIPLE BLINDED REVIEWS OF THE 2 MANUSCRIPTS - EFFECTS OF REFEREE CHARACTERISTICS AND PUBLICATION LANGUAGE JAMA-J AM MED ASSOC 1994 51
MILES_M(1994): Miles, MB Qualitative Data Analysis book 1994 47
SCHULZ_K(1995)273:408 SCHULZ, KF EMPIRICAL-EVIDENCE OF BIAS - DIMENSIONS OF METHODOLOGICAL QUALITY ASSOCIATED WITH ESTIMATES OF TREATMENT EFFECTS IN CONTROLLED TRIALS JAMA-J AM MED ASSOC 1995 60
JADAD_A(1996)17:1 Jadad, AR Assessing the quality of reports of randomized clinical trials: Is blinding necessary? CONTROL CLIN TRIALS 1996 97
BEGG_C(1996)276:637 Begg, C Improving the quality of reporting of randomized controlled trials - The CONSORT statement JAMA-J AM MED ASSOC 1996 73
GOLD_M(1996): Gold, MR Cost-Effectiveness in Health and Medicine book 1996 59
SACKETT_D(1996)312:71 Sackett, DL Evidence based medicine: What it is and what it isn't - It's about integrating individual clinical expertise and the best external evidence BRIT MED J 1996 58
EGGER_M(1997)315:629 Egger, M Bias in meta-analysis detected by a simple, graphical test BRIT MED J 1997 119
SEGLEN_P(1997)314:498 Seglen, PO Why the impact factor of journals should not be used for evaluating research BRIT MED J 1997 74
WENNERAS_C(1997)387:341 Wenneras, C Nepotism and sexism in peer-review NATURE 1997 72
ARMSTRON_J(1997)3:63 Armstrong, JS Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation Science and Engineering Ethics1997 43
GODLEE_F(1998)280:237 Godlee, F Effect on the quality of peer review of blinding reviewers and asking them to sign their reports - A randomized controlled trial JAMA-J AM MED ASSOC 1998 98
JUSTICE_A(1998)280:240 Justice, AC Does masking author identity improve peer review quality? - A randomized controlled trial JAMA-J AM MED ASSOC 1998 94
VANROOYE_S(1998)280:234 van Rooyen, S Effect of blinding and unmasking on the quality of peer review - A randomized trial JAMA-J AM MED ASSOC 1998 88
BLACK_N(1998)280:231 Black, N What makes a good reviewer and a good review for a general medical journal? JAMA-J AM MED ASSOC 1998 85
DOWNS_S(1998)52:377 Downs, SH The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventionsJ EPIDEMIOL COMMUN H 1998 68
WEBER_E(1998)280:257 Weber, EJ Unpublished research from a medical specialty meeting - Why investigators fail to publish JAMA-J AM MED ASSOC 1998 57
CAMPANAR_J(1998)19:181 Campanario, JM Peer review for journals as it stands today - Part 1 SCI COMMUN 1998 51
CALLAHAM_M(1998)280:229 Callaham, ML Reliability of editors' subjective quality ratings of peer reviews of manuscripts JAMA-J AM MED ASSOC 1998 45
VANROOYE_S(1999)318:23 van Rooyen, S Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial BRIT MED J 1999 104
MOHER_D(1999)354:1896 Moher, D Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement LANCET 1999 95
VANROOYE_S(1999)52:625 van Rooyen, S Development of the Review Quality Instrument (RQI) for assessing peer reviews of manuscripts J CLIN EPIDEMIOL 1999 52
JUNI_P(1999)282:1054 Juni, P The hazards of scoring the quality of clinical trials for meta-analysis JAMA-J AM MED ASSOC 1999 46
SMITH_R(1999)318:4 Smith, R Opening up BMJ peer review - A beginning that should lead to complete transparency BRIT MED J 1999 44
STROUP_D(2000)283:2008 Stroup, DF Meta-analysis of observational studies in epidemiology - A proposal for reporting JAMA-J AM MED ASSOC 2000 132
ROTHWELL_P(2000)123:1964 Rothwell, PM Reproducibility of peer review in clinical neuroscience - Is agreement between reviewers any greater than would be expected by chance alone? BRAIN 2000 74
WALSH_E(2000)176:47 Walsh, E Open peer review: a randomised controlled trial BRIT J PSYCHIAT 2000 46
OUDEJANS_R(2000)404:33 Oudejans, RRD Errors in judging 'offside' in football - Optical trickery can undermine the assistant referee's view of this ruling. NATURE 2000 44
ALTMAN_D(2001)134:663 Altman, DG The revised CONSORT statement for reporting randomized trials: Explanation and elaboration ANN INTERN MED 2001 65
KRUSTRUP_P(2001)19:881 Krustrup, P Physiological demands of top-class soccer refereeing in relation to physical capacity: effect of intense intermittent exercise training J SPORT SCI 2001 63
WELLER_A(2001): Weller, AC Editorial Peer Review: Its Strengths and Weaknesses book 2001 54
HARRIS_R(2001)20:21 Harris, RP Current methods of the US Preventive Services Task Force - A review of the process AM J PREV MED 2001 53
MOHER_D(2001)357:1191 Moher, D The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials LANCET 2001 45
LIPSEY_M(2001): Lipsey, MW Practical Meta-Analysis (Applied Social Research Methods) book 2001 44
MOHER_D(2001)285:1987 Moher, D The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials JAMA-J AM MED ASSOC 2001 43
JEFFERSO_T(2002)287:2784 Jefferson, T Effects of editorial peer review - A systematic review JAMA-J AM MED ASSOC 2002 75
HIGGINS_J(2002)21:1539 Higgins, JPT Quantifying heterogeneity in a meta-analysis STAT MED 2002 66
NEVILL_A(2002)3:261 Nevill, AM The influence of crowd noise and experience upon refereeing decisions in football PSYCHOL SPORT EXERC 2002 63
JEFFERSO_T(2002)287:2786 Jefferson, T Measuring the quality of editorial peer review JAMA-J AM MED ASSOC 2002 55
GODLEE_F(2002)287:2762 Godlee, F Making reviewers visible - Openness, accountability, and credit JAMA-J AM MED ASSOC 2002 55
HIGGINS_J(2003)327:557 Higgins, JPT Measuring inconsistency in meta-analyses BRIT MED J 2003 97
SPRAGUE_S(2003)85A:158 Sprague, S Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings J BONE JOINT SURG AM 2003 55
BEKELMAN_J(2003)289:454 Bekelman, JE Scope and impact of financial conflicts of interest in biomedical research - A systematic review JAMA-J AM MED ASSOC 2003 45
HELSEN_W(2004)22:179 Helsen, W Physical and perceptual-cognitive demands of top-class refereeing in association football J SPORT SCI 2004 70
SCHROTER_S(2004)328:673 Schroter, S Effects of training on quality of peer review: randomised controlled trial BRIT MED J 2004 54
CHAN_A(2004)291:2457 Chan, AW Empirical evidence for selective reporting of outcomes in randomized trials - Comparison of Protocols to published articles JAMA-J AM MED ASSOC 2004 48
HIRSCH_J(2005)102:16569 Hirsch, JE An index to quantify an individual's scientific research output P NATL ACAD SCI USA 2005 105
IOANNIDI_J(2005)2:696 Ioannidis, JPA Why most published research findings are false PLOS MED 2005 43
SMITH_R(2006)99:178 Smith, R Peer review: a flawed process at the heart of science and journals J ROY SOC MED 2006 71
GARFIELD_E(2006)295:90 Garfield, E The history and meaning of the journal impact factor JAMA-J AM MED ASSOC 2006 43
BENOS_D(2007)31:145 Benos, DJ The ups and downs of peer review ADV PHYSIOL EDUC 2007 44
SCHERER_R(2007):pub3 SCHERER, RW Full publication of results initially presented in abstracts Cochrane Database of Systematic Reviews2007 43
HIGGINS_J(2008): Higgins, JPT Cochrane Handbook for Systematic Reviews of Interventions book 2008 57
MOHER_D(2009)6:1000097 Moher, D Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement PLOS MED 2009 83
LIBERATI_A(2009)6:1000100 Liberati, A The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration PLOS MED 2009 54
MOHER_D(2009)151:264 Moher, D Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement ANN INTERN MED 2009 54
HIGGINS_J(2011): Higgins, JPT Cochrane Handbook for Systematic Reviews of Interventions book 2011 84
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most cited books
1962 (49) 1967 (49) 1973 (59) 1978 (44)
1985 (96) 1985 (67) 1988 (149) 1990 (83)
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most cited books
1990 (57) 1993 (51) 1994 (47) 1996 (59)
2001 (44) 2001 (54) 2008 (57) 2011 (84)
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most citing worksCiteAll outdeg best
rank node outdeg work title
1 290441 2306 GOLDSTEI_R(2010)53:4343 Heat transfer-A review of 2004 literature
2 175921 2127 GOLDSTEI_R(2010)53:4397 Heat transfer-A review of 2005 literature
3 5541 1259 HILLIS_L(2011)58:E123 2011 ACCF/AHA Guideline for Coronary Artery Bypass Graft Surgery
4 6068 872 LEVINE_G(2011)58:E44 2011 ACCF/AHA/SCAI Guideline for Percutaneous Coronary Intervention
5 144032 837 DELEON_G(2007):1 Trematode parasites (Platyhelminthes) of wildlife vertebrates in Mexico
6 1585 747 ADAMS_H(2007)38:1655 Guidelines for the early management of adults with ischemic stroke - A guideline from the American Heart Association
7 4415 729 GOLDSTEI_L(2011)42:517 Guidelines for the Primary Prevention of Stroke A Guideline for Healthcare Professionals From the American Heart Association
8 525591 711 WILLHITE_C(2014)44:1 Systematic review of potential health risks posed by pharmaceutical, occupational and consumer exposures to metallic and nanoscale aluminum, …
9 132542 710 SMITH_J(2012)17:510 Single-Case Experimental Designs: A Systematic Review of Published Research and Current Standards
10 211490 671 AKSULU_A(2010)11:576 A Comprehensive Review and Synthesis of Open Source Research
11 492189 651 MILLER_C(1990)62:700 Groundwater - A review of the 1989 literature
12 3943 586 GOLDSTEI_L(2006)37:1583 Primary prevention of ischemic stroke - A guideline from the American Heart Association
13 313264 577 FLEGAL_A(2013)43:1869 All the Lead in China
14 15310 569 GOLDSTEI_L(2006)113:E873 Primary prevention of ischemic stroke - A guideline from the American heart Association
15 199315 538 EVANS_E(2004)19:775 Atomic spectrometry update. Advances in atomic emission, absorption and fluorescence spectrometry and related techniques
16 4714 533 ROACH_E(2008)39:2644 Management of stroke in infants and children - A scientific statement from a special writing group of the American Heart Association
17 23743 517 SANTEN_R(2010)95:S7 Postmenopausal Hormone Therapy: An Endocrine Society Scientific Statement
18 6565 505 GORELICK_P(2011)42:2672 Vascular Contributions to Cognitive Impairment and Dementia A Statement for Healthcare Professionals From the American Heart Association
19 285306 493 HILLIS_L(2011)58:2584 2011 ACCF/AHA Guideline for Coronary Artery Bypass Graft Surgery: Executive Summary A Report of the American College of Cardiology Foundation
20 115699 492 HILLIS_L(2012)143:4 2011 ACCF/AHA guideline for coronary artery bypass graft surgery: Executive summary
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The most cited authorsWciA = Cite * WA weighted indegree
i f Id i f Id i f Id i f Id---------------------------------------------------------------------------------1 976 ALTMAN_D 26 285 FLETCHER_S 51 171 THOMPSON_S 76 133 ZUCKERMA_H2 973 MOHER_D 27 251 GUYATT_G 52 168 EASTWOOD_S 77 132 BECKER_B3 756 RENNIE_D 28 250 OXMAN_A 53 167 CICCHETT_D 78 132 SIPE_T4 634 SMITH_R 29 241 D’OTTAVI_S 54 167 PETERS_D 79 132 MORTON_S5 584 GODLEE_F 30 238 CHO_M 55 166 JADAD_A 80 129 WINKER_M6 523 BLACK_N 31 228 MULROW_C 56 165 MATTHEWS_D 81 128 WAECKERL_J7 514 BERLIN_J 32 222 CHALMERS_T 57 155 EVANS_A 82 124 SCHNEIDE_M8 421 EVANS_S 33 221 MERTON_R 58 154 WEBER_E 83 121 CHALMERS_I9 417 BORNMANN_L 34 219 WEARS_R 59 154 MCNUTT_R 84 120 FLANAGIN_A
10 386 CASTAGNA_C 35 215 DEVEREAU_P 60 149 JUSTICE_A 85 119 SMITH_G11 381 DANIEL_H 36 208 GARFIELD_E 61 146 VANRAAN_A 86 119 MINDER_C12 376 VANROOYE_S 37 201 WESTON_M 62 146 KRUSTRUP_P 87 119 SQUIRES_B13 368 SCHULZ_K 38 193 BERO_L 63 145 COLE_J 88 117 COHEN_J14 360 CALLAHAM_M 39 192 JEFFERSO_T 64 145 DERSIMON_R 89 116 BHANDARI_M15 351 DICKERSI_K 40 191 ABT_G 65 145 BEGG_C 90 113 GOODMAN_S16 348 HELSEN_W 41 190 WAGER_E 66 143 NEVILL_A 91 111 GOPALAN_R17 344 LIBERATI_A 42 190 HIGGINS_J 67 142 LUNDBERG_G 92 111 DAVIS_D18 333 COOK_D 43 189 LAIRD_N 68 140 SCHROTER_S 93 111 EASTERBR_P19 333 EGGER_M 44 183 MARSH_H 69 139 HAYNES_R 94 109 SMITH_H20 311 GOTZSCHE_P 45 180 HORROBIN_D 70 138 COLE_S 95 107 SCHERER_R21 307 TETZLAFF_J 46 178 IOANNIDI_J 71 136 THACKER_S 96 106 PLESSNER_H22 301 STROUP_D 47 178 GILIS_B 72 136 WILLIAMS_G 97 106 JAYASING_U23 300 OLKIN_I 48 178 CECI_S 73 136 CAMPANAR_J 98 105 HIRSCH_J24 298 DAVIDOFF_F 49 175 BANGSBO_J 74 133 ZUCKERMA_ 99 105 WILSON_J25 285 FLETCHER_R 50 172 MARTYN_C 75 133 HORTON_R 100 104 CATTEEUW_P---------------------------------------------------------------------------------
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Distribution of papers from WoS (DC=1) by year
●●●●●●●●●●● ●●● ●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●
●●
●●
●●●●
●●
●●
●
●
●
●
●
●
●
●
1950 1960 1970 1980 1990 2000 2010
050
010
0015
0020
00
Papers per year
year
num
ber
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Publication year distribution
It can be fitted with lognormal distribution:
> setwd("C:\\Users\\batagelj\\work\\Python\\WoS\\peere1")> years <- read.table(file="Year1.clu",header=FALSE,skip=2,
colClasses="character")$V1> y <- as.numeric(years)> t <- table(y[!is.na(y)])> year <- as.integer(names(t))> freq <- as.vector(t[1950<=year & year<=2015])> y <- 1950:2015> plot(y,freq,pch=16,cex=0.8)> model <- nls(freq~c*dlnorm(2016-y,a,b),start=list(c=350000,a=2,b=0.7))> modelNonlinear regression model
model: freq ~ c * dlnorm(2016 - y, a, b)data: parent.frame()
c a b6.415e+05 2.631e+00 6.762e-01residual sum-of-squares: 45256778
Number of iterations to convergence: 7Achieved convergence tolerance: 8.714e-06> lines(y,predict(model,list(x=2016-y)),col=’red’,lw=2)
∑freq = 625557
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Publication year distribution
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
1950 1960 1970 1980 1990 2000 2010
050
0015
000
2500
035
000
y
freq
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Authors with the largest number of worksWAD indeg
Rank Deg Id Rank Deg Id-------------------------- -------------------------
1 288 [ANONYMO_ 23 27 SQUIRES_B2 98 BORNMANN_L 24 27 WANG_J3 62 SMITH_R 25 27 COHEN_J4 61 ALTMAN_D 26 25 WILLIAMS_A5 59 DANIEL_H 27 24 MAYER_A6 55 MOHER_D 28 24 MARSHALL_E7 51 LEE_J 29 23 LEE_M8 47 RENNIE_D 30 23 BROWN_C9 43 SMITH_J 31 23 SMITH_M
10 40 REYES_H 32 22 ANDERSON_P11 37 BROWN_D 33 22 ZHANG_Y12 37 CURTIS_K 34 22 JONES_A13 35 CASTAGNA_C 35 22 ADAMS_J14 32 GARFIELD_E 36 22 CHENG_J15 32 LEE_S 37 21 ZHANG_L16 31 THOENNES_M 38 21 BJORK_B17 30 JOHNSON_J 39 21 KIM_Y18 29 WESTON_M 40 21 KIM_J19 29 WILLIAMS_J 41 21 CICCHETT_D20 28 CALLAHAM_M 42 21 KIM_S21 28 JOHNSON_C 43 21 MARSH_H22 28 HELSEN_W 44 20 SAPER_C
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Authors with the largest contribution to the fieldWeighted indegree of normalized WAD
Rank Value Id Rank Value Id------------------------------ ------------------------------
1 280.0000 [ANONYMO_ 23 9.8333 HARNAD_S2 30.7310 BORNMANN_L 24 9.8333 ROY_R3 17.6667 MARSHALL_E 25 9.7000 CURTIS_K4 17.6569 SMITH_R 26 9.5583 CALLAHAM_M5 17.2262 DANIEL_H 27 9.5333 ROUKIS_T6 16.0000 GARFIELD_E 28 9.1286 JONES_R7 15.3543 RENNIE_D 29 9.0484 MOHER_D8 15.2101 ALTMAN_D 30 9.0184 ANDERSON_M9 14.9762 SQUIRES_B 31 9.0000 KOSTOFF_R
10 14.5333 CHENG_J 32 8.9167 LIESEGAN_T11 14.0179 REYES_H 33 8.8323 JOHNSON_J12 13.6286 THOENNES_M 34 8.6429 FONTANAR_P13 12.7796 SMITH_J 35 8.5000 CICCHETT_D14 12.5167 COHEN_J 36 8.3792 WILLIAMS_A15 11.7466 LEE_J 37 8.3274 CASTAGNA_C16 10.9091 BROWN_C 38 8.2500 DONOVAN_S17 10.8333 WELLER_A 39 8.1667 CHUBIN_D18 10.8314 BROWN_D 40 8.0354 JONES_A19 10.7102 JOHNSON_C 41 8.0000 REINDOLL_W20 10.5833 BJORK_B 42 7.5180 WILLIAMS_J21 10.5000 MERVIS_J 43 7.4787 MARTIN_B22 10.0000 BEREZIN_A 44 7.3333 WALSH_J
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Main journals in which PR papers were publishedWJD indeg
Rank Freq Id Rank Freq Id-------------------------------- --------------------------------
1 286 JAMA-J AM MED ASSOC 26 57 PEDIATRICS2 286 BMJ OPEN 27 55 CHEM ENG NEWS3 175 NATURE 28 53 J NANOSCI NANOTECHNO4 174 BRIT MED J 29 50 J SEX MED5 165 SCIENCE 30 50 J GEN INTERN MED6 163 SCIENTOMETRICS 31 49 J SPORT SCI7 133 PLOS ONE 32 48 J SCHOLARLY PUBL8 96 ACAD MED 33 48 MED EDUC9 96 LANCET 34 47 *****
10 92 SCIENTIST 35 46 RES EVALUAT11 88 LEARN PUBL 36 46 AM J PREV MED12 80 PHYS TODAY 37 45 ENVIRON HEALTH PERSP13 75 J ASSOC OFF AGR CHEM 38 44 J AM DENT ASSOC14 73 J UROLOGY 39 44 BRIT J SPORT MED15 72 CAN MED ASSOC J 40 43 CLIN THER16 70 ARCH PATHOL LAB MED 41 42 ANN EMERG MED17 67 ABSTR PAP AM CHEM S 42 42 AM J ROENTGENOL18 66 ANN PHARMACOTHER 43 42 NUCLEIC ACIDS RES19 66 ANN INTERN MED 44 41 BMC PUBLIC HEALTH20 65 J AM COLL RADIOL 45 41 CLIN ORTHOP RELAT R21 62 CUTIS 46 41 CURR MED RES OPIN22 62 NEW ENGL J MED 47 41 J MED INTERNET RES23 59 BEHAV BRAIN SCI 48 40 PLAST RECONSTR SURG24 58 ANN ALLERG ASTHMA IM 49 39 J PROSTHET DENT25 57 MED J AUSTRALIA 50 37 BMC HEALTH SERV RES
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Main keywordsWKD indeg
Rank Freq Id Rank Freq Id--------------------------- --------------------------------
1 6635 review 26 1003 treatment2 3700 peer 27 996 assessment3 2279 research 28 992 risk4 1978 quality 29 967 outcome5 1748 health 30 878 model6 1610 journal 31 876 performance7 1575 systematic 32 832 program8 1456 referee 33 829 education9 1449 peer-review 34 818 evaluation
10 1399 management 35 810 intervention11 1344 care 36 808 scientific12 1274 publication 37 802 cancer13 1262 trial 38 799 practice14 1222 analysis 39 787 control15 1220 study 40 772 child16 1188 science 41 725 process17 1171 use 42 707 effect18 1163 impact 43 690 report19 1142 therapy 44 688 medicine20 1125 patient 45 679 guideline21 1092 clinical 46 658 factor22 1054 randomize 47 627 controlled-trial23 1043 literature 48 619 disorder24 1032 disease 49 607 surgery25 1009 medical 50 601 development
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Citation networks
To solve the boundary problem we manually added some “missing”works and removed nodes with DC = 0 and indeg < 3 – obtainig thenetwork CiteB.
The bounded citation network (21513 nodes) has one largecomponent (8927 nodes), 66 small components, and 11783 isolatednodes (not cited and citing only unimportant works).
The SPC method used for its analysis requires that the citationnetwork is acyclic. Determining strong components in CiteB we seethat our network is not acyclic. We transform it into an acyclicnetwork, CiteAcy, using the Preprint transformation. It hasn = 21533 nodes and m = 24031 arcs.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Weak components in the bounded citation network
Cluster Freq Freq% CumFreq% Representative---------------------------------------------------------
0 11783 54.7715 54.7715 BABILONI_F(2000)8:1861 8927 41.4958 96.2674 CHALMERS_I(1989):2 83 0.3858 96.6532 AUDI_G(2003)729:33 81 0.3765 97.0297 MCKENDRY_P(2002)83:374 62 0.2882 97.3179 SUMMERS_R(1994)15:7025 39 0.1813 97.4992 AWLIYA_W(1998)56:96 39 0.1813 97.6805 FRENCH_S(1997)97:10087 31 0.1441 97.8246 MCMAHON_C(2004)1:588 24 0.1116 97.9361 BLANCO_I(2001)95:1099 22 0.1023 98.0384 ALFARANO_C(2005)33:D418
10 20 0.0930 98.1314 MOURITZE_U(2003)62:33211 19 0.0883 98.2197 DEMUNCK_J(2003)82:13612 18 0.0837 98.3034 ABRAMS_P(1988)114:514 17 0.0790 98.4660 WARD_J(1995)23:22615 16 0.0744 98.5404 DELBEKE_D(1997)38:119616 14 0.0651 98.6055 BRUNEKRE_B(1989)140:136318 13 0.0604 98.7310 YOVINE_A(2004)22:89020 12 0.0558 98.8472 WHITEHEA_M(1984)81:48722 11 0.0511 98.9541 GRAHAM_J(1995):27 10 0.0465 99.2051 BARRIOS_L(2004)41:7228 9 0.0418 99.2470 HAMOSH_A(2005)33:D51432 7 0.0325 99.3911 MCCANN_J(1989)13:17934 6 0.0279 99.4515 JEMAL_A(2009)59:22538 5 0.0232 99.5584 HAN_Y(2005)116:150646 4 0.0186 99.7397 LOBBEZOO_F(2006)33:29350 3 0.0139 99.8094 BRADLEY_J(1994):58 2 0.0093 99.9163 YAGER_R(1988)18:18367 2 0.0093 100.0000 REYES_H(2004)132:7
---------------------------------------------------------Sum 21513 100.0000
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – short cycles
TURNER_R{1998}352:854
TURNER_R{1998}352:837
DICKERSI_K{1990}263:1385
CHALMERS_I{1990}263:1401
MOHER_D{2001}285:1987
LIBERATI_A{2009}6:1000100
BLACK_N{1998}280:231
VANROOYE_S{1998}280:234ALTMAN_D{2001}134:663
CHALMERS_I{1990}263:1405
ELLISON_G{2002}110:947
JEFFERSO_T{2002}287:2786
JEFFERSO_T{2002}287:2784
EISENHAR_M{2002}32:241
MOHER_D{2009}6:1000097
LAROCHEL_M{2002}32:181
ROTH_W{2002}32:215
OSSENBLO_T{2012}21:280
ENGELS_T{2012}93:373
ELLISON_G{2002}110:994
Pajek
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Search path count method
The search path count (SPC)method is based on countersn(u, v) that count the numberof different paths from s to tthrough the arc (u, v).
arXiv, Wiley book
The Main path starts in a linkwith the largest SPC weight andexpands in both directions fol-lowing the adjacent link withthe largest SPC weight.
The CPM path is determinedusing the Critical Path Methodfrom Operations Research.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Citation networks – SPC weights analysis
In the network we compute the SPC weights and on their basisdetermine the Main path, the CPM path and link islands [20 200].Islands labels:
12(1):199 11(2):89 4(3):83 1:81
6:62 10(4):46 9:39 2(5):39
3:31 7:24 5(6):22 8:20
picture (extract): size
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Main Path and CPM path
MCNUTT_R{1990}263:1371
JEFFERSO_T{1995}4:383
DRUMMOND_M{1996}313:275
CICCHETT_D{1980}35:300
GOTTFRED_S{1978}33:920
PETERS_D{1982}5:187
POLANYI_M{1958}:
BAILAR_J{1985}312:654
RENNIE_D{1986}256:2391
ROBIN_E{1987}91:252
JEFFERSO_T{1998}280:275
RENNIE_D{1994}272:91
WALSH_E{2000}176:47
JEFFERSO_T{2002}287:2784
BAYER_A{1966}39:381
COLE_S{1967}32:377
SMITH_R{1994}309:143
MERTON_R{1957}22:635
RENNIE_D{1993}270:2856
GARFIELD_E{1972}178:471
SMITH_R{1999}318:4
BORNMANN_L{2007}1:204
BORNMANN_L{2006}68:427
BORNMANN_L{2005}14:15
GOLDBECK_S{1999}318:44
RENNIE_D{1998}280:214
DANIEL_H{2005}18:143
CRANE_D{1965}30:699
MELTZER_B{1949}55:25
STORER_N{1966}:
RENNIE_D{1992}13:443
RENNIE_D{2002}287:2759
BORNMANN_L{2010}32:5
BORNMANN_L{2011}174:857
OPTHOF_T{2002}56:339
BORNMANN_L{2009}81:407
BORNMANN_L{2008}2:217
BORNMANN_L{2013}7:286
WALTMAN_L{2014}65:433
CARTTER_A{1966}:
DENNIS_W{1954}79:180
=JEFFERSO_T{2002}287:2786
Pajek
MCNUTT_R{1990}263:1371
JEFFERSO_T{1995}4:383
DRUMMOND_M{1996}313:275
CICCHETT_D{1980}35:300
INGELFIN_F{1974}56:686
ZUCKERMA_H{1971}9:66
PETERS_D{1982}5:187
POLANYI_M{1958}:
BAILAR_J{1985}312:654
CRANE_D{1967}2:195
RENNIE_D{1986}256:2391
ROBIN_E{1987}91:252
JEFFERSO_T{1998}280:275
RENNIE_D{1994}272:91
WALSH_E{2000}176:47
JEFFERSO_T{2002}287:2784
BAYER_A{1966}39:381
COLE_S{1967}32:377
SMITH_R{1994}309:143
MERTON_R{1957}22:635
RENNIE_D{1993}270:2856
SMITH_R{1999}318:4
BORNMANN_L{2007}1:204
BORNMANN_L{2009}22:117
BORNMANN_L{2008}59:1841BORNMANN_L{2006}68:427
BORNMANN_L{2005}14:15
BORNMANN_L{2011}45:199
GOLDBECK_S{1999}318:44
RENNIE_D{1998}280:214
DANIEL_H{2005}18:143
CRANE_D{1965}30:699
MELTZER_B{1949}55:25
STORER_N{1966}:
RENNIE_D{1992}13:443
RENNIE_D{2002}287:2759
BORNMANN_L{2011}174:857
OPTHOF_T{2002}56:339
BORNMANN_L{2013}7:286
WALTMAN_L{2014}65:433
CARTTER_A{1966}:
DENNIS_W{1954}79:180
=JEFFERSO_T{2002}287:2786
Pajek
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Main path and CPM path list of titles
name author title journal year
MELTZER_B(1949)55:25 Meltzer, BN The productivity of social scientists AM J SOCIOL 1949
DENNIS_W(1954)79:180 Dennis, W Bibliographies of eminent scientists Scient. Monthly 1954
MERTON_R(1957)22:635 MERTON, RK PRIORITIES IN SCIENTIFIC DISCOVERY - A CHAPTER IN THE SOCIOLOGY OF SCIENCE AM SOCIOL REV 1957
POLANYI_M(1958): Polanyi, M Personal Knowledge: Towards a Post-Critical Philosophy book 1958
CRANE_D(1965)30:699 Crane, D Scientists at major and minor universities AM SOCIOL REV 1965
BAYER_A(1966)39:381 Bayer, AE Some Correlates of a Citation Measure of Productivity in Science Sociology of Education 1966
STORER_N(1966): Storer, N The Social System of Science book 1966
CARTTER_A(1966): Carter, AM An assessment of quality in graduate education AM Council on Education 1966
COLE_S(1967)32:377 COLE, S SCIENTIFIC OUTPUT AND RECOGNITION - STUDY IN OPERATION OF REWARD SYSTEM IN SCIENCE AM SOCIOL REV 1967
GARFIELD_E(1972)178:471 GARFIELD, E CITATION ANALYSIS AS A TOOL IN JOURNAL EVALUATION - JOURNALS CAN BE RANKED BY FREQUENCY AND IMPACT OF CITATIONS FOR SCIENCE POLICY STUDIESSCIENCE 1972
GOTTFRED_S(1978)33:920 GOTTFREDSON, SD EVALUATING PSYCHOLOGICAL-RESEARCH REPORTS - DIMENSIONS, RELIABILITY, AND CORRELATES OF QUALITY JUDGMENTS AM PSYCHOL 1978
CICCHETT_D(1980)35:300 CICCHETTI, DV RELIABILITY OF REVIEWS FOR THE AMERICAN-PSYCHOLOGIST - A BIOSTATISTICAL ASSESSMENT OF THE DATA AM PSYCHOL 1980
PETERS_D(1982)5:187 PETERS, DP PEER-REVIEW PRACTICES OF PSYCHOLOGICAL JOURNALS - THE FATE OF ACCEPTED, PUBLISHED ARTICLES, SUBMITTED AGAIN BEHAV BRAIN SCI 1982
BAILAR_J(1985)312:654 BAILAR, JC JOURNAL PEER-REVIEW - THE NEED FOR A RESEARCH AGENDA NEW ENGL J MED 1985
RENNIE_D(1986)256:2391 RENNIE, D GUARDING THE GUARDIANS - A CONFERENCE ON EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC 1986
ROBIN_E(1987)91:252 ROBIN, ED PEER-REVIEW IN MEDICAL JOURNALS CHEST 1987
MCNUTT_R(1990)263:1371 MCNUTT, RA THE EFFECTS OF BLINDING ON THE QUALITY OF PEER-REVIEW - A RANDOMIZED TRIAL JAMA-J AM MED ASSOC 1990
RENNIE_D(1992)13:443 RENNIE, D SUSPENDED JUDGMENT - EDITORIAL PEER-REVIEW - LET US PUT IT ON TRIAL CONTROL CLIN TRIALS 1992
RENNIE_D(1993)270:2856 RENNIE, D MORE PEERING INTO EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC 1993
RENNIE_D(1994)272:91 RENNIE, D THE 2ND INTERNATIONAL-CONGRESS ON PEER-REVIEW IN BIOMEDICAL PUBLICATION JAMA-J AM MED ASSOC 1994
SMITH_R(1994)309:143 SMITH, R PROMOTING RESEARCH INTO PEER-REVIEW BRIT MED J 1994
JEFFERSO_T(1995)4:383 JEFFERSON, T ARE GUIDELINES FOR PEER-REVIEWING ECONOMIC EVALUATIONS NECESSARY - A SURVEY OF CURRENT EDITORIAL PRACTICE HEALTH ECON 1995
DRUMMOND_M(1996)313:275 Drummond, MF Guidelines for authors and peer reviewers of economic submissions to the BMJ BRIT MED J 1996
JEFFERSO_T(1998)280:275 Jefferson, T Evaluating the BMJ guidelines for economic submissions - Prospective audit of economic submissions to BMJ and The Lancet JAMA-J AM MED ASSOC 1998
RENNIE_D(1998)280:214 Rennie, D Peer review in Prague JAMA-J AM MED ASSOC 1998
SMITH_R(1999)318:4 Smith, R Opening up BMJ peer review - A beginning that should lead to complete transparency BRIT MED J 1999
GOLDBECK_S(1999)318:44 Goldbeck-Wood, S Evidence on peer review - scientific quality control or smokescreen? BRIT MED J 1999
WALSH_E(2000)176:47 Walsh, E Open peer review: a randomised controlled trial BRIT J PSYCHIAT 2000
JEFFERSO_T(2002)287:2784 Jefferson, T Effects of editorial peer review - A systematic review JAMA-J AM MED ASSOC 2002
RENNIE_D(2002)287:2759 Rennie, D Fourth International Congress on Peer Review in Biomedical Publication JAMA-J AM MED ASSOC 2002
OPTHOF_T(2002)56:339 Opthof, T The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical biasCARDIOVASC RES 2002
JEFFERSO_T(2002)287:2786 Jefferson, T Measuring the quality of editorial peer review JAMA-J AM MED ASSOC 2002
BORNMANN_L(2005)14:15 Bornmann, L Committee peer review at an international research foundation: predictive validity and fairness of selection decisions on post-graduate fellowship applicationsRES EVALUAT 2005
DANIEL_H(2005)18:143 Daniel, HD Publications as a measure of scientific advancement and of scientists' productivity LEARN PUBL 2005
BORNMANN_L(2006)68:427 Bornmann, L Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicantsSCIENTOMETRICS 2006
BORNMANN_L(2007)1:204 Bornmann, L Convergent validation of peer review decisions using the h index - Extent of and reasons for type I and type II errors J INFORMETR 2007
BORNMANN_L(2008)2:217 Bornmann, L Latent Markov modeling applied to grant peer review J INFORMETR 2008
BORNMANN_L(2009)81:407 Bornmann, L The influence of the applicants' gender on the modeling of a peer review process by using latent Markov models SCIENTOMETRICS 2009
BORNMANN_L(2010)32:5 Bornmann, L The manuscript reviewing process: Empirical research on review requests, review sequences, and decision rules in peer review LIBR INFORM SCI RES 2010
BORNMANN_L(2011)174:857 Bornmann, L A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?J R STAT SOC A STAT 2011
BORNMANN_L(2013)7:286 Bornmann, L The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000J INFORMETR 2013
WALTMAN_L(2014)65:433 Waltman, L F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations J ASSOC INF SCI TECH 2014
name author title journal year
MELTZER_B(1949)55:25 Meltzer, BN The productivity of social scientists AM J SOCIOL 1949
DENNIS_W(1954)79:180 Dennis, W Bibliographies of eminent scientists Scient. Monthly 1954
MERTON_R(1957)22:635 MERTON, RK PRIORITIES IN SCIENTIFIC DISCOVERY - A CHAPTER IN THE SOCIOLOGY OF SCIENCE AM SOCIOL REV 1957
POLANYI_M(1958): Polanyi, M Personal Knowledge: Towards a Post-Critical Philosophy book 1958
CRANE_D(1965)30:699 Crane, D Scientists at major and minor universities AM SOCIOL REV 1965
BAYER_A(1966)39:381 Bayer, AE Some Correlates of a Citation Measure of Productivity in Science Sociology of Education 1966
STORER_N(1966): Storer, N The Social System of Science book 1966
CARTTER_A(1966): Carter, AM An assessment of quality in graduate education AM Council on Education1966
CRANE_D(1967)2:195 CRANE, D GATEKEEPERS OF SCIENCE - SOME FACTORS AFFECTING SELECTION OF ARTICLES FOR SCIENTIFIC JOURNALS AM SOCIOL 1967
COLE_S(1967)32:377 COLE, S SCIENTIFIC OUTPUT AND RECOGNITION - STUDY IN OPERATION OF REWARD SYSTEM IN SCIENCE AM SOCIOL REV 1967
ZUCKERMA_H(1971)9:66 ZUCKERMA.H PATTERNS OF EVALUATION IN SCIENCE - INSTITUTIONALISATION, STRUCTURE AND FUNCTIONS OF REFEREE SYSTEM MINERVA 1971
INGELFIN_F(1974)56:686 INGELFIN.FJ PEER REVIEW IN BIOMEDICAL PUBLICATION AM J MED 1974
CICCHETT_D(1980)35:300 CICCHETTI, DV RELIABILITY OF REVIEWS FOR THE AMERICAN-PSYCHOLOGIST - A BIOSTATISTICAL ASSESSMENT OF THE DATA AM PSYCHOL 1980
PETERS_D(1982)5:187 PETERS, DP PEER-REVIEW PRACTICES OF PSYCHOLOGICAL JOURNALS - THE FATE OF ACCEPTED, PUBLISHED ARTICLES, SUBMITTED AGAIN BEHAV BRAIN SCI 1982
BAILAR_J(1985)312:654 BAILAR, JC JOURNAL PEER-REVIEW - THE NEED FOR A RESEARCH AGENDA NEW ENGL J MED 1985
RENNIE_D(1986)256:2391 RENNIE, D GUARDING THE GUARDIANS - A CONFERENCE ON EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC 1986
ROBIN_E(1987)91:252 ROBIN, ED PEER-REVIEW IN MEDICAL JOURNALS CHEST 1987
MCNUTT_R(1990)263:1371 MCNUTT, RA THE EFFECTS OF BLINDING ON THE QUALITY OF PEER-REVIEW - A RANDOMIZED TRIAL JAMA-J AM MED ASSOC 1990
RENNIE_D(1992)13:443 RENNIE, D SUSPENDED JUDGMENT - EDITORIAL PEER-REVIEW - LET US PUT IT ON TRIAL CONTROL CLIN TRIALS 1992
RENNIE_D(1993)270:2856 RENNIE, D MORE PEERING INTO EDITORIAL PEER-REVIEW JAMA-J AM MED ASSOC 1993
RENNIE_D(1994)272:91 RENNIE, D THE 2ND INTERNATIONAL-CONGRESS ON PEER-REVIEW IN BIOMEDICAL PUBLICATION JAMA-J AM MED ASSOC 1994
SMITH_R(1994)309:143 SMITH, R PROMOTING RESEARCH INTO PEER-REVIEW BRIT MED J 1994
JEFFERSO_T(1995)4:383 JEFFERSON, T ARE GUIDELINES FOR PEER-REVIEWING ECONOMIC EVALUATIONS NECESSARY - A SURVEY OF CURRENT EDITORIAL PRACTICE HEALTH ECON 1995
DRUMMOND_M(1996)313:275 Drummond, MF Guidelines for authors and peer reviewers of economic submissions to the BMJ BRIT MED J 1996
JEFFERSO_T(1998)280:275 Jefferson, T Evaluating the BMJ guidelines for economic submissions - Prospective audit of economic submissions to BMJ and The Lancet JAMA-J AM MED ASSOC 1998
RENNIE_D(1998)280:214 Rennie, D Peer review in Prague JAMA-J AM MED ASSOC 1998
SMITH_R(1999)318:4 Smith, R Opening up BMJ peer review - A beginning that should lead to complete transparency BRIT MED J 1999
GOLDBECK_S(1999)318:44 Goldbeck-Wood, S Evidence on peer review - scientific quality control or smokescreen? BRIT MED J 1999
WALSH_E(2000)176:47 Walsh, E Open peer review: a randomised controlled trial BRIT J PSYCHIAT 2000
JEFFERSO_T(2002)287:2784 Jefferson, T Effects of editorial peer review - A systematic review JAMA-J AM MED ASSOC 2002
RENNIE_D(2002)287:2759 Rennie, D Fourth International Congress on Peer Review in Biomedical Publication JAMA-J AM MED ASSOC 2002
OPTHOF_T(2002)56:339 Opthof, T The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical biasCARDIOVASC RES 2002
JEFFERSO_T(2002)287:2786 Jefferson, T Measuring the quality of editorial peer review JAMA-J AM MED ASSOC 2002
BORNMANN_L(2005)14:15 Bornmann, L Committee peer review at an international research foundation: predictive validity and fairness of selection decisions on post-graduate fellowship applicationsRES EVALUAT 2005
DANIEL_H(2005)18:143 Daniel, HD Publications as a measure of scientific advancement and of scientists' productivity LEARN PUBL 2005
BORNMANN_L(2006)68:427 Bornmann, L Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicantsSCIENTOMETRICS 2006
BORNMANN_L(2007)1:204 Bornmann, L Convergent validation of peer review decisions using the h index - Extent of and reasons for type I and type II errors J INFORMETR 2007
BORNMANN_L(2008)59:1841 Bornmann, L Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhereJ AM SOC INF SCI TEC 2008
BORNMANN_L(2009)22:117 Bornmann, L The luck of the referee draw: the effect of exchanging reviews LEARN PUBL 2009
BORNMANN_L(2011)45:199 Bornmann, L Scientific Peer Review ANNU REV INFORM SCI 2011
BORNMANN_L(2011)174:857 Bornmann, L A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?J R STAT SOC A STAT 2011
BORNMANN_L(2013)7:286 Bornmann, L The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000 J INFORMETR 2013
WALTMAN_L(2014)65:433 Waltman, L F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations J ASSOC INF SCI TECH 2014
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The main path publicationstill 1982
Journals: social science journals (sociological, psychological,educational,...).
The most influential authors: Cole and Cole (1967), Zuckermanand Merton (1971), Garfield (1972), Gottfredson (1978), and Petersand Ceci (1982).
Topics: scientific productivity, citation measures as measures ofscientific accomplishement, scientific output and recognition,evaluation in science, referee system, journal evaluation,peer-evaluation system, review process, peer review practices.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The main path publicationsfrom 1983 to 2002
Journals: biomedical journals, mainly in JAMA. From 1986 theInternational Congress on Peer Review and Biomedical Publication isorganized every four years.
The most influential authors: Rennie (1986, 1992, 1993, 1994,2002), Smith (1994, 1999), Jefferson (1995, 1998, 2002), and theircollaborators.
Topics: the effects of blinding on review quality, research into peerreview, guidelines for peer reviewing, monitoring the peer reviewperformance, open peer review, bias in peer review system, measuringthe quality of editorial peer review. Development of meta-analysisand systematic reviews approaches.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
The main path publicationsfrom 2003 on
Journals: specialized journals on science studies: Scientometrics, ResearchEvaluation, Journal of Informetrics, JASIST.
The most influential authors: Bornmann and Daniel (2005, 2006, 2007,2008, 2009, 2010, 2011). The last paper on the main path is Waltman andCostas (2014).
Topics: Bornmann and Daniel studied the validity of commitee peer reviewprocess for awarding long-term fellowship to post-graduate researchers, theuse of h-index and pre-screening of applications at Boehringer IngelheimFonds. They also analysed citations of accepted and rejected papers at aprime chemistry journal (Angewandte Chemie Internationa Edition -AC-IE), the effect of exchanging reviews, the peer review process in thisjournal, the validity of its editorial decisions. The last two papers(Bormann and Leydesdorff, 2013; Waltman and Costas, 2014) use F1000(a post-publication peer review system) recommendations as a source ofresearch evaluation.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
Cuts, islands, cores
The node-cut of a network N = (V,L, p), p : V → R, at level t is a subnetworkN (t) = (V ′,L(V ′), p), determined by the set V ′ = {v ∈ V : p(v) ≥ t} and L(V ′)is the set of links from L that have both endnodes in V ′.
The link-cut of a network N = (V,L,w), w : L → R, at level t is a subnetworkN (t) = (V(L′),L′,w), determined by the set L′ = {e ∈ L : w(e) ≥ t} andV(L′) is the set of all endnodes of the links from L′.If we represent a given or computed value of nodes / links as a height of nodes /links and we immerse the network into a water up to selected level we get islands.Varying the level we get different islands.
The subgraph H = (C , E|C) induced by the set C ⊆ V is a p-core at level t ∈ Riff ∀v ∈ C : t ≤ p(v ,C) and C is a maximal such set.Ordinary core: p(v ,C) = degH(v)pS -core: p(v ,C) =
∑u∈C w(v , u), w(v , u) is the weight of link (v , u)
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC islands [20 200]
Pajek
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Island1 [100]/[200]wmax = 2.936 · 10−1
FISHER_M{1994}272:143
MCNUTT_R{1990}263:1371
JEFFERSO_T{1995}4:383
DRUMMOND_M{1996}313:275
CICCHETT_D{1980}35:300
GOTTFRED_S{1978}33:920
INGELFIN_F{1974}56:686
RUDERFER_M{1980}3:533
ZUCKERMA_H{1971}9:66
PETERS_D{1982}5:187
CALLAHAM_M{1998}280:254
WEBER_E{1998}280:257
POLANYI_M{1958}:
BAILAR_J{1985}312:654
CRANE_D{1967}2:195
WILSON_J{1978}61:1697
RENNIE_D{1990}263:1317
RENNIE_D{1986}256:2391
CLEARY_J{1988}22:601
ROBIN_E{1987}91:252
GODLEE_F{1998}280:237
BLACK_N{1998}280:231
JUSTICE_A{1998}280:240
VANROOYE_S{1998}280:234
VANROOYE_S{1999}318:23
MOOSSY_J{1985}44:225
CAMPANAR_J{1998}19:181
JEFFERSO_T{1998}280:275
RENNIE_D{1994}272:91
WALSH_E{2000}176:47
JEFFERSO_T{2002}287:2784
LOCK_S{1990}263:1341
BORNMANN_L{2005}63:297
MARSH_H{2008}63:160
BAYER_A{1966}39:381
COLE_S{1967}32:377
FEURER_I{1994}272:98
SMITH_R{1994}309:143
LINDSEY_D{1978}12:45
MERTON_R{1957}22:635
MICHELS_R{1995}76:217
RENNIE_D{1993}270:2856
GARFIELD_E{1972}178:471
SMITH_R{1999}318:4
BORNMANN_L{2008}47:7173
BORNMANN_L{2007}1:204BORNMANN_L{2005}13:296
BORNMANN_L{2009}22:117
BORNMANN_L{2008}2:280BORNMANN_L{2007}1:83
BORNMANN_L{2008}59:1841 BORNMANN_L{2006}68:427
BORNMANN_L{2005}14:15HOJAT_M{2003}8:75
SULS_J{2009}4:40
BORNMANN_L{2011}45:199
CALLAHAM_M{1998}280:229
GOLDBECK_S{1999}318:44
BORNMANN_L{2008}59:830
CHO_M{1998}280:243
HOWARD_L{1998}173:110
RENNIE_D{1998}280:214
DANIEL_H{2005}18:143
JAYASING_U{2006}69:591
CRANE_D{1965}30:699
MELTZER_B{1949}55:25
STORER_N{1966}:
WESSELY_S{1996}26:883
RENNIE_D{1992}13:443
RENNIE_D{2002}287:2759
ROWLAND_F{2002}15:247
DAVIDOFF_F{1998}128:66
BORNMANN_L{2010}32:5
BORNMANN_L{2011}174:857
BORNMANN_L{2006}15:209
OPTHOF_T{2002}56:339
WETS_K{2003}16:249
ALPERT_J{2007}120:287
BORNMANN_L{2009}81:407
SEN_C{2012}16:293
LEE_C{2013}64:2
FRANCESC_M{2011}5:275
BORNMANN_L{2008}2:217
BENDA_W{2011}27:166
BORNMANN_L{2013}7:286
SQUAZZON_F{2012}6:265
ABRAMO_G{2011}89:929
WALTMAN_L{2014}65:433BORNMANN_L{2010}85:681
SQUAZZON_F{2013}42:287
WALTMAN_L{2011}88:1017
MOHAMMAD_E{2013}97:383
CARTTER_A{1966}:
VERLEYSE_F{2013}64:428
VANARENS_P{2012}25:381
BORNMANN_L{2010}64:72
DENNIS_W{1954}79:180
=BLACK_N{1998}280:231
=VANROOYE_S{1998}280:234
=JEFFERSO_T{2002}287:2786
Pajek
FISHER_M{1994}272:143
KASSIRER_J{1994}272:96
MCNUTT_R{1990}263:1371
MOHER_D{1995}16:62
OXMAN_A{1991}44:91
JADAD_A{1996}17:1
JEFFERSO_T{1995}4:383
DRUMMOND_M{1996}313:275
CICCHETT_D{1991}14:119
DERSIMON_R{1986}7:177
MOHER_D{1999}354:1896BEGG_C{1996}276:637
BOWEN_D{1972}27:221
BRACKBIL_Y{1970}25:937
CICCHETT_D{1980}35:300
COLMAN_A{1979}32:390
GOTTFRED_S{1978}33:920
INGELFIN_F{1974}56:686MCREYNOL_P{1971}26:400
MERTON_R{1968}:
RUDERFER_M{1980}3:533
SCOTT_W{1974}29:698
WOLFF_W{1973}28:257
YOTOPOUL_P{1961}51:665
ZUCKERMA_H{1971}9:66
PETERS_D{1982}5:187
SACKS_H{1987}316:450
DICKERSI_K{1990}263:1385
DICKERSI_K{1992}267:374
EASTERBR_P{1991}337:867
SCHERER_R{1994}272:158
GOODMAN_S{1994}121:11
CHALMERS_I{1990}263:1401
DEBELLEF_C{1992}3:187
DICKERSI_K{1987}8:343
CALLAHAM_M{1998}280:254
WEBER_E{1998}280:257
POLANYI_M{1958}:
BAILAR_J{1985}312:654
BERELSON_B{1960}:
CHASE_J{1970}5:262CLEARY_F{1960}50:1011
CRANE_D{1967}2:195GOODRICH_D{1945}10:716
HARGENS_L{1988}53:139
MARSH_H{1981}73:872
SMIGEL_E{1970}5:19
WHITEHUR_G{1984}39:22
WILSON_J{1978}61:1697
GARFUNKE_J{1990}263:1369
HAGGARD_E{1958}:
RENNIE_D{1990}263:1317
RENNIE_D{1986}256:2391
CLEARY_J{1988}22:601
ROBIN_E{1987}91:252
NYLENNA_M{1994}272:149
GODLEE_F{1998}280:237
BLACK_N{1998}280:231
JUSTICE_A{1998}280:240
VANROOYE_S{1998}280:234
VANROOYE_S{1999}318:23
MOOSSY_J{1985}44:225
MERTON_R{1968}159:56
CHALMERS_I{1990}263:1405
BAUM_M{1981}305:795
CAMPANAR_J{1998}19:181
WOLFF_W{1970}25:636
MOHER_D{1998}352:609
BINGHAM_C{1998}352:441CALLAHAM_M{1998}32:318
JEFFERSO_T{1998}280:275
PIERIE_J{1996}348:1480
RENNIE_D{1998}280:213
RENNIE_D{1994}272:91
STRAYHOR_J{1993}150:947
WALSH_E{2000}176:47
JEFFERSO_T{2002}287:2784
GARFIELD_E{1955}122:108
SCHROTER_S{2004}328:673
LOCK_S{1990}263:1341
BORNMANN_L{2005}63:297
JAYASING_U{2003}166:279
MARSH_H{2008}63:160
MARSH_H{2009}79:1290
CECI_S{2011}108:3157
WESSELY_S{1998}352:301
ROTHWELL_P{2000}123:1964
SHARP_D{1990}263:1390
BURNHAM_J{1990}263:1323
BAYER_A{1966}39:381
COLE_S{1967}32:377
FEURER_I{1994}272:98
JUSTICE_A{1994}272:117
RAY_J{2000}109:131
VANROOYE_S{1999}52:625
WAGER_E{2001}14:257
SMITH_R{1994}309:143
JUDSON_H{1994}272:92
LEGENDRE_A{1995}207:36
LINDSEY_D{1978}12:45
MERTON_R{1957}22:635
MICHELS_R{1995}76:217
RENNIE_D{1993}270:2856
GARFIELD_E{1972}178:471
SMITH_R{1999}318:4
GODLEE_F{2002}287:2762
BORNMANN_L{2008}47:7173
BORNMANN_L{2007}1:204BORNMANN_L{2005}13:296
BORNMANN_L{2009}22:117
BORNMANN_L{2008}2:280BORNMANN_L{2007}1:83
BORNMANN_L{2008}59:1841 BORNMANN_L{2006}68:427
BORNMANN_L{2005}14:15CAMPANAR_J{1998}19:277
GIBSON_M{2008}112:646
GILBERT_J{1994}272:139
HOJAT_M{2003}8:75
MARSH_H{2007}42:33
OPTHOF_T{2009}17:145
SULS_J{2009}4:40
BORNMANN_L{2011}45:199
CALLAHAM_M{1998}280:229
GOLDBECK_S{1999}318:44
BORNMANN_L{2008}59:830
CHO_M{1998}280:243
HOWARD_L{1998}173:110
RENNIE_D{1998}280:214
DANIEL_H{2005}18:143
JAYASING_U{2006}69:591
SCHROTER_S{2006}295:314
CRANE_D{1965}30:699
MELTZER_B{1949}55:25
GOLDBECK_S{1998}316:86
STORER_N{1966}:
WESSELY_S{1996}26:883
SMITH_R{1997}315:759
HORROBIN_D{2001}22:51
RENNIE_D{1992}13:443
PASTERNA_S{1966}19:38
PORTER_J{1964}28:211
KRONICK_D{1962}:
POLAK_J{1995}165:685
RENNIE_D{2002}287:2759
BORNMANN_L{2008}64:45
BORNMANN_L{2010}4:211
VANROOYE_S{2001}14:85
ROWLAND_F{2002}15:247
DAVIDOFF_F{1998}128:66
BORNMANN_L{2010}32:5
BORNMANN_L{2011}174:857
BORNMANN_L{2006}15:209
OPTHOF_T{2002}56:339
WETS_K{2003}16:249
ALPERT_J{2007}120:287
BORNMANN_L{2010}83:493
BORNMANN_L{2009}81:407
SCHULTZ_D{2010}84:277
SEN_C{2012}16:293
LEE_C{2013}64:2
FRANCESC_M{2011}5:275
TILDEN_V{2002}51:275
FREDA_M{2005}37:87
KEARNEY_M{2005}28:444
BORNMANN_L{2008}2:217
BENDA_W{2011}27:166
OPTHOF_T{2011}88:1011
BORNMANN_L{2013}7:286
SQUAZZON_F{2012}6:265
ABRAMO_G{2011}89:929
WALTMAN_L{2014}65:433BORNMANN_L{2010}85:681
OLBRECHT_M{2010}19:293
SQUAZZON_F{2013}42:287
WALTMAN_L{2011}88:1017
MOHAMMAD_E{2013}97:383
CARTTER_A{1966}:
VERLEYSE_F{2013}64:428
VANARENS_P{2012}25:381
BORNMANN_L{2010}64:72
COLE_S{1968}33:397
FRANTZ_T{1968}47:384
GOUDSMIT_S{1969}22:23
GOUDSMIT_S{1967}20:12
KENISTON_H{1959}:
LIBBEY_M{1967}:49
MERTON_R{1961}105:470
POLANYI_M{1958}:CH6
RAFFEL_S{1968}:
WHITLEY_R{1970}18:241
ZIMAN_J{1966}:148
DENNIS_W{1954}79:180
=DICKERSI_K{1990}263:1385
=CHALMERS_I{1990}263:1401
=BLACK_N{1998}280:231
=VANROOYE_S{1998}280:234
=CHALMERS_I{1990}263:1405
=JEFFERSO_T{2002}287:2786
Pajek
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – island1 / list of titles
name author title journal year
GOODRICH_D(1945)10:�16 N� N� N� 1945
M��T��R_B(1949)55:�5 N� N� N� 1949
D�NNIS_W(1954)�9:180 N� N� N� 1954
G�RFI��D_�(1955)1��:108 G�RFI��D, � CIT�TION IND�E�S FOR SCI�NC� - N�W DIM�NSION IN DOCUM�NT�TION THROUGH �SSOCI�TIONOF ID��S SCI�NC� 1955
M�RTON_R(195�)��:635 M�RTON, RK PRIORITI�S IN SCI�NTIFIC DISCOV�RY - � CH�PT�R IN TH� SOCIO�OGY OF SCI�NC� �M SOCIO� R�V 195�
PO��NYI_M(1958): N� N� N� 1958
H�GG�RD_�(1958): N� N� N� 1958
PO��NYI_M(1958):CH6 N� N� N� 1958
K�NISTON_H(1959): N� N� N� 1959
B�R��SON_B(1960): N� N� N� 1960
C���RY_F(1960)50:1011 N� N� N� 1960
YOTOPOU�_P(1961)51:665 N� N� N� 1961
M�RTON_R(1961)105:4�0 N� N� N� 1961
KRONICK_D(196�): N� N� N� 196�
PORT�R_J(1964)�8:�11 N� N� N� 1964
CR�N�_D(1965)30:699 N� N� N� 1965
B�Y�R_�(1966)39:381 N� N� N� 1966
STOR�R_N(1966): N� N� N� 1966
P�ST�RN�_S(1966)19:38 N� N� N� 1966
C�RTT�R_�(1966): N� N� N� 1966
�IM�N_J(1966):148 N� N� N� 1966
CR�N�_D(196�)�:195 CR�N�, D G�T�K��P�RS OF SCI�NC� - SOM� F�CTORS �FF�CTING S���CTIONOF �RTIC��S FOR SCI�NTIFIC JOURN��S �M SOCIO� 196�
CO��_S(196�)3�:3�� CO��, S SCI�NTIFIC OUTPUT �ND R�COGNITION - STUDY IN OP�R�TIONOF R�W�RD SYST�M IN SCI�NC� �M SOCIO� R�V 196�
GOUDSMIT_S(196�)�0:1� N� N� N� 196�
�IBB�Y_M(196�):49 N� N� N� 196�
M�RTON_R(1968): N� N� N� 1968
M�RTON_R(1968)159:56 M�RTON, RK M�TTH�W �FF�CT IN SCI�NC� SCI�NC� 1968
CO��_S(1968)33:39� N� N� N� 1968
FR�NT�_T(1968)4�:384 N� N� N� 1968
R�FF��_S(1968): N� N� N� 1968
GOUDSMIT_S(1969)��:�3 N� N� N� 1969
BR�CKBI�_Y(19�0)�5:93� N� N� N� 19�0
CH�S�_J(19�0)5:�6� N� N� N� 19�0
SMIG��_�(19�0)5:19 SMIG��, �O F�CTORS IN �DITORI�� D�CISION �M SOCIO� 19�0
WO�FF_W(19�0)�5:636 N� N� N� 19�0
WHIT��Y_R(19�0)18:�41 N� N� N� 19�0
MCR�YNO�_P(19�1)�6:400 N� N� N� 19�1
�UCK�RM�_H(19�1)9:66 �UCK�RM�.H P�TT�RNS OF �V��U�TION IN SCI�NC� - INSTITUTION��IS�TION, STRUCTUR� �ND FUNCTIONS OF R�F�R�� SYST�M MIN�RV� 19�1
BOW�N_D(19��)��:��1 N� N� N� 19��
G�RFI��D_�(19��)1�8:4�1 G�RFI��D, � CIT�TION �N��YSIS �S � TOO� IN JOURN�� �V��U�TION - JOURN��S C�N B� R�NK�D BY FR��U�NCY �ND IMP�CT OF CIT�TIONS FOR SCI�SCI�NC� 19��
WO�FF_W(19�3)�8:�5� N� N� N� 19�3
ING��FIN_F(19�4)56:686 ING��FIN.FJ P��R R�VI�W IN BIOM�DIC�� PUB�IC�TION �M J M�D 19�4
SCOTT_W(19�4)�9:698 SCOTT,W� INT�R-R�F�R�� �GR��M�NT ON SOM� CH�R�CT�RISTICS OF M�NUSCRIPTS SUBMITT�D TO JOURN�� OF P�RSON��ITY �ND SOCI��-PSYCH�M PSYCHO� 19�4
GOTTFR�D_S(19�8)33:9�0 GOTTFR�DSON, SD �V��U�TING PSYCHO�OGIC��-R�S��RCH R�PORTS - DIM�NSIONS, R��I�BI�ITY, �ND CORR���T�S OF �U��ITY JUDGM�NTS �M PSYCHO� 19�8
WI�SON_J(19�8)61:169� WI�SON, JD �0TH �NNU��-M��TING OF �M�RIC�N-SOCI�TY-FOR-C�INIC��-INV�STIG�TION, S�N-FR�NCISCO, C��IFORNI�, 30 �PRI� 19�8 - PR�SID�NT J C�IN INV�ST 19�8
�INDS�Y_D(19�8)1�:45 �INDS�Y, D OUT�OOK OF JOURN�� �DITORS �ND R�F�R��S ON NORM�TIV� CRIT�RI� OF SCI�NTIFIC CR�FTSM�NSHIP - VI�WPOINTS FROM PSYCHO�O�U�� �U�NT 19�8
CO�M�N_�(19�9)3�:390 CO�M�N, �M �DITORI�� RO�� IN �UTHOR - R�F�R�� DIS�GR��M�NTS B BRIT PSYCHO� SOC 19�9
CICCH�TT_D(1980)35:300 CICCH�TTI, DV R��I�BI�ITY OF R�VI�WS FOR TH� �M�RIC�N-PSYCHO�OGIST - � BIOST�TISTIC�� �SS�SSM�NT OF TH� D�T� �M PSYCHO� 1980
RUD�RF�R_M(1980)3:533 RUD�RF�R, M TH� F����CY OF P��R-R�VI�W - JUDGM�NT WITHOUT SCI�NC� �ND � C�S�-HISTORY SP�CU��T SCI T�CHNO 1980
M�RSH_H(1981)�3:8�� M�RSH, HW INT�RJUDGM�NT�� R��I�BI�ITY OF R�VI�WS FOR TH� JOURN��-OF-�DUC�TION��-PSYCHO�OGY J �DUC PSYCHO� 1981
B�UM_M(1981)305:�95 B�UM, M� � SURV�Y OF C�INIC��-TRI��S OF �NTIBIOTIC-PROPHY��EIS IN CO�ON SURG�RY - �VID�NC� �G�INST FURTH�R US� OF NO-TR��TM�NT CON�W �NG� J M�D 1981
P�T�RS_D(198�)5:18� P�T�RS, DP P��R-R�VI�W PR�CTIC�S OF PSYCHO�OGIC�� JOURN��S - TH� F�T� OF �CC�PT�D, PUB�ISH�D �RTIC��S, SUBMITT�D �G�IN B�H�V BR�IN SCI 198�
WHIT�HUR_G(1984)39:�� WHIT�HURST, GJ INT�RR�T�R �GR��M�NT FOR JOURN�� M�NUSCRIPT R�VI�WS �M PSYCHO� 1984
B�I��R_J(1985)31�:654 B�I��R, JC JOURN�� P��R-R�VI�W - TH� N��D FOR � R�S��RCH �G�ND� N�W �NG� J M�D 1985
MOOSSY_J(1985)44:��5 MOOSSY, J �NONYMOUS �UTHORS, �NONYMOUS R�F�R��S - �N �DITORI�� �EP�OR�TION J N�UROP�TH �EP N�U 1985
D�RSIMON_R(1986)�:1�� D�RSIMONI�N, R M�T��N��YSIS IN C�INIC��-TRI��S CONTRO� C�IN TRI��S 1986
R�NNI�_D(1986)�56:�391 R�NNI�, D GU�RDING TH� GU�RDI�NS - � CONF�R�NC� ON �DITORI�� P��R-R�VI�W J�M�-J �MM�D �SSOC 1986
S�CKS_H(198�)316:450 S�CKS, HS M�T�-�N��YS�S OF R�NDOMI��D CONTRO���D TRI��S N�W �NG� J M�D 198�
DICK�RSI_K(198�)8:343 DICK�RSIN, K PUB�IC�TION BI�S �ND C�INIC��-TRI��S CONTRO� C�IN TRI��S 198�
ROBIN_�(198�)91:�5� ROBIN, �D P��R-R�VI�W INM�DIC�� JOURN��S CH�ST 198�
H�RG�NS_�(1988)53:139 H�RG�NS, �� SCHO��R�Y CONS�NSUS �ND JOURN�� R�J�CTION R�T�S �M SOCIO� R�V 1988
C���RY_J(1988)��:601 C���RY, JD B�IND V�RSUS NONB�IND R�VI�W - SURV�Y OF S���CT�D M�DIC�� JOURN��S DRUG INT�� C�IN PH�R 1988
MCNUTT_R(1990)�63:13�1 MCNUTT, R� TH� �FF�CTS OF B�INDING ON TH� �U��ITY OF P��R-R�VI�W - � R�NDOMI��D TRI�� J�M�-J �MM�D �SSOC 1990
DICK�RSI_K(1990)�63:1385 DICK�RSIN, K TH� �EIST�NC� OF PUB�IC�TION BI�S �ND RISK-F�CTORS FOR ITS OCCURR�NC� J�M�-J �MM�D �SSOC 1990
CH��M�RS_I(1990)�63:1401 CH��M�RS, I � COHORT STUDY OF SUMM�RY R�PORTS OF CONTRO���D TRI��S J�M�-J �MM�D �SSOC 1990
G�RFUNK�_J(1990)�63:1369 G�RFUNK��, JM PROB��MS ID�NTIFI�D BY S�COND�RY R�VI�W OF �CC�PT�D M�NUSCRIPTS J�M�-J �MM�D �SSOC 1990
R�NNI�_D(1990)�63:131� R�NNI�, D �DITORI�� P��R-R�VI�W IN BIOM�DIC�� PUB�IC�TION - TH� 1ST-INT�RN�TION��-CONGR�SS J�M�-J �MM�D �SSOC 1990
CH��M�RS_I(1990)�63:1405 CH��M�RS, I UND�RR�PORTING R�S��RCH IS SCI�NTIFIC MISCONDUCT J�M�-J �MM�D �SSOC 1990
�OCK_S(1990)�63:1341 �OCK, S WH�T DO P��R R�VI�W�RS DO J�M�-J �MM�D �SSOC 1990
SH�RP_D(1990)�63:1390 SH�RP, DW WH�T C�N �ND SHOU�D B� DON� TO R�DUC� PUB�IC�TION BI�S - TH� P�RSP�CTIV� OF �N �DITOR J�M�-J �MM�D �SSOC 1990
BURNH�M_J(1990)�63:13�3 BURNH�M, JC TH� �VO�UTIONOF �DITORI�� P��R-R�VI�W J�M�-J �MM�D �SSOC 1990
DICK�RSI_K(1990)�63:1385 DICK�RSIN, K TH� �EIST�NC� OF PUB�IC�TION BI�S �ND RISK-F�CTORS FOR ITS OCCURR�NC� J�M�-J �MM�D �SSOC 1990
CH��M�RS_I(1990)�63:1401 CH��M�RS, I � COHORT STUDY OF SUMM�RY R�PORTS OF CONTRO���D TRI��S J�M�-J �MM�D �SSOC 1990
CH��M�RS_I(1990)�63:1405 CH��M�RS, I UND�RR�PORTING R�S��RCH IS SCI�NTIFIC MISCONDUCT J�M�-J �MM�D �SSOC 1990
OEM�N_�(1991)44:91 OEM�N, �D �GR��M�NT �MONG R�VI�W�RS OF R�VI�W �RTIC��S J C�IN �PID�MIO� 1991
CICCH�TT_D(1991)14:119 CICCH�TTI, DV TH� R��I�BI�ITY OF P��R-R�VI�W FOR M�NUSCRIPT �ND GR�NT SUBMISSIONS - � CROSS-DISCIP�IN�RY INV�STIG�TION B�H�V BR�IN SCI 1991
��ST�RBR_P(1991)33�:86� ��ST�RBROOK, PJ PUB�IC�TION BI�S IN C�INIC�� R�S��RCH ��NC�T 1991
DICK�RSI_K(199�)�6�:3�4 DICK�RSIN, K F�CTORS INF�U�NCING PUB�IC�TIONOF R�S��RCH R�SU�TS - FO��OW-UP OF �PP�IC�TIONS SUBMITT�D TO � INSTITUTION�� R�VI�W BO J�M�-J �MM�D �SSOC 199�
D�B����F_C(199�)3:18� D�B����F�UI���, C TH� F�T� OF �BSTR�CTS SUBMITT�D TO � C�NC�R M��TING - F�CTORSWHICH INF�U�NC� PR�S�NT�TION �ND SUBS��U�NT PUB�IC�TIO�NNONCO� 199�
R�NNI�_D(199�)13:443 R�NNI�, D SUSP�ND�D JUDGM�NT - �DITORI�� P��R-R�VI�W - ��T US PUT IT ON TRI�� CONTRO� C�IN TRI��S 199�
STR�YHOR_J(1993)150:94� STR�YHORN, J �N INT�RV�NTION TO IMPROV� TH� R��I�BI�ITY OF M�NUSCRIPT R�VI�WS FOR TH� JOURN��-OF-TH�-�M�RIC�N-�C�D�MY-OF-CHI�D-�N�M J PSYCHI�T 1993
R�NNI�_D(1993)��0:�856 R�NNI�, D MOR� P��RING INTO �DITORI�� P��R-R�VI�W J�M�-J �MM�D �SSOC 1993
FISH�R_M(1994)���:143 FISH�R, M TH� �FF�CTS OF B�INDING ON �CC�PT�NC� OF R�S��RCH P�P�RS BY P��R-R�VI�W J�M�-J �MM�D �SSOC 1994
K�SSIR�R_J(1994)���:96 K�SSIR�R, JP P��R-R�VI�W - CRUD� �ND UND�RSTUDI�D, BUT INDISP�NS�B�� J�M�-J �MM�D �SSOC 1994
SCH�R�R_R(1994)���:158 SCH�R�R, RW FU�� PUB�IC�TIONOF R�SU�TS INITI���Y PR�S�NT�D IN �BSTR�CTS - � M�T��N��YSIS J�M�-J �MM�D �SSOC 1994
GOODM�N_S(1994)1�1:11 GOODM�N, SN M�NUSCRIPT �U��ITY B�FOR� �ND �FT�R P��R-R�VI�W �ND �DITING �T �NN��S OF INT�RN��-M�DICIN� �NN INT�RNM�D 1994
NY��NN�_M(1994)���:149 NY��NN�, M MU�TIP�� B�IND�D R�VI�WS OF TH� � M�NUSCRIPTS - �FF�CTS OF R�F�R�� CH�R�CT�RISTICS �ND PUB�IC�TION ��NGU�G� J�M�-J �MM�D �SSOC 1994
R�NNI�_D(1994)���:91 R�NNI�, D TH� �ND INT�RN�TION��-CONGR�SS ON P��R-R�VI�W IN BIOM�DIC�� PUB�IC�TION J�M�-J �MM�D �SSOC 1994
F�UR�R_I(1994)���:98 F�UR�R, ID �V��U�TING P��R R�VI�WS - PI�OT T�STING OF � GR�DING INSTRUM�NT J�M�-J �MM�D �SSOC 1994
JUSTIC�_�(1994)���:11� JUSTIC�, �C DO R��D�RS �ND P��R R�VI�W�RS �GR�� ONM�NUSCRIPT �U��ITY J�M�-J �MM�D �SSOC 1994
SMITH_R(1994)309:143 SMITH, R PROMOTING R�S��RCH INTO P��R-R�VI�W BRIT M�D J 1994
JUDSON_H(1994)���:9� JUDSON, HF STRUCTUR�� TR�NSFORM�TIONS OF TH� SCI�NC�S �ND TH� �ND OF P��R-R�VI�W J�M�-J �MM�D �SSOC 1994
GI�B�RT_J(1994)���:139 GI�B�RT, JR IS TH�R� G�ND�R BI�S IN J�M�S P��R-R�VI�W PROC�SS J�M�-J �MM�D �SSOC 1994
MOH�R_D(1995)16:6� MOH�R, D �SS�SSING TH� �U��ITY OF R�NDOMI��D CONTRO���D TRI��S - �N �NNOT�T�D-BIB�IOGR�PHY OF SC���S �ND CH�CK�ISTS CONTRO� C�IN TRI��S 1995
J�FF�RSO_T(1995)4:383 J�FF�RSON, T �R� GUID��IN�S FOR P��R-R�VI�WING �CONOMIC �V��U�TIONS N�C�SS�RY - � SURV�Y OF CURR�NT �DITORI�� PR�CTIC� H���TH �CON 1995
��G�NDR�_�(1995)�0�:36 ��G�NDR�, �M P��R-R�VI�W OF M�NUSCRIPTS FOR BIOM�DIC�� JOURN��S J �M V�T M�D �SSOC 1995
MICH��S_R(1995)�6:�1� MICH��S, R P��R-R�VI�W INT J PSYCHO�N�� 1995
PO��K_J(1995)165:685 PO��K, JF TH� RO�� OF TH� M�NUSCRIPT R�VI�W�R IN TH� P��R-R�VI�W PROC�SS �M J RO�NTG�NO� 1995
J�D�D_�(1996)1�:1 Jadad, �R �ssessing the quality of reports of randomized clinical trials: Is blinding necessary? CONTRO� C�IN TRI��S 1996
DRUMMOND_M(1996)313:��5 Drummond, MF Guidelines for authors and peer reviewers of economic submissions to the BMJ BRIT M�D J 1996
B�GG_C(1996)��6:63� Begg, C Improving the quality of reporting of randomized controlled trials - The CONSORT statement J�M�-J �MM�D �SSOC 1996
PI�RI�_J(1996)348:1480 Pierie, JP�N Readers� evaluation of effect of peer review and editing on quality of articles in the Nederlands Tijdschrift voor Genees�unde ��NC�T 1996
W�SS��Y_S(1996)�6:883 Wessely, S What do we �now about peer review? PSYCHO� M�D 1996
SMITH_R(199�)315:�59 Smith, R Peer review: Reform or revolution? BRIT M�D J 199�
CALLAHAM�M�1998)280:254 Callaham,ML Pos�t�ve-outcome b�as and other l�m�tat�ons �n the outcome of research abstracts subm�tted to a sc�ent�f�c meet�ng JAMA-J AMM�� ASSOC 1998
W�E�����1998)280:257 Weber, �J Unpubl�shed research from a med�cal spec�alty meet�ng - Why �nvest�gators fa�l to publ�sh JAMA-J AMM�� ASSOC 1998
GO�L���F�1998)280:237 Godlee, F �ffect on the qual�ty of peer rev�ew of bl�nd�ng rev�ewers and as��ng them to s�gn the�r reports - A random�zed controlled tr�al JAMA-J AMM�� ASSOC 1998
ELACK�N�1998)280:231 Elac�, N What ma�es a good rev�ewer and a good rev�ew for a general med�cal journal? JAMA-J AMM�� ASSOC 1998
JUST�C��A�1998)280:240 Just�ce, AC �oes mas��ng author �dent�ty �mprove peer rev�ew qual�ty? - A random�zed controlled tr�al JAMA-J AMM�� ASSOC 1998
VAN�OOY��S�1998)280:234 van �ooyen, S �ffect of bl�nd�ng and unmas��ng on the qual�ty of peer rev�ew - A random�zed tr�al JAMA-J AMM�� ASSOC 1998
CAMPANA��J�1998)19:181 Campanar�o, JM Peer rev�ew for journals as �t stands today - Part 1 SC� COMMUN 1998
MOH�����1998)352:�09 Moher, � �oes qual�ty of reports of random�sed tr�als affect est�mates of �ntervent�on eff�cacy reported �n meta-analyses? LANC�T 1998
E�NGHAM�C�1998)352:441 E�ngham, CM TheMed�cal Journal of Austral�a �nternet peer-rev�ew study LANC�T 1998
CALLAHAM�M�1998)32:318 Callaham,ML �ffect of attendance at a tra�n�ng sess�on on peer rev�ewer qual�ty and performance ANN �M��GM�� 1998
J�FF��SO�T�1998)280:275 Jefferson, T �valuat�ng the EMJ gu�del�nes for econom�c subm�ss�ons - Prospect�ve aud�t of econom�c subm�ss�ons to EMJ and The Lancet JAMA-J AMM�� ASSOC 1998
��NN�����1998)280:213 �enn�e, � Congress on E�omed�cal Peer �ev�ew - H�story, eth�cs, and plans for the future JAMA-J AMM�� ASSOC 1998
W�SS�LY�S�1998)352:301 Wessely, S Peer rev�ew of grant appl�cat�ons: what do we �now? LANC�T 1998
CAMPANA��J�1998)19:277 Campanar�o, JM Peer rev�ew for journals as �t stands today - Part 2 SC� COMMUN 1998
CALLAHAM�M�1998)280:229 Callaham,ML �el�ab�l�ty of ed�tors subject�ve qual�ty rat�ngs of peer rev�ews of manuscr�pts JAMA-J AMM�� ASSOC 1998
CHO�M�1998)280:243 Cho, MK Mas��ng author �dent�ty �n peer rev�ew - What factors �nfluence mas��ng success? JAMA-J AMM�� ASSOC 1998
HOWA���L�1998)173:110 Howard, L Peer rev�ew and ed�tor�al dec�s�on-ma��ng E��T J PSYCH�AT 1998
��NN�����1998)280:214 �enn�e, � Peer rev�ew �n Prague JAMA-J AMM�� ASSOC 1998
GOL�E�CK�S�1998)31�:8� Goldbec�-Wood, S What ma�es a good rev�ewer of manuscr�pts? The EMJ �nv�tes you to jo�n �ts peer rev�ew process E��T M�� J 1998
�AV��OFF�F�1998)128:�� �av�doff, F Mas��ng, bl�nd�ng, and peer rev�ew: The bl�nd lead�ng the bl�nded ANN �NT��NM�� 1998
ELACK�N�1998)280:231 Elac�, N What ma�es a good rev�ewer and a good rev�ew for a general med�cal journal? JAMA-J AMM�� ASSOC 1998
VAN�OOY��S�1998)280:234 van �ooyen, S �ffect of bl�nd�ng and unmas��ng on the qual�ty of peer rev�ew - A random�zed tr�al JAMA-J AMM�� ASSOC 1998
MOH�����1999)354:189� Moher, � �mprov�ng the qual�ty of reports of meta-analyses of random�sed controlled tr�als: the QUO�OM statement LANC�T 1999
VAN�OOY��S�1999)318:23 van �ooyen, S �ffect of open peer rev�ew on qual�ty of rev�ews and on rev�ewers recommendat�ons: a random�sed tr�al E��T M�� J 1999
VAN�OOY��S�1999)52:�25 van �ooyen, S �evelopment of the �ev�ew Qual�ty �nstrument ��Q�) for assess�ng peer rev�ews of manuscr�pts J CL�N �P���M�OL 1999
SM�TH���1999)318:4 Sm�th, � Open�ng up EMJ peer rev�ew - A beg�nn�ng that should lead to complete transparency E��T M�� J 1999
GOL�E�CK�S�1999)318:44 Goldbec�-Wood, S �v�dence on peer rev�ew - sc�ent�f�c qual�ty control or smo�escreen? E��T M�� J 1999
WALSH���2000)17�:47 Walsh, � Open peer rev�ew: a random�sed controlled tr�al E��T J PSYCH�AT 2000
�OTHW�LL�P�2000)123:19�4 �othwell, PM �eproduc�b�l�ty of peer rev�ew �n cl�n�cal neurosc�ence - �s agreement between rev�ewers any greater than would be expected by chance aloE�A�N 2000
�AY�J�2000)109:131 �ay, J The fate of manuscr�pts rejected by a general med�cal journal AM J M�� 2000
WAG�����2001)14:257 Wager, � Shortcom�ngs of peer rev�ew �n b�omed�cal journals L�A�N PUEL 2001
HO��OE�N���2001)22:51 Horrob�n, �F Someth�ng rotten at the core of sc�ence? T��N�S PHA�MACOL S 2001
VAN�OOY��S�2001)14:85 van �ooyen, S The evaluat�on of peer-rev�ew qual�ty L�A�N PUEL 2001
J�FF��SO�T�2002)287:2784 Jefferson, T �ffects of ed�tor�al peer rev�ew - A systemat�c rev�ew JAMA-J AMM�� ASSOC 2002
GO�L���F�2002)287:27�2 Godlee, F Ma��ng rev�ewers v�s�ble - Openness, accountab�l�ty, and cred�t JAMA-J AMM�� ASSOC 2002
��NN�����2002)287:2759 �enn�e, � Fourth �nternat�onal Congress on Peer �ev�ew �n E�omed�cal Publ�cat�on JAMA-J AMM�� ASSOC 2002
�OWLAN��F�2002)15:247 �owland, F The peer-rev�ew process L�A�N PUEL 2002
OPTHOF�T�2002)5�:339 Opthof, T The s�gn�f�cance of the peer rev�ew process aga�nst the bac�ground of b�as: pr�or�ty rat�ngs of rev�ewers and ed�tors and the pred�ct�on of c�CA���OVASC ��S 2002
T�L��N�V�2002)51:275 T�lden, V Peer rev�ew: �v�dence-based or sacred cow? NU�S ��S 2002
J�FF��SO�T�2002)287:278� Jefferson, T Measur�ng the qual�ty of ed�tor�al peer rev�ew JAMA-J AMM�� ASSOC 2002
JAYAS�NG�U�2003)1��:279 Jayas�nghe, UW A mult�level cross-class�f�ed modell�ng approach to peer rev�ew of grant proposals: the effects of assessor and researcher attr�butes on asseJ �OY STAT SOC A STA 2003
HOJAT�M�2003)8:75 Hojat, M �mpart�al judgment by the "gate�eepers" of sc�ence: Fall�b�l�ty and accountab�l�ty �n the peer rev�ew process A�V H�ALTH SC� ��UC 2003
W�TS�K�2003)1�:249 Wets, K Post-publ�cat�on f�lter�ng and evaluat�on: Faculty of 1000 L�A�N PUEL 2003
SCH�OT���S�2004)328:�73 Schroter, S �ffects of tra�n�ng on qual�ty of peer rev�ew: random�sed controlled tr�al E��T M�� J 2004
EO�NMANN�L�2005)�3:297 Eornmann, L Select�on of research fellowsh�p rec�p�ents by comm�ttee peer rev�ew. �el�ab�l�ty, fa�rness and pred�ct�ve val�d�ty of Eoard of Trustees dec�s SC��NTOM�T��CS 2005
EO�NMANN�L�2005)13:29� Eornmann, L Cr�ter�a used by a peer rev�ew comm�ttee for select�on of research fellows - A boolean prob�t analys�s �NT J S�L�CT ASS�SS 2005
EO�NMANN�L�2005)14:15 Eornmann, L Comm�ttee peer rev�ew at an �nternat�onal research foundat�on: pred�ct�ve val�d�ty and fa�rness of select�on dec�s�ons on post-graduate fell��S �VALUAT 2005
�AN��L�H�2005)18:143 �an�el, H� Publ�cat�ons as a measure of sc�ent�f�c advancement and of sc�ent�sts product�v�ty L�A�N PUEL 2005
F���A�M�2005)37:87 Freda, MC An �nternat�onal survey of nurse ed�tors roles and pract�ces J NU�S SCHOLA�SH�P 2005
K�A�N�Y�M�2005)28:444 Kearney, MH Nurse ed�tors v�ews on the peer rev�ew process ��S NU�S H�ALTH 2005
EO�NMANN�L�200�)�8:427 Eornmann, L Select�ng sc�ent�f�c excellence through comm�ttee peer rev�ew - A c�tat�on analys�s of publ�cat�ons prev�ously publ�shed to approval or rejec SC��NTOM�T��CS 200�
JAYAS�NG�U�200�)�9:591 Jayas�nghe, UW A new reader tr�al approach to peer rev�ew �n fund�ng research grants: An Austral�an exper�ment SC��NTOM�T��CS 200�
SCH�OT���S�200�)295:314 Schroter, S ��fferences �n rev�ew qual�ty and recommendat�ons for publ�cat�on between peer rev�ewers suggested by authors or by ed�tors JAMA-J AMM�� ASSOC 200�
EO�NMANN�L�200�)15:209 Eornmann, L Potent�al sources of b�as �n research fellowsh�p assessments: effects of un�vers�ty prest�ge and f�eld of study ��S �VALUAT 200�
EO�NMANN�L�2007)1:204 Eornmann, L Convergent val�dat�on of peer rev�ew dec�s�ons us�ng the h �ndex - �xtent of and reasons for type � and type �� errors J �NFO�M�T� 2007
EO�NMANN�L�2007)1:83 Eornmann, L Gate�eepers of sc�ence - �ffects of external rev�ewers attr�butes on the assessments of fellowsh�p appl�cat�ons J �NFO�M�T� 2007
MA�SH�H�2007)42:33 Marsh, HW Peer rev�ew process: Assessments by appl�cant-nom�nated referees are b�ased, �nflated, unrel�able and �nval�d AUST PSYCHOL 2007
ALP��T�J�2007)120:287 Alpert, JS Peer rev�ew: The best of the blem�shed? AM J M�� 2007
MA�SH�H�2008)�3:1�0 Marsh, HW �mprov�ng the peer-rev�ew process for grant appl�cat�ons - �el�ab�l�ty, val�d�ty, b�as, and general�zab�l�ty AM PSYCHOL 2008
EO�NMANN�L�2008)47:7173 Eornmann, L The effect�veness of the peer rev�ew process: �nter-referee agreement and pred�ct�ve val�d�ty of manuscr�pt referee�ng at Angewandte chemANG�W CH�M �NT ���T 2008
EO�NMANN�L�2008)2:280 Eornmann, L How to detect �nd�cat�ons of potent�al sources of b�as �n peer rev�ew: A general�zed latent var�able model�ng approach exempl�f�ed by a genJ �NFO�M�T� 2008
EO�NMANN�L�2008)59:1841 Eornmann, L Select�ng manuscr�pts for a h�gh-�mpact journal through peer rev�ew: A c�tat�on analys�s of commun�cat�ons that were accepted by AngewaJ AM SOC �NF SC� T�C 2008
G�ESON�M�2008)112:�4� G�bson, M Author percept�on of peer rev�ew OEST�T GYN�COL 2008
EO�NMANN�L�2008)59:830 Eornmann, L Are there better �nd�ces for evaluat�on purposes than the h �ndex? a compar�son of n�ne d�fferent var�ants of the h �ndex us�ng data from b J AM SOC �NF SC� T�C 2008
EO�NMANN�L�2008)�4:45 Eornmann, L What do c�tat�on counts measure? A rev�ew of stud�es on c�t�ng behav�or J �OC 2008
EO�NMANN�L�2008)2:217 Eornmann, L Latent Mar�ov model�ng appl�ed to grant peer rev�ew J �NFO�M�T� 2008
MA�SH�H�2009)79:1290 Marsh, HW Gender �ffects �n the Peer �ev�ews of Grant Proposals: A Comprehens�veMeta-Analys�s Compar�ng Trad�t�onal and Mult�level Approaches ��V ��UC ��S 2009
EO�NMANN�L�2009)22:117 Eornmann, L The luc� of the referee draw: the effect of exchang�ng rev�ews L�A�N PUEL 2009
OPTHOF�T�2009)17:145 Opthof, T The H�rsch-�ndex: a s�mple, new tool for the assessment of sc�ent�f�c output of �nd�v�dual sc�ent�sts The case of �utch professors �n cl�n�cal c N�TH H�A�T J 2009
SULS�J�2009)4:40 Suls, J The A�r We Ereathe: A Cr�t�cal Loo� at Pract�ces and Alternat�ves �n the Peer-�ev�ew Process P��SP�CT PSYCHOL SC� 2009
EO�NMANN�L�2009)81:407 Eornmann, L The �nfluence of the appl�cants gender on the model�ng of a peer rev�ew process by us�ng latent Mar�ov models SC��NTOM�T��CS 2009
EO�NMANN�L�2010)4:211 Eornmann, L A meta-evaluat�on of sc�ent�f�c research proposals: ��fferent ways of compar�ng rejected to awarded appl�cat�ons J �NFO�M�T� 2010
EO�NMANN�L�2010)32:5 Eornmann, L The manuscr�pt rev�ew�ng process: �mp�r�cal research on rev�ew requests, rev�ew sequences, and dec�s�on rules �n peer rev�ew L�E� �NFO�M SC� ��S 2010
EO�NMANN�L�2010)83:493 Eornmann, L A content analys�s of referees comments: how do comments on manuscr�pts rejected by a h�gh-�mpact journal and later publ�shed �n e�theSC��NTOM�T��CS 2010
SCHULT���2010)84:277 Schultz, �M Are three heads better than two? How the number of rev�ewers and ed�tor behav�or affect the reject�on rate SC��NTOM�T��CS 2010
EO�NMANN�L�2010)85:�81 Eornmann, L The val�d�ty of staff ed�tors �n�t�al evaluat�ons of manuscr�pts: a case study of Angewandte Chem�e �nternat�onal �d�t�on SC��NTOM�T��CS 2010
OLE��CHT�M�2010)19:293 Olbrecht, M Panel peer rev�ew of grant appl�cat�ons: what do we �now from research �n soc�al psychology on judgment and dec�s�on-ma��ng �n groups?��S �VALUAT 2010
EO�NMANN�L�2010)�4:72 Eornmann, L How Long �s the Peer �ev�ew Process for Journal Manuscr�pts? A Case Study on Angewandte Chem�e �nternat�onal �d�t�on CH�M�A 2010
C�C��S�2011)108:3157 Cec�, SJ Understand�ng current causes of womens underrepresentat�on �n sc�ence P NATL ACA� SC� USA 2011
EO�NMANN�L�2011)45:199 Eornmann, L Sc�ent�f�c Peer �ev�ew ANNU ��V �NFO�M SC� 2011
EO�NMANN�L�2011)174:857 Eornmann, L A mult�level modell�ng approach to �nvest�gat�ng the pred�ct�ve val�d�ty of ed�tor�al dec�s�ons: do the ed�tors of a h�gh prof�le journal select mJ � STAT SOC A STAT 2011
F�ANC�SC�M�2011)5:275 Franceschet, M The f�rst �tal�an research assessment exerc�se: A b�bl�ometr�c perspect�ve J �NFO�M�T� 2011
E�N�A�W�2011)27:1�� Eenda, WGG The pred�ct�ve val�d�ty of peer rev�ew: A select�ve rev�ew of the judgmental forecast�ng qual�t�es of peers, and �mpl�cat�ons for �nnovat�on �n�NT J FO��CAST�NG 2011
OPTHOF�T�2011)88:1011 Opthof, T A comment to the paper by Waltman et al., Sc�entometr�cs, 87, 4�7-481, 2011 SC��NTOM�T��CS 2011
AE�AMO�G�2011)89:929 Abramo, G Nat�onal research assessment exerc�ses: a compar�son of peer rev�ew and b�bl�ometr�cs ran��ngs SC��NTOM�T��CS 2011
WALTMAN�L�2011)88:1017 Waltman, L On the correlat�on between b�bl�ometr�c �nd�cators and peer rev�ew: reply to Opthof and Leydesdorff SC��NTOM�T��CS 2011
S�N�C�2012)1�:293 Sen, CK �ebound Peer �ev�ew: A V�able �ecourse for Aggr�eved Authors? ANT�OX�� ���OX S�GN 2012
SQUAON�F�2012)�:2�5 Squazzon�, F Sa�nt Matthew str��es aga�n: An agent-based model of peer rev�ew and the sc�ent�f�c commun�ty structure J �NFO�M�T� 2012
VANA��NS�P�2012)25:381 van Arensbergen, P The Select�on of Sc�ent�f�c Talent �n the Allocat�on of �esearch Grants H�GH ��UC POL�CY 2012
L���C�2013)�4:2 Lee, CJ E�as �n peer rev�ew J AM SOC �NF SC� T�C 2013
EO�NMANN�L�2013)7:28� Eornmann, L The val�dat�on of �advanced) b�bl�ometr�c �nd�cators through peer assessments: A comparat�ve study us�ng data from �nC�tes and F1000 J �NFO�M�T� 2013
SQUAON�F�2013)42:287 Squazzon�, F �oes �ncent�ve prov�s�on �ncrease the qual�ty of peer rev�ew? An exper�mental study ��S POL�CY 2013
MOHAMMA����2013)97:383 Mohammad�, � Assess�ng non-standard art�cle �mpact us�ng F1000 labels SC��NTOM�T��CS 2013
V��L�YS��F�2013)�4:428 Verleysen, FT A label for peer-rev�ewed boo�s J AM SOC �NF SC� T�C 2013
WALTMAN�L�2014)�5:433 Waltman, L F1000 �ecommendat�ons as a Potent�al New �ata Source for �esearch �valuat�on: A Compar�son W�th C�tat�ons J ASSOC �NF SC� T�CH 2014
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Island2wmax = 7.715 · 10−5
STOLEN_T{2005}35:501
BANGSBO_J{1991}16:110
EKBLOM_B{1986}3:50
KRUSTRUP_P{2002}20:861
KRUSTRUP_P{2001}19:881
REILLY_T{1997}15:257
SALTIN_B{1973}5:137
VANGOOL_D{1988}:51
WITHERS_R{1982}8:159
MOHR_M{2003}21:519
COURNEYA_K{1992}14:13
NEVILL_A{1997}15:437
NEVILL_A{1996}14:181
NEVILL_A{1999}28:221
CATTEEUW_P{2009}27:1125
HELSEN_W{2004}22:179
NEVILL_A{2002}3:261
PLESSNER_H{2009}174:151
PLESSNER_H{2006}7:555
SCHWEIZE_G{2011}23:429
BIZZINI_M{2009}19:95
BIZZINI_M{2009}43:936
BIZZINI_M{2009}43:490
CASTAGNA_C{2007}37:625
WESTON_M{2012}42:615
SUTTER_M{2004}25:461
BALMER_N{2005}23:409
CATTEEUW_P{2009}31:786
DOSSEVIL_F{2011}25:67
HANCOCK_D{2013}14:66
D’OTTAVI_S{2001}41:27
CASTAGNA_C{2004}18:486
MALLO_J{2012}30:1437
OUDEJANS_R{2005}36:3
WESTON_M{2007}10:390
PIZZERA_A{2012}24:59
VANQUAQU_N{2010}32:3
CATTEEUW_P{2010}32:845
CATTEEUW_P{2010}28:471
CATTEEUW_P{2010}28:1027
CATTEEUW_P{2010}32:828
GILIS_B{2009}27:551
MASCAREN_D{2005}19:131
MASCAREN_D{2005}19:364
MALLO_J{2010}13:129
ANSHEL_M{1995}7:11
COULOMB-_G{2005}52:547
SOUCHON_N{2004}51:445
REILLY_T{2006}24:795
CASAJUS_J{2007}10:382
CASTAGNA_C{2005}19:785
WESTON_M{2011}32:190
WESTON_M{2010}13:96
BRAND_R{2006}28:93
CASTAGNA_C{2005}19:805
GILIS_B{2008}14:21
HELSEN_W{2006}24:521
MALLO_J{2007}25:805
WESTON_M{2006}9:256
TIRYAKI_M{2005}100:51
KAISSIDI_A{1997}15:427
JONES_M{2002}20:991
UNKELBAC_C{2008}30:95
MASCAREN_D{2006}37:99
STEWART_M{1998}87:1275
D’OTTAVI_S{2001}15:167
BARBERO-_J{2012}26:1383
CASTAGNA_C{2003}17:388
MALLO_J{2009}27:9
CASTAGNA_C{2002}16:623
CASTAGNA_C{2002}16:231
WESTON_M{2009}27:1177
MASCAREN_D{2002}24:328
MALLO_J{2008}22:235
REILLY_T{1984}58:149
CASTAGNA_C{2001}15:420
MALLO_J{2009}30:331
HELSEN_W{2007}25:991
CATTERAL_C{1993}27:153
ASAMI_T{1988}:341
MEMMERT_D{2008}15:1
BANGSBO_J{1994}:
REILLY_T{1986}29:601
CASTAGNA_C{1999}:67
REILLY_T{1996}:65
OHASHI_J{1988}:329
AGNEVIK_G{1970}:
REILLY_T{1986}:114
THOMAS_V{1976}19:530 Pajek
Island2 – Refereeing in sport.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Island3 and Island4wmax = 1.415 · 10−8 and wmax = 1.132 · 10−8
AUDI_G{2003}729:3
ALBURGER_D{1978}18:1875
BATCHELD_J{1998}57:R1042
BATCHELD_J{1993}47:2038
BERNAS_M{1997}415:111
BLANK_B{1995}74:4611
BLANK_B{1996}77:2893
BLANK_B{1994}50:2398
BORREL_V{1992}344:135
DAVIDS_C{1998}80:1849
FIRESTON_R{1996}:
GUILLEMA_D{1989}332:189
HENCHECK_M{1994}50:2219
HOFMANN_S{1981}299:281
IUPAC_T{1991}63:879
JANAS_Z{1999}82:295
KASHY_E{1976}14:1773
LIVINGOO_J{1940}12:0030
LUKYANOV_S{2002}28:L41
MAHMUD_H{2002}15:85
MOHAR_M{1991}66:1571
MORRIS_C{1982}25:3218
NOTANI_M{2002}542:49
POUGHEON_F{1987}327:17
ROBERTSO_R{1974}32:1207
RYKACZEW_K{1995}52:R2310
SAKURAI_H{1996}54:R2802
SCHARDT_D{1981}368:153
SOULIOTI_G{2000}T88:153
SYMONS_T{1979}42:40
WESTFALL_G{1979}43:1859
XU_S{1999}60:061302
YENNELLO_S{1992}46:2620
THOENNES_M{2004}67:1187
GINEPRO_J{2009}95:805
GORIELY_S{2007}75:064312
BUTEMENT_F{1950}165:149
CORYELL_C{1951}9:
GINEPRO_G{2009}95:805
JOHN_W{1970}2:1451
NITSCHKE_J{1984}316:249
POOL_M{1938}53:437
RASMUSSE_J{1953}89:33
THE_P{1946}68:2411
WILKINSO_G{1948}74:1733
AMALDI_E{1935}149:0522
FERMI_E{1934}146:0483
MCLENNAN_J{1935}135:147
SCHNEIDE_J{1983}312:21
BUTEMENT_F{1951}64:395
THE_P{1946}18:513
WILHELMY_J{1970}25:1122
ASTON_F{1920}105:8
BERNAS_M{1994}331:19
CREUTZ_E{1940}58:481
GLASOE_G{1940}58:1
HANSEN_P{1969}B 28:415
WAHL_A{1962}126:1112
BENLLIUR_J{1999}660:87
HOFMANN_S{1979}291:53
MCMILLAN_E{1937}52:0375
POOL_M{1937}52:0239
SMITH_W{1955}98:1258
HEYN_F{1939}143:516
STOVER_B{1951}81:8
WATSON_R{1970}141A:6
[ANONYMO{0}:ENSDF_EV_NUCL_STRUCT
HOPKINS_H{1950}77:717
HOPKINS_H{1948}73:1406
MORINAGA_H{1960}15:213
HAHN_O{1943}31:249
HAHN_O{1943}121:729
SNELL_A{1937}52:1007
ASTON_F{1925}49:1191
GRUNDER_H{1977}27:353
PLUTONIU_P{1946}18:513
HAHN_O{1940}28:543
ALBOUY_G{1960}21:751
FRITSCH_A{2010}96:315
GINEPRO_J{2009}95:905
HEIM_M{2010}96:333
SCHUH_A{2010}96:307
SHORE_A{2010}96:299
Pajek
CLARK_P{1997}37:441
STRAUSS_A{1990}:
THOMAS_L{1995}80:6
THOMPSON_C{1999}54:392
GLASER_B{1967}:
PATTON_M{1990}:
LINCOLN_Y{1985}:
CRESWELL_J{1998}:
MAXWELL_J{1996}:
STRAUSS_A{1998}2ND:
WILENSKY_H{1964}70:137
BECKER_H{1961}:
BALOGUN_J{2002}31:131
HENDRIX_A{2000}35:139
MALASARN_R{2002}37:55
PITNEY_W{2002}37:63
PITNEY_W{2006}41:189
ERLANDSO_D{1993}:
MERRIAM_S{1988}:
STROOT_S{1993}12:337
TIERNEY_W{1996}:
BABIN_B{1996}72:57
BOLES_J{2001}13:376
CAPEL_S{1990}25:34
DENZIN_N{2000}:
DIXON_M{2007}21:377
DIXON_M{2007}78:236
DIXON_M{2005}8:227
MAZEROLL_S{2008}43:513
MAZEROLL_S{2008}43:505
MAZEROLL_S{2006}11:47
MEHO_L{2006}57:1284
MILAZZO_S{2006}41:S73
PASTORE_D{1996}20:427
PASTORE_D{1991}5:128
PITNEY_W{2010}45:198
PITNEY_W{2002}37:286
PITNEY_W{2009}:42
RICE_L{2001}36:S73
MAZEROLL_S{2011}46:194
HAYES_L{2006}43:237
PITNEY_W{2002}37:S168
GOODMAN_A{2010}45:287
MAZEROLL_S{2012}47:679
MAZEROLL_S{2012}47:694
MAZEROLL_S{2013}48:668
Pajek
Island3 – isotopes / nuclear physics;Island4 – athletic training.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1
Peer Reviewfrom WoS / 1
V. Batagelj,A. Ferligoj
Data
Distributions
Citations
SPC – Island5 and Island6wmax = 1.516 · 10−9 and wmax = 1.112 · 10−9
AWLIYA_W{1998}56:9
DELLA_B{2000}2:305
HORN_H{1983}27:671
KERN_M{1998}14:64
MALAMENT_K{1999}81:23
MCLAREN_E{1998}19:307
MCLEAN_J{1965}119:251
PEUMANS_M{2000}28:163
SEGHI_R{1995}8:239
TINSCHER_J{2001}14:231
ZENG_K{1998}11:183
ZENG_K{1996}9:434
BLATZ_M{2003}89:268
CHONG_K{2002}15:183
ESQUIVEL_J{2004}17:469
GUAZZA_M{2004}25:5045
HEFFERNA_M{2002}88:10
KOSMAC_T{1999}15:426
LUTHARDT_R{2004}20:655
MILLER_L{1977}21:699
ODEN_A{1998}80:450
OH_W{2002}81:623
OLSSON_K{2003}16:150
PICONI_C{1999}20:1
RAIGRODS_A{2004}48:531
RAIGRODS_A{2004}92:557
RAIGRODS_A{2001}86:520
SEGHI_R{1995}74:145
SORENSEN_J{1998}26:207
SUAREZ_M{2004}17:35
VONSTEYE_P{2001}14:379
WALTON_T{2003}16:177
CHRISTEL_P{1989}23:45
COVACCI_V{1999}20:371
HEYDECKE_G{2002}87:438
ICHIKAWA_Y{1992}68:322
KELLY_J{1995}74:1253
SCARANO_A{2003}29:8
SULAIMAN_F{1997}10:478Pajek
ALFARANO_C{2005}33:D418
HERMJAKO_H{2004}32:D452
SALWINSK_L{2004}32:D449
ZANZONI_A{2002}513:135
CHATR-AR_A{2007}35:D572
ASHBURNE_M{2000}25:25
STEIN_L{2002}12:1599
PETERSON_J{2001}29:123
TATUSOV_R{2000}28:33
EISEN_M{1998}95:14863
MULLER_H{2004}2:1984
BOECKMAN_B{2003}31:365
OGATA_H{1999}27:29
BECKER_K{2006}296:179
BRAY_T{1998}:
XIANG_Z{2006}7:347
XIANG_Z{2007}8:r150
XIANG_Z{2008}36:D923
CHAIN_P{2005}73:8353
DELVECCH_V{2002}99:443
HALLING_S{2005}187:2715
GARRETT_J{2005}: Pajek
Island5 – dentures;Island6 – genome.
V. Batagelj, A. Ferligoj Peer Review from WoS / 1