New Tools for Measuring the UniversityIn the Global Science Network

Turning Data into Insight

2

What Constitutes a “U.S. Research University”?

Growth Indicator (15 years)

1 = world average3.8 to 14.4 (11)1.2 to 3.8 (47)0.7 to 1.2 (92)0.3 to 0.7 (27

0 to 0.3 (7)too few papers (67)

Science is becoming more:

InternationalGlobal

Collaborative

Networked

Interdisciplinary

International collaborations in science have more than doubled since 1990…

3

Counting Within “Boxes” is Limited

Shift in boundaries (nations, disciplines of science) make science difficult to measure

Rankings are widely used but simplistic Inputs (R&D spending) are only loosely correlated to outputs Surveys are expensive and difficult to reproduce Knowledge creation, absorption, diffusion, retention are dynamic and

ever-changing Indicators are needed that reflect the movement of knowledge, people,

and information …The data are the same but we have found new theoretically-based

uses for them…

4

Our Value Add – Quality and Insight

comparative analyses of performance and capability international, national, faculty and departmental level positional

analyses impact and uptake analyses collaborative trends analyses strengths, weaknesses, and opportunities analyses

We measure performance - trends, impact, and specialization

Our analyses are based on the most up-to-date information available from world-class databases.

5

We Measure Performance

Trends

Specialization

Impact

Growth Index (GI): a measure of the increase in output for an area relative to thedatabase overall.

Average Relative Citation (ARC): an indicator of observed impact of a given entitycalculated from citation counts.

Average Relative Impact Factor (ARIF): an indicator of expected impact basedon the journals in which papers are published.

Specialization Index (SI): a measure of the research intensity of an entity in a given area relative to a reference entity in the same area.

6

Performance Rankings Top 25 Institutions in California (by # papers) in the Field of Optics

rankings can be done based on a variety of indices:

- output- ARC/ARIF- specialization- growth indices

multi-criteria rankings can also be assessed

Institution Papers ARCCalifornia Institute of Technology 1,809 1.8NASA 1,422 1.2Stanford University 1,098 2.3Lawrence Livermore Natl. Laboratory 950 1.9UC Berkeley 885 1.6UC San Diego 772 1.4University of Southern California 763 1.3Los Alamos National Laboratory 674 1.4UC Irvine 638 1.9UC Los Angeles 634 1.5UC Santa Barbara 522 2.0Lawrence Berkeley Natl. Laboratory 376 1.8UC Davis 368 1.9US Navy 288 0.9UC Santa Cruz 236 2.0Intel Corporation 209 2.9Sandia National Laboratories 176 2.6Raytheon Corporation 168 0.8Lockheed Martin Corporation 167 0.6IBM 167 2.7Aerospace Corporation 156 0.7UC San Francisco 155 0.9KLA-Tencor 151 0.9Hewlett-Packard 150 0.9San Diego State University 118 1.8

7

Trends - Output Relative to Size

Entity Population(2003 Estimates)

GERD(billion$ &

PPP)(2002)

Papers Papers/103 inhab.

Papers/Billion $1 ARIF ARC

California 35,484,453 63.2 206,480 5.8 653 1.32 1.61

Washington 6,131,445 12.9 39,290 6.4 608 1.27 1.60

Canada 31,676,100 23.5 161,715 5.1 1,374 1.08 1.17

World 6,297,409,158 1,168.5 3,483,166 0.6 596 1 1

1Papers from 2001-2005 divided by 2002 GERD in Canadian dollars at purchasing power parities

For the examples below, the overall number of papers per inhabitant are relatively similar

The number of papers produced per billion $ is much lower in California & Washington

In return for the higher ‘per paper cost’, a higher impact is observed (measured by ARC and ARIF)

8

Strength of leading Canadian institutions in forestry, 1991–2006 (only the top 20 institutions in absolute output presented)

Institution Papers Publication Trend Impact (ARC) Impact TrendNRCan-CFS 2,321 +N+ N+ N+ N+ + NN+ NN+ NN+ NN+UBC 1,562 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of Alberta 1,038 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Université Laval 773 +N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of Toronto 552 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+BC Ministry of Forests 529 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Environment Canada 415 +N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of Victoria 381 + N+N+ N+ N+ + NN+ NN+ NN+ NN+Simon Fraser University 374 + N+ N+ N+ N+ + NN+NN+ NN+ NN+University of Guelph 353 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of New Brunswick 344 +N+ N+ N+ N+ + NN+ NN+ NN+ NN+AAFC 327 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+McGill University 320 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of Calgary 309 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Ontario Ministry of Nat Res 296 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Ctr for Forest Res (CFR/CEF) 243 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+University of Saskatchewan 239 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Lakehead University 201 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Université de Montréal 198 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+Minist des Res Nat et Faune (QC) 170 + N+ N+ N+ N+ + NN+ NN+ NN+ NN+

9

Network of Co-authorships in Cosmology/Gravity Studies (Sized by Impact, 2000-2005)

10

Indicators can be Created from Networks

Network index 1990 network 2000 network 2005 network Number of nodes 172 192 194 Number of links 1 926 3 537 9400 Size of core component 37 54 66 Network density 0.1310 0.1929 0.2511 Average degree 22.442 36.896 48.649 Average distance 1.954 1.851 1.76 Diameter 3 3 3 Graph betweenness 0.2589 0.1617 0.144 Average clustering coefficient 0.784 0.787 0.789

The Global Network of Science over 15 Years, based onall scientific publications in the Web of Science

11

Impact & Specialization - International Level

shows the size of an entity’s research output in a given area

shows the level of specialization of an entity in a given area

shows the impact of an entity in a given area

shows an entity’s relative position vis-à-vis other countries

12

Impact & Specialization – A Research Institution in a National Context

13

Impact & Specialization – Institutional Level(for Management and Business Finance)

assess the performance of specific faculties, departments or subfields

identify areas of institutional strength and weakness

evaluate opportunities for strategic development

14

Mapping Knowledge Networks(citations within the field of quantum materials)

citation networks map the links between researchers

networks can show various types of clusters

- subject matter- geographical

networks help characterize collaboration patterns

15

Let Us Sharpen Your Profile

Science-Metrix Corporation

Caroline S. Wagner, Ph.D.404 Russell RoadAlexandria VA 22301703-836-4569

Caroline.wagner@science-Metrix.com