Upload
lonna
View
37
Download
0
Embed Size (px)
DESCRIPTION
World University Rankings Symposium Identifying Excellence and Diversity in International Education: Rankings and Beyond École Normale Supérieure (ENS), Paris. May 18 th 2011. Why web-based rankings of universities should not be neglected Isidro F. Aguillo. The Cybermetrics Lab. - PowerPoint PPT Presentation
Citation preview
Why web-based rankings of universities should not be neglected
Isidro F. Aguillo
World University Rankings SymposiumIdentifying Excellence and Diversity in International Education: Rankings and Beyond École Normale Supérieure (ENS), Paris. May 18th 2011
2
A rigorous scientific approach to the ranking of universities
Scholars making scientific research
– Researchers belonging to the National Research Council (CSIC)
• The largest Spanish research public organization
– Recognised by our peers
• 15 years experience in quantitative analysis and evaluation of scholar communication and academic institutions
• Papers in referred scientific journals, contributions to international conferences, reports to governmental bodies
– Funded by public resources
• International cooperation projects funded by European Commission
Research Agenda
– Promote Open Access initiatives
– Global coverage, including universities from developing countries
– Building Cybermetrics/Webometrics as an emerging discipline
The Cybermetrics Lab
3
A new powerful tool with a significant impact on Higher Education
The problems of the Higher Education Science in the ‘90s
– Limited impact, not suitable for students/scholars
• “Experts” driven
• Complex reports, difficult to understand
– Lack of global scenarios
• Focus on “US research university” model
International Rankings (ARWU, 2003)
– Huge impact, including governments
• Data driven
• Easy to understand, ONE composite indicator (rank)
– World and continental scenarios
• Excellence clearly defined
• Useful for both individual universities and national systems
Why Global Rankings of Universities?
ACTIVITY IMPACT
How? A couple of models
4
ACTIVITY IMPACT
ARWU (Shanghai) 40% 60%
RANKING
QS 30% 70%
TIMES HE (2010) 35% 65%
HEEACT (Taiwan) 20% 80%
WR (Webometrics) 50% 50%
Customizing the modelMost of the Global Rankings use an arbitrary weighting system, without an underlying model
5
Feasibility
6
It is not so easy, sometimes there is no data, or they are not trusted or they are not useful
7
Quality can be measured by consensus or majority of experts evaluating the targeted topic
Surveyo Topic: Perceived (subjective) academic reputation
o Expertise: Peers with global knowledge (not available)
o Size: Medium (several thousands)
Citation analysiso Topic: Empirical (objective) scientific impact (formal communication)
o Expertise: Peers
o Size: Small (a few hundreds)
Link analysiso Topic: Quality of academic and other web contents
o Expertise: Web-editors (peers and non-peers)
o Size: Huge (millions)
Quality
8
Surprisingly most of the Rankings are being accepted without a critical review of the authorship, the methodology and the results
A critical review
Ranking Missions Indicators Model Results Comments
ARWU Research Scientific excellence
No model (Top 200) Correlated
World class universities(incomplete, mistakes, old data)
WEBOMETRICS All Activity & Impact (Web)
Impact factor based
Correlated (relevant exceptions)
World universities(extreme penalties for bad web policies, search engines biases)
HEEACT Research Activity & Impact (Papers, Citations)
Striking not explained model
Correlated World research universities (variables not independent)
LEIDEN Research Impact (Citations/Papers)
Crown indicator
Several “rankings”
World research universities (questioned methodology)
SCIMAGO Research Activity OR Impact (Papers, Citations)
Ranked by size (!) as default
Several “rankings”
World research institutions (no composite ranking)
THE All Activities, Prestige (Surveys) & Impact (Citations)
No model Not valid Methodology applied incorrectly, unethical (non-cooperating universities are not identified)
CHEPS All Multi-Rank Customizable by end-users!
No Ranking A collection of colorful smileys
Power-law Distribution
9
SCORE
RANK
WR
QS
CWTS
ARWUHEEACT
log-norm
z-score
•Published since 2004, two editions per year (January & July)
•The largest and most updated Directory with more than 20,000 Higher Education Institutions from all over the World
•The Ranking provides the Top 12,000 universities according to web indicators
•The hypothesis is that in the 21st century the web reflects the organization, activities, research results, knowledge transfer, prestige, and international visibility of the universities
•If the web performance is below the expected position could be due to lack of commitment to the electronic publication, or bad web practices
Webometrics RankingThe Ranking Web is one of the first World Rankings to be published and currently it is the one with the largest coverage
10
3484336
15435
5943
6176148
339259
3975
5069414
19266(Top 1000)
WRJan 2011
Geographical coverage
11
12
Digital Divide
67115
5069(Top 500)
WRJan 2011
Europe
13
58148
43319411
66330
0
23648
23378
1116
16114
16213
20348
11114
10710
10010
958
757
779
676
6410
601
5712
5016
508
519
480
483
632
352
351
31
338
334
322
151
91
370
260
14
Universities in France
15
Different views are correlated but the position of French universities, also in the Web ranking, is below expectations
Coverageo Rankings based on research performance where the English is the “lingua
franca” of scholar communication are penalizing French language
o Webometrics takes into consideration also other university missions including traditional and off campus (distance) teaching in French or transfer of knowledge and technology, also in local language
University roleso Research-based rankings describe “World-class” universities
o Web ranking also identify “nation-building” institutions, probably more important to the local communities (social commitment) and governments (economic and political involvement)
Bad practiceso Surprisingly for the 21st century, there are academic institutions (its leaders or
scholars) that are ignoring the Web: No enough contents
o Web policies are inadequate, incorrect or insufficient. Too many institutions, frequent web domain changes, too few repositories
Comparative Analysis
16
French and Norwegian (and others) Higher Education Ministries visited Shanghai for advice!!. Suggestions provided here are tentative:
Rankings should be useful
ARWUoriented
Choose 2-3 universities as World-class candidates
Increase their budget at the expense of the other universities
Give legal capability for offering better salaries and grants
Attract international & national best candidates to these universities
Reinforce research capabilities, transferring elite institutes and hospitals
Fire low performers & close not very productive or international disciplines (Humanities!)
Webometricsoriented
All the universities should reinforce their web presence
Budget should be available to web publication for …
* Reflecting the institution’s performance
* Attracting international students & scholars
* Disseminating the research results
* Giving more visibility to “neglected” disciplines (Humanities!)
* Improve the transfer of technology to industry and other companies
* Increase the knowledge transfer and the community engagement
17
Web should be the showcase of the University, the place for publishing its basic info & facts, organisation, activities, policies, results …
Contentso First mission: Traditional and off campus (distance) teaching supporting
pages
o Second mission: Research groups and (scientists) personal pages, Research projects websites
o Third mission: Hosting third parties sites
Quality Contentso Portal of journals / Repository of papers
o Media (videos, webinars, objects) and software portals
o Data repository / (Web) Archive
o Social commitments
International Contentso Multilingual (also English) versions
o Guides to foreign students/scholars
o Super-sites
Web strategies
Learning management systems (LMS)
MIT Open CourseWare
19
Personal Pages
ENS (?) Repository
21
http://hal-ens.archives-ouvertes.fr/
Why not?http://archiveouverte.ens.fr/
Thesis at USP
22
Web 2.0: Priorities
Rank Facebook YouTube Social Bookmarking
External Blogs Institutional Blogs
1 Campus Life Events Courses Faculty Campus Life
2 Sports Campus Life Projects, Non-Research
Research, Physical Sciences Events
3 Technology Faculty Research, Physical Sciences Institution Overall Institution Overall
4 Product Services Courses Events Expert Commentary Institution Sub-Groups
5 Events Institution Overall Faculty Events Admissions
24
Why the Web?o Web presence is an indicator. Web is reflecting overall performance and not
only online activities
o Web is already the most important scholarly communication tool
o Web increases significantly the visibility and impact of the activities and the results of scholars and researchers
o Web is the best channel for online and distance learning, the best showcase for attracting international talent (foreign students and professors), the meeting point with the society, the economic and industrial stakeholders
Why the Ranking Web?o Webometrics is offering the larger coverage, ranking World-class universities
but also institutions from emerging and developing countries
o Discrepancies on the web ranking informs about problems with the governance and long-term strategies of the universities, obsolete or inadequate policies and bad web practices
o US universities leadership in Webometrics is showing a concerning and a long term threat to the scientific and cultural presence of the Europeans (and their languages) in the Web
Summarizing
25
Questions? …Thank you!
Contact:Isidro F. Aguillo, HcPhDThe Cybermetrics Lab. [email protected]
Open Forum