Achieving efficiency and effectiveness of the R&I...Source: SciVal 2011-2016 . REPUBLIC OF BULGARIA Ministry of Education and Science ... introducing international quality standards

Achieving efficiency and

effectiveness of the R&I

system

REPUBLIC OF BULGARIA

Ministry of Education and Science

PSS Kick-off meeting, Brussels, February 2017



Europe 2020 and

Innovation Union

More investments in

R&I – 3% of GDP European Research Area

• Effective national research

systems

• Open labor market for

researchers

• Optimal transnational cooperation

• Open access, circulation and

transfer of knowledge

• Gender equality and

mainstreaming

• Research Infrastructure

European scene



2

With 0.96% of GDP in R&D (Eurostat 2015), Bulgaria

is still behind the EU average

Source: DG Research and Innovation, EC (2017)





4

BULGARIA: EU/National Funding Instruments

5-th Framework (1998-2002) 6-th Framework 2002-2006 7-th Framework 2007-2013

20 mln. Euro 45 mln. Euro 105 mln. Euro

NATIONAL FUNDING

INSTRUMENTS

AVAILABLE FUNDING (in Euro)

National Innovation Fund (2005-2016) 50 mln.

National Science Fund (2005-2016) 168 mln.

NIF 2016 3.2 mln.

NSF 2016 9.8 mln.

NSF 2012 7.6 mln.

NSF 2010 0

NSF 2009 44 mln.

Source: National Statistical Institute (2016)



The Bulgarian Academy of Sciences (46 Institutes, the Agricultural Academy (35 centers) and key Bulgarian universities - Sofia University “St. Kl. Ohridski”, Medical Universities in Sofia, Plovdiv, Varna and Pleven, Plovdiv University, Technical Universities represent the major public research performing institutions in Bulgaria. The private performers are private universities, private research organizations, and enterprises involved in R&D.

2 Research Organisations, 51 HEIs and privet centers



6

Source: SciVal, 2011-2016

Top 15 best performing research organization in

Bulgaria, 2011-2016



7

R&D Researchers in Bulgaria (FTE)

Businesses State Academia Non-Commercial Organizations

Source: National Statistical Institute (2016)



8

Dynamics of Publication Activity in Bulgaria

Source: InCites, Web of Science (2016)



9

Distribution of science areas as per indicators

for quality

Source: SciVal 2011-2016



2003 –ORDINANCE No 9 OF 8.08.2003 ON THE CONDITIONS AND PROCEDURE FOR PLANNING, DISTRIBUTION AND

EXPENDITURE OF FUNDS EARMARKED BY THE STATE BUDGET FOR SCIENTIFIC OR CREATIVE AND ARTISTIC ACTIVITY

INTRINSIC TO STATE HIGHER EDUCATION INSTITUTIONS (Since 2014 differentiated distribution, based on the assessment for the

previous two years period)

1995 – National and Evaluation and Accreditation Agency is founded. With the enforcement of the amendments

of the Higher Education Act on June 4, 2004 some changes occurred in the evaluation and accreditation

procedures of higher education institutions, imposing on NEAA the functions of a specialized governmental body

for carrying out evaluation, accreditation and quality control in the activities of higher education institutions and

carrying out post-accreditation monitoring and control – a new task in the activity of NEAA.



2010 – Bulgarian University Ranking System has been developed to support education service users in their

choice of a university http://rsvu.mon.bg/rsvu3/?locale=en Considering that users’ individual priorities and

interests will vary greatly, the system allows for producing rankings of different scope and type in each

professional field. Rankings are produced based on a set of indicators included in the system.

2005 – Online, free access, bilingual Current research information system www.cris.mon.bg

2015 – RULES of monitoring and evaluation of research undertaken by Universities and scientific organizations

as well as the activities of the “Science Fund”

http://rsvu.mon.bg/rsvu3/?locale=en

http://www.cris.mon.bg/



Better Science for a Better Bulgaria 2025 Vision for a research policy strategy in support of society and economy

- The commitment to raise public investment in research by 2020 and 2025.

- Public investment increase will obviously be implemented gradually. - Bulgaria should radically improve the organisation of its public investment in research:

boosting the quality, efficiency and effectiveness of its public research system. Last, but not least, Bulgaria’s engagement in building the European Research Area also relates to effective national research systems.

- Shift to performance-based funding by default



Shift to performance-based funding

Based on implemented assessment and control, the greater part

of institutional funding for each research organization will be

result-based, introduction of multi-annual plans and performance

contracts involving regular institutional evaluations using

objective international criteria related to performance indicators.

Budget plans will integrate monitoring and evaluation

mechanisms, publicity and traceability of inputs and justification

of expenditures based on analytical statistics and constant data

exchange.

WHAT IS

DONE?

National research

Infrastructure Roadmap is

on actualization

OP SESG

Revision of the R&P of

the National Science

Fud

ORDINANCE for conditions and order of assessment, planning, allocation and expenditure of funds from the state budget for financing the inherent of State Universities scientific or artistic activity

RULES of monitoring and evaluation of research undertaken by Universities and scientific organizations as well as the activities of the “Science Fund

1 2 3

The evaluation system provides conditions for:

improving the quality of research by introducing international quality standards applicable to research activities

improving the management of research activities

reporting of research results and the Fund activity to the scientific community, the relevant state bodies and other institutions which finance it, as well as to the public

THE MAIN OBJECTIVE OF THE SYSTEM IS TO

SUPPORT THE CREATION OF AN EFFECTIVE AND

EFFICIENT NATIONAL POLICY ON RESEARCH,

AIMED AT ACHIEVING HIGH COMPETITIVE

RESEARCH RESULTS AND DEVELOPMENT OF

THE BULGARIAN SCIENTIFIC ORGANIZATIONS

AS AN EQUAL PARTNERS OF LEADING

SCIENTIFIC ORGANIZATIONS IN THE EUROPEAN

UNION (EU) AND ON A GLOBAL SCALE

SPECIFIC OBJECTIVES OF THE SYSTEM ARE:

1. evaluation of research organization activities and analysing their positioning in the European and global research area;

2. identifying and supporting research activities which have proven potential of national significance and / or international recognition;

3. stimulating organizations to reach high, internationally recognized results of research activities, based on a system of objective, measurable and transparent evaluation criteria;

4. analysis of the effectiveness of investment in research for innovation and economic growth and planning of budget funds for organizations and the Fund;

5. ensuring publicity in the process of implementing the national policy on research and transparency of their funding.

Based on citation

analyses and the

impact of scientific

publications

The monitoring is a collection of objective data and establishes

the dynamics of scientific development of organizations and effectiveness of the

Fund.

The assessment is made pursuant for research activities carried out by organizations and Fund Activity

The system includes annual monitoring, evaluation and analysis of research carried out by the organizations and activities of the Fund

555,5 mln. BGN

Centers of excellence

Centers of competence

Combination of “hard”

and “soft” measures

Optimization of research

infrastructures

Applied research

programmes

Building high-quality

research potential

Coordination with

Horizon 2020

Widening participation

OP “SCIENCE AND EDUCATION FOR SMART GROWTH”

MAIN SOURCES AND TOOLS OF THE SYSTEM ARE

THE FOLLOWING:

• international scientific databases – SCOPUS, WEB of SCIENCE, Harzing’s Publish or Perish (Google Scholar);

• statistical databases maintained by the National Statistical Institute (NSI)

and Eurostat;

• Bulgarian Current Research Information System (BulCRIS); • databases maintained by the National Centre for Information and

Documentation, Patent Office of the Republic of Bulgaria and international patent databases;

• activity reports of the Fund.

Independent assessments and reports to the EU institutions and the Ministry of Education and Science can be additional sources of the system.

The system provides an opportunity for annual evaluation of the results of research organizations based on objective and measurable internationally recognized indicators in three basic criteria: • scientific results • scientific capacity • national and international recognition

Annually by the end of March, organizations provides digital information tо the Ministry of Education and Science for the previous calendar year for their research activities as follows:

• list of research staff on basic labour contract with the names under which they

publish, their scientific degrees and academic positions; • the average salary of research staff in the organization for the relevant reporting

period; • list of national and international projects which have received funds from

competitive-project financing over the reporting period and the amount of funds

received; • list of contracts with Bulgarian and foreign companies and / or organizations, which

have received funds during the reporting period and the amount of funds received; • list of people successfully defended doctoral dissertation during the reporting

period; • list of memberships in international editorial boards of journals, registered in

international databases • list of memberships of organizations and members of the research staff in

international scientific networks and / or scientific societies;

• list of printed textbooks, teaching aids, popular science publications of anticipated significance for society over the reporting period

1 2 3

CRITERIA AND INDICATORS FOR EVALUATION

OF RESEARCH CARRIED OUT BY

ORGANIZATIONS

SCIENTIFIC RESULTS IN THE REPORTING PERIOD

RESEARCH

CAPACITY IN

THE

REPORTING

PERIOD

NATIONAL AND INTERNATIONAL RECOGNITION DURING THE REPORTING PERIOD

The report is published on the website of the Ministry of Education and Science

On the basis of monitoring and evaluation, the Committee prepares an analysis of research work carried out by organizations and the effectiveness of funding.

Annually by the end of March, the National Science Fund provides digital information tо the Ministry of Education and Science for the previous calendar year as follows:

• list of conducted competitions for funding of scientific programs and projects

during the reporting period and tender documentation to them; • reports of temporary scientific expert committees on the results of

competitions conducted and ranked projects during the reporting period;

• decisions of the Executive Council to determine the projects for financing and the amount of their available means respectively;

• rules of the Executive Board for current financial control and accountability in the implementation of contracts for research funding;

• reference for the implementation and reporting of financed projects during the reporting period ;

• list of constructed during the reporting period scientific infrastructures as a result of the implementation of programs and projects financed by the Fund;

• list of research results (patents, publications, models, algorithms, programs,

methodologies, etc.). during the reporting period as a result of the implementation of projects financed by the Fund;

• list of research results used in innovative projects during the reporting period.

The criteria for evaluating the National Science Fund activity are:

• Fund contribution to achieving the objectives of the National

Strategy for Development Research, European Framework

Programmes, the European and National Roadmap for Research

Infrastructure;

• impact and effectiveness of implementing the annual operational

program of the Fund for research and innovation development in

Bulgaria;

• results of prioritizing the funding of programs and projects in

scientific fields

• efficiency of the implementation control and reporting of the

financed projects;

• socio-economic impact of the results of the financed projects;

• quality of work processes and practices, applicable to the Fund

during the evaluation period.

In accordance with the criteria and on the basis of the information the

Committee shall evaluate the activities of the Fund during the reporting

period. On the basis of monitoring and evaluation the Commission

prepares an analysis of the Fund activity.

Criteria for monitoring and accounting the achieved results by the

inherent of State Universities scientific and artistic activity

Approved internal institutional priorities for scientific

work:

- number of financed projects under the relevant

priorities of total amount.

Scientific results:

– list of the scientific publications, which are refered and indexed in world

secondary literary sources;

– number of scientific publications, published in editions with impact factor (Web

of Science) and impact rank (Sсopus);

– number of quotes in the accounting period of the research team of the state

universities on the basis of data from Web of Science and Scopus of scientific

publications of the previous two years;

– list of the registered patent applications, patents and patents, in a result of

concluded contracts with companies;

– number of articles in collections of scientific conferences, introduced in

Conference Proceedings in Thomson Reuters and/or SCOPUS;

– list of the published treatises;

– list of the performed spectacles;

– list of the performed concerts;

– list of the organized exhibitions;

– list of the filmed movies;

– list of other artistic manifestations (for example, broadcast radio- and/or TV-

programs, written scores, etc.) ;

– list of the artistic manifestations with international recognition, proven with a

relevant document (diploma., certificate, etc.)

• Number of the research team on basic employment contract in

a state university :

– number of the acquired educational and scientific degree

“doctor” during the respective year;

– number of attracted researchers from outside the structure

of the State University (from Bulgarian and foreign

Universities and scientific organizations).

• Detected problems during the implementation of the financed

projects and measures to overcome them.

• Measures to ensure publicity of the results.

• Annual financial report for the received and expended funds,

released purposely from the state budget for the inherent of

state universities scientific or artistic activity.

• In accordance with the received assessment

of the results of the artistic and scientific

work for the previous three years, the State

Universities shall receive a percentage of the

funds for the inherent scientific or artistic

activity, determined with the State Budget

Law for each year, as following:

a) in final assessment above 1,50 - 100 percent;

b) in final assessment from 0,40 to 1,5 - 90

percent;

c) in final assessment from 0,01 to 0,39 - 80

percent.

• The remainder of the funds of the state

budget for scientific or artistic activity,

released in result of the implemented

assessment, shall be apportioned between

the State Universities with a final assessment

above 1,50.

• Thanks to PSF

• RECOMMENDATION 2.5: Performance-based funding for PROs • The present model for funding Bulgarian higher education and research organisations is clearly inadequate when it

comes to supporting research and encouraging building up high-level research environments. The PSF panel recommends to move to a progressively higher concentration of resources based on performance-based funding, to facilitate transparent, fair and efficient allocation of resources and enhancing incentives for high research performance. It should be a system recognizing the binary nature of the Bulgarian system (with a separation between research institutes and higher education teaching establishments).

• Recommendation # 3.7 • The Bulgarian government and universities/BAS should ensure that the assessment of researchers for recruitment,

promotion and research grant funding purposes is done in an open, transparent and merit-based manner on the basis of research excellence. Measurement should be based on a combination of metrics including publications, citation impact (and other bibliometrics), patents, throughput of graduate students, international and industry collaboration.

TO CONCLUDE

We are all aware on the problems, scarcities and

weaknesses of Bulgarian Research and Innovation system

It is necessary to build a system bringing together

national information resources scientometric data from

Web of Science and SCOPUS

We have to concentrate our efforts on potentials,

perspectives and strengths

Our ultimate goal is to re-build trust in research

profession and in research institutions in Bulgaria, in

order to be able to develop our new generation of

motivated, highly–skilled and excellent researchers

THANK YOU FOR YOUR ATTENTION

Documents

Achieving efficiency and effectiveness of the R&I...Source: SciVal 2011-2016 . REPUBLIC OF BULGARIA Ministry of Education and Science ... introducing international quality standards