Document of
The World Bank
Report No: ICR00003911
IMPLEMENTATION COMPLETION AND RESULTS REPORT
(TF071742)
ON A
GRANT
IN THE AMOUNT OF €8.4 MILLION
(US$11.9 MILLION EQUIVALENT)
TO THE
REPUBLIC OF SERBIA
FOR THE
INNOVATION SERBIA PROJECT
October 8, 2016
Trade and Competitiveness Global Practice
Europe and Central Asia Region
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
CURRENCY EQUIVALENTS
(Exchange Rate Effective July 6, 2016)
Currency Unit = Euro (€)
€1.00 = US$1.09
US$1.00 = €0.917
FISCAL YEAR
July 1 – June 30
ABBREVIATIONS AND ACRONYMS
C&JP Competitiveness and Jobs Project
CFP Call for Proposals
EPO European Patent Office
ENIF Enterprise Innovation Fund
EU European Union
EUD Delegation of the European Union
EU IPA European Union Instrument for Pre-Accession
FINS Institute of Food Technology Novi Sad
FM Financial Management
GDP Gross Domestic Product
GM Grant Manual
GoS Government of Serbia
GRA Global Research Alliance
IC Investment Committee
ICR Implementation Completion and Results Report
ICT Information and Communication Technology
IF Innovation Fund
IFVC Institute of Field and Vegetable Crops
IMGGE Institute of Molecular Genetics and Genetic Engineering
IMPR Institute of Medicinal Plants Research
IP Intellectual Property
IPA Instrument for Pre-Accession
IPB Institute of Physics, Belgrade
MOERD Ministry of Economy and Regional Development
MOSTD Ministry of Science and Technological Development
MOESTD Ministry of Education, Science and Technological Development
MPI Mihajlo Pupin Institute
MRI Maize Research Institute
PAD Project Appraisal Document
PDO Project Development Objective
R&D Research and Development
RDI Research and Development Institution
SIP
SME
Serbia Innovation Project
Small and Medium Enterprise
SRITTP Serbia Research, Innovation and Technology Transfer Project
TA Technical Assistance
TTF Technology Transfer Facility
VC Venture Capital
WB EDIF Western Balkans Enterprise Development and Innovation Facility
Senior Global Practice Director: Anabel Gonzalez
Sector Manager: Paulo Guilherme Correa
Project Team Leader: Natasha Kapil
ICR Team Leader: Natasha Kapil
SERBIA
Innovation Serbia Project
CONTENTS
Data Sheet
A. Basic Information………………………………………………………………….i
B. Key Dates………………………………………………………………………….i
C. Ratings Summary……………………………………………………………….....i
D. Sector and Theme Codes………………………………………………………......ii
E. Bank Staff………………………………………………………………………….ii
F. Results Framework Analysis……………………………………………………...iii
G. Ratings of Project Performance in ISRs………………………………………......vi
H. Restructuring ……………………………………………………………………..vi
I. Disbursement Graph……………………………………………………………...vii
1. Project Context, Development Objectives and Design ............................................... 1
2. Key Factors Affecting Implementation and Outcomes ............................................ 10
3. Assessment of Outcomes .......................................................................................... 16
4. Assessment of Risk to Development Outcome ......................................................... 27
5. Assessment of Bank and Recipient Performance ..................................................... 28
6. Lessons Learned ....................................................................................................... 30
7. Comments on Issues Raised by Recipient/Implementing Agencies/Partners .......... 31
Annex 1. Project Costs and Financing .......................................................................... 32
Annex 2. Outputs by Component ................................................................................. 33
Annex 3. Economic and Financial Analysis ................................................................. 40
Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 41
Annex 5. Beneficiary Survey Results ........................................................................... 43
Annex 6. Stakeholder Workshop Report and Results ................................................... 47
Annex 7. Summary of Recipient's ICR and/or Comments on Draft ICR ..................... 48
Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 50
Annex 9. List of Supporting Documents ...................................................................... 51
MAP …………………………………………………………………………………..52
i
Datasheet
A. Basic Information
Country: Serbia Project Name: Innovation Serbia
Project
Project ID: P126229 L/C/TF Number(s): TF11257
ICR Date: 10/08/2016 ICR Type: Core ICR
Lending Instrument: TAL Borrower/Recipient: Government of Serbia
Original Total
Commitment: US$9.52 million Disbursed Amount: US$8.35 million
1
Revised Amount: US$7.15 million
Environmental Category: B
Implementing Agencies: Ministry of Education, Science and Technological Development
(MOESTD); Serbia Innovation Fund (IF)
Co-financiers and Other External Partners: The Donor is the EU Delegation in Serbia
B. Key Dates
Process Date Process Original Date Revised/Actual
Date(s)
Concept Review: 05/05/2011 Effectiveness: 12/08/2011
Appraisal: 06/01/2011 Restructuring(s): 09/05/2014
Approval: 11/23/2011 Mid-term Review: 08/25/2014 08/25/2014
Closing: 11/30/2014 01/10/2016
C. Ratings Summary
C.1 Performance Rating by ICR
Outcomes: Satisfactory
Risk to Development Outcome: Low or Negligible
Bank Performance: Satisfactory
Recipient Performance: Satisfactory
1 This system generated datasheet only reflects Recipient Executed disbursement amounts and
does not reflect Bank Executed components or fees associated with this hybrid trust fund. This
table would suggest significant undisbursed and cancelled resources at the end. However, only
Euro 655,000 were cancelled at project closing. Differences between the Revised amount and the
Disbursed Amount are most likely explained by the Euro vs. US$ exchange rate differential
between appraisal and closing.
ii
C.2 Detailed Ratings of Bank and Recipient Performance (by ICR)
Bank Ratings Recipient Ratings
Quality at Entry: Satisfactory Government: Moderately Satisfactory
Quality of Supervision: Highly Satisfactory Implementing
Agency/Agencies: Highly Satisfactory
Overall Bank
Performance: Satisfactory
Overall Recipient
Performance: Satisfactory
C.3 Quality at Entry and Implementation Performance Indicators
Implementation
Performance Indicators
QAG Assessments
(if any) Rating
Potential Problem Project
at any time (Yes/No): No
Quality at Entry
(QEA): None
Problem Project at any
time (Yes/No): No
Quality at Entry
(QEA): None
DO rating before
Closing/Inactive status: Satisfactory
Quality of
Supervision (QSA): None
D. Sector and Theme Codes
Original Actual
Sector Code (as % of total Bank financing)
Public administration - Financial Sector 40 40
Public administration - Industry and Trade 10 10
Tertiary Education 50 50
Theme Code (as % of total Bank financing)
Technology diffusion 100 100
E. Bank Staff
Positions At ICR At Approval
Vice President: Cyril E. Muller Philippe H. Le Houerou
Country Director: Ellen Goldstein Jane Armitage
Sector Manager: Paulo Guilherme Correa Lalit Raina
Project Team Leader: Natasha Kapil Natasha Kapil
ICR Team Leader: Natasha Kapil N/A
ICR Primary Author: Hiran Herat N/A
iii
F. Results Framework Analysis
Project Development Objectives (from Project Appraisal Document) The development objective of the project is to assist in building the institutional capacity to
stimulate innovative activities in the enterprise sector by:
(a) Supporting the operationalization of the Serbia Innovation Fund (IF);
(b) Piloting financial instruments for technological development and innovation in
enterprises; and
(c) Encouraging selected research and development institutes (RDIs) to engage in
technology transfer and commercialization, and assisting in formulating RDI sector
reform policy.
Revised Project Development Objectives (No changes, as approved by original approving
authority
(b) Intermediate Outcome Indicator(s)
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised Target
Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1 : Number of financial instruments designed and introduced
Value
(quantitative
or Qualitative)
0 3 4
Date achieved 12/05/2011 08/01/2014 08/01/2014
Comments
(incl. %
achievement)
Target exceeded by 33%
Indicator 2 : Number of IF managers trained and applying newly learnt skills
Value
(quantitative
or Qualitative)
0 4 10
Date achieved 12/05/2011 08/01/2014 12/11/2015
Comments
(incl. %
achievement)
Target exceeded by 150%
Indicator 3 : Number of start-ups mentored
Value
(quantitative
or Qualitative)
0 10 51
Date achieved 12/05/2011 08/01/2014 12/11/2015
Comments
(incl. %
achievement)
Target exceeded by 410%
Indicator 4 : Mechanism to support training/mentoring of innovative start-ups established
Value
(quantitative No Yes Yes
iv
or Qualitative)
Date achieved 12/05/2011 04/30/2013 04/30/2013
Comments
(incl. %
achievement)
Target met.
Indicator 5 : Number of networking and educational events organized for enterprises, academia and
research
Value
(quantitative
or Qualitative)
0 8 22
Date achieved 12/05/2011 08/01/2014 12/11/2015
Comments
(incl. %
achievement)
Target exceeded by 175%
Indicator 6 : Number of funded projects
Value
(quantitative
or Qualitative)
0 20 52
Date achieved 12/05/2011 04/30/2013 04/30/2013
Comments
(incl. %
achievement)
Target exceeded by 160%
Indicator 7 : Number of funded enterprises
Value
(quantitative
or Qualitative)
0 15 48
Date achieved 12/05/2011 08/01/2014 08/01/2014
Comments
(incl. %
achievement)
Target exceeded by 220%
Indicator 8 : Number of RDIs identified for technology transfer and commercialization support
Value
(quantitative
or Qualitative)
0 4 10
Date achieved 12/05/2015 08/01/2014 08/01/2014
Comments
(incl. %
achievement)
Target exceeded by 150%
Indicator 9 : Number of RDIs assessed jointly by international experts & RDI management
Value
(quantitative
or Qualitative)
0 2 4
Date achieved 12/05/2011 08/01/2014 08/01/2014
Comments
(incl. %
achievement)
Target exceeded by 100%
v
No. Date ISR
Archived DO IP
Actual
Disbursements
G. Ratings of Project Performance in ISRs
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1 : Amount of innovation financing mobilized by IF in addition to this EU IPA project
Value
quantitative or
Qualitative)
0 EUR 20 million
WB EDIF EUR 28
million
SRITTP Project EUR
6.9 million
Date achieved 12/05/2011 01/08/2014 12/11/2015
Comments
(incl. %
achievement)
Target exceeded by 75%
Indicator 2 : Number of active technology start-ups funded
Value
quantitative or
Qualitative)
0 10 38
Date achieved 12/05/2011 04/08/2015 12/11/2015
Comments
(incl. %
achievement)
Target exceeded by 280%
Indicator 3 : Number of new products and processes launched by beneficiary enterprises
Value
quantitative or
Qualitative)
0 8 19
Date achieved 12/05/2011 08/01/2014 11/30/2014
Comments
(incl. %
achievement)
Target exceeded by 138%
Indicator 4 : Number of technologies transferred by RDIs
Value
quantitative or
Qualitative)
0 3 3
Date achieved 12/05/2011 08/01/2014 01/04/2016
Comments
(incl. % achievement) Target met.
Indicator 5 : Dissemination of policy recommendations for RDI sector reform
Value
quantitative or
Qualitative)
No strategy/Policy
exists
Disseminated to
stakeholders
Disseminated to
stakeholders
Date achieved 12/05/2011 08/01/2014 01/04/2016
Comments
(incl. % achievement) Target met.
vi
(US$, millions)
1 05/28/2012 Satisfactory Satisfactory 1.03
2 11/26/2012 Satisfactory Satisfactory 1.71
3 07/04/2013 Satisfactory Satisfactory 2.78
4 04/05/2014 Moderately Satisfactory Moderately Satisfactory 5.59
5 10/17/2014 Satisfactory Satisfactory 7.02
6 04/16/2015 Satisfactory Satisfactory 8.11
7 01/11/2016 Satisfactory Satisfactory 8.49
H. Restructuring (if any)
Restructuring
Date(s)
Board
Approved
PDO Change
ISR Ratings at
Restructuring
Amount
Disbursed at
Restructuring
in US$,
millions
Reason for Restructuring &
Key Changes Made DO IP
September 5,
2014 No S S 6.36
Closing date extended from
November 30, 2014 to January
10, 2016. The extension
allowed IF to complete core
program activities, including
contracting of the new awardees
from the fourth call for
proposals as well as disbursing
earlier approved funds to the
existing awardees.
I. Disbursement Profile2
2 This table would suggest significant undisbursed and cancelled resources at the end. In fact,
there was a small cancellation of remaining funds at Closing, equaling Euro 655,000. This chart
is explained by the exchange rate differential at appraisal and closing of the Euro vs. US$.
1
1. Project Context, Development Objectives and Design3
1.1 Context at Appraisal
1. The national innovation system and composition of research and development
(R&D) funding in 2012 did not support enterprise innovation at any significant level, and
the research sector modernization agenda was in its infancy. The Serbian R&D sector was
dominated by the public sector, largely inefficient, and most importantly delinked from industry
needs, particularly those of local small and medium enterprises (SMEs). The Serbian industry’s
capacity, both in terms of human capital and financial resources for R&D and innovation, had
been severely weakened post-transition and exacerbated by the lingering effects of the financial
crisis. Brain drain was a major concern, in the private sector and even more so in the scientific
community.
2. Serbia spent roughly €100 million (0.3 percent of gross domestic product [GDP] in
2010) on R&D, with a goal to reach 1 percent by 2014. While R&D spending had increased
from €28 million in 2001, Serbia still lagged significantly behind its neighbors. Slovenia, the
Czech Republic, and Croatia spend more than 1 percent of GDP, and the European average was
1.8 percent. In Serbia, the majority of R&D spending went to basic research, which accounted
for 50 percent of all R&D funding. A significant shift from basic to applied research was needed
for research commercialization and development of an innovative SME sector. This, along with
developing stronger connections between science and industry, was to be key in raising R&D
expenditures to 2 percent of GDP by 2020, with a target of leveraging half of this from the
private sector.
3. Outputs from the R&D sector were not commensurate with the public resources
being invested and did not support modernization of the Serbian economy. While the
number of scientific publications had increased during the previous few years, quality remained
poor. Intellectual property (IP) was either not being created or not being protected. Merely 21
patents were registered by public research and development institutes (RDIs) from 2003 to 2008,
with only 36 patent applications, mostly in Serbia. Figures for the private sector were similar.
4. In an attempt to modernize the national innovation system, the authorities had
implemented a reform program over the previous two years to revamp the regulatory and
institutional framework governing science, technology, and innovation. It had enacted a
number of laws to promote science and innovation and formulated the Serbian Scientific and
Technological Development Strategy 2009–2014. In addition, the Ministry of Education and
Science was created as the central body responsible for science and innovation in Serbia.4
3The PDO was not revised over the course of the Project.
3 This section is based on the Project Appraisal Document (PAD).
4 The March 2011 restructuring of the Government of Serbia (GoS) led to a merger of the Ministry of Science and
Technological Development (MOSTD) with the Ministry of Education, thus creating a new entity, the Ministry of
Education, Science and Technological Development (MOESTD). The innovation and technology development
priorities have remained unchanged as a result of restructuring. The review of programs described herein was
prepared by MOSTD.
2
5. Science infrastructure improvement had been the main focus to that point, while
stimulation of private sector-led R&D and innovation had been negligible. The European
Investment Bank provided a loan of €200 million to upgrade infrastructure at universities and
RDIs and to set up science parks as well as Centers of Excellence in priority areas including
biotechnology, nanotechnology, and advanced computing. In 2010, the Government disbursed
approximately €70 million for basic research, technological development, integral and
interdisciplinary research, and innovation activities. However, almost 80 percent of the funds
intended for science projects actually went to salaries for researchers.
6. RDIs were the primary recipients of financing, with private firms receiving funding
only under the technological development component. The 471 technological development
projects supported by the MOSTD were led by RDIs and not by industry. Public RDIs depended
on these projects as they did not receive regular institutional budget allocations. Projects
generally had a 90 percent approval rate, creating disincentives for researchers to innovate.
Except in a few cases, technology transfer and diffusion from RDIs was quite low with only a
few spin-offs and real technology-based start-ups, indicating the need to place greater emphasis
on supporting the various stages of technology development and innovation within enterprises—
for example, technology transfer, commercialization, diffusion, absorption, adaption, and
application.
7. Few mechanisms to incentivize private sector R&D and innovation had been
implemented to that point in time. The Ministry of Economy and Regional Development
(MOERD) administered the Program to Strengthen Innovation in SMEs, that is, matching grants,
to support SME investment in innovation through co-financing. However, the total budget for
2010 was RSD 40 million (€380,000). All legal entities were eligible for co-financing of up to 50
percent of justified expenses for innovation activity. Matching funds had to be provided from the
SME’s own resources and could not include any other public support. The awarded amounts
could not exceed RSD 800,000 (€7,500) for the first group of eligible activities, and RSD 1.5
million (€14,000) for the second group. In addition, under the Instrument for Pre-Accession
(IPA) 2010 program, €3 million was allocated for the Integrated Innovation Support Program
with the objective to ‘increase the competitiveness and economic growth in Serbia, through
strengthening of innovation in SMEs in accordance to National Strategy for Development of
Competitive and Innovative SMEs 2008–2013.’ However, implementation of this program had
not yet started.
8. In the post-crisis environment, the Government was expected to play a critical role
in stimulating economic recovery through policies that target firm-based R&D and
innovation. While the Government and other international donors had demonstrated efforts to
support R&D and innovation, these were focused on renewing infrastructure investments in
public RDIs with negligible support for private sector innovation. Given the World Bank’s
experience garnered in supporting private sector development in post-transition economies,
including in neighboring Croatia, the GoS was keen on a World Bank engagement on the
innovation agenda. Hence, the World Bank’s technical assistance (TA) to the GoS and to the
future Innovation Fund (IF), being established under the project as the implementing agency,
focused on filling these gaps and developing mechanisms to stimulate enterprise-led technology
development and innovation.
3
9. The objective of the FY12-15 CPS was to support Serbia’s EU accession and help
the Government strengthen competitiveness and improve the efficiency and outcomes of
social spending. Under the Strengthening Competitiveness pillar Improved innovation capacity
was identified as one of objectives. The CPS noted that the system and composition of R&D
funding did not support Serbia’s agenda to modernize and enhance its competitiveness. The CPS
sought to change this situation by supporting the establishment of institutional capacity to
stimulate activities in the enterprise sector. The capacity would arise from creating a Serbia
Innovation Fund, using financial instruments for innovation and technological development in
enterprises, and having R&D institutes engaged in transferring and commercializing technology
and in helping formulate RDI reform policy.
1.2 Original Project Development Objectives (PDO) and Key Indicators
10. The project development objective (PDO) was to assist in building the institutional
capacity to stimulate innovative activities in the enterprise sector by:
(a) Supporting the operationalization of the Serbia Innovation Fund (IF);
(b) Piloting financial instruments for technological development and innovation in
enterprises; and
(c) Encouraging selected research and development institutes (RDIs) to engage in
technology transfer and commercialization, and assisting in formulating RDI sector
reform policy.
11. The key performance indicators for this project included:
(a) Amount of innovation financing mobilized in addition to this European Union
Instrument for Pre-Accession (EU IPA) project;
(b) Number of active technology start-ups funded;
(c) Number of new products and processes launched by beneficiary enterprises;
(d) Number of technologies transferred by RDIs; and
(e) Dissemination of policy recommendations for RDI sector reform.
1.3 Revised PDO and Key Indicators
12. The PDO was not revised. PDO-level indicators 1 and 2 were refined for clarity’s sake
during negotiations with the donor and recipient, and are fundamentally the same as in the PAD.
1.4 Main Beneficiaries
13. The primary beneficiaries of this pilot project were the staff of the IF whose capacity was
built under Component 1; entrepreneurs, start-ups, and firms that received financing and
mentoring/training under Component 2; researchers and leadership of select RDIs that benefited
4
from exposure to modern practices in IP management, technology transfer, and
commercialization, as well as the Ministry of Education, Science and Technological
Development (MOESTD) that also benefited from the policy recommendations developed based
on the RDI TA program under Component 3. Important secondary beneficiaries included
stakeholders engaged during project launch and implementation through numerous open
houses/workshops/conferences on such topics as entrepreneurship, research commercialization,
or innovation finance. These stakeholders ranged from across Serbia’s national innovation
system, including leading universities, RDIs, incubators, technology transfer offices, nascent
early stage investors, and private sector actors.
1.5 Original Components
14. In order to build institutional capacity to stimulate innovative activities in the enterprise
sector, the project design intended to leverage three complementary components that would
enable piloting of institutional, financial, and commercialization support mechanisms absent
from the national innovation ecosystem with the view to support and ideally demonstrate
Serbia’s enterprise innovation potential.
Component 1: Capacity Building of the Serbia Innovation Fund - Recipient-Executed (€1.1
million)
15. The capacity of the IF was being established to encourage entrepreneurship, including by
financing enterprise innovation, as well as participating in long-term programming efforts by
national authorities, international organizations, financial institutions, and the private sector. In
this context, this component provided support for:
(a) operations of the Investment Committee and due diligence of grant applications;
(b) engagement of strategic and operations advisors to assist the IF’s current and future
programs, strategy, operations, and procedures;
(c) operational project management in relation to the European Union (EU) and World
Bank;
(d) IF staff training;
(e) IF beneficiary enterprise mentoring and training;
(f) project visibility events and workshops; and
(g) IF operational infrastructure, including information and communication technology
(ICT) and financial management (FM) development.
Component 2: Implementation of Financial Instruments Supporting Enterprise Innovation
- Recipient-Executed (€6.0 million)
16. This component financed (a) Mini Grants targeting the proof of concept and prototyping
stages, including but not limited to IP protection and business plan preparation for initial capital
5
mobilization; and (b) Matching Grants targeting R&D in technology development projects, for
new or improved technologies, products and processes. Incorporated entrepreneurs, innovative
start-ups, spin-offs, micro and small enterprises, with majority Serbian private sector ownership
were eligible for Mini and Matching grants. The Mini and Matching Grants applications were
evaluated by the IF’s independent Investment Committee (IC), with input from international peer
reviewers.
17. The resources under this component were earmarked notionally per instrument with the
objective of flexible reallocation among instruments based on demand. The terms and conditions
of the grant programs were designed with the IF advisors with sufficient flexibility to
accommodate changing market conditions and the quality and volume of the project pipeline. A
guiding principle was to decrease the matching contribution from the IF over time, particularly
for the Matching Grants instrument.
Table 1. Design Features of the Mini Grant Programs at Appraisal
Mini Grants
Baby Seed €1,500,000–€3,000,000
Objective and stage Proof of concept, prototyping stage, IP protection, business plan preparation for
mobilization of initial capital
Recipient Incorporated entrepreneurs, innovative startups, spin offs, micro and SMEs, all
with majority Serbian private sector ownership
Grant size Up to €80,000 for projects that will be completed within 12 months
Features Up to 85% of approved project costs
Table 2. Design Features of the Mini Grant Programs at Appraisal
Matching Grants €3,000,000–€4,500,000
Objective and stage R&D (technology development) and commercialization projects for new or
improved technologies, products, and processes
Recipient Incorporated entrepreneurs, innovative startups, spin offs, micro and SMEs, all
with majority Serbian private sector ownership
Grant size Up to €300,000 for projects that will be completed within 2 years.
Features Up to 75% Matching Grant with an optional 3–6% royalty component based on
sales revenue, up to 120% of the original grant within a predetermined time
frame
Component 3: Provision of Technical Assistance to Research and Development Institutes
(RDIs) - Bank-Executed (€0.7 million)
18. Component 3 was to provide (a) customized TA to up to two RDIs based on a detailed
needs assessment; and (b) limited TA to up to four RDIs based on a general needs assessment;
and (c) technical input to the Government’s future RDI sector reform program based on lessons
learned from the aforementioned TA program.
19. A customized TA program for up to two select RDIs entailed conducting a detailed
assessment to be carried out by international experts in partnership with the RDIs’ management.
The assessment was to cover issues, including but not limited to, the organization’s management
structure, human and budgetary resources, researcher promotion and incentive system,
infrastructure and laboratory facilities, research capabilities and outputs, IP protection,
challenges and potential for applied R&D, and technology transfer and commercialization. Based
6
on this diagnosis, technical support was to be provided for the design and implementation of
RDI-specific upgrading programs. Upgrading of labs and other infrastructure would not be
supported. The RDIs were identified jointly by MOESTD and the World Bank based on several
key factors, including the level of commitment to reform from the RDI management team,
potential for improvement, likelihood of success, sectoral coverage, and availability of resources.
The lessons derived from this program were expected to inform future strategic planning and
reforms in the RDI sector. Specifically, the recommendations were expected to be focused and
actionable, while identifying quick wins. The World Bank team was to organize a workshop with
the ministry in charge to disseminate the experience, lessons learned, and recommendations for a
future RDI system reform program.
1.6 Revised Components
20. There were no revisions to the components.
1.7 Other Significant Changes
21. There were no significant changes. A simple project restructuring was conducted with
approval from the management of the Europe and Central Asia Region and the donor to extend
the closing date from November 30, 2014 to January 10, 2016. The extension allowed the IF to
complete core activities, such as contracting of awardees from the fourth call and disbursing
previously approved funds to awardees.
22. The US$ appreciated almost 30 percent against the EURO during project implementation.
The US dollar amount at project approval was $11.9 million. The final disbursement amount –
both Bank and Recipient Executed - was approximately US$ 9.7 million, with disbursement via
the Recipient amounting to US$8.35 million. Of the total EURO 8.4 million, EURO 655,000
was cancelled. Fortunately, the exchange rate differential in dollars did not affect
implementation as the grant and expenditures were in Euros.
2. Key Factors Affecting Implementation and Outcomes
2.1 Project Preparation, Design and Quality at Entry
(a) Soundness of the Background Analysis
23. The background analysis for this project was sound, given its pilot nature and little
government experience in supporting enterprise innovation or RDI sector reforms. The project
was aligned with the FY08–FY11 Country Partnership Strategy, wherein a key priority included
encouraging dynamic private sector-led growth to promote convergence with European levels. In
FY09 and FY10, the World Bank team delivered a TA program, supporting MOESTD to update
legislative frameworks that encouraged firm uptake of innovation support. Specifically, the
World Bank team worked on identifying challenges with existing innovation programs,
reviewing the legal framework governing R&D and innovation, and a new innovation strategy.
Further, at entry, the focus was on (a) appraising institutional arrangements supporting research
and innovation; (b) surveying and assessing firm perception of existing innovation and
technology absorption support programs and appetite for proposed instruments; and (c)
discerning level of opposition for support to firms and for research sector reform. The analysis
and consultations were instrumental in sharing regional and global experience, building
7
stakeholder confidence on the need to deploy a pilot project that would initiate a steady transition
to an enterprise-led model.
(b) Assessment of the Project Design
24. Pilot. The project was designed carefully as a modest pilot initiative that aimed at
building the missing institutional capacity in the Serbian innovation system, testing financial
instruments and mentoring mechanisms to reveal the emerging innovative entrepreneurial sector,
and identifying viable approaches to the challenges of Serbia’s ailing research sector. Given high
expectations from this pilot, the risky nature of both demand- and supply-side interventions,
accompanied by significant latent opposition from researchers in the public RDIs, the split
between World Bank and recipient-executed components was prudent. Equally judicious was the
attraction of top global expertise to build credibility for this pilot, and demonstration of key
principles in innovation policy management, including horizontality, independence, good
governance, efficiency, transparency, monitoring and evaluation (M&E), and open
communication with stakeholders.
25. Selection Procedures. Noteworthy design features included the IF’s efficient and
independent grant selection processes that engaged international peer reviewers and an IC
featuring international and diaspora experts with a background in scientific research,
entrepreneurship, technology commercialization, market trends, and venture investments.
26. Visibility. The donor placed significant emphasis on organizing project visibility
activities and these ended up being instrumental given the risky and novel nature of the pilot.
Most visibility activities were high level in nature and co-hosted by the recipient, the donor, and
the World Bank, which helped reinforce the Government’s commitment in spite of multiple
ministerial reshufflings within the lifetime of the project.
27. Ownership. The World Bank-executed TA component was designed to support
knowledge transfer and research commercialization within the lifetime of the project. Only those
RDIs where management demonstrated (a) willingness to be assessed by international expert
teams convened for their institution and (b) commitment to act on key recommendations from
such assessments, were included in the RDI TA program. This allowed the World Bank team to
narrow down the list of RDIs it chose to provide TA support. In retrospect, this important design
feature afforded flexibility, as both time and financial resources were limited.
(c) Adequacy of Government’s Commitment
28. The Government’s commitment toward the project—and especially for the enterprise
innovation objectives—was high at entry. The Government demonstrated this by pursuing the
Delegation of the European Union (EUD) to earmark EU IPA funding for enterprise innovation
for the very first time, approving a notional operational budget for the IF from 2011 to 2015, and
attracting a senior diaspora member to lead the IF. The World Bank was nominated by the GoS
to enter into a trust fund arrangement and execute Component 3, which was controversial at the
time.
8
(d) Assessment of Risks
29. Several risks were identified during project preparation and adequate risk mitigation
measures were incorporated into the project design as follows:
A pioneer on several fronts, there were important coordination and
reputational risks stemming from this pilot. This was the first ever EU-financed
World Bank-administered trust fund in Serbia, addressing unchartered policy
territory, facing skepticism by entrenched interests and significant political
instability. Through meticulous supervision and dialogue with the EUD, MOESTD,
MOERD, IF, and other stakeholders during missions and steering committee
meetings, the World Bank fostered shared awareness on project progress and
ownership of challenges as they arose.
Significant resistance from entrenched interests to RDI sector reforms. The
World Bank team managed the RDI TA program closely, expanding the scope of
assistance only to volunteer RDIs where management demonstrated firm
commitment to optimization and knowledge transfer and research commercialization
outcomes.
The IF had no demonstrable capacity to transparently select projects, manage
FM, procurement, environmental screenings, or support the private sector. The
World Bank’s no objection to the selection of IC members, and to procedures
followed during the first call, supported the credibility of the selection processes.
The international advisors provided continuous capacity building for IF staff on
program design couched in global practice, whereas the IC safeguarded the IF’s
independence, together tempering undue influence from the research community or
political interference.
2.2 Implementation
30. The following factors had an impact on project implementation:
Factors having a positive impact
o The establishment of the IF as an efficient independent institution with strong
and transparent governance practices was instrumental in smooth project
implementation.
o The independent and professional manner in which beneficiaries were selected
and monitored during implementation ensured that the selected subprojects
were successfully implemented.
o Continuous access to the IF Advisory team, oversight from the World Bank,
and access to MOESTD, EUD, and the IF’s Board, was important in allowing
the IF to overcome teething and political challenges over the course of the pilot,
focus on creating a positive environment for its beneficiaries, and introduce
new innovation and technology transfer programs for Serbia.
9
o In line with the concept of the pilot, both recipient- and World Bank-executed
components were monitored closely and pivoted incrementally to enhance
program outcomes and inform longer-term instrument design. A few examples
include design of the mentoring program to accommodate mandatory and
customized training elements for IF staff and beneficiary entrepreneurs;
deploying the Mini and Matching Grant programs sequentially so the IF could
incorporate lessons from the first call of the Mini Grant Program; increasing the
private sector contribution (from 25 percent to 30 percent ) under the Matching
Grant program; converging on design details for the royalty scheme; and
engaging commercialization brokers to work with RDIs to identify
commercialization wins within the lifetime of the project.
o Under Component 3, the growing acceptance of knowledge transfer and
research commercialization was an important cultural shift among researchers
and managers in the RDI community, enabling the recipient to consider sector
wide reforms in the future.
o Continuous access to the IF Advisory team, oversight from the World Bank,
and access to MOESTD, EUD, and the IF’s Board, was important in allowing
the IF to overcome teething and political challenges over the course of the pilot,
focus on creating a positive environment for its beneficiaries, and introduce
new innovation and technology transfer programs for Serbia.
Factors having a negative impact
o 2012–2013. During the early stages of project implementation, disbursements
were lower than planned at entry. This was due to GoS insistence at entry to
plan as many as two Mini and two Matching Grant Calls for Proposals (CFPs)
in 2012. The IF had just been established and did not have the capacity at the
time to handle the proposed workload. However, based on guidance the IF
received from its advisors, it was eventually agreed to phase the rollout of the
programs and to pilot the first CFP for the Mini Grant Program separately, and
incorporate any lessons from this call into future ones. In retrospect, this was an
important adjustment, enabling IF staff to be better prepared for subsequent
calls.
o 2014. Serbia faced a turbulent year in 2014 and so did the IF. This included
catastrophic flooding, multiple changes in the Government, delayed funding,
and management restructuring within the IF. Implementation progress slowed
due to the lack of budget funds for the IF’s operations and given the
uncertainty, the IF experienced significant staff departure. Additionally,
MOESTD’s interest in engaging the World Bank on the public research sector
reform agenda diminished in the absence of a larger World Bank operation. The
project was downgraded from Satisfactory to Moderately Satisfactory.
Nonetheless, with guidance from its advisory team, the IF was able to rebuild
staff capacity by conducting additional trainings to bring the new cohort to
speed. During this period, project implementation continued, albeit at a slower
10
speed. By July 2014, the IF was fully staffed, the budget fully funded, and
operations commenced satisfactorily. After the mid-term review in August
2014, the project was again rated Satisfactory, and this status was maintained
through closing in January 10, 2016.
2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization
Rating: Substantial
31. M&E design. There was no systematic effort to capture firm-level innovation or research
commercialization data in Serbia at entry. Publicly available indicators were limited to typical
inputs (for example, public R&D spend) or research outputs (for example, publications and
patents). Given minimal baseline information, skepticism around the pilot in Serbia, the growing
debate on effectiveness of Matching Grants in the World Bank, and high donor and recipient
expectations, the M&E activities envisaged under this project were deliberate. Beyond the set of
input, output, and outcome indicators in the results matrix, the project focused on building the
IF’s long-term capacity to identify additional outcome indicators, build its internal systems to
capture and analyze firm-level innovation data to support future evidence-based policy making in
Serbia beyond this modest pilot. At project inception, M&E activities were managed and
supported by two IF staff members, a program manager, and a senior associate, whose tasks
entailed assisting in data collection, overseeing deliverables, and payment schedules. However,
as implementation progressed, the IF realized that it lacked the M&E expertise to establish and
operate an in–house system. As a result, a reputable external think tank with a track record in
conducting innovation program impact evaluations was engaged by the IF from May 2013 to
November 2015 to ensure the quality of methodology and implementation, and to design the
most relevant approach for the small sample size of beneficiaries expected under the pilot.
32. M&E implementation. The consulting firm developed a framework and procedures for
M&E activities under the project which was incorporated in an M&E manual for the IF, and in
addition, trained the relevant staff. Staff training provided by the external consultant focused on
identifying relevant indicators, monitoring processes, specifically deploying IF data collection
systems. As part of their assignment, the firm and IF gathered extensive data on both
beneficiaries and applicants. M&E functions were to monitor and analyze the performance of the
Mini and Matching Grant programs’ subprojects. IF staff were also trained to monitor financing
programs to be managed by the IF in the future. The IF produced semiannual M&E reports.
These reports were shared with the Government, World Bank, and EUD, and served as a useful
tool for project monitoring and progress reporting. The independent evaluation was led by an
independent firm. For the World Bank-executed component, M&E data was collected and
analysed by the World Bank team.
33. M&E utilization. The IF and the World Bank monitored the interim outputs of the
project systematically to assess progress toward end-of-project target outcomes, identifying
issues and risks, and reporting on time-bound action plans to mitigate them. In August 2015,
with participation of IF staff, the consulting firm presented its findings to key stakeholders,
including senior representatives from MOESTD, MOERD, EUD, Ministry of Finance, and IP
office, among others. The focus of the presentation was on the selection process, demonstrating
the additionality of IF programs and recommendations for scaling of future programing. The
11
evaluations of the Mini and Matching Grant programs justified the EUD’s decision to provide
additional funding for these programs under its future programming and also justified inclusion
of these programs under the World Bank-funded Competitiveness and Jobs Project (C&JP), a
results-based operation (2015). On Component 3, the monitoring indicators proved useful in
concentrating efforts of RDI management and researchers on knowledge transfer and
commercialization activities. Importantly, it allowed the World Bank team to estimate the
financing gap for knowledge transfer and research commercialization in Serbia, enabling the
GoS to advocate for the sequel EU IPA-financed program with the EUD, which supported the
establishment of the national Technology Transfer Facility (TTF) and the Collaborative Grant
Scheme under the ongoing Serbia Research, Innovation and Technology Transfer Project
(SRITTP).
2.4 Safeguard and Fiduciary Compliance
34. The project did not entail any major environmental risk, include construction or
land acquisition. The project was rated category ‘B’. There were no issues in compliance with
environmental safeguards. Due to the limited size and scope of the grants, there was no
significant environmental impact associated with the project’s activities. The overall innovation
process under this component did not support environmentally unfriendly technologies and
practices. The Environmental Management Framework was implemented successfully. The
project did not entail any social risk or risk of adverse social impact.
35. FM was assessed as Satisfactory. There were no outstanding audit reports for the
project and all audit reports were found satisfactory to the World Bank. Quarterly interim
unaudited financial reports were submitted to the World Bank on time. The project audits
received ‘unqualified’ opinions of the audited financial statements and no major systems and
control issues were identified by the auditors.
36. Procurement was rated Satisfactory, based on review of prior review documents
and post review missions. The IF and beneficiaries followed the World Bank's procurement
guidelines and no major issues were encountered. For procurement under the grants, World
Bank-approved simplified commercial practices were used.
2.5 Post-completion Operation/Next Phase
37. In 2014, the Government was unable to finance post-pilot calls for the IF grant programs
as budget resources were diverted to post-flood remediation efforts. However, key elements for a
post-pilot transition are now in place with the approval of the EU IPA SRITTP and the C&JP
operations in 2015. SRITTP provides support for national technology transfer activities,
collaborative industry/academia research activities, and support for development of the new
strategy, the infrastructure road map, and their related action plans. C&JP supports among other
activities, the integration of the IF’s operational budget into the GoS’s annual budget,
continuation of the Mini and Matching Grant Programs, scaling of technology transfer initiatives,
and RDI sector reforms through a results-based approach.
3. Assessment of Outcomes
12
38. There were no major shortcomings in achievement of project objectives, efficiency, or
relevance.
3.1 Relevance of Objectives, Design, and Implementation
(a) Relevance of Objectives
39. The relevance of the objective is rated High. The objectives and related outcomes of
this project are as relevant at the time of the Implementation Completion and Results Report
(ICR) as they were at entry and remain strongly aligned with the World Bank’s engagement in
Serbia and the Government’s development agenda. This project remains highly relevant for the
overarching objectives laid out in the FY16-20 CPF which is to support Serbia in creating a
competitive and inclusive economy and, through this, to promote integration into the EU. One of
two broad CPF focus areas - Private sector growth and economic inclusion should address
significant constraints to private sector development and economic inclusion: financing,
investment, connectivity, and labor market constraints. This Project was both timely and relevant
as it supported the creation of institutional capacity and programming for private sector
innovation that can be scaled with future IPA and eventually EU funding.
40. The GoS is undertaking measures to rebalance the economy, enhance the quality and
efficiency of public expenditures and service delivery (including state-owned enterprise reform),
and stimulate private sector initiatives to enhance Serbia’s competitiveness. In March 2016, the
Government adopted the Strategy for Science and Technology Development: Research for
Innovation 2016–2020, specifically calling for reforms in the public RDI sector and reinforcing
the importance of enterprise innovation and technology transfer for the economy, including
support for the activities carried out by the IF. This pilot furnishes evidence to engage on
innovation policy and is demonstrated by GoS engagement in the SRITTP and C&JP.
(b) Relevance of Design
41. The relevance of the design is rated Substantial. Project design was essential in
supporting the delivery of PDO outcomes and enabled successful piloting of missing elements of
the Serbian innovation system. The core design features remain relevant and are therefore being
continued or expanded under SRITTP and C&JP. They would warrant being supplemented by
increased research infrastructure investments, coupled with reforms in the public research sector,
and direct support instruments for enterprise innovation and market access in the future. Under
Component 1, the IF leveraged funding and partnerships with international donor organizations
to strategically build entrepreneurship and innovation programming from traditional early-stage
matching grant funding (SRITTP, C&JP) to early stage equity funding (that is, Western Balkans
Enterprise Development and Innovation Facility [WB EDIF]). Through Component 2, the IF
demonstrated long-term demand for early-stage enterprise innovation funding and the ability of
Serbia’s private sector to lead innovation projects (often in partnership with the research sector)
and co-finance these with own matching resources. Under Component 3, the World Bank
managed to provide in-depth TA to four RDIs and deliver technology transfer targets by being
selective and staying focused. This built the case to establish a national TTF to stimulate
collaborative research under the SRITTP and support RDI sector reform activities under the
C&JP. At entry, there were no visible private ecosystem players present in Serbia; hence, the
13
pilot could not be designed to engage them. In future iterations of innovation programming and
financial instrument design (including of the Matching Grant programs), the IF could explore
leveraging local private ecosystem actors—for example, business angels, other early stage
investors, accelerators, and so on—in its selection, co-funding, and/or mentoring processes.
(c) Relevance of Implementation
42. Relevance of implementation is rated Substantial. Implementation outcomes remain
relevant, although implementation arrangements can be further strengthened in the future. The
emphasis placed on good international practices, efficiency, horizontality, independence, and
transparency is deemed relevant for the implementation outcomes of the pilot. So much so, that
the donors and the World Bank continue to leverage the IF and MOESTD for additional
innovation activities under the SRITTP and C&JP. Although currently manageable, if the IF and
MOESTD become solely responsible for innovation, without commensurate staffing and
capacity building in the new areas of responsibility (for example, technology transfer), there
could be trade-offs in terms of quality, delivery, and overall direction of the agenda. Further
expansion in the IF’s implementation responsibilities ought to be accompanied by wider
ownership and endorsement by key line ministries of the cross-cutting research and innovation
agenda, such as ministries in charge of the finance, economy, agriculture, and so on.
3.2 Achievement of Project Development Objectives
43. Achievement of PDO is rated High. Overall, the project achieved or exceeded its stated
objectives of building institutional capacity to stimulate innovative activities in the enterprise
sector as explained below. This was achieved through the operationalization of the IF, piloting of
financial instruments for technological development and innovation in enterprises; and
encouraging selected RDIs to engage in technology transfer and commercialization, and assisting
in formulating RDI sector reform policy.
44. Operationalization of the IF. As the main implementing agency, the IF was responsible
for (a) successfully designing, piloting, and managing enterprise innovation programs; (b)
supporting proof of concept and prototyping activities and demonstrating a strong pipeline of
entrepreneurs and innovative firms in Serbia; and (c) participating in co-financing of programs
and leveraging other activities organized by international organizations, financial institutions,
and the private sector, all to further the innovation agenda on behalf of the GoS. The IF engaged
its international strategic, operations, and regional advisors to develop its own capacity and
operational procedures and provide guidance to enterprises and the GoS on enterprise innovation
policy and programs.
45. Building Institutional Capacity within the IF and Beyond. An essential service
provided by the IF was the mentoring provided through Enterprise Training, in addition to the
specialized training programs that entrepreneurs would elect themselves. This was conducted by
international advisors covering key innovation management issues, such as business formation
and expansion, IP protection and commercialization, investment readiness, and early stage
technology and business development. Entrepreneurs reported these trainings to be very useful,
valuable, and otherwise inaccessible. This training also prepared IF staff for the advanced
capacity building led by the advisory team, and especially for the international trainings in Israel,
14
Finland, and Croatia that focused on ecosystem development, institutional, policy, early-stage
entrepreneurship and innovation program design, sequencing, and evaluation. The IF today is
well-regarded by its peers, and newer innovation agencies, including Macedonia and Georgia,
have requested the IF to conduct training programs for their staff.
46. Supporting Evolution of the Ecosystem. The IF played a vital awareness-building role
for the nascent innovation ecosystem by conducting around 22 educational and networking
events for its beneficiaries and a broad set of institutional stakeholders. These activities included
project launch activities across six cities in Serbia, successive launches of the Mini and Matching
Grants Programs, IP and Technology Transfer Workshop, WB EDIF workshop and four info-
days for the awardees (addressing project implementation and training held by the Serbian IP
office), four Enterprise Training Sessions for awardees, presentation of the IF Annual Report,
Finland-Israel Knowledge Economy Workshop, consultations with four major technology
transfer offices on the design of the centralized TTF, industry-RDI consultations on the design of
the Collaborative Grant Scheme, and project closing.
47. Leveraging of Resources by the IF. During design, the intention was for the IF be
established as an entity that is both experienced at managing and capable of raising additional
funds beyond this pilot. The aforementioned factors and IF’s strong delivery positioned it as an
institution with significant credibility to manage the EU and other donor funds in a professional
independent and efficient manner, far beyond what was originally planned at the onset of the
pilot, including for the following activities:
(a) WB EDIF (€140 million). The IF led the preparation of this regional program and
mobilized €28 million from the GoS.
(b) SRITTP (€6.9 million). A sequel to the current pilot, the SRITTP is financed
through the 2013 EU IPA.
(c) Serbia C&JP (€89.5 million). Under C&JP, the IF can access €12 million for its
operational budget, financing of new calls for the Mini and Matching Grant
Programs, and technology transfer activities.
44. Piloting financial instruments. The responsible governance mechanisms established by
the IF—through its 60-plus independent international peer reviewers, 5-member professional
diaspora-led independent IC, efficient under-90-day selection procedures codified in its Mini and
Matching Grant Manuals (GMs)—underscored the transparent merit-based selection processes it
developed. The IF completed four CFPs for the Mini and Matching Grants programs by October
2013, announcing the last batch of awardees on February 14, 2014. Funding was awarded for 41
Mini Grants and 11 Matching Grants to start-ups and firms with innovative projects and high
potential for commercialization. Nineteen new products and processes were launched by the
beneficiary companies, over 300 high-value jobs were supported, and 51 companies were
mentored through Enterprise Training.5 By closing, €5.4 million had been disbursed to awardees
who expressed their gratitude to the IF and authorities for designing and managing the grant
5 Detailed description of each sub-grant and the amounts disbursed can be found in project files. Annex 2 provides a
few examples of Mini and Matching Grant Projects.
15
facility in a transparent manner. Several of these early-stage projects are beginning to
demonstrate successes, with some receiving noteworthy coverage of their achievements in the
local and international press, including in the Financial Times, the BBC, and the New York
Times. At the time of ICR completion, 36 of the 38 startups funded under the Mini Grant
program and all 10 of the small and medium size firms financed under the Matching Grant
program were still active, demonstrating a 95 percent survival rate. In line with international
good practice, this pilot project was subject to a rigorous independent evaluation and annex 5
captures both key findings on additionality of programs as well as critical recommendations for
future scale up by GoS.
Box 1. Case Study of a Successful Mini Grant Program: Start-up Strawberry Energy
Strawberry Energy was a Mini Grant recipient. This Serbian startup made renewable energy sources more
accessible by developing, designing, and implementing clean technologies. It developed a solar charger and then
applied the expertise gained to create the ‘Strawberry Tree Mini’—a flexible and transportable mini solar charger
for cell phones—with IF financing. Besides improved characteristics of power production, consumption
management, and functionality of solar panels and batteries, the mini charger includes an interactive component
that shares renewable energy information with end users. A finalist of the 2013 World Technology Award,
Strawberry Energy has been recognized as an innovation leader in environmental and energy efficiency.
Strawberry Tree was featured on Mashable among 25 top technologies every smart city should have. Following
completion of the IF project, Strawberry Energy has continued development of its technology and has created the
‘Strawberry Mini Rural’, a solar charger suitable for areas with low access to electrical energy. The Accelerator
Venture Fund Eleven from Bulgaria made a €100,000 investment in Strawberry Energy in 2014. In October 2014,
Strawberry Energy was the only participating company from outside the United States to win the international
start-up contest organized by the GreenBiz Group, held in San Francisco, California.
Table 3. Additional Achievements of Mini and Matching Grant Awardees
Activity Number of Firms
Revenue generated as a result of the grants 19
International cooperation with firms 9
Foreign market distribution secured (United States) 2
New business collaboration formed 18
Applications for Patent Cooperation Treaty 13
Applications to the European Patent Office (EPO) 4
Patent applications: United States, Australia, Euro-Asia, and Japan 5
National patent applications 33
Trademarks 10
Copyrights 6
Source: IF.
45. Technology transfer and commercialization. Another activity that this pilot tackled
was to encourage RDIs to engage in institutional assessments and knowledge transfer and
research commercialization activities to glean insights for sector wide policy reforms. The World
Bank team conducted detailed diagnostic assessments of four RDIs (classified as Category I);
that is, two more RDIs than planned at entry: Institute of Physics, Belgrade (IPB), Institute of
Molecular Genetics and Genetic Engineering (IMGGE), Institute of Food Technology (FINS),
Novi Sad, and Institute of Medicinal Plants Research (IMPR). These assessments provided
detailed recommendations (and specific action plans) for improved institutional capacity,
performance management, and knowledge transfer and research commercialization practices that
were the basis for customized technical support provided to these RDIs. As part of the TA, the
World Bank organized trainings for a broader group of 10 RDIs (classified as Category II) to
16
identify opportunities for institutional improvements. These institutions participated in
discussions to identify cross-cutting topics of interest to RDIs, including: Program/Project
Management, IP Management, R&D Marketing and Sales, Performance Evaluation and Career
Planning/Development, and Managing/Developing R&D Capabilities. Based on expressed
interest, the World Bank delivered three high-level training workshops on topics of interest to the
RDI sector community.
46. Research Sector Reform. Given its modest means, the TA program made important
contributions to the voluntary institutional adjustments and technology transfer efforts at IPB and
IMGGE. These two RDIs were responsible for three of the early-stage technology transfer
targets met under this project. However, supporting technology transfer and commercialization
was very challenging owing to basic research orientation of most RDIs and the weak institutional
capabilities, low level of technological and even lower market readiness of the proposed R&D
projects. With the establishment of an Innovation Center at the IPB, technology transfer
activities progress picked up during the project. IPB leadership as well as many department
heads and researchers demonstrated strong interest and willingness to undertake some difficult
institutional and mind set changes. The IMGGE transformed itself from a university-like
organization to a research institute comparable with its international counterparts and at closing
was on the verge of exporting a new product supported under the TA (see annex 3 for details).
Both RDIs reported a significant change in attitudes among its management teams and
researchers in engaging the private sector on potential knowledge transfer and commercialization
projects. Both RDIs reported the culture as more favorable toward conducting applied research
with commercialization potential by partnering with the private sector, internal reforms in
performance and institutional management, and the need to look for non-budgetary sources of
revenues for the future. Both RDIs were able to obtain several knowledge transfer contracts from
the private sector firms and European organizations, produce patent filings and high quality
publications.
47. The management of the FINS and the IMPR—the two other RDIs supported under the
detailed TA—appreciated the in-depth assessments, but found it difficult to motivate staff to
proceed with the recommended institutional reforms without an explicit mandate push or support
from MOESTD.
48. Overall, this World Bank-led activity provided extensive insight into the market
relevance of the RDI projects and revealed the financing gap to support commercialization and
conduct RDI sector reforms nationally. In October 2013, the World Bank delivered a Policy Note
on RDI Sector Reform to MOESTD. In 2015, subsequent MOESTD leadership took up
recommendations from the Policy Note to initiate systematic reforms in the R&D sector. The
World Bank and IF teams also provided specific inputs to the ministry officials for the design of
two projects: (a) the EU IPA-funded SRITTP and (b) World Bank’s C&JP. For instance, under
the C&JP, the GoS is adopting a strategy that commits to undertaking reforms in the public
research sector by conducting RDI-level assessments with global experts and establishing
market-aligned key performance indicators that are for both individual researchers and RDIs.
Both projects were approved and currently are under implementation.
3.3 Efficiency
17
Rating: Substantial
49. Overall, this pilot achieved its objectives efficiently and is rated Substantial. It is not
useful to conduct a cost-benefit analysis for a project (a) that was attempting to pilot multiple
missing elements in an innovation ecosystem; (b) that was going to trigger systemic changes; (c)
that required significant scaling of financing to demonstrate impact; and (d) where the full effect
of the outcomes would be seen only several years post pilot. Comparing costs of similar
programs elsewhere can lead to erroneous inferences as initial circumstances vary by country.
The PDO was successfully achieved with the operationalization of the IF (€1.1 million) and the
project achieved better outcomes than originally envisaged: (a) leverage of at least €35 million
through three donor projects versus €20 million target; (b) grant decision making in 60–90 days
versus regional average of 150 days; (c) design of four financial instruments during the project
versus target of three; (d) Funding of 38 active startup that launched 19 new products versus 10
and 8, respectively; (e) mentoring of 51 startups versus initial target of 10; (f) rebuilding staff
capacity in spite of two waves of staff departures; and (g) being recognized by regional peers.
50. The independent quantitative evaluation (see annex 3 for analysis) delivered by the
external consulting firm in October 2015, titled ‘Estimation of the Effect of Participating in the
IF Financing Programs on Firm Performance,’ concluded that (a) the IF was a credible and
transparent platform, and accessible to many Serbian businesses, not just a small group of
entrepreneurs; (b) Mini and Matching Grant Programs, developed and piloted within the project,
were found to be clear, accessible, and effective, with the application process being similar to
other international programs and the evaluation and decision process being transparent and
trustworthy; (c) the IF succeeded in expanding the opportunities for tech-based entrepreneurs by
creating a number of innovative start-ups/spin-offs. Almost half (156 out of 326) of the firms that
submitted the application for IF financing were created for them to apply for the IF operation. In
addition, the evaluation found a high survival rate among supported firms. The IF succeeded in
expanding the network of tech-based entrepreneurs by creating a relatively high amount of
collaborations with academia and industry (also internationally); and (d) under the Mini Grants
Program (€2.9 million), 40 firms were approved, of which 37 were active at the end of the
project period (January 2016). Simple differences in sales changes between firms that received a
Mini Grant and firms that applied but were rejected revealed that the former increased their sales
by an additional €27,000 and their employment by an additional five employees, on average.
Based on these numbers, on average, the additional sales generated as a result of the Mini Grants
was approximately €1 million. Based on which, the investment in this component would be
recouped in a period of three years.
51. The RDI TA (€0.7 million), conducted four instead of two in-depth RDI assessments
planned, identified 10 RDIs for potential institutional improvements support instead of five,
delivered on three technologies transferred, and brought a new commercialization culture to
these RDIs. Most importantly, perhaps, learnings under this component informed the design of
the SRITTP and C&JP.
3.4 Justification of Overall Outcome Rating
Rating: Satisfactory
18
Table 4.Summary of Ratings Supporting Overall Outcome Rating
Relevance of Achievement of PDO
Efficacy Efficiency Overall
Outcome Objective Design Implementation Objective
High Substantial Substantial High Substantial Satisfactory
52. The overall outcome rating is based on ratings for Relevance, Efficacy, and Efficiency as
broken down in Table 4. Project Objective remains relevant and aligned with Serbia’s overall
development priorities and is rated as High. Project Design and Implementation are both
deemed to be relevant in the EU pre-accession context and are therefore rated as Substantial.
Some delays were experienced during early stages of implementation and there was also a need
to extend the closing date of the project. The delays were justified (and agreed to by the World
Bank, donor, and recipient) given strategic guidance provided to the IF by its advisory team to
stagger the launch of the Mini and Matching Grant Programs early during implementation to
allow for learning to be incorporated in subsequent rounds. The extension facilitated the
completion of all awardees’ projects, achievement of technology transfer targets with RDIs, and,
most importantly, rebuilt the ministry’s commitment on innovation policy and RDI reforms. In
this context, given that the Project mostly exceeded the PDO outcome targets, its Efficacy is
rated as High and Efficiency was rated as Substantial. All of the aforementioned ratings lead to
the Overall Outcome Rating of Satisfactory.
3.5 Overarching Themes, Other Outcomes and Impacts
(a) Poverty Impacts, Gender Aspects, and Social Development
53. This was a small pilot project aimed at stimulating innovative activities in the enterprise
sector. The 51 projects financed through the pilot in the total amount of €6 million over four
years were too small to have an impact on poverty, gender aspects, or social development.
However, the increased income and employment generated by the pilot program have given an
indication that, if scaled up substantially, the program could have a positive impact on poverty
reduction and economic development (see annex 3).
54. Women entrepreneurs. The IF programs supported a few remarkable women researcher
become entrepreneurs who were even featured in the international media. They have become an
inspiration for younger women entrepreneurs. It would be important to strengthen gender
emphasis in future programming given the very specific challenges Serbian women
entrepreneurs face in entering the market, accessing both debt and equity finance, and extensive
harassment while running a business.
(b) Institutional Change/Strengthening
55. The project has had a significant impact with respect to institutional changes, especially
with respect to the IF, but also the management teams and researchers at RDIs supported under
the TA program, as well as MOESTD, as discussed above.
19
(c) Other Unintended Outcomes and Impacts (positive or negative)
56. At entry, there were few players of any significance in Serbia’s innovation and
entrepreneurship ecosystem, with the exception of incubators in Belgrade and Novi Sad. By
project closing multiple public and private actors were active in Serbia, including business
angels, accelerators, incubators, technology transfer offices, science parks, and so on. As the
project did not actively finance ecosystem development, causality cannot be inferred between the
project and its evolution. However, stakeholders and beneficiaries have credited the IF and its
programming for shining a spotlight on Serbia’s early-stage entrepreneurial pipeline, attracting
regional investors, highlighting its policy efforts to promote research and enterprise innovation.
20
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops
57. Based on the interviews carried out with the beneficiaries as part of the M&E process, the
M&E consultants and World Bank staff concluded that (a) overall project objectives were
adequately specified; (b) the Mini and Matching Grants Programs were clear, accessible, and
effective; (c) program evaluation and decision process was transparent and trustworthy; (d)
application process was similar to other programs (internationally); (e) the IF staff ranked high
on professionalism and willingness to help, and were flexible; (f) implementation was
effective—most of the projects achieved their objectives; (g) RDI management valued the
changes in researcher behavior toward technology transfer; and (h) the Government takes
ownership of R&D sector reform agenda.
4. Assessment of Risk to Development Outcome
Rating: Negligible
58. The risk to development outcome is considered Negligible. All activities are either
completed or sustainable under follow-on projects. Although there was a change of Government
during project implementation, the risk that the program would be derailed did not materialize.
Sustainability of the pilot’s achievements has been further enhanced by the Government and the
World Bank’s commitment to further support the PDO through the SRITTP and C&JP.
5. Assessment of Bank and Recipient Performance
5.1 Bank Performance
(a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory
59. The World Bank’s performance for quality at entry was rated Satisfactory. The World
Bank heavily leveraged top-notch global expertise and lessons learned from multiple post-
transition economies to recast the innovation policy discourse to focus on the role of the
Government and entrepreneurship in building that national innovation system. Additionally, it
emphasized the importance of piloting and stronger than usual M&E practices for this
institutional capacity-building project.
(b) Quality of Supervision
Rating: Highly Satisfactory
60. Quality of supervision was far more intense than a regular World Bank project and very
meticulous given the pilot nature of the project and first time relationship with the donor.
Supervision missions were carried out at least twice a year and up to five times in the early
stages of the project owing to the flexibility afforded by the World Bank-executed TA
component. The World Bank team ensured continuity in project management and a strong
thematic and operational skills mix with strong support from staff in the local office. The World
Bank-executed component enabled mobilization of leading global practitioners in their
respective fields, thus earning the trust and respect of the research community. The team was
proactive and responsive to the multiple GoS requests for change during project implementation
and demonstrated flexibility and understanding in the use of project resources. The World Bank
led project restructuring in consultation with the donor and recipient to maximize achievement of
21
the PDO. On fiduciary requirements, the performance was Highly Satisfactory with FM,
procurement, and environment activities under the project implemented effectively with
significant capacity building and no major issues.
(c) Justification of Rating for Overall Bank Performance
Rating: Satisfactory
61. Overall, the World Bank was very proactive and responsive to the client’s needs from the
design stage through to project closing, with close supervision and excellent support from the
country office. The World Bank team also played a very important role in establishing the IF’s
capacity by offering its own experience and guidance. Based on the above, the overall World
Bank performance in ensuring quality at entry and quality of supervision is rated Satisfactory.
5.2 Recipient Performance
(a) Government Performance
Rating: Moderately Satisfactory
62. The recipient worked closely with the World Bank during project preparation in a
satisfactory manner. During implementation, the recipient could have been more proactive in
providing support to the IF (especially its operating budget), and introducing some quick win
policy recommendations resulting from the RDI TA program. Although the Recipient complied
with all covenants under the Grant Agreement and development outcomes were met by the
project, the shortcomings in budget support to the IF slowed down implementation progress in
early 2014 and destabilized the IF’s operations. Therefore, the overall performance of the
Recipient is rated as Moderately Satisfactory.
(b) Implementing Agency or Agencies Performance
Rating: Highly Satisfactory
63. The IF as the implementing arm of the Recipient was very effective in providing efficient
implementation, monitoring, and reporting, allowing for better management and implementation
of the project and facilitating the World Bank’s supervision activities. The IF was staffed with
qualified experts, selected through a competitive process and adequately compensated for their
work. Although there were teething problems such as the absence of a permanent managing
director and lack of timely GoS budget support on multiple occasions, the IF staff have done a
remarkable job in implementing the project and achieving the PDOs. The IF staff should be
commended for the professionalism exhibited in the discharge of their duties and the
performance is rated Highly Satisfactory.
(c) Justification of Rating for Overall Recipient Performance
Rating: Satisfactory
64. The overall Recipient performance is rated Satisfactory. The success of the project was to
a large extent due to the efficiency and professionalism demonstrated by IF staff in the
implementation of the program and they should be highly commended. The Recipient, although
22
committed, could have been more proactive in providing budget and policy support to the IF,
RDIs, and the sector overall.
6. Lessons Learned
65. Governance arrangements are key for institutional integrity and program delivery. Governance mechanisms established at the IF were underscored by the transparent selection
process that placed a premium on merit and commercialization, and integrated feedback from
international peer reviewers, decision making by an independent IC and continuous guidance
from a professional global advisory team. This positioned the IF as an institution with significant
credibility and capacity to absorb and distribute EU funds in a thorough and independent
manner. This aspect was confirmed by beneficiaries, who commended the transparent and
independent process and the mentoring received from peer reviewers.
66. Institutional capacity matters for delivery of innovation programming. A well-
functioning national innovation agency, staffed with competent and committed people and fairly
remunerated is essential for implementation of a complex project, such as this, and is an
important element of the national innovation system. The IF is an institution that is well regarded
by its beneficiaries and its peers, and should be nurtured to continue to serve the country and
remain agile in an ever changing global innovation landscape. This will mean additional rounds
of capacity building and staff renewal to meet new client and market demands. This should
remain part of the IF’s strategic planning efforts.
67. Horizontality of innovation policy and programming matters in nascent ecosystems.
One of the principles of design considered important in nascent post-transition innovation
ecosystems—dominated by entrenched state enterprise and RDIs—is horizontality of the
innovation policy and interventions, that is, sector agnosticism. At entry, many assumed that the
deal flow would be limited to innovative projects in ICT and digital technology from Novi Sad
and Belgrade. The pilot—by adhering to horizontal selection criteria that emphasized merit and
commercialization potential—revealed a diverse pipeline in terms of sectors, cities, and gender.
68. Size of innovation effort matters and so do rhythm and scale. The 51 projects the IF
financed over four years was too small a number to have an economy-wide impact, but
manageable for capacity building. However, to move the needle at an economy-wide level, the
independent evaluation estimates that the IF would need to provide funding to 200–300 projects
annually. Having established the IF and piloted instruments that demonstrate additionality for
enterprise innovation, the GoS should ensure continuous uninterrupted support to retain IF
capacity, generate innovative entrepreneurs, rebalance the enterprise innovation system, and
continue to position Serbia as a budding knowledge and IP creator in the global innovation
landscape. Losing the momentum generated would be ill advised and create significant loss of
trust in public institutions and could undermine future efforts.
69. Visibility of innovation activities really matters. World Bank operations could take a
serious lesson from the EU’s approach and benefit from a more systematic approach to providing
visibility to project activities with stakeholders, sharing lessons learned, and celebrating
successes. The Project Opening Ceremony engaged the IF’s advisory and management teams,
and its IC to showcase the capacity engaged and discuss intentions transparently, thus building
credibility among the sceptical research and entrepreneurial communities. The Closing
23
Ceremony—a celebration of the journey the Serbian ecosystem had made over the course of the
project—displayed the institutional capacity of the IF and RDIs, the remarkable 50 plus
entrepreneurs who demonstrated their firms’ prototypes and viable products, as well as the
investor groups and entrepreneurship support institutions that did not exist at the onset of the
project. The unrehearsed testimonials provided by these ecosystem actors during visibility events
were picked up by policy makers and donors and are informing future decisions on financing.
70. Support for the evolution of an innovation ecosystem is critical. In the early days of
the project, the IF was a lonely advocate for the ecosystem and would often come under undue
scrutiny and criticism. Eventually the strong entrepreneurial pipeline attracted regional partners
and global investors, making it easier to advocate for support to other actors and promote long-
term symbiotic relationships in the ecosystem.
71. The policy world is changing fast. The innovation arena is witnessing an increase in
well-networked private sector investors and support institutions globally, including in incubation,
acceleration, and financing. Policy makers are beginning to review policies and adopt legislation
to enable crowd funding, which could further democratize early-stage innovation finance. While
not an immediate concern for any of the post-pilot operations, the GoS and IF should familiarize
themselves with these developments and policy implications to ensure relevance of future design
and implementation.
72. Flexibility during design and implementation are key for pilots. During design and
especially during implementation of this pilot, the World Bank team worked closely with the IF
to monitor project progress. When issues were encountered, the teams worked quickly to find
solutions. This flexibility allowed for smoother and efficient implementation. A few examples
include (a) design of the mentoring program to accommodate mandatory and customized training
elements for IF staff and beneficiary entrepreneurs to be more useful; (b) deploying the Mini and
Matching Grant programs sequentially so the IF could incorporate lessons from the first call of
the Mini Grant Program for subsequent programs; (c) increasing the private sector contribution
(from 25 percent to 30 percent) under the Matching Grant program; (d) converging on design
details for the royalty scheme; and (e) engaging commercialization brokers to work with RDIs to
identify commercialization wins within the lifetime of the project.
73. Incentives Matters: Entrepreneurs must have “Skin in the Game”. The requirement
for the beneficiary entrepreneurs to provide a match in case only (not in kind) and the success
royalty component weeded out several serial ‘grant writers’ during the project. This scenario is
known from international practice and hence influenced project design and played out in Serbia.
74. Building relationships with development partners takes times but is worthwhile. This was the first time in Serbia that a Trust Fund Agreement was forged between the Bank and
the EU Delegation. As both organizations were unfamiliar with policies and procedures of their
counterparts, this led to some delays during project preparation. However, once both
organizations were comfortable with each other’s policies and procedures, project
implementation progressed smoothly. It would be prudent that for future joint collaborations to
allow some lead time for project preparation so that any procedural issues could be sorted out
and not delay project preparation. Additionally, the Bank team ensured that it met with the EUD
24
counterparts during each mission to allow maximum information sharing on project progress and
any challenges that could be jointly addressed in bilateral conversations with the GoS.
7. Comments on Issues Raised by Recipient/Implementing Agencies/Partners
(a) Recipient/implementing agencies: please see Annex 7 for report and statement from IF.
(b) Co-financiers: please see Annex 8 for statement from EUD.
(c) Other partners and stakeholders: Please see Annex 8 from key partner: Belgrade Incubator
25
Annex 1. Project Costs and Financing
(a) Project Costs by Component (in Euro)
Appraisal Estimate
Components Project Cost
(€, millions)
Grant
Financing (€,
millions)
Government
Financing*
% of EU
Financing
Component 1. Capacity Building of the
Serbia Innovation Fund 3.21 1.11 2.10 35
Component 2. Implementation of Financial
Instruments Supporting Enterprise Innovation 6.00 6.00 0.0 100
Component 3. Provision of Technical
Assistance to Research and Development
Institutes (RDIs)
0.70 0.70 0.0 100
Total base cost 9.91 7.81 2.10
Contingency 0.12 0 0.12
Total project cost 10.03 7.81 2.22 78
World Bank administrative fees (7%) 0.59
Total EU Grant 8.40
Final Project Cost
Components Project Cost
(€, millions)
Grant
Financing (€,
millions)
Government
Financing*
% of EU
Financing
Component 1. Capacity Building of the
Serbia Innovation Fund 3.82 1.06 2.76 28
Component 2. Implementation of Financial
Instruments Supporting Enterprise Innovation 6.51 5.41 1.1** 83
Component 3. Provision of Technical
Assistance to Research and Development
Institutes (RDIs)
0.80 0.80 0 100
Total Project Cost 11.13 7.27 3.86 65
World Bank Administrative Fees (7%) 0.59
Total EU Grant 7.86***
Note: *Government financing covering period from 2011–2015 (funds are expressed in € using middle exchange
rate of National Bank of Serbia on the last day of each year). Initially, project closing date was set on November 30,
2014. However, it was extended to January 10, 2016, which caused increased Government financing (Government
financing in 2015 equals around €534,000).
**Contributions from the Mini and Matching Grant private sector enterprises were at least €1.1 million.
*** EURO 0.655 million was cancelled.
(b) Financing
Source of Funds
Appraisal
Estimate (€,
millions)
Actual/Latest
Estimate (€,
millions)
Percentage of
Appraisal
EU Grant 8.40 7.86 94
Government Funds 2.22 3.86 175
26
Annex 2. Outputs by Component
Component 1: Capacity Building of the Serbia Innovation Fund - Recipient Executed
1. The IF was to be established with the objective of encouraging entrepreneurship and
managing financing for innovation, as well as participating in co-financing of programs, projects,
and other activities organized by international organizations, financial institutions, and the
private sector. This has been fully achieved. Under this component, the IF developed the
requisite capacity for designing and managing grant instruments geared toward technology-based
firms. Importantly, the IF has built and retained an impressive international team, including
strategic, operations, and regional advisors, in addition to strong governance practices espoused
by its international diaspora-based independent IC.
2. The major networking and educational events held by the IF were project launch and
launch of the Mini Grants Program, followed by the launch of the Matching Grants Program, the
IP workshop (for RDIs and key stakeholders), WB EDIF workshop (for RDIs and private sector
and key government stakeholders), and four info-days for the awardees (addressing project
implementation and training held by Serbian IP office), presentation of the IF Annual Report,
knowledge economy workshop, industry and RDI consultation regarding the new technology
transfer project design, consultations with local Technology Transfer Offices on the cooperation
and functioning of centralized TTF.
3. The IF also provided several training programs for its staff and beneficiary enterprises.
One of the essential services the IF provided to its grant awardees was Enterprise Training,
which was conducted by international advisors covering topics of innovation, business formation
and expansion, IP protection and commercialization, investment readiness, and other valuable
themes associated with early-stage technology and business development. As part of its program,
the IF also conducted several project visibility events and workshops. Overall, this component is
considered highly satisfactory for the establishment of the IF, with its remarkable governance
and the institutional capacity.
4. Staff training for the IF. The following training courses were attended by IF staff.
Preparation and management of the World Bank-funded projects (World Bank
office, Belgrade). IF staff received training on all aspects on the preparation and
implementation of World Bank-funded projects.
Management of innovation and technology support programs (at BICRO,
Croatia). Training provided at BICRO was expected to introduce IF staff to BICRO
programs and organization, management of calls for financing, due diligence, M&E.
Introduction to programs for supporting operation of incubators and technology
transfer programs was also provided.
FM (Turin, Italy). Training was provided to the financial manager to design and
operate the FM systems and execute corresponding payments/disbursements in
accordance with sound professional standards and the harmonized requirements of
donors and their respective governments.
27
Venture capital (VC, Tel Aviv, Israel). Training was provided to acquire critical
skills in five key areas: (a) interpreting governance diagnostics and political
economy analyses, (b) creating multi-stakeholder coalition building strategies and
tactics to support reform, (c) providing communication skills that support the
implementation on governance reforms, (d) leveraging social/digital media tools and
analytics effectively, and (e) developing communication metrics and applying M&E
frameworks. Training was attended by the IF’s interim managing director.
VC (Tel Aviv, Israel - two times for staff). Training consisted of introduction to
organization, operations, and principles of private sector VC funds, transaction, and
due diligence process for early-stage financing. Training was useful for the IF
managing director and program managers to structure and implement privately
managed/publically funded equity instruments and to efficiently connect Serbian
companies with private sector VC funds.
World Bank procurement and FM procedures (Sarajevo, Bosnia and
Herzegovina). Introduction to the World Bank procurement and FM procedures and
guidelines was provided. Training was attended by the procurement associate and
financial manager.
Innovation funding programs (Helsinki, Finland). Tailor-made training for the IF
managing director and program managers at TEKES and SITRA in Finland was
provided to get an overview of financing mechanisms for private sector companies
and enterprise R&D as well as fund management in Finland; applications evaluation;
monitoring, funds management; and programs for large companies.
Program management (Turin, Italy). Two-week training on project and program
cycle management based on the logical framework approach, including project
identification, stakeholder analysis, project design, and the development of M&E
systems was provided. Training was attended by the IF’s senior associate.
Represent. The IF’s public relations (PR) person and a person in charge of
managing the social network pages (LinkedIn and Facebook) have received training
to provide knowledge on the most important aspects of presenting the institution
through social network profiles/pages.
Training on EU PRAG procedures. IF employees have attended the tailor-made
training designed to provide better understanding of the PRAG procedures that was
required in the implementation of the projects financed by IPA funds.
PR training. IF employees received PR training from the World Bank
representative (Belgrade office) in Belgrade.
5. A critical ingredient for the success of the project was that a responsible governance
mechanism was established at the IF, underscored by the transparent selection process that
placed a premium on merit and commercialization potential of projects, and included inputs from
international peer reviewers and decision making by an independent IC. This has positioned the
28
IF as an institution with significant credibility and capacity to absorb and distribute EU funds in
a professional and independent manner.
Component 2: Implementation of Financial Instruments Supporting Enterprise Innovation
- Recipient Executed
6. The IF started its operation on December 5, 2011, as Serbia’s main agency for innovation
support through two main programs: the Mini Grants Program and the Matching Grants
Program.
7. The purpose of the Mini Grants Program was to expand opportunities for tech-based
entrepreneurs by stimulating commercialization of R&D and the creation of innovative
enterprises based on knowledge through start-ups and/or spin-offs. The Matching Grants
Program aimed to expand collaboration opportunities for innovative micro and small companies
with strategic partners (for example, private sector industry, R&D organizations, and VC/private
equity funds) with the goal of increasing private sector investment in R&D and the
commercialization of projects promoting new and improved products and services.
8. The financing offered by the Mini Grants Program covered a maximum of 85 percent of
the total approved project budget and up to €80,000 for a one-year project. The reminder of the
total approved project budget was to be secured by the applicant from other, preferably private
sector sources, independent of the IF.
9. The financing offered by the Matching Grants Program covered a maximum of 70
percent of the total approved project budget and up to €300,000 for a two-year project. The
remainder of the total approved project Budget was to be secured by the applicant from other
sources independent of the IF such as private sector industry, private investors/VC/private equity
funds or the applicant’s own internal resources.6
10. Financing decisions were made on a competitive basis by the IF’s independent IC. The
number of awards was determined by the quality of the proposals and subject to the total funds
available (allocated) to the programs. In addition, these programs provided support for FM, IP
protection, and business development training, as well as mentoring of the awardees to enable
them to prepare for the next stage of development.
11. Between January 31, 2012 and October 28, 2013, the IF managed four financing calls
during which 326 Serbian enterprises submitted applications for either the Mini or Matching
Grants Programs. The IF held about 20 visibility events per call to promote the CFPs and to
explain the Mini and Matching Grants Programs. Some 91 firms applied to the Matching Grants
Program, while 247 firms applied to the Mini Grants Program (12 firms applied to both the
6 Upon successful technology commercialization, the revenue (and not just the profits) derived from the sales of the
product/service and any or subsequent products/service based on the technology developed in the Matching project
financed by the IF, become the basis for a royalty payment in compensation for the IF financing, at a rate of 5
percent of annual revenue derived from sales of product or service or 15 percent of licensing revenue derived from
such product/service up to 120 percent of financing received from the IF for a period of up to five years after project
completion, whichever is achieved first.
29
Matching and Mini Grant Programs). Overall, 471 projects were submitted by the 326 firms. A
total of 55 projects were approved for IF support, out of which 52 projects were financed (out of
the 471 projects); a 12 percent approval rate. Details of the selected awardees from the last CFPs
and their achievements can be found in project files. Some samples of Mini and Matching Grants
Awardees are stated below:
Mini Grants
12. 365ads. The project ‘Brides2bride’ (B2B) was developed by a Belgrade ICT start-up
‘365ads’. ‘Brides2bride’ was a new platform through which ex brides—sharing their own
experience—help future brides organize their weddings. B2B connects different actors in the
process—former brides enter the world of wedding organization by recommending companies
they were satisfied with, future brides get free help in wedding organization and first-hand
information, and companies get the best possible advertisement, direct recommendation. IF
financing, €76,460.
13. COFA Games. The Belgrade-based company, COFA Games, operating in the gaming
industry, developed a new multiplayer, action real-time strategy video game titled ‘Awakening of
Heroes. The game will be deployed to different operating systems and will also enable players to
access the game through different devices, such as tablets, desktop computers, or Oculus Rift VR
Headset. The game was built using the Unity 3D platform, which supports browser games, so
that it can be played anywhere and does not require additional installation. IF financing, €80,000.
14. Coprix Media. The company has profiled itself into a specialized ICT-based education
aid company, with their latest project (following ‘Interactive spelling book’ and ‘Interactive
mathematics’) being ‘Interactive the world around us’. Through it, the company created a new
application for touch-sensitive devices (tablets, smart phones), with the result being a new
gamification-driven learning tool for preschool and primary school children. The project’s main
aim is to help keep education interesting and technologically updated for the generations to
come. IF financing, €80,000.
15. Eipix Entertainment. A young, but rapidly growing company from Novi Sad developed
a new free-to-play (F2P) video game that combines hidden object puzzle adventure play style
with rich interactive content, set in a story-driven environment. F2P is a very popular approach to
modern gaming, represented in digital hybrid form and usually consisting of freemium (full
product for free for nonpaying users, and a premium version for paying users), micro
transactions, and a virtual economy. IF financing, €70,880.
16. Ekofungi. The new mushroom-growing technology that was developed by the Belgrade-
based company, Ekofungi, yields an increase in quality of the mushroom’s fruit body, and also
results in a significant reduction or even complete elimination of harmful compounds used
during the growing of mushrooms. Due to the absorption of various minerals from the substrate
and a substantially greater presence of microelements (for example, selenium, magnesium), as
well as use of other natural compounds, this new type of mushrooms will have a much higher
nutritional value and will be devoid of mycotoxins that can harm humans. IF financing, €72,580.
30
17. Euro Cal. From the city of Kragujevac, Euro Cal developed a new type of ultrasonic heat
meter that finds application in chemical, processing, food, pharmaceutical, and various other
industries. The company’s new product will be manufactured in Serbia and is expected to offer
several competitive characteristics: energy efficiency, increased functionality, high
manufacturing quality, and an affordable price compared to other ultrasonic heat meters already
offered on the market. IF financing, €80,000.
18. Novel IC. The young Belgrade ICT company, Novel IC produced a new solution to help
people with visual impairments. The system, which is embedded in a spectacle-shaped frame,
will help users to detect the distance and speed of objects that are in nearby surroundings by
using the integrated radar sensors (60 GHz RF ASIC). This new solution will also find its
applications among individuals functioning in dark and fog-like environments. IF financing,
€79,710.
19. NS Web Development. NS Web Development’s ICT project named ‘Clinic in Cloud’
represents a new business framework for patient and appointment management systems. The
most important modules of the project are patient data, appointment and insurance policy
management, as well as data security and advanced reports. This software-as-a-service solution
is designed for various clinics and the approach is to be piloted in Serbia before moving into the
international market. IF financing, €61,900.
20. Projektinženjering Tim. Throughout their new project, the construction engineering
company ‘Projektinženjering Tim’ developed an innovative system for seismic protection of
buildings during an earthquake. The system is based on new GOSEB technology that will enable
efficient multilevel and multidirectional protection of the entire buildings. Using special
technical solutions, the system is designed to optimally disperse and reduce the shock energy
(forces) in all directions, as well as to adapt its own vibration features depending on the intensity
of the earthquake. The company is planning to export the solution to various international
markets. IF financing, €78,700.
21. Unitehna Alati from Kruševac developed a new, cost-efficient device for green pruning
with improved characteristics in optimal field of visibility, pruning accuracy, and responsiveness
to commands. The improvement will be achieved by employing an innovative pneumatic
controller system for the device, instead of relying on traditional hydraulic solutions. The device
is easily adaptable to different models of tractors and is expected to be 50 percent lighter and
cheaper than the products of the competition. IF financing, €80,000.
Matching Grants Program
22. Techno Naiss Group. Hailing from the city of Niš, ‘Techno Naiss Group’ developed a
new model of X-ray collimator with a durable, affordable, and modular mechanism for automatic
filter replacement. The effective electromechanical solution behind the new filter replacement
mechanism eliminates the need to have manual operators handling filter changing. Furthermore,
it will be possible to apply the filter replacement system to other existing types of collimators,
offering greater versatility to the project. Relying on their globally present partner ‘X-Alliance’,
Techno Naiss Group plans to sell the new collimator model in various international markets. IF
financing, €259,959.
31
23. BSK. The main activities of the company are design, manufacturing, and sales of
innovative products based on technologies in the field of hydrodynamics. One of the innovations
of BSK is the Air Berry Harvester KOKAN 500S, intended for picking of raspberry, blackberry,
blueberry, and currants. It uses the air current’s force to pick the fruit off the plant, then purifies
it from the dried leaves and twigs and places the fruit into crates for storage. The project is
completed. After setting up a new time frame following the setback caused by last year floods,
the company has completed the building of the Air Berry Harvester KOKAN 500S, for picking
fruits (raspberry, blackberry, blueberry, and so on). This new prototype will be used as a model
for commercial production. First unit sale has been reported. IF financing, €282,669.
24. Vizus. Vizus operates from Niš and it is a producer of aluminum and aluminum-wood
windows and doors. The company offers an innovative solution for manufacturing of high
efficiency windows and doorframes for passive construction with low coefficient of thermal
conductivity, increased durability, and higher resistance to penetration of water and air, and
increased stability. The company successfully developed innovative solution for manufacturing
of high efficiency windows and doorframes. The serial production was set up during the
beginning of 2015. Interest was generated from potential partners from Germany and the United
Kingdom. First sale of the new windows stared in June 2015. IF financing, €246,365.
25. Techno Foam. The company is operating in the machine industry, developing
specialized machines for the needs of companies dealing in sponge production. Techno Foam is
designing and developing a new machine for cutting, profiling, and packaging of sponges. The
company has completed prototyping and establishing the production of the new automatic
machine for cutting, profiling, and packaging of sponges. Development of this automatic
machine is a continuation of the Mini Grants Program that was completed in 2013. Interests for
purchase of the machine have been expressed from companies abroad: United States, Argentina,
and the United Arab Emirates. IF financing, €296,740.
26. Milinković Company. The company is improving a new integrated system of
construction, using prefabricated building blocks (‘big bricks’) to construct multipurpose halls of
various dimensions. Their solution represents a quick and simple way of construction of long
lasting, durable objects, and results in energy-efficient objects of high quality, which are
simultaneously resistant to storm winds, fire, and earthquakes. The company has successfully
completed its project of improving a new integrated system of construction, using a prefabricated
building block. Final certification was also completed within the project implementation period.
It has secured local contracts for the new product. IF financing, €183,070.
Component 3: Provision of Technical Assistance to Research and Development Institutes -
Bank Executed
27. The work on Component 3 started with the first visit of the World Bank team to Serbia in
December 2011. The team discussed key issues, challenges and opportunities in the Serbian RDI
sector, and the proposed approach and gauged interest of RDI management to participate in the
TA program. This was to be followed by building support for the action plans, dissemination
workshop, and assisting RDIs’ management during implementation of the respective action
plans. The work on a policy note outlining systematic RDI sector reform issues started around
32
mid-2012 and was completed in October 2013. Highly qualified international technical experts
were engaged as needed and when specific needs of RDIs were identified or requested by RDIs.
28. Prevailing R&D environment. This work was launched under prevailing difficult
conditions in the Serbian R&D sector. There were serious budget constraints nationally affecting
the R&D sector. This was compounded by the frequent changes in ministry leadership (during
2011–2014 there were four ministers), and a lack of policy stability and commitment to
implement any reforms. Further, there was no reliable information available on the RDIs quality
and volume of R&D outputs and their overall performance. Further, it was expected that the
ministry would undertake systematic R&D sector reforms and request a US$100 million World
Bank project. Both these critical initiatives were not pursued by previous ministry leadership, but
were followed up by the subsequent administration in 2015.
29. Key R&D sectoral issues. Initially, RDI management teams identified a number of
issues in the Serbian RDI sector. These included brain drain of scientists, aging of the scientific
community, difficulty in hiring of young researchers, limited size of the Serbian market for their
research services, difficulties in penetrating external markets and funding, weak managerial and
business-related skills, a lack of culture and skills related to the protection of IP, technology
transfer and commercialization, spin-offs, shortage of financial resources, inadequate laboratory
infrastructure, perverse incentives to conduct applied R&D, and so on. While some of these
issues were common to most RDIs, some RDIs seem to be doing quite well in several areas. For
example, institutes such as the Mihajlo Pupin Institute (MPI), Institute of Field and Vegetable
Crops (IFVC), Maize Research Institute (MRI), FINS, and IMPR were mostly self-sufficient,
earning revenues from the market and receiving only a fraction of support from the ministry in
the form of project grants for specific purposes.
Selection of RDIs
30. The RDIs to be assisted were identified jointly by the ministry and the World Bank team
keeping in mind (a) interest of RDIs management to participate in the TA program, (b)
commitment of RDI management for reform and potential for improvement, (c) diversified
sectoral coverage, and (d) resources under the project. The demand from RDIs to participate in
the TA program was strong, as many RDIs wanted to benefit from World Bank assistance. At the
request of RDIs’ management, the World Bank team agreed with the ministry to include the
following RDIs (more than originally anticipated) to benefit from the TA program.
Category 1 RDIs (detailed TA)
(a) IPB, Belgrade
(b) FINS, Novi Sad
(c) IMPR, ‘Josif Pančić’, Belgrade
(d) IMGGE, Belgrade
33
Category 2 RDIs (limited TA focused on institutional improvements)
(a) MPI, Belgrade
(b) Institute of Chemistry, Technology and Metallurgy (ICTM), Belgrade
(c) Institute of Biological Research (IBR), Belgrade
(d) VINCA Institute of Nuclear Science (VINCA), Belgrade
(e) IFVC, Novi Sad
(f) MRI, Zemun Polje
31. Category 1 RDIs (detailed TA). Scope of work for Category 1 RDIs included
conducting a detailed assessment, carried out jointly by international experts and RDI
management. This involved an assessment of the RDI’s organizational structure, human and
budgetary resources, researcher promotion and incentive system, infrastructure and laboratory
facilities, research capabilities and outputs, IP protection, challenges and potential for applied
R&D, technology transfer and commercialization, and so on. Based on this diagnosis, technical
support was provided for the design and implementation of a RDI-specific upgrading program
(action plan). This included technical expertise, training workshops, and advice, but no support
for the upgrading of labs and other infrastructure. The four selected institutes reflected a
diversified portfolio. Key differences between them include the share of basic versus applied
research conducted at a given RDI (IPB is almost entirely basic research, while IMPR is
predominantly applied); IPB is almost fully state-financed, while IMPR is predominantly
commercial income-based by selling its own products. The IMGGE is a mix of basic and applied
R&D as is the FINS.
32. Category 2 RDI (limited TA). This work included design and delivery of training
workshops on multiple cross-cutting topics of interest to RDIs. The topics included IP;
technology transfer and commercialization; training of research managers; and market
development, collaboration, and EU-Framework Program funding.
The Approach
33. The World Bank’s approach was based on strong participation of all key stakeholders,
both during the assessment and design of RDI sector reforms. These included lead ministry
officials, EUD, Serbian Academy of Science and Arts, other ministries and public sector
agencies, RDIs management and researchers, private sector, nongovernmental organization, and
relevant international organizations such as the Global Research Alliance (GRA).7 The work
7 The GRA, a virtual alliance, is a network of nine of the world’s most prestigious knowledge-intensive R&D and
innovation organizations with a goal of creating ‘A Global Knowledge Pool for Global Good’. The GRA has a
capacity to undertake projects with a magnitude and complexity that transcend the capabilities of any single
organization. It includes organizations of the northern and southern hemisphere, with more than 60,000 scientists
and engineers. The alliance partners perform basic and applied research, technology transfer and commercialization,
and specialize in the implementation of innovative commercially and socially viable solutions. The GRA has within
34
involved analyses of available data supplemented by interviews with senior management and
focus groups, and discussions with the ministry and industry officials.
34. The World Bank team engaged an extensive roster of reputable global experts and
practitioners, most of whom had managed R&D organizations and/or designed and implemented
systematic organizational reforms.
Deliverables
35. Component 3 includes the following deliverables:
(a) Assessment reports including action plans for two Category 1 RDIs
(b) Brief diagnosis of four Category 2 RDIs
(c) Design and delivery of in-depth TA to two RDIs
(d) Technical inputs to the Serbian RDI sector reform program
(e) Dissemination and consultation workshop
(f) Input to research sector reforms
36. The project has either met or exceeded expectations delivering a higher level of quality
outputs in comparison with what was expected in the project agreements. Key outputs include
the following:
(a) Assessment reports including action plans for two Category 1 RDIs. The team
has delivered comprehensive assessment reports including actions plans for four
RDIs: (a) IPB, Belgrade, (b) IMGGE, Belgrade, (c) IMPR, Belgrade, and (d) FINS,
Novi Sad. These RDIs are provided with TA by high quality international experts
from relevant fields, who conducted in-depth analyses of each RDI, identifying its
position in the Serbian R&D sector, and Serbian and international business society,
and outlining specific reform programs. Each RDI was provided a comprehensive
but separate, tailored document entitled ‘Challenges and Opportunities’, presenting
the experts findings and advice for further transformation. Customized TA is
provided to each RDI to implement the recommended action plans, tailored to meet
their specific needs and willingness to undertake reforms. Summary information on
four Category 1 RDIs is provided in the attachment.
(b) Brief diagnosis of Four Category 2 RDIs. This work provided input to the design
of training workshops on cross-cutting topics of interest to most RDIs. The topics
its ranks highly innovative members who among other breakthroughs have developed an MP3 player, Xerox
copying technology, carbon fiber composite wings for light combat aircraft, a space satellite, drought resistant millet,
mesh networking for rural internet provision, the world’s first flu drug, low-cost HIV and tuberculosis drugs, an
open-source drug discovery platform, and so on.
35
included IP; technology transfer and commercialization; training of research
managers; and market development, collaboration, and EU program funding.
(c) Design and delivery of TA to selected RDIs. The expert team designed training
workshops on a number of cross-cutting topics of interest to RDIs, including
Governance and Institutional Management, Program/Project Management, IP
Management, R&D Marketing and Sales, Performance Evaluation and Career
Planning, and Managing/Developing R&D Capabilities. Three training workshops
on these issues were delivered to the participating RDIs and ministry officials. All
ten RDIs and some ministry and industry officials participated in the workshops.
(i) Two-day Workshop on RDI Institute Management, September 2012
(ii) High-level Workshop on Strategic RDI Sector Reform, December 2012
(iii) Workshop on Commercialization and Technology Transfer, December 2013
(d) Technical inputs to the Serbian RDI sector reform program. The Bank team
prepared a comprehensive policy note outlining key RDI systematic sectoral issues
and recommendations, ‘Enhancing the Productivity of Serbian Research and
Development Institutes: Initial Observations and Recommendations’, delivered to
MOESTD in August 2012. At the request of the new minister, an updated policy
note was prepared and delivered to MOESTD in October 2013. These policy notes
analyzed the Serbian RDI environment and identified issues and critical factors and
outlined key recommendations for successful reforms of the RDI sector. These
recommendations have provided a strong platform for the new ministry leadership to
undertake systematic reforms in the R&D sector. Many of the recommendations for
these policy notes have been incorporated in the design of recently adopted Serbian
Strategy for Research and Innovation 2016–2020.
(e) Dissemination and consultation workshop. A dissemination and consultation
workshop on systematic RDI sector wide reforms was organized in December 2012,
with the participation of the MOESTD minister and officials, Serbian Academy of
Science and Arts, RDIs, and industry. Presentations were made by MOESTD, RDI
and academy officials, and technical experts. The World Bank team provided
specific and detailed inputs to the ministry officials for the design of two projects
currently under implementation: (a) the EU IPA SRITTP, and (b) the World Bank’s
C&JP.
Key Performance Indicators and Outcomes
37. The result indicators related to Component 3 included the following:
(a) Number of technologies transferred by RDIs
(b) Dissemination of policy recommendations for RDI sector reform
36
38. The project fully achieved both indicators, as explained in the following paragraphs. The
project outcomes were especially noteworthy, considering frequent changes in the ministry
leadership, their weak capacity and unwillingness to undertake any meaningful reforms, the huge
funding crunch, and postponement of the request for World Bank project support.
39. Overall, the project has a huge positive impact on the general attitude toward RDI reform
including applied R&D, commercialization, and innovative opportunities in general, the area
with great potential, which was (and still is) not properly utilized in the Serbian R&D system.
The new ministry leadership is highly committed to reform the sector and has asked the World
Bank team to assist in designing a comprehensive reform agenda for the R&D sector. They have
already started implementing several policy and regulatory changes.
40. Management of all four Category 1 RDIs expressed deep appreciation for the TA and
opined that owing to the TA they are experiencing positive changes in their institutions, with the
extent of positive changes varying from highest at IPB and lowest at IMPR. RDI management
and a significant number of researchers are energized and motivated to pursue applied R&D and
technology commercialization. They are actively pursuing new nongovernment sources of
revenues and liaising with industry by exploring technology transfer and service opportunities,
both within Serbia and in Europe. They have also increased their efforts to participate in the
European R&D projects. In some cases, RDIs have secured R&D/service project funding from
external sources, including from a large global company and European Patent Office.
Summary of Category 1 RDIs
Institute of Physics, Belgrade
41. IPB is among the largest RDIs in Serbia, employing over 120 PhDs, with an annual
budget of about €4 million, where over 90 percent comes from the Government, through research
projects. IPB was affiliated with about 10 percent of all scientific publications published by
Serbian scientists. Being an almost exclusively basic research-oriented RDI, it surprisingly
expressed the largest enthusiasm toward applied R&D and commercialization, as well as the
institutional changes in general. In accordance with the World Bank team’s recommendations,
IPB management formed an internal team (strategic growth unit), which consists of five people,
and actively supports the project at IPB and continuously acts within the space temporarily
allowed by the legal framework. Through the process, IPB customized the technology readiness
level scale for IPB needs, and established an innovation center. It also developed the long-term
strategy, in which IPB aims to switch from almost exclusively government financing, to one-
third from the Government, one-third from commercial operations, and one-third from externally
financed scientific research, such as EU’s Framework Program or Horizon, and so on.
42. Commercialization. IPB initially identified 14 R&D commercialization proposals,
which reduced to 8 through the review process, and further down to 4 actively participating, after
critical assessment. IPB researchers compiled four new, technologically interesting proposals.
One IPB proposal was rather mature, with the prototype successfully tested in a working
environment. IPB proposals are technologically more demanding, and therefore in most cases
required commercial gap funding to reach technology readiness levels, where the project may be
presentable to an investor. IPB has been successful in achieving a mind-set change of its
37
managerial team and majority of researchers, a very difficult but fundamental element to pursue
any meaningful applied R&D and commercialization program. It has executed knowledge
transfer contracts with private sector firms, including a large multinational company and
increased participation in EU FP programs and outreach to industry has increased its visibility.
The institute has filed European and national patents for its breakthrough currency security
technology (called butterfly). It is also advising smaller RDIs in their transformation process,
including supplying a customized institutional management and monitoring system. IPB aims to
build the capacity (institutional and TRL of projects proposed) to have a solid base to compete in
coming IF CFPs.
Institute of Molecular Genetics and Genetic Engineering
43. The IMGGE was acknowledged as one of the premier R&D institutions in Serbia. It has a
long history, excellent talent, and reasonable laboratory facilities. Its research output (mainly
publications) has been growing, as has the quantity and quality of its collaborations within the
international R&D community. However, in recent times, IMGGE has focused mostly on basic
research, with very limited technology transfer and R&D commercialization outcomes, thereby
limiting its contributions to Serbian enterprises and the overall economy. Further, like many
other RDIs, IMGGE operates under a government policy regime that does not provide stable
base funding to support R&D operations and has limited authority to establish efficient and cost-
effective operations. As a consequence, its governance and management systems are not yet
aligned with international RDI best practices. The World Bank TA to IMGGE although slow at
the start because of difficulties in deploying reputed global experts has been instrumental in
achieving significant institutional-level improvements and mind-set change among its researcher
community.
44. Commercialization. IMGGE researchers submitted nine projects with commercial
innovative potential. After initial assessment, seven proposals were labeled as in the research
phase, as the disclosures resembled applications for further research rather than
commercialization. One proposal was marked as potentially interesting, for further market
analysis. Another proposal identified held potential as a service. IMGGE has entered into a
contract with a private sector company to transfer its probiotic technology and filed European
and national patents. In the knowledge transfer area also, IMGGE has been quite successful, for
example, securing an advisory services contract (over €1.5 million) from the European Patent
Office. IMGGE expects to pursue further knowledge and R&D transfer opportunities.
Institute for Medicinal Plant Research
45. IMPR was identified as a small-size research organization (out of 147 employees, 12 PhD
staff, 69 employees in production plant), established in 1948 to conduct scientific research on
herbal-based medicine. Later on, IMPR started operating its own production facility, which, at
present, provides about 85 percent of IMPR annual income (€3.7 million). Through the years,
IMPR built a reputation with customers in former Yugoslavia, predominantly selling teas and
herbal extracts of good quality at affordable prices. The World Bank team identified a number of
areas for improvement in management systems, as well as in the existing business model
(exploring new areas of research, opening up for private funding, exploring innovative
potentials, optimizing production, reaching out to the international market, and so on). IMPR
38
leadership did not show serious commitment for internal reforms (despite agreeing with the
World Bank team’s recommendations), citing lack of overall GoS commitment to sector wide
reforms. As a result, in early 2013, the World Bank team stopped providing further TA to IMPR.
46. Commercialization. IMPR has over 400 commercial products. Its dominant source of
income comes from the market, so it is successful in commercializing its R&D outputs. It has a
strong reputation in the market. However, the institute is not exploring its full potential or
increasing its product output by focusing on higher value added products.
Institute of Food Technology, Novi Sad
47. The World Bank team delivered a comprehensive report titled ‘Challenges and
Opportunities’, to FINS’ management outlining the current state and advice on short-, mid-, and
long-term actions to be taken to foster innovation to contribute more actively to the Serbian food
industry. FINS was identified as a small-size research organization, counting about 100 staff
members, with 55 researchers (25 hold a PhD degree). The annual income of FINS is about €2
million, with 25 percent coming from commercial operations (mostly services), while the rest
comes from the government-financed research projects. In the past, FINS had active
collaboration with the food processing industry in Serbia, which decayed as production vanished.
The staff and management attitude toward commercialization of research results was supportive,
although they were lacking resources and know-how in technology marketing. As FINS was
highly dependent on government financing, which is exclusively project based, FINS falls in the
category of RDIs whose commercialization potentials are heavily hindered by an inappropriate
financing mechanism. Although having a strategically good place in the Serbian food industry,
well-equipped laboratories, and staff motivated to compete on the market, FINS, as all other
RDIs in Serbia, is financially stimulated only to publish scientific articles. Unfortunately, FINS
management has not shown any serious interest in moving forward with institutional reforms or
supporting significant commercialization efforts. Therefore, the World Bank team scaled down
its TA support to FINS.
48. Commercialization. Following several visits by World Bank experts, FINS researchers
proposed six innovative projects for commercialization, from the fields of food supplements,
functional food, animal feed, and animal feed supplements. Although FINS did not have a
technology readiness level model adopted yet, according to their internal grading, their products
are rather advanced in technology readiness terms, although only one of the projects was in
negotiation with a potential client, which did not materialize owing to typical mismatched
expectations on IP value between the researcher and the prospective buyer.
39
Annex 3. Economic and Financial Analysis
1. Economic analysis was not carried out during preparation or recorded in the PAD, as it
was not required for Track 1 projects. However, below are important extracts from the report
titled ‘Estimation of the Effect of Participating in the IF Financing Program on Firm
Performance’, conducted by an independent consultant. It should be noted that the report was
concluded a few months before project closing in January 2016 and the final data was not
available.
2. Most funding (€7.12 million) under this pilot was executed by the IF and focused on the
‘Implementation of Financial Instruments Supporting Enterprise Innovation’, which consisted of
the Mini and Matching Grants Programs. The purpose and aim of the Mini Grants Program was
to expand opportunities for tech-based entrepreneurs by stimulating commercialization of R&D
and the creation of innovative enterprises based on knowledge through start-ups and/or spin-offs.
The Matching Grants Program aimed to expand collaboration opportunities for innovative micro
and small companies with strategic partners (for example,, private sector industry, R&D
organizations, and VC/private equity funds) with the goal of increasing private sector investment
in R&D and the commercialization of projects promoting new and improved products and
services. Overall, 469 projects were submitted by 326 firms. A total of 55 projects were
approved for IF support (out of the 469 projects); a 12 percent approval rate. These projects were
the subject of the impact evaluation analysis.
Table 3.1. Number of Applications, Program and Call
Applications Projects
Call # Call Date Matching Mini Total Matching Mini Total
1 January 31,
2012 0 57 57 0 11 11
2 August 31,
2012 40 106 146 5 9 14
3 April 15,
2013 39 85 124 5 13 18
4 October 28,
2013 40 102 142 2 10 12
Total 119 350 469 12 43 55
3. Under the Mini Grants Program, the financing was approved to 38 firms, out of which 36
were active at the end of the project period (January 2016). Over 80 percent of firms approved
for Mini Grants were established between 2012 and 2013 and 90 percent were start-ups or spin-
off firms established as a special purpose company to participate under the project. The amount
of funds disbursed for this program was €2.9 million. The contribution made by the participating
firms was at least €0.4 million. The quantitative evaluation for the Mini Grants was based on the
difference between the observed performance outcome of the supported firm (firms that were
awarded grants) and its hypothetical or counterfactual outcome if it would not have participated
under the program. Simple differences in sales changes between firms that received a Mini Grant
and firms that applied but were rejected revealed that the former increased their sales by an
additional €27,000 and employment by an additional five employees, on average. These are very
large effects because the pretreatment sales and employment were zero and zero to one
employees, respectively, for the vast majority of the supported firms. About 75 percent of Mini
40
Grant firms finished their projects as planned. About 16 out of the 38 (i.e. 42 percent) Mini Grant
firms had products that reached the marketing stage and 89 products were introduced by those
firms. Based on these numbers, the additional sales generated as a result of the Mini Grants were
approximately €1 million and it is expected that the investment in this component would be
recouped in a period of three years.
4. Under the Matching Grants Program only 11 firms could receive support as the amounts
awarded were much larger than under the Mini Grant Program. This number is inherently too
low to perform any reliable statistical analysis. Given the small sample, the analysis generated
very imprecise estimates and the independent evaluation report could not formally reject the null
hypothesis that the Matching Grants Program did not have a statistically significant effect on
sales and employment. The report reiterated that this result was likely because of the fact that
there were not enough observations to generate meaningful estimates of the Matching Grants
effect. The amount of funds disbursed for this program was €2.6 million. The contribution made
by the participating firms was approximately €0.63 million. Of the 11 firms, 4 submitted patent
applications, there were 13 collaborations with academia (including foreign universities), and 13
cooperative efforts with industry (23 percent of which were foreign partners). Data was
unavailable to quantify the value of these collaborations.
5. The report concluded that the budget of €6 million for the early-stage grant programs was
too low to have an economy-wide impact. The report estimates that the GoS should resume calls
and allocate at least €30 million per year to support 200–300 R&D projects for economy-wide
impacts. The firm contributions to match the government contributions (based on the above
examples) would be approximately €5.1 million. The €35 million injection into the system could
prove a great catalyst for the sector. The report suggested that the GoS address the ‘Valley of
Death’ and create government support for a second round of investment for IF graduate firms,
(which should at least partially be addressed by WB EDIF going forward).
6. Knowledge spillovers. The GoS should not expect that R&D knowledge, especially from
high-tech industries or services (most of the grants awardees), will drill down and assimilate
quickly in the traditional SME industries. Theory and experience suggests that traditional SMEs
require special attention and resources for spillovers to occur. To avoid polarization of the
Serbian economic structure, the GoS could consider special funding to increase innovative
activities to increase productivity of firms in the traditional industry.
41
Annex 4. Bank Lending and Implementation Support/Supervision Processes
(a) Task Team Members
Names Title Unit Responsibility/
Specialty
Lending
Natasha Kapil Private Sector Development Specialist GTCIE Task team leader
Andrej Popovic Private Sector Development Specialist GFM01
Private sector
development
Hiran Herat Senior Operations Specialist GFM01 Implementation
Vinod K. Goel Science, Technology and Innovation
Advisor GTCDR
Science, technology,
and innovation
Itzhak Goldberg Innovation Policy Advisor Innovation
Kashmira Daruwalla Senior Procurement Specialist GGO03 Procurement
Aleksandar Crnomarkovic Financial Management Specialist GGO21 FM
Esma Kreso Environmental Safeguards Specialist GEN03
Environmental
safeguards
Amy Evans Environmental Safeguards Specialist GEN03
Environmental
safeguards
Lola Ibragimova Social Safeguards Specialist GSU03 Social safeguards
Nicholay Chistyakov Senior Finance Officer Disbursements
Ignacio Jauregui Legal Counsel LEGLE Legal
Supervision/ICR
Natasha Kapil Senior Private Sector Specialist GTCIE Task team leader
Vesna Kostic Senior Communications Officer ECAEC Project Visibility
Jasna Vukoje Senior Program Assistant ECCYU Liaison
Hiran Herat Senior Operations Specialist GFM01 Implementation
Vinod K. Goel Science, Technology, Innovation Advisor GTCDR STI
Kashmira Daruwalla Senior Procurement Specialist GGO03 Procurement
Aleksandar Crnomarkovic Senior Financial Management Specialist GGO21 FM
Nina Rinnerberger Environmental Safeguards Specialist GEN03
Environmental
Safeguards
(b) Staff Time and Cost
Stage of Project Cycle
Staff Time and Cost (Bank Budget Only)
No. of Staff Weeks US$, thousands (Including Travel
and Consultant Costs)
Lending 12.46 77,137.92
Total: — —
Supervision/ICR 53.53 266,563.43
Total: 65.99 343,701.35
Note: Includes budget for the additional financing.
42
Annex 5. Beneficiary Survey Results: Excerpts from an Impact Evaluation
1. Between January 31, 2012 and October 28, 2013, the IF managed four financing calls
during which 326 Serbian firms submitted applications for either the Mini or Matching Grants
Programs. The IF held about 20 visibility events per call to promote the CFPs and to explain the
Mini and Matching Grants Programs. Of the firms, 91 applied to the Matching Grants Program,
while 247 applied to the Mini Grants Program (12 firms applied to both the Matching and Mini
Grant Programs). Overall, 469 projects were submitted by the 326 firms. Of the 469 projects, 55
were approved for IF support; a 12 percent approval rate. These projects are the subject of the
IF’s impact evaluation analysis.
2. The qualitative evaluation of the IF consisted of personal interviews with the IF staff, the
Investment Committee members, several peer reviews, and many firms (both accepted and
rejected from Mini and Matching Grants Programs) as well as on-site visits to several firms and
to the local university.
3. Based on the quantitative evaluation, the main findings are as follows:
The IF is a credible and transparent platform, accessible to many Serbian businesses,
not just to a small group of entrepreneurs.
The IF succeeded in expanding the opportunities for tech-based entrepreneurs by
creating a lot of innovative start-ups/spin-offs. Almost half (156 out of 326) of the
firms that submitted the application for IF financing were created because of the IF
operation. In addition, there is a high survival rate among supported firms.
The IF succeeded in expanding the network of tech-based entrepreneurs by creating
a relatively high amount of collaborations with academia and industry (also
internationally).
Overall project objectives are adequately specified.
The Mini and Matching Grants Programs are clear, accessible, and effective.
Program evaluation and decision process is transparent and trustworthy.
Application process is similar to other programs (internationally).
The IF staff ranks high in professionalism and willingness to help and in flexibility.
The IF has overcome organizational difficulties.
The program’s implementation is effective—most of the projects achieved their
objectives.
4. The quantitative evaluation of the IF’s performance is based on a rich data set that was
gathered using specially designed questionnaires to firms, which had applied for the IF’s support,
as well as supplementary information from the Serbian Business Registry. At this stage,
43
information on Serbian firms from the Amadeus database had proven to add very little, if at all,
to the analysis.
5. The IF effect is defined as the difference between the observed performance outcome of
the supported firm and its hypothetical or counterfactual outcome in the case it would not have
participated in the IF program. The counterfactual outcome was estimated using statistical
techniques. This is done using a propensity score-matching algorithm to match a ‘twin’ firm
from among the non-supported firms to each supported (treated) firm. The matching ensures that
a twin firm is ex ante and is as likely to receive an IF grant as the firm that actually received the
grant. Then a matched difference-in-difference estimator was computed, which is based on the
differences in outcome differences (for example, growth in sales or employment) between the
treated firm and its twin.
6. Simple differences in sales changes between firms that received a Mini Grant and firms
that applied but were rejected reveal that the former increased their sales by an additional
€27,000 and employment by an additional five employees, on average. The evaluation analysis
generates similar results to the raw data: receiving a Mini Grant increases sales by €20,000 and
employment by five employees, on average, relative to what these firms would have experienced
had they not received the Mini Grant. The effect on employment is also statistically
significant, but the effect on sales is not. These are very large effects because the pre-treatment
sales and employment were 0 and 0–1 employees, respectively, for the vast majority of the
supported firms.
7. Only 11 firms received a Matching Grant. This is too low a number to perform any
reliable statistical analysis. Indeed, the evaluation analysis generates very imprecise estimates
and the null hypothesis that the Matching Grants Program did not have a statistically significant
effect on sales and employment cannot be formally rejected. It is reiterated that this result is
likely because there are not enough observations to generate meaningful estimates of the
Matching Grants effect. The results cannot therefore be interpreted as meaning that the
Matching Grants Program had little or no effect.
8. To fully measure the impact of IF grant, there is need for a longer time-horizon. The
evaluation analysis takes place at a point in time which is very close to the completion of the
project (or, more precisely, the part of the project being financed by the IF) and even closer to
the time new products and services were introduced to the market and the beginning of sales.
This may give a relative disadvantage, in the short run, to firms that actually did invest in R&D
to create new products and entering new markets, thereby, underestimating the effect of IF
support.
9. The finding of a positive IF effect on employment in the Mini Grants program is
consistent with increased R&D activity by these firms (as most of the R&D expenditures are due
to wages). The modal number of employees in 2011 among firms receiving a Mini Grant (before
the IF program was established) was zero. These firms were created to benefit from IF support
and are taking their first steps in establishing an R&D program. Simply put, these firms are still
at the inventive stage. Hence, increases in employment have been observed (but not a
corresponding increase in sales), which is interpreted to indicate that the IF has had a positive
effect on firms R&D expenditures (positive additionality of R&D).
44
Our policy recommendations are as follows:
(a) 52 projects in 4 years is too small a number to have an impact on the economy.
Unless the IF reaches a critical mass of 200–300 R&D projects at work annually,
efforts will have little impact on the Serbian economy. The GoS should resume
calls for application submission as soon as possible and allocate a budget of €30
million per year to meet the need for 300 R&D projects at work.
(b) The GoS should consider dealing with the ‘Valley of Death’ to create Government
support for a second round of investment for the IF graduate firms.
(c) The GoS should consider the creation of a special fund to increase the productivity
of firms in the traditional industry.
45
Annex 6. Stakeholder Workshop Report and Results
Press Releases and summaries from Stakeholder Workshops are available on the Innovation
Fund’s website at www.inovacionifond.rs under the Serbia Innovation Project section.
46
Annex 7. Summary of Recipient's ICR and/or Comments on Draft ICR
Overview
1. From December 2011 to January 2016, the IF implemented the SIP the first EU IPA
Grant (€8.4 million) allocated to promote innovation in Serbia and administered by the World
Bank. The main objectives of the SIP are to encourage innovative entrepreneurship, boost
awareness regarding technological development and innovation in the economy, and address
missing elements of the Serbian innovation system, which is likely to be vital for improving the
competitiveness of the enterprise sector and Serbia’s long-term growth prospects.
2. The SIP consists of three core components:
Component 1: Supporting the operationalization and institutional capacity building
of the recently established Serbia IF
Component 2: Piloting financial instruments for technological development and
innovation by enterprises
Component 3: Encouraging selected public RDIs to engage in technology transfer
and assisting in formulating a RDI sector reform policy
3. The IF was responsible for implementation of Components 1 and 2 under the SIP, while
the third component was World Bank-executed.
SIP Implementation
Component 1: Capacity Building of the Serbia Innovation Fund
4. The IF was established in 2005 by the Innovation Law to encourage entrepreneurship and
manage financing for innovation, including participating in co-financing of programs, projects,
and other activities organized by international organizations, financial institutions, and the
private sector. As the SIP was the first pilot project implemented by the IF, one of the initial and
primary goals was operationalization and institutional capacity building of the IF.
5. Capacity building during the SIP focused on IF operational management and
infrastructure, as well as staff trainings on management of innovation and R&D financing
programs. The IF has engaged competent staff (15 permanent employees) and has established an
organizational structure that works effectively on the development of the financial instruments,
as well as the implementation and the monitoring of the awarded projects. Trainings cover
various aspects of the IF operations, both the administrative and business development, such as
management of innovation and technology support programs, management of innovations, FM,
and due diligence in early-stage technology development.
6. Additionally, a strong governance structure and processes in line with international good
practice have been established, based on trainings the IF staff has received during visits to similar
institutions in Israel, Finland, and neighboring Croatia. The IF has established a fully transparent
47
evaluation process that includes international peer review and an independent body IC,
responsible for evaluation of the project’s financing decisions.
7. The IF has also engaged a team of prominent advisors, ranging from strategic, operations,
and regional levels, who assisted and mentored the IF during the SIP implementation and
advised on future programs, strategy, operations, and procedures, in turn strengthening the IF
institutional capacity.
8. As of the SIP closing, the IF capacity is stable and operational even though it is still not
operating at full capacity due to the Government-imposed hiring restrictions and budgetary
measures for public sector organizations. The IF’s organizational structure has continued to
evolve with the introduction of the operations manager and senior legal advisor positions to
ensure expanded program operations (SRITTP, WB EDIF, Enterprise Innovation Fund [ENIF])
go uninterrupted and efficiently. Continued organizational improvements are expected on the
performance-based employee evaluation system front.
Component 2: Implementation of Financial Instruments Supporting Enterprise Innovation
9. With the fundamental TA of the World Bank, the IF has established and piloted two
R&D grant programs: (a) the Mini Grants Program, targeting proof of concept and prototyping
stages among start-ups and (b) Matching Grants Program, targeting R&D in more mature
companies for development of technologies, products, and processes.
10. During the SIP, four CFPs were successfully conducted and the entire €6 million were
committed for the financing of 43 Mini Grants (approximately, €3 million) and 12 Matching
Grant (approximately, €3 million) projects.
11. By introducing the Mini Grants Program, the IF aimed to support early-stage, private, and
micro and small enterprises, which possess a technological innovation that have a potential for
creation of a new IP and clear market need. Following the initial success of the Mini Grants
Program, the IF introduced the second financial instrument called the Matching Grants Program,
designed to help companies struggling to address the significant financial investment associated
with the development cycle and the high cost of translating research into a commercially viable
product. The main goal of the Matching Grants Program is to expand collaboration opportunities
for Serbian innovative micro, small, and medium companies with strategic partners and increase
private sector investment in technology development and commercialization projects for new
and improved products/services.
12. After the initial completion of the first CFP for financing under Mini Grants Program, the
IF ran the CFP for both Mini and Matching Grants Programs simultaneously, by setting the same
deadline for project submission:
January 31, 2012 - CFP 1 Mini Grants
August 31, 2012 - CFP 2 for Mini Grants and CFP 1 for Matching Grants Programs
April 15, 2013 - CFP 3 for Mini Grants and CFP 2 for Matching Grants Programs
48
October 28, 2013 - CFP 4 for Mini Grants and CFP 3 for Matching Grants Programs
13. With respect to the implementation of the awarded project, the IF has been continuously
working on improvement of the GMs, procedures, and program documentation to make the
processes more fluid and easier for the companies. The IF team put significant effort to maintain
the evaluation process, not only fully transparent, but also within the set deadline of 8–10 weeks,
which complies with the some of the best examples of international practice such as Tekes
(Finland) and Matimop (Israel).
14. To complete its core program activities, including contracting of the new awardees and
implementation of the projects from the last CFP as well as disbursing funds to the existing
awardees, the SIP closing date was extended from November 30, 2014 to January 10, 2016. By
this date, all awarded projects were completed.
15. Overall, the implementation of the SIP was successfully conducted and issues that came
up during the implementation were resolved in collaboration with key IF stakeholders
(MOESTD, the EUD, and the World Bank) who also contributed to the IF’s organizational
capacity building. Internal monitoring of the set indicators have shown that all PDOs were
reached and/or exceeded (for an overview of the indicators, please see appendix 1).
Project Outcome (Achievements)
16. Addressing missing elements in the development of the innovation ecosystem.
Aiming to provide necessary funding to the innovative micro and small enterprises, the IF has
addressed some of the missing elements of the Serbian innovation system that are vital for
improving the competitiveness of the enterprise sector and Serbia’s long-term growth prospects.
Even though the financing under the SIP is mainly dedicated to the R&D activities, the programs
provide much needed support in the initial phases and give the companies a chance to focus on
the product/service development. Being the first financial instruments of its kind in Serbia, the
Mini and Matching Grants Programs initiated changes in the local innovation ecosystem and
contributed not only to the expending of complementary elements such as business incubators
and hubs, but also to the emerging of VC funds. Scaling-up of activities established under the
SIP is a step toward a more competitive start-up community, leading to establishing the
principles of the knowledge-based economy.
17. Encouraging innovative entrepreneurship. By providing programs that offer essential
seed financing for innovative projects, the IF has encouraged a certain portion of entrepreneurs
to think about further implication of their technologies. Furthermore, the IF has received
numerous project proposals coming from the university spin-offs, which have shown researchers’
potential and willingness to shift toward market-implicated research. With a total number of 471
submitted applications, of which 421 were eligible for evaluation, Mini and Matching Grants
Programs have shown that there is a clear demand for financing of the innovative SMEs’ R&D
activities. Most of the applications are from the ICT sector, combined with other high-tech
industries, but some are also from traditional industries such as mechanical engineering and food
and agriculture. Besides diversity in the industry area, the IF has also received project proposals
from all across Serbia, showing demand for the Mini and Matching Grants Program financing to
be present across various cities and university centers.
49
18. Enhancing awareness of the importance of innovation. By showcasing the results of
the financed projects and the SIP itself, the IF has been working on boosting the awareness of
technological development and innovation in the economy as a whole. Together with its
stakeholders, the IF has engaged in a number of events and promotion activities aimed to
increase knowledge on the significance of an innovation-driven economy, including recent
events such as SRITTP Launch (February 2015, Belgrade); visit to the Fund for Innovations and
Technology Development of Macedonia (March 2015, Skopje); Start-up Sauna at the Royal
Palace (March 2015, Belgrade); Europe Day Assembly of the City of Belgrade (May 2015,
Belgrade); 59th International Fair of Technique and Technical Achievements (May 2015,
Belgrade); Project Fair ‘The EU and Serbia in Action’ (May 2015, Niš); Local Slush
Competition (August 27, 2015 in Belgrade); Georgia's Innovation and Technology Agency visit
to the IF (October, 2015 Belgrade); and the SIP Closing Event (December 2015, Belgrade), to
name a few.
Project Outputs (Deliverables)
19. Since the start of the SIP, the IF has conducted four CFPs under the Mini Grants Program
and three under the Matching Grants Program, through which it provided financial support for 41
projects of young innovative companies under the Mini Grants Program, together with the 11
projects under the Matching Grants Program.
20. Overall, the awarded projects have shown a high percentage of success in technology
development, with most companies reaching their expected milestones of a working prototype, a
fully developed technology, and/or even a market-ready product.
21. Through four CFPs under the Mini Grants Program and three CFPs under the Matching
Grants program, the IF has supported over 300 jobs within innovative Serbian SMEs.
Additionally, the IF has also supported the engagement of around 90 highly qualified R&D
advisors on these projects. More than 60 persons engaged in the financed projects hold a PhD
degree from various technological backgrounds. The IF’s programs have also proven to foster
the collaboration between the private sector companies and the academia, with more than 20
different RDIs and faculties engaged on the financed projects.
22. Although, it is still early to determine the full impact of the IF’s financing on companies’
results, the awardees from Mini and Matching Grants Programs have already achieved some
notable results:
19 awardees have generated revenue from the sale or lease of their innovative
products.
9 awardees have established international cooperation.
2 awardees have secured distributors for the U.S. market.
On a national level, 33 patent applications have been submitted, together with 10
trademark applications, and 6 applications for authorship protection.
50
To protect their IP abroad, awardees have submitted 13 Patent Cooperation Treaty
applications, 4 EPO applications, 5 U.S. patent applications, along with patent
applications in Australia, Euro-Asia, and Japan.
18 awardees have established new business cooperation or partnerships.
Bank Performance
23. The World Bank team has been very active in supporting the IF team, pertaining to both
the building of overall organizational capacity and designing and managing of the financial
instruments. The World Bank encouraged the IF to take a more decisive role in the
implementation of the SIP and be more proactive in strategic planning, which contributed highly
to establishing the IF’s capacity. Key areas in which the World Bank has provided TA are the
following:
Establishing IF capacity through organizing various in-house training for the IF staff,
as well as working with the IF team on the training plan
Designing of the financing programs, including program documentation
Organizing and managing of the visibility events
Continuation of financing and scaling up of the existing grant schemes
Development of new financing programs
Strategic planning
24. The results achieved during project implementation were in large part due to the
accessibility and support of the World Bank’s task team leader. The World Bank team and their
advisors were instrumental during the drafting of program documentation and highly valuable
during project implementation and improvement of the project processes.
25. The World Bank team was also very determined in securing funds for the next rounds of
financing under Mini and Matching Grants, as well as expending the portfolio of IF programs
through the SRITTP that includes a new collaborative grant scheme and the establishment of the
TTF.
Conclusions and Final Remarks
26. During the period of the SIP implementation, the IF has managed to build capacity and
has established itself as a trustworthy institution, fully capable of absorbing funds and creating
and implementing new projects. M&E analysis of IF’s Mini and Matching Grants Programs
based on indicators defined by the SIP was completed in November 2015 by the consultant
company, Applied Economics (Israel). Key findings of this analysis include that the IF is a
credible and transparent institution, accessible to many Serbian businesses. In addition, the report
identified relatively high levels of collaboration between academia and the private sector
industry (both domestically and internationally) involvement in the beneficiaries’ projects. Mini
51
and Matching Grant Programs were found to be clear, accessible, and effective, with the
application process being similar to other international programs and the evaluation and decision
process being transparent and trustworthy.
27. Following the achievements of the SIP, the IF has been actively working on securing
additional funds to scale up activities under the existing programs to achieve higher impact on
the Serbian economy. This effort resulted in €8.1 million being provisionally allocated to the IF’s
Mini and Matching Grant Programs over the next three years as part of the Loan Agreement
between the World Bank and the Republic of Serbia under the Serbia C&JP. Furthermore,
additional funds are expected from the IPA 2014 pot to be available to the IF in late 2016. With
the prospect of securing funds for resuming the Mini and Matching Grants Programs in 2016 and
with adequate Government support, the expectations are for the IF to remain an integral catalyst
in the Serbian innovation ecosystem.
1
Appendix 1. Performance and Monitoring Indicators
(as per Attachment to Supplemental Letter No.2 of the Project Agreement)
Indicators Number and Definition Cumulative Target Values Actual Values
Frequency Data Source/Methodology YR1 YR2 YR3 YR4 June 30, 2015
Indicator 1: Amount of innovation financing
mobilized by IF in addition to this EU IPA
project
— — €10
million €20 million
WB EDIF €28
million1
SRITTP €6.9
million2
6 months Internal reports of IF
Indicator 2: Number of active technology
start-ups funded 2 5 8 10 38
3 6 months Internal reports of IF
Indicator 3: Number of new products and
processes launched by beneficiary
enterprises
0 2 6 8 194 6 months Internal reports of IF
Indicator 4: Number of technologies
transferred by RDI’s 0 1 2 3 3
5 6 months
Consultants’ reports to IF
and IBRD
Indicator 5: Dissemination of policy
recommendations for RDI sector reform — — —
Disseminated to
stakeholders
Disseminated to
stakeholders
Upon
completion of
TA
Consultants reports,
MOESTD and IBRD
Note:
1. WB EDIF: Support approved by the Western Balkan Investment Framework on December 7, 2011 for ENIF, a regional VC fund. The GoS passed the
conclusion 05 no. 337-195212012 of 22 March 2012, appointing the IF as an organization for implementing the WB EDIF on behalf of the Republic of Serbia
and approved the provision of financial contribution of the Republic of Serbia in ENIF in 2012 by approving the transfer and payment of RSD 21,528,000.00 in
2012 through IF.
2. Budget support for SRITTP, to be financed through IPA 2013 is launched in February 2015 (total project budget is €6.9 million, out of which €4.9 million is
funds of IPA, €1 million budget of the Republic of Serbia, and €1 million private co-financing ).
3. IC decision to finance project proposed by Stamar was revoked by the IC; Project Fastro was terminated; number of Mini Grants projects financed in each of
four CFPs, 10, 8, 13 (excluding Stamar and Fastro), and 8 respectively. Companies DNET Inovacioni Centar, Ekofungi and Coprix Media were awarded a
second Mini Grant and one company that finished a Mini Grant project was subsequently awarded a Matching Grant in one of the later CFPs (Techno Foam).
4. Techno Foam, Strawberry Energy, Angros, Adecom, iStreetLight, Euroheat, Ekofungi, Coprix Media, mBraintrain, Trk inovacije, Spik Briket, Brainstorm,
Novel IC, Eipix Entertainment, Intens Net, Vizus, BSK, TajfunHil, and Morena Inzenjering reported product launch.
5. Spin-off from IMGGE; IMGGE - service licensed/contract research; IPB - product sale (sale started before the implementation of the SIP, but the TA from the
project resulted in the increase).
Intermediate Results (Part A): Built IF’s capacity for designing and managing financial instruments for enterprise innovation, technology transfer and
mobilizing funds
Intermediate Result Indicator 1: Number of
financial instruments designed and 2 2 3 3 4
1 6 months Internal reports of IF
2
Indicators Number and Definition Cumulative Target Values Actual Values Frequency Data Source/Methodology
introduced
Intermediate Result Indicator 2: Number of
IF managers trained and applying newly
learnt skills
4 4 4 4 102 6 months Internal reports of IF
Intermediate Result Indicator 3: Number of
start-ups mentored 2 5 8 10 51
3 6 months Internal reports of IF
Intermediate Result Indicator 4: Mechanism
to support training/mentoring of innovative
start-ups established
No Yes Yes Yes Yes 6 months Internal reports of IF
Intermediate Result Indicator 5: Number of
networking and educational events
organized for enterprises, academia and
research
2 4 6 8 224 6 months Internal reports of IF
Intermediate Result (Part B): Piloted financial schemes for technology development and innovation in enterprises
Intermediate Result Indicator 1: Number of
funded projects 5 10 15 20 52
5 6 months Internal reports of IF
Intermediate Result Indicator 2: Number of
funded enterprises 5 10 10 15 48
6 6 months Internal reports of IF
Intermediate Result (Part C): Assessed a selected number of RDIs and their innovation potential with recommendations for technology
commercialization
Intermediate Result indicator 1: No. of
RDIs identified for technology transfer and
commercialization
3 4 4 4 10 6 months Consultant reports,
MOESTD and IBRD
Intermediate Result Indicator 2: Number of
RDIs assessed jointly by international
experts and RDI management
0 2 2 2 4 6 months Consultant reports,
MOESTD and IBRD
Note:
1. The IF has designed the Collaborative R&D Grant Scheme under the SRITTP, while TTF was established under the same program. It is expected that both
programs will officially commence in the first part of 2016.
2. Total number of IF managers trained from the beginning of the IF operations.
3. All Mini Grants awardee companies have received Enterprise Training together with three companies that were preselected in CFP1. Besides that, the IF has
organized a local ‘Slush’ competition that also included mentoring sessions for participating companies.
4. The major networking and educational events held were project launch and launch of the Mini Grants Program, launch of the Matching Grants Program, the IP
workshop (for RDI’s and key stakeholders), EDIF workshop (for RDIs and private sector and key government stakeholders), and four info days for the 53
selected projects (addressing project implementation and training held by Serbian IP office), four Enterprise Training sessions for awardees, presentation of the
IF Annual Report, knowledge economy workshop, industry and RDI consultation regarding the new technology transfer project design, consultations with local
3
Indicators Number and Definition Cumulative Target Values Actual Values Frequency Data Source/Methodology
technology transfer offices on the cooperation and functioning of centralized TTF.
5. IF terminated the Financing Agreement with one of the CFP4 Matching Grants awardee, the Stan Technologies company, because it was unable to secure its
part of project co-financing, and subsequently, the company requested to be withdrawn from project implementation. On December 18, 2014, the IC made a
decision to terminate the project.
1
Annex 8. Comments of Co-financiers and Other Partners/Stakeholders
Comments from Co-financier: Delegation of the European Union to the Republic of Serbia
As a general comment, this is a very specific project – the first one of this kind in the EU Delegation to
Serbia – that provided significant resources in the form of grants (to the amount of EUR 6 million) to the
innovative private sector companies. Furthermore and as far as I am aware, it was for the first time in
Serbia that start-up companies were eligible for grants as well. Both of these initiatives proved to be very
successful and turned this icebreaker into one of the flagships of the EU funded projects in the area of
Research, Development (R&D) and Innovation.
The piloting of financial instruments for technological development and innovation in enterprise, in the
most transparent way, was rather revolutionary for a Serbian innovation market. At the same time, the
capacity building of the Innovation Fund was not less challenging. Although the Innovation Fund was
established by the Innovation Law in 2006, it became operational only in 2011, with resources provided
from the European Union through the Instrument for Pre-Accession Assistance (IPA) and under the
umbrella of the Serbia Innovation Project.
Finally, the customized technical assistance was successfully provided to selected Research and
Development Institutes in Serbia.
All of the above was achieved in a very difficult and challenging political and economic environment.
During the project life cycle, three Ministers changed in the Ministry of Education, Science and
Technological Development. These changes included significant fluctuations at the level of State
Secretaries and Assistant Ministers as well. The political instability was followed with budgetary
constraints and delayed funding. At one point, the employees of the Fund were without salary for three
months. Lack of general understanding of importance of innovation and R&D for the economic recovery
was always present at various levels. On the other side, three Acting Directors of the Innovation Fund
have also changed and the Fund was understaffed most of the time.
Piloting this intensive project, with two different grant schemes and EUR 8.5 million budget, under the
above mentioned circumstances was rather difficult, to say the least. It was even more demanding if we
have in mind the specific reputational risk of the EU as the biggest donor in the region and the World
Bank.
Apart from the excellent results achieved during the project implementation – and most probably because
of them, the Serbia Innovation Project laid ground for further interventions in this area. In that regard, we
are already implementing the Serbia Research, Innovation and Technology Transfer Project with the
Collaborative grant scheme for public sector R&Ds and SMEs. Furthermore, replication of the Mini and
Matching grant scheme is in the pipeline for IPA 2014. The project also demonstrated that Serbia has
many brilliant, young and innovative entrepreneurs that just need a more conducive environment for their
success.
An excellent visibility of the project (including the one-page story in the Financial Times) proved to be
relevant for the change of perspective and demonstrated that costs of innovation (and R&D) should not be
regarded as expenditure but as an investment. Of course, much more will have to be done to make this
perspective live.
As one of the lessons learned, it would be good to underline the necessity for having the project contact
point in Belgrade, dealing mostly with this or any other specific project. This would improve daily
communication and contribute to resolution of urgent and technical issues.
2
I would also like to use this opportunity to commend the very good World Bank team and specifically the
Team Leader, Ms Natasha Kapil. Their knowledge, experience, professionalism, efficiency and
persistence contributed significantly to the huge success of this project.
The ICR report provides a realistic and correct picture of the project and offer an excellent analysis of all
relevant project elements: from context and design to implementation and assessment of the outcome.
Notwithstanding this, please note that the above comments should not be regarded as an approval of the
final report for this project. The final narrative and financial report will be commented once submitted for
approval, as per the administration agreement with the World Bank.
Dejan M. Šuvakov Programme Manager for SMEs, Competitiveness, Innovations - Operations II
Delegation of the European Union to the Republic of Serbia Vladimira Popovića 40, 11070 Belgrade, Serbia
Comments from Stakeholder: Business Technology Incubator of Technical Faculties Belgrade
The Business and Technology Incubator of Technical Faculties Belgrade (BITF) was the first technology
incubator launched in Serbia in 2008. Since the early days of BITF, we have faced the lack of funding
sources for startups and innovation programs. In addition, there was no systematic approach and funding
for incubators in general, which made this period of time extremely difficult. At that time, there was the
Innovation Fund (IF), established under the Innovation Law in 2006, however it was not operational until
2011, when it was activated with the World Bank support through the Serbia Innovation Project (SIP),
financed by the European Union through the Instrument for Pre-Accession Assistance (IPA 2011) funds
for Serbia.
The beginning of the Serbia Innovation Project implementation was marked by distrust of the government
in the potential for innovation and fear of not having enough projects on the one hand, and by distrust of
startups that the main criterion in selecting projects for funding would be innovation on the other hand. At
the same time, it was more than difficult to raise the capacity of the Innovation Fund due to lack of local
experts in the field of innovation, especially in the field of financing innovation.
As for startups, it was noticeable that the selection of the first group of projects funded from the
Innovation Fund (beginning of 2012) encouraged other startups, so that every subsequent call for
proposals for small grants (Mini Grants) attracted more and more project proposals, which resulted in the
greater visibility of the innovation potential of Serbia at all levels. There is no doubt that through the
entire implementation period of the Serbia Innovation Project, the level of support to innovation in Serbia
was constantly increasing. At the same time, all stakeholders learned a lot at the macro level from the
example of impressive development in Israel, and at the micro level, with the preparation of project
proposals according to the demanding World Bank methodology which was a learning process for small
companies in Serbia.
Now it is necessary to multiply the positive effects of this project through the future work of the IF in
order to advance the technological development of the country. For that purpose, the Innovation Fond,
being an organization that received a significant capacity building support through the Serbia Innovation
Project, must secure funding and continuously invest in innovation development and startups by
providing small grants or similar types of support.
Gordana Danilovic Grkovic,
BITF, Manager
3
Annex 9. List of Supporting Documents Available Upon Request and in Project Files in
WB Docs
(a) Project Appraisal Document
(b) Data and statistics on grant beneficiaries
(c) Impact Evaluation Report - Estimation of the Effect of Participating in the IF
Financing Programs on Firm Performance (October 2015)
(d) Project Aide Memoires
(e) Project Progress Reports
(f) Project Implementation Status Reports
(g) IF Annual Reports
(h) RDI reports, workshop materials, manuals, and so on
4