Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Ex post evaluation of
Cohesion Policy programmes
2007-2013, focusing on the
European Regional
Development Fund (ERDF)
and Cohesion Fund (CF) –
Work Package nine: Culture
and Tourism
2014CE16BAT034
DATE:
November 2014
Submitted by:
Irs, Csil, Ciset, BOP Consulting
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 3
Table of Contents
1 Foreword ................................................................................................................. 5 2 Aims and scope of the evaluation ................................................................................ 7
2.1 Culture and tourism: conceptualisation and preliminary definitions........................... 8 2.2 Culture and tourism can be strongly interrelated and mutually interdependent. ......... 9 2.3 Culture and tourism can play an important role in fostering growth and supporting
social innovation, social cohesion and better living conditions… .......................................... 9 2.4 ...however culture and tourism are fragmented sectors and are highly dependent on
public funds ............................................................................................................... 10 2.5 Cohesion Policy and the culture and tourism sectors.............................................. 11
3 Fine tuning of the evaluation approach and methodology .............................................. 13 3.1 The evaluation approach .................................................................................... 13
3.1.1 Evaluation issues ......................................................................................... 13 3.1.2 The proposed evaluation approach ................................................................ 15 3.1.3 Evaluation methods ..................................................................................... 23
3.2 Evaluation activities and tasks ............................................................................ 25 3.2.1 Task 1 – Review of the existing literature on culture and tourism ...................... 26 3.2.2 Task 2 - Data analysis and sampling .............................................................. 38 3.2.3 Task 3 - Case Studies and Mini Case Studies “What Happens on the Ground?” .... 54 3.2.4 Task 4 - Cross-Task Analysis and Presentation of the Final Report ..................... 73
4 Organisation and management .................................................................................. 75 4.1 Roles and tasks ................................................................................................ 75 4.2 Allocation of activities ........................................................................................ 81 4.3 The quality control system ................................................................................. 82
4.3.1 The team’s quality management .................................................................... 84 4.3.2 Risk management and contingency plan ......................................................... 85 4.3.3 Specific quality check ................................................................................... 87
5 Time plan, deliverables and upcoming activities ........................................................... 91 5.1 Time plan and upcoming activities ...................................................................... 91 5.2 Deliverables ..................................................................................................... 94
Annexes ....................................................................................................................... 97 Annex I Indicative Bibliography ................................................................................ 97 Annex II First elaboration on ERDF AIR raw data (Project selection), 2007-2013 ........... 104 Annex III Draft Questionnaire for Managing Authority ................................................. 122 Annex IV Pilot case study proposal –Puglia ERDF ROP - Motivation ............................... 137
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 4
List of abbreviations
AIR Annual Implementation Report
CCI Cultural and Creative Industries
CF Cohesion Fund
CP Cohesion Policy
DG REGIO Directorate General for Regional and Urban Policy
EC European Commission
ERDF European Regional Development Fund
ESF European Social Fund
ETC European Territorial Cooperation
EU European Union
GDP Gross Domestic Product
H&R Hotels and Restaurants
IB Intermediate Body
ICTs Information and Communication Technologies
LAU Local Administrative Units
LTSs Local Tourism Systems
MA Managing Authority
MS Member State
NUTS Nomenclature of Territorial Units for Statistics
OECD Organisation for Economic Co-operation and Development
OP Operational Programme
RCE Regional Competitiveness and Employment
RTA Regional Tourism Agency
ROP Regional Operation Programme
SF Structural Funds
SFC Structural Funds Common database
SME Small and Medium Enterprise
TPAs Touristic Promotion Agencies
WP0 Work package 0
WP9 Work package 9
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 5
1 Foreword
The Consortium IRS-CSIL-CISET-BOP has been selected for work package nine (WP9):
“Culture and Tourism”, within the framework of the ex post evaluation of the 2007-
2013 Cohesion Policy Programmes co-financed by the ERDF/CF.
This draft of the Inception Report revises the methodological approach and the general
project organization, developing the original proposal in response to the discussions
and clarifications during the kick-off meeting (held on 21 August 2014).
This report:
clarifies the aims and scope of the evaluation (chapter 2);
reviews and develops the evaluation approach and methodology (chapter 3);
refines the composition of the team and the external experts, clarifies the tasks
to be carried out and the responsible team members, and the number of days
allocated (chapter 4);
lists forthcoming activities and clarifies the schedule for the evaluation;
provides an indicative structure for future deliverables (chapter 5).
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 7
2 Aims and scope of the evaluation
This ex post evaluation aims to analyse the use of ERDF1 funds invested in the sectors
of culture and tourism through the 2007-2013 Operational Programmes, and to
provide evidence of both the nature of ERDF investments and their effectiveness in
supporting regional development.
The terms of reference and discussion in both the kick off and the 1st Steering
Committee meetings provide a framework for this analysis and identified a range of
issues that the evaluation approach will address. These issues include:
given the scattered and scant evidence available on the economic effectiveness
of ERDF investments in culture and tourism, the main objective of this
evaluation is to gain a methodologically and analytically sound picture of the
nature of ERDF investments in the culture and tourism sectors and their
contribution to the main objectives of cohesion policy, by identifying – if any –
the main types of strategies used, the way investments are organised and
designed, the results achieved and the financial sustainability and expected
impact of ERDF interventions in selected regions where we undertake more
detailed investigation (using case study analysis);
the methodological approach will therefore have a specific focus on the
robustness and accountability of the evaluation and so be based on a sound
and defensible methodological design so that credible evaluation results and
impartial conclusions can be presented on the main rationales behind
interventions to support culture and tourism (including sport and creative
industries) through the financial support from ERDF;
fieldwork (in the form of an online survey and case study research) will be the
main source for primary data. Other relevant evidence will come from
statistical analysis of the DG REGIO Cohesion Policy database and the
categorisation codes currently in use, as well as evidence from programming
documents and thematic evaluations.
the analysis will have to search for a balanced approach to the analysis of
investments in both the culture and tourism sectors;
great attention to the presentation and organisation of the work will be given to
ensure high quality written outputs and that the project meets its deadlines.
To help contextualise and define the scope of the evaluation, and to ensure that these
issues are addressed, the following sections describe important characteristics of the
culture and tourism sectors that will be considered in the evaluation.
1 Hereafter we will consider only ERDF programmes given the marginal role of CF in the culture and tourism
sectors.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 8
2.1 Culture and tourism: conceptualisation and preliminary definitions
Culture and Tourism are complex policy fields to conceptualise as they cover a wide
range of activities, have sizeable overlaps between them and other policy fields and/or
sectors such as information and communications technology, transport, distribution,
construction, etc.
Culture embraces a range of activities. These activities range from preservation and
restoration of natural, historical and cultural heritage, to museums and exhibitions,
activities in the arts (music, theatre, etc.), and entertainment, media and e-culture
platforms. The development of economic activities concerned with the creation and
distribution of goods and services that are cultural in nature and protected by
intellectual property rights, has led to a new dimension of cultural supply. The rise of
“cultural or creative industries” - which include architecture, visual and performing
arts, music, artistic crafts and fashion design, but also web and graphic design,
publishing, film, TV and radio production, etc. – is a growing source of jobs and wealth
and have also an important role to play in urban regeneration and community
cohesion. It is difficult to define culture2 as it is often an “intangible” product, and in
recent years there has been a move away from product-based definitions to process-
based definitions of culture (OECD, 2009).
Tourism is considered to be a major economic activity which can have a positive
impact on economic growth and employment in Europe. The current acknowledged
definition of tourism is the one developed by the United Nations’ Agency World
Tourism Organization (UNWTO): “tourism comprises the activities of persons travelling
to and staying in places outside their usual environment for not more than one
consecutive year for leisure, business and other purposes not related to the exercise
of an activity remunerated from within the place visited”.
The tourism product is thus a complex mix of non-reproducible resources (mainly
relating to natural heritage such as beaches, national parks, lakes, mountains, etc.,
but also to historical and cultural heritage such as castles, museums, etc.) and of
goods and services produced by the private sector in different fields (such as
hospitality, the activities of tour operators and travel agencies, tourist assistance
services and cultural, recreational and sporting activities) that can be identified only
when a tourist reaches a destination and experiences it (Ritchie & Goeldner, 2007;
2 Frequently used definitions of culture include:
UNESCO Framework for Cultural Statistics (2009) includes: Cultural and Natural Heritage; Performance and Celebration; Visual Arts and Crafts; Books and Press; Audio-Visual and Interactive Media; Design and Creative Services. It also makes a distinction with two “related domains”: Tourism; Sports and Recreation.
UNCTAD (2008) defines the creative industries as “the cycles of creation, production and distribution of
goods and services that use creativity and intellectual capital as primary inputs” and comprise traditional arts and crafts; publishing; music; visual and performing arts; film, television, and radio broadcasting; new media; and design. (p13).
The “Vilnius definition” is commonly used for sport which provides three related definitions: Statistical, which is a very narrow definition focused on individual-related sport services; Narrow, which includes anything related to “doing” sport e.g. the construction of stadia; Broad, which includes all related activities e.g. sports media or sports gambling.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 9
Costa & Manente, 2000).
A key feature of the tourism sector (and to a lesser extent the cultural industries
sector) is that the market is strongly differentiated being composed by very different
market segments (e.g. same-day, overnight, cruise and activity tourism, of which
there are various sub-types including cultural tourism). The market is also segmented
by visitor origins, either from within the region or from elsewhere.
2.2 Culture and tourism can be strongly interrelated and mutually
interdependent.
Culture can generate tourism: it is often an important determinant of the
attractiveness of a tourist region (Ritchie and Zins, 1978). Tourism can also have
important effects on culture. For example, cultural heritage may be an important
factor in attracting tourists to a location, and tourism may generate funds that make
the conservation of cultural heritage possible. In this way, tourism can support
“culture”. The development of cultural productions (films, TV programmes, books,
museums, etc.) regarding a particular destination may also affect the attractiveness of
the destination to tourists, shaping its tourist image and the visitors’ behavior and
activities (Crang, 1997, 2011; Sangkyun, Long and Robinson, 2009). Therefore, the
creative industries also play a crucial role in linking these two sectors.
Cultural tourism is one of the largest and fastest-growing tourism segments and has
become an important driver for urban development. Many cities, locations and regions
develop their tangible and intangible cultural assets in order to enhance place
distinctiveness and gain a comparative advantage in an increasingly competitive
tourism marketplace. This highlights a growing awareness of the driving role of culture
in helping places become more attractive to tourists or other visitors or to live, work
and invest in. There are many examples of destinations that have adopted this
strategy, such as London and Vienna.
2.3 Culture and tourism can play an important role in fostering growth and
supporting social innovation, social cohesion and better living conditions…
Investing in culture and tourism can boost a region’s economy and enhance social
inclusion. Research undertaken for the European Commission by Ecorys3 and based on
Eurostat data claims that the EU tourism industry is an engine for economic growth
having realized in 2006 a total turnover equal to 5% of total EU-27 GDP and
employing 9.7 million people, equal to 5.2% of the total employment in the EU27.
Cultural and creative industries are a growing source of jobs and wealth and may have
an important role to play in community cohesion. Investments in culture may enhance
the quality of residents' life and sense of belonging. Culture is also increasingly
recognised as one of the factors considered in destination decisions not only in terms
of what makes people visit a certain region, but also what makes people want to stay
3 Ecorys (2009), Study on the Competitiveness of the EU tourism industry - with specific focus on the
accommodation and tour operator & travel agent industries, Final Report, Sept. 2009
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 10
in that region (Wikhal, 2002). Estimates by Tera Consultants for the European
Commission4 claim that the cultural and creative industries accounted for 4.5% of EU
GDP in 2008 and employed about 3.8% of its workforce.
During the recent economic crisis employment in culture and tourism proved more
resilient than the EU economy as a whole5.
Another important feature of the culture and tourism sectors is that they are strongly
place based, relying on natural and/or historical/cultural assets which represent a
comparative advantage for specific destinations. They are therefore an important
potential driver for regional development, as recognized by the sixth priority of the
Territorial Agenda 20206. For instance, by improving the attractiveness of a region,
they may help to limit or redress depopulation. They may also support cultural
diversity, education, social inclusion and social innovation and the maintenance of
natural and historical/cultural assets.
2.4 ...however culture and tourism are fragmented sectors and are highly
dependent on public funds
Although these two sectors are closely connected, and policies adopted in Culture
and/or Tourism can (directly or indirectly) affect other economic and social activities,
strategies for investments in these sectors are often reported as being developed in
isolation from each other. They are therefore not in accordance with or underpinned
by a long term, integrated, territorial strategy. Their success is also strongly
interrelated with other fields of policy intervention (such as urban regeneration, rural
development, environmental policies, employment and education policies, innovation
policies, transportation policies, etc.). However policies and initiatives in these policy
fields are often not consistent with a strategic vision towards culture and tourism.
This fragmentation can hamper the sustainable and competitive development of
culture and tourism industries (Manente, Minghetti, Montaguti, 2013). This contributes
to the risk of limiting the capacity of these sectors’ to address key challenges, such as:
globalisation of the tourist and cultural markets, the impact of the crisis on
consumption levels and patterns and on the availability of public resources, changing
4 Tera Consultants (2014), The economic contribution of the creative industries to EU GDP and employment,
September 2014 http://www.forum-avignon.org/sites/default/files/editeur/2014-Oct-European-Creative-
Industry-GDP-Jobs-full-Report-ENG.pdf
5 Recent data provided by Eurostat show that employment in the Accomodation and food activities increased
by +3.4 between 2008 and 2013 compared to an overall employment decline in the same period of – 2.9%
(European Labour Survey Data). Data resulting from the SBS – Structural Business Survey also report that
between 2008 and 2012 in the sectors G476 - Retail sale of cultural and recreation goods, I-
Accommodation and food service activities and N79 - Travel agency, tour operator reservation service and
related activities, the number of active businesses increased by 3.4% (compared to a 3.3% increase for the
overall economy) and the number of workers increased by 5.6% compared to a decline of -2.8% for the
overall economy. 6 Territorial Agenda of the European Union 2020, Towards an Inclusive, Smart and Sustainable Europe of
Diverse Regions, agreed at the Informal Ministerial Meeting of Ministers responsible for Spatial Planning and
Territorial Development on 19th May 2011 Gödöllő, Hungary.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 11
preferences due to demographic trends, and the impact of new technologies on
traditional business models.
Other factors for consideration include the high dependence of these sectors on public
funding and the relative importance of strategic and management capacity of public
institutions and operators. The lack of strategic and management capacity can make
potentially attractive regions less successful than they could be. Endowed assets
(natural resources, such as beaches, climate, and historical and cultural endowments)
that have traditionally been important for destination attractiveness are no longer
sufficient by themselves. Stronger international competition has made created assets
(infrastructure, accessibility, facilitating resources, hospitality, product and service
innovation) and management capacity (marketing, finance and venture capital,
organisation, human resource development, information/research, quality of service,
visitor and resources management) crucial to utilise all the regional tourism and
cultural assets effectively, for instance, through industry restructuring, product
innovation and territorial marketing/branding (OECD, 2009).
However, the wide array of public, private and, in the culture sector, third sector
actors involved in the provision of culture and tourism products and services makes
strategic management complex. Smaller operators may also lack the financial and
strategic resources to implement the integrated governance systems necessary to
support sustainable tourism or enhance culture as a driver of development.
Recently there has been a growing awareness of the need for a more strategic and
integrated territorial approach and European countries are reportedly revising culture
and tourism strategies in order to better address this issue (OECD, 2009).
For example, there have been moves to integrate culture and tourism policies into
national or regional economic plans to create jobs, to promote regional development,
and to promote export growth. New financing schemes and public-private partnerships
have been implemented to relieve pressure on public budgets and to encourage a
higher level of cooperation particularly in territorial strategies and marketing activities.
Clusters of cultural and creative industries are being supported with universities acting
often as important catalysts for these clusters.
Some countries/regions are also rationalising tourism policy delivery functions and
programmes, and focusing more on natural and environmental assets as well as new
technologies and social media. At the same time, there is a growing awareness of the
importance of domestic tourism (including cultural tourism) in providing a stable
source of revenue in uncertain times, as well as of the social benefits deriving from
the promotion of sustainable social tourism accessible to all (OECD, 2014).
2.5 Cohesion Policy and the culture and tourism sectors
Structural Funds are key financial sources for the culture and tourism sectors, yet
evidence of the effectiveness of ERDF support to these sectors is at present limited.
No dedicated evaluation has been carried out by the European Commission, while few
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 12
evaluations have been conducted or are ongoing in the Member States.
Around EUR 13.5 billion of ERDF funds (equivalent to 5.7% of total allocated ERDF
funds) have been allocated to culture and tourism in the 2007-2013 programming
period. Existing data on expenditures available in the Infoview database show that:
around EUR 12 billion allocation under the 6 culture and tourism priority
themes in 235 OPs (see Table A1 in Annex II), including EUR 1 billion under the
Territorial Cooperation Objective;
an additional EUR 2 billion are categorized under the economic activity "hotels
and restaurants", mainly under innovation and business support priority
themes.
In addition, support for culture and tourism - related infrastructure and services can
be provided under other headings, such as innovation, business support, infrastructure
development, information technology applications and human capital.
Supported projects range from investments in large public infrastructure (including
tourist or cultural infrastructures like theatres, museums, holiday and sporting/leisure
centres, etc., as well as basic infrastructures, like transportation systems, waterfronts,
etc.) to shape and improve the attractiveness of cities and rural areas; to investments
in private enterprises and the third sector; to small local or cross-border projects to
support the upgrading of tourist or cultural services. Even if the ESF does not explicitly
address culture and tourism, it often provides support to the development of skills in
cultural and tourism sectors.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 13
3 Fine tuning of the evaluation approach and methodology
3.1 The evaluation approach
3.1.1 Evaluation issues
Besides usual challenges facing an evaluation of this type, any study assessing the
contribution of ERDF invested in the culture and tourism sectors, and the rationale for
public interventions, must address the following issues:
• The lack of a universally agreed definition of these sectors, especially in the
case of culture. This makes the collation of comparable evidence across
different countries or regions difficult (O’Brien, 2010, p11). Because of this, in
the literature review detailed in Task 1 we will analyse a range of definitions of
both culture and tourism which are used in academic and grey literature, to
ensure we choose one appropriate for this evaluation;
• The heterogeneity of activities in these sectors will have to be considered when
selecting case studies and mini case studies and in the assessment of the
evidence resulting from the evaluation;
• The lack of clarity in the logic underpinning the interventions in these sectors
and their scattered nature may make it difficult to identify and evaluate results,
making it necessary to focus on outputs and expected results, rather than
actual results.
As indicated in Chapter 2, culture and tourism can play an important role in fostering
regional socio-economic development. However, their potential contribution is
hindered by their highly fragmented structure, high dependence on public funding and
their often marginal role in territorial strategic planning. To fully exploit the potential
of culture and tourism for regional development, institutional learning and capacity
building are therefore particularly important in order to address these sector-specific
issues. The effectiveness and performance of these sectors are differentiated not only
in relation to existing socio-economic conditions, but also in relation to the strategic
and innovation capacity of public and private institutions and operators.
Cohesion policy may contribute to the competitive capacity of these sectors by
supporting related infrastructures and businesses, ICT applications and innovation, as
well as institution-building, social capital formation and partnerships. However, this
potential added value of Cohesion Policy to culture and tourism is not always created.
The academic and evaluation literature provides some examples of culture and tourism
projects producing a positive contribution to regional socio-economic growth, however
other anecdotal evidence suggests that investments in these sectors have been
sometimes isolated and over scaled one-off events, not integrated in long term
strategies and thus failing to achieve any significant economic impact. Indeed, the lack
of a coherent strategy is often limiting the effectiveness of individual interventions and
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 14
their success.
For example, the performance audit of the European Court of Auditors (2011) on the
effectiveness of 206 randomly sampled ERDF co-financed tourism projects in 9 MSs in
the 2000-2006 programming period7 showed concerns over additionality and
administration. Of the surveyed project promoters, 26% stated they would have
undertaken the project even without a public grant; 42% of promoters underlined the
considerable administrative burden tied to ERDF funding. The Audit also highlighted
limitations regarding the unclear articulation of objectives and projects’ selection
criteria, weak orientation towards results, weak financial sustainability and a lack of
monitoring and evaluation mechanisms.
The main impact of Structural Funds on (coastal) tourism in the 2000-2006
programming period has been on institutional and capacity building, according to a
study carried out by CSIL (CSIL, 2008). This study found that the implementation of
the Structural Funds increased attention on tourism as a strategy to foster regional
development, as well as on the need to adopt an integrated approach, especially in
terms of sustainable tourism, to the preservation of territorial areas’ historical, cultural
and natural resources and service diversification.
Other recent evaluations of the contribution of ERDF to regional development in the
previous programming periods8 show that in regions with specific geographical
features - islands, mountainous and sparsely populated areas – tourism, exploiting
cultural and natural specificities, has been a key asset-based sectors used to promote
or sustain economic development. The support to economic diversification through
tourism has thus been an effective medium-term response for regions with lagging
economies, especially when integrated strategies were implemented with synergies
with transport policies and ESF and EARDF programmes. However, when regional
development is only based on tourism assets, these economies are relatively
vulnerable to external economic shocks (such as changes in the price of natural
resources; changes in tourism preferences etc.) as well as climate change.
Furthermore, these ‘touristic asset-based’ strategies require constant monitoring and
improvement to maintain a competitive advantage over other regions.
The findings from existing studies also reveal a relative lack of projects to encourage
investment in innovation and research in these sectors. This was particularly the case
in the Convergence regions where the focus was on developing hard infrastructure
projects (e.g. visitor attractions, sometimes in the private sector), or providing
incentives to the hospitality sector with subsidies to hotel operators.
The lack of data and evaluation evidence on the specific contribution of culture to
7 European Court of Auditors (2011), Were ERDF co-financed tourism projects effective?, Special report no 6. 8 European Policies Research Centre, University of Strathclyde (2013), Evaluation of the main achievements of Cohesion Policy programmes and projects over the longer term in 15 selected regions (from 1989-1993 programme period to the present); and ADE (2012), Relevance and effectiveness of ERDF and cohesion fund support to regions with specific geographical features - islands, mountainous and sparsely populated areas.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 15
regional development makes a comprehensive assessment particularly challenging.
The few available studies and evaluations on the contribution of the ERDF (overall) to
the development of a competitive and sustainable cultural sector in the EU (CSES,
2010, KEA, 2012) underline the importance of the strategies and of the
implementation mechanisms adopted in supporting the growth of the culture sector as
a driver for growth. According to the KEA (KEA, 2012) study, in the 2007-2013
Community Strategic Guidelines culture is considered as a:
soft location factor, improving the attractiveness of regions and cities through
the regeneration and development of cultural assets and the supply of cultural
services;
tool to create and attract more and better jobs;
driver for tourism led economic development.
Existing studies show however that culture is rarely cited as contributing to goals
relating to innovation or sustainability (unless in relation to sustainable tourism), even
if it is increasingly acknowledged as contributing to innovation, attractiveness of the
territory, entrepreneurship, and economic development. This approach does not
exploit the potential of investments in the Cultural sector in driving non-technological
and social innovation, contributing to urban regeneration or the green economy, and
to job creation.
3.1.2 The proposed evaluation approach
The proposed analytical approach aims to: (i) assess the type of interventions
supported with Structural Funds, their outputs and results in the culture and tourism
sectors and their contribution to regional development; and (ii) answer the “what
works” question, i.e. what factors and social mechanisms in the policy making process
support the policy results. To this end the theory of change will be used as main
theoretical reference, while the social mechanisms theory will be considered as an
additional explanatory approach.
The focus of the theory of change is whether it is clear what the intervention activities
are to produce in relation to the desired or intended change. It helps to analyse the
theoretical assumptions underpinning the implemented policies, and to define the
sequence in which results are likely to happen. It allows the identification of the
primary factors triggering change, as well as questioning what other sets of conditions
and circumstances, apart from the intervention activities, are necessary for the
desired outcome to occur. The theory of change clarifies the causal connections
between results, outputs and inputs in ERDF support to the culture and tourism
sectors. In looking at the causal connections, particular attention is paid to the
assumptions made about the external environment in which an intervention is
implemented (Dente, 2011). It also allows us to assess the extent to which the
interventions are on course to achieve their expected results.
As an additional explanatory approach, the social mechanisms theoretical framework
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 16
(Mayntz 20049, Barzelay 2007) could also help explaining why results occur.
The evaluation will focus on the ERDF resources allocated to and spent on
interventions in the culture and tourism sectors, their targeted and achieved outputs
and the intended and actual results in order to evaluate the ERDF contribution to the
culture and tourism sectors and indirectly to regional growth and jobs.
In the following we illustrate the main issues that will be to taken into consideration in
our evaluation approach against three headings:
1) The evaluation focus in terms of policy boundaries and ERDF interventions to
be considered;
2) the rationales for ERDF support to these sectors and the strategies ERDF can
implement given its final aim to enhance regional development;
3) the expected contribution of ERDF interventions.
Evaluation focus
As specified in paragraph 2.1, definitions of culture and tourism are very broad. Within
our evaluation activities we will align with the definitions provided in the Terms of
Reference, as follows:
For culture we will refer to three specific sectors: the cultural sector, the
creative industries and sport. Thus, we will consider the preservation and
restoration of natural and cultural heritage, activities in the arts, including
venues and museums, and entertainment, media and e-culture platforms, sport
and outdoor activities.
The tourism industry includes hotels and restaurants, activities of travel
agencies and tour operators, tourist assistance activities as well as certain
recreational, cultural and sporting activities developed to attract visitors.
These definitions will be refined and developed during our literature review (Task 1).
Attention will be paid to all ERDF areas of intervention that are directly linked to
culture and tourism: protection and preservation of the cultural heritage, development
of cultural infrastructures, other assistance to improve cultural services; promotion of
natural assets, protection and development of natural heritage, other assistance to
improve tourist services, and hotels and restaurants in the case of tourism.
This list of areas of intervention will be reviewed as the evaluation progresses and with
specific reference to the reality of strategies, programming and implementation at the
regional level. Within the identified regional strategies, the selection of mini case
studies will consider ERDF-supported projects that are representative of the various
types of economic activities and interventions that are included in the culture and
tourism sectors.
9 Mayntz R., (2004), Mechanisms in the Analysis of Social Macro-Phenomena, Philosophy of the Social Sciences June 2004 34: 237-259.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 17
Identification of framework strategies and of the rationale for public intervention
As underlined in Chapter 2, the capacity of culture and tourism to be competitive and
to support sustainable regional development be competitive and to support
sustainable regional development depends on: the region’s socio-economic conditions;
the role of culture and tourism in the adopted regional strategies; which culture and
tourism sector(s) have been targeted by development policies and what have been
major trends for the target sectors(s) over the period of the programme.
In order to provide a functional structure to refer to during the evaluation, the
following preliminary general strategies based on culture and tourism as drivers for
socio-economic regional development have been identified:
- the first strategy is mainly focused on investments in culture and tourism
industries to support economic diversification, especially in declining industrial
or agricultural regions or in regions where tourism has a marginal role in the
economy. These strategies are usually regionally or nationally focused and their
typical objectives are to improve job and economic opportunities in the region
by increasing tourists numbers and/or diversifying their type. This may be done
through, for instance, improving accessibility, support to territorial marketing
activities, the upgrading and extension of accommodation infrastructures and
enhancing the attractiveness of the regions (including improving cultural and
natural assets);
- the second strategy is mainly focused on investments in culture and tourism
industries to support economic regeneration and social cohesion of urban or
rural areas within regions. A typical objective is to improve wellbeing and social
cohesion, by upgrading local cultural and natural assets, supporting sport and
cultural infrastructures and promoting events;
- the third strategy is mainly focused on support to innovation and
competitiveness in these sectors. The objective in this case is the upgrading of
cultural and tourism services with the use of new technologies or exploring new
market niches, as well as supporting spillover effects on other sectors through,
for example, cluster strategies;
- in regions largely dependent on the tourism sector, a tourism specific strategy
focused on supporting socio-economic and environmental sustainability. In this
case the objective is to encourage tourism operators and service providers to
focus on economic and environmental sustainability.
These strategies will be better defined and verified in the evaluation process, on the
basis of the survey on Managing Authorities and the case studies and mini case
studies.
ERDF investments can refer to more than one of these strategies, used in combination
and according to the regional contexts or the different tools used: physical and ICT
infrastructures; support to business start-ups and consolidation; support to business
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 18
clusters/districts and to networking; grants and loans schemes; soft infrastructures
e.g. spectacles, events, skills upgrading, territorial marketing; etc.. However, a
catching-all basket of the above mentioned interventions without a clear focus and
well-identified objective can be a signal of poor planning capacity and lack of strategic
underpinnings.
Underpinning these strategies, one of the main rationale for ERDF intervention in the
culture and tourism sectors (as well as for other public intervention) could be that of
addressing possible market failures and so to improve these sectors’ performance and
the economic and social conditions of the targeted regions.
For both sectors market failures mainly relate to the presence of public goods (natural,
historical and cultural resources) and the need to ensure their sustainable use, as well
as these sectors’ externalities in terms of social cohesion and quality of life for the
resident population, their spillover effects on other sectors, their contribution to
physical and socio-economic regeneration and to contrast de-population. These
rationales for ERDF intervention are detailed in tables 3.1 and 3.2 below and further
discussed in the methodology for Task 1 (page 29). Again, these rationales will be
refined and developed throughout the evaluation.
The expected contribution of ERDF investments
In the Culture sector ERDF interventions can support industrial clusters or districts
around key major cultural assets; promote a greater involvement of private operators
in the provision of cultural products and services; support innovation and the
development of new business models and networking; enhance the economic and
social spillover effects of culture and creative industries on social and economic
innovation and growth, in particular through the design of new products and services
that can support otherwise declining industries and make room for high-value market
niches in peripheral regions; support to the development of creative clusters, creative
incubators and the digitisation of cultural contents to support Europe’s peripheral
regions; support to cultural and creative business start-ups, development and
networking; invest in cultural infrastructures.
In the Tourism sector, mainly based on private providers and the regions’ key natural
and/or historical/cultural assets, ERDF intervention can incentive long term sustainable
strategies, support tourism models focused on attracting foreign visitors (export-led
model) and based on environmental and economic sustainability; it can support the
upgrading of the management and innovation capacity of regional operators, in order
to reduce seasonality and improve employment conditions; it can support the
development of new business models and tourism services (like for example “diffused
hospitality” or “agritourism” in rural areas; or tourism facilities and services accessible
to the elderly or the disabled; etc.). Given the high dependency of some EU regions on
tourism, diversification contributes to reduce dependency and to ensure economic
activity and jobs outside the tourist season.
Tables 3.1 and 3.2 below summarise the main general strategies (as indicated in
point 2 above) and market failures that could justify public support to culture
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 19
and tourism in regional development policy, as well as types of ERDF interventions in
the two sectors of interest (as indicated in point 3 above).
Table 3.1. Strategies for cultural and creative industries in economic
development10
General strategy and
description
Possible market failure
examples that the
strategy may address
Examples ERDF interventions
1) Economic
diversification
Support to the growth of
cultural and creative
industries in response to
deindustrialisation or in
economically declining rural
areas.
Enhancing digital and ICT
capacity may be used to help
the cultural and creative
industries themselves
diversify
Public intervention to
facilitate change.
Asymmetry of information /
incentives.
Market forces cannot always
ensure adequate long-term
funding for investment due
to differences between social
and private returns
Stimulating private
investments (in a promising
sector still too weak to
attract investments on its
own)
Refurbishment/recovery and protection of historical monuments, buildings or archeological sites
Construction/extension/recovery
of cultural infrastructure (e.g. museum, libraries, archives, etc.) or of infrastructure providing cultural services (e.g. theatre, concert hall, opera
house, auditorium, art galleries, planetarium, etc.) Organisation of cultural events (e.g. painting, sculpture, photography, etc.) or performing arts (e.g. film production, book
publishing, etc.) Business support for SMEs operating in the cultural and creative industries.
Support to ICT and transport
infrastructures
2) Regeneration
Developing cultural
infrastructure (e.g.
museums, heritage sites,
public art) and fostering
cultural activity (e.g.
investing in an art school)
may play a role in improving
the social and economic
conditions of neighbourhoods
and cities.
Cultural and creative
industries can contribute to
the public image of regions
and foster a sense of identity
Market forces cannot always
ensure adequate long-term
funding for investment due
to differences between social
and private returns.
Positive externalities:
culture and creative
industries support identity
development, civic pride,
promoting social or
community inclusion or
cohesion. They also create
interactions and exchanges
between individuals from
Refurbishment/recovery and
protection of historical
monuments, buildings or
archeological sites
Construction/extension/recovery
of cultural infrastructure (e.g.
museum, libraries, archives,
etc.) or of infrastructure
providing cultural services (e.g.
theatre, concert hall, opera
house, auditorium, art galleries,
planetarium, etc.)
Organisation of cultural events
10 Adapted from: Regional Policy contributing to smart growth in Europe 2020, COM(2010) 553 final and
the Common Strategic Framework for 2014-2020
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 20
and territorial cohesion.
Investment in culture may
be used to trigger a virtuous
development process and
contribute positively to the
four realms of sustainable
development, including
environmental.
different backgrounds,
whether they are actively
taking part in the creative
process (e.g. singing in a
community choir) or are just
appreciating the cultural
good (e.g. attending a music
festival). Cultural activities
also offer a relatively high
level of volunteering
opportunities, compared to
other sectors, bringing
people together from other
backgrounds.
(e.g. painting, sculpture,
photography, etc.) or performing
arts (e.g. film production, book
publishing, etc.)
Business support
Support to ICT and transport
infrastructures
Support to (clusters of) creative
industries (sports, media and
information technology, etc.)
Promotion of sports and
recreational activities and events
3) Support to innovation
and competitiveness
Investment in the cultural
and creative industries in
order to develop innovation
capacity, either within the
sector or by linking it with
other industries to enhance
spillover effects.
Market forces cannot always
ensure adequate long-term
funding for investment due
to differences between social
and private returns and
uncertainty of innovation
results.
Awareness of returns from
adaptation to new
technology.
Business support Support to (clusters of) creative industries (sports, media and information technology, etc.)
Support to ICT infrastructures
Table 3.2. Tourism: Strategies for supporting regional development11 General strategy
Possible market failure addressed by
the strategy and other rationales for
public intervention
Examples of
ERDF
interventions
1a Economic diversification
a) in declining industrial or
agricultural regions
Investment in tourism as a
response to economic decline
and unemployment in other
sectors (e.g. industry,
agriculture) or to industry
delocalization/depopulation
Preserving public goods (natural,
historical and cultural resources), whose
characteristics (non-rivalry and non-
excludability) require an actor super
partes to ensure their efficient
conservation and use for future
generations (missing markets)
Lack of basic know-how at private level.
Decline of the regional economy,
depopulation and difficulty in attracting
Physical investments for the promotion
and
development of the tourism sector (e.g.
11 Adapted from: Regional Policy contributing to smart growth in Europe 2020, COM(2010) 553 final and
the Common Strategic Framework for 2014-2020
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 21
private investments
Local resources and the tourist
superstructure are not developed
enough to attract private investments
from other regions/countries
Some of primary attractions (cultural
heritage, natural areas, local traditions,
itineraries etc.) are public goods
Need for basic infrastructure that can
also affect the development of the
region (roads, airports, etc.)
Need to accelerate the development
process to prevent further economic and
social problems
Information failure: information
provided on the region is based on its
previous economic specialisation or on
its decline and problems
information centres, etc.)
Investments to improve the safety and protection of natural assets
Territorial
marketing
activities (e.g.
promotional
activities,
networking,
conferences and
trade fairs)
Support to private initiatives (e.g. hotel and restaurants and
other tourism service providers)
Infrastructure to
improve
accessibility
b) in emerging tourism
regions and in regions
where tourism has a
marginal role
Investment in tourism to:
diversify the regional economy
and make it less affected by
economic cycles
provide a driver for
development in natural
protected areas
Preventing market failures related to an
un-governed development of the
tourism sector (missing markets,
negative externalities, income gaps):
tourists and tourism enterprises may fail
to take into account the effects of their
actions on third-parties (e.g. other
tourists, residents’, other businesses)
Managing unstable markets: due to
climate change or business cycles
Spreading tourism benefits to a wider
area and reducing congestion on major
destinations, when the market itself
would tend to address tourists
exclusively on the main attractors
Stimulating private investments (in a
promising sector, still too weak to
attract investments on its own)
Managing economic spillovers to other
regions (most profit derived from
tourism activities may benefit other
regions/countries)
Ensuring that tourism development
supports cultural and creative
industries, and/or cultural and natural
heritage preservation
Information failure on new or less
developed destinations
Provision of key infrastructure to
support the region’s accessibility with
spillover effects on regional
development and welfare
2. Regeneration
Support to tourism accessibility:
Important role of public goods such as
cultural heritage but also other public
Physical
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 22
Regenerate degraded urban
or natural/rural areas
improve the economic value
and social conditions of the
area
Prevent depopulation of rural
semi-abandoned areas
goods such as roads, infrastructures,
public parks or public urban spaces,
etc., which are essential components of
the area to be regenerated
Management and maintenance of new
infrastructures (e.g. cycling routes)
Coordination role of different
institutional levels (e.g. cycling routes
encompassing different regions,
municipalities, etc.)
Information failure: inaccurate
information on these areas is, or
focused exclusively on its
decline/negative aspects
Preventing and managing negative
externalities: tourists and tourism
enterprises may fail to take into account
the effects of their actions on third-
parties (e.g. other tourists, residents’
community, other businesses), who
indirectly benefit or suffer as a result of
their actions
Spreading tourism benefits on a wider
area and reduce congestion on the
major destinations, when the market
itself would tend to address tourists
exclusively on the main attracting
destinations
investments for the promotion
and development of the tourism sector (e.g. information centres, congress
centres, cycling routes etc.) Investments to improve the safety and protection of
natural assets
Infrastructure to
improve the area
accessibility
Marketing
activities (e.g.
promotional
activities,
networking,
conferences and
trade fairs)
Support to private initiatives
(e.g. hotel and restaurants and other tourism
services providers)
3. Innovation and
competitiveness
in specialized tourist regions
and emerging tourist regions:
Investment in tourism as a way
to support innovation and
competitiveness capacity within
the sector or to stimulate
innovation in other sectors (e.g.
agriculture; culture; sport; etc.)
Market forces cannot always ensure
adequate long-term funding for
investment due to differences between
social and private returns and
uncertainty around the results of
innovation processes or investments.
Compensate the lack of motivation by
private businesses to introduce
innovation processes or new
investments, mainly due to the
prevalence of local demand, firms’ size
(SMEs), entrepreneurs’ skills or
knowledge.
Integrate the training
requested/delivered by the private
sector, which can be insufficient due to
the isolation of purchasers or lack of
demand, etc.
Support to
private initiatives
to improve the
quality and
innovation of the
tourism offer
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 23
4.Socio-economic and
environmental sustainability
in specialized touristic
regions: incentive economic
and environmental
sustainability
Preserving public goods (such as
natural, historical and cultural
resources), whose characteristics (non
rivalry and non-excludability) require an
actor super partes to ensure their
efficient conservation and use for future
generations (missing markets).
Preventing and managing negative
externalities: tourists and tourism
enterprises may fail to take into account
the effects of their actions on third-
parties (e.g. other tourists, residents’,
other businesses).
Managing unstable markets (for
instance, as a result of climate change).
Reducing congestion at major
destinations, when the market itself
would be likely to focus primarily on the
main attractions, and incentive tourism
provides to focus on economic and
environmental sustainability.
Investments to improve the
safety and protection of natural assets Support to private initiatives to improve the
quality and innovation of the tourism offer Investments in
new ICT or other
infrastructures
3.1.3 Evaluation methods
Given the evaluation approach and framework developed in the previous section, the
evaluation will focus on:
The territorial socio-economic context and the culture and tourism
sectors’ structure and performance, to explore the main features and recent
trends of the regional socio-economic structure; the features, performance and
trends of the culture and tourism sectors in EU countries and regions, the
challenges and opportunities they are facing, their contribution to regional
development and the rationale for public intervention in these sectors. This
analysis will be mainly based on indications resulting from the Literature review
(see Task 1 below for details) and the statistical analysis of context and sector
specific indicators available in the “General and regional statistics” section of the
EUROSTAT statistics website (see Task 2c below for details).
The relevance of ERDF programmes in the policy areas of interest, to
assess the extent to which ERDF programmes address the specific needs and
challenges of the culture and tourism sectors in EU regions and contribute to
their performance. This analysis will be based on the indications emerging from
the literature review (Task 1) and on data available in the database of Infoview
raw data on expenditures and the survey on the MAs of the largest CP
programmes (Tasks 2a and 2b).
The contribution and effects of ERDF on the culture and tourism sectors,
to assess the type of interventions supported with ERDF, their outputs/results
and their value added relative to domestic policies through the case studies.
Particular emphasis will be put on how ERDF strategies incentivise innovative
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 24
and integrated interventions aimed at overcoming the seasonality issue and at
promoting a high quality and sustainable culture and tourism activities, as well
as the contribution of the supported activities to regional development. The
evaluation of the potential and actual effects of ERDF on culture and tourism will
consider the following dimensions (see Task 3 for details):
promotion of sustainable and quality tourism and of less known
destinations;
job and business creation/preservation;
innovation in products, services and processes;
development of cultural and creative industries;
contribution of tourism and cultural interventions to the regeneration and
attractiveness of rural and urban areas and community development;
contribution to the attraction and retention of highly skilled workers;
contribution to job quality.
In assessing the main national and regional strategies in the policy areas of interest,
additional attention will also be devoted to understanding the evolution of the
strategies in the programming period within the given institutional setting. This
analysis will be based on the indications resulting from the Literature review (Task1),
inputs from thematic experts (internal and external to the research team) and the
survey on the MAs and some Implementation bodies (when possible) of the largest
Cohesion Policy programmes (see Task 2b for details).
A combination of methods will be used in the evaluation:
Desk research and literature review on public investment in the culture and
tourism sectors (including the literature on creative industries and sport) will
support the definition of the analytical framework for the evaluation, by providing a
typology of investment strategies in these sectors; an indication of the challenges
and opportunities faced by the culture and tourism sectors at the regional level and
their potential contribution to regional development; the rationale for public
intervention in these sectors and the factors to be considered in the analysis; initial
indications of more and less successful projects. The literature that will be taken
into account will cover different sources: academic contributions, empirical
research and studies, documents produced by the European Institutions in the
framework of the Structural Funds and of the European Cohesion Policies;
evaluation studies.
Statistical analysis based on available comparative descriptive indicators mainly
derived from the “General and regional statistics” section of the EUROSTAT
statistics website and the data stored in the DG for Regional and Urban Policy's
databases (InfoView). Statistical analysis will assess the importance and the main
features of the culture and tourism sectors in the national/regional economy and
the incidence of ERDF financial support to these sectors. A Cluster analysis on
culture and tourism context indicators and ERDF spending in related categories will
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 25
support the mapping of EU regions according to these dimensions to highlight
similarities and differences across and within European regions/countries.
Fieldwork based on:
An online Survey addressed to MAs on the strategies, typologies of
activities, and policy tools implemented in a sample of regions with ERDF
allocations in the culture and tourism projects in their areas.
Case studies involving direct interviews with (regional) programming
authorities and local stakeholders to gain: (a) a deeper understanding of
the culture and tourism activities supported by ERDF and their contribution to
improve the quality and performance of these sectors at the regional level and
the factors supporting or hindering success; (b) a detailed understanding of the
complex interactions and processes between ERDF supported measures and
developments in the culture and tourism sectors affecting expected and
unexpected results, with attention to the context and implementation
processes.
The following paragraphs describe in detail how these methods will be combined to
deliver the various tasks and to answer the evaluation questions.
3.2 Evaluation activities and tasks
In summary, the evaluation will include the following activities:
literature review and typology of interventions (Task 1);
data analysis and sampling (Task 2a);
survey to MAs and some Implementation bodies (Task 2b);
tabulation of findings and proposal for case selection (Task 2c);
case study analysis, including mini case studies (Task 3);
cross-task analysis and presentation of the final report (Task 4).
Figure 3.1 shows the relationship between the different tasks, with arrows
representing the main links between the tasks.
The following sections present for each task a detailed description of aims, evaluation
questions, methodology and deliverables
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 26
Figure 3.1. Relationship between Tasks of WP9
3.2.1 Task 1 – Review of the existing literature on culture and tourism
Aims
Task 1 aims to review the existing literature on culture and tourism (including creative
industries and sport), and in particular on the role and use of public investments, in
order to construct a functional theoretical framework which can be used for the
assessment of ERDF support in these sectors. Therefore, it aims to:
identify the socio-economic rationales that justify public financial support in
culture and tourism, distinguishing between public and private initiatives;
identify the main typologies of investment strategies implemented by public
authorities in the recent past (taking into account the type of strategy, the
expected results, etc.);
establish the context for the evaluation of the effects of ERDF support in these
sectors, either in general or according to different type of investment, and then
of the contribution that these investments make to growth and job creation.
These aims are intended to support the thorough analysis of ERDF investments in the
fields of culture and tourism which will be undertaken in Task 2, as well as the
selection of the programme case studies which will be used to assess the
Task 1
Literature review and typology of interventions
Task 2a
Data analysis and sampling
Task 2b
Survey to MAs
Task 2cTabulation of findings and proposal for case selection
Task 4
Cross-task analysis and final report
Task 3a
6 Case studies
Task 3b
12 Mini Case studies
Task 3cFinal comparative analysis and lessons
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 27
achievements of investments and their effectiveness in contributing to the objectives
of long-term growth and jobs (Task 3). To do this the analysis in the literature review
will:
1. highlight, with appropriate key evidence, the role that culture and tourism play to
support long-term growth and jobs and to reduce economic and social disparities
in the EU, in line with the objectives of the ERDF;
2. give evidence of the socio-economic foundations behind the role of public
authorities in supporting culture and tourism development and planning, through
the discussion of the specific characteristics each of the two sectors (i.e. the role
of public goods, the structural complexity of both sectors, the multiplicity of actors
and industries involved, the spread of economic impacts);
3. provide a concise overview of the main political and strategic approaches to
culture and tourism in the EU28, which demonstrate the theoretical assumptions
followed by public authorities in developing multi-annual programs, against a set
of appropriate criteria to classify general policies and Operative Programmes;
4. building on point 3, provide evidence of the main typologies of public investment
strategies pursued in culture and tourism, using a set of specific criteria;
5. contribute to the assessment of the effects of public financial support, either in
general or according to the specific type of investment made and its correlation to
the overall strategy, through the use of a set of indicators already applied in other
studies on the achievements of Cohesion Policy Programs.
The analysis will provide clear and concise answers to the evaluation questions
indicated in the Terms of Reference and raised during the Kick-off meeting and the 1st
Steering Committee. Table 3.3 shows how the evaluation questions map across to
corresponding activities which will be undertaken and included in the final report.
Table 3.3. Task 1 - Evaluation questions and activities
Evaluation questions Evaluation activities
What is the socio-economic rationale behind the public financial support to these sectors?
Discussion of the main socio-economic theoretical foundations behind the role of public financial support, based on the specific characteristics of the culture and tourism Sectors, and that derive from acknowledged academic and practitioner literature in the field, which also inform official government decision making.
What are the main types of public investment strategies pursued in CULTURE AND TOURISM?
Concise overview of the main political and strategic approaches to public support for culture and tourism developed in the EU28, which demonstrate the theoretical assumptions followed by public authorities to guide changes in these two sectors, according to a set of appropriate criteria derived from the literature.
Identification of the main typologies of public investment strategies pursued in culture and tourism, according to a set of specific variables.
Is there evidence of the effects of public financial support, either in general or according to the different type of investments?
Contribution to the assessment of the effects of public financial support in general and according to the type of investment, through a set of indicators already applied in other studies on the achievements of Cohesion Policy Programs.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 28
Methodology
This Task will provide:
the instruments, criteria, and structure required to identify the main typologies of
investment strategies in culture and tourism highlighting the (sometime implicit)
motivation, and logic behind the ERDF investments choices made by the EU regions
in relation to their broad development strategies;
the reference framework to understand if things worked as expected in order to
produce the desired change regarding (directly and indirectly) culture and tourism
or, in other words, how and why the desired change is expected to come about.
The literature review will focus on four interrelated categories of documents, papers
and studies. For each evaluation question, the most appropriate literature to support
the development of the corresponding evaluation activity will be selected. Figure 3.2
summarises the procedure and shows the links between questions, activities and
literature.
Figure 3.2. Task 1 - Scheme of the links among questions, activities and
literature considered
Note: Arrows between boxes relate to links between activities and the evaluation questions or literature.
For each category of literature, Table 3.4 specifies the main sources that will be used
alongside an example of which ones are likely to be most relevant. Priority will be
given to official statistics and to acknowledged academic literature. Special attention
What are the socio-economic rationales
behind the public financial support to
these sectors?
What are the main types of public investments
strategies pursued in Culture and
Tourism?
Is there evidence of the effects of public financial support,
either in general or according to the different type of
investments?
3. Investigation of main strategic approaches and identification of main typologies of investment initiatives in EU28
4. Identification of a number of indicators
through which to assess the effects of
ERDFsupport
2. Discussion of the main socio-economic foundations behind the role, based on a number of relevant aspects that characterise the two sectors
Evaluation questions
Evaluation activities
Literature investigated
• Academic literature• ‘Grey’ literature• EC studies and
publications (evaluation studies)
• Academic literature• ‘Grey’ literature• EC studies and
publications (sector-based studies)
• Academic literature• ‘Grey’ literature• EC studies and
publications(evaluation studies)
1. Statistical data on the role of Culture and
Tourism for a homogeneous economic
and social development of the EU, coherently with
ERDF tools
• International statistics and related studies
• Academic literature• ‘Grey’ literature• EC studies and
publications
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 29
will be given to the selection of grey literature and of EC studies and publications, in
order to identify those that provide a balanced, substantiated view on the two sectors.
In general, equal attention will be given to sector-based literature as well as to studies
and research that analyses and assess the relationships between culture and tourism.
Table 3.4. Task 1 – Main sources of information and data
Type of literature
Description Examples of relevant sources
International statistical data and related studies
Statistical information describing the consistency and evolution of culture and tourism in Europe (physical and economic size: e.g. no. tourists/visitors: tourism receipts; tourism GDP; jobs created), in total and in different countries;
Specific studies carried out by the same sources, through an in-depth analysis of this information.
UNWTO, OECD,
Eurostat, National
Statistics bureaus,
Banque de France,
Banca d’Italia, etc.
Academic literature
Relevant studies and research published by the
international academic community on:
the economic value of culture and tourism (including creative industries and sport);
the economic and social impacts of these sectors at global and territorial level;
the rationale for public sector intervention in supporting
tourism and cultural development and the evolution of public investments in culture and tourism (Books, papers, articles, research works, conference proceedings, etc.)
Costa, Manente, 2000; Dwyer, Forsyth and Papatheodorou, 2011; Blake and Sinclair, 2007; Costa, Panik and Buhalis, 2013 and 2014; etc.
Practitioner or ‘grey’ literature
Selection of relevant studies and research published by National, Regional Tourist and Cultural Offices, independent associations and research centres, national and regional governmental authorities on: the economic value of culture and tourism (including
creative industries and sport); the economic and social impacts of these sectors at
global and territorial level; the rationale for public sector intervention in supporting
tourism and cultural development and the evolution of public investments in culture and tourism
(Books, papers, articles, research works, conference proceedings, reports, assessments, etc.)
OECD, World Economic
Forum, European
Travel Commission,
Visit Britain and other
national and regional
tourist board and
development agencies,
GLA Economics, etc.
EC studies and publications
Selection of studies and reports regarding the aspects mentioned just above and developed or commissioned by the European Commission or by EU Directorates directly and indirectly involved with culture and tourism or by national and regional authorities;
Selection of studies on the relevance and effectiveness of ERDF and CF support and on the main achievements
of CP programmes commissioned by the DGREGIO, which include and assessment of investments in the sectors of culture and tourism
European Commission, DG Enterprise & Industry, DG Culture and Education, Council of Europe, DGREGIO, etc.
The limited availability of data, especially for culture, will be addressed through a wide
search of available sources. The homepage for cultural statistics in the Eurostat portal
states that “European cultural statistics presenting data on enterprises in cultural
sectors, employment and occupation, external trade in cultural goods, cultural
participation and other cultural statistics will be made available as from 2015 onward”
which may mean it is available in the lifetime of this study
(http://epp.eurostat.ec.europa.eu/portal/page/portal/culture/introduction).
In order to provide a robust theoretical framework for the evaluation, the literature
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 30
will be selected using a two-step process:
Step 1: Preliminary Search. The collection of materials already available to the
thematic experts involved in the task will be accompanied by a comprehensive
search on the widely used electronic databases (e.g. Google Scholar, Science Direct
Scopus, Web of Science). The search will be based upon Boolean combinations of
keywords that are considered relevant according to the evaluation questions (e.g.
“public sector investments in tourism”, “economic impact of tourism”). The search
terms used will be recorded for reference. The electronic search will be
supplemented by complementary methods such as reference checking and follow-
up, Internet and hand searching journals, and email communication with key
authors.
Step 2: Initial Review and Selection. The list of potentially relevant material
collected will be analysed and refined. The definitive sources to be reviewed will be
selected and organized according to their relevance to the evaluation questions.
An indicative bibliography of relevant key publications and documentation on culture
and tourism is provided in Annex I.
The two external experts will be involved in the selection phase and will be asked for
their suggestions regarding any additional literature not collated by the investigation.
Outputs
As indicated in the Tender specification, the output of Task 1 will be included in the
First Interim Report. The report will provide a theoretical framework to the evaluation
of ERDF support to culture and tourism. The key findings and results will be presented
using logic models (which will outline the key assumptions made and the Theory of
Change behind investments in culture and tourism), alongside the main objectives of
the Task, the main categories of literature identified, and the methodology described
for the literature review.
The report will be relatively short (30-35 pages) in comparison to the number of topics
dealt with, in order to summarise and bring out the main points of discussion and
provide a concise framework and guidance for the analysis carried out in the following
Tasks.
The report will be supported by appropriate tables, graphs and/or figures in order to
present the data in a clear and informative way. The report will, therefore, consist of
four main sections:
1. culture and tourism in European regions: key figures, development trends
and evolutionary aspects;
2. the socio-economic rationales behind public financial support in culture and
tourism;
3. main strategic approaches and investment initiatives in culture and tourism
in the EU28: A comparative analysis;
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 31
4. assessment of the effects of public financial support in relation to the
strategies in culture and tourism.
The proposed contents and methodology for each section of the report follows.
1. Culture and Tourism in European regions: key figures, development trends
and evolution aspects.
This section will review the most important statistical sources and statistics-related
studies at international and EU level (UNWTO, EUROSTAT, OECD, National Statistics
Bureaus, etc.) to provide a concise descriptive analysis of the:
characteristics and development of culture and tourism in the EU28 Member
States and, particularly, in NUTS2 regions
contribution of the two sectors to the countries/regions’ economy and welfare;
main expected trends shaping their evolution in the next 5-10 years.
To support the first two points, a number of quantitative indicators will be used
(according to data availability) such as: no. of tourists, in total and by country of
residence; no. of cultural visitors; no. of nights spent in all accommodation
establishments, in total and by country of residence; volume of cultural and tourism
receipts; volume of tourism and cultural GDP, in total and as % share of total national
GDP; proportion of cultural and tourism receipts on country GDP; number of
enterprises and of jobs created. These elements will contribute to:
set the scene, highlighting the relevance of culture and tourism (including creative
industries and sport) in the global and regional EU social and economic scenario,
especially in the light of the unprecedented economic crisis that Europe has been
facing;
identify the NUTS2 regions where Culture and/or Tourism make the highest
contribution to social and economic development and, vice versa, those where the
two sectors, or one of them, have a comparatively limited role compared to other
regions or other industrial and service activities. This consideration is important to
help contextualise and understand the logic of intervention followed by public
authorities, and therefore the typologies of investments made (e.g. ‘hard’ vs. ‘soft’
actions).
Culture and tourism will be presented separately, alongside a unifying analysis. The
text will be supported by appropriate tables, graphs and maps, based on available and
relevant data.
The analysis of culture will generally use this concept to refer to three specific sectors:
the cultural sector; the creative industries; and sport. This range of meanings provides
an additional challenge for any evaluation, as the difference in makes the collation of
comparable evidence difficult, especially across different countries or regions (O’Brien,
2010, p11). By clearly defining the sector this approach will avoid some complications
caused by this imprecise terminology.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 32
2. The socio-economic rationales behind public financial support in culture
and tourism
The second section of the report will discuss the socio-economic rationales that
underpin public financial support to both culture and tourism, including the distinction
between public and private initiatives. This analysis will support the identification of
the socio-economic objectives behind the interventions surveyed in Task 2b. It will be
presented as separate sections on Culture, and on Tourism, as well as a unifying
analysis.
The discussion will be based on a number of relevant features and market failures
characterising each of the two sectors resulting from the acknowledged academic and
practitioner literature in the field and underpinning official government decisions (see,
for example, Blake and Sinclair, 2007, for the UK Department for Culture, Media and
Sport; GLA Economics & London Development Agency, 2006; Dwyer, Forsyth and
Papatheodorou, 2011). Examples of these policy rationales include:
Rationales or characteristics applying to both culture and tourism to be tested on the
basis of the evidence provided by the literature review:
presence of public goods (natural and cultural resources) in culture and tourism,
and the importance of ensuring their sustainable use (non-rivalry and non
excludability) and their availability also for future generations;
peculiarity of cultural and tourism demand (seasonality, strong variability
according to a number of factors, etc.);
complex nature of these sectors and multiplicity of (public and private) actors and
industries directly and indirectly involved (for tourism: accommodation, transport,
intermediaries, entertainment, but also agriculture, etc.), which usually have
different and diverging interests and objectives;
reduction of economic disparities, through the spread of impacts produced by
tourist and cultural consumption over different industries and territories, not
directly offering products and services to tourists and cultural visitors;
strong links among the location of main assets and the related investments (the
financing of place-specific sectors tends to produce local benefits);
linked to the previous point, supporting job-intensive sectors and stimulating job
creation at local level (opportunities to ensure gender equality, disadvantage
workers, less-skilled workers especially in less developed regions or declining
economies);
contribution to social cohesion, such as enhancing residents’ quality of life,
preserving cultural identity and local values, reducing isolation and stimulating the
spread of innovation at a territorial level;
Rationales or characteristics that apply primarily to the Culture sector to be tested on
the basis of the evidence provided by the literature review:
growing economic importance of the cultural and creative industries;
cultural sector as a strategic asset for Europe, because of its historic strengths;
role of the cultural and creative industries in stimulating innovation and providing
other economic spillovers;
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 33
role in promoting intercultural dialogue and in international relations ("soft"
power).
Rationales or characteristics that apply primarily to the Tourism sector to be tested on
the basis of the evidence provided by the literature review:
different impacts that public-owned initiatives vs. private investments in tourism
infrastructure can have on the territory (as, for example, the effects that the
opening of a new flight route by a private low-cost airline can have in marginal
areas);
complex nature of these sectors and their interactions with other different sectors
that, directly and/or indirectly, contribute to the “tourism product”, so that public
investments in other sectors (like, for example, agriculture, transports, energy,
etc.) can have an important effect on tourism (e.g. green innovation). Therefore,
public investment can help capitalise or adjust to these effects.
These elements (and, in particular, the presence of public goods and the sector
complexity) highlight the probability of inefficiencies in both sectors and therefore the
need for public intervention in order to improve the functioning of the market, through
strategic and financial support. According to the literature (see, for example, Jeffries
(2001:105-113), there are six main reasons of public intervention in tourism: the
need to have wider objectives than private stakeholders; the importance of transport
policies; the complexity of the tourism sector; the presence of market failure; the
public sector provision of leisure services and the resolution of conflict. This means
that “rationales are also not related to market failure, but some include reasons that
do derive from one of more forms of market failure in tourism” (Blake and Sinclair,
2007: 3). Examples of market failure in tourism are the following: missing markets
(the preservation of public goods, i.e. natural, historical and cultural assets require an
actor super partes who ensure their efficient conservation); externalities (tourists and
tourism enterprises may fail to take into account the positive and negative effects of
their actions on third-parties, e.g. other tourists, residents’ community, other
businesses); information failure (markets may not provide enough information
because, during a market transaction, it may not be in the interests of one party to
provide full information to the other party); unstable markets (sometimes markets
become highly unstable (e.g. because of sudden climate change) and such volatility
may require intervention); economic inequality or income gaps (market transactions
reward consumers and producers with incomes and profits, but these rewards may be
concentrated in the hands of a few).
3. Main strategic approaches and investment initiatives in culture and
tourism in the EU28: A comparative analysis
Having set the scene on the characteristics, structure and role of culture and tourism,
and analysed the socio-economic rationales behind public support to the two sectors,
the third section of the Report will investigate the main typologies of investment for
culture and tourism developed in the EU28 regions. This will support Task 2b (MA
survey of selected programmes) and Task 3 (Selection and analysis of
regional/programme case studies).
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 34
The first part of this section will provide a concise overview of the main approaches
adopted in the EU28 for culture and tourism, building on the general strategies
illustrated in sub-section 3.1.2. This analysis will be developed taking into account, if
relevant, the differences between Convergence and Competitiveness & Employment
regions. The approaches considered are those that guide the theoretical assumptions
behind the intervention logic followed by national and regional public authorities in
developing programmes and include the identification of specific objectives and of
intended results or outcomes.
The main initiatives developed by Member States and representative regions for
culture and tourism will be identified through a review of EU official publications (e.g.
LSE&EPRC, 2013), of National and Regional programming documents defining policies
and financial instruments, and of recent academic research in comparative policy
analysis (Costa, Panyik and Buhalis, 2013 and 2014). The evaluation will also assess
the role of culture and tourism initiatives within the framework of general national and
regional policies, in order to contextualise and understand whether and how they are
part of a broader strategy for encouraging the development of a territory in terms of
growth and job creation and also for improving the residents’ quality of life.
The overview will be based on an appropriate set of criteria derived from the literature
review and will be supported by concise, illustrative case studies at national and/or
territorial level. A first set of criteria for identifying and grouping the main approaches
has been summarised in Table 3.5., with some examples of their application.
Other criteria will be identified during the analysis. The literature will also provide the
basis to interpret the identified approaches in relation to each region’s characteristics
as far as culture and tourism are concerned (e.g. tourism GDP and employment,
number of tourist arrivals, seasonality, destination life cycle, etc.)
Table 3.5 Task 1 – Intervention approaches for culture and tourism in the
EU28: preliminary classification criteria and some examples of application
from both sectors
Criteria Examples Culture Examples Tourism
Strategic level and scope
General development strategy for culture and tourism, i.e. planned independently from the funds used, and the fund sources
Amsterdam’s ‘Kunstenplan 2013-2016' sets out the city cultural strategy, and highlights key areas
of focus or the sector, as well as spending. Amsterdam has a separate ERDF strategy, which includes Creative Industries under the Knowledge development and innovation theme.
The Basilicata strategy for tourism development is illustrated in the Regional Tourism development plan
(PTR). The Basilicata ERDF OP has a specific Axis dedicated to culture and tourism (Axis IV) . This is one of the axis with the higher amount of investments
The Andalucia tourist areas requalification plan (Plan qualifica) is
ERDF strategy, i.e. developed in the OP according to the available ERDF funds or other Eu funds
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 35
partially financed through ERDF and partially by other
sources, including private ones (Programa Operativo de Andalucia, Plan Qualifica)
Main focus and relationships between culture and tourism
Culture, Tourism or both sectors. In the last case indication of which sector is the main pillar of the strategy
The main focus of Andalucia ERDF strategy is tourism more than culture, as tourism is mentioned among the sectors to be considered for new business development, water supply management, technological improvement, sustainable urban development, etc.
Approach Holistic vs. sectoral approach to culture and/or tourism
Dublin’s Arts culture and tourism Strategy does not include EU programmes or ERDF, which are covered by a separate strategy.
Basilicata adopts a sectoral interrelated approach, where the exploitation of natural/cultural heritage also relate to tourism development goals (the same happens in the Apulia or Liverpool cases)
Specificity of the objectives (generic vs. specific)
Generic development of one or both sectors vs. development/promotion of specific markets/products/services
In Swansea (UK), culture and tourism fit within an overarching city economic strategy, as well as within Welsh strategies for culture and tourism.
A focus on tourism in general, or on specific aspects such as seaside tourism, accessible tourism, or similar
Explicit/implicit goals
Whether the strategy addresses culture and tourism explicitly or implicitly, i.e. whether the objectives indicated in programme documents (in accordance to ERDF guidance) differ from public institutions’ real goals for tourism and/or culture development (underlying
aims, which can also emerge in the implementation phase)
A focus on cultural sector, or on a subsector such as cultural heritage, performing arts or similar
In the Andalucian OP, the infrastructure development is not explicitly addressed as part of tourism development, but measures concerning airports infrastructure improvements “in view of demand increase” or the railway connection
Malaga-Algericias are possibly intended for tourism
Continuity with previous ERDF support programmes
If the strategy builds on previous ERDF strategies in this field
The National Strategic Reference Framework for Ireland guides the spending of European funds, and culture and the
creative industries are not mentioned explicitly or implicitly.
In the case of Liverpool and the Merseyside, the strategy remained focused on the development of visitors’ economy throughout the three programming
periods the area benefited from ERDF
The relationship between culture and tourism can heavily influence the strategic
planning at national/regional level. This close relationship explains why some
Operational Programmes in some EU regions have a specific Priority Axis that
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 36
addresses directly natural and cultural resources (i.e. their development/preservation
is the main focus of the Axe) and indirectly tourism12. It is also likely to explain why
specific projects under these programmes address culture directly and tourism
indirectly13.
Moving from the approaches listed above, the Report will also give evidence of the
main typologies of public investments specifically addressing culture and tourism
and of those investments that, although developed in other sectors, can indirectly
cause a change in the development of the culture and tourism sectors. The definition
of these typologies will be based on a set of criteria derived from the literature, which
will set the context in which the investments take place, identify the expected
outcomes/results and their possible strengths and weaknesses. Some of the
classification criteria that can be identified ex ante are described in Table 3.6. Further
criteria will identified during the analysis.
The use of these criteria, derived from the literature, will:
provide a common framework to compare the different investment strategies at
a regional level;
provide a framework to assess the main aspects and determinants
underpinning such differences;
allow us to analyse why the results of some typologies of investments are more
or less effective, and which kind of strategies are likely to be most successful
depending on the specific context.
12 For example, in the Basilicata OP 2007-2013: the development of sustainable tourism and the growth of territorial competitiveness is the main focus of specific/operative objectives and action lines of Priority Axis 4: Exploitation of natural and cultural assets. http://ec.europa.eu/regional_policy/country/prordn/details_new.cfm?gv_PAY=IT&gv_reg=562&gv_PGM=1050&LAN=7&gv_PER=2&gv_defL=7 13 For example, the Campania project “Protecting Pompei for future generations”, aims to stop the degradation of the area and to increase tourism, by attracting an average of 300,000 additional visitors each year by 2017. http://ec.europa.eu/regional_policy/projects/stories/details_new.cfm?pay=IT&the=79&sto=2556&lan=7®ion=1218&obj=ALL&per=2&defL=en. Also in the Castilla-La Mancha project “From mines to mining park”, the building of a cultural site to preserve the mining and industrial heritage of the world’s largest mercury mines and to reverse the environmental damage of years of extraction generated over 52,000 visitors URL: http://ec.europa.eu/regional_policy/projects/stories/details_new.cfm?pay=ES&the=79&sto=1874&lan=7®ion=574&obj=ALL&per=2&defL=en).
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 37
Table 3.6. Task 1 –Main classification criteria for public investments in
culture and tourism in the EU28 regions
Classification Criteria Illustration - Culture
Illustration - Tourism
Output nature ‘Hard’ vs. ‘soft’ outputs Construction of a new theatre vs. a business support programme for the creative
industries
Construction of new hotels vs. the creation of tourism packages or the development of a
promotional strategy
(For investments on ‘hard’ outputs) Structure level
Infrastructure vs. superstructure A digital archive vs. a new theatre
Dedicated transport vs. a hotel, a congress centre, a theme park
Economic activity The sector in which the investments have been developed (e.g. tourism, culture, urban regeneration, environment, energy). If different from culture and tourism, what is the main goal (tourism/cultural development or other)?
Investment in superfast broadband may support creative businesses
Investment in a new waste treatment centre may support the tourist economy
Territorial area The type of area where the investment takes place
For instance, urban vs. rural areas
For instance, urban vs. rural areas
Type of beneficiaries Public and/or private stakeholders (distinguishing NGOs, third sector operators)
Intended results The sector where the main results are expected, whatever the output
For example, the creation of a new concert hall may be intended to improve cultural engagement at a local level or develop the city tourist brand.
Actual results The sector where the main impact occurs, whatever the output
For example, the actual results of the restoration of an historic building are mainly in the tourism sector.
4. Assessment of the expected effects of ERDF financial support to culture
and tourism
This final section of the Report on Task 1 will assess the anticipated effects of ERDF
support on in the two sectors, in general and/or according to the different type of
investments and strategies implemented, according to the literature review. The
conditions under which the intended results have been achieved or not will also be
considered.
The assessment will build on the results of the investigation on the strategic
approaches adopted and the investment initiatives. This analysis will provide guidance
for Task 2b (MA survey of selected programmes) and Task 3 (Selection and analysis of
regional/programme case studies).
As already mentioned, this part of the evaluation is likely to be challenging. First,
because of the lack or poor quality of the data available (Martin and Tyler, 2006).
Second, because of the difficulties in understanding how and why the support provided
by ERDF “brings about changes in the regions where they have been targeted” (Martin
and Tyler, 2006; LSE & EPRC, 2013). Third, the difficulty of disentangling the specific
impact of ERDF support (e.g. in terms of visitors growth) from those of other
investments, wider trends or from other factors. For example, the general evolution of
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 38
the tourism industry brought about by big private companies (e.g. low-cost airlines,
tourist portals and online travel agencies) may provide easier accessibility to regions
and easier booking procedures for accommodation and consequently stimulate an
increase of tourist flows at local level, and if these changes happen alongside ERDF
investment, causality is hard to identify with precision (LSE and EPRC, 2013: 63).
Given these limitations and capitalising on the experience of previous studies on the
achievements of Cohesion Policy Programmes (LSE and EPRC, 2013), the following
table summarises the evaluation criteria to assess the main achievements of ERDF
support, as required in the Terms of Reference. Other possible criteria and indicators
will be identified during the analysis.
Table 3.7. Task 1 – Assessing the effects of ERDF public financial support:
Main evaluation criteria
Dimensions Definition Significance
Relevance Appropriateness of the explicit objectives of an intervention with regard to the socio-
economic problems the intervention is meant to solve
It indicates the strategic structure and internal coherence of programs, i.e. whether the programme strategies, goals and priorities
respond to the needs of the regions where they apply
Effectiveness The extent to which programmes/investments achieved the goals stated (coherence between intended
and actual results)
It is intended to assess, on the one hand, the results specified in the programmes; on the other, the changes which were intended to occur as a result of the policy intervention. In
other words, the anticipatedeffect of the intervention
The two external experts will be involved in the development of Chapter 3 and 4 of the
Interim Report of Task 1. In particular, they will be asked to give advice on further
initiatives and investment strategies developed by some countries, regions, or
destinations that can be illustrative of the investment strategies implemented, and to
assess the potential ERDF intervention areas identified.
3.2.2 Task 2 - Data analysis and sampling
Task 2 a - Distribution of investments and reported output
Aims
As required by the Terms of Reference, the aim of Task 2a is to carry out a statistical
analysis of the distribution of ERDF investments and reported outputs, in order to
derive a picture of the formal allocation of funds by priority themes, their geographical
distribution, their economic relevance and reported achievements. As part of this sub-
task the programmes allocating more than 15 million Euros to culture and tourism
activities (55-60 Priority Themes) plus hotels and restaurants (Economic Activity 14
not included in 55-60 Priority Themes) will be identified in order to set up the survey
envisaged in Task 2b.
Table 3.8 presents the evaluation questions and the corresponding activities and
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 39
deliverables.
Table 3.8. Task 2a – Evaluation questions and activities
Evaluation questions Activities and deliverables
What are the main differences in the relative distribution of investments in culture and tourism across ERDF programmes?
Statistical analysis of the distribution of financial allocations and reported outputs based on DG REGIO categorisation data and achievement indicators (core indicators and other relevant achievement indicators resulting from the
activities of WP0, as well as programme specific indicators resulting from the survey on MAs carried out in task 2b)
This task will: Map and assessment of the distribution of
investments and reported outputs by Priority
Themes, Regions/countries, Economic sectors and Territorial Dimensions
Identification of the programmes with the highest aggregate allocations for these policy areas for the survey of MAs.
Assessment of the core and programme
specific indicators used by the Mas (resulting from WP0 and the Survey), individuation of main strengths and weaknesses and of significant gaps in data reporting (between input and output indicators).
What are the economic sectors (distinguishing between public and private) benefiting most from investments
associated with culture and tourism?
What is the ‘geographical’ and ‘territorial’ distribution across investments?
How reliable is the reporting of relevant core indicators across the programmes?
What programme specific indicators are used and what are their reported achievements?
Methodology
The statistical analysis will be mainly based on: (i) the categorisation of expenditures
and (ii) the achievement indicators reported by Member States in the Annual
Implementation Reports and available from the database created by WP014, as well as
those indicated by the surveyed MAs in Task 2b.
As for the allocation of ERDF Funds, in order to answer the evaluation questions,
we will carry out a descriptive analysis of the distribution of the resources allocated for
culture and tourism and their incidence on the total amount by the following five
dimensions identified in the DG Regio Categorisation System:
1) The Priority themes, which identify the areas of intervention (86 codes). The
analysis on ERDF investments will take into account those priority themes which are
directly linked to culture and tourism:
For Tourism: 55 Promotion of natural assets, 56 Protection and development of
natural heritage, 57 Other assistance to improve tourist services.
14 'Ex post' evaluation of Cohesion Policy programmes 2007–2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) - Work Package Zero: Data collection and Quality
Assessment.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 40
For Culture: 58 Protection and preservation of the cultural heritage, 59
Development of cultural infrastructure, 60 Other assistance to improve cultural
services.
In addition, through the "economic" dimension of the categorisation system, we
will take into account support for the economic sector "code 14: Hotels and
restaurants" (not included in the Priority themes 55-60).
2) the nature of financial support, giving information on the nature of support:
Non-repayable aid, aid, venture capital, other forms;
3) the territorial dimension, defining the type of territory in which the allocated
investment takes place (i.e. urban, mountains, islands, rural, etc.).
4) the economic activity dimension, describing the sector in which the investments
have taken place. These codes are derived from the statistical NACE classification,
with codes added for the services sector. As requested by the Terms of Reference, this
dimension will also allow us to distinguish the public sector (code 17). A preliminary
cross tabulation of ERDF financial allocations in culture and tourism by economic
activity (Table A3 in Annex II) shows however a high percentage of not applicable (00)
or other unspecified services (22) that will be further investigated through the survey
and the case studies.
5) the location dimension, reporting the code of the NUTS area in which the
investment takes place. Descriptive statistics will be provided at European, National
(NUTS 0) and Regional level (NUTS 2 or NUTS 1 for countries with regional OP
referring to NUTS 1 level, such as Germany and The United Kingdom).
First elaborations on available data provided by DG Regio15 are presented in Annex II
as well as the list of OPs investing more than 15 million ERDF in culture and tourism
(Table A2).
Around EUR 13.5 billion of ERDF funds (equivalent to 5.7% of total allocated ERDF
funds) have been invested in the sectors of culture and tourism in the 2007-2013
programming period (Table A1 in Annex II). The countries with higher amount of ERDF
allocations in culture and tourism are PL, GR, ES, HU and IT while the incidence of
culture and tourism on the total ERDF allocations is particularly high in Cross-border
programmes (16.6%), in MT (15,9%), in DK and AT (around 13%). Considering
economic activities, ERDF allocations are concentrated in the Public Administration
sector (13.5%), Hotels and Restaurants (7.7%) and Construction (5.9%) while the
distribution by territorial dimension (Table A4 in Annex II) shows a concentration of
culture and tourism interventions in urban areas (48.2%).
15 http://ec.europa.eu/regional_policy/sources/docgener/evaluation/data/erdf_projectselection_air_rawdata_
2007_13.xls. The 2017 - 2013 categorisation data (Project selection) should be available in October 2014 and
should be considered in the interim report.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 41
As regards the output indicators, the shortage of data for culture and tourism
must be highlighted. There are only two core indicators for ERDF to account for
achievements in the targeted sectors. They are:
Indicator n. (34) Number of projects in Tourism defined as the number of projects
aimed to develop the tourism industry in the region.
Indicator n. (35) Number of jobs created, defined as the gross direct jobs created
(in full time equivalents) by tourism projects counted under indicator 316.
These indicators will be analysed to highlight gaps with respect to the target value in
all OPs. Whenever feasible, the average financial allocation per project will be
calculated. We will also calculate the coverage rate of jobs created on the total
employment in the tourism sector (resulting from LFS microdata).
Core indicators were not mandatory but only recommended by the European
Commission, and so data on core indicators is likely to be scarce. A first indication on
WP0 data provided by DG Regio shows that only 120 OPs out of 253 with allocations in
priority themes directly linked to culture and tourism (including 104 OPs investing
more than 15 millions of Euros in these areas) report data on Number of Projects in
Tourism (Core Indicator 34), and only 70 (56 OPs with more than 15 millions of Euros
allocated) on jobs created (Core Indicator 35). Furthermore, in some case these
indicators are not consistent and reliable. As shown in table 3.9, at the end of 2012
the number of tourism projects were 8,600, 42% of the target, concentrated in Spain
(1,642 projects), Italy (1,443) and Portugal (1,076).
Jobs created were around 12,000 up to the end of 2012, 53% of the final target. The
highest achievement was 3,300 reported in HU (51.5% of target); also PL reports a
high number, with 1,900 tourism jobs (43.3% of final target) and DE with 1,347 (four-
times the final target) (Table 3.10).
16 Indicator 39 “Number of projects ensuring sustainability and improving the attractiveness of towns and
cities” will not be considered because it is not possible to derive the actual relationship with culture and
tourism from the available data.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 42
Table 3.9. Core indicator n. 34 - Number of tourism projects at the end of
2012 and % of the final target (ordered by Number of projects)
Country Number of projects % of the final target
ES 1642 21.9
IT 1443 31.5
PT 1076 81.3
HU 998 92.2
CZ 813 79.4
DE 647 159.8
PL 555 54.8
FR 291 67.4
LT 231 192.5
RO 180 45.0
SI 146 116.8
SK 136 36.2
GR 118 6.1
SE 102 510.0
UK 92
BE 37 160.9
NL 31 34.1
LV 28
MT 20 100.0
AT 14 155.6
DK 10 100.0
BG 8 11.4
HR 2 13.3
Total 8620 41.9
Source: calculation on DG Regio – Core Indicator AIR data
Table 3.10. Core indicator n. 35 - Number of jobs created in tourism at the
end of 2012 and % on the final target (ordered by Number of Job created)
Country Jobs created % of the final target
HU 3324 51.5
PL 1947 43.3
DE 1347 443.1
CZ 1092 67.1
IT 1050 76.0
SI 840 84.0
UK 520 1040.0
LT 461 92.2
SK 404 10.2
FR 349 103.8
PT 166 6.9
Total 11928 53.0
Source: calculation on DG Regio – Core Indicator AIR data
The interpretation of these data on achievements will take into account the strengths
and weaknesses in the reporting of the data, emphasizing significant gaps that
prevent a clear understanding of achievements. For example, there may be differences
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 43
in the definitions and methods of calculation adopted by MAs; differences in the
projects considered to calculate the indicator (on-going projects or only concluded
ones); differences in the measurement of jobs (often only the number of jobs is
reported without adjustment in full-time equivalents). The analysis of the reliability
and comparability of the core indicators will take into account the results of WP0.
In order to complete the analysis of available output indicators, we will also consider
those collected in OPs Monitoring Systems. Starting from the list of relevant indicators
identified by WP0 (see Table A8 in Annex II), we will classify and analyse all the
specific indicators related to culture and tourism. We expect that only an analysis at
the Programme Level will be possible. A first recognition on WP0 data provided by DG
Regio shows that only 110 OPs (86 investing more than 15 millions of Euros culture
and tourism) provide additional programme specific indicators related to culture and
tourism and that these indicators refer to: Additional and/or modernized tourist beds,
facilities; Number of visitors; Increase in number of nights; and Improved sites or
monuments.
The survey on MAs (task 2b) and the case studies will provide additional information
of the Programme results.
Task 2b - Survey of selected programmes
Aims
As required by the Terms of Reference, the survey will complement the assessment
performed under Task 2a. In particular, it will collect fine-grained information and data
on ERDF strategic expenditures as well as on outputs and results achieved in the field
of culture and tourism. Both quantitative and qualitative data will be collected in order
to investigate the rationales of ERDF investments for culture and tourism as well as to
understand the logic of intervention. The aim is to provide an overview of the
strategies underpinning these investments, a classification of the most common
typologies of activities implemented and of beneficiaries targeted. The survey will
serve as a tool to investigate the ERDF resources directly and indirectly addressed to
these two sectors as well as to assess whether synergies and complementarities exist
between ERDF interventions and other ESI funds (e.g. EAFRD, Cohesion Fund, ESF,
etc.).
The survey will target Operational Programmes investing more than EUR 15 million of
ERDF17 in support of culture and tourism activities. In order to select the Operational
Programmes for the survey, the amount of Structural Funds allocated to the
categories of expenditures in culture and tourism (55-60 plus hotels and restaurants)
was taken into account. On this basis, 158 OPs will be addressed, representing about
96% of total allocation of funds to culture and tourism. As shown below, more than
half of the number of OPs and total financial allocation are concentrated in a small
number.
17 As required by ToR, pag 14.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 44
Figure 3.3. Number of targeted OPs by countries
Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
Figure 3.4. Financial allocation of targeted OPs by countries
Source: calculation on DG Regio – ERDF AIR raw data (Project selection), 2007-2013
Table 3.11 shows the evaluation questions and the corresponding activities and parts
of the questionnaire which address them.
Table 3.11. Task 2b – Matching ToRs’ evaluation questions and questionnaire
Evaluation questions Questionnaire
What is the nature of the activities and main beneficiaries of the financial support (e.g. services vs. infrastructure, public vs. private, etc.,)?
Questions C.1, C.2, E.1
On which socio-economic objectives do the interventions focus on and what is the logic underpinning these
interventions? To what extent do interventions address
identified market failures?
Questions B.1, B.2, B.3, B.4, B.5,
B.6, B.7
Are clear changes (specific objectives) being targeted and quantified and how are they being monitored?
Questions B.8, B.9, B.10, D.1, D.2
What is the nature of the financial support (grants vs. financial engineering instruments)?
Question C.1
In the case of business support to private companies (e.g. hotels and restaurants) are the support measures targeted exclusively to tourism enterprises or do these enterprises compete on the same criteria applied to
enterprises in other sectors?
Question C.4
0
5
10
15
20
25
30
35C
B PL IT DE
FR ES PT
HU CZ
GR
UK SK EE BE
NL FI SE RO SI LT BG
MT
LV DK
AT
CY IE
Nu
mb
er
of
OP
s
79 Ops (50% of
targetedOps)
0
500
1000
1500
2000
2500
PL
GR IT PT
HU CB CZ ES DE
RO FR SK SI LT UK
BG EE MT
BE
NL FI SE LV DK
AT
CY IE
Mill
ion
s o
f EU
RO
6,8 Billions(53% of
targetedOps)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 45
What are the most commonly used measures of achievements (core indicators as well as programme
specific indicators - e.g. capacity of hotels, number of visitors and occupancy rate of facilities)?
Question D.1
What is the nature of the selection processes? Do selection processes, selection criteria and the choice of indicators address the assessment of the financial sustainability of projects?
Questions E.2 and E.3
Methodology
The Questionnaire
The draft version of the questionnaire included in the Technical Offer has been revised
according to the comments provided during the kick-off meeting and those received
on a previous versions of the Inception Report. In particular, more emphasis has been
placed onto the collection of quantitative data on the achievements indicators of the
OPs. Also, some multiple choice questions were changed into open-ended questions,
especially those related to the description of the strategies. The questionnaire is still
an early draft, expected to be improved and finalised on the basis of the literature
review (Task 1) and the analysis of ERDF expenditures and reported achievements
from MAs on core and specific indicators (WP0 considered in Task 2a). A final
questionnaire will be tested with a small number of MAs. The revised draft version of
the questionnaire is provided in Annex III.
The questionnaire is deliberately ambitious. Great efforts will be made - e.g. through
telephone recalls and e-mail reminders - to increase response rates. We will also
consider whether to make a selection of questions mandatory in order to allow more
reluctant respondents to concentrate on at least a minimum set of high priority data.
Running the Survey
The questionnaire will be available in different languages18. It will be administered with
an online tool and will be supported by follow-up calls carried out by country experts
in different languages. The survey will be undertaken through:
Step 1 – Preparatory activities
Preparatory activities for the launch of the survey, including:
collecting contacts of all the targeted Managing Authorities. The list of contacts
will be provided by the Commission services.
testing the questionnaire with a small number of MAs and fine-tuning if needed.
translating the questionnaires into relevant languages.
drafting an invitation and reminder letters, as well as a script for the follow-up
call.
18 Bulgarian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian,
Latvian, Lithuanian, Spanish, Polish, Portuguese.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 46
Step 2 – Launch of the survey to MAs
Once the contact list of the MAs has been gathered, and the questionnaire tested and
adjusted, the full-scale survey distribution takes place. Invitations to the survey will be
sent out via e-mail through an electronic system. The system provides a link to the
online questionnaire, which can be accessed through a standard Internet browser. By
clicking the link and selecting the preferred language, the MAs are able to respond to
the questionnaire.
A letter of presentation of the study will be attached to the invitations. It will focus on
the objectives of the exercise, including the advantages that feedback of the overall
evaluation exercise may have on their activities. The letter will make explicit that the
information provided will be used only in aggregate form, which should remove a
possible hesitation MAs may have about making their data available.
As the questionnaire is ambitious and is likely to require the collaboration of different
persons within the Managing Authorities to answer, a printable version will also be
attached to the invitation letter in order to provide the respondents with a preview of
the information required. . After the questionnaire has been filled in and submitted by
the respondent, it is transferred to a central database for storage and later processing
and analysis.
The survey will be open for about two months in order to achieve a balanced
distribution of respondents. During this period, a hotline will be available to answer
both technical and content-related questions. A reminder will be sent two weeks after
the launch of the survey to non-respondents.
Step 3 – Telephone recall
The IT solution5 used enables continuous monitoring of the survey and, in the specific
case, provides information on the number of respondents and their distribution by
country. The project team plans to spend significant resources on telephone follow-
ups, to achieve a balanced distribution of respondents, and so preserving the
geographical representativeness of the collected data.
In answering the questionnaire, MAs will be required to provide at least three contacts
of Implementation bodies. This will enable the project team to build a database of
contacts of Implementation bodies who will be contacted and required to answer
specific questions with the aim of having a more comprehensive picture of ERDF
effectiveness in supporting culture and tourism activities. The launch of an on-line
survey to Implementation Bodies – through a structured questionnaire - will be
considered on the basis of the response rates from MAs in providing useful contacts,
as well as of the time available for data collection.
Analysis of collected data
Collected data will be processed using standard statistical tools. The main statistics
describing variability of strategies, activities implemented across countries and OPs,
beneficiaries of interventions, results and outcomes achieved will be provided, e.g.,
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 47
standard deviation, max/min, quantile statistics. An extensive cross tabulation
analysis will be performed to provide a picture of the main features of the strategies
and operations e.g. typologies of actions, support, type of beneficiaries and the
relationship and the interrelationship between these characteristics and the types of
strategies and regions targeted.
In addition to descriptive statistics, specific graphical devices will be used to
investigate the data, such as histograms, (cumulative) density plots and box plots.
Finally, a segmentation analysis will be performed to identify the groups of OPs that
share similar characteristics in terms of strategies and actions implemented.
Segmentation analysis will help to answer questions like: how different are strategies
in the relevant Member States?
Table 3.12. Timetable of the survey activities October/November 1st week 2nd week 3rd week 4th week 5th week 6th
week Preparatory activities
Launch of the survey to MAs
Telephone recall
Interviews with IBs
Analysis of survey’s results
The table below provides an initial indicative illustration of the ERDF interventions and
their expected outputs and intended results (changes) in the two sectors of interest.
Table 3.13. Culture and tourism: ERDF interventions, expected outputs and
intended results ERDF interventions Expected outputs Expected results (changes)
CULTURE
Refurbishment/recovery and protection of historical monuments, buildings or archeological sites
• No. and type of new, restored/refurbished facilities or sites
• No. of visitors
• Contribution to job creation/maintenance and to the attraction and retention of highly skilled workers
• Increased attendance/ participation
to cultural events • • Increase and diversification of
audiences / participants/tourists • More competitive, and more
sustainable businesses / organisations
• Greater territorial attractiveness • Development of creative industries
and innovation in products, services and processes
• Improved community identity • Crossovers / spillovers effects
contributing to innovation and growth, such as the value design adds to other sectors
• Leverage effects on investments • Improved social cohesion and
living conditions • Improved institutional and
administrative capacity
Construction/extension/recovery of cultural infrastructure (e.g. museum, libraries, archives, etc.) or of infrastructure providing cultural services (e.g. theatre, concert hall, opera house, auditorium, art galleries, planetarium, etc.)
• No. and type of new, restored/refurbished facilities or sites
• No. of visitors
Organisation of cultural events (e.g. painting, sculpture, photography, etc.) or performing arts (e.g. film production, book publishing, etc.)
• No. and type of events promoted
• No. of audiences or participants
Support to creative industries (sports, media and information technology, etc.)
• No. and sector of firms supported
• No. of adaptations or innovations
Promotion of sports and recreational activities and events
• no. of events and recreational activities promoted
• Increased numbers and diversification of tourism flows
• Lower seasonality of tourism inflows
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 48
ERDF interventions Expected outputs Expected results (changes)
• Improved social cohesion and living conditions
Support to transport and ICT infrastructures
• no. and type of infrastructure supported
• Improved territorial accessibility • Increased numbers and
diversification of cultural tourism flows
• Lower seasonality of tourism
TOURISM
Investments to improve the safety and protection of natural assets
• no. and type of interventions and projects
• Reduction of adverse natural events
• Improvement in the quality of life for local population
• Increased tourism inflows • Contribution to job
maintenance/creation
Physical investments for the promotion and development of the tourism sector (e.g. information centres, etc.)
• No. and types of new/restored/refurbished infrastructures
• Leverage effect on investments • Greater quality /efficiency
/sustainability of the infrastructural endowment
• Increased arrivals/nights at tourists accommodation establishments
• Contribution to job maintenance/creation
• Improved supply of attractions and quality of life for inhabitants
Territorial marketing activities (e.g. promotional activities, networking, conferences and trade fairs)
• no. promotional activities, conferences and trade fairs
• Increased numbers and diversification of tourism inflows (arrivals/nights)
• Lower seasonality of tourism inflows
Support to private initiatives (e.g. hotel and restaurants)
• no. and type of support
• types of interventions supported (e.g. technological innovation, networking, environmental sustainability, etc.)
• no. and sector of firms supported
• contribution to Business maintenance/creation
• Improved quality of the hospitality sector
• Contribution to job maintenance/creation
• Improved quality /efficiency /sustainability of the sector
Transport and ICT Infrastructure • no. and type of
infrastructure supported
• Improved territorial accessibility • Increased numbers and
diversification of tourism flows • Lower seasonality of tourism
Source: Authors processing regulations and policy documents19
19 Part A – Code by Dimensions of the COMMISSION REGULATION (EC) No 1828/2006 of 8 December 2006
setting out rules for the implementation of Council Regulation (EC) No 1083/2006 laying down general
provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund
and of Regulation (EC) No 1080/2006 of the European Parliament and of the Council on the European
Regional Development Fund; Commission of the European Communities (2007b), White Paper on Sport,
COM(2007) 391 final. European Commission (2006), the Economy of Culture in Europe, available for
download on http://ec.europa.eu/culture/keydocuments/doc873_en.htm European Commission (2010b),
Green Paper: Unlocking the potential of cultural and creative industries, COM(2010) 183. European
Commission (2011), Ex-post Evaluation of 2011 European Capitals of Culture, final report drafted by Ecorys
on behalf of DG EAC. European Commission (2012), Communication from the Commission to the European
Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions:
Promoting cultural and creative sectors for growth and jobs in the EU, COM(2012) 537 final. European
Parliament (2012), Use of Structural Funds for cultural projects, DG for Internal Policies, Brussels. JASPERS
(2011), Best Practice in the Preparation of Projects in the Culture Sector, Staff Working papers.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 49
Task 2c - Tabulation of findings and proposal for case selection
The aim of this sub-task is to cross-analyse the outcomes of Task 2a and 2b in order
to classify EU regions according to the different culture and tourism strategies
identified in the literature review (Task 1). As requested in the ToRs, this task will
provide a systematic mapping of EU regions and a summary table presenting for each
region/programme: the “initial financial allocation”; the “financial allocation at the
moment of the MAs survey”; the “expected final allocation”; the “outputs of
interventions”; and the “achieved/expected results”.
This multi-dimensional analysis will be at the basis of the selection of nine regional or
programme case studies and 18 “mini case studies" (two for each of the 9
regional/programme strategies proposed). From among the nine proposed programme
cases, six will be then identified with the associated pair of mini case studies in
agreement with the European Commission and will be subject to the in-depth analysis
described in task 3.
Table 3.14 shows the evaluation questions and the corresponding activities.
Table 3.14. Task 2c – Evaluation questions and activities Evaluation questions Activities and deliverables
Proposal of nine regional or
programme case studies and of 18
mini case studies
Classification of EU regions based on ERDF and context indicators Selection of nine regional or programme case studies and the corresponding 18 mini case studies (2 for
each regional/programme case) on the basis of the
findings of the previous Tasks context indicators on the relevance of the culture and tourism sectors in EU regions. Description of the proposed programmes’ strategies
Methodology
The selection of the 9 case studies (Programmes) and 18 mini case studies (Projects)
will be based on the following steps:
1. Classification of the EU regions according the culture and tourism strategy
undertaken and relevance of the sectors in the regional economy;
2. Identification of additional criteria to describe regional patterns and to select the 9
case studies;
3. Identification of 2 more and less successful mini case studies for each one of the 9
case studies.
Step 1 - Classification of the EU regions
The classification will provide a mapping of the surveyed regions on the basis of the
different regional/national strategies in culture and tourism individuated in the
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 50
previous Tasks. While the Literature review (Tasks 1) will identify the different
strategy models of culture and tourism, data and analysis from Task 2a and 2b will
allow us to classify EU regions according to the strategy models identified, the ERDF
financial allocation, the output and results indicators and the relevance of the culture
and tourism sectors in the regional economy.
In particular we will consider the following data:
For the strategy dimension (D1)
o The presence in the OP of a specific axis on culture and/or tourism and the
average amount of financial allocation per project (questions B7 and C1 of the
survey). Also the presence of big projects will be taken into account.
o The typology of investment strategy in terms of objectives (questions B8 and
B9 of the survey) and the nature of activities implemented for the culture and
tourism sectors (Question C1 of the Survey);
o The distribution of ERDF allocations by each priority themes (resulting from
task 2a);
o The beneficiaries of the activities implemented (Question E1 of the survey).
For the ERDF financial dimension (D2) the 2013 ERDF allocations to the
sectors of interest (resulting from task 2a) in order to individuate regions with
high/medium/low ERDF relevance in these sectors (with respect to the EU
average). We propose to consider the incidence of ERDF allocations for culture and
tourism (plus hotels and restaurants) on the overall OP.
For the achievements dimension (D3): Core Indicators 34 and 35 as well as
other relevant indicators (see task 2a and question D1 of the survey), we propose
to classify the OPs according to the gaps between the achievement in 2013 with
respect to the target value. We will also consider the physical (Completed
projects/Total projects) and financial performance (from questions C1 of the
Survey);
For the relevance of the sectors in the regional economy (D4) we will
individuate similar EU regions in terms of the relevance of culture and tourism
sectors and their trends during the programming period on the basis of a cluster
analysis (see Box 3.1) performed on the culture and tourism indicators usually
adopted in the literature and available at the regional level. Compared to a
descriptive analysis, which does not allow an easy handling of different variables
simultaneously, the cluster analysis allow us to “classify” multivariate entities in
few categories a priori not defined. Given the regional focus of the analysis, we
have to use harmonized comparative data available in the EUROSTAT Regio
database or in other European official sources of information (e.g. the European
Structural Business Survey and the LFS Microdata, see the following Box). The
selection of the proposed case studies will take into account the coverage of the
different regional clusters resulting from the analysis.
The output of this first step will be a qualitative table mapping the surveyed
programmes/regions according to the type of strategy undertaken; the initial, current
and expected final ERDF allocation; ERDF outputs and achieved/expected results;
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 51
relevance of the culture and tourism sector in the regional economy.
Box 3.1 - Relevance of the culture and tourism sector in the regional economy: a cluster analysis
A cluster analysis will be carried out in order to classify EU regions according to comparable context indicators and to individuate similar EU regions in terms of the relevance of culture and tourism in the regional economy and their trends during the programming period. On the basis of the above mentioned indicators, regions will be clustered according to an agglomerative hierarchical technique which undertakes a succession of mergers of n units, starting from the baseline situation in which each unit is a group in its own right up to the stage n-1 which forms a group that includes them all. Agglomerative hierarchical clustering methods begin with each
observation considered as a separate group (n group each of size 1). The closest of two groups are then combined (n-1 groups, one of size 2 and the rest of size 1), and this process continues until all observation belong to the same group. To cluster the individual units, we will use the method of Ward in which the pair entity to be aggregated to a certain step of the analysis is the one that minimizes the deviance between the centroid of the groups. Ward’s method joins the
two clusters that result in the minimum increase in the error sum of squares. The Euclidean distance between an entity k and a new group (i, j) is the square root of
where nk is the number of units that make up the group k and n(ij)= ni+nj. The territorial unit of analysis is the NUTS II level (except for Germany and UK where the NUTS I level is considered), and we will consider changes between average 2005-2007 and average 2012-2013 (when available). On the basis of a first recognition of available data, we propose to consider the following context
indicators: List of illustrative “context” indicators
INDICATOR NATIONAL/ REGIONAL FIELD PERIOD
SOURCE
Business Local Unit in culture and tourism (Number and Incidence on
the total)
Regional culture
and tourism
2008-2012 Eurostat: Structural Business Survey
(sbs_r_nuts06_r2)*
Number of persons employed
(Number and Incidence on the
total)
Regional culture
and tourism
2008-2012 Eurostat: Structural Business Survey
(sbs_r_nuts06_r2)*
Growth rate of employment (%)
Regional culture
and tourism
2008-2012
Eurostat: Structural Business Survey (sbs_r_nuts06_r2)*
Employment in the
accommodation sector
Regional Culture
and Tourism
2005-2012
LFS Microdata**
Number of establishments, bedrooms and
bed-places
Regional Tourism
2005-2013 Eurostat: Regional Tourism Statistics
(tour_cap_nuts2)*
Nights spent at tourist
accomodation establishments
Regional Tourism
2005-2013
Eurostat: Regional Tourism Statistics (tour_occ_nin2)
Arrivals at tourist accommodation establishments
Regional Tourism 2005-2013
Eurostat: Regional Tourism Statistics (tour_occ_arn2))
*We will consider the following codes: G476 - Retail sale of cultural and recreation goods, I- Accommodation and food service activities, N79 - Travel agency, tour operator reservation service and related activities.
)(2
)
)(
(ijk
ijk
ijkd
nn
nn
= ijkjkkjikki
kji
dndnndnnnnn
2221
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 52
** unpublished Eurostat LFS microdata on employment in culture and tourism sectors. We will consider the following NACE Rev. 2 codes: 55 – Accommodation; 56 – Food and beverage activities; 90 – Creative, arts and entertainment activities; 91 – Libraries, archives, museum and other cultural activities; 93 Sports activities and amusement and recreation activities; 96.04 – Physical well-being activities (where available). A specific request to obtain these microdata will be sent through the DG Regio Evaluation Unit. It should be pointed out that this data will be considered according the Eurostat reliability (limits a and b).
Step 2 - Identification of additional qualitative criteria and selection of nine case
studies
The nine programmes/regions will be selected among those grouped according to the
metholdology presented in step 1 and the following complementary and integrative
criteria:
Programme coverage: One Cross Border Programme and a combination of
“Convergence” and “Competitiveness and Employment” regions according to
the weight of the financial allocations in the two groups of regions;
Geographical distribution: north, south, central and eastern Europe and
distribution between old/new member states;
Urban vs rural regions and topographical characteristics: Mountains, Coastal,
Other;
The selected nine cases will have to cover at least the two main types of
culture and of tourism industries.
The 9 case studies will thus be selected to represent different culture and tourism
strategies and the above-mentioned additional criteria.
Step 3 - Identification of the Mini Case-Studies
As foreseen by the tender, besides case studies (representative of different typologies
of investment strategies), the evaluation will include also two mini case studies
(representative of different typologies and performance of projects) for each
programme/region case study.
The 18 examples of Mini Case studies will cover both successful and less successful
practices.
Successful practices refer to innovative projects that achieved:
good financial and physical performance;
sustainable results that contributed to the tourism and/or culture sectors and
so to the region’s socio-economic development (for further details on potential
results in the two sectors see task 1);
improvements in the capacity to capitalize the results produced (i.e. exploit
infrastructure created).
Other success criteria include: adoption of an integrated strategy; pluralism and
stakeholders participation; grounding; and transferability20.
20 European Commission, DG REGIO, (2008) Analysing ERDF co-financed innovative projects. Case study
manual.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 53
Less successful practices are those characterised by financial/physical/partnership
criticalities that blocked the development of the project or where a major challenge
was overcome in the implementation. We are interested especially in this latter type of
projects as they allow us to derive lessons on potential barriers and how to overcome
them.
In selecting both successful and less successful projects, we will look at projects that
are either relevant to the strategy objectives or from a financial perspective. Selected
projects will be representative of the different types of strategies identified during the
literature review.
The selection of mini case studies will be based on information deriving both from the
literature review (task 1) and the indications from survey to the MAs (questions F1
and F2).
Outputs
The results of task 2 will be presented in Part II of the intermediate report. This will be
structured in four parts:
1) A descriptive picture of EU regions/programmes according to the culture and
tourism indicators in terms of financial allocation and achievements of ERDF
interventions; in addition to descriptive statistics, specific graphical and mapping
devices will be used to present the data at EU regional level (dispersion and plot
graphs and maps).
2) The evidence resulting from the survey on the MAs of programmes investing
more than EUR 15 million in culture and tourism. An extensive cross tabulation
analysis will be performed to provide a picture of the main features of the
strategies and operations (e.g. typologies of actions, support, type of
beneficiaries and the interrelation between these characteristics and the types of
strategies and regions targeted) and of the main outputs and results of the ERDF
supported interventions.
3) The results of the classification of EU regions according to the strategy models
resulting from Task 1 by combining the findings from task 2a with the evidence
from the MAs survey.
4) The selection of the 9 case studies. For each one of the proposed 9 case studies
and of the 18 mini case studies the report will present a brief description and the
motivation of the choice. Moreover a synthetic table resuming the nine case
studies in terms of strategy undertaken (for example the presence of a specific
axis in Culture and/or Tourism , if they are part of any urban development
strategies or fall outside of urban strategies), relevance of culture and tourism,
motivation of choice and Mini Case Studies will be elaborated.
Table 3.15 presents the main challenges and the possible solution in carry out the
activities of Task 2.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 54
Table 3.15. Task 2 - Main challenges and possible solutions
Risks expected Solutions
Task 2a Difficulties in identifying and assessing specific programme indicators.
Integration with qualitative information on outputs/results achieved from the Web Survey (Task 2b) and stakeholders‘ interviews (Task 3).
Task 2b Low response rate from MAs to the questionnaire MAs reluctant to answer all the questionnaire since it looks too time-consuming. After three weeks from the launch of the survey, the response rate is still far from target. The questionnaire may require the collaboration of different persons within the Managing Authorities to provide all the answers. MAs are reluctant to provide contacts on Implementation bodies.
A reminder to non-responders will be sent after two weeks from the launch of the survey (or every week if needed). Only a selection of questions will be made mandatory in order to allow more reluctant respondents to concentrate on at least a minimum set of high priority data. Resource allocated to telephone follow-ups to ensure that a sufficient number of MAs respond. It will be made possible within the system that different people access the on-line questionnaires in different moments and that partial responses are saved and stored until the overall questionnaire is responded. Resource allocated to telephone follow-ups to ensure that a sufficient number of contacts of the Implementation bodies are provided.
Task 2c Difficulties in classifying EU regional strategies Missing data for the context indicators Shortage of comparable regional indicators on culture Difficulties in clustering EU regions
Triangulation of information from Task 1 and 2b and indications from Thematic Experts and Scientific Advisors We will consider the possibility to use proxies (e.g. national values or available values for comparable regions) Use of indications from task 1 We will consider different clustering specifications (different variables, different unit of measure and definitions)
3.2.3 Task 3 - Case Studies and Mini Case Studies “What Happens on the Ground?”
Task 3a - Regional/Programme Case studies
Aims
The case studies aim to gain a better understanding of the ERDF contribution to the
overall objectives of cohesion policy in the sectors of culture and tourism.
Following the selection of the case studies (Task 2), we will carry out 6 in-depth ERDF
programme/regional case studies, thereafter called case studies. The analysis of the
case studies will address different types of ERDF strategies implemented in the sectors
of culture and tourism. Each case study will be accompanied by two mini case studies,
pursuing one more and one less successful project implemented within the regional
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 55
programme.
Case studies will explore whether and how the expected results of the ERDF strategy
are achieved and what factors contributed to the achievement of results. Case studies
will also provide information on how the different ERDF strategies deliver growth and
jobs in the sectors of culture and tourism. By doing so, case studies will test the
hypotheses of change resulting from task 1 , which will be further refined in the
literature review.
In detail, the case study analysis will consider the following issues:
the context of the intervention with specific regards to the sectors of culture and
tourism and their specific barriers to growth;
the rationale of the public/ERDF intervention in the sectors of culture and tourism,
with a particular focus on the take-up of market failure and analysis of future
demand, typologies of interventions selected, policy mix and expected changes;
the features of the ERDF strategy that has been implemented;
the implementation process of the ERDF strategy, with a particular focus on the
actors involved, and the main barriers to address;
the effectiveness and sustainability of the ERDF strategy;
the lessons for future policies in the two sectors, with a particular focus on the
drivers (social mechanisms) useful to explain specific outstanding results and/or
failures.
The table below provides a match of the evaluation questions to be addressed by this
task and the evaluation activities put in place.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 56
Table 3.16. Task 3 - Evaluation questions and activities
Evaluation questions How information is collected: case study activities
How information is synthesized in the case study template
What are the regional strategies underpinning public
support and how do they identify priority
investments in these specific sectors? How has
future demand been analysed?
Desk analysis
Regional briefing
Semi-structured
interviews
Rationale and intervention
logic
Programme supporting
features
Are the supported projects within the investigated
cases financially sustainable? How do selection
criteria and the choice of indicators contribute to
assure the financial sustainability of projects in the
selection process?
Desk analysis
Regional briefing
Semi-structured
interviews
Mini case studies
Programme supporting
features
Outputs achieved
Results achieved
What are the main changes (objectives) that
programmes are attempting to achieve and what
results have been achieved (or are likely to be
achieved) via the support to specific projects?
Desk analysis
Regional briefing
Semi-structured
interviews
Rationale and intervention
logic
Outputs achieved
Results achieved
To what extent do the projects supported represent
a cost-effective means to foster economic growth for
the Member State or region? Could such a support
be better targeted to this end?
Desk analysis
Regional briefing
Semi-structured
interviews
Mini case studies
Outputs achieved
Results achieved
5. Conclusions and
lessons learned
Comparative analysis and
lessons
What are the main drivers of success and failure
associated with these strategies and projects?
Desk analysis
Regional briefing
Semi-structured
interviews
Mini case studies
Conclusions and lessons
learned
Comparative analysis and
lessons
Methodology
Case studies will be conducted by country experts on the basis of a common
methodological framework that will be provided by the core team.
A. Contents of the case study
The box below presents the case study draft template. The template will be further
refined during the project based on the findings of the literature review and web-
based survey.
Box 3.2 - Proposed regional case study template
Executive summary
1) The regional context
2.1. Overall picture of the socio-economic conditions and trends in the region 2.2 Overall picture of the culture and tourism conditions and trends in the region
2) The regional ERDF 2007-2013 strategy in the culture and tourism sectors
3.1 Chronology
3.2 The rationale and the intervention logic 3.3. The programme supportive features
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 57
3.4. The implementation process: phases and actors
3) Effectiveness and sustainability of the strategy: outputs and results
5.1 Outputs achieved (qualitative and quantitative evidence) 5.2 Results achieved (qualitative and quantitative evidence)
4) Conclusions and lessons learnt
The following sections detail the methodology proposed for each section of the case
study.
1. The regional context
This chapter aims to set the context of the ERDF regional programme strategy at the
beginning, during and at the end of the 2007-2013 programming period. It will
provide information on the:
socio-economic conditions of the analysed region;
main features of the culture and tourism sectors in the region.
Socio-economic conditions and trends in the region
The section aims to provide information on the regional socio–economic structure and
its position in the national and EU context, underlining the main challenges for
regional socio-economic development faced by the Region.
The analysis of the features and changes in the socio-economic situation of the region
over the 2007-2013 period will include information on the following indicators.
Table 3.17. Relevant socio-economic indicators of the Region in the 2007-13
period
Regional National EU
2007 2013 ∆% 2007 2013 ∆% 2007 2013 ∆%
Total Population
Population over 65
Population Education level
Net migration
GDP per capita
Working age population
Employment rate
Unemployment rate
Where available, data should be presented by gender and additional information that
supports the analysis will be included.
Information sources: Eurostat database, regional and national official statistic
databases. The core team will provide country experts with preliminary information on
these indicators. Statistics will be integrated with information on the main changes
and challenges related to the socio-economic and territorial development in the region
as identified from the regional briefing with stakeholders and the semi-structured
interviews.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 58
Main features of the culture and tourism sectors in the region
The section provides information on the main characteristics of the culture and
tourism sectors in the region over the 2007-2013 programming period and their
influence on the regional/national economy during this period. It includes quantitative
and qualitative information on the characteristics of the two sectors, with particular
regard to:
General data on the culture and tourism sectors, and
The policy framework governing the two sectors.
General data on the culture and tourism sectors
The section has the goal of describing the relevance of the culture and tourism sectors
for the regional economy, the main trends and challenges of the two sectors in the
period under consideration.
The analysis will provide general data and descriptions on the evolution of the two
sectors before, during and after the 2007-2013 period. Particular attention will be paid
to the impact of the economic crisis on the culture and tourism sectors.
Furthermore, if data is available, the analysis of the evolution of the two sectors will
go back before 2007.
The table below provides an indication of the minimum information to be analysed in
this paragraph.
Table 3.18. – A general overview of the culture and tourism sectors
Regional National EU
2007 2013 ∆% 2007 2013 ∆% 2007 2013 ∆%
Arrivals at tourist accommodation establishments
Nights spent at tourist accommodation establishments
Number of establishments, bedrooms and bed-places
Incidence of international tourism receipts on GDP (%)
Business Local Units in culture and tourism (number and incidence on the total)
Number of workers employed in the sectors of culture and tourism (number and incidence on the total)
Share of the region covered by a policy or plan that protects cultural heritage
Share of the region’s biggest events that are focused on traditional/local culture and assets
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 59
Additional information21 that supports this analysis will be included, if available.
In commenting on data, the main challenges in the region in terms of development of
the culture and tourism sectors have to be underlined.
Information sources: Eurostat database, regional and national official statistic
databases. The core team will provide country experts with preliminary information on
these indicators. Statistics will be included, alongside information from regional
briefing with stakeholders and semi-structured interviews.
The policy framework governing the two sectors
This subsection aims to place the ERDF 2007-13 regional case within the general
policy framework at national and regional level.
The analysis will describe the main national/regional policies implemented in the two
sectors, their goals and their organization, with particular reference to the main public
and private actors involved in the programming and delivering of culture and tourism
policies.
Particular attention will be devoted to the stability and continuity (or the main
changes) of the policy framework with respect to the previous programming period
(2000-2006).
The analysis will summarize the resources allocated globally to the culture and tourism
policy during the 2007-2013 programming period and the incidence of ERDF funds on
the total allocations in this period.
Information sources: the information will be collected mainly through desk analysis. It
may be integrated with information from the briefing to regional stakeholders and
semi-structured interviews.
2. The regional 2007-2013 ERDF strategy in the culture and tourism sectors
This chapter provides information on the analysed strategy and, in particular, on:
main events (chronology); the rationale of the ERDF intervention in the sectors of
culture and tourism; the main features of the strategy; the governance and supportive
features; the implementation process and actors involved.
21 If available for the analysed regions, some other indicators to be considered might be: nights spent at
tourist accommodation establishments by degree of urbanisation; Nights spent at tourist accommodation
establishments by coastal and non-coastal area; net occupancy rate of bed-places and bedrooms in hotels
and similar accommodation; number of establishments, bedrooms and bed-places by degree of
urbanisation; number of establishments, bedrooms and bed-places by coastal and non-coastal area.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 60
Chronology
This section provides a brief chronology of the main events/phases in the
programming, implementation, monitoring and evaluation of the analysed strategy
(e.g. starting date of the programming; starting date of the implementation; etc).
The rationale and the intervention logic
The goal is to reconstruct the rationale behind the analyzed ERDF programme
strategy, providing insights into:
The justification of the public intervention, and in particular the ERDF
intervention in the sectors of culture and tourism. Particular attention will be
paid to the consideration of specific market failures22 in the sectors of culture
and tourism that underpinned the development of the strategy.
The main changes (objectives) pursued by the strategy and the expected
results. Particular attention will be paid to the targets identified;
The elements of the strategy (types of interventions/operations) intended to
contribute to the expected changes in these two sectors, and in particular, the
types of interventions and the main involved beneficiaries.
The reconstruction of the underlining theory of change of the strategy will serve as a
reference for the assessment of its contribution to the expected results, and for the
identification of unintended/unexpected results.
The reconstruction of the ERDF regional strategy will be based on the web survey
(task 2b), refined by country experts (phase B Collecting preliminary information)
according to a shared template, and discussed during the regional briefing with
stakeholders in order to be refined/completed.
A check list will be provided by the core team to allow comparisons among the two
sectors and the six cases under analysis. The checklist will be refined after the
completion of the literature review and will consider the following dimensions:
Types of intervention: which are the main types of operations foreseen by the
strategy (with reference to the operations defined in Task 1);
Type of beneficiaries: which are the main beneficiaries of the strategy: i.e.
economic actors, local public institutions, universities/research centres, social
actors, etc;
Synergies with other wider strategies: whether the analyzed ERDF strategy for
culture and tourism is part of wider urban or rural development strategies
and/or other relevant regional/national frameworks;
22 Some examples of market failures in the two sectors might be : lack of private operators in the sectors ;
constraints in accessing capital markets; lack of profitability; lack of accessibility infrastructures; lack of
cultural/leisure infrastructures; small dimensions/fragmentation of the market; lack of entrepreneurial
capacities; etc
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 61
Continuity: whether it draws on previous ERDF strategies in these fields;
Centralization/decentralization: the use of subdelegation by the Mas;
Holistic integration: whether it addresses culture and tourism in an integrated
and holistic way (namely including interventions in different tourism and
cultural fields,23 and/or addressing interventions regarding both internal and
external tourism);
Sectoral and financial integration: whether it is sector or policy integrated24;
whether it integrates infrastructural interventions with immaterial ones
(training, awareness raising, etc); whether it is integrated with other different
type of funds (EU/national/regional funds);
Financial sustainability: if and how it considers/addresses financial
sustainability, and market risks of the interventions implemented;
Expected changes.
Information sources: the information will be collected through desk analysis, regional
briefing with relevant stakeholders and semi-structured interviews
The programme supportive features
The analysis aims to assess to what extent the ERDF strategy has been supported by
appropriate supportive features25 to favour the achievement of results.
The analysis will take into consideration the available documentation (AIRs,
evaluations) and the opinions/judgments of MAs and other stakeholders.
Particular attention will be paid to triangulate the sources of information, to
appropriately balance personal opinions.
Mini case studies will provide for more details and specifications on these elements.
A checklist will be prepared by the core team in order to facilitate the gathering of
information and final comparisons among the two sectors and the six cases under
analysis (cross analysis).
The supportive features under analysis will be the following:
Use of information and tools in order to analyse and monitor future demand;
23 Such as: costal tourism, mountain tourism, cultural tourism, creative tourism, educational tourism,
gastronomic tourism, religious tourism, arts tourism, heritage tourism, wellness and spa tourism, sports
tourism; social tourism; etc.
24 Namely, it includes interventions in different policy fields, such as: tourism/culture and ICT,
tourism/culture and environment; tourism/culture and energy, tourism/culture and transport,
tourism/culture and innovation, etc
25 “Implementing features” are those that directly implement basic mechanisms. For example, actors’
enhanced propensity to renovate tourism accommodations could be fostered by a substantial grant.
Supportive features are those elements (programme processes, inputs, resources…) used to bring the
implementing features into being. They have a less directly instrumental role in fostering change, but they
support the general working of the programme. The difference between implementing features and
supportive features has been underlined by Bardach, 2004, Presidential Address. The Extrapolation
Problem: How Can We Learn from the Experience of Others?
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 62
Presence and adequacy of selection criteria in order to select financial
sustainable projects;
Presence and quality of the monitoring and evaluation systems and the use of
performance indicators (to assure the financial sustainability of projects in the
selection process);
Use of public-private partnerships for the delivery of the financed interventions;
Typologies of contracts agreed for the delivery of the interventions (grants;
tenders; conventions; etc) and how they intended to favour the implementation
process26;
Use and effectiveness of tools for the involvement of the stakeholders and the
management of partnerships.
Information sources: the information will be collected through desk analysis, regional
briefing and semi-structured interviews with relevant stakeholders, including
implementing bodies, business association representatives, civil society organisations
and representatives of beneficiaries, etc..
The implementation process: phases and actors
The aim of this section is to provide an overview of main phases of the programme
and the policy network involved.
This section provides information on:
the main phases the programme (length of the programming/implementing
phases; problems encountered during each phases),
the main actors involved and, in particular: their type
(bureaucrats/administrators, politicians, economic actors, social actors,
experts) and their level of action (international/EU, national, regional and
local).
Information sources: The information will be collected through desk analysis, regional
briefing with relevant stakeholders and semi-structured interviews
3. Effectiveness and sustainability of the strategy: outputs and results
This chapter aims to assess the effectiveness of the ERDF strategy. It considers the
outputs of the strategy and its direct and indirect results on the two sectors and on
regional development.
Priority is placed on the collection of objective quantitative and qualitative
evidence. Quantitative data, in particular outputs and some results indicators, are in
principle available from the monitoring systems of the MA27. Evaluation reports could
26For example: existence of clauses referring to solutions of controversies, rules for decision-making,
sanctions/rewards for the respect of time schedules, process monitoring, setting of performance indicators
to be achieved, use of ex ante, mid-term and ex post evaluation
27 A qualitative assessment of the way this principle is reflected into reality will be also provided.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 63
also contain additional information on results. Furthermore, quantitative data on
outputs and results indicators will also be found in the Cohesion Policy database
collecting all indicators presented in the final annual reports of the ERDF programmes.
Other more qualitative information will be collected from the interviewed MA and
relevant stakeholders. In order to limit the subjectivity of such information, particular
attention will be paid to the triangulation of sources and their integration with other
more “objective” evidence such as quantitative data.
Output achieved
The output analysis takes into consideration the financial (expenditure) and physical
performance of the strategy.
The financial performance considers the overall degree of absorption of the
strategy (resources programmed, allocated and spent) and of the different
types of interventions promoted within the strategy.
The physical performance includes an analysis of the strategy’s capacity to
achieve the programmed physical targets. Moreover, it will also include details
on the most relevant projects financed within the different types of
interventions and their financial and physical achievements.
The sustainability analysis focuses on whether 1) the projects financed are
likely to continue to operate in the absence of ongoing ERDF or other public
financial support and 2) to what degree sustainability has been promoted using
appropriate selection criteria. The assessment will verify whether the main
financed projects include, for instance, an analysis of changes in demand and
appropriate governance structures, which will assure the continuity of the
results achieved; whether other funds are needed/available for the sustaining
the interventions after the end of the ERDF co-financing; whether there is an
improvement in the capacity of the actors involved to exploit and manage the
infrastructures and facilities created. Specific information will be drawn from
the mini case study analysis.
Information sources: The information will be collected through desk analysis, regional
briefing with relevant stakeholders and semi-structured interviews, mini case study
analysis.
Results achieved
The analysis of results aims to test the theory of change underlined in chapter 3.1.
Particular emphasis is put on the main effects of the strategy on the performance of
the two sectors, in terms of contribution to the regional economic and social
development.
Attention will be paid to expected results as well as to obtained results (even if not
expected). Where results are not fully deployed due to the reduced time span between
implementation and evaluation, intermediate results or expected results will be
considered as well.
Information sources: The information will be collected through desk analysis, regional
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 64
briefing with relevant stakeholders and semi-structured interviews
4. Conclusions and lessons learnt
This chapter aims to provide conclusions and specific lessons for future policy in the
two sectors on the following issues:
Whether the regional strategy is based on and centered around a sound
analysis of future demand;
Whether and to what degree the intervention achieved the expected outputs
and results;
What are the main differences between actual (or likely to be achieved) results
and expected ones, and what explains the possible differences
(wrong/irrelevant strategy; implementation failures; etc.)
Whether the programme features include appropriate selection criteria and
indicators and the extent to which and how they contribute to fostering the
financial sustainability of projects;
which are the more relevant (positive or negative) effects achieved.
Furthermore, social mechanism theory will be used as an additional tool to better
understand what has or has not worked and why.
Conclusions and lessons of each case study will be horizontally assessed in a final
comparative analysis (see below).
Information sources: The section will draw on the analysis carried out in the previous
sections.
B. Activities and steps to carry out the case study
The development of the case studies will follow seven steps, as illustrated below.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 65
Figure 3.5. – Steps in the development of the case studies
Each of these steps is described in detail in the following paragraphs.
A. Drafting templates and guidelines
Prior to the case study analysis, the core team will draft a case study template and
guidelines for the semi-structured interviews that will guide country experts in data
collection and analysis.
The drafting of the case study template and interview guidelines will draw on the
findings of the literature review (Task 1) and of the survey to MAs and implementation
bodies (Task 2b).
Case studies will be a maximum of 25 pages. Table, graphs and maps will be included
as required to clearly communicate the main points.
B. Pilot case study
The pilot case study will be developed by the core team using the template drafted
under phase A. The aim is to test the case study template and research tools,
adjust/refine them if necessary, and to provide country experts with an example of
how to conduct and draft the case study.
On the base of a preliminary analysis of the main OPs indicated in the tender and
dedicating significant financial allocations to culture and tourism, we propose the
Puglia ROP for the pilot case study (see Annex 4 for the rationale for this choice).
C. Online briefing with the experts involved in the drafting of the case study
Because of the complexity of the subject and the need for a coherent approach, we
will run an online briefing for all country experts involved in conducting the case
A
Drafting templates
and guidelines
B
1 Pilot case study
C
Online briefing
with country experts
D
Collecting preliminary information:
Web survey
Desk research
Contact with MA
E
Gathering data and
information:
Desk research,Regional briefing,
Interviews to stakeholders
F
Drafting 6 case studes
and two mini case
studies
G
Conclusions and lessons
xxCaCase studies 6 Case studies
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 66
studies.
The briefing is crucial in ensuring experts have a good understanding of the case study
approach and methodology, for achieving a common view on how case studies can
make the most effective contribution to the overall evaluation. Templates and
interview guidelines will be presented and discussed, as well as the timeframe and
deliverables. The briefing will be a morning or afternoon online workshop.
Before attending the briefing, experts will be asked to carefully read the reports on
Task 1 and 2 and the Pilot case study produced by the core team.
D. Collecting preliminary information from secondary data
Different documents on the regional ERDF strategy will be reviewed through desk
analysis in order to collect information on the issues identified in the case study
template.
The desk analysis phase starts with collecting and reading through relevant documents
(programme/s documents and reports, annual implementation reports or other
available monitoring reports, ex ante/mid-term/ex post evaluations, regional/national
studies, official programme websites, regional managing authority websites, press
review, etc).
The desk analysis phase also includes an analysis of quantitative data (deriving mainly
from the monitoring system put in place by the MA and from the evaluation reports
produced), with reference to the analysis of the effectiveness of the ERDF strategy,
and which will be completed using information from the semi-structured interviews.
Prior to the desk analysis, country experts will be asked to contact the MA, introduce
themselves (if the previous contact is done by the core team), and explain the project
and aims of the case study. They will first make an approach by email, ahead of an
introductory phone call.
The first conversation is crucial as it facilitates a more in-depth introduction to the
study. It should be also used to ask for official and unofficial documents on the
programme and for the monitoring data (monitoring reports or access to the
database). Furthermore, it is useful for moving forward with the planning of the
fieldwork activity, and, in particular, of the regional briefing with relevant stakeholders
of the ERDF strategy.
All the materials collected by the country experts in this phase will be organized along
the structure of the case study template.
Country experts should also make a list of the actors involved in the case to be
analysed as described in the official and unofficial documents. The list will be useful
during fieldwork when identifying actors to be interviewed.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 67
E. Fieldwork: Collecting primary data
The fieldwork consists of a regional briefing with relevant stakeholders of the analysed
ERDF strategy, and of semi-structured face-to-face interviews.
Regional briefing with the main stakeholders of the ERDF strategy
The regional briefing will introduce the project and the case study to the programme
authorities (representatives of the MA), and to relevant stakeholders. The regional
briefing aims to integrate the secondary data obtained from the desk analysis with
other quantitative and qualitative information on the issues indicated in the case study
template. Moreover, during the regional briefing country experts are urged to plan the
fieldwork.
Stakeholders to be involved in the regional briefing will be identified based on the list
of actors involved in the strategy, as identified during the desk analysis and the
contacts indicated by the MA. Up to 10 stakeholders will be invited to take part in the
briefing.
Semi-structured interviews
In order to collect information on the issues mentioned in the case study template,
country experts will carry out face to face semi-structured interviews with policy
makers (MA of the ERDF regional/programme strategy, regional and other
administrations’ staff in charge of designing and implementing the regional strategy,
etc), relevant stakeholders (politicians, bureaucrats, economic actors, social actors,
academic and practitioners), relevant beneficiaries, and independent evaluators.
Based on the actors involved in the analyzed case, the number of interviews should
vary between a minimum of 10 and a maximum of 15.
Interviews will be conducted face-to-face in order to allow him/her to freely express
his/her opinions. Country experts can also consider a group interview if it is not
feasible to meet stakeholders separately or if they think this would bring an added
value to better understand some of the process dynamics. In case of limited time for
the face-to-face interview, experts can also consider realizing some complementary
phone interviews afterwards. However, phone interviews should occur soon after the
face-to-face interview so that respondents are still familiar with the issues discussed.
A careful selection of interviewees will be carried out to ensure those chosen include
the key managers who have the ‘institutional knowledge’ of what happened in the past
and why. Actors to be interviewed will be selected through the snowball method28. As
the success of this technique depends greatly on the initial contacts and connections
28 Snowball sampling is a special nonprobability method used when the desired sample characteristic is rare.
Snowball sampling relies on referrals from initial subjects to generate additional subjects. As the success of
this technique depends greatly on the initial contacts and connections made, it is important to know the
case and the main actors involved.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 68
made, it is important to know the case and the main actors involved. Therefore, in this
phase, experts are urged to use the list of actors drafted in the previous phase and
the information deriving from the regional briefing with relevant stakeholders.
Interviews should start with the MA representatives, who know the strategy best. As
they might be selective in their choice of actors with “good stories” to tell, country
experts will ask each actor interviewed who are the other relevant actors involved in
the case who in his/her opinion should be interviewed. In doing so, experts are asked
to use their knowledge on the case, the list of actors involved resulting from the
previous phase and the indications deriving from the regional briefing with
stakeholders involved.
In conducting the face-to-face semi-structured interviews, country experts will follow
the case study template and indicative questions included in the interview guidelines.
However, country experts are urged to let the conversation follow smoothly based on
the answers given by the interviewee. Country experts have to pay attention to the
fact that all the issues specified in the template have to be covered during the
interview. Whether they feel that that the conversation diverges too far from the
issues relevant for the case study, they can return to the indicative questions included
in the case study template.
Country experts are asked to prepare a report of the interviews that have to be sent
to the core team together with the case study.
For each interview, country experts have to fill in the following fiche:
Name of interviewee
Position of interviewee
Name of the organisation
Type of organisation
Country and region
Address
Contact telephone
E-mail address
Date of interview
Time of interview
Interviewer
In conducting the face-to-face interviews, country experts have to:
give assurance of anonymity and confidentiality if this is requested by the
interviewees;
reassure interviewees that the interview data will be analyzed collectively, and
that names will be included in the research report only after consensus;
ask permission to record the interview (if using a recorder). However, it is
advisable not to use a recorder as this may inhibit responses, and this can
influence conversation negatively. It is better to directly type the answers,
using the structure of the case study template;
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 69
ask the respondent if they would like to see a copy of the interview note when
it is written up;
remind the interviewee they can stop the interview at any time. If at any point
during the interview, interviewee would prefer to provide particular views ‘off
the record”, they should let the interviewer know;
ask permission to use quotes when presenting results (quotes could also be
attributed by broad designation);
conduct the interview in an informal way to encourage open expression;
avoid expressing personal opinions about the case during the interviews.
F. Analysing the collected data and filling in the case study template
In this phase, it is important to organize the material collected during the previous
phases and write up the case study report.
Country experts will have to note that the case study is not a mere collection of
information from official or unofficial documents and fieldwork, but an elaboration and
a critical analysis of the information collected in the previous research phases.
In writing up the case study, country experts will have to provide evidence of their
analysis referring to any relevant material, such as official/unofficial documents,
evaluations or interviews. Experts will be asked to avoid using jargon, including
technical terms without a proper explanation or acronyms without the full name.
The completed document with both descriptive and analytical elements becomes a first
working draft.
Once completed, the draft case study will be sent to the core team that will review it.
The review is aimed at:
commenting on gaps, incomplete information or missing references;
using information and evidence gathered for the critical analysis identifying
logical flaws or unclear passages in the case study report;
checking the writing style and presentation.
Following the review, country experts are asked to integrate comments received from
the core team within the deadline agreed with the core team.
Outputs
The main deliverables are:
Template and guidelines guiding country experts in charge of the case studies
drafting;
One pilot case study;
One regional briefing for each case study;
A case study report including all the six regional case studies.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 70
Task 3b - Mini case studies
Aims
As foreseen by the tender, besides case studies (at the programme level), the
evaluation will include also two mini case studies (projects) for each programme case
study. Mini case studies will be selected from the 18 proposed at the end of Task 2
and will include both successful and less successful projects representative of the
different typologies of projects supported by ERDF programmes. We will search for
practices that are singular, for whatever reasons, and can offer an exemplary record
of success and less success.
Methodology
As for programme case studies, mini case studies will be based on a common template
in order to gather and present homogenous information.
The following box illustrates an indicative mini case study structure. The proposed
template will be further revised during the study, based especially on the findings of
the literature review (Task 1) and the survey on MAs (Task 2b).
Project title
1. Synthesis includes brief information on:
o the project objectives; o key project activities and their beneficiaries; o main results achieved – with specific reference to any innovative aspect - and
impacts
2. Background information Includes information on:
o the country and region;
o programme type; o duration of project; o funding; o ERDF objective.
3. Project description Includes an analysis of:
o the overall objectives/objectives and purpose; o beneficiaries; o description of activities; o project main outputs and results.
4. Political and strategic context This section will “set the scene for the story of the project”. It places the project in a wider
context providing a synthetic description of the regional economy and governance system and
of the culture and tourism sectors in the implementation area. It provides information on the
role and importance of the project in the context of regional (or national) strategies and
policies. Furthermore, it retraces the story of the intervention from the most recent event
(usually results) until the programming and agenda setting phases. In addition the purpose of
this phase is to provide information on the actors involved in the policy process.
5. Implementation This section will complement the main analysis addressing the following issues:
the process of project design and planning; management, monitoring and evaluation system of the project;
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 71
governance arrangements of the project; innovative elements and novel approaches to implementation; key implementation obstacles and problem-solving practices.
6. Key results
This section focuses on identifying, explaining and assessing the results of the described project.
Emphasis will be given to any results that may be considered innovative.
Findings will include any quotes which summarise key themes or findings in the evaluation and
which can be used to support the findings from the data.
This section will also include information on whether the intervention represents a cost-effective
way to foster economic growth of the implementation area.
7. Sustainability and transferability This section focuses on assessing the sustainability of the project in the absence of ERDF or
other public funding and providing information on other funding schemes that could sustain
results and impacts as well as on the other ways through which the results and effects could be
maintained.
For successful cases, this section will provide information on the capacity and contextual
conditions that are needed for target sites to ensure a similar successful implementation. In
particular, it will include a check list of the social mechanisms that have favoured the successful
implementation process and that could be activated in other contexts for ensuring a smooth
implementation of similar interventions.
8. Lessons learnt Drawing on previous information, this section will synthesize information on the main lessons
learnt from the project, particularly regarding the key factors behind either successful or less
successful projects.
9. Contact details
The following research methods will be applied for the analysis of the mini cases:
Desk research;
4-8 semi-structured and face to face interviews with the main actors involved
in the project;
focus-group with other stakeholders (if necessary to get a complete picture of
the case).
Outputs
Mini case study template;
Guidelines on how to draft mini case studies provided by the Core team to
regional experts;
Two pilot mini case studies;
12 mini case studies (6 successful projects and 6 less successful projects).
Task 3 c. Final comparative analysis and lessons
In this task the core team will analyse horizontally the case studies and mini case
studies reports in order to draw conclusions and lessons from the information
acquired.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 72
The cross-analysis will aim to identify similarities and differences among the strategic
approaches to culture and tourism sectors, and in particular:
Whether the regional strategies are based on and centered around a sound analysis
of future demand;
Which are the main reasons explaining the differences between actual (or likely to
be achieved) results and expected ones;
Whether the supported projects within the regional strategy analyzed are financially
sustainable;
Which selection criteria and indicators and how they contribute to assure the
financial sustainability of projects in the selection process;
Which are the more relevant (positive or negative) effects achieved, which
programme features and mechanisms fostered those results.
Moreover, the analysis will assess the programme features which favoured the
achievement of outstanding results (positive or negative), and the underlying
mechanisms which provided those results. Positive cases (including mini case studies
experiences) will provide knowledge on solutions and mechanisms which are likely to
lead to comparatively cost-effective results.
Outputs
The main deliverables is a Comparative analysis of 6 regional case studies and 12 mini
case studies
Quality and risk management for Task 3.
The table below includes an overview of the main risks related to this task.
Table 3.19. – Task 3- Main challenges and possible solutions
Risk Mitigation measures
Lack of data on programme results (high chance of occurring; high impact)
Data obtained from the monitoring reports and evaluations will be integrated with data from the regional briefing and semi-structured interviews. Moreover, the contractor will also consider the perceptions of relevant stakeholders on the expected results, in case the strategy is not yet completed. Triangulation of information and data sources and combination of qualitative views and evidence will mitigate the subjectivity of these perceptions.
Confidentiality of data (low chance of occurring; medium impact)
Country members engaged on the ground will clearly state that data collected will remain confidential
Availability of interviewees (chance of occurring medium; impact medium)
The proposal puts forward a reasonable budget dedicated to chasing interlocutors. Support from the EC will be requested in extreme cases
Less successful practices to be analysed within mini case studies (medium chance of occurring: high impact)
MA or other relevant stakeholders may not be willing to provide information on interventions judged as failures. Particular attention will be paid to the selection of mini case studies, selecting "promising" or "relevant" interventions that faced a specific problem (for example, the management of the partnership; a significant obstacle in the implementation process, etc). The aim is to adopt a learning perspective (and not a naming-and-shaming one) in the evaluation, in order to raise knowledge on the specific problem faced and its causes, the consequences for the process and the possible lessons learnt for the future and for other policy makers.
Consistency and horizontal analysis of
Strong coordination in the implementation of case studies is needed to make sure that the collated information from the case studies is aligned with the aims
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 73
the information presented in case studies (medium/high chance of occurring; high impact)
of the study, and to ease the horizontal analysis. To this end, the core team will develop the case studies template, undertake a pilot case to test and fine tune it, and will draft interview guidelines. Furthermore, a briefing with the experts involved in the case studies will be organized before they begin research. The implementation of the case studies will be strictly monitored by the project leader and quality manager, to ensure a close alignment with the study overall methodological approach and Steering Group’s expectations.
3.2.4 Task 4 - Cross-Task Analysis and Presentation of the Final Report
Aims
This task draws together the results of the previous Tasks 1-3 in order to:
analyse and synthesise results;
draw conclusions on the basis of these results;
put forward recommendations;
contribute to the validation and dissemination of the results after approval
of the final report.
Methodology
The results of the different analyses will be discussed according to the analytical
framework presented in the methodological approach and the triangulation of different
sources in order to identify:
The main strategies and types of intervention in culture and tourism adopted at
national and regional level;
The nature of co-financed investments in these sectors and their effectiveness
in contributing to the development of these sectors;
The contribution of culture and tourism strategies to regional development;
The process and context related factors that might condition the success of
regional strategies in the sectors of interest.
The results will address the evaluation questions indicated in the tender specifications
and pinpoint a set of causal mechanisms that could explain the success of the ERDF
strategies at different levels, as well as operational recommendations on how to
enhance the contribution of cohesion policy in the culture and tourism policy areas.
Outputs
The output of this phase will consist of a final synthesis report including the overall
findings of the analysis and the main implications for policies targeting the sectors of
culture and tourism.
The final report will include a thorough analysis of the information collected; a
revalidation of the preliminary findings from the previous tasks; a synthesis of the
overall findings and lessons learnt; conclusions and operation recommendations. The
report will be completed by an executive summary in English, French and German,
and by annexes presenting in detail the results of each task:
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 74
The literature review
The survey on MAs
The 6 case studies
The 12 mini case studies.
The final report will have an emphasis on clear communication to ensure that it is able
to be shared widely29.
A Validation Seminar with Member States to discuss the findings of the evaluation and
deepening the analysis for policy orientation will be organised in Brussels before the
draft final report is submitted to the Commission. The results of the discussion will be
incorporated into the final report.
The specific arrangements for the organization of the seminar will be discussed at a
later stage. Indicatively the seminar could be held on the 10th of September 2015. As
required by the tender specifications the evaluator will be responsible for the
organisation and content of the seminar, including identifying participants, preparing
the content of the seminars, leading discussions and writing up the event.
29 http://ec.europa.eu/translation/english/guidelines/documents/styleguide_english_dgt_en.pdf
http://www.economist.com/styleguide/introduction
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 75
4 Organisation and management
4.1 Roles and tasks
As specified in the proposal, the project will be implemented by a Consortium of four
organizations, led by IRS.
The lead organization (IRS) will guarantee an efficient project management process
that ensures:
Central management and coordination of the work streams and tasks;
Reliable service integrity and continuity;
Services delivered to quality professional standards (Quality assurance).
Research and communication activities, including the regional stakeholders’ workshops
and presentation of project results will involve all the partners of the Consortium.
The members of the consortium are used to managing large projects, at international
and national level, coordinating mixed teams and facilitating close multidisciplinary
collaborations by large numbers of experts from different countries and in multiple
languages, and maintaining the highest standards of quality and resource efficiency.
Moreover the consortium can activate a large network of European experts in the
evaluation of Cohesion Policy and culture and tourism policies to be involved in the
drafting of the programme/regional case studies and the related mini case studies.
The team includes (i) a Project Manager with more than 10 years experience in project
management, including overseeing project delivery, quality control of delivered
service, client orientation and conflict resolution experience and several years
experience in management of team of at least 5 people; (ii) more than two members
with native-level language skills in English; and iii) two high level external experts.
On the basis of discussion during and after the kick-off meeting with the DG REGIO
Evaluation Unit, the composition of the project team has been modified with the
substitution of the two external experts to be involved in the study.
In detail, the proposed research team includes:
a core team made of:
√ the project leader (Manuela Samek, who will also act as task manager of Task
4);
√ the project operative coordinator (Flavia Pesce)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 76
√ the quality manager (Paul Owens supported by Callum Lee30)
√ three tasks managers (Valeria Minghetti, Erica Melloni, Silvia Vignetti);
a team of scientific advisors made of:
√ two high level external experts (Dr. Beatriz Garcia and Professor Harvey
Armstrong);
√ an internal high level expert (Prof. Bruno Dente);
a team of thematic experts in cohesion policy, tourism, culture, local and
regional development, urban development, public governance and public
management and capacity building (Mara Manente, Paul Owens, Claudio Calvaresi,
Ivana Fellini, Luca Moreschini, Nicoletta Torchio, David Bradley);
a team of researchers and research assistants with national and international
experience in the evaluation of public policies, and in particular culture and tourism
policies, cohesion policy, territorial development policies, capacity building policies
(Jessica Catalano, Erica Mingotto, Federica Montaguti, Bethany Lewis, Monica
Patrizio, Paulo Rosario, Emanuela Sirtori);
a team of country experts in cohesion policy and/or culture and tourism to carry
out the case studies (Davide Barbieri, Anelia Damianova, Heitor José Rocha Gomes,
Nicholas Georgios Karachalis, Andris Klepers, Tomasz Komornicki, Callum Lee,
Sandra Naaf, Julie Pellegrin, Davide Sirtori, Vladimir Sodomka, Dmitrij Svec,
Cristina Vasilescu). If necessary the team of country experts may be modified
according to the case studies selected for the field work;
a financial manager (Nadia Naldi);
an administrative secretariat (Rosa Rainieri).
The figure below describes the links between the members of the research team with
arrows representing the main communication channels.
30 Given the great importance given to the quality of the outputs, editorial and linguistic check, we
integrated the proposed quality manger with an additional support. Callum is an experienced consultant with
over ten years experience in research and evaluation in the cultural and creative sectors (Category II) and
is an Associate Director at BOP Consulting. This supplements the time provided by Paul Owens and Callum
potential role as a Country Expert, on the delivery of the research component, and ensures we have
continuity with one individual managing all processes and therefore ensuring a higher quality output.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 77
Figure 4.1. Project Management
The Core team will be responsible for:
planning the tasks required by the tender;
choosing the best team for execution of the tasks;
fine tuning the theoretical approach and methodologies, jointly with the
internal scientific advisor;
defining and approving methods of implementation, analysis of findings and
dissemination of results;
supervising the work done by the team assigned to the task;
assisting thematic experts, country experts and researchers in the
implementation of the tasks;
coordinating the drafting of all reports foreseen by the tender (inception
report, first interim report, report on the pilot case study and two mini case
studies, report on the 6 case studies and 12 mini case studies, draft final
report, final report) and presentations requested by the Commission;
ensuring that the all changes/requests of integration pointed out during the
Steering Meetings and/or in exchanges with the Commission office and the
external experts are fully considered and integrated in the reports to be
submitted;
project management and risk management;
project monitoring;
quality management;
administrative and general coordination.
In particular, the project leader is responsible for representing the consortium with the
European Commission; scientific and project coordination, implementation and results;
the management of the work team and delivery of qualitative and timing project
deliverables. Furthermore, the project leader will also be in charge of: work planning
in order to ensure that the project stays within its original timescale; facilitating
regular interaction between the project team members; monitoring of the work;
managing the risk(s) associated with project execution; communicating and liaising
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 78
with the client on a regular basis.
In carrying out these tasks the project leader will be supported by:
the project operative coordinator for internal and external operative project
coordination; day-to-day activities and communication exchanges between the
partners of the consortium; production of the Monthly Progress Reports;
the tasks managers in charge of the implementation of all four project tasks;
the quality manager responsible for quality checks of all project deliverables.
The scientific advisors are in charge of ensuring high quality standards in the
development of the evaluation design and its implementation.
The internal expert will be involved in the fine tuning of the methodologies for each
tasks and in providing feedback during the drafting process.
The external experts will comment on each deliverable, based on their area(s) of
expertise. The experts’ comments have to include:
an assessment of the quality of the deliverable;
brief, specific, constructive suggestions for change or improvement based on their
expertise.
External expert will not however be responsible for the final deliverable, which remains
in all cases the responsibility of the contractor.
The experts comments will have to be provided both in written and oral form.
Written comments will be provided on the draft deliverables and sent to DG Regio at
least 2 days before the Steering Committee. Oral comments will be provided at three
expert meetings to be held around the same time as the Steering Group in Brussels.
The following table shows the contribution of the external experts to the deliverables.
If these contributions need to be changed later in the contract this will be done by
prior agreement with DG Regio.
Table 4.1. Contribustion of experts to the deliverables
Expert 1: Beatriz Garcia Expert 2: Harvey Armstrong
Deliverable 1: Methodological
Inception Report
Written comments Written comments
Deliverable 2: First Interim
report presenting the results
of Task 1 and Task 2
Written comments and
Meeting in Brussels
Written comments and
Meeting in Brussels
Deliverable 3: Presentation of
a Pilot Case Study, including
two mini case studies
Written comments and
Meeting in Brussels
Written comments and
Meeting in Brussels
Deliverable 4: Report on the
six case studies, including all
mini case studies
Written comments Written comments
Deliverable 5: Draft Final
Report
Written comments and
Meeting in Brussels
Written comments and
Meeting in Brussels
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 79
The country experts’ team. Depending on the choice of the regional case studies to
be analysed, a number of country experts will be selected from the network presented
in the technical offer. Their contribution will be crucial in:
the identification of national and regional statistical sources for collection of
data;
the identification of key stakeholders at national and regional level to
interview;
carrying out the regional case studies with the support of the Core team
(Task 3) and drafting the mini case studies;
providing insights to the core team during the preparation of the first,
second, third interim reports, draft final report and final report.
The thematic experts team cover the different areas of policy required by the tender
and will be involved in the different project tasks according to their competences. They
are individually and as a group responsible for:
data collection and qualitative and quantitative data analysis;
drafting the project reports;
ensuring the highest scientific level of their contributions.
The team of researchers provides research support to the core team and to the
thematic experts.
The administrative secretariat is in charge of operative support in the general
coordination tasks, in day-to-day logistics and editorial support.
The financial manager is in charge of all the financial aspects of the contract. She
will work with the partners on any financial matters.
Table 4.2 presents in details the role of each member of the consortium and their
task.
Table 4.2. Research team expertise, roles, tasks
Role in the Projec Name Category Task Language
Core team Project Leader
Manuela Samek
Lodovici
I Project management,
Manager of Task 4
Contribution in all
tasks
IT (MT),
EN
Core team Operative
Coordinator
Flavia Pesce I Project management
Contribution in all
tasks
IT (MT),
EN, DE
Scientific advisor
team
High level internal
expert (Scientific
Advisor)
Bruno Dente I Scientific advice and
contribution in all tasks
IT (MT)
EN, FR, SP
Scientific advisor
team
High level external
expert for Culture
(Scientific Advisor)
Dr. Beatriz
Garcia
I Scientific advice and
contribution in all tasks
EN (Es,Fr)
Scientific advisor
team
High level external
expert for Tourism
(Scientific Advisor)
Prof. Harvey
Armstrong
I Scientific advice and
contribution in all tasks
EN
Core team Task Manager
Senior Thematic
Expert (Tourism
Economics)
Valeria Minghetti I
Manager of task 1 and
contribution in Task 4
IT (MT)
EN, FR
Core team Task Manager Silvia Vignetti II Manager of Task 2 and IT (MT),
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 80
Senior Thematic
Expert (Evaluation
and Structural Fund)
contribution in Task 4 EN, FR
Core team Task manager
Senior Thematic
Expert (Evalutation,
Local Development
and Structural
Funds)
Erica Melloni II
Manager of Task 3 and
contribution in Task 4
IT (MT)
EN, FR
Core team Quality Manager
Senior Thematic
Expert (Evaluation
and Cultural and
Creative sectors)
Paul Owens I
Manager for the quality
check and contribution
in Tasks 1 and 4
EN (MT)
FR, Hausa
Thematic Experts
team
Senior Thematic
Expert (Evaluation
and Tourism sector)
Mara Manente I
Tasks 1 and 4
IT (MT)
EN, FR
Thematic Experts
team
Senior Thematic
Expert (Evaluation,
Regional and Urban
Development,
Spatial planning)
Claudio
Calvaresi
I
Tasks 1, 2c, 4
IT (MT)
EN, SP
Thematic Experts
team
Senior Thematic
Expert (Tourism
Economics)
Ivana Fellini II
Tasks 2a, 2c, 4
IT (MT)
EN, FR
Thematic Experts
team
Senior Thematic
Expert (Evaluation,
Culture, Structural
Funds)
Luca Moreschini II
Task 2a, 2c, 4
IT (MT)
EN, FR, ES
Thematic Experts
team
Senior Thematic
Expert (Evaluation
and Structural
Funds)
Nicoletta Torchio II
Task 2a, 2c, 4
IT (MT)
EN, FR, ES
Thematic Experts
team
Senior Thematic
Expert (Evaluation
of Structural Funds,
Tourism sector)
Country Expert (IE,
UK)
David Bradley I
Tasks 1, 3 and 4
EN (MT)
FR
Researchers’ team Researcher Jessica Catalano IV Tasks 2b, 2c IT, EN, FR
Researchers’ team Researcher Erica Mingotto IV Task 1
IT (MT)
EN
Researchers’ team Researcher Federica
Montaguti
II
Task 1
IT (MT
EN, FR, ES,
Japanese
Researchers’ team Proof Reader
Researcher
Bethany Lewis III Task 1 and Proof
reading in all tasks
EN, DE (Bi-
lingual)
FR, NL
Researchers’ team Researcher Monica Patrizio III Tasks 2a, 2c and 3
IT (MT)
EN, FR
Researchers’ team Researcher Paulo Rosario III Task 1
PT (MT)
EN, ES
Researchers’ team Researcher Emanuela Sirtori IV Tasks 2b, 2c
IT (MT)
EN, ES
Country Experts’
team
County Expert (IT) Davide Barbieri II Task 3
IT (MT)
EN, ES, FR
Country Experts’
team
County Expert (BG) Anelia
Damianova
I Task 3
BG (MT)
EB, FR, Russian
Country Experts’
team
Country Expert (PT) Heitor José
Rocha Gomes
II Task 3 PT (MT)
EN, FR, ES
Country Experts’
team
Country Expert (GR,
CY)
Nicholas
Georgios
Karachalis
II Task 3 GR (MT)
NL, EN, FR
Country Experts’
team
Country Expert (LV,
LT, EE)
Andris Klepers III Task 3 LV (MT)
EN, DE, Russian
Country Experts’
team
Country Expert (PL) Tomasz
Komornicki
II Task 3 PL (MT)
EN, FR, RU
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 81
4.2 Allocation of activities
The allocation of human resources by Tasks is presented in table 4.3.
Table 4.3. Working days allocated per task and researcher
Name
Project
Manageme
nt
Task
1
Task
2a
Task
2b
Task
2c
Task
3
Task
4
Tota
l
Manuela Samek Lodovici 27 5 5 5 5 5 10 62
Flavia Pesce 20 2 2 2 2 2 10 40
Bruno Dente 2 2 2 2 8
Beatriz Garcia 10 10
Harvey Armstrong 10 10
Valeria Minghetti 5 10 8 10 12 45
Silvia Vignetti 5 10 15 10 15 55
Erica Melloni 15 10 25
Paul Owens 6 8 8 22
Mara Manente 5 3 5 5 18
Claudio Calvaresi 7 7 14
Ivana Fellini 7 7 14
Luca Moreschini 5 5 5 15
Nicoletta Torchio 5 5 5 15
David Bradley 5 10 15
Jessica Catalano 8 30 6 10 54
Erica Mingotto 10 10
Federica Montaguti 10 10
Monica Patrizio 8 15 10 15 48
Paulo Rosario 10 10 20
Emanuela Sirtori 8 30 10 48
Callum Lee 4 5 5 14
6 Experts for selected case
studies 180 180
Rosa Rainieri 10 10
Nadia Naldi 10 10
TOTAL days 112 72 62 87 79 224 136 772
% 14.5 9.3 8.0 11.3 10.2 29.0 17.6 100
Country Experts’
team + support to
Quality Manager
Country Expert (NL) Callum Lee II Task 3 EN (MT)
FR, DE
Country Experts’
team
Country Expert (DE,
AT)
Sandra Naaf III Task 3 DE (MT)
EN, ES, FR, PT,
IT
Country Experts’
team
Country Expert (FR,
BE)
Julie Pellegrin I Task 3
FR (PT)
IT, EN
Country Experts’
team
Country Expert (ES) Davide Sirtori III Task 3 IT (MT)
EN, ES
Country Experts’
team
Country Expert (SK) Vladimir
Sodomka
II Task 3 CZ (MT)
EN, FR, Russian
Country Experts’
team
Country Expert (CZ) Dmitrij Svec I Task 3 CZ (MT)
EN, Russian
Country Experts’
team
Country Expert (RO) Cristina
Vasilescu
III Task 3 RO (MT)
EN, FR, IT
Financial Manager Nadia Naldi I IT (MT)
EN, FR
Secretariat and
Editing
Rosa Rainieri I IT (MT)
EN, DE
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 82
Figure 4.2 shows a balanced allocation of resources for the tasks to be carried out.
More resources, in terms of working days, are allocated to Tasks 2 and 3 due to the
important time consuming activity of the fieldwork to be carried out in the Survey
(Task 2b) and the programme/regional case studies (Task 3). Over 40% of the total
working days are spent on fieldwork (the survey and on the case studies).
Figure 4.2. Allocation of human resources by tasks (%)
4.3 The quality control system
As indicated in the proposal, the consortium leader (IRS) will manage the overall
quality control system, assisted by the Consortium partners.
IRS and the consortium partners have a strong commitment to quality. Significant
past experience in managing complex projects with wide EU coverage and large teams
enabled the team experts to set up and further refine a well proven internal quality
control system for this project.
To ensure that the services provided under this contract meet the highest professional
standards, a quality assurance system will be used, specifying the Consortium
approach to quality assurance and how it intends to control and ensure the effective
monitoring of the services supplied to the Commission as part of the execution of this
contract.
The quality plan covers the following areas:
The procedures the Consortium intends to implement to ensure the quality of
the services provided;
Personnel policy and management, including the mechanisms for notification to
the Commission, and timely and full replacement of any reduction in capacity,
in order to ensure the committed level of expertise and resources throughout
the whole duration of the contract;
Project Management;
14,5
Task 1; 9,3
Task 2a; 8
Task 2b; 11,3
Task 2c; 10,2
Task 3; 29
Task 4; 17,6
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 83
The structure set up for coordinating the work between the different members
of the Consortium, including working criteria for the distribution of assignments
between the members of the consortium;
The procedures we intend to use for quality control, assurance of client
satisfaction and complaint management.
The quality system aims to ensure the following overall standards:
Consistency with the research questions set out in the Terms of Reference
and with requests and considerations from meetings with the Commission,
the steering Group and the external experts.
High scientific relevance and accuracy. A peer review mechanism aims
to ensure a sound methodology and access to the latest research and
evidence. A standard practice on IRS’s assignments is the mobilisation of
scientific advisors of renowned reputation in their fields of specialisation to
guarantee the quality of IRS’s intellectual input. They are involved either as
external critical reviewers and as members of the team ensuring scientific
quality in the overall methodological development;
Reliability of information provided. All material from the work of others
which is used for specific assignments, such as data, information, ideas,
concepts, methodologies, quotes and literature will be clearly identified
and cross checked for reliability and referred to in a systematic reference
system. The most recently available information and data will be taken into
account;
Clarity and quality of language. From past working experience for
European institutions, the Consortium partners know that the study report
should inform discussion and policy debate. Thus a special effort will be paid
to the way the key messages are communicated and the adopted style and
language will be reader friendly and policy oriented, straightforward and
easy to be understood to all the relevant stakeholders. As agreed in the
Inception Meeting, the Economist Style Guide will be taken as a model for
this where possible, taking into account the specific language and
terminology required for a evaluation of this type. The Consortium experts
have extensively experience in producing deliverables for policy makers
(e.g. officials in national and European public institutions) and the general
public (e.g. public dissemination of findings through conferences, mass
media). This ensures that the deliverables will be easily accessible. Good
linguistic coverage of the research teams enables to avoid
misinterpretations and misunderstandings in a multilingual environment.
Finally, the presence of a number of native English speakers as well as of a
proof-reader will assure that all the deliverables (especially the final ones)
will meet high standard of written English.
Besides these key principles which guide this evaluation, a tailor-made quality
management system will be implemented as specified below.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 84
4.3.1 The team’s quality management
Roles and responsibilities
Aware of the strategic importance of quality control activities, especially for the
management of large international teams, the allocation of resources includes a
specific allocation for activities aimed at ensuring a smooth implementation of the
study tasks, and specific quality check activities for the production of individual
deliverables.
The quality management of the project will be ensured by the Project Manager
(Manuela Samek Lodovici) and the Operative Coordinator (Flavia Pesce) who will take
overall responsibility for the quality of the reports produced and submitted, for
covering all issues as identified by the Terms of Reference, for considering all issues
raised during Steering meetings and other exchanges with the Commision services
and external experts, as well as for the quality of work undertaken and overall
implementation of the assignment. In this role they will rely on the support of the
feedback by the Internal Scientific Advisor (Bruno Dente) who will guarantee the
general quality of the content of the reports as well as the Quality Manager (Paul
Owens supported by Callum Lee) and a Proof-reader (Bethany Lewis), who will ensure
the editorial and linguistic quality.
The role of the Project Manager as regards the internal quality system includes the
following tasks:
Monitoring the progress of each activity;
Supervising and ensuring the smooth and effective implementation of the
overall activities performed in the framework of the quality control process;
Ensuring the clarity and reliability of any documents submitted to the Steering
Group;
Ensuring communication with the Commission officials.
In addition, the appropriate attribution of responsibilities to the various experts, a
constant monitoring of the implementation of the project and the capacity to manage
a contingency plan will ensure the successful and efficient execution of the
assignments. In order to guarantee a smooth implementation of the quality
management plan, the following methods will be used:
the study will accord with the Tender Specifications, addressing the right focus and
scope, including the further fine-tuning needed to comply with the Steering
Group’s specific requests during the implementation of the study. The participation
of the Project Manager and the Operative Coordinator to all meetings and
interactions with the External and Internal Scientific Advisors as well as the Task
Managers and the internal thematic experts will ensure that the project
implementation will address the SG requirements and agreed refinements;
the Project Manager and the Operative Coordinator will ensure the alignment of
the whole team to such requirements. They will keep Task Managers and the
research team constantly informed about the development of the study and will
ensure an open communication and exchange of information with all members. In
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 85
order to facilitate the sharing of the information and documents a shared
management environment will be set up and populated under the responsibility of
Task managers with the help of the researchers in order to ensure a real-time
sharing of documents and reports.
The study process will be carried out in a transparent way. Besides the scheduled
meetings, an open communication with the Steering Group will be pursued by the
Project Manager, who will be the main contact point vis-à-vis the client, supported
by the Operative Coordinator. If any force majeure problem occurs, the Project
Manager will inform the Commission immediately by providing a possible solution.
The allocation of the workload among the team’s members is made in order to
maximise the comparative advantage of each expert and will be adjusted if
necessary to ensure his/her full availability when specific input is required.
The whole team is committed to providing the necessary high quality service to the
Client. In the unlikely case of the absence of one of the experts, the continuity of
the service is assured by the Project Manager who will reshuffle the internal time
allocation in order to allow other team members to take over responsibilities. This
will be possible since each role within the team is ensured by more than one
expert, so that in case some of them are not available in the foreseen timeframe
his/her role may be covered by other experts in the same category. The
Consortium partners also belong to extended networks of experts in the evaluation
of Structural Funds and in the sectors of interest. This ensures the capacity to
rapidly activate additional experts in case of need. Furthermore the wide range of
experts that can be activated ensures that each expert will be in charge of one
specific case study, given the depth of analysis required.
4.3.2 Risk management and contingency plan
The assignment poses some specific challenges which have been carefully considered
in setting up the quality management system and contingency plan. In our view, the
most important are:
the large size of the consortium and the broad spectrum of tasks running in
parallel require the strong coordination of the team and the capacity to manage
heavy workloads. For this reason, special attention will be dedicated to project
monitoring and coordination, as well as a continuous alignment of the team’s
members to the state of implementation and methodological development. A
strong coordination role is ensured by the core team and flexibility is
guaranteed by the availability of a wide range of expertise within the team,
with the awareness that the schedule and time allocation may well be subject
to constant revisions and updates in order to promptly mobilise the best
available resources.
the need to ensure consistency among case studies. It is acknowledge that
strong coordination in the implementation of case studies is needed to make
sure that the set of information from different case studies are fully aligned
with the study aim, so as to ease the subsequent comparative analysis. To this
end, the Task Manager, with the help of the core team, will develop the case
studies template, undertake a pilot case to test and fine tune it, and will if
required support the Member State experts in the drafting of the case studies
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 86
to ensure the consistency of information in them. Continuous exchanges with
the country experts will be ensured throughout the implementation of the case
studies. The implementation of the case studies will be strictly monitored by
the core team, to ensure a close alignment with the study overall
methodological approach and Steering Group’s expectations as developed
during the first phase of the project implementation.
The statistical analysis to be implemented in task 2 and the on line survey on
MAs is subject to the inherent uncertainty related to data availability, response
rate as well as quality of responses. The Team’s experience from previous
studies suggests that any attempt to predict a credible survey response rate is
challenging, especially considering conditions such as the motivation of of those
invited to participate, and how accurate or current the contacts available are.
In order to minimise these risks, the Consortium will put significant effort into
the careful design of the questionnaire as well as in a follow up approach, after
the initial survey invitation. In our experience, the design of the questionnaire
is also crucial in gaining the required information and ensuring a good response
rate.
In addition to these challenges, the Consortium have prepared plans to mitigate or
deal with some specific risks that may arise from the project: those related to the
management and organisation of the work and those related to the methodological
approach adopted. The main risks are detailed in chapter 3.2. These are summarized
below, from an operational perspective.
Table 4.4. Risk management related to team management
Risk Impact Chance
of occurring
Mitigation measures Responsible
Lack of experts Medium Low The partners have a wide network of contacts throughout Member States and can rapidly find individual experts to perform the required task
Lead partner (IRS)
Individual country experts are too busy
Low Medium
Advanced planning will allow early warning for individual country experts so to allow them to commit to deadlines. In any case, partners and subcontractors have a wide network of contacts throughout Member States and can rapidly find individual experts to perform the required task
Lead partner (IRS)
Senior staff leaves a consortium organisation
Medium / high if it concerns partners
Medium
Partners have sufficient in-house capacity to substitute the experts with equal level of qualification and experience
Partner concerned
Senior experts are too busy, unavailable to travel
Medium Medium
Telephone and writing exchanges may help to address this. If not, the proposal includes a large number of senior experts to deal with the issue.
Lead partner (IRS)
Experts or Partners deliver work of poor quality
Low High
Partners and experts have been chosen for their excellent records. In case the issue arises, the management of outputs delivery will allow identifying it in time to take corrective measures, such as redrafting a deliverable, or finding substitutes if no progress is made.
Lead partner (IRS)
Delays Medium Medium The management of the delivery of outputs–internal or concerning deliverables to the EC – will be designed to allow the project team to identify issues in time to address them. Monitoring the project development will be one of the main responsibilities of the project manager
Lead partner (IRS)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 87
Table 4.5. Risk management related to the methodological approach
Risk Impact
Chance
of
occurring
Mitigation measures Responsible
Lack of data
availability (Tasks
2 and 3)
High High
This risk depends on the data
concerned and is likely to be higher
for the culture sector. Monitoring
data will be checked against the
results of WP0, the MAs survey and
the interviews carried out in the
case studies. The triangulation of
sources will help in covering the
main data gaps.
Partners and experts
concerned
Confidentiality of
data (Tasks 2,3) Medium Low
Consortium members engaged on
the ground will clearly state that
data collected will remain
confidential
Partners experts and
subcontractors
concerned
Low availability of
interviewees
(Tasks 1,2, 3)
Medium Medium
The proposal puts forward a
reasonable budget dedicated to
conducting interviews and their
arrangement. Support from the EC
will be requested in extreme cases
Partners concerned,
EC.
Low survey
response rate
(Task 2b)
Medium /
High High
See above
Partners concerned
Validity of the
typologies used
(task 2, 3)
Medium High
The different typologies (of policy
instruments, of sectors, of
operators) will be used
pragmatically, allowing for
progressive refinement and
adjustment
Lead partner (IRS)
Consistency and
comparability of
outputs (tasks
2,3,4)
High Medium
See above
Lead partner (IRS)
Poor readability of
outputs High Medium
Native team members will edit all
deliverables to the EC with
emphasis not only on correct
English but also on stylistic issues
to ensure they use a clear and
concise style. Final language
checks are foreseen as part as the
quality control process.
Lead partner (IRS)
4.3.3 Specific quality check
The quality management plan will be implemented through a joint collaboration of the
Internal Scientific Advisor (Bruno Dente), the Project Manager (Manuela Samek
Lodovici), the Quality Manager (Paul Owens supported by Callum Lee) and the proof-
reader (Bethany Lewis). The process will include four main activities:
1. Scientific review of each report before it is submitted to the Steering Group. This
is performed by the Internal Scientific Advisor who is convened to discuss the
proposed results. His main responsibility will be to ensure that the outputs
delivered meet the highest academic standards especially in terms of content and
methods. In addition the two External Scientific Advisors will provide the
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 88
Team’s experts with comments and suggestions on how to improve the
deliverables in order to ensure the highest scientific quality of the research.
2. An editorial check aimed at ensuring the clarity and consistency of the
information provided as well as the ease of reading of the final output. This check
will be performed by the Quality Manager whose main role is to carefully read
documents drafted by the Team’s experts and verify that the information are
consistent and presented according to the European Commission’s requirements in
terms of layout, style, format, length and structure. The Quality Manager will
suggest improvements in the style and format of the information presented to the
Team’s experts. In addition, all the reports will have the same layout, style and
format and will follow a common structure in order to ease the readability and
intelligibility of the results.
3. A linguistic check aimed at ensuring that the clarity and highest quality of the
language. Each deliverable will be submitted to the proof-reader who is an
English native speaker. This ensures that requirements of clarity and the highest
quality of language are met. When proofreading activities have to be carried out
for different deliverables in parallel (for example for the case studies report) the
British partner (BOP) will support this with a linguistic check. In addition, the
British partner will be in charge of directly drafting some of the planned
deliverables or sections of them (e.g. literature review in Task 1, case studies for
UK and/or IE or MT in Task 3, final reporting in Task 4) which significantly reduces
the number and length of deliverables expected to undergo the professional proof-
reading.
The Project Manager will lead the quality process monitoring with the support of the
Operative Coordinator. This involves: i) monitoring the progress of each
aforementioned activity of the quality process, by holding regular progress discussions
with the Internal Scientific Advisor and the Task managers; ii) supervising and
assuring the smooth and effective implementation of the overall activities performed in
the framework of the quality control process; iii) managing a contingency plan to
overcome problems and ensure a successful and efficient functioning and execution of
the assignments.
This process is visualized in the Figure below.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 89
Figure 4.3. Illustration of the quality control system
Team’s experts producing the reports
Review by the Scientific Advisor
(checking the scientific and academic quality)
OK
Editorial reviewer (checking the style, layout,
clarity of presentation)
Proof - reader (checking the linguistic
quality)
OK
Project Manager (checking the overall quality
and consistency of the deliverables)
OK
Project Manager
Steering Group
NO
NO
NO
YES
YES
YES
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 91
5 Time plan, deliverables and upcoming activities
5.1 Time plan and upcoming activities
Deadlines relating to the study have been discussed and agreed with the DG REGIO
Evaluation Unit at the kick-off meeting.
All the future activities will proceed as agreed, taking into consideration any
methodological fine tuning proposed in chapter 3.
The following table presents the updated timeframe of activities and deliverables.
Table 5.1. Time plan and deliverables
Deliverable Deadline Steering Committee and other
meetings
Kick off meeting 21 August 2014
Inception Report Draft 24 September 2014
(by noon)
Final 21 November 2014
1st Steering Committee: 1 October
2014, 2.30 p.m
Progress Report By the 21st of each month
First intermediate report:
results of T1 and T2
15 January, 2014 2nd Steering Committee and
1st Meeting with external experts:
indicatively in the third/fourth week of
January 2015
Presentation of a pilot case-
study and 2 mini case-studies
21 February 2015 2nd Meeting with external experts:
indicatively by end of February/early
March 2015
Report on the six case-studies
and 12 mini case-studies
21 May 2015
Draft final report 31 July (No later than 21
August) 2015
3rd Meeting with external experts –
indicatively by early Sept. 2015
3rd Steering Committee: 10 September
2015, 10.30 a.m.
Validation Seminar with regional and
national representatives: 10
September 2015, 2.30 p.m.
Final report 21 October 2015
Tasks 1 and 2 have already started during the inception phase, pending any fine
tuning of the methodology outlined in this Inception Report and according to the
discussion at the 1st Steering Committee on October 1, 2014.
As shown in the GANTT chart (table 5.2) below the activities to be carried out in Task
1 and Task 2a will proceed in parallel and will end indicatively by December 2014.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 92
Table 5.2 Gantt
Phase
Months
21/8/14
21/9/14
21/9/14
22/10/14
22/10/14
22/11/14
22/11/14
22/12/14
22/12/14
22/01/15
22/01/15
22/02/15
22/02/15
22/03/15
22/03/15
22/04/15
22/04/15
22/05/15
22/05/15
22/06/15
22/06/15
22/07/15
22/07/15
22/08/15
22/08/15
22/09/15
22/09/15
22/10/15
Phase 1
Kick off meeting (21
August 2014)
Scoping
Inception Report 24/9
1st Steering group 1/10
Phase 2
Literature review
Data analysis and
sampling
Web survey on MAs
First Intermediate Report 15/1
2nd Steering group + 1st meeting with external
experts
To be
defined
Phase 3
Fieldwork: pilot
regional/programme case
study and two mini case
studies
Report including the pilot
case study and two mini
case studies
21/2
2nd meeting with
external experts
To be
defined
Phase 4
Fieldwork:
regional/programme case
studies and mini case
studies
Report including the 6
regional case studies and
12 mini case studies
21/5
Phase 5
Synthesizing results of all
the previous tasks
Draft final report 31/7
3rd Steering group 10/9
Phase 6
Drafting the final report
Final report 21/10
Phase 7
Dissemination (to be
defined with the
Commission)
Project management
Project monitoring
Progress reports 21/9 21/10 21/11 21/12 21/1 21/2 21/3 21/4 21/5 21/6 21/7 21/8 21/9
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 93
As for Task 2b, all the preliminary steps (collection of contacts, fine tuning of the
questionnaire, setting up of the on line questionnaire and testing) have been
undertaken in November, in order to be ready to launch the on line survey by the end
of November 2014. We expect the survey to be open for about two months in order to
achieve a balanced distribution of respondents.
In January 2014 the research team will work on the tabulation of findings and the
selection and description of the 9 programme case studies and 18 mini case studies
(Task 2c).
A draft first Intermediate Report will be produced by the 15th of January 2015. The
report will incorporate the outcomes of Task 1 and Task 2a, as well as the initial
findings of the Web survey (Task 2b) and of Task 2c. The report will be discussed in a
1st Steering Committee to be held between the third and fourth week of January 2015.
A separate meeting with the two external experts will take place in the same date.
Between December 2014 and mid-February 2015 the evaluation team will also
implement the Pilot case study and the corresponding 2 mini case studies,
according to the methodology proposed in the Inception report and refined in the
Intermediate Report on the basis of the results of Task 1 and 2. The Report on the
Pilot case study and the corresponding 2 mini case studies will be produced by
February 21, 2015 and discussed in a 2nd meeting with the external experts to be held
by the end of February.
The methodology for the case studies and mini case studies will then be adjusted
according to the results of the pilot case study and of the discussion with the external
experts. On the basis of this adjusted methodology and of the tools provided by the
Core Team, the country experts will carry out the remaining 5 programme case
studies and 10 corresponding mini-case studies during March and April 2015. A
second intermediate report presenting the result of all the case studies and mini
case studies will then be produced by 21st May 2015.
During June and July 2015 the Core team, with the support of the internal high level
advisor, the thematic experts and of the research staff, will work on the draft final
report to be delivered by the end of July and certainly no later than August 21 2015.
The draft final report will be presented to the 3rd Steering Committee on 10th
September 2015 and to the external experts early in September 2015 to allow
discussion and input before it is finalised. Following discussion with the Steering
Committee there will be a presentation in a validation seminar with regional and
national representatives. The arrangements of the validation seminar will be discussed
at a later stage. A possible date for the seminar is the 10th of September 2015 in the
afternoon, following the 3rd Steering meeting.
The final version of the report will be delivered after one month (by the 21st of October
2015) and will internalize the results of the discussion with the Steering committee,
the external experts and the validation seminar.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 94
5.2 Deliverables
The First Intermediate Report will present the results of Tasks 1 and 2, namely:
The findings of the literature review, with a focus on: recent trends in these
sectors in the EU; the socio-economic rationales that justify public financial
support in the culture and tourism sectors and the effects of public financial
support in these fields; the main types of investment strategies and
interventions pursued in these areas.
The findings of the analysis of the ERDF support in culture and tourism, and in
particular those resulting from the statistical analysis of the main financial and
core indicators reported by MS.
The initial findings from the survey to the MAs of programmes investing more
than 15 million euro of ERDF funds in these areas and of the analysis on the
relevance of the culture and tourism sectors and of the ERDF financial support
by programme, region and type of intervention strategy and the proposed 9
programme case studies and 18 mini case studies with a brief description (1-2
pages) of each one.
The refined overall methodological approach for the programme case studies
and mini case studies as well as the template and guide to the research tools to
be used in fieldwork.
An indicative table of contents for the first intermediate report is below.
Indicative table of contents for the first intermediate report
Foreword and synthesis of the main results
Introduction: aims of the study and methodology
Part I - Task 1 - Literature review and analytical approach
1. culture and tourism in European regions: key figures, development trends
and evolution aspects
2. The socio-economic rationale behind public support to culture and tourism
3. Main strategic approaches and investment initiatives in culture and
tourism in the EU28: A comparative analysis
4. Assessment of the expected effects of ERDFfinancial support to culture
and tourism
Part II – Data analysis and sampling
1. ERDF contribution to culture and tourism: distribution of investments and
reported outputs
2. Main features of the adopted strategies and ongoing results of ERDF
Programmes allocating more than EUR 15 million to culture and tourism
(initial results from the Survey on MAs and other operators)
3. Classification of regions/programmes with similar features and
preliminary identification of 9 programme case studies and 18 mini case
studies for field work. Brief description of each proposed programme.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 95
Part III- Upcoming activities
Refining of the case studies and mini case studies methodology
ANNEXES
1. Annex to Part I: List of references
2. Annex to Part II: Methodology and source of data for the statistical
analysis and Survey Questionnaire; Additional data and statistics
(tables and figures)
3. Annex to Part III: Template for the case studies and mini case studies;
Consolidated check list for the case studies and mini case studies;
Interview guide for the semi-structured interviews
The second deliverable will present a pilot case study and two corresponding mini
case studies. Both the pilot case study and mini case studies are aimed at testing the
methodology and hypothesis drawing on the previous tasks. The report will also
include the finalized case study and mini case study methodology and the related
templates and guidelines for the drafting of case studies/mini case studies.
The third deliverable will be a report presenting all 6 regional/programme case
studies and 12 mini case studies, to provide answers to the evaluation questions
identified in the Terms of Reference. The report will present a transversal analysis of
all regional/programme case studies and mini case studies and a table of contents of
the draft final report. The 6 case studies and 12 mini case studies reports will be
annexed to the report. (An indicative structure of the case studies and mini case
studies has been presented in a section on Task 3 in chapter 3.)
The Final Report will cover the findings resulting from all activities under tasks 1 – 3.
The Report will provide: in depth analysis of the information collected in a specific
chapter for each task; a revalidation of the findings from the programme case studies,
mini case studies and seminars with the members states; a synthesis of the overall
findings and lessons learnt; the policy implications in the two areas of culture and
tourism, which will also take into account the further policy orientations resulting from
the discussion in the Validation seminar).
The report will be complemented by an Executive Summary in English, French and
German and a short document (1-2 pages) of Key Findings. The case studies and mini
case studies will be provided in an annex to the final report. Additional annexes will
present the data sources, the methodology and tools adopted for the statistical
analysis, the survey, and the case studies.
Monthly Progress Reports will be sent to the Commission on the activities carried
out and on the resources used (human and financial) as well as future planned tasks.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 97
Annexes
Annex I Indicative Bibliography
International statistical data and related studies
UNWTO
UNWTO, Statistical Yearbook 2008-2012, 2014 Edition
UNWTO, Compendium of tourism statistics 2008-2012, 2014 Edition
UNWTO, Tourism Highlights, 2014 Edition
UNWTO World Tourism Barometer and Statistical Annex, June 2014
UNWTO, Tourism towards 2030: Global Overview, 2011 Edition
EUROSTAT
EUROSTAT, Statistical Database. Tourism
(http://epp.eurostat.ec.europa.eu/portal/page/portal/tourism/data/database:
monthly and annual data on tourism industries; annual data on trips of EU
residents)
EUROSTAT, Eurostat Yearbook. Tourism (Online publication:
http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tourism_trends)
EUROSTAT, Eurostat Regional Yearbook. Tourism (Online publication:
http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tourism_statistics
_at_regional_level)
EUROSTAT, Tourism Industries – Economic analysis, Statistics in Focus 32/2013
(http://epp.eurostat.ec.europa.eu/statistics_explained/extensions/EurostatPDFGene
rator/getfile.php?file=157.138.95.30_1409649975_68.pdf)
EUROSTAT, Tourism trips of Europeans (Online publication:
http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tourism_trips_-
_introduction_and_key_figures
OECD
OECD (2014), OECD Tourism Trends and Policies 2014, Paris. URL:
http://www.oecd.org/cfe/tourism/oecdtourismtrendsandpolicies2014.htm
OECD (2013), Indicators for measuring competitiveness in tourism, OECD Tourism
Papers 2013/02. URL:
http://www.oecd.org/cfe/tourism/Indicators%20for%20Measuring%20Competitiven
ess%20in%20Tourism.pdf
OECD (2009), The impact of culture and tourism, URL:
http://www.oecd.org/cfe/tourism/theimpactofcultureontourism.htm
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 98
UN
UN (2013), Creative Economy Report 2013. Widening Local Development
Pathways, Geneva and New York: United Nations.
UN (2010), Creative Economy Report 2010. Creative Economy: A Feasible
Development Option, Geneva and New York: United Nations.
UN (2008), Creative Economy Report 2008. The Challenge of Assessing the
Creative Economy: Towards Informed Policy Making. Geneva and New York: United
Nations.
UNCTAD (2008), Creative Economy Report.
UNESCO (2009), Framework for Cultural Statistics.
Academic literature
Ashworth, G., & Larkham, P. (Eds.). (2013), Building A New Heritage, London:
Routledge.
Bachtler, J., & Gorzelak, G. (2007). Reforming EU Cohesion Policy: A reappraisal
of the performance of the Structural Funds, Policy Studies, 28(4), 309-326.
Bachtler J. and Wren C. (2006), Evaluation of EU Cohesion Policy: Research
Questions and Policy Challenges, Regional Studies, Vol. 40 (2), pp. 143-153, April.
Bachtler, J., & Michie, R. (1995), A new era in EU regional policy evaluation? The
appraisal of the Structural Funds, Regional Studies, 29(8), 745-751.
Balmey A.M., MacMillan F., Fitzsimons C.F., Shaw R., Mutrie N. (2013), “Using
programme theory to strengthen research protocol and intervention design within
an RCT of a walking intervention”, in Evaluation, volume 19, number 1.
Barzelay M. (2007), "Learning from Second-Hand Experience: Methodology for
Extrapolation-Oriented Case Research" in Governance 20(3), 521-543.
Benhamou F. (1996), L’Économie de la Culture. Éditions La Découverte, Paris.
Blake A., Sinclair T. (2007), The Economic Rational for Government Intervention
in Tourism, Report for the UK Department for Culture, Media and Sport.
Beeton, S. (2005), Film-induced tourism, (No. 25). Channel View Publications.
Burkart and Medlik (1992), Tourism: past, present and future, Butterworth-
Heinemann, London, Third Edition.
Butler, R.W., 1980. The concept of a tourist area cycle of evolution: Implications
for management of resource Canadian Geographer. Vol 14, No. 1, pp. 5-12.
Cappelen, A., Castellacci, F., Fagerberg, J., & Verspagen, B. (2003), The impact of
EU regional support on growth and convergence in the European Union. JCMS:
Journal of Common Market Studies, 41(4), 621-644.
Bound K., Briggs, R., Holden, J. & Jones, S. (2007), Cultural Diplomacy, London:
Demos.
Boukas, N., Ziakas, V., & Boustras, G. (2013), Olympic legacy and cultural
tourism: exploring the facets of Athens’ Olympic heritage. International Journal of
Heritage Studies, 19(2), 203-228.
Costa C., Panyik E., Buhalis D. (eds.)(2014), European Tourism Planning and
Organisation Systems. The EU Member States, Channel View.
Costa C., Panyik E., Buhalis D. (eds.)(2013), Trends in European Tourism Planning
and Organisation, Channel View.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 99
Costa P., Manente M. (2000), Economia del turismo. Milano: TUP –Touring
University Press.
Crang M. (2011), Tourist: moving places, becoming tourist, becoming
ethnographer. In T. Cresswell ed. Geographies of mobilities: practices, places,
subjects. Farnham: Ashgate 2011, pp.205-224.
Crang, M. (1997), “Picturing practices: research through the tourist gaze” in
Progress in Human Geography, June 1997 vol. 21 no. 3, pp. 359-373.
Curiel J. (2005), “Cultural tourism and contemporary culture in Madrid” in
Literature Review -Cultural Tourism in Spain.
De la Fuente, A., Vives, X, Dolado, JJ and Faini, R. (1995), Infrastructure and
Education as Instruments of Regional Policy: Evidence from Spain. Economic
Policy, Vol. 10, No. 20, 11-51.
Dente Bruno (2011), Policy decisions: how they are taken and how they are
studied, Il Mulino.
Dredge D, Jenkins J. (2003) Destination place identity and regional tourism policy.
Tourism Geographies: An International Journal of Tourism Space, Place and
Environment, Vol. 5, No. 4, pp.383-407.
Dwyer L., Forsyth, P., Papatheodorou A. (2011), Economics of Tourism,
Contemporary Tourism Review, URL:
http://www.goodfellowpublishers.com/free_files/fileEconomics.pdf.
Farrell B.H., Twining-Ward L.(2004), Reconceptualizing Tourism, Annals of
Tourism Research, Vol. 31, No. 2, pp. 274–295
Flew T. (2010), The Creative Industries: Culture and Policy, SAGE.
García, B. (2004), Cultural policy and urban regeneration in Western European
cities: lessons from experience, prospects for the future. Local economy, 19(4),
312-326.
Heidenreich, M., & Plaza, B. (2013). Renewal through culture? The role of
museums in the renewal of industrial regions in Europe. European Planning
Studies, (ahead-of-print), 1-15.
Howard, P., & Papayannis, T. (Eds.). (2013). Natural heritage: At the interface of
nature and culture. London: Routledge.
Jarvie, G. (2013). Sport, culture and society: an introduction. London: Routledge.
Manente M., Minghetti V., Montaguti F. (2013), "The Role of the EU in Defining
Tourism Policies for a Competitive Destination Governance" in C. Costa, E. Panyk,
D. Buhalis (eds), Trends in European Tourism Planning and Organisation. London,
Channel View, 2013
See: http://www.channelviewpublications.com/display.asp?isb=9781845414108
Martin R., Tyler P. (2006), Evaluating the impacts of the Structural Funds on
Objectives 1 Regions: An Exploratory Discussion, Regional Studies, Vol. 40(2), pp.
201-2010.
Mieczkowski Z. (1990), World Trends in Tourism and Recreation, Peter Lang, New
York.
Pellegrini, G., Terribile, F., Tarola, O., Muccigrosso, T., & Busillo, F. (2013).
Measuring the effects of European Regional Policy on economic growth: A
regression discontinuity approach. Papers in Regional Science, 92(1), 217-233.
Pforr C (2006) Tourism policy in the making. An Australian network study, Annals
of Tourism Research, Vol. 33, No. 1, pp. 87-108.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 100
Ritchie, J.R.B. and Zins, M. (1978). Culture as determinant of the attractiveness of
a tourism region. Annals of Tourism Research, 5 (2), 252-267
Ritchie B.J.R., Goeldner C.R. (2007), Tourism Principles, Practices, Philosophies,
10Th Ed Paperback, J.Wiley&Sons;
Richards G. (2010), Increasing the attractiveness of places through cultural
resources, Tourism, Culture & Communication, Vol. 10, Cognizant
RodrÍguez-Pos, A., & Fratesi, U. (2004), Between Development and Social
Policies: The Impact of European Structural Funds in Objective 1 Regions,
Regional Studies, 38:1, 97-113.
Sacco, P. L., Tavano Blessi, G., & Nuccio, M. (2008), Culture as an engine of local
development processes: system-wide Cultural Districts. Working paper. University
IUAV of Venice.
Sakai, M., 2006, Public Sector Investment in Tourism Infrastructure, in L.Dwyer
and P. Forsyth (eds.), International Handbook on the Economics of Tourism:
Edward Elgar, pp. 266-279.
Sánchez-Moral, S., Méndez, R., & Arellano, A. (2014), Creative economy and
employment quality in large urban areas in Spain. Urban Geography, 35(2), 264-
289.
Sangkyun K., Long P., Robinson M. (2009), Small Screen, Big Tourism: The Role
of Popular Korean Television Dramas in South Korean Tourism, Tourism
Geographies - TOUR GEOGR. 01/2009; 11(3):308-333.
Steiner, L., Frey, B., & Hotz, S. (2014), European capitals of culture and life
satisfaction. Urban Studies, (ahead of print) doi: 10.1177/0042098014524609.
Throsby, D. (2014), The role of culture in sustainable urban development: some
economic issues. International Journal of Global Environmental Issues, 13(2), 89-
99.
Stevenson N., Airey D., Miller G. (2008), Tourism policy making: the
policymakers’ perspective, Annals of Tourism Research, Vol. 35, No. 3, pp-732-
750.
Vanhove (2011), The Economics of Tourism Destination, Elsevier, London. Second
Edition.
Wanhill, S., 1994, The Measurement of Tourism Income Multipliers, Tourism
Management,, 15(4): 281-283.
Wanhill, S., 2005, Role of Government Incentives, in W. F. Theobald (ed.), Global
Tourism, Butterworth-Heinemann.
Practitioner Literature
Atout France (AAVV) Tableau de bord du tourisme
British Council (2010), Mapping the creative industries: a toolkit, Creative and
cultural economic series 2, URL:
http://www.britishcouncil.org/mapping_the_creative_industries_a_toolkit_2-2.pdf.
CSES Centre for Strategy & Evaluation Services (2010), Study on the Contribution
of Culture to Local and Regional Development – Evidence from the Structural
Funds, Final Report, September.
Csapo J. (2012), “The Role and Importance of Cultural Tourism in Modern Tourism
Industry” in TechOpen, Published on: 2012-04-20.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 101
Deloitte, Oxford Economics, for VisitBritain (2013) Tourism, jobs and growth: the
economic contribution of the tourism economy in the UK.
Department for Culture, Media and Sport (2014), Creative Industries Economic
Estimates.
Ecorys (2009), Study on the Competitiveness of the EU tourism industry - with
specific focus on the accommodation and tour operator & travel agent industries,
Final Report, Sept. 2009
http://ec.europa.eu/enterprise/newsroom/cf/itemdetail.cfm?item_id=3702
Ecotec (1992) Study of Methods and Concepts: Visitor Expenditure and Economic
Effects, English Tourist Board.
Exceltur con el Gobierno de La Rioja (2012) Impacto económico del turismo para
2012.
GLA Economics (2006), The rationale for public sector intervention in the
economy, URL:
http://www.london.gov.uk/mayor/economic_unit/docs/rationale_for_public_sector
_intervention.pdf.
Empresa Pública para la Gestión del Turismo y del Deporte de Andalucía ,
CONSEJERÍA DE TURISMO Y COMERCIO (AAVV) El turismo como actividad
estratégica de la economía andaluza.
Impact assessment of the draft 2007-2013 EU structural funds regulations.
Executive Summary. Study financed by the Office of the Prime Minister of the
Republic of Lithuania.
IET, Impacto Economico del turismo (AAAV)
http://www.iet.turismoencifras.es/index.php?option=com_k2&view=itemlist&layo
ut=category&task=category&id=11&Itemid=6.
KEA European Affairs. (2012). Measuring economic impact of CCIs policies - How
to justify investment in cultural and creative assets – Prepared for the INTERREG
IVC network CREA.RE, Brussels.
KEA European Affairs (2010). Promoting Investment in the Cultural and Creative
Sector: Financing Needs, Trends and Opportunities- Prepared for ECCE Innovation
– Nantes Metropole.
KEA European Affairs. (2009). The impact of Culture on Creativity – Prepared for
the European Commission (DG CULT), Brussels.
KEA European Affairs. (2006). The Economy of Culture in Europe – Prepared for
the European Commission (DG CULT), Brussels.
Manente, M. (AAVV) Il turismo nell’economia italiana. In E. Becheri, G. Maggiore
(a cura di). XIX Rapporto sul turismo italiano. Milano: Franco Angeli.
O’Brien D. (2010), AHRC/ESRC Placement Fellow, Measuring the value of culture:
a report to the Department for Culture Media and Sport, Department for Culture
Media and Sport.
Nogué, F. (2014) Le tourisme,« filière d’avenir . Développer l’emploi dans le
Tourisme. Rapport remis à Michel Sapin,Ministre du Travail, de l’Emploi, de la
Formation professionnelleet du Dialogue social et à Sylvia Pinel Ministre du
Commerce, de l’Artisanat et du Tourisme, Republique Francaise.
Oxford Economics (AAVV) The Economic Impact of the UK Film Industry.
Oxford Economics (2013) The Economic Impact of the UK Heritage Tourism
Economy.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 102
Sillignakis C. (2005), The role of private and public sector in Tourism planning and
development, URL:
http://www.sillignakis.com/PDF/The%20role%20of%20private%20and%20public
%20sector%20in%20Tourism%20planning%20and%20development.pdf
SWK Review of the 2007-13 Convergence Programmes & the contribution of the
local authority Specialist European Teams in South West Wales . A Final Report to
the Four Local Authorities in South West Wales , 18 March 2013.
Tera Consultants (2014), The economic contribution of the creative industries to
EU GDP and employment, September 2014 http://www.forum-
avignon.org/sites/default/files/editeur/2014-Oct-European-Creative-Industry-
GDP-Jobs-full-Report-ENG.pdf
UK Music (AAVV) Destination: Music. The contribution of music festivals and major
concerts for tourism in the UK.
World Economic Forum (2013), Travel and Tourism Competitiveness Report 2013,
URL: http://www.weforum.org/issues/travel-and-tourism-competitiveness.
EC Studies and publications
ADE (2012), Study on the relevance and the effectiveness of ERDF and Cohesion
Fund support to Regions with Specific Geographical Features – Islands,
Mountainous and Sparsely Populated Areas, Voll. I and II, European Commission,
(URL:
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/eval2007/ge
ographical_final1.pdf;
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/eval2007/ge
ographical_final2.pdf)
CSIL (2008), The Impact of Tourism on Coastal Areas: Regional Development
Aspects, European Parliament- Policy Department B: Structural and Cohesion
Policies.
European Court of Auditors (2011), Were ERDF co-financed tourism projects
effective?, Special report no 6.
European Commission (2014), Preference of Europeans towards tourism, Flash
Eurobarometer 392, February,
http://ec.europa.eu/public_opinion/flash/fl_392_en.pdf.
European Commission, DG Regional and Urban Policy (2013), Job creation as an
indicator of outcomes in ERDF programmes, Expert evaluation network delivering
policy analysis on the performance of Cohesion Policy 2007-2013.
European Commission (2010). Europe 2020: A strategy for smart, sustainable and
inclusive growth- COM(2010) 202. European Union (2010) Investing in our regions: 150 examples of projects co-
funded by European Regional Policy.
LSE, EPRC (2013), Evaluation of the main achievements of Cohesion Policy
Programmes and Projects over the longer term in 15 selected regions. Final
Report, (URL:
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/eval2007/co
hesion_achievements/final_report.pdf).
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 103
SEC (2011) 1141 Final Commission Staff Working Paper. Impact assessment
Accompanying the document Proposal for a REGULATION OF THE EUROPEAN
PARLIAMENT AND OF THE COUNCIL laying down common provisions on the
European Regional Development Fund, the European Social Fund, the Cohesion
Fund, the European Agricultural Fund for Rural Development and the European
Maritime and Fisheries Fund covered by the Common Strategic Framework and
laying down general provisions on the European Regional Development Fund, the
European Social Fund and the Cohesion Fund and repealing Regulation (EC) No
1083/2006.
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 104
Annex II First elaboration on ERDF AIR raw data (Project selection), 2007-2013
Table A1 - Number of OPs and allocations in culture and tourism by Country and Priority Theme (ordered by Total
Amount) – 2012 data
Country
Number of Ops AR Community Amount in M. €
Tourism
55-57
Culture
58-60
14 -
H&R
Total 55-
60+H&R
Total
OPs
Number of OPs with allocation
in culture and
tourism
/Number of
Total OPs
Tourism
55-57
Culture
58-60 14 - H&R
Total 55-
60+H&R
Total
Amount
Amount in
culture and
tourism Priority
Themes in % of
Total
PL 18 17 16 19 20 95.0 878.5 980.4 199.1 2057.9 48844.7 4.2
GR 6 6 7 7 10 70.0 197.5 687.0 764.7 1649.2 21750.2 7.6
ES 14 13 14 18 23 78.3 370.7 346.4 67.2 784.3 21634.3 3.6
HU 7 8 5 10 13 76.9 646.3 375.1 25.8 1047.2 19458.1 5.4
IT 19 19 17 22 28 78.6 571.6 456.3 78.8 1106.7 18994.1 5.8
CZ 9 9 4 9 14 64.3 525.0 424.0 2.5 951.5 16544.9 5.8
PT 7 8 8 9 10 90.0 235.6 344.0 468.4 1048.1 13595.6 7.7
DE 14 14 13 17 18 94.4 256.6 276.0 162.9 695.4 13343.4 5.2
RO 1 1 1 1 5 20.0 380.4 295.6 3.1 679.0 12309.7 5.5
SK 3 2 3 9 33.3 146.8 129.8 276.6 7495.8 3.7
CB 61 63 5 65 71 91.5 635.6 553.0 2.9 1191.5 7195.7 16.6
FR 24 20 14 27 31 87.1 255.4 125.7 23.7 404.8 6154.7 6.6
LT 1 1 1 2 2 100.0 113.3 101.3 0.1 214.6 5204.3 4.1
BG 1 1 1 5 20.0 28.9 135.0 163.9 5190.5 3.2
UK 7 3 2 7 16 43.8 152.5 32.0 0.9 185.4 4418.0 4.2
LV 1 1 2 2 100.0 34.5 0.5 35.0 3653.0 1.0
EE 2 1 1 2 2 100.0 114.7 31.8 0.1 146.6 2767.8 5.3
SI 1 1 1 1 2 50.0 112.2 109.5 30.8 252.5 2709.2 9.3
BE 4 3 4 4 100.0 46.4 48.2 94.6 997.2 9.5
SE 5 5 6 8 75.0 59.4 10.0 69.4 982.4 7.1
FI 4 4 4 5 5 100.0 65.5 22.5 7.2 95.2 887.7 10.7
NL 3 3 2 4 4 100.0 63.4 34.4 2.7 100.5 816.3 12.3
MT 1 1 1 1 100.0 31.7 70.2 101.9 641.7 15.9
CY 1 1 1 1 100.0 10.4 14.6 25.0 579.8 4.3
AT 5 2 8 8 9 88.9 6.0 7.0 54.0 67.0 514.2 13.0
IE 1 1 1 2 50.0 4.4 0.8 5.1 402.2 1.3
DK 1 1 1 1 1 100.0 7.5 8.4 14.8 30.7 232.1 13.2
LU 1 0.0 24.9 0.0
Total 219 209 126 253 317 79.8 5905.7 5649.2 1924.7 13479.5 237342.4 5.7
For Tourism: 55 Promotion of natural assets, 56 Protection and development of natural heritage, 57 Other assistance to improve tourist services.
For Culture: 58 Protection and preservation of the cultural heritage, 59 Development of cultural infrastructure, 60 Other assistance to improve cultural services.
14 H&R: Hotel & Restaurant Economic Activity not included in 55-60 Priority Themes
Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 105
Table A2 - OPs investing more than 15 millions of Euro in culture and tourism (ordered by Total allocations in Tourism,
culture and tourism) - 2012 data
Country OP Code Name of the OP Allocations in Tourism
(55-57)
Allocations in Culture (58-60)
Allocations in H&R
(Economic activity=14)
Total allocations in Tourism, Culture and
H&R
Overall amount of OP
Incidence % of
culture and tourism on the total amount
RO 2007RO161PO001 Regional Operational Programme 380.4 295.6 3.1 679.0 3531.3 19.2
GR 2007GR161PO001 Ανταγωνιστικότητα και Επιχειρηματικότητα 101.0 78.5 455.4 634.8 1881.0 33.7
PL 2007PL161PO002 Program Operacyjny Infrastruktura i Środowisko 0.0 405.0 0.0 405.0 24051.0 1.7
GR 2007GR161PO008 Μακεδονία - Θράκη 8.9 248.1 89.6 346.6 3751.1 9.2
PT 2007PT161PO002 PO Regional do Norte 2007-2013 90.5 136.5 96.3 323.4 2545.9 12.7
ES 2007ES161PO008 Programa Operativo FEDER de Andalucía 203.0 101.4 8.7 313.1 5879.9 5.3
HU 2007HU161PO011 Operational Programme for South Transdanubia 93.0 164.7 0.0 257.7 714.1 36.1
SI 2007SI161PO001 Operativni program krepitve regionalnih razvojnih 112.2 109.5 30.8 252.5 1546.8 16.3
CZ 2007CZ16UPO002 Integrovaný operační program 50.9 202.9 0.2 254.0 1303.6 19.5
IT 2007IT161PO010 Programma Operativo FESR Puglia 2007-2013 90.3 138.5 14.8 243.7 2171.0 11.2
LT 2007LT161PO001 2007-2013 m. Sanglaudos skatinimo veiksmų programa 113.3 101.3 0.0 214.6 2491.3 8.6
IT 2007IT161PO011 Por Sicilia FESR 145.8 47.3 6.2 199.3 3513.5 5.7
IT 2007IT161PO001 Poin Attrattori culturali, naturali e turismo 146.9 11.1 35.1 193.1 217.9 88.7
GR 2007GR161PO006 Αττική 8.5 154.8 19.3 182.6 2750.4 6.6
GR 2007GR16UPO002 Κρήτη & Νήσοι Αιγαίου 12.0 75.8 90.0 177.7 1259.3 14.1
GR 2007GR16UPO001 Θεσσαλία - Στερεά Ελλάδα - Ήπειρος 35.1 52.3 85.4 172.8 1631.4 10.6
BG 2007BG161PO001 Operational Programme Regional Development 28.9 135.0 0.0 163.9 1231.3 13.3
PL 2007PL161PO001 Program Operacyjny Innowacyjna Gospodarka, 2007-20 164.6 0.0 2.8 167.3 7369.2 2.3
PT 2007PT161PO003 PO Regional do Centro 2007-2013 19.8 67.1 77.5 164.4 1645.6 10.0
PL 2007PL161PO010 Małopolski Regionalny Program Operacyjny na lata 2 47.1 103.9 9.7 160.7 1200.8 13.4
PT 2007PT161PO001 PO Factores de Competitividade 2007-2013 0.0 0.0 163.4 163.4 3105.1 5.3
HU 2007HU161PO006 Operational Programme for North Hungary 136.0 15.9 0.0 151.9 820.3 18.5
HU 2007HU161PO009 Operational Programme for North Great Plain 139.8 14.5 0.0 154.3 897.0 17.2
PL 2007PL161PO005 Regionalny Program Operacyjny dla Województwa Doln 63.0 34.2 52.3 149.5 1052.3 14.2
IT 2007IT161PO009 Por Campania FESR 79.1 65.8 0.8 145.7 3268.1 4.5
SK 2007SK161PO006 OP Competitiveness and Economic Growth 138.2 5.1 0.0 143.2 716.0 20.0
HU 2007HU161PO004 Operational Programme for South Great Plain 122.2 11.2 0.3 133.8 714.8 18.7
CZ 2007CZ161PO008 ROP NUTS II Severozápad 92.4 38.3 0.0 130.7 562.6 23.2
PL 2007PL161PO020 Regionalny Program Operacyjny Województwa Warmińsk 77.5 28.4 24.7 130.7 853.9 15.3
SK 2007SK161PO003 Regional Operational Programme 7.7 124.7 0.0 132.4 1488.7 8.9
PL 2007PL161PO019 Regionalny Program Operacyjny Województwa Śląskieg 61.1 50.1 19.3 130.5 1556.2 8.4
PT 2007PT161PO006 Programa Operacional dos Açores para a Convergênci 71.2 26.6 32.5 130.3 955.7 13.6
CZ 2007CZ161PO005 ROP NUTS II Severovýchod 104.0 15.6 1.2 120.9 567.5 21.3
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 106
Country OP Code Name of the OP Allocations in Tourism
(55-57)
Allocations in Culture (58-60)
Allocations in H&R
(Economic activity=14)
Total allocations in Tourism, Culture and
H&R
Overall amount of OP
Incidence % of
culture and tourism on the total amount
GR 2007GR161PO007 Δυτική Ελλάδα - Πελοπόννησος - Ιόνιοι Νήσοι 32.1 77.6 5.3 114.9 1250.0 9.2
DE 2007DE162PO007 Operationelles Programm EFRE Nordrhein-Westfalen 2 58.8 53.4 0.0 112.2 1220.6 9.2
PT 2007PT161PO004 PO Regional do Alentejo 2007-2013 13.4 31.5 69.1 114.0 793.2 14.4
HU 2007HU161PO003 Operational Programme for West Pannon 48.3 57.5 0.0 105.8 440.6 24.0
CZ 2007CZ161PO001 ROP NUTS II Jihovýchod 60.6 41.3 0.0 101.9 544.4 18.7
HU 2007HU161PO005 Operational Programme for Central Transdanubia 49.0 58.3 0.0 107.3 481.7 22.3
MT 2007MT161PO001 Operational Programme I - Investing in Competitive 31.7 70.2 0.0 101.9 641.7 15.9
PL 2007PL161PO011 Regionalny Program Operacyjny Województwa Mazowiec 33.7 69.5 2.8 106.0 1556.0 6.8
ES 2007ES161PO005 Programa Operativo FEDER de Galicia 59.6 32.8 9.0 101.3 2294.4 4.4
EE 2007EE161PO002 Operational Programme for the Development of Livin 70.3 31.8 0.0 102.1 1380.4 7.4
PL 2007PL161PO016 Regionalny Program Operacyjny Województwa Zachodni 40.4 51.7 2.7 94.8 703.8 13.5
PL 2007PL161PO007 Regionalny Program Operacyjny Województwa Lubelski 52.9 35.1 5.7 93.8 927.7 10.1
CZ 2007CZ161PO002 ROP NUTS II Střední Morava 70.7 19.9 0.2 90.8 440.5 20.6
CZ 2007CZ161PO013 ROP NUTS II Jihozápad 57.2 31.6 0.0 88.9 523.6 17.0
PL 2007PL161PO009 Regionalny Program Operacyjny Województwa Łódzkieg 46.5 44.5 4.2 95.2 972.4 9.8
HU 2007HU162PO001 Operational Programme for Central Hungary 57.9 30.3 5.1 93.3 1561.6 6.0
CZ 2007CZ161PO009 ROP NUTS II Střední Čechy 45.3 35.6 0.0 80.9 458.1 17.7
UK 2007UK161PO002 West Wales and the Valleys ERDF Convergence programme 80.6 0.0 0.0 80.6 1287.3 6.3
ES 2007ES162PO006 Programa Operativo FEDER de Cataluña 31.1 44.5 0.0 75.6 638.1 11.9
CZ 2007CZ161PO010 ROP NUTS II Moravskoslezsko 42.0 30.6 0.0 72.6 572.8 12.7
ES 2007ES162PO009 Programa Operativo FEDER de Castilla y León 2.5 57.8 10.5 70.8 851.0 8.3
DE 2007DE162PO010 Operationelles Programm EFRE Niedersachsen (ohne R 30.1 34.9 7.5 72.4 506.3 14.3
ES 2007ES161PO006 Programa Operativo FEDER de Extremadura 12.3 49.7 5.8 67.9 1282.2 5.3
DE 2007DE162PO001 Operationelles Programm EFRE Bayern 2007 - 2013 21.1 27.5 19.2 67.8 476.4 14.2
PL 2007PL161PO015 Regionalny Program Operacyjny Województwa Pomorski 37.8 27.2 2.7 67.6 877.3 7.7
PL 2007PL161PO014 Regionalny Program Operacyjny Województwa Podlaski 30.0 16.1 19.6 65.7 543.7 12.1
PL 2007PL161PO003 Program Operacyjny Rozwój Polski Wschodniej 2007-2 61.7 0.0 0.0 61.7 2292.8 2.7
PL 2007PL161PO012 Regionalny Program Operacyjny Województwa Opolskie 20.6 13.1 29.7 63.4 446.5 14.2
DE 2007DE161PO002 Operationelles Programm EFRE Brandenburg 2007-2013 27.9 16.4 20.8 65.1 1247.5 5.2
DE 2007DE161PO007 Operationelles Programm EFRE Sachsen-Anhalt 2007-2 18.2 31.1 14.0 63.3 1663.1 3.8
PL 2007PL161PO006 Regionalny Program Operacyjny Województwa Kujawsko 44.0 18.5 0.7 63.2 773.5 8.2
CB 2007CB163PO005 Programa Operativo FEDER Cooperación Transfronteri 36.1 21.4 0.0 57.5 257.7 22.3
CB 2007CB163PO017 Programm Ziel 3 / Cíl 3 zur Förderung der grenzübe 47.2 14.0 0.0 61.1 196.0 31.2
PL 2007PL161PO017 Regionalny Program Operacyjny Województwa Wielkopo 30.4 24.5 1.7 56.6 1192.1 4.7
IT 2007IT162PO005 Por Liguria FESR 0.0 52.1 2.5 54.6 489.7 11.2
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 107
Country OP Code Name of the OP Allocations in Tourism
(55-57)
Allocations in Culture (58-60)
Allocations in H&R
(Economic activity=14)
Total allocations in Tourism, Culture and
H&R
Overall amount of OP
Incidence % of
culture and tourism on the total amount
CB 2007CB163PO063 INTERREG IV France-Wallonie-Vlaanderen 25.6 28.9 0.0 54.5 140.6 38.8
BE 2007BE161PO001 Programme opérationnel 'Convergence' Hainaut - FED 36.3 15.2 0.0 51.6 455.2 11.3
DE 2007DE161PO001 Operationelles Programm EFRE Thüringen 2007 bis 20 9.8 37.8 3.0 50.6 1053.1 4.8
DE 2007DE162PO003 Operationelles Programm EFRE Schleswig-Holstein 20 19.1 13.1 18.3 50.6 316.2 16.0
CB 2007CB163PO025 OP Česká republika - Polsko 40.5 12.5 0.2 53.2 168.1 31.7
PL 2007PL161PO018 Regionalny Program Operacyjny Województwa Świętokr 41.9 13.1 0.0 55.0 634.0 8.7
PT 2007PT16UPO001 PO Temático Valorização do Território 2007-2013 0.0 53.1 0.0 53.1 3775.0 1.4
CB 2007CB163PO006 Programa Operativo FEDER Cooperación Transfronteri 39.2 13.8 0.0 53.0 163.9 32.3
PL 2007PL161PO013 Regionalny Program Operacyjny Województwa Podkarpa 5.6 29.9 13.9 49.4 1055.4 4.7
PT 2007PT162PO002 PO Valorização do Potencial Económico e Coesão Ter 24.9 7.8 15.7 48.4 281.7 17.2
DE 2007DE161PO006 Operationelles Programm EFRE Niedersachsen - Regio 21.9 21.8 4.6 48.3 486.5 9.9
FR 2007FR161PO003 Programme opérationnel FEDER Martinique 33.9 14.4 0.0 48.3 318.4 15.2
CB 2007CB163PO038 Programme des 2 mers 16.1 29.4 0.0 45.5 164.5 27.7
EE 2007EE161PO001 Operational Programme for the Development of Econo 44.4 0.0 0.1 44.5 1387.4 3.2
PL 2007PL161PO008 Regionalny Program Operacyjny Województwa Lubuskie 19.8 15.5 6.5 41.7 406.0 10.3
BE 2007BE162PO003 Programme opérationnel 'Compétitivité régionale et 8.5 30.3 0.0 38.8 290.0 13.4
FR 2007FR161PO004 Programme opérationnel FEDER Réunion 32.2 3.3 7.8 43.3 672.1 6.5
NL 2007NL162PO001 Operationeel Programma Noord 2007-2013 31.0 13.2 0.0 44.2 188.1 23.5
NL 2007NL162PO002 Operationeel Programma West 2007-2013 24.3 15.4 0.9 40.6 301.3 13.5
UK 2007UK162PO008 North West England ERDF Regional Competitiveness and Employment 18.6 20.2 0.0 38.8 568.6 6.8
DE 2007DE161PO003 Operationelles Programm EFRE 2007 - 2013 Mecklenbu 0.0 0.0 38.7 38.7 912.8 4.2
DE 2007DE162PO011 Operationelles Programm EFRE Rheinland-Pfalz 2007- 11.8 4.7 17.5 33.9 158.9 21.4
IT 2007IT162PO002 Por Emilia Romagna FESR 8.2 25.7 0.0 33.9 172.3 19.7
CB 2007CB163PO034 PO Italia-Francia Alpi (ALCOTRA) 14.0 22.5 0.0 36.5 144.0 25.4
CB 2007CB163PO068 Hungary-Slovakia Cross-border Co-operation Program 24.3 8.5 0.0 32.8 140.3 23.4
FI 2007FI162PO002 Pohjois-Suomen EAKR-toimenpideohjelma 2007-2013 26.5 8.8 4.7 40.0 301.0 13.3
IT 2007IT161PO012 Por Basilicata ST FESR 2.5 30.8 1.4 34.6 252.3 13.7
CB 2007CB163PO019 Ziel 3-Programm zur grenzüberschreitenden Zusammen 14.0 21.4 0.0 35.4 116.2 30.4
DE 2007DE162PO004 Operationelles Programm EFRE Berlin 2007-2013 3.8 29.4 0.5 33.6 814.7 4.1
IT 2007IT161PO008 POR Calabria FESR 2007 - 2013 7.3 26.3 0.0 33.6 932.6 3.6
IT 2007IT162PO011 PO Regione Piemonte FESR 30.3 0.0 0.6 30.9 309.2 10.0
LV 2007LV161PO002 Infrastructure and Services 0.0 34.5 0.0 34.5 2975.7 1.2
CB 2007CB163PO040 Interreg IV A programme de coopération transfronta 7.1 22.7 1.8 31.7 153.4 20.6
SE 2007SE162PO007 Mellersta Norrland 27.1 4.6 0.0 31.7 181.7 17.4
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 108
Country OP Code Name of the OP Allocations in Tourism
(55-57)
Allocations in Culture (58-60)
Allocations in H&R
(Economic activity=14)
Total allocations in Tourism, Culture and
H&R
Overall amount of OP
Incidence % of
culture and tourism on the total amount
DK 2007DK162PO001 Innovation og Viden 7.5 8.4 14.8 30.7 232.1 13.2
CB 2007CB163PO012 Program współpracy przygranicznej Polska-Słowacja 13.6 19.7 0.0 33.3 140.3 23.8
ES 2007ES161PO004 Programa Operativo FEDER de Asturias 13.9 14.6 3.0 31.5 467.1 6.7
ES 2007ES161PO007 Programa Operativo FEDER de Castilla La Mancha 16.5 5.5 11.3 33.3 1232.0 2.7
FI 2007FI162PO001 Itä-Suomen EAKR-toimenpideohjelma 2007-2013 24.0 3.7 2.0 29.7 314.7 9.4
FR 2007FR162PO017 Programme opérationnel FEDER NORD PAS-DE-CALAIS 0.0 35.0 0.0 35.0 533.0 6.6
UK 2007UK162PO003 Nothern Ireland ERDF Regional Competitiveness and Employment 33.2 0.0 0.0 33.2 177.2 18.7
DE 2007DE162PO005 Operationelles Programm EFRE Hessen 2007-2013 24.7 0.5 1.6 26.8 222.8 12.0
PT 2007PT162PO001 PO Regional de Lisboa 2007-2013 8.4 16.8 0.7 25.9 316.8 8.2
CB 2007CB163PO009 Ziel 3-Programm zur grenzübergreifenden Zusammenar 9.8 17.8 0.0 27.6 105.5 26.1
CB 2007CB163PO035 Programma Operativo di Cooperazione Transfrontalie 21.2 10.9 0.0 32.1 63.8 50.3
FR 2007FR162PO016 Programme opérationnel FEDER PAYS DE LA LOIRE 5.8 20.0 0.0 25.8 237.2 10.9
ES 2007ES162PO004 Programa Operativo FEDER de Madrid 6.6 20.9 0.0 27.5 219.1 12.6
CB 2007CB163PO018 Operationelles Programm zur grenzübergreifenden Zu 10.6 14.1 0.0 24.7 98.6 25.1
CB 2007CB163PO059 Πρόγραμμα Ευρωπαϊκής Εδαφικής Συνεργασίας Ελλάδα – 7.9 15.0 0.0 22.9 115.9 19.7
SE 2007SE162PO008 Övre Norrland 24.9 3.4 0.0 28.3 255.4 11.1
AT 2007AT161PO001 OP Burgenland 2007-2013: Ziel Konvergenz/Phasing O 3.6 6.1 18.5 28.2 96.2 29.3
FR 2007FR162PO004 Programme opérationnel FEDER AUVERGNE 20.3 1.6 0.9 22.8 168.5 13.5
CB 2007CB163PO036 Programma per la cooperazione transfrontaliera Ita 10.4 18.7 0.0 29.0 114.2 25.4
CB 2007CB163PO061 Central Europe 2007-2013 6.3 14.6 0.0 20.9 233.7 8.9
HU 2007HU161PO008 Operational Programme for Social Infrastructure 0.0 22.7 0.0 22.7 1410.8 1.6
CB 2007CB163PO069 South East Europe (SEE) Transnational Co-operation 10.2 10.5 0.0 20.8 218.7 9.5
CB 2007CB163PO011 Program współpracy przygranicznej Polska-Niemcy (w 20.8 5.2 0.0 26.1 113.3 23.0
PT 2007PT161PO005 PO Regional do Algarve 2007-2013 7.5 4.5 13.2 25.2 141.7 17.8
CB 2007CB163PO026 Interreg IV Öresund-Kattegatt-Skagerrak 8.6 9.3 0.0 17.9 111.9 16.0
ES 2007ES162PO010 Programa Operativo FEDER de la Comunidad Valencian 15.2 6.2 1.0 22.4 1245.7 1.8
FR 2007FR162PO022 Programme opérationnel FEDER RHONE-ALPES 22.1 3.9 0.0 26.0 296.2 8.8
GR 2007GR161PO002 Ψηφιακή Σύγκλιση 0.0 0.0 19.7 19.7 837.9 2.4
IT 2007IT162PO004 Por Lazio FESR 21.5 2.8 0.0 24.2 373.4 6.5
CB 2007CB163PO066 Central Baltic INTERREG IV A Programme 2007-2013 4.8 12.1 0.0 16.9 100.0 16.9
CY 2007CY16UPO001 Sustainable Development and Competitiveness 0.0 10.4 14.6 25.0 579.8 4.3
DE 2007DE161PO004 Operationelles Programm EFRE Sachsen 2007-2013 5.9 0.0 17.3 23.2 2782.3 0.8
FR 2007FR162PO001 Programme opérationnel FEDER AQUITAINE 22.3 0.0 0.0 22.3 331.8 6.7
CB 2007CB163PO054 Operational Programme Slovenia-Austria 2007-2013 12.6 6.0 0.0 18.6 66.7 27.9
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 109
Country OP Code Name of the OP Allocations in Tourism
(55-57)
Allocations in Culture (58-60)
Allocations in H&R
(Economic activity=14)
Total allocations in Tourism, Culture and
H&R
Overall amount of OP
Incidence % of
culture and tourism on the total amount
FR 2007FR162PO014 Programme opérationnel FEDER LIMOUSIN 5.7 11.9 0.0 17.7 96.6 18.3
IT 2007IT162PO016 Por Sardegna ST FESR 3.4 13.1 1.2 17.7 551.2 3.2
UK 2007UK161PO001 Highlands and Islands of Scotland ERDF phasing out 11.1 9.5 0.7 21.3 120.2 17.7
CB 2008CB163PO001 Programa de Cooperación Territorial Transfronteriz 13.6 8.7 0.0 22.2 88.6 25.1
CB 2007CB163PO030 Program cezhraničnej spolupráce Slovenská republik 9.2 7.4 0.0 16.7 82.1 20.3
CB 2007CB163PO031 Lithuania - Poland 2007-2013 European Teritorial C 4.4 14.2 0.0 18.5 72.2 25.7
CB 2007CB163PO064 Programme transfrontalier Grande Région 7.7 10.0 0.0 17.6 102.0 17.3
CB 2007CB163PO065 Grensregio Vlaanderen - Nederland - Operationeel p 9.9 9.6 0.0 19.4 94.0 20.7
IT 2007IT162PO012 Por Toscana FESR 11.0 8.7 0.7 20.3 332.7 6.1
CB 2007CB163PO002 Operational Programme Objective European Territori 10.4 9.0 0.0 19.3 91.4 21.2
CB 2007CB163PO047 EU Programme for Cross Border Territorial Cooperat 15.4 3.9 0.0 19.3 161.3 11.9
CB 2007CB163PO052 INTERREG IV A Italia/Austria 12.0 5.5 0.0 17.5 56.6 30.9
FR 2007FR161PO002 Programme opérationnel FEDER Guadeloupe 20.1 0.1 0.0 20.2 317.9 6.4
CB 2007CB163PO045 Programme opérationnel CTE MED - Méditerranée 10.3 6.1 0.0 16.4 185.8 8.8
CB 2007CB163PO060 Πρόγραμμα Ευρωπαϊκής Εδαφικής Συνεργασίας Ελλάδα – 5.5 13.6 0.0 19.0 59.5 31.9
FR 2007FR162PO021 Programme opérationnel FEDER MIDI-PYRENEES 11.0 0.2 6.0 17.2 377.1 4.6
CB 2007CB163PO067 Hungary-Romania Cross-border Co-operation Programm 7.6 7.8 0.0 15.3 224.5 6.8
ES 2007ES162PO011 Programa Operativo FEDER de Canarias 0.0 0.0 16.2 16.2 863.8 1.9
IT 2007IT162PO006 Por Lombardia FESR 11.8 6.2 0.0 18.0 189.1 9.5
FR 2007FR162PO006 Programme opérationnel FEDER BOURGOGNE 11.2 3.9 0.0 15.2 158.0 9.6
FR 2007FR162PO010 Programme opérationnel FEDER FRANCHE-COMTE 9.3 6.6 0.0 16.0 120.8 13.2
IT 2007IT162PO015 Por Veneto FESR 3.9 9.4 1.8 15.1 193.0 7.8
Total targeted OPs 5641.9 5446.5 1839.6 12928.0 157913.2 8.2
Overall OPs with al allocations in culture and tourism 5905.7 5649.2 1924.7 13479.5 180331.7 7.5 For Tourism: 55 Promotion of natural assets, 56 Protection and development of natural heritage, 57 Other assistance to improve tourist services; For Culture: 58 Protection and preservation of
the cultural heritage, 59 Development of cultural infrastructure, 60 Other assistance to improve cultural services.
14 H&R: Hotel & Restaurant Economic Activity not included in 55-60 Priority Themes
Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 110
Table A3 - Allocations in culture and tourism by Economic sector (ordered by % of the total) - 2012 data
For Tourism: 55 Promotion of natural assets, 56 Protection and development of natural heritage, 57 Other assistance to improve tourist services.
For Culture: 58 Protection and preservation of the cultural heritage, 59 Development of cultural infrastructure, 60 Other assistance to improve cultural services.
14 H&R: Hotel & Restaurant Economic Activity not included in 55-60 Priority Themes
Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
M. € % M. € % M. € % M. € % M. € % M. € % M. € % M. € % M. € %
Other unspecified services 295.6 36.4 86.5 12.6 2046.0 46.4 2477.1 40.6 1585.6 49.8 697.9 33.1 182.4 51.2 2548.8 43.6 4893.9 42.4
Not applicable 40.1 4.9 168.1 24.6 432.4 9.8 670.2 11.0 653.1 20.5 569.8 27.0 83.3 23.4 1353.7 23.1 1946.9 16.8
Public administration 136.4 16.8 101.6 14.8 470.7 10.7 740.3 12.1 356.7 11.2 461.8 21.9 35.1 9.9 886.7 15.2 1562.2 13.5
Hotels and restaurants 42.6 5.3 8.2 1.2 781.4 17.7 838.6 13.7 49.4 1.6 5.4 0.3 1.7 0.5 58.4 1.0 888.8 7.7
Construction 33.9 4.2 32.2 4.7 276.7 6.3 351.6 5.8 202.1 6.4 128.4 6.1 10.9 3.1 353.9 6.1 684.2 5.9
Activities linked to the environment 155.1 19.1 217.3 31.8 84.7 1.9 508.0 8.3 55.7 1.7 10.8 0.5 1.6 0.5 70.4 1.2 525.2 4.5
Social work, community, social and personal services 44.9 5.5 32.8 4.8 100.2 2.3 188.2 3.1 140.5 4.4 145.9 6.9 19.0 5.3 316.7 5.4 483.3 4.2
Real estate, renting and business activities 44.0 5.4 1.8 0.3 104.2 2.4 155.6 2.5 62.4 2.0 43.7 2.1 8.1 2.3 118.2 2.0 264.1 2.3
Education 2.4 0.3 0.7 0.1 17.3 0.4 20.8 0.3 49.8 1.6 44.7 2.1 11.0 3.1 109.2 1.9 125.9 1.1
Transport 3.5 0.4 18.2 2.7 30.6 0.7 55.4 0.9 8.3 0.3 0.0 0.5 0.2 9.1 0.2 61.2 0.5
Human health activities 5.2 0.6 6.9 1.0 38.0 0.9 51.8 0.8 8.1 0.3 0.0 0.0 0.1 0.0 8.4 0.1 58.3 0.5
Wholesale and retail trade 0.0 4.2 0.6 15.5 0.4 20.4 0.3 1.7 0.1 0.1 0.0 0.4 0.1 2.3 0.0 21.9 0.2
Agricuture, hunting and forestry 4.4 0.5 5.5 0.8 5.9 0.1 17.2 0.3 4.3 0.1 1.0 0.0 0.0 5.5 0.1 21.1 0.2
Collection, purification and distribution fo water 2.8 0.3 0.0 0.0 3.2 0.1 0.8 0.0 0.0 0.0 0.8 0.0 3.6 0.0
Manufacture of food products and beverages 0.0 0.0 0.0 2.0 0.0 2.0 0.0 1.2 0.0 0.0 0.0 1.3 0.0 3.2 0.0
Manufacture of textiles and textile products 0.0 0.0 0.0 0.0 0.0 0.0 1.8 0.1 1.3 0.4 3.1 0.1 3.0 0.0
Unspecified manufacturing industries 0.0 0.1 0.0 2.3 0.1 2.4 0.0 0.6 0.0 0.0 0.0 0.6 0.0 3.0 0.0
Financial intermediation 0.0 0.0 0.1 0.0 0.5 0.0 0.6 0.0 1.5 0.0 0.0 0.0 1.6 0.0 2.2 0.0
Fishing 1.1 0.1 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.0
Manufacture of tranport equipment 0.0 0.0 0.5 0.0 0.5 0.0 0.1 0.0 0.0 0.0 0.1 0.0 0.6 0.0
Mining and quarrying of energy producting materials 0.0 0.0 0.5 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
Post and telecommunications 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.1 0.5 0.0 0.5 0.0
Total 812.0 100.0 684.3 100.0 4409.4 100.0 6105.7 100.0 3181.9 100.0 2111.3 100.0 356.0 100.0 5849.2 100.0 11554.9 100.0
55 56 57 55-57
TotalTourism Culture
58 59Economic Cd
60 58-60
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 111
Table A4 - Allocations in culture and tourism by territorial dimension (ordered by % of the total) - 2012 data
For Tourism: 55 Promotion of natural assets, 56 Protection and development of natural heritage, 57 Other assistance to improve tourist services.
For Culture: 58 Protection and preservation of the cultural heritage, 59 Development of cultural infrastructure, 60 Other assistance to improve cultural services.
14 H&R: Hotel & Restaurant Economic Activity not included in 55-60 Priority Themes
Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
M. € % M. € % M. € % M. € % M. € % M. € % M. € % M. € % M. € % M. € %
Urban 338,5 41,7 238,5 34,9 2107,7 47,8 2761,2 45,2 1716,1 53,9 1281,0 60,7 125,8 35,3 3237,5 55,3 692,8 36,0 6500,3 48,2
Rural areas 246,1 30,3 117,4 17,2 942,8 21,4 1353,8 22,2 736,5 23,1 223,1 10,6 19,4 5,4 1012,6 17,3 448,7 23,3 2733,9 20,3
Not applicable 58,8 7,2 60,2 8,8 578,3 13,1 713,4 11,7 214,4 6,7 347,3 16,4 14,6 4,1 599,4 10,2 262,9 13,7 1536,5 11,4
Cross-border cooperation area 52,4 6,5 131,6 19,2 387,7 8,8 597,5 9,8 235,5 7,4 117,3 5,6 146,4 41,1 512,2 8,8 2,9 0,1 1073,9 8,0
Islands 8,8 1,1 9,0 1,3 129,7 2,9 149,9 2,5 96,8 3,0 34,0 1,6 11,8 3,3 147,3 2,5 257,7 13,4 547,8 4,1
Outermost region 65,1 8,0 36,8 5,4 86,9 2,0 202,2 3,3 21,6 0,7 29,0 1,4 5,5 1,5 58,1 1,0 63,9 3,3 308,7 2,3
Sparsely and very sparsely
pupulated areas 23,0 2,8 19,3 2,8 86,7 2,0 134,6 2,2 35,4 1,1 34,7 1,6 5,1 1,4 78,0 1,3 97,7 5,1 301,9 2,2
Mountains 2,1 0,3 48,3 7,1 28,1 0,6 85,8 1,4 96,9 3,0 23,6 1,1 3,5 1,0 128,1 2,2 34,5 1,8 236,9 1,8
Transnational cooperation area 13,7 1,7 18,4 2,7 37,1 0,8 73,6 1,2 23,4 0,7 8,2 0,4 18,9 5,3 51,6 0,9 0,0 119,7 0,9
Former UE external borders (after
30.04.2004 ) 3,1 0,4 3,2 0,5 16,9 0,4 24,0 0,4 5,2 0,2 13,2 0,6 5,0 1,4 24,3 0,4 63,6 3,3 110,2 0,8
Inter-regional cooperation area 0,3 0,0 1,7 0,3 7,4 0,2 9,7 0,2 0,2 0,0 0,0 0,0 0,2 0,0 0,0 9,6 0,1
Total 812,0 100,0 684,3 100,0 4409,4 100,0 6105,7 100,0 3181,9 100,0 2111,3 100,0 356,0 100,0 5849,2 100,0 1924,7 100,0 13479,5 100,0
55 56 57 55-57
Tourism Culture Total
58Territory Cd
59 60 58-60
14 - H&R
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 112
Table A5 – Hotel & Restaurant Economic Sector: Allocations by Priority
themes - 2012 data
Millions of EURO %
01 - R&TD activities in research centres 0.3 0.0 03 - Technology transfer and improvement of cooperation networks between small business (SMEs), between these and other business and universities, post-secondary education establishments of all kinds, regional authorities, research centres and scientific and technological poles
1.4 0.0
04 - Assistance to R&TD, particularly in SMEs 0.7 0.0 05 - Advances support services for firms and groups of firms 6.8 0.2 06 - Assistance to SMEs for the promotion of environmentally-friendly products and production processs
31.5 1.1
07 - Investment in firms directly linked to research and innovation 410.6 14.6 08 - Other investment in firms 958.6 34.1 09 - Other measures to stimulate research and innovation and entrepreneurship in SMEs 426.6 15.2 11 - Information and communication technologies (access, security, interoperability, risk-prevention, research, innovation, e-content, etc).
1.6 0.1
13 - Services and applications for the citizen (e-health, e-government, etc.). 0.4 0.0 14 - Services and applications for SMEs (e-commerce, education and training, networking, etc.).
35.7 1.3
15 - Other measures for improving access and efficient use of ICT by SMEs 4.1 0.1 24 - Cycle tracks 0.0 0.0 40 - Renewable energy: solar 1.7 0.1 41 - Renewable energy: biomass 3.3 0.1 42 - Renewable energy: hydroelectric, geothermal and other 1.1 0.0 43 - Energy efficiency, co-generation, energy management 9.3 0.3 48 - Integrated prevention and pollution control 0.0 0.0 50 - Rehabilitation of industrial sites and contaminated land 0.0 0.0 51 - Promotion of biodiversity and nature protecion (including Natura 2000) 2.1 0.1 53 - Riks prevention (including the drafting and implementation of plans and measures to prevent and manage natural and technological risks)
0.1 0.0
54 - Other measures to preserve the environment and prevent risks 4.1 0.1 55 - Promotion of natural assets 42.6 1.5 56 - Protection and development of natural heritage 8.2 0.3 57 - Other assistance to improve tourist services 781.4 27.8 58 - Protection and preservation of the cultural heritage 49.4 1.8 59 - Development of cultural infrastructure 5.4 0.2 60 - Other assistance to improve cultural services 1.7 0.1 61 - Integrated projects for urban and rural regeneration 11.7 0.4 68 - Support for self-employment and business start-up 2.3 0.1 71 - Pathways to integration and re-entry into employment for disadvantaged people; combating discrimination in accessing and progressing in the labour market and promoting acceptance of diversity at the workplace
0.4 0.0
72 - Design, introduction and implementation of reforms in education and training systems in order to develop employbility, improving the labour market relevance of initial and vocational education and training
1.8 0.1
77 - Childcare infrastructure 0.1 0.0 79 - Other social infrastructure 3.0 0.1 81 - Mechanisms for improving good policy and programme design, monitoring and evaluation at national, regional and local level, capacity building in the delivery of policies and programmes.
0.1 0.0
83 - Specific action addressed to compensate additional costs due to size market factors 5.1 0.2 85 - Preparation, implementation, monitoring and inspection 0.0 0.0 86 - Evaluation and studies; information and communication 0.0 0.0 Total 2813.4 100.0
Total 888.8 31.6 Source: calculation on DG Regio – ERDF AIR rawdata (Project selection), 2007-2013
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 113
Table A6 – List of relevant indicators related to culture and tourism
OP Unit of
measurement Indicator name
2007AT162PO003 € Investment induced in the field of the improvement of touristic services
2007BE161PO001 enterprises Number of enterprises participating to a touristic network
touristic networks Number of touristic networks initiated or supported
2007BE162PO002 Realisations Number of realisations in improving the touristic and (socio-)cultural of (less know) city areas)
2007BE162PO003
enterprises Number of enterprises participating to a touristic network
networks Number of touristic networks initiated or supported
touristic networks Number of touristic networks initiated or supported
2007BG161PO001
persons Annual number of participants (organisations, companies) in international, national and regional tourism fairs and exhibitions
facilities Cultural facilities improved
% Bed occupancy rate
Satisfaction of visitors with attractions and information services
visitors Additional annual number of visitors of attractions supported
€ m Net annual revenues from international tourism
enterprises Number of enterprises involved in promotion projects in Bulgaria
Number of enterprises involved in promotion projects abroad
Events Number of promotional events to promote conformity assessment, certification and product quality
km Kilometres constructed / rehabilitated gas connection with relevant borudvane
projects Number of investment projects
Number of completed investment projects in the sectors targeted
services Number of new or improved business services provided by organizations NQI
start-ups Number of start-ups backed by financial products
centres Number of supported / created centres for technology transfer, technology incubators, technology centres, technology parks and other pro-innovative firms
Incubators Number of supported regional business incubators created / updated
% Survival of innovative start-ups
visitors Increase the number of people using the website of the ASME
companies Number of supported companies that use specialized consulting services
innovations Number of innovations introduced / ready to be brought to market
applications Number of applications from commercial to trade marks, designs, patents, etc. by supported enterprises and research organization
certificates Number of certificates introduced in supported enterprises
installations Number of introduced CHP installations in supported enterprises
compressor stations
Number of purchased and installed compressor stations
inquiries Number of inquiries from potential investors in targeted sectors
labs Number of supported labs
2007CB163PO003 projects Number of common cultural actions
person No visitor of cultural events
2007CB163PO004 number Elaborated concepts (e.g. tourism)
Established tourism infrastructure
2007CB163PO009 projects Education and general learning, science, research, culture, health, social, civil protection and disaster prevention - Support of culture
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 114
OP Unit of
measurement Indicator name
2007CB163PO017 projects Improvement of the cooperation in the field of art and culture
2007CB163PO017 projects Improvement of the cooperation in the field of art and culture
m Length of the built or maintained touristic roads
2007CB163PO025 Products Number of tourism products / services
buildings Number of new / reconstructed tourism facilities
2007CB163PO027 solutions RI 7: No. of new or improved solutions for sustainable management of cultural heritage
2007CB163PO028 cooperations
Cooperation and knowledge sharing among linguistic minorities
Cooperation in order to create the conditions for an environmentally sound use of nature as a resource for processing and tourism
Cooperation in order to develop activities based on the cultural values that exist in communities in the archipelago , coastal and mountain areas and World Heritage sites
Cooperation in order to preserve and develop the cultural history as a resource for regional growth
Creation of communities in areas such as culture and recreation for young people , students and newcomers
Cooperation between regional newspapers , television , Internet and radio media which help to increase the knowledge of neighbours
2007CB163PO031
Products
No of ecotourism products/services developed/ improved due to supported operations
No of developed joint tourism products
Places No of places for incoming tourists created on both sides of the border due to supported operations
objects No of developed/ renewed cultural/historical/tourism infrastructure and objects
2007CB163PO033 networks
Networks of harbours and touristic services
Networks of wildlife and marine parks
Number of networks (training, cultural, athletical social and touristic) created in cooperation areas
2007CB163PO034 towns Number of towns covered by touristic itineraries
itineraries Number of touristic itineraries
2007CB163PO035 persons
Number of persons buying touristic packages
Number of participants to cultural events
packages Number of thematic touristic packages
2007CB163PO036
networks Number of network in tourism
persons Number of participants in cultural events
infrastructures Number of cultural infrastructures used jointly
2007CB163PO037 countries / tour operators
Number of countries / tour operators achieved by the actions of territorial marketing and promotion of culture and tourism.
2007CB163PO039 number Number of new tourism offerings created
2007CB163PO056 products and service offers
Number of new intercultural offers
2007CB163PO058 Actions Number of actions of management, utilization and promotion of cultural resources
2007CB163PO059 co operations Establishment of joint co operations for the promotion of cultural assets
2007CB163PO060 Events Number of joint events and cooperation in the fields of tourism, culture and natural heritage promotion
2007CB163PO068 projects Tourism cooperation (Number of jointly developed tourist attractions)
2007CY16UPO001 beds Number of beds upgraded in tourist establishments
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 115
OP Unit of
measurement Indicator name
2007CZ161PO001
beds Number of newly created or reconstructed beds total
projects
Number of created marketing and promotion products for tourism
Number of supported cultural events
Number of products created for navigation and routing of visitors
institutions Number of newly certified facilities in tourism
objects Number of new and technically improved objects of tourism infrastructure
Number of renovated historical monuments
2007CZ161PO002
projects Number of created marketing and promotion products for tourism
% Share of overnight stays of guests in hotel facilities of the region on the total number of stays in the Czech Republic except for Prague)
2007CZ161PO005 beds Number of newly created or reconstructed beds total
objects Number of renovated historical monuments
2007CZ161PO008 projects Number of newly created or reconstructed beds total
2007CZ161PO009 beds Number of newly created or reconstructed beds total
objects Number of renovated historical monuments
2007CZ161PO010 beds Number of newly created or reconstructed beds total
overnight stays Visitors of the region – number of overnight stays
2007CZ161PO013
beds Number of newly created or reconstructed beds total
projects Number of renovated historical monuments
Number of products created for navigation and routing of visitors
visitors Visitors of the region – number of overnight stays
2007CZ162PO001 % Use of regenerated or recultivated area - increase in number of visitors
objects Number of renovated historical monuments
2007CZ16UPO002
Products Number of created publicity or marketing products for tourism
facilities Number of newly built or modernised cultural facilities
%
Increase in the share of tourists in the number of visitors to the CR
Increase in the number of visitors to monuments and cultural facilities
Share of entities operating in tourism in the CR, which will be connected to the information system
monuments Number of regenerated immovable cultural monuments
methodologies Number of created methodologies in the cultural heritage area
2007DE161PO004 projects Rehabilitated adits
m Functioning adits
2007DE161PO006
jobs Jobs safeguarded due to supported film and media projects
visitors Number of visitors in supported facilities
institutions Number of visitors in supported cultural institutions
Upgraded cultural institutions
infrastructures Number of newly established or upgraded infrastructures (edutainment, cycle paths, bridleways etc.)
2007DE161PO007 ha Newly created area: Trades, industries, tourism, rehabilitated area (in ha)
2007DE162PO001 m2 Area of new museums
2007DE162PO002 visitors Number of visitors’ arrivals (arrivals of day tourism)
overnight stays Number of overnights
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 116
OP Unit of
measurement Indicator name
2007DE162PO003
persons Thereof involved in culture
services Number of new developed cultural economy products and services
visitors Increase of visiting numbers in funded institutions (here absolute visitors numbers)
2007DE162PO005 persons Number of visitors / users p.a. in 1000 (tourism infrastructure)
2007DE162PO009
centres Modernised cultural centres in neighbourhood districts
% relative increase of people using cultural centres in neighbourhood districts
partnerships Initiated development partnership for urban development through culture
2007DE162PO010 institutions Number of modernised culture institutions
overnight stays Development of the overnight stays (tourism)
2007EE161PO001 persons Increase in overnight visitors in accommodation establishments
% Decrease in seasonality
2007EE161PO002 Sites Number of visitor sites created or qualitatively improved
2007ES161PO004 number Number of plans and dissemination campaigns for touristic promotion developed
2007FR161PO002
% % Of tourists other than metropolitan
Increased number of nights
number Number of licenses of brand quality in tourism
number of promotional campaign
Number of Guadeloupe promotional campaign oriented towards new targets
Number of rooms Capacity accuil created or furnished
2007FR161PO003 Sites Number of cultural and heritage sites restored
nights Increase the number of hotel nights
2007FR161PO004
rooms Additional host capacity of hotels
€ m
Amounts of investment (tourism and tic)
Amount of investment projects related to facilities and equipment for tourism
2007FR162PO003 number Number of host structure created
persons Number of visitors participate in events organized thanks to the program
2007FR162PO005 number Virtual tourist routes
2007FR162PO006 M nights Numbers of nights hotels
2007FR162PO009 centres Number of tourist centres created
2007FR162PO010 number Number of beds created
Number of requalified beds
2007FR162PO014 number Number of nights hotels
nights Number of nights spent in hotels
2007FR162PO023 number
Number of shelters in which were improved the quality of reception and comfort, the working conditions of the keepers, work up to standards and improving environmental quality and access
€ m Amount of investment in shelters summer / winter
2007FR162PO024 m Number of annual visitors paying a right to enter on large sites
2007GR161PO001
beds Number of new beds in tourist establishments
% Percentage of beds upgraded in tourist establishments
heritage attractions
Number of tourism - cultural infrastructures supported
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 117
OP Unit of
measurement Indicator name
2007GR161PO006 number
Number of museums - monuments rehabilitated
Tourist beds modernised
Investment projects on alternative forms of tourism
2007GR161PO007 beds Number of tourist beds modernised
monuments Number of museums - monuments rehabilitated
2007GR161PO008 beds
Number of tourist beds modernised
Number of new tourist beds
number Number of museums - monuments rehabilitated
2007GR16UPO001
% Tourist beds modernised
Contemporary culture buildings
Contemporary culture buildings
infrastructure Number of cultural tourism infrastructure supported
tourism promotion programmes
Tourism promotion programmes
monuments/archaeological sites
Monuments / archaeological sites rehabilitated
2007GR16UPO002 number Tourist beds modernised
Monuments / archaeological sites rehabilitated
2007HU161PO004 person Number of guest nights at commercial/business accommodations (per 1000 persons) (nights)
2007HU161PO005
persons Average number of visitors at the supported tourism attraction (persons)
nights Number of guest nights at commercial/business accommodations (per 1000 persons) (nights)
2007HU161PO006
persons Average number of visitors at the supported tourism attraction (persons)
nights/T persons Number of guest nights at commercial/business accommodations (per 1000 persons) (nights)
2007HU162PO001
persons Average number of visitors at the supported tourism attraction (persons)
nights/1T persons Number of guest nights at commercial/business accommodations (per 1000 persons) (nights)
2007IE162PO001 visitors No. of visitors to 2 New National Park Visitor Centres
2007IT161PO001
programmes Number of international cultural programme
interventions
Number of interventions for renovation, conservation and requalification of cultural assets
Number of interventions for improving buildings used for cultural activities
Number of intangible interventions for promoting the integration of inter-regional touristic offer
2007IT161PO008
beds Number of beds in high quality accommodations of regional touristic destination
enterprises
Number of enterprises (recovery and renovation of cultural assets)
Number of enterprises (marketing for network of cultural services)
Number of enterprises (production and diffusion of cultural events)
Number of new/updated touristic enterprises
Number of enterprises (services for cultural networks)
centres Number of new cultural multifunctional centres
% Presence of foreign tourists (%)
Increase in bed in accommodations (%)
days/inhabitant Days of touristic presence (non-summer time)
2007IT161PO009 % Valorisation of cultural and historical assets
days Days of touristic presences (except for summer time)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 118
OP Unit of
measurement Indicator name
2007IT161PO010
% Increase in touristic presence in areas covered by valorisation/riqualification projects (%)
days/inhabitant Days of touristic presences (except for summer time)
Days of touristic presences in accommodations
2007IT161PO012
enterprises Number of enterprises benefiting from touristic projects
persons Number of additional touristic arrivals
Number of additional touristic arrivals (protected areas)
presences per capita
Tourist presences per capita in non-summer months
2007IT162PO001
% Recovery of touristic demand (%)
days Number of touristic presence days
buildings Number of high relevant historic, cultural and architectural buildings for cultural activities
2007IT162PO002 € m Enterprise investments in sustainable culture and tourism projects financed by OP
2007IT162PO003
beds Number of new beds
buildings Number of recovered buildings for touristic purposes
accomodations Number of accommodations that can be reserved through touristic portal
2007IT162PO004 beds Number of beds created in accommodation facilities of valorised protected areas
2007IT162PO006
persons Increasing in persons visiting rehabilited buildings and areas
assets
Number of upgraded cultural assets (religious buildings)
Number of upgraded cultural assets (historical buildings)
Number of upgraded cultural assets (mobile assets)
Number of upgraded cultural assets
2007IT162PO007 projects
Number of historical and cultural assets recovered
Number of recovered/valorised cultural areas
Number of interventions to upgrade cultural heritage
2007IT162PO008
enterprises Number of touristic enterprises benefiting from facilitations
Number of female touristic enterprises benefiting from facilitations
infrastructures Number of cultural infrastructures
€ m Investments for recreation areas and cultural infrastructures
2007IT162PO011 % Increase in number of persons visiting recovered buildings and monuments
2007IT162PO012 assets Number of recovered cultural assets
€ m Investments in sustainable tourism (millions of euro)
2007IT162PO014 persons Number of visitors in recovered and valorised area and buildings
areas Touristic areas covered by support and valorisation projects
2007IT162PO015 % Additional tourists in area covered by territorial valorisation projects (%)
2007LT161PO001
number of protected territories
Number of protected territories (national parks and reserves) where tourist centres and visual information systems are established
percentage points Increase in percentage of protected territories where conditions to visit without damaging nature are provided
2007PL161PO001 million persons Number of people benefiting from tourism products resulting from the support of the OP IE
2007PL161PO002
persons Increase in the number of visitors to museums
items
Number of restored and renovated historical buildings
Increase in the number of new / built, expanded and reconstructed cultural institutions (museums, galleries, theaters and music institutions)
€ bn Growth of foreign tourist expenditures in Poland
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 119
OP Unit of
measurement Indicator name
2007PL161PO003 items Number of tourist attractions made available
2007PL161PO005
persons Number of people visiting supported cultural institutions
items Number of cultural heritage sites made accessible to tourists
Number of supported cultural institutions
2007PL161PO006
persons Number of tourists using produced / modernized tourist products
items Number of supported enterprises in the field of tourism
units Number of culture projects
2007PL161PO007
persons Number of tourists
items
Number of accommodation beds resulting from the implementation of projects
Number of constructed / modernized objects of culture
Number of jobs created in culture
New types of services / products / attractions offered as a result of implementation of projects
2007PL161PO008 persons
Number of people using the supported infrastructure of culture and tourism - foreign citizens thereof
Number of people using the supported infrastructure of culture and tourism
M PLN Revenue from tourism infrastructure under the support of the program
2007PL161PO009 items Number of culture projects
Number of jobs created in the area of culture
2007PL161PO010
persons Number of visitors to Malopolska Region
persons/year Number of tourists visiting Krakow
items
Number of new, modernized, equipped objects of culture and heritage
Number of supported objects in the field of cultural infrastructure in Krakow Metropolitan Area
2007PL161PO011 items Number of new, modernized facilities of culture and heritage
Total number of directly created new jobs (FTE) in the culture
2007PL161PO012 items Number of supported institutions of culture
2007PL161PO013 items Number of supported cultural institutions
Number of cultural heritage buildings opened to tourism
2007PL161PO014
persons Number of people using tourist attractions arising from the implementation of the program
items
Number of infrastructure projects in the field of culture and protection of cultural heritage
Number of new or modernized objects of cultural infrastructure
2007PL161PO015
persons/year Number of people benefiting from the tourist information system as a result of projects
items Number of created / developed tourism products
Number of education, sports and cultural projects
2007PL161PO016
persons Number of people benefiting from the tourist information system
items
Number of culture projects
Number of new / modernized cultural buildings
Number of new / modernized tourist infrastructure
2007PL161PO017 items Number of constructed / modernized facilities of culture and heritage
2007PL161PO018 items Number of built / modernized cultural objects
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 120
OP Unit of
measurement Indicator name
2007PL161PO019
items
Number of buildings of culture undergoing modernization
Number of new program offers in terms of culture
Number of program officers promoted for tourism
€ m Income from tourism infrastructure covered by the support of the program
T persons
Number of people benefiting from the support of tourism infrastructure covered under the program
Number of people using cultural infrastructure covered by the support of the program
2007PL161PO020 items Number of newly created / upgraded cultural objects
Number of newly created / upgraded tourist facilities (hotels - catering)
2007PT161PO002
Events Supported events for region internationalization
persons
Participants in supported actions for qualification and promotion of touristic resources integrated into PDTVD
Participants in supported actions for qualification and promotion of touristic resources of Oporto/North Region
Visitors of supported events
Participants in supported actions for qualification and promotion of touristic resources integrated into regional development plans
Visitors and users of supported infrastructures and collective equipment to support the economic valorisation of marine resources
Covered population by cultural equipment
Actions
Supported actions for qualification and promotion of touristic resources integrated into PDTVD
Supported actions for qualification and promotion of touristic resources of Oporto/North Region
Supported actions for qualification and promotion of touristic resources integrated into regional development plans
equipments
Cultural equipment (public libraries, public archives, theatres, cinetheatres, digital cinema, contemporary art centres)
Cultural equipment
2007PT161PO003 persons
Covered population by cultural equipment (public libraries, public archives, theatres, cinetheatres, digital cinema, contemporary art centres)
Actions Interventions for monuments valorisation and rehabilitation
2007PT161PO004
persons Covered population by cultural equipment (public libraries, public archives, theatres, cinetheatres, digital cinema, contemporary art centres)
equipments Cultural equipment (public libraries, public archives, theatres, cinetheatres, digital cinema, contemporary art centres)
2007PT161PO006
Visitors (% of growth)
Additional visitors of cultural spaces
€ m Hospitality income
2007RO161PO001
nights Overnights-staying within the accommodation structures
Tourists Tourists visiting the Information and Promotion Centres
Tourists visiting the rehabilitated/equipped accommodation structures
campaigns Promotional campaigns for advertising the tourism brand
informtion centers
National Tourism Information and Promotion Centres supported
2007SI161PO001
projects Number of renovated cultural heritage and public cultural heritage infrastructure facilities
%
Increase in value added per employee in companies receiving financial support (at least 24 months after the project completion at the end of financial year compared to the 31. December of the year of the project start)
overnight stays Number of tourist overnight stays (in millions)
tourist beds Increase in tourist accommodation capacities
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 121
OP Unit of
measurement Indicator name
2007SK161PO001
services Number of online services available in culture
% Share of users of electronic services in culture on the total number of Internet population
2007SK161PO003
services
Number of new and improved services provided in supported immovable cultural monuments
Number of new and improved services provided in tourism
Number of new and improved services provided in supported facilities of cultural infrastructure
Number of new and improved services provided in supported immovable facilities of cultural infrastructure
facilities Number of supported facilities of cultural infrastructure
monuments
Number of revitalized immovable cultural monuments in order to preserve the cultural heritage or their utilization in the cultural-cognitive tourism
Number of revitalized immovable cultural monuments
Source: WP0
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 122
Annex III Draft Questionnaire for Managing Authority
SECTION A. GENERAL INFORMATIONGENERAL INFORMATION
A.1 Operational Programme List of Operational Programmes
A.2 Name of Respondent(s)
A.3 Position Programme Manager
Responsible for the implementation of
a specific priority axis
Responsible for the implementation of
a specific measure
Other, please specify
SECTION B. STRATEGIC UNDERPINNINGS OF CULTURE AND TOURISM
INVESTMENTS
B.1. Please indicate if there is a specific strategy for CULTURE in your OP, if
this strategy is inspired by regional/national strategies on CULTURE or if
other rationales apply (indicate in this case how investment priorities are
decided and funds allocated)
Specific strategy on CULTURE set in the programme document (please
summarise goals and expected changes):
____________________________________________________________________
___________________________________________________________________
Regional /national general strategies on CULTURE outside the programme
document (please summarise main goals and expected changes):
____________________________________________________________________
___________________________________________________________________
Other (please indicate how investment priorities are decided and funds
allocated if there is no ERDF specific or regional/national strategy on
CULTURE):
____________________________________________________________________
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 123
B.2. What is the rationale for investing in CULTURE in the strategy?
Rationale is based on identified market failure(s) (e.g. lack of private
operators, small dimensions/fragmentation of the market, lack of
entrepreneurial capacities, etc.)
____________________________________________________________________
___________________________________________________________________
Rationale is based on financing needs of the sector:
____________________________________________________________________
___________________________________________________________________
Other (please specify):
____________________________________________________________________
___________________________________________________________________
B.3. Please indicate if there is a specific strategy for TOURISM in your OP, if
this strategy is inspired by regional/national strategies on TOURISM or if
other rationales apply (indicate in this case how investment priorities are
decided and funds allocated)
Specific strategy on TOURISM set in the programme document (please
summarise goals and expected changes):
____________________________________________________________________
____________________________________________________________________
Regional /national general strategies on TOURISM outside the programme
document (please summarise main goals and expected changes):
____________________________________________________________________
____________________________________________________________________
Other (please indicate how investment priorities are decided and funds
allocated if there is no ERDF specific or regional/national strategy on
TOURISM):
____________________________________________________________________
____________________________________________________________________
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 124
B.4. What is the rationale for investing in TOURISM in your ERDF Operational
Programme?
Rationale is based on identified market failure(s) (e.g. lack of private
operators, small dimensions/fragmentation of the market, lack of
entrepreneurial capacities, etc.)
____________________________________________________________________
____________________________________________________________________
Rationale is based on financing needs of the sector:
____________________________________________________________________
____________________________________________________________________
Other (please specify):
____________________________________________________________________
____________________________________________________________________
B.5. To what extent are strategies on CULTURE and on TOURISM integrated?
Please, provide a brief description.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
B.6. To what extent have strategies on CULTURE and on TOURISM been
integrated with other strategies (e.g. with strategies on competitiveness of
SMEs, on innovation etc.)? Please, provide a brief description.
CULTURE
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 125
TOURISM
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
B.7. In the ERDF Operational Programme of your country/region: (more than
one response is possible)
There is a specific priority axis for CULTURE
There is a specific priority axis for TOURISM
Culture and tourism are addressed together in a specific priority axis
There are specific measures for Tourism/Culture within a broader priority axis
(please specify which
one)..............................................................................................
Tourism/Culture are indirectly addressed in the framework of integrated
projects for urban and regeneration
Tourism/Culture are indirectly addressed under other headings (e.g. research
and innovation, promotion of small and medium-sized enterprises, information
society, transport, energy, human capital, etc.). If so, please provide examples:
__________________________________________________________________
___
A mix of the above
Other (please specify)
B.8. Which are the SPECIFIC OBJECTIVES for CULTURE directly or indirectly
addressed by the ERDF Operational Programme of your country/region?
(more than one response is possible)
Support the development and networking of cultural resources and activities
Encourage the development of entrepreneurship in the cultural sector (e.g.
creative industries)
Develop the culture sector (including creative industries) to diversify the
economic structure of declining industrial zones and rural areas
Support the culture sector (including culture industries) to foster innovation and
spill-over effects to other industries
Support culture sector (including creative industries) to improve urban
regeneration and social cohesion
Diversify the cultural offer and support the production/promotion of new culture
services
Other, please specify:
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 126
B.9. Which are the SPECIFIC OBJECTIVES for TOURISM directly or indirectly
addressed by the ERDF Operational Programme of your country/region?
(more than one response is possible)
Support the development of sustainable economic activities in natural heritage
areas
Encourage the development of entrepreneurship in the tourism sector
Develop the tourism sector to diversify the economic structure of declining
industrial zones and rural areas
Diversify the tourism offer and create targeted tourism products/packages
Support the upgrading of hotels and other accommodation facilities
Improve the tourist attractiveness of the region and enhance tourism
promotion, also through the use of ICT
Support the development of territorial marketing actions
Other, please specify:
B.10. Have targets been set for each specific objective of the ERDF
investments supported by the Operational Programme of your country/region
and is progress on achievements systematically monitored?
Yes, but only for some of them
Yes, for all of them
No
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 127
SECTION C. NATURE OF ACTIVITIES
C.1 Considering the following typologies of activities, please identify those
listed in the ERDF Operational Programme of your country/region and
provide an indication of the financial allocation and the number of projects
implemented.
CULTURE
Financial
allocation
in 2007
Financial
allocation
up to
June 2014
Expenditure
up to June
2014
Number of
projects to June
2014
EUR EUR EUR Total Of which
completed
Refurbishment/recovery and
protection of historical
monuments, buildings or
archeological sites
Construction/extension/recovery
of cultural infrastructure (e.g.
museum, libraries, archives,
etc.) or of infrastructure
providing cultural or sporting
services (e.g. theatre, concert
hall, opera house, auditorium,
arty gallery, planetarium,
stadium, etc.)
Organisation of cultural events
(e.g. painting, sculpture,
photography, etc.) or performing
arts (e.g. film production, book
publishing, etc.)
Support to private initiatives in
creative industries (media and
information technology, cultural
operators), of which:
Grants
Loan, interest subsidy,
guarantees
Venture capital
Financial engineering
instruments
Non financial support
Other, please specify:
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 128
TOURISM
Financial
allocation
in 2007
Financial
allocation
to June
2014
Expenditure
to June 2014
Number of
projects to June
2014
EUR EUR EUR Total
Of which
completed
Investments to improve the
safety and protection of natural
assets
Physical investments for the
promotion and development of
the tourism sector (e.g.
information centres, etc.) or
recreational touristic
infrastructures and services
Territorial marketing activities
(e.g. promotional activities,
networking, conferences and
trade fairs)
Promotion of sports and
recreational activities and events
Support to private initiatives
(e.g. sports, hotel and
restaurants) in the tourism
sector, of which:
Grants
Loan, interest subsidy,
guarantees
Venture capital
Financial engineering
instruments
Non-financial support
Integrated tourism initiatives
Other, please specify:
C.2. On the basis of the typologies of activities identified under question C1,
please describe the most relevant initiative supported by the ERDF
Operational Programme of your country/region to strengthen the
development of culture and tourism and describe the allocated and expected
expenditure and the actual (or expected) results.
CULTURE
_____________________________________________________________________
_____________________________________________________________________
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 129
_____________________________________________________________________
TOURISM
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
C.3. Are there projects financed in other policy areas by the ERDF Operational
Programme of your country/region with an indirect impact on culture and
tourism. Please provide a brief description of the main ones.
CULTURE
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
TOURISM
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
C.4 In the case of business support to private operators in the culture and
tourism sectors (e.g. hotel and restaurants or support schemes for culture
industries), are support measures targeted exclusively to tourism /culture
operators or do these operators compete for the same support measures
applied to operators in other sectors?
Only for tourism/culture operators
Generally for all operators
Mixed
C.5 In addition to your ERDF Operational Programme, have funding
opportunities for culture and tourism sectors been available also under other
Cohesion Policy’s funds (e.g. EAFRD) and/or national/regional programmes
in your country/region?
Yes
No
I don’t know
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 130
C.5.1 If yes, could you please specify which activities have been supported?
(more than one response is possible)
Cultural and leisure activities
Renovation and development of rural villages
Protection and conservation of the rural heritage
Financial support to SMEs operating in the tourism/cultural sector
Other, please specify:
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 131
SECTION D. ACHIEVEMENTS OF ERDF INTERVENTIONS IN CULTURE AND
TOURISM
D.1. For each of the following typologies of interventions, please specify the
most relevant achievement indicators (distinguishing between output and
result) adopted in your monitoring system (maximum three), provide a
quantitative measure of the achievements to 2013 and describe any other
additional evidence available (qualitative or quantitative).
CULTURE Output/
Result
Enter the
name of
the
relevant
indicator
Target
(if any)
Achievement
s to June
2014
Additional
evidence
Refurbishment/recovery and
protection of historical
monuments, buildings or
archeological sites
1)
2)
3)
1)
2)
3)
1)
2)
3)
Construction/extension/recov
ery of cultural infrastructure
(e.g. museum, libraries,
archives, etc.) or of
infrastructure providing
cultural services (e.g.
theatre, concert hall, opera
house, auditorium, arty
gallery, planetarium, etc.)
1)
2)
3)
1)
2)
3)
1)
2)
3)
Organisation of cultural
events (e.g. painting,
sculpture, photography, etc.)
or performing arts (e.g. film
production, book publishing,
etc.)
1)
2)
3)
1)
2)
3)
1)
2)
3)
Support to creative industries
(sports, media and
information technology)
1)
2)
3)
1)
2)
3)
1)
2)
3)
Other, please specify:
1)
2)
3)
1)
2)
3)
1)
2)
3)
TOURISM Output/
Result
Enter the
name of
the
relevant
indicator
Target
(if any)
Achievement
s to June
2014
Additional
evidence
Investments to improve the
safety and protection of
natural assets
1)
2)
3)
1)
2)
3)
1)
2)
3)
Physical investments for the
promotion and development
of the tourism sector (e.g.
information centres, etc.)
1)
2)
3)
1)
2)
3)
1)
2)
3)
Territorial marketing 1) 1) 1)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 132
activities (e.g. promotional
activities, networking,
conferences and trade fairs)
2)
3)
2)
3)
2)
3)
Promotion of sports and
recreational activities and
events
1)
2)
3)
1)
2)
3)
1)
2)
3)
Support to private initiatives
(e.g. hotel and restaurants)
in the tourism sector
1)
2)
3)
1)
2)
3)
1)
2)
3)
Other, please specify:
1)
2)
3)
1)
2)
3)
1)
2)
3)
D.2 Do indicators listed under question D1 appropriately describe the
achievements of culture and tourism interventions? Please identify the
strengths and weaknesses of the indicators used.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 133
SECTION E. TYPOLOGIES OF BENEFICIARIES
SECTION E. TYPOLOGIES OF BENEFICIARIES
E.1 Who are the main beneficiaries of the interventions implemented? (more
than one responses are possible)
Individ
ual
Large Enterprises
Individual
SMEs
Consortium/
networks of
private operator
s
Individual
public bodie
s
Consortium/partnershi
ps of public bodies
NGOs
Cons
ortium of public and
private
opera
tors
Other
(specify)
CULTURE
Refurbishment/recovery and
protection of historical monuments, buildings or archeological sites
Construction/extension/recovery of cultural infrastructure (e.g. museum, libraries,
archives, etc.) or of infrastructure providing cultural
services (e.g. theatre, concert hall, opera house, auditorium, arty
gallery, planetarium, etc.)
Organisation of cultural events (e.g. painting, sculpture,
photography, etc.) or performing arts (e.g. film production, book publishing, etc.)
Support to
creative industries (sports, media and information technology)
Other (please specify..)
TOURISM
Investments to
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 134
improve the safety and
protection of natural assets
Physical investments for the promotion and development
of the tourism sector (e.g. information centres, etc.)
Territorial
marketing activities (e.g. promotional
activities, networking, conferences and trade fairs)
Promotion of sports and recreational activities and events
Support to private initiatives (e.g. hotel and restaurants) in the tourism sector
Other (please specify..)
E.2 How are beneficiaries selected? (more than one responses are possible)
Open
competitive
procedure
Negotiated
procedure
Direct
appointment
Other
(please
specify)
Individual Large Enterprises
Individual SMEs
Consortium/ networks of private
operators
Individual public bodies
Consortium/partnerships of public
bodies
NGOs
Consortium of public and private
operators
Other (specify)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 135
E.3 What are the most common selection criteria for beneficiaries?
Never Sometimes Often Always
Employment expected impact
Technical quality
Financial robustness and
sustainability
Capacity to cooperate/build
networks
Innovativeness
Previous experience in the field
Other (specify)
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 136
***
SECTION F. EXAMPLES OF PROJECTS
F.1 Please indicate one interesting example of project co-financed by the
ERDF programme in your region/country in tourism/culture, from which
lessons can be drawn on the effective and timely delivery:
Title
Brief Description
Contact Person
(Please, provide name and e-mail
address)
Link to website, if any
F.2. Please indicate one interesting example of project co-financed by the
ERDF programmes in your region/country in tourism/culture, from which
lessons can be drawn on typical challenges and problems encountered:
Title
Brief Description
Contact Person
(Please, provide name and e-mail
address)
Link to website, if any
***
The European Commission is interested to know what is the effectiveness of
the ERDF/Cohesion Fund interventions from the point of view of actors
involved in the implementation of the Programme. Please provide us with
contact details of at least three implementing bodies of the most significant
interventions in the field of culture and tourism:
1 Contact name:
E-mail address:
2 Contact name:
E-mail address:
3 Contact name:
E-mail address:
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 137
Annex IV Pilot case study proposal –Puglia ERDF ROP - Motivation
Puglia ERDF ROP is an relevant and insightful regional case study given its strong effort in
investing in tourism, culture and creativity in recent years. The project provides insights because of both the governance process used and the global touristic and cultural strategy implemented. The present governance structure for tourism in Apulia is the result of a complex reform
process which redesigned the organisation of regional tourism by setting up Local Tourism Systems (LTSs) (as provided for by the Regulations of the R.L. 11 February 2002 no. 1, and by the Regulation n. 19/2011,– concerning the reorganisation of the regional tourism system). New regulations dissolved the existing five provincial TPAs (Touristic Promotion Agencies) and established the RTA (Regional Tourism Agency) which took over their tasks and duties. The overall regional strategy stated its objectives as: - to develop a network of integrated tourist micro - systems focused on different regional
attractions. This approach aims to reduce the gap between the coastal and internal areas and so to balance socio - economic opportunities at a regional level.
- to support the tourist regional offer, reinforce tourist demand and improve regional infrastructures.
- the adoption of different participatory models to elaborate the overall regional strategy of tourist development by reinforcing democratic and community accountability.
- to rationalise the expenditure of the limited national and European funds at disposal. At the core of Apulia tourism policy are a series of actions aiming to develop local tourist systems which can diversify the tourist offer and improve the regional attractiveness through the integration between tourism and natural and cultural resources. This integration corresponds to the specific vocations and needs of the 10 – “area vasta”, (broad area) who are sub-regional aggregations of municipalities which represent part of the regional territory and
which are experimenting with an innovative process of multilevel governance leading to the definition of strategic plans per each “area vasta”. Therefore the strategy is elaborated at local level through several processes of sub-regional strategic planning enhancing the territorial stakeholders and democratic participation and contributing to the empowerment of local authorities to achieve greater territorial cohesion. The model of tourism development assumed in the Apulia region is thus focused on the promotion of specific local resources of the territory.
The imperative is therefore to accentuate and enhance the specificity of Apulia, both in terms of
natural and cultural resources and accomodation (including local variations such as trulli, masserizie, etc.), aiming to encourage a highly experiential tourism, concentrated on land resources. From the perspective of the culture sector, the Apulia global strategy included a broad and integrated policy supporting the production chain of the creative industries, by
implementing several programs and regulations aiming to ensure its presence within institutions that have, as their mission, an activity in the creative industries sector. The Apulia Region has become the main partner within the Teatro pubblico pugliese, a public circuit for prose and dance, operating in the whole Region in association with a large number of Apulian municipalities and provinces. Also, the Apulia Film Commission has been established, to promote the area and attract film production, as well as promoting video content, audiovisual content and film across the region.
Within this context, the ROP has a dedicated Axis (Axis 4) specifically addressing the “Enhancement of natural and cultural resources for attraction and development” which aims to
make the region more attractive to tourists with environmental policies, and actions directed at goods, and cultural events and the tourism offer. With specific reference to regional tourism policy, Axis IV is focused on (i) promoting the
tourism economy by qualification, diversification and the promotion of an integrated tourism offer in the territorial structure, the completion and qualification of the infrastructure and territorial marketing actions, with specific actions relating to the completion of infrastructure supporting the tourism economy (especially tourist ports); (ii) developing territorial marketing and promotion, with the aim of taking Apulian tourist destinations to national and international markets; and finally (iii) creating hospitality structures and leisure facilities for social tourism, in
Ex post evaluation of Cohesion Policy programmes 2007-2013, focusing on the European Regional
Development Fund (ERDF) and Cohesion Fund (CF) – Work Package nine
November 2014 138
rural and beach contexts as well as in the old town centres. With reference to culture, the Axis covers both (i) maintaining cultural heritage and improving
its accessibility (see for example, Intramoenia extra art – Castelli di Puglia) and (ii) the promotion of cultural and creative initiatives (see for example, the initiatives Teatri abitati which assigns the management of small local theatres to small companies and artistic groups. It has involved, to date, 12 theatres and 185 operators). At the present time, the Axis has a good level of implementation: the number of implemented projects are higher than the target objective; the financial commitment amounts
to 93% of the dedicated resources and the level of payment is 63% of the committed resources. Actions implemented in the tourism sector record a positive performance reaching 83% of the target value. The actions undertaken and interventions have contributed to increase the number of tourists visiting Puglia. Several of the integrated projects (among which mini case studies can be selected) are already implemented, and these affect the entire Region and cover all its tourism and cultural specificities.
Contact persons: Palumbo Francesco, Responsible for Axis 4 Tel. 0039/080 5405615 Email: [email protected]