Economic Impact Toolkits for Archives, Museums and
LibrariesALMA UK
1st December 2010
Oliver Allies & Jamie Buttrick
Introduction
Our Brief To analyse economic impact methodologies
for the sector Assess the pros and cons and thinking behind
each method Recommend methods for the creation of
toolkits, suitable for application across a range of organisations
Recommend an outline process for piloting these
Our Approach Extensive desk based review of economic
impact research (reviewed c.60 research documents)
Consultation with (19) key stakeholders from each representative sector to discuss feasibility of data capture
Progress session – (today) Workshop session with ALMA Working Group 1st
December
Economic Impact Definition
“Examines the economic effect of an activity on a specific geographical area”
OriginSporadic usage in 1980s (although said to have emerged in 1960s – John Galbraith)More commonplace in the 1990s linked to appraisal and evaluation in the public sector (HM Treasury Green Book first published in 1991)Widespread usage over the last 10 years linked to the need to identify “additionality” and return on investment (English Partnerships Additionality Guide 2001, Impact Evaluation Framework, BERR 2008)
Economic Impact Approaches Many different types of impact assessment.
Three techniques are most commonplace for the assessment of economic impact: Multipliers – calculating expenditure impacts and the
“multiplier effects” of these Contingent Valuation – (willingness to pay/willingness
to accept) Return on Investment – broadly combining the two
approaches above to generate a “ratio”
Number of Contingent Valuation Studies Identified Worldwide to Mid 2002 (Morris et al 2002)
0
1
2
3
4
5
6
7
8
9
YearNumber of Studies
Economic Impact Research in the Cultural Sector Analysed in this Study
0
2
4
6
8
10
12
14
16
1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
YearResearch Studies
Economic Impact Approaches
Multiplier Seeks to capture and map expenditure
related impacts from an organisation over a given area/areas Procurement spend Spend on employees Visitor related spend
Utilises Keynesian Multipliers to assess the “indirect” and “induced” effect of this spend
Economic Impact Approaches - Multiplier
Organisation
Procurement
Wages
Visitor Spend
Secondary expenditure impacts (the multiplier)
Economic Impact Approaches - Multiplier
Pros “Relatively” straightforward and resource efficient “Benchmarks” available and considered appropriate Most widely used and recognised impact approach
Cons Relatively narrow in focus – tendency to overlook
social impacts Most suited to those activities that offer an economic
return Additionality and attribution aspects can cause
confusion – particularly visitor spend
Economic Impact Approaches
Contingent Valuation More widely associated with an assessment
of the public’s “willingness to pay” (WTP) or “willingness to accept” (WTA)
WTP - Surveys of users and non-users of services to assess what value they would pay for those services (if they were free)
WTA – the level of compensation an individual (user and non) is willing to accept for the loss of a service/good
Popular in the United States and used by the British Library
Economic Impact Approaches – Contingent Valuation
Pros A means to capture value placed on non-marketed
goods and services An effective way of capturing “intrinsic value” Could utilise benchmarks (derived from pilot or
targeted analysis) for aggregation Cons
Demands extensive survey consultation carefully worded to avoid confusion
Is a challenging concept for the public to grasp in relation to both libraries and archives in particular
The challenge of capturing non-user value
Economic Impact Approaches
Return on Investment Offers a combination of contingent valuation
(CV) and multiplier techniques Seeks to provide a ratio of return per pound
invested (e.g. £2.60 return for every £1 invested)
Is increasingly utilised to assess government spend (particularly RDA investment)
Has driven the desire to capture economic values for all activity delivered…emergence of Social Return on Investment (SROI)
Economic Impact Approaches – Return on Investment
Pros Enables the greatest breadth of activity to be captured Is equipped (where possible) to provide values
associated with social activities (through the adoption of SROI techniques)
A focus on user value would overcome some confusion
Cons Can demand extensive resources Due to diversity of socially related activity it would not
be possible to capture social returns through a toolkit approach
Demands training and knowledge of SROI
Assessing the Impact of Socially Oriented Activities
Danger of overlooking diversity of activities delivered within the sector
Capturing social value increasingly popular over the last 5-10 years
Emerged within this sector e.g. Generic Learning Outcomes and Generic Social Outcomes
Challenge of capturing this data in an effective, robust and consistent manner… e.g. 22 different approaches identified (NEF 2005)
Social Audit and Accounting and Social Return on Investment (SROI) most prominent
Stakeholder Feedback
Broadly, greater understanding and experience of impact assessment in museums sector than in libraries/archives
In general information related to the multiplier approach was not deemed too onerous
General recognition of the need for impact assessment and willingness to participate
Consensus of the need to know what to collect, when and why
Stakeholder Feedback
Require clarity on what value they will gain from the exercise
Resource (time and personnel) main barrier to participation
Alignment to frameworks or measurements or performance measurements will boost participation
Represented a broad and extremely diverse spectrum of social activity
Implications of Findings
Clarity on what are the goals Advocacy? Benchmarking performance?
Clarity on what are the returns Fine balance of an organisation or individual’s
return on investment from participating in this research
Consistency on what messages are provided
Implications of Findings – Toolkit Potential?
Is the development of a “toolkit” and its roll-out a feasible prospect?
Does a one-size fits all approach seem possible?
What type of “toolkits” are already being used?
Toolkit Review
Few toolkits exist with a specific economic impact remit
Toolkits as guides AIM Toolkit (DC Research) Contingent Valuation Toolkit (JURA)
Toolkits as “tools” Scottish Enterprise Additionality toolkit
For effective aggregation a combined guide and toolkit is required
Needs to be simple and straightforward but robust
Toolkit Options
Format Electronic favoured by stakeholders (albeit
with scope to provide hard copy) Could enable aggregation of impact yet
provide individual organisations with data (a “hook”)
Ownership and support challenges
Toolkits – Initial Proposals
Impact Approach – Museums/Archives Multiplier methodology for assessment of
Museums and Archives is fit for purpose
Options of Approach (resource implications) Low - Hard copy guide Medium - Provide a electronic/hardcopy
guide and toolkit High - Provide online resource for
download/upload of returns
Toolkits – Initial Proposals – Museums and Archives
“Medium” resource approach (ERS preferred approach) as: Facilitates organisational impact Scope for aggregation Utilise existing data where possible to minimise
resource investment (PSQG and Gift Aid for example) Potential agreement for required primary research
Piloting Suggest testing internally prior to initial roll-out Clarity of messages and promotion Managed roll-out and testing with “early adopters”
Toolkits – Initial Proposals
Impact Approach – Libraries Multiplier – whilst possible, in danger of underplaying
scale of impact Within the United States ROI increasingly utilised –
based on UK research (Economic Value of Public Libraries in the UK – Morris et al 2002)
ROI approach in US now standardised and available as an online calculator - http://www.chelmsfordlibrary.org/library_info/calculator.html
ROI approach in the UK would incorporate elements of the Museum/Archive tool (procurement and employment) alongside user value of services – derived from a survey)
Toolkits – Initial Proposals – Libraries
Proposed Option ROI – user values combined with procurement and
employment expenditure
Approach Design survey through ALMA UK to capture range of
library services that demand a user-value Survey of xxxx library users to obtain perceived value
of services Establish Calculator tool to supplement multiplier tool
to establish return on investment.
Recommended
