The fork in the road for innovation measurement: which way should we

go?Jonathan Haskel

Imperial College Business School, Imperial College London and CEPR, UKIRC

j.haskel@ic.ac.uk

National Academy of Sciences, Washington DC, 11th July 2011Slides plus spares, links to papers etc. in spares section

1

The exam questions(to be answered in final slide…)

• The objective of this session is to identify – recent developments in measuring STI and – what is currently planned for the future.

• Discussion should reveal – what has been successfully and unsuccessfully measured. – What are critical bottlenecks and perceived

opportunities?– What global STI metrics and indicators should NCSES

develop in the near and medium term (the next 5-10 years)?

2

The fork in the road for innovation measurement

• Approach 1: EU-type innovation surveys– Tried for 15+ years in EU, Applied in latest US work– Key questions

• “have you process/product innovated?”• Spending on innovation e.g. software, marketing, training• Potpourri of other questions

– Information for innovation (e.g. importance of information from universities high/med/low)– What stopped you from innovating?

• Approach 2: extended R&D/spending type surveys– Follows Griliches/Jorgenson/CHS/Nat Accounts to measure knowledge capital– Implementation: software and (forthcoming) R&D spend– CHS agenda: need coinvestments with R&D e.g. design, training, marketing,

business process• Current method: potpourri of surveys• UK: intangibles investment survey, similar style to R&D survey• Kauffman

3

What theory do we have to guide measurement?

• Innovation as TFP i.e. output net of tangible K and labour– Logically consistent, suggests measurement (e.g. depreciation,

investment)– Appropriate if all knowledge from free spillovers

• Corrado, Hulten, Sichel – Recognises innovation is routinised in many firms e.g. organised spending

on marketing, training , design etc. – Helps understand innovation in non-R&D industries e.g. finance– Strength: CHS categories are matched to functional bodies in (large) firms

e.g. HR dept for training , marketing dept for brand etc. – Still scope for knowledge spillovers

• Measurement implications– Spending, investment, depreciation, asset and rental prices

European Innovation survey questionnaires

• UK example: product innovation

5

• Process innovation

6US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26

Spa

in

Icel

and

(200

2-04

)

Irela

nd

Net

herla

nds

Aus

tralia

(200

6-07

)

Uni

ted

Kin

gdom Ita

ly

Bel

gium

Est

onia

Aus

tria

Finl

and

Ger

man

y

US

A (2

006-

8)

Cze

ch R

epub

lic

Luxe

mbo

urg

Sou

th A

frica

(200

2...

Por

tuga

l

Nor

way

Den

mar

k

Japa

n (1

999-

2001

)

Chi

le

0

2

4

6

8

10

12

14

16

18

Weighted % of companies introducing new or significantly improved product or process: service industries%

Michael Mandel: “You can’t be an innovative economy if only 9% of your companies are innovating”

What do innovation surveys find?

What does the “have you innovated” question measure?

• Other findings– the total is procyclical: rises strongly in IT boom end of 1990s – Item non-response can be very high e.g. 70% to intangible spending

questions– Large firms show poor response rates

• So what does the question measure? Crespi , Criscuolo, Haskel (2007, Appendix 1), UK CIS3 (98-00)– 45% of firms reported technological product/process– they were asked for text describing their innovation– They are given instructions on what to include and exclude e.g.

include “robotised welding”, exclude “packaging” – Analysis of 874 firms showed

7

The “have you process innovated “ question measures new capital equipment

• Summary: – of firms who provided the information, reported process innovation is

mostly new capital spending– This is why measured “innovation” is highest in IT boom– Conjecture: we would get same/better(?) information from a

standard investment survey (+ good deflators) 8

Analysis pf 874 process innovation using firm-reported text

Type of Word Share (%)

HARDWARE word 22.54 SOFTWARE word 15.33

LEAN PROCESS word 8.92 Other (mostly capital equipment) 53.20

Total 100

An alternative approach guided by CHS

• What does this framework suggest you need to know?– Knowledge services within the firm = knowledge capital

stock and rental rates• Innovation spending on broad range of knowledge assets:

software, design etc. • Convert to investment• Convert to real stock: need depreciation and deflators

– Implication: need a spending survey. Can piggy back onto R&D survey or new one (or modify innovation survey)

– Knowledge from outside the firm• Information flows

– Patents is one source: but service sector?– CIS information flows

Possible model: Investment in Intangible Assets Survey(www.ceriba.org.uk/bin/view/CERIBA/InvestIntangAssetsSurvey)

)

Extension of UK R&D survey, Autumn 2009, through ONS: so linked to national accounts data and R&D and CIS surveySurvey follows (for large firms) typical business functions

10

Each section has a filter question which defines the asset with examples

11

Then asks purchased and own-account

12

Finally life lengths

13

14

Life lengths

• All asset life lengths are greater than one year and range between 2.75 years for training and reputation to around 4.5 years for R&D.

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

Training Software Reputation & Branding R&D Design Business ProcessImprovement

Asset Category

Ave

rage

ben

efit

lives

(yea

rs)

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

ALLUpperMidLower

My answers to the exam questions• The objective of this session is to identify

– recent developments in measuring STI • US innovation survey raises puzzles for survey design. Intangible assets framework gaining

policy traction. Some surveys moving in this direction. – what is currently planned for the future

• Review by OECD of innovation surveys. More UK intangible surveys. Kauffman survey to continue?

• Discussion should reveal – what has been successfully and unsuccessfully measured.

• yes/no innovation questions have not worked. I• Intangible spending can work if asked to business functions, own-account explained.• Business process has not worked in UK survey

– What are critical bottlenecks and perceived opportunities?• Shortages of funding for more surveys. Opportunity (a) in EU, convert Innovation surveys to

intangible spending surveys, (b) in US to convert supplement to BERDIS to intangible spending survey.

• General lesson: take best practice from each survey– What global STI metrics and indicators should NCSES develop in the near and

medium term (the next 5-10 years)?• Intangible spending, life lengths. Integrate with National Accounts work on deflators, service

sector output. 15

Spares

16

What do innovation surveys find?

17

US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26

Irela

nd

US

A (2

006-

08)

Sou

th A

frica

(200

2...

Icel

and

(200

2-04

)

Net

herla

nds

Uni

ted

Kin

gdom

Aus

tralia

(200

6-07

)

Kor

ea (2

005-

07)

Est

onia

Italy

Finl

and

Spa

in

Bel

gium

Cze

ch R

epub

lic

Luxe

mbo

urg

Aus

tria

Nor

way

Den

mar

k

Chi

le

Por

tuga

l

Ger

man

y

Japa

n (1

999-

2001

)0

5

10

15

20

25

Weighted % companies introducing new or significantly improved product or process: mfring %

Crespi et al, 2008. What do firms write down as their most significant process

innovation?

• Replies – 53% are other, mostly including capital equipment (e.g. steel wire machine, spectrophotometer..)– 23% are hardware related (computer, CAD, PC) and software related (website, email, software, internet etc.)

• Summary: of firms who provided the information, reported process innovation it new capital spending

• Not surprising that measured “innovation” is highest in IT boom• We likely get equivalent information from a standard investment survey (and spend resources on

better deflators) 18

Analysis pf 874 process innovation using firm-reported text

Type of Word Share (%)

HARDWARE word 22.54 SOFTWARE word 15.33

LEAN PROCESS word 8.92 Other (mostly capital equipment) 53.20

Total 100

The future• Take best practice from elements of different surveys

– Innovation surveys• Don’t ask % new, % innovation• Do ask information sources

– R&D/IAS• Do ask for spending • Keep structure of questionaire allied with business function: ICT dept for software,

HR dept for training, Marketing dept for branding• Complement with

– What do firms do? Innovation data at project level ?– What do young firms do? Likely very intangible intensive. Kauffman survey. – What do consultants do, especially intangible consultants?– What do financial institutions do esp young firms, they are trying to value

innovation?– Measuring business process huge challenge

• Chance for NAS/OECD to lead as OECD did on software