106
Open (Gov) Data The What, Why, and How in 50 examples or less Guest lecture, 30 October 2014 Ton Zijlstra, @ton_zylstra

Guest Lecture Open Data

Embed Size (px)

DESCRIPTION

Guest lecture on open government data, for students of Saxion University for Applied Sciences

Citation preview

Page 1: Guest Lecture Open Data

Open (Gov) DataThe What, Why, and How in 50 examples or less

Guest lecture, 30 October 2014

Ton Zijlstra, @ton_zylstra

Page 2: Guest Lecture Open Data

Lessons from…

Enschede

Amsterdam

European Commission

World Bank

Province North-Holland

Leeuwarden

emerging patterns

Province Utrecht

Schiedam

European Space Agency

High Court of Audit

Min Interior

Min Economics

Dutch base registers

Danish base registers

Moldovan gov

Heerlen

Flemish Chancellary

Min Housing (DK)

Min Fin (DK)

ePSIplatform

LAPSI

Page 3: Guest Lecture Open Data

Husetsweb

impacts energy savings, employment, subsidy use

http://husetsweb.dk/

Page 4: Guest Lecture Open Data

hansjebrinker.com

impacts infrastructure, mobility, coastal defence, costs

http://hansjebrinker.com

Page 5: Guest Lecture Open Data

impacts sanitation, health care, costs

http://opendata.go.ke

Page 6: Guest Lecture Open Data
Page 7: Guest Lecture Open Data

&What it is

How to do

Why do it &

Page 8: Guest Lecture Open Data

What it is

Page 9: Guest Lecture Open Data

Data that is

Gathered for a public task !

Pro-actively published !

To be used by others

Page 10: Guest Lecture Open Data

No legal barriers

Everyone has equal access !

Open licensed (PD, or attribution) !

No statement of interest !

No usage restrictions

Page 11: Guest Lecture Open Data

No tech barriers

Open standards !

Findable !

In bulk and/or in pieces !

Raw and timely

Page 12: Guest Lecture Open Data

No monetary barriers

Free of charge !

Marginal costs at most

Page 13: Guest Lecture Open Data

untapped abundance

Page 14: Guest Lecture Open Data

https://secure.flickr.com/photos/opensourceway/5535034664

towards by default, by design

Page 15: Guest Lecture Open Data

Why do it

Page 16: Guest Lecture Open Data

Market and civic domain !!!!!!!!!!!!!!!

Government domain !!!!!!!!!!!!!!!

There always was value in data

Government Data

Re-user 1

Re-user 2

Open Data

2: effect of ‘Open’ !- free or not? - - ∆ demand - Ev price !

- ∆ fiscal revenue > ∆ sales revenue ?

!OXFORD 2009 POPSIS 2011

1: Market value PSI !!!!!!!!!

PIRA 2000 (€95 B) MEPSIR 2006 (€ 37 B)

3: cost of ‘Open’ <<1% of costs

Page 17: Guest Lecture Open Data

Market and civic domain !!!!!!!!!!!!!!!

Government domain !!!!!!!!!!!!!!!

Gov body 1 DataOpen Data

Government - Interaction w market/citizens changes

- Quality of data improves - Increased efficiency & effectiveness - public tasks shift (smaller, different)

Digitization has shifted that valueMarket

- Barriers to entry drop away

- market dynamics: chains change

- paradigm shift: from owning to using

Value looks different: - less lineair - not monetary - more equally spread - hard to measure - hard to correlate - but doable

Page 18: Guest Lecture Open Data

four valuable reasons

more efficient government !

better public services !

more transparent government !

new socio-economic value

Page 19: Guest Lecture Open Data

four valuable reasons

more efficient government !

better public services !

more transparent government !

new socio-economic value!

Page 20: Guest Lecture Open Data

http://www.flickr.com/photos/59937401@N07/5858059202/

DK addresses 2010: value > 70 * cost !

ETLA, SME’s geo data grow 15% faster !

Spain 2011/12, up to 600 million Euro !

POPSIS 2012, 21 cases !

Vickery / EC 2012, 2% GDP EU !

McKinsey 2014, $3 trillion+ !

„Open Data for Economic Growth” (WB, june 2014)

!!

all empirical evidence points same way

Page 21: Guest Lecture Open Data

open data in the EU

Page 22: Guest Lecture Open Data

Public, unless!

openbaar, tenzij!

openbaar, tenzij!

openbaar, tenzij!

openbaar, tenzij!

openbaar, tenzij!

openbaar, tenzij!

35 yrs FOIA, 10 yrs PSI Directive, 6 yrs open standards

Page 23: Guest Lecture Open Data

four valuable reasons

more efficient government !

better public services !

more transparent government!!

new socio-economic value

Page 24: Guest Lecture Open Data
Page 25: Guest Lecture Open Data

various reasons, a transparant sector results

284 organisations, aidtransparency.net

Page 26: Guest Lecture Open Data

showing how government works

data.gov.uk/organogram

Page 27: Guest Lecture Open Data

http://www.flickr.com/photos/59937401@N07/5858059202/

economic value? transparency?

not my departments job!

Page 28: Guest Lecture Open Data

four valuable reasons

more efficient government!!

better public services !

more transparent government !

new socio-economic value

Page 29: Guest Lecture Open Data

29

participatory budgeting

Page 30: Guest Lecture Open Data

30

government is a strong re-user: 6.5m GBP saved

Page 31: Guest Lecture Open Data

Denmark projects 26M+ Euro annual savings

Page 32: Guest Lecture Open Data

BC #1 user, 33% of downloads

Page 33: Guest Lecture Open Data

open data shows 200m GBP potential savings

Page 34: Guest Lecture Open Data

four valuable reasons

more efficient government !

better public services!!

more transparent government !

new socio-economic value

Page 35: Guest Lecture Open Data

35

http://wiki.openstreetmap.org/wiki/File:Haiti_earthquake_damage_map.png

http://hot.openstreetmap.org/

allowing disaster response

Page 36: Guest Lecture Open Data

http://www.theguardian.com/society/2013/mar/12/nhs-transparency-open-data-initiative

1000 less heart surgery deaths / year

https://secure.flickr.com/photos/76652722@N04/6878044757

Page 37: Guest Lecture Open Data

pupil and elderly public transport: better = cheaper

Page 38: Guest Lecture Open Data

Uganda: Open Data and Community Health Monitoring

28

z 33% reduction in under-5 mortality

z 20% extra utilisation of out-patient services

z Significant improvements in: z Immunization z Waiting times z Absenteeism

source: Andrew Stott, World Bank

Page 39: Guest Lecture Open Data

policy problem

new ways

Page 40: Guest Lecture Open Data

four valuable reasons

more efficient government!!

better public services!!

more transparent government!!

new socio-economic value!

Page 41: Guest Lecture Open Data

Not just gov

Page 42: Guest Lecture Open Data

42

Page 43: Guest Lecture Open Data

43

opencorporates.com

Page 44: Guest Lecture Open Data

44

encourage what you cannot do yourself

Page 45: Guest Lecture Open Data

How to do

Page 46: Guest Lecture Open Data

Costs too much; What’s the business case; Has commercial value; Possible privacy issues; Confidential info; It’s not ours, and we don’t know who owns it; It’s not ours and supplier won’t allow it; The quality isn’t very good; We don’t know where it is; Not our job; It’s in a useless format anyway; I don’t have the authority; People will misuse the data; People will use it wrongly; Only we understand our data; We’ll get sued; Files are just too big; Too little bandwith; It starts with this, but where’s the end? It’s there, but can’t be opened; Data is dated/too old; It’s not in digital format; Is this even legal?; Our Minister says no; We never have done this before, why start now?; I don’t see the use; Nobody will be interested; No time; No resources; Just do FOIA requests; We’ll publish it redacted; It’s not complete; It contains errors; It’s commercially sensitive; Combining this with other data is dangerous; People will come to wrong conclusions; People will get lost and confused; It will trigger endless discussions; We can’t confirm or deny we have that data; We’ll get feedback, and can’t handle that; Our IT supplier says it’s not possible; Our IT supplier will charge too much; Our site will crash; It’s already online! (but in unfindable PDFs); If people download it and use it later it will be outdated; I can’t take responsibility for all the reuse; People will get angry; Our data is in contradiction to the data of the department that is in charge of the topic; Only we truly understand.....statistics/meteo/geo/laws; We’ll disrupt the market; It will only be used to attack us.

many shades of ‘no’

Page 47: Guest Lecture Open Data

Слишком дорого; В чем выгода; Есть коммерческая ценность; Возможные проблемы с личными данными; Конфиденциальные данные; Эти данные не наши, мы не знаем, кому они принадлежат; Мы не владеем данными, а поставщик не разрешит; Качество не очень хорошее; Мы не знаем, где они; Не наша работа; Они в бесполезном формате; Я за это не отвечаю; Люди воспользуются данными неправильно; Люди используют данные в неправильных целях; Только мы понимаем наши данные; Нас засудят; Файлы очень тяжелые; Низкая скорость загрузки; Данные устарели; Данные не в цифровом формате; Это вообще законно?; Министр запретил; Мы никогда этим не занимались, зачем начинать?; Я не вижу в этом пользы; Никому не будет интересно; Нет времени; Нет ресурсов; Мы будем отвечать только на запросы, связанные со свободой информации; Мы опубликуем информацию выборочно; Данные неполные; В файле есть ошибки; Это коммерчески важная информация; Если объединить эти данные с другими, они могут быть опасными; Люди сделают неверные выводы; Люди запутаются; Начнутся бесконечные обсуждения; Мы не можем ни подтвердить, ни опровергнуть, что у нас есть эти данные; Начнется обратная связь, у нас нет возможности всем отвечать; поставщик наших ИТ систем говорит, что это невозможно; Наш сайт упадет; Эти данные уже онлайн! (но в PDF, и их невозможно найти); Если люди загрузят их сейчас, а используют потом, они устареют; Я не могу отвечать за всех повторных пользователей; Люди разозлятся; Наши данные не совпадают с данными ведомства, которое отвечает за этот вопрос; Только мы по-настоящему понимаем … статистику, погоду, законы; Мы нарушим баланс на рынке; Ими воспользуются, чтобы нас атаковать.

Множество оттенков «нет»

Page 48: Guest Lecture Open Data

census.okfn.org

Page 49: Guest Lecture Open Data

Mins I&E / EA 2015, e.g. NDW 600M records / day

Page 50: Guest Lecture Open Data

100% CC0, 90% dode links

April 2014 Oct 2014

Page 51: Guest Lecture Open Data

http://epsiplatform.eu/content/european-psi-scoreboard

Page 52: Guest Lecture Open Data

OGP Action plan

Page 53: Guest Lecture Open Data

ready, but for what?

Page 54: Guest Lecture Open Data

real barriers: it’s a transitionSee http://www.flickr.com/photos/epsiplatform/5737203950/

Page 55: Guest Lecture Open Data

55

top

bottom

middle

Page 56: Guest Lecture Open Data

56

top down leadership

Page 57: Guest Lecture Open Data
Page 58: Guest Lecture Open Data

Networked life, networked work, networked learning

know the gov does not exist. find 1 civil servant

bottom up trailblazing

Page 59: Guest Lecture Open Data

Actief uitnodigend

community engagement

http://www.flickr.com/photos/dhammza/492882480/

Page 60: Guest Lecture Open Data

iinformal is important

Page 61: Guest Lecture Open Data

61

middle

middle out organizing

Page 62: Guest Lecture Open Data

where to start?

Page 63: Guest Lecture Open Data

1 Core reference data

Page 64: Guest Lecture Open Data

accessibility

Page 65: Guest Lecture Open Data

65

2 Data people care about

Page 66: Guest Lecture Open Data

cleaning up the country side

Page 67: Guest Lecture Open Data

3 Data gov cares about

Page 68: Guest Lecture Open Data

OPEN data

open DATAvs

Page 69: Guest Lecture Open Data

postcodes….if you darehttp://www.zylstra.org/blog/2013/10/theoretically-open-post-codes-not-so-much-in-practice/

Page 70: Guest Lecture Open Data

2 years ago

Page 71: Guest Lecture Open Data

now

Page 72: Guest Lecture Open Data
Page 73: Guest Lecture Open Data

DK Roadmap: open is key ingredient

Page 74: Guest Lecture Open Data

serious issues

serious data=

Page 75: Guest Lecture Open Data
Page 76: Guest Lecture Open Data

health care? gov spending?

Page 77: Guest Lecture Open Data

„just add openness”

Now

Page 78: Guest Lecture Open Data

78

instru ment

inter vention&

Page 79: Guest Lecture Open Data

value, efficiency, impact

use open data as a policy instrument

policy issue

stakeholders open data

Page 80: Guest Lecture Open Data

!• Texel • Hollands Kroon • Heerhugowaard • Purmerend • Velsen • Haarlem • Medemblik • Den Helder • Schagen

!• tourism • real estate • neighborhoods • public spaces • flash floods • local entrepreneurs • school/elderly transport • disused shops • tbd

9 local communities & themes

#pnhslim

open data as intervention

Page 81: Guest Lecture Open Data

flash flood prevention with greener gardens

Page 82: Guest Lecture Open Data

compare local spending

Page 83: Guest Lecture Open Data

83

<<1%

rule of thumb

Page 84: Guest Lecture Open Data

84

how to change existing financing models?

https://secure.flickr.com/photos/alexjbutler/14953247679/

Page 85: Guest Lecture Open Data

Who is to pay the bill? 5 options

Data setsRegistration+ updating

Delivery to users

Public sector

Private sector + society

Reporting fee

Registration fee

User fee

User fee

State

budget

Page 86: Guest Lecture Open Data

Who is to pay the bill?

Data setsRegistration+ updating

Delivery to users

Public sector

Private sector + society

Reporting fee

Registration fee

User fee

User fee

State

budget➢Discourage use

➢Affect quality data

➢Discourage use ➢ govt pays govt ➢Admin costs

➢If there is a concrete return ➢If admin costs low

➢Discourage use ➢Miss out on value

Page 87: Guest Lecture Open Data

Some Dutch figures on income key registers 2011 * 1 million euros

Registration fees

Public sector use

Private sector use

Own re-use activities

State budget Total

Buisness registers

67 6 42 5 - 120

Cadastral registers

130 15 35 30 - 210

Topogra-phical map

- 9,5 0,5 - 14 24

Adresses - - - - 4 4

Total 197 30,5 77,5 35 18 358

Page 88: Guest Lecture Open Data

embedding into internal policy?

Page 89: Guest Lecture Open Data

leadership commitment !

procurement procedures !

procurement contracts !

IT specs & change management !

policy plan writing !

project plan writing

Page 90: Guest Lecture Open Data

https://secure.flickr.com/photos/driek/2938311931

internal policy addresses data holder concerns

Page 91: Guest Lecture Open Data

open by design

privacy by design&

Page 92: Guest Lecture Open Data

http://www.flickr.com/photos/mdid/3271972434/

too important to just fix at the end

Page 93: Guest Lecture Open Data

treating openness as an incident

#pnhslim

Page 94: Guest Lecture Open Data

mission, infrastructure, operational

Page 95: Guest Lecture Open Data

Issue driven

Tech drivenvs

Page 96: Guest Lecture Open Data

96

data pitfalls

What, but not why Evidence based: correlation / causation Direct re-use vs proxies Pre-hypothesis tool: q’s, context, experiments Big data also Σ small data (access / ownership)

Page 97: Guest Lecture Open Data

97

seeing impact

Page 98: Guest Lecture Open Data

http://www.flickr.com/photos/chipdatajeffb/8421625784/

yay! impact!

http://www.flickr.com/photos/chipdatajeffb/8421625784/

Page 99: Guest Lecture Open Data

where is the next Google OR where is the killer app for open data

„where’s the first 1B+ exit?”

Page 100: Guest Lecture Open Data

making money count

Page 101: Guest Lecture Open Data

„We’re not an open data company”

Page 102: Guest Lecture Open Data

costs 500k? or yields 3M5 & 100 jobs?

Page 103: Guest Lecture Open Data

http://www.flickr.com/photos/chipdatajeffb/8421625784/

yay! impact!

http://www.flickr.com/photos/chipdatajeffb/8421625784/

Page 104: Guest Lecture Open Data

if you know why, you’ll see impact

http://www.flickr.com/photos/ringroadproductions/144808172/

Page 105: Guest Lecture Open Data
Page 106: Guest Lecture Open Data

CreditsOpen Data!

The What, Why, and How in 50 examples or less

Photos: all CC BY NC SA Ton Zijlstra, except where mentioned on photo (CC respective authors) !Slides: CC BY NC SA except where source stated downloads at http://slideshare.net/thegreenland

Ton Zijlstra, @ton_zylstra, [email protected]