Common monitoring and evaluation framework - Eurocitiesnws.eurocities.eu/.../media/CommonMonitoringandEvaluationFramework.pdf · Common framework: The project will create a common

Proposal Acronym: SHAR-LLM Proposal ID: 691895

Call: H2020-SCC-2015

Project Title: Sharing Cities

Common monitoring and evaluation framework

Planned submission date: Month 9 Actual submission date: Month 9

Imperial College London

Ref. Ares(2016)7142640 - 22/12/2016

2

Common monitoring and evaluation framework WP number: D8.1

Document History

Version Comments Date Authorised by

Version 0.1 1/6/2016 IC

Version 1 25/7/2016 IC

Version 1.2 19/9/2016 IC

Version 1.3 30/9/2016 IC

Number of pages: 61

Number of annexes: 2

Responsible Organisation: Imperial College London Contributing Organisation(s): Instituto Superior Tecnico Politecnico di MIlano - Departmento of Energy Ricerca sul Sistema Energetico - RSE SpA

Principal Author(s): Konstantinos Zavitsas Yanjie Dong John Polak

Contributing Author(s): Francesco Causone Sonia Cunha Pierpaolo Girardi Andre Pina Andrea Temporelli

3

TABLE OF CONTENTS

Table of Contents .................................................................................................................................... 3

1 Introduction .................................................................................................................................... 4

1.1 The SHARING CITIES Project .................................................................................................. 4

1.2 Monitoring and evaluation framework ................................................................................. 4

1.3 Framework elements ............................................................................................................. 5

1.4 This deliverable ...................................................................................................................... 6

2 Overview of demonstrator programme ......................................................................................... 7

2.1 Building retrofitting and local renewable energy generation ............................................... 7

2.2 Sustainable energy management system ............................................................................ 11

2.3 E-Mobility ............................................................................................................................ 13

2.4 Smart Lamppost................................................................................................................... 19

3 Monitoring and evaluation targets ............................................................................................... 21

3.1 Building Retrofit ................................................................................................................... 21

3.2 Sustainable Energy Management System ........................................................................... 23

3.3 Shared e-Mobility ................................................................................................................ 25

3.4 Lamppost ............................................................................................................................. 29

4 Measurable Indicators definitions and influencing factors .......................................................... 31

4.1 Assessment methods ........................................................................................................... 31

4.2 Measurable indicators ......................................................................................................... 34

5 Initial appraisal of existing and potential data sources ................................................................ 43

5.1 Data sources ........................................................................................................................ 43

5.2 Data gap analysis ................................................................................................................. 47

5.3 Data gap analysis discussion ................................................................................................ 49

6 Concluding remarks and future steps ........................................................................................... 50

7 References .................................................................................................................................... 51

8 Appendix ....................................................................................................................................... 52

8.1 Appendix A........................................................................................................................... 52

8.2 Appendix B ........................................................................................................................... 57

4

1 INTRODUCTION 1.1 The SHARING CITIES Project The SHARING CITIES project brings together city authorities, business and research organisations to develop a vision of a more agile and more collaborative smart cities market. The aim is to dramatically increase the speed and scale at smart solutions are implemented across Europe by engaging citizens in new ways that enable them to play an active role in the transformation of their communities – delivering more vibrant, liveable, economically active and resource efficient cities. Underpinning this are shared solutions that apply a “digital first” approach and that provide “building blocks” incorporating European and worldwide leading practices that can be deployed at scale, yet tailored to cities of different size and stage of development. The vision and objectives are delivered through implementation of a number of measures which are categorised into three core subjects of the project: People, Place, Platform and each of them includes the following contents: PEOPLE – Approaches and tools to develop a deep understanding of society, and the means by which citizens can actively participate in making their districts better places, through sharing services, delivering better outcomes. PLACE – Comprising four main streams of work that address city infrastructure and services that support low energy districts, electrification of mobility, and integration of infrastructures and processes. These include: Building Retrofit; Sustainable Energy Management System; Shared eMobility; and Smart Lampposts. PLATFORM – An urban sharing platform (USP) that manages data from a wide range of sources including sensors as well as more traditional data sources. The USP will be built using open technologies and standards, building London’s DataStore expertise, Milan’s work on an API marketplace and Lisbon’s work on sensor data and gateways. 1.2 Monitoring and evaluation framework Monitoring and evaluation forms a key element of SHARING CITIES, since it provides the means by which the work undertaken in the project becomes relevant to the wider policy and innovation community. The overall aim of this work is to deliver a comprehensive monitoring and evaluation of the effects of the People/Place/Platform (PPP) measures developed and deployed as part of the SHARING CITIES project. This monitoring and evaluation work consists of two elements:

1. Methods to enable the impacts of the specific PPP measures implemented in the partner cities to be reliably understood, quantified and evaluated.

2. A Toolbox of models and methods to enable these results to be used as a basis for the development of future policy, technology and business models. In particular enabling both the scaling up of existing PPP measures and the translation, replication and evolution of these measures to cities across Europe.

The monitoring and evaluation will be based on a clear and explicit set of principles that will guide the selection of evaluation targets and the development of evaluation methods. Such a principles-based approach will avoid the risks associated with an ad hoc and fragmented case-based approach. There are six key principles that will inform our work:

5

Common framework: The project will create a common monitoring and evaluation framework which will define the evaluation targets to be addressed and the evaluation methods to be used including processes covering data collection, data standards, data quality, data stewardship and the definition of key evaluation indicators.

Local implementation: Although the overall evaluation framework will be developed centrally, responsibility for the implementation of the framework will reside locally with relevant research and delivery partners in each city. This is because the successful implementation of complex data collection protocols depends on detailed local knowledge which is only available in the local partners. Moreover, local knowledge is critical for the design of proper control.

Target salience: Each PPP measure will entail a set of technical developments and will have a range of direct and indirect effects on people, business and the public sector. Since it is impractical to monitor and evaluate every possible technical and impact dimension, the selection of relevant evaluation targets will be a critical part of the common framework. This selection will be based on consideration of the salience of each potential evaluation target in respect of its policy and market significance, its practical contribution to scaling and replication together with the practical opportunities for the collection of relevant high quality monitoring data.

Control for covariates: Each PPP measure will be introduced into a complex environment in which many different factors can influence a particular outcome or evaluation target. For example, when considering the impact of a building retrofit measure on energy use and expenditure, we need to recognise that energy expenditure will be affected by energy prices, weather conditions, appliance ownership and use and patterns of building occupancy as well as the retrofit measure itself. It is vital that the monitoring and evaluation activities collect sufficient information on these covariates to enable proper statistical control for their effect. An important element of this is to ensure that a sufficient time series of data are collected not only after but also before the implementation of the PPP measures.

Common core: A key element of the common evaluation framework will be the development of a common core of evaluation targets and associated KPIs and data and measurement processes that will be implemented in a consistent manner across all three cities. This common core will provide the fundamental mechanism by which the SHARING CITIES will be able to aggregate experience and learning across the participating cities and indeed more widely. This common core will be selectively augmented by additional evaluation targets that are specific to a particular city and/or a particular PPP measure.

Dimensions of impact: In developing evaluation targets, it is recognised that the PPP measures implemented by SHARING CITIES will have a wide range of different types of impacts on different stakeholders and that these impacts may change over time as stakeholders learn and adapt their behaviour and as the measures themselves are evolved. Our experience suggests that it is useful to structure consideration of these impacts under five broad headings:

o technical performance o institutional and business consequences o impacts on attitudes and behaviours o wider systemic impacts including environmental, security, safety and sustainability o economic and social implications including those affected by efficiency, equity and

social inclusion This structuring provides a useful simplification of what might otherwise be an overly complex domain and additionally assists the task of designing data collection protocols. 1.3 Framework elements The common monitoring and evaluation framework (CMEF) defines the following key elements:

6

The specific evaluation targets: These are the research questions of relevance and interest to SHARING CITIES. For example, in the case of PPP measures in the transport domain such questions might relate to the adoption and use of shared mobility services and the impact of such services on car ownership, energy use and emission. Likewise, for the platform technologies developed in the project, interest might focus on the quality of the data attracted to the platform and the use made of it by individuals and business. Developing an agreed set of evaluation targets will be a key early activity in the project. These will be divided between core targets that are addressed.

Measurable indicators: Corresponding to each evaluation target we will define one or more measurable indicators. For instance, in the case of the shared mobility example considered above, adoption and use could be measured using indicators such as mode share and trip frequency. In general, the evaluation indicators will be quantitative but in some instances, such as in understanding the impact of a new disruptive service on existing business relationships and regulatory framework, it may be more appropriate for indicators to include both quantitative and qualitative elements.

Data standards: Standards are necessary both in the definition of underlying data and indicators (e.g., what exactly do we mean by a trip?) and in the manner in which relevant information is stored, pre-processed and stewarded through the lifetime of the project, and beyond. The project will draw on relevant industry and academic standards wherever possible, to ensure that the data are as transparent and transferable as possible.

Data collection methods: This task will also identify and agree the broad types of data collection methods that will be used to obtain the information required for the development of the evaluation indicators. A wide range of different methods of data collection is available including the harvesting of information from operation data streams, the undertaking of polls and questionnaires, panel surveys, the administration of structured and unstructured interviews, hypothetical choice experiments, case studies and narratives. Consideration will also be given to the duration over which data should be collected including identify those case where a before-and-after approach is required. The types of methods used will be carefully matched to the nature of the research targets and indicators.

1.4 This deliverable The structure of this deliverable is organised as follows. Chapter 2 provides a brief summary of each of the demonstrator activities in each of the cities. These template-based summaries are focused on key information such as type, location, scale, technologies, etc. which are important for performance assessment. In chapter 3, the evaluation targets for each demonstrator are stated, including both desired outcomes (e.g. improved air quality and car ownership reduction) and collateral or unintended effects. For each of the evaluation targets, quantitative indicators and corresponding measurement quantities are introduced in chapter 4. Chapter 5 provides an initial overview and appraisal of the existing and potential data resources. At this stage, the focuses principally on the activities to be undertaken in WP3, since these are currently the most mature. We will update the scope to include relevant elements of the work of WP2 and WP4 as these streams of work develop.

7

2 OVERVIEW OF DEMONSTRATOR PROGRAMME The key evaluation and assessment targets presented in this document concern the “Place” demonstrations from the PPP (People, Place and Platform) measures. “Place” itself comprises of four different repeatable measures:

Building Retrofit & Local Renewable Energy Generation;

Sustainable Energy Management Systems;

Shared eMobility, which includes EV car sharing, eBike sharing, eLogistics, EV charging facilities and Smart Parking;

Smart Lamposts. This chapter provides a brief summary of each of the demonstrator activities in each of the cities. It is worth to be noted that because it is still early stage of the project when this report is written, some of the demonstration activities are subject to change as the project is moving on. To minimise the discrepancy of information across the participating cities, a data collection proforma (see Appendix A) was designed aiming to:

Collect up to date information on all demonstrators,

Amend existing data, stressing areas where little or no data were available,

Collect information on data sources and data collection equipment, and

Standardise demonstrator data across all cities The proforma was distributed across WP8 city partners and all data presented in this deliverable have been updated to be comprehensive, up-to-date and consistent. 2.1 Building retrofitting and local renewable energy generation Building retrofit in the three cities will involve common deep-retrofit approaches (windows replacement and insulation); innovative approaches and materials (e.g. ‘cool’ materials for external walls, e.g. green walls; roofs; and some pavements); and ICT-enabled building monitoring and control systems. The selection of buildings in the three districts seeks to address building typologies that offer high replication potential within the districts, across the three different cities, the followers and across Europe. Local renewable energy generation will be installed in the cities: Milan and Lisbon have expertise in solar PV that will be shared between them, London and the Followers during the design, installation and maintenance - the innovative Lisbon solar potential chart provides a useful and replicable tool to exploit across the cities. London will be leading on heat pump renewables and heat network integration, with particular expertise in capturing secondary heat to be shared with the other cities. The specific activities are summarised in the following table for all the three cities:

8

Table 2.1.1: Summary of demonstration activities – building retrofitting and renewable energy generation

Type of buildings

Number of buildings

Number of dwellings

Tenants Total floor areas (sqm)

Retrofitting measures & priority (see 2.1.2) Renewable energy priority (see 2.1.3)

1 2 3 4 5 6 7 8 9 10 1 2 3 4

Lisbon Public housing

2 248 Social housing

20609 h L x x

Public offices

1 N/A Municipality h1 L h h

Private housing

TBD TBD Private residential

London Public housing

13 (3 estates)

304 Mixed use 25274 h m l/m m h x2 H l/m x3 h h l/m l

Milan

Public housing

2 66 Social housing

4633 h h m h m h m L h h

Private residential

5 300 Mixed use 21000 h l h h L h m

x indicates a measure is considered; if, priority data are available: h: high, m: medium, l: low; Beyond the data presented in Table 2.1.1 , as part of retrofit demonstrators London anticipates the connection to heat network and the generation of energy through a river heat source pump.

1 To be implemented in one of the two buildings 2 Not to be included on site, but a CHP/water source heat pump will be used to heat the district heating network 3 Solar panels will be considered where feasible, but not on every block- most likely on one or two blocks at Flamsteed Estate.

9

Table 2.1.2: Building retrofitting measures

ID Retrofitting measures

1 thermal insulation – walls and/or ceilings

2 thermal insulation windows

3 air tightness improvements

4 hybrid/mechanical ventilation with heat recovery

5 thermostatic valves

6 high efficiency generation system

7 solar shading

8 LED lighting

9 photovoltaic panels

10 solar thermal panels

Table 2.1.3: Renewable energy generation measures

ID Renewable energy generation measures

1 District heating

2 Low carbon energy heat

3 PV/solar

4 EV charging

The timeframe of retrofit measures application in Lisbon, London and Milan range as shown in the following Table:

10

Lisbon London Milan

Public

Residential Buildings

Public offices

Private Residential Buildings

Public


Public


Private


Building selection

Jan 2016 – Mar 2016

Jan 2016 - Mar2016

Jan 2016 - Dec 2016

Surveys and Specifications

Jan 2016- Dec 2016

Launch public tender for building selection

Jan 2016 – Mar 2016

Building evaluation

Apr 2016 Apr 2016 - Dec 2016

Jan 2017 - Mar 2017

Do on site analysis

Jan 2016 – Mar 2016

Feasibility studies on nominated buildings

Apr 2016 – Nov 2016

Design contract

May 2016 Jan2017 - Mar2017

May 2017 Do detail

design and approval

Apr 2016 – Sep 2016

Design finalization

Jun 2016 - Aug 2016

Apr 2017 - Aug 2017

Apr 2017 - Jun 2017

Executive design

Jan 2017 - Jun 2017

Energy audits and detail design of first

buildings

Jul 2016 – Jun 2017

Development

of Tender Jan 2017- Mar 2017

Procurement documentation

approval

Jul 2017 – Sep 2017

Construction contract

Sep 2016- Dec 2016

Sep 2017 - Dec 2017

Apr 2017 - Jun 2017

Issuing of tender and

appointment of contractor

Apr 2017- Sept 2017

Public tender process

Oct 2017 – Mar 2018

Assembly approvals and work

procurements

Apr 2017 – Feb 2018

Construction work

Jan 2016 - Sep 2016

Jan 2018 - Dec 2018

Jul 2017 – Mar 2018

Undertaking of works

Oct 2017- Mar 2018

Apr 2018 – Dec 2018

Construction works Oct 2017 – Dec 2018

Monitoring design Apr 2016 – Jun 2016

Install

monitoring systems

Oct 2016 – Dec 2016

Install monitoring systems on selected

buildings

Sep 2016 – Dec 2016

Monitoring data

Oct 2016 - Dec 2018

Jan 2019 - Dec 2020

Apr 2018 - Dec 2018

11

2.2 Sustainable energy management system Energy Management in a typical district is typically run by isolated digital and hardware solutions. Sharing cities aims to enhance the existing solutions in the districts with interoperable sustainable energy management systems (SEMS) integrated with the urban shared platform (USP) (WP4) that provide coordinated, integrated (with renewables and EV charging), optimised (secure, stable, balanced supply and demand) and interoperable energy management across urban infrastructures with information to better manage and optimise the citizens’ energy demand to reduce their energy use and bills. Sharing Cities proposes the development of an advanced, data-rich, management system which gains maximum benefits from the retrofitted buildings, sharing energy data through the open platform enabling energy services to be provided that reduce energy use and bills. This will enable the design and roll out of higher level applications for citizens and authorities, taking advantage of the sensing layers and actuators installed. The specific activities for the SEMS are summarised in the table below.

12

Table 2.2: Summary of demonstration activities – sustainable energy management system

Type Scale Usage Factors considered Capability Other impacts

Lisb

on

District / regional/ building

Energy consumption/ production prediction. Energy monitoring, energy efficiency, demand-response. EV charging & flexible loads.

Electric Distribution Network; measures from primary substations and secondary substations; MV/LV transformer data; public lighting system; lighting consumption; electric mobility; measures for EV recharging; electric meters in retrofitted buildings; thermal meters in Buildings; environmental data

Monet as a Smart City System will:

Collect energy data for each Smart Grid system: public lighting, electric mobility.

Provide (real-time) energy monitoring and energy reporting (electric) at municipality level

Integrate data coming from other systems to correlate consumptions information

Integrate energy tariffs model to estimate and simulate energy costs

Lon

do

n

Heat District To determine the best times to operate the pump and building heating controllers and then put this plan into action.

Heat requirements from the citizens at the buildings, metering, weather, carbon impact, electricity prices, renewable generation, and potentially other environmental concerns.

Control of energy assets (RSHP/Gas CHP DH; Street Lighting; Solar PV; Thermal Storage; EV Charging Points) through direct control mechanism (i.e. turning asset on/off or switching between energy sources) or demand response/behaviour change (i.e. provide incentive to residents to change consumption patterns)

pushing billing and energy use information to the citizens’ mobile phones /websites leading to reduced energy use and bills, carbon emissions and support balancing of grid energy supply and demand by shifting their demand (manual and automated) from peak to off-peak times for energy use.

Mila

n

Electricity

Municipality and Building

Better match micro-generation for PV panels.

Currently, the SEMS system can acquire data from the energy field, but the devices that can provide the measurements are not defined or are not yet available interfaces

Monet as a Smart City System will:

Collect energy data for each Smart Grid system: distribution network, public, lighting, electric mobility.

Provide (real-time) energy monitoring and energy reporting (electric /thermal / gas) at municipality level

Integrate data coming from other systems to correlate consumptions information

Integrate energy tariffs model to estimate and simulate energy costs

13

Based on the data provided via the info proforma, the timeframe for the implementation of SEMS demonstrators is:

Lisbon London Milan

Defining SEMS requirements Jan 2016- Oct 2016

SEMS procurement Nov 2016- Jun 2017

SEMS Implementation and Commissioning

Jul 2017- Dec 2017

SEMS Operation and Monitoring Jan 2017- Dec 2018 Autumn 2016 For Milan it is anticipated that the interface with DSO will become available in autumn 2016, making possible to measure energy consumption at a building level. 2.3 E-Mobility A bold and multi-action suite of measures for the elevating of eMobility districts in the three core cities, including:

EV car sharing – building on and learning from Milan’s 10yrs and London’s 20yrs of car sharing experience applying different business models (public/private) and shift to EV car clubs in recent years;

eBikes as part of the sustainable and integrate mobility-as-a-service offer in the cities, building on and integrating (Milan will be the first city in Italy to do this) with very substantial conventional bike share schemes (i.e. 11,500 public hire bikes in London);

smart parking to incentivize the use of eMobility and eMobility services, reduce search time, optimise limited parking space, reduce road km and emissions;

eLogistics to streamline the growing volume of light freight caused by increasing on-line- delivered customer/business purchasers; and

EV charging stations maintained by an interoperable network (i.e., mobi.me already successfully implemented across Portugal, including significantly in Lisbon), with 100 new smart charge points as part of this project.

These integrated and mixed measures create a co-created, connected and shared package of initiatives that will test and demonstrate the scalability of new technologies and services for eMobility in the cities, how to integrate within the complex mobility sector and across energy and ICT sectors. The detailed demonstration plans for each city is summarised in Table 2.3.

14

Table 2.3: Summary of demonstration activities - Mobility

Cit

y

Measure Initiative Description Nº

Vehicles

Nº Infrastruc

ture elements

Lisb

on

EV Car Sharing

CML Corporate Car-Sharing

New system to create an automated car-sharing initiative for municipality people based on a fleet of EVs (starting with 15 Peugeot iONs),

15+30 eCars

N/A

eBikes EMEL eBike Sharing

New mobility initiative. As part of the new bike-sharing initiative, with at least 30 eBikes. E-bike sharing scheme and a park & bike scheme will be deployed: when air pollution conditions are expected to deteriorate, a set of committed citizens will be incentivized by reduced/eliminated parking charges for those that switch to eBikes outside the district.

30 eBikes

2 stations

Smart Parking

EMEL smart parking

New mobility initiatives namely focusing on the following use cases: •Delivery bays occupation monitoring (possibly check-in/check-out processes to be supported); •Monitoring of the illegal occupation of electric vehicle charging points parking spots; •Overall parking spaces occupation at the street level.

N/A N/A

Logistics

EDP eLogistics

Currently, electric vehicles are being used by EDP on their normal operations without taking into account neither the operational constraints nor the benefits of EVs. No dedicated fleet management solution has been proposed. Utility fleet for maintenance actives within the district.

6 + 15 eCars/e

Vans N/A

EMEL eLogistics

New fleet for use in their parking meters maintenance and cash collection activities throughout the downtown.

5 + 6 eVans

N/A

CML eLogistics

Fleet for delivery, garbage collection, street monitoring.

17 + 31 eCars/ eVans

N/A

eV Charging Points

EDP Public Charging Network

eV Charging Installation of public charging points of 20KW with three charging modes. Installation of one new public rapid charge point.

N/A

6 normal chargers + 1 fast charger

EDP Private Charging Network

Smart charging points in private locations to be coupled with the availability of local PV generation. The combination of user/business requirements with technical grid requirements in order to manage loads, generation, and e-mobility systems for the optimization of charging profiles.

N/A 2(4)

home chargers

15

Cit

y


Vehicles

Nº Infrastruc

ture elements

CML Private Charging Network

Added-value services between the public and private networks, namely new energy pricing schemes and incentives

N/A 24 smart

home-chargers

Other

CML/EMEL Corporate eMoto Sharing

Fleet for use in their parking meters maintenance and cash collection activities throughout the downtown.

3 + 17 eMotos

N/A

Lisbon CML eBus

Electric buses in the urban environment. Exact application to be defined.

2 eBuses

N/A

Lon

do

n

eV Car Sharing

RBG eCar Club Trial

Six to 20 vehicles in demonstration area, likely to be a ‘back to base’ model

6-10 eCars

N/A

Autonomous EVs

Run three trials (design, operation, maintenance, evaluation) of these vehicles in 2016-17: (i) last mile “point-to-to point” shuttles to connect major transport hubs in the district with main employment and leisure destinations; (ii) self parking cars – allowing the space allocated to car parking in high value locations to be optimised; (iii) the automation of freight vehicles to allow more efficient use of road space

TBD N/A

eBikes RBG eBike Trial

Up to 30 bikes at up to 5 different locations, likely to be a ‘back to base’ model for simplicity

25 eBikes

5 stations

Smart Parking

RBG Smart Parking

TBC- Censor system to be incorporated in smart lampposts trial – App based real time information and data collection

N/A 300

parking sensors

Logistics RBG eLogistics

Autonomous vehicle delivery system pilot (University of Greenwich campus)

4 eVans N/A

eV Charging Points

RBG Bolloré EV Charging Rapid EVCP

Standard on street as part of borough wide Source London based roll out in partnership with BluePoint (Bollore Group) TBC – design for on street station potentially in conservation area to be explored

N/A

20 normal

chargers + 1 fast charger

Other

RBG Smarted Shared Space

Smarter shared space trial – Greenwich Foot Tunnel

N/A

TBD Sensors

and cameras,

digital messagin

g

Smart Square/

In line with Smart Squares in Milan and Lisbon look to provide an area within the

As part of other

pilots

As part of other pilots

16

Cit

y


Vehicles

Nº Infrastruc

ture elements

neighbourhood

demonstrator to ‘show case’ eCar club, eBikes, smart parking and EVCPs

Mila

n

eV Car Sharing

Milan Public e-Car Sharing

Deploy 60 electric vehicles into car-sharing schemes with 10 eCar sharing stations (including 60 EV charge points).

60 eCars

40 normal + 20 fast

chargers

Milan e-Car sharing for condominium

2 electric vehicles dedicated to the trial for “condominium car sharing” test. The stations will be equipped with plants photovoltaic, able to recharge both eCars and the eBikes batteries.

2 eCars

Milan e-Car sharing for condominium Symbiosis district

N/A 10 eCars 5 home chargers

eBikes Milan E-Bike Sharing

The system will allow the reservation of e-bikes, to guarantee availability and boost modal shift from car to eBike. A mixed traditional and a user- based reallocation systems will guarantee a constant level of service in terms of a number of e-bikes at disposal for the users. The batteries of e-bikes will be recharged in correspondence with the e-car sharing stations

150 eBikes

7 new stations

(for a total of

14 stations)

10 charging

points

Smart Parking

Milan Smart Parking

Sensors installation in 125 parking bays for city parking, such as: E.V. freight, disabled, car sharing and for unauthorized metered parking. Parking App, to track users (GPS) for mobility habits will be at disposal. 300 RFID with real-time information to be available on parking bay and charging point availability. Smart parking service will be based on a predictive algorithm to guide the drivers to available parking places.

N/A 125

parking sensors

Logistics and eV Charging Points

Milan e-Logistics

Elogistics platform with 9 vans (equipped with on-board tracking) and 2 eBikes. The elogistic platform (i.e. UCDC, the urban consolidation/distribution centre), equipped with fast recharging points will host the e-vehicles.

9 eVans and 2 eBikes

1 fast charger

17

The following table presents the timeframes for the implementation of Mobility demonstrators. A more detailed table is available in

18

Measure Lisbon London Milan

Site-specific Initiative Timeframe Site-specific Initiative Timeframe Site-specific Initiative Timeframe

eV Car Sharing CML Corporate eCar Sharing

Jan 2016 – Jun 2017

RBG e-Car Club Trial Apr 2016 – Jun 2017


2016 - 2018 Milan e-Car sharing for condominium

Autonomous EVs Jan 2016 – Mar 2017


eBikes EMEL eBike Sharing Apr 2016 – Dec 2017

RBG e-Bike Trial Apr 2016 – Sep 2017

Milan E-Bike Sharing 2016 - 2018

Smart Parking EMEL Smart Parking Apr 2016 – Sep 2016

RBG Smart Parking Jul 2016 – Dec 2017

Milan Smart Parking 2017 - 2018

eLogistics

EDP eLogistics Sep 2016 – Dec 2016

RBG e-Logistics Milan e-Logistics 2017 - 2018 EMEL eLogistics Apr 2016 – Dec 2017

CML eLogistics Apr 2016 – Dec 2017

eV Charging Points

EDP Public and Private Charging Network

Jan 2016 – Apr 2017

RBG Bolloré EV Charging Rapid EVCP

Jul 2016 – Dec 2017


2016 - 2018



Jan 2016 – Apr 2017


Other CML eBus Jul 2017


Jan 2016 – Jun 2016

CML/EMEL Corporate eMoto Sharing

Smart Square/ neighbourhood

19

2.4 Smart Lamppost Smart lamppost presents a very visible “quick win” for smart cities; and the well-proven lighting and maintenance savings offer an attractive bankable initiative. The smart approach is to consider how to develop business models and funding mechanisms that incentivise implementation of ‘smart’ measures (WiFi; air quality, parking, eV charging, etc.) alongside lighting exploiting what is typically a considerable network of existing assets – in other words to multi-purpose the ‘humble’ lamppost. The core cities have considerable experience of and plans for smart lighting. The detailed implementation plan for the smart lamppost is summarised in the table below.

Table 2.4.1: Summary of demonstration activities – number of smart lampposts

City Population Total Lamppost LED conversion Demonstration numbers

RBG 0.3m 23,000 100% by ‘20 400

Lisbon 0.5m 66,000 2k in; 3k ’16; 6k ‘16/17 250

Milan 1.3m 140,000 140k ‘15/16 300

Table 2.4.2: Summary of demonstration activities (magnitude and priority where available) – Smart lamppost functions

Function Lisbon London Milan

Wi-Fi, Mobile & Mesh X High x 30 Low

App based wireless control X High 300 High

Environmental sensing (air quality, noise) X High x 2 High

Façade lighting (colours) x

RGBA notification

Digital signage X Low x

Water level/ flood monitoring X High

PV, power for lamp, mobile phone x

Smart lighting – LED X High x 300 High

Smart lighting – Photocell control

Smart lighting – 0-100% dimming X High 300 High

Smart lighting – On-demand lighting X Low

Concealed speakers

Image sensing X High

Push-to-talk system

eV charging x 5 Low

Bat sensors x

Speed x

Traffic and pedestrian movements x

Car Parking x

IoT Gateway (Lorawan/Wireless Mbus) 3 High

The timeframe for lamppost demonstrator implementation in London is:

Lisbon London Milan

Use cases and city data capture 3000 by 2016 and 14000 by 2017

Jan 2016- Sep 2016

Developing Business Cases Jul 2016- Oct 2016

Implementation of symbol lamppost Nov 2016- Dec 206

Procurement Jan 2017- Mar 2017

Deployment of Pilot Apr 2017- Jul 2017

20

Contracting and Implementation at Scale Aug 2017- Dec 2017

Measurement and Evidence Jan 2018- Dec 2018

21

3 MONITORING AND EVALUATION TARGETS The Common Monitoring and Evaluation Framework (CMEF) proposed in this report attempts to cover a wide range of relevant demonstrator impacts. The evaluation targets that are briefly set in this section follow the principles described in section 1 and are categorized into:

technical performance

impacts on attitudes and behaviours

wider systemic impacts including environmental, security, safety and sustainability

institutional and business consequences

economic and social implications including those affected by efficiency, equity and social inclusion

Aside from the anticipated demonstrator impacts, the CMEF proposed covers a wide range of evaluation targets in order to capture a holistic view of demonstrator performance. This includes unintended effects that are more difficult to pin down and inevitably rather open ended. Due to the small scale of the demonstrators, it is anticipated that the wider system level impacts might be difficult or even in cases impossible to truck at a city wide level. Therefore, the CMEF evaluation targets should be focused on both monitoring aggregate performance and more refined effects. A major challenge in delivering the CMEF for SHARING CITIES project is the diverse nature of the demonstrators. Although common thematic areas and work packages have been introduced in the Description of Work, the actual demonstrators (described in chapter 2) make evident that there can be substantial differences across cities even for the same type of demonstrator (e.g., social housing retrofit, commercial building retrofit, private retrofit). To use the data produced via the monitoring processes of different demonstrators in a complementary and comparative way, and to undertake useful analysis, common evaluation targets should be used where possible. Although little similarity is anticipated in evaluation targets dealing with technical performance and impacts on attitudes and behaviours, more common ground exists in wider systemic impacts, institutional and business consequences and economic and social implications (e.g. emission savings, social inclusion). In fact, for “Institutional and business consequences” and “economic and social implications” identical sets of evaluation targets are frequently used across demonstrators, as their primal aim is to reflect on the management and policy decisions made in planning and implementing a demonstrator. A major limitation when attempting to capture city wide business effects, is the great uncertainty associated with the causality of the impact monitored. For example, if an increase in city jobs is recorded after a demonstrator is introduced, cannot be attributed to a demonstrator impact as there are several other influencing factors. Furthermore, to enable the project to deliver a replicable and scalable evaluation framework able to be applied to a wide range of cities, it is essential to identify an appropriate level of demonstrator description that is simultaneously technology agnostic and sufficiently detailed. This enables impact analysis to be conducted both at an individual demonstrator level, and at a city level. 3.1 Building Retrofit The aim of building retrofits is to reduce the energy consumption while maintaining or increasing comfort for occupants. To evaluate a demonstrator’s impact of an individual building, it is necessary to determine its performance before and after intervention. The before performance is of particular significance in assessing retrofit impact as each building will begin at different performance level. Furthermore, the baseline condition of a building is indicative of the amount of room for improvement. For example, a city with an efficient building stock will have less ability to improve its

22

performance than a city with an inefficient building stock. It is also important to point out that energy consumption in some buildings might be low due to its tenants being incapable of paying for the necessary energy to maintain it at comfort conditions. In these cases, building retrofits could instead lead to higher comfort while maintaining the same energy consumption as before. The role a building is used for, can also influence its energy consumption performance as there are different requirements for various types of usage. For example, hospital buildings have strict temperature and humidity requirements leading them to use more energy than a residential building. It is therefore essential to assess a retrofit demonstrator’s impact with respect to the magnitude of improvements possible for a building or set of buildings. 3.1.1 Technical performance Buildings are bespoke systems each having different internal systems to provide comfort. Therefore, it is prudent to have an evaluation framework that is technology agnostic. This allows demonstrators to implement varied technologies that work best for their system. With respect to the technical performance of retrofit demonstrators the aim is to assess the impact of retrofit and green energy equipment installations as described in Tables 2.1.2 and 2.1.3 respectively. Due to the complexity of measuring flat or building energy use, for the evaluation of retrofit technical performance aggregate indicators are more suitable, rather than capturing a specific measures impact. For example, the installation of new wall insulation implies a heat loss reduction that is evaluated by monitoring the energy use before and after the interventions, assuming other influencing factors do not vary. Therefore, the following evaluation targets can be used for assessing the technical performance for retrofit demonstrators at either building or flat levels:

How much energy is used for heating?

How much energy is used for cooling?

How much energy is used for ventilation?

How much energy is used for lighting?

How much energy is used for domestic hot water?

How much energy is used by plug load/ appliances?

How much maintenance is required?

3.1.2 Impacts on attitudes and behaviours Tenants’ and operators’ perception of energy use in buildings is dynamic and can change when energy is supplied more efficiently and at a lower cost. Therefore, the following evaluation targets are considered for demonstrator impacts on attitudes and behaviour of tenants, building operators and other stakeholders: For tenants’ indoor environment quality:

Does the thermal comfort level change?

Does the visual comfort level change?

Does the acoustic comfort level change?

Does the indoor air quality (IAQ) level change?

How satisfied are tenants with the retrofit? For building operators:

Does their perception of system functionality change?

Does their perception of system control change? For retrofit demonstrator policy makers/ stakeholders:

Does their willingness to retrofit change?

23

3.1.3 Wider systemic impacts The buildings retrofit demonstrators are small scale interventions that are anticipated to have limited impact on city sustainability and city wide air quality. Therefore, the wider system impacts evaluation targets considered require to focus on quantifiable impacts, while at the same time be compatible/ comparable with evaluation targets of other demonstrators. In that context, the wider systemic impact evaluation targets considered are: Per buildings:

Does energy use change?

Do environmental emissions change?

Does energy supply become more reliable? Per city:

Does air quality change?

Is there a relief for energy generation capacity?

Are distribution and transmission networks relieved? 3.1.4 Institutional & business consequences Retrofit demonstrators institutional and business consequences are primarily concerned with the management and policy choices made at an institutional level and how these choices reflect business performance. Institutional and business consequences can be defined with respect to demonstrator performance and city performance, although it is recognized that it is difficult to isolate demonstrator impact at a city level. Therefore, the following evaluation targets for buildings retrofit demonstrators focus on capturing management decisions impacts at both levels:

How successful has the demonstrator been financially?

How successful have demonstrator related policies been?

How successful have the procurement mechanisms been?

How has the productivity of the affected area changed?

3.1.5 Economic and social implications With respect to retrofit demonstrators, economic and social implications include the indirect economic and social effects on the local population. As with institutional and business consequences such implications can be captured at a demonstrator specific or city wide levels, although for the latter there is limited clarity on the causality. Therefore, the following evaluation targets are considered focusing on both the demonstrator and the city contexts:

Does demonstrator property value change?

Does household upkeep cost change?

Does local land value change?

Does neighbourhood liveability change?

Does the demonstrator encourage social inclusion? 3.2 Sustainable Energy Management System The aim of Sustainable Energy Management Systems’ (SEMS) is to integrate and optimise (e.g. demand and supply balancing) energy from all local sources in a building/ district (interfacing with other energy systems), and provide a means that supports users in understanding and being incentivised to get informed and be more efficient in their energy consumption. To capture the impact of introducing an SEMS a before and after analysis is required. The level of renewable energy supply, energy management, energy demand response and electric vehicle charging prior to introducing an SEMS at the specific location, influence the potential for improvement and require to be captured.

24

It is worth noting SEMS performance monitoring can be complemented by retrofit monitoring data. Recognising that similar monitoring requirements are described in section 3.1 for buildings retrofit, it is worth establishing a common framework to improve the efficiency of data collection. 3.2.1 Technical performance Energy systems are bespoke, composed of different energy production, distribution and consumption components that SEMSs can dynamically monitor and control to optimise performance. To account for the plethora of technological sub-systems that can compose an SEMS, it is prudent to establish a CMEF for SEMS that is technology agnostic. Evaluation targets for SEMS technical performance are focusing on:

How efficient is heat/ cool supply?

How efficient is electricity supply?

How efficient is gas supply?

How efficient is hot water supply?

How much maintenance is required? 3.2.2 Impacts on attitudes and behaviours Although local residents might have indirect benefits through the introduction of an SEMS system, their interaction with it is very limited as SEMS is primarily focusing on the efficiency of supply rather than the quantity. On the other hand, local energy operators and stakeholders are much more involved and aware of SEMS effects on local energy management, and evaluation targets are considered to capture the impacts on their attitudes and behaviours. For building operators:


Does their perception of system control change (e.g. demand spikes)? For retrofit demonstrator policy makers/ stakeholders:

Does their willingness to install SEMS change? 3.2.3 Wider systemic impacts As discussed in 3.1.3, the wider system evaluation targets considered require to focus on quantifiable impacts, while at the same time be compatible/ comparable with evaluation targets of other demonstrators. Therefore, at city context a similar evaluation target set as in 3.1.3 is used. In that context, the wider systemic impact evaluation targets considered are: SEMS specific:

Does energy efficiency change?

Do environmental emissions change?

Does electricity supply become more reliable?

Does the amount of water leakage reduce? Per city:

Does air quality change?

Is there a relief for energy generation capacity?

Are distribution and transmission networks relieved? 3.2.4 Institutional & business consequences As in 3.1.4, SMES demonstrators institutional and business consequences are primarily concerned with the management and policy choices made at an institutional level and how these choices reflect business performance. Although it is recognized that it is difficult to isolate demonstrator impact at a city level, institutional and business consequences are defined with respect to demonstrator performance and city performance. Therefore, the following evaluation targets for SEMS demonstrators focus on capturing management decisions impacts at both levels:


25



Are more labour force training and specialization required?

How has the productivity of the affected area changed? 3.2.5 Economic and social implications As in 3.1.5, SEMS demonstrators economic and social implications include the indirect economic and social effects on the local population. As with institutional and business consequences such implications can be captured at a demonstrator specific or city wide levels, although for the latter there is limited clarity on the causality. Therefore, the following evaluation targets are considered focusing on both the demonstrator and the city contexts:

Does energy delivery cost change?

Does household upkeep cost change?



Does the demonstrator encourage social inclusion? 3.3 Shared e-Mobility The aim of shared mobility is to improve the transport network performance and to support the shift to low carbon systems, while the aim of electro-mobility is to reduce transportation pollution (particularly in urban areas) while maintaining the convenience performance associated with existing modes. As discussed in Section 2.3, the SHARING CITIES project partners envisage the implementation of various types of mobility systems (e.g. eV car sharing, eBikes, eV charging points, smart parking, logistics, etc.) with different functionality and usage. Table 3.3.0.1 summarizes the information from Table 2.3, categorizing them per demonstrator (rather than per city) to emphasize the similarities and differences across similar demonstrator categories. For example, Milan is interested in utilising eBikes for logistics purposes aside eBike sharing, while Lisbon is interested to introduce dedicated users to its eCar sharing scheme. It is therefore essential to introduce a flexible evaluation framework able to account for the various functions and usage cases. Considering the various demonstrator scales discussed in Section 2.3 the mobility CMEF also requires to be scalable and replicable. As each city has a unique transportation system, it is similarly important to accurately capture the performance of the transport network before the demonstrator implementation. Cities with more developed and functional transport systems have a lower potential for improvement. Therefore, to determine the impact of shared e-mobility demonstrators it is essential to assess transport network performance before and after implementation.

Table 3.3.0.1: Unique mobility demonstrator features per city Milan Lisbon London

eCar share 60 eV charge points/ 10 stations, 60 eVs, "condominium" test vehicles to be charged by PVs

sharing for Municipality workers, EV charging with 3 charge modes, 1 fast charger, "private" eV charging using PVs and SEMS

return to base model, "autonomous eVs": last mile point to point shuttles to connect major transport hubs, self-parking cars, freight vehicles

26

eBike share

eBike reservation/ guaranteed availability, common battery re-charging with eCars, user-based redistribution

park & bike scheme, air pollution based outskirt parking incentive scheme

return to base model

eLogistics 2 vans with on-board tracking, e-bikes, fast recharging at distribution centre

"EDP" for district maintenance activities, "EMEL" for parking meter maintenance and cash collection activities, "CML" for garbage collection and street monitoring

Smart Parking

sensors at parking bays for priority, checking unauthorized parking, user GPS tracking, predictive algorithm for guidance

sensors at parking bays for priority (incl. committed park and eBike users)

eMoto share

"CML" & "EMEL" to be used for parking meters maintenance and cash collection

eBus

Electric Bus for urban environment use

Smart Shared Space

(River Thames) Pedestrian Tunnel rules enforcement using sensors, cameras and messaging

The introduction of shared e-Mobility demonstrators of different types is anticipated to yield common impacts such as reducing car emissions and ownership or encouraging multi-modal trips. Unique impacts are also identifiable across different demonstrators, particularly in terms of technical performance and attitude/ behavioural impacts (e.g. changes in driving style are applicable to eCar Sharing and eLogistics demonstrators). Therefore, as mobility demonstrators evaluation targets can be associated with one or more of the mobility demonstrator sub-categories (i.e. e-Car share, e-Bike share and e-Logistics), a tabular form is used to accommodate this feature. This approach adds flexibility to the evaluation framework, as evaluation targets can be considered or not at specific cities, depending on local needs. For example, in the case of Milan, it is envisaged to introduce integrated charging for eCars and eBikes, while this is not the case for the other two cities. Furthermore, unique impacts can also be identified across same type demonstrators. For example, the familiarity of a driver with the vehicle is relevant only when the scheme is open to the public. In the case of Lisbon, where the e-vehicles will have dedicated users, the CMEF can be made more efficient by removing this evaluation target. For identifying the evaluation targets for mobility demonstrators, shared electro-mobility projects evaluation frameworks and mobility monitoring literature were reviewed (DfT, 2015; JRC, 2014; EVUE, 2012).

27

3.3.1 Technical performance Vehicles and associated mobility infrastructure are bespoke systems each having unique characteristics. Therefore, as in the cases of retrofit and SEMS, it is prudent to have an evaluation framework that is technology agnostic. This allows demonstrators to implement varied technologies that work best for their system.

e-C

ar S

har

e

e-B

ike

Shar

e

eV

-Ch

argi

ng

e-L

ogi

stic

s

Smar

t P

arki

ng

How efficiently are eV being driven? X X X

What is the battery charge level at hire/ drop-off? X X X X How easy is it to use the docking station interface to hire an eV? X X How easy is it to reach a docking station to hire an eV? X X How easy is it to find a parking spot/ docking station/ charging station?

X X X X X

How much are demonstrator vehicles utilized? X X X Is there range anxiety for the users? X X What is the minimum reliable battery charge at hire? X X How much eV rebalancing is required (between empty full stations)?

X X

How accurate are deliveries by eV? X Does performance reliability change? X X X X How much maintenance is required? X X X X X How frequently do vehicles run out of battery? X X X X

3.3.2 Impacts on attitudes and behaviours With respect to mobility demonstrators, impacts on attitude and behaviours are anticipated for users, operators and stakeholders. Therefore, the following evaluation targets are considered:

e-C

ar S

har

e

e-B

ike

Shar

e

eV-C

har

gin

g

e-Lo

gist

ics

Smar

t P

arki

ng

Does car ownership change? X X X X

Does citizens’ level of mobility change? X X X Does electro-mobility demand change? X X Does trip distance distribution/ average change? X X X X Does trip purpose change? X X Is the trip mode choice influenced? X X X X Does vehicle occupancy change? X Can users easily involve eVs in multi-modal trips? X X X Do route choice criteria change (between simpler, faster, shorter)?

X X X X

Does driving style change (aggressive/ eco-friendly)? X X X Do users comply with safety rules? X

28

Is there shared electro-mobility awareness across citizens? X X X X X Is there shared electro-mobility familiarity across citizens? X X X X X How satisfied are citizens with demonstrator? X X X X X Do policy makers favour similar smart-mobility investments? X X X X X

3.3.3 Wider systemic impacts At a wider systemic level, mobility demonstrator impacts concern the performance of the entire transportation network. The modes introduced can freely use all transport infrastructure whose performance requires to be captured, while taking into account the wide variety of causes that can influence city wide indicators and the associated uncertainty. The city-wide mobility evaluation targets considered, attempt to exploit the common ground with evaluation targets presented in sections 3.1.3. and 3.2.3.

e-C

ar S

har

e

e-B

ike

Shar

e

eV

-Ch

argi

ng

e-L

ogi

stic

s

Smar

t P

arki

ng

Do local environmental emissions change? X X X X X

Do global environmental emissions change? X X X X

Does air quality change? X X X X X

Does local noise pollution change? X X X Does mobility become safer? X X X X Does road congestion change? X X X X Does asset deterioration/ maintenance change? X X X

3.3.4 Institutional & business consequences As in 3.1.4 and 3.2.4, mobility demonstrators’ institutional and business consequences are primarily concerned with the management and policy choices made at an institutional level and how these choices reflect business performance. Although it is recognized that it is difficult to isolate demonstrator impact at a city level, institutional and business consequences are defined with respect to demonstrator and city performance. Therefore, the following evaluation targets for mobility demonstrators are considered for all mobility demonstrator sub-categories:




How has the productivity of the affected area changed? 3.3.5 Economic and social implications As in 3.1.5 and 3.2.5, mobility demonstrators’ economic and social implications include the indirect economic and social effects on the local population. As with institutional and business consequences such implications can be captured at a demonstrator specific or city wide levels, although for the latter there is limited clarity on the causality. Therefore, the following evaluation targets are considered focusing on both the demonstrator and the city contexts:

Does the generalized cost of travel change?



Does the demonstrator encourage social inclusion?

29

3.4 Lamppost Aside from the anticipated installation of more energy efficient LED lighting, as discussed in Section 2 (Table 2.4.2), streetlamps can accommodate several functions to contribute towards an improved urban efficiency and performance. Lampposts are relatively simple city assets that all too often are purchased at an individual city level in relatively low volumes (to often bespoke specifications). An integrated function lamppost is by nature a highly replicable and scalable solution that can have a modular form to accommodate functions depending on local needs. Therefore, the impacts of a lamppost are highly open-ended as they are module/ function based. The evaluation targets proposed, focus on the fundamental functions of a lamppost (i.e. LED lighting and light dimming) and its utilization level as street furniture. Evaluation targets can be further extended to accommodate targets for individual functions. For example, for eV charging capability of lamp posts, the evaluation targets presented in section 3.3 on eV charging can be used. 3.4.1 Technical performance The technical performance evaluation targets of lamp posts considered focus on lighting and adaptive light control (dimming).

How much energy is consumed for lighting?

Is lighting provided sufficient?

How accurate are the data collected?

How many modular functions are accommodated?

How much maintenance is required? 3.4.2 Impacts on attitudes and behaviours Lamp posts LED lighting and light dimming are anticipated to have an impact on attitudes and behaviours of citizens, operators and stakeholders. The evaluation targets considered monitor: For residents:

How satisfied are residents’?

How satisfied are visitors?

Does lighting effect route choice in walk trips?

Does lighting effect route choice in vehicle trips? For operators:


Does their perception of system control change? For local policy makers/ stakeholders:

Does their willingness to install new smart lamp posts change? 3.4.3 Wider systemic impact The lamp posts demonstrators wide systemic impact evaluation targets require to focus on quantifiable impacts, while at the same time be compatible/ comparable with evaluation targets of other demonstrators. In that context, the wider systemic impact evaluation targets considered are: Lamp post specific:

Is road safety influenced?

Does local criminality change? City-wide:

Does lighting energy efficiency change? 3.4.4 Institutional & business consequences Lamp post demonstrators’ institutional and business consequences are limited when only LED lighting and light dimming functions are considered. Therefore, the following evaluation targets for lamp posts act primarily as a feedback loop for future demonstrators:

30


How has the productivity of the affected area changed?

3.4.5 Economic and social implications With respect to lamp posts demonstrators, economic and social implications include the indirect economic and social effects on the local population. Therefore, the following evaluation targets are considered:

Does street lighting upkeep cost change?



Does the demonstrator encourage social inclusion?

31

4 MEASURABLE INDICATORS DEFINITIONS AND INFLUENCING FACTORS For each of the targets presented in chapter 3, this chapter discusses how they can be quantified using measurable indicators. As discussed in section 1.3 measurable indicators of quantitative nature are adequate for monitoring adoption and use of a demonstrator, however qualitative indicators might also be required for capturing unquantifiable impacts, such as regulatory framework changes and business relationships. In case an evaluation target described in chapter 3 cannot be measured directly, estimation models require to be considered for capturing its performance as accurately as possible. A major challenge in undertaking the evaluation task is that many of the demonstrators will be small scale, so their direct measurable impacts will be minimal. In such cases, instead of relying on raw data collection, other indicators and measurement quantities require to be defined, that when linked to suitable modelling assumptions and estimation models can yield sufficiently accurate evaluations of impact at a city wide level. The additional input data required by those estimation models, are also considered. Another major challenge, is the appropriate association of impacts recorded with causes. This problem is particularly important when dealing with city-wide evaluation targets, where the effects of several demonstrators might emerge simultaneously. The challenge of associating impacts with causes extends even further, to external influencing factors that might create bias in the results. To deal with this evaluation problem a comprehensive list of influencing factors is introduced for each evaluation target, aiming to minimise evaluation biases. This chapter initially discussed the assessment methods available for data collection and monitoring, attempting to identify their strengths and weaknesses. Also, provided the plethora of demonstrators and evaluation targets in chapter 3, it discusses how various assessment methods can be used complementarily both to utilize direct monitoring and to feed data to analytic models. The latter section of this chapter, defines the measurable indicators for each evaluation target, alongside data standards and influencing factors that might create evaluation biases. Finally, for each demonstrator the data required are summarized, so that they can be easily compared with data sources that are discussed in the following chapter. 4.1 Assessment methods Several data collection and analytic methods are available for answering questions defined in chapter 3. The assessment methods applied depend on the specific requirements of each evaluation targets and will also vary according to the context and requirements of each specific demonstrators and the city they are applied in. Assessment methods associated with the evaluation targets presented in chapter 3 include:

Monitoring

Experience surveys

Process evaluation

Modelling (large scale) impacts Each assessment method and data requirements have strong links between thematic parts of the assessment framework. For example, modelling primarily refers to the use of transport and energy distribution models capable of capturing the system level impacts of a demonstrator, when complete and accurate city-wide data are not available.

32

4.1.1 Monitoring Monitoring of technical and operational parameters is required yield the data necessary for each evaluation target in order to assess all demonstrator impacts. Monitoring data are also required for running models in order to capture system level impacts. Although, most technical parameters of each demonstrator can be monitored after demonstrator implementation, as discussed in the introduction of chapter 3, it is also required to establish each demonstrator’s baseline conditions. Therefore, before demonstrator implementation data are also essential. For example, comparable before and after data require to be collected on:

the comfort temperature of residents before and after retrofit is applied, and

the electricity consumption of lampposts before and after new lights installation. Depending on the variety of data sources available, more than one measurable indicators might be adequate for capturing the impact associated with a specific evaluation target, or more than one data collection methods might be available for quantifying a measurable indicator. The tables of measurable indicators presented in this section attempt to capture all possible data collection streams available to add robustness to the evaluation framework. For instance, if GPS data are collected and combined with an appropriate map matching technique, they can replace the data from the odometer of a vehicle when monitoring eVs trip distance. When more than one data streams are available for assessing an evaluation target, the evaluation efficiency and accuracy require to be considered for assessing its usefulness. Although, it is reasonable to remove the secondary data stream to make the evaluation process more efficient, it is essential to assess its usefulness with respect to the possibility of using the extra piece of information to reduce evaluation biases. Table 4.1.1 illustrates how measurable indicators are defined for each evaluation target, and how more than one measurable indicators and data collection methods might be available. Such monitoring data can be analysed and combined to reduce evaluation biases and to assess a demonstrator’s impact (per evaluation target) more accurately.

Table 4.1.1: Analysis of monitoring data

Evaluation target Measurable indicator(s) Data collection method(s)

Route choice Path distance/ Straight line distance

On-Board GPS

Vehicle logger data

Path travel time/ Total distance

On-Board GPS

Station logger data

Driving style behaviour

Distance driven per battery energy used

Vehicle logger data & Station logger data

For shared electric vehicles typical data sets used in other projects and research literature (Aunedi M. et al, 2014; Corchero C., 2014)) include:

The vehicle logger data that provide information on: o Vehicle ID o Start and end times o Start and end address o Distance travelled o Average and maximum speed o Start and end battery State of Charge (SoC) o Energy transferred during charge o State of heating/ AC

Docking station logger data: o Timestamps for charging start and completion o Energy transferred during charging event

33

o Charging Network Operator ID, charging point ID and plug ID o Vehicle ID

4.1.2 Experience surveys Experience survey data are collected in order to evaluate the wider attitudinal and social impacts of demonstrators. Surveys are suitable for quantifying happiness, familiarity and perception evaluation targets from the perspectives of users, operators and stakeholders. For example, user surveys can yield information on evaluation targets such as:

Changes in driving style/ routing of eVs and eBikes drivers;

Changes in the perception of citizens on ease of access/ connectivity for mobility demonstrators; and

Changes in the definition of comfort and energy use after a building retrofit. For operators and decision makers, surveys can yield information on evaluation targets such as:

The willingness to encourage a specific mobility solution through policy framework;

The perception of operators on a demonstrator and their willingness to invest further Surveys require to be carefully planned and designed, as the quality of response is very sensitive to the willingness of the person surveyed to participate. Past survey experience (Willmack et al., 1995) suggests that long surveys and difficult to comprehend survey questions yield lower quality data, while response incentives improve data quality. As in the case of monitoring, surveys can be conducted “before” and “after” the introduction of a scheme, as for most demonstrators’ evaluation targets it is required to establish the baseline condition. The survey format will mostly be self-completion survey forms, although telephone interviews and face-to-face interviews may be carried out if necessary. For ease of data collection, the preferable way to carry out a survey is to use on-line survey forms. Translation will be provided if the respondents are not English speakers. Surveys can be designed to target various audiences. In the context of the Sharing Cities project demonstrators, a number of key respondents are identified below:

Local residents,

Local visitors,

Scheme users,

Operators, and Policy makers and other stakeholders.

4.1.3 Process evaluation The process evaluation is performed in order to analyse and quantify the implementation of a demonstrator and to highlight the problems and success areas. Such data can prove useful for analyzing the scale-up and replication potential of demonstrators that is an activity led by WP5. Process evaluation information of the evaluation framework can potentially act as inputs for the analysis of governance, procurement and policy making. The main goal of the process evaluation is to develop new findings of factors of success of the demonstrator and to define strategies to overcome possible barriers hampering implementation. The three implementation stages of each demonstrator that can be considered in the process evaluation:

1. Planning and preparation of individual demonstration projects: during this stage all the preparatory work to actually start demonstration are taking place (demonstration planning process is developed in detail, choice of assets and infrastructure is done, operational processes to be implemented are discussed with relevant stakeholders, ICT solutions are

34

defined, etc.). At the end of this phase all planning details are fixed, including all decisions and permissions that are a pre-conditioning for procuring and implementing the demonstrator.

2. Implementation phase: consists of purchase of assets (where applicable) and additional infrastructure, installation and or construction and approval of regulation measures (where applicable), and

3. Operational phase: where the demonstrator scheme is running and is available to the public. 4.1.4 Modelling system impacts Based on the evaluation targets presented in chapter 3 for wider systemic impacts, various modelling approaches can be utilized depending on data availability from each demonstrator, such as:

Descriptive statistics (direct impacts analysis) –for all demonstrators; Scenarios/Sensitivity tests (systemic impacts at different market penetration levels) - by traffic

simulation modelling if a traffic model is available, and by energy distribution/ efficiency models, and

Where suitable data available, monetize the systemic impacts. To monetise the system impacts, standard transport appraisal methodologies are available that consider the value of time savings, the value of life etc. Such models can be used, provided suitable data sets are made available per demonstrators. Furthermore, descriptive statistics can be used to analyse the direct systemic impacts of introduced shared e-mobility on the transport network and the environment. To overcome the small scale deployment, a local or regional traffic, simulation models can be used in order to estimate wider systemic and environmental impacts, such as network congestion and air quality. Similarly, a small scale energy distribution simulation model can be used in order to estimate wider system and environmental impacts of building retrofit and SEMS demonstrators. The system impact modelling is significant for capturing the impact of solutions considered in larger scale interventions, as well as the scalability and replicability of each demonstrator examined by WP5. The evaluation targets that can be analysed using traffic and energy distribution models include: For mobility demonstrators:

Traffic congestion, based on the amount of shared electric-mobility, and

Local CO2 emission, based on shared electric-mobility utilization level. For buildings retrofit and SEMS demonstrators:

Energy distribution efficiency, based on energy use, transmission and energy generation infrastructure.

4.2 Measurable indicators In this section, a list of possible measurable indicators that can be used to assess the proposed evaluation targets is listed and characterized in terms of their units and possible data collection methods that can be employed to monitor them. The partner cities of the project will choose some of the listed indicators that better fit with local and national requirements, standards and technical constraints. Moreover, since the project is still at an early stage, new indicators may be proposed and used, at the operational phase, that will integrate with the ones listed in Section 4.2. The table is therefore to be considered as a developing tool that will change along the project, following local requirements. The measurable indicators presented in this section focus on evaluation targets on technical performance, impacts on attitudes and behaviours and wider systemic impacts. 4.2.1 Building retrofit

Evaluation target

Measurable indicator(s)

Unit Data collection method(s)

35

Energy used for heating

Primary energy kWh Delivered energy + primary energy factor

Delivered energy

kWh,

m3,kg Gas meter, flow meter, barrels delivered, pellets delivered, electrical energy meters

Energy delivered by the generation system kWh Temperature sensors + Flow meters

Energy used for cooling


Delivered energy kWh, m3,kg Electrical energy meter, gas meter


Energy used for ventilation


Delivered energy kWh Electrical energy meter

Energy used for lighting


Delivered energy kWh Electrical energy meter

Energy used for domestic hot water


Delivered energy kWh, m3,kg

Gas meter, flow meter, barrels delivered, pellets delivered, electricity meters


Energy used by plug load/ appliances

Electric energy kWh Electrical energy meter

Electric energy appliances kWh Electrical energy meter

Performance reliability

Minor repair per year Operator data

Major repair per year Operator data

Tenants thermal comfort level

Operative temperature oC Temperature sensors

PMV Temperature & RH sensors, anemometer

PPD Temperature & RH sensors, anemometer

Tenants visual comfort level Illuminance

Lux (lm/m2) Light sensor

Tenants acoustic comfort level Sound Pressure Level dB(A) Phonometer

Tenants indoor air quality level

CO, NOx, PM concentration μg/m3 Air pollutant sensor

Tenants satisfaction Grade 1-5 Tenants survey

Operator perception of system functionality Grade 1-5 Operators survey

Operators perception of system control Grade 1-5 Operators survey

Stakeholder willingness to retrofit Grade 1-5 Stakeholders survey

36

Building energy supply reliability

Frequency of blackouts Operator data

Air pollution Pollutants emitted (NOx, PM) kg Emission model

City energy generation relief

Generation capacity factor % Energy model

City electricity networks infrastructure relief

Distribution network capacity % Energy model

Transition network capacity % Energy model

It is noted that for measuring primary energy To accurately quantify the effects of retrofit demonstrators it is required to capture or monitor (if applicable) several influencing factors that can potentially create biases in the data. The influencing factors for retrofit evaluation targets and measurable indicators include:

Building size, including: o Floor area, and o Height,

Local weather conditions, including: o Temperature, o Humidity, o Wind speed, and o Precipitation,

Building occupancy and equipment, and

Building characteristics. Summarizing the data collection methods for all evaluation targets and measurable indicators, the data required per retrofit demonstrator include:

Energy monitoring via electricity meters (including amount of locally generated energy) per building function,

Delivered (physically) energy monitoring via gas meters, flow meters, barrels and pellets, per building function,

Temperature and RH sensors, anemometer

Air pollutant sensor(s)

Operational data for reliability measurements

Tenants, operators and stakeholders survey 4.2.2 SEMS

Evaluation target Measurable indicator Unit Data collection method

Efficiency of heat/cool supply Utilization of local heat used % System logger data

Efficiency of electricity supply

Local production used % System logger data

Green production used % System logger data

Substation thermal constraint breaches System logger data

Voltage stability System logger data

Efficiency of gas supply Energy used kWh System logger data

37

Efficiency of hot water supply Energy used kWh System logger data

Performance reliability

Electricity blackouts Hours/year Operational data

Quantity Operational data

Heat pump system out Hours/year Operational data

Electricity substation thermal constraint breaches Quantity Operational data

Operator perception of system functionality Grade (1-5) Operator survey

Operators perception of system control (e.g. demand spikes) Grade (1-5) Operator survey

Stakeholder willingness to retrofit Grade (1-5) Stakeholder survey

Energy efficiency

Utilization of local resources % System logger data

Utilization of green resources % System logger data

Energy used from storage? kWh System logger data

Energy supply reliability Frequency of supply shortage Operator data

Leakage

Water volume m3 Operator data

Gas volume m3 Operator data

Air pollution Pollutants emitted (NOx, PM) kg Emission model

City energy generation relief Generation capacity Operator data

City distribution and transition networks infrastructure relief

Distribution network capacity Operator data

Transition network capacity Operator data

To accurately quantify the effects of SEMS demonstrators it is required to capture or monitor (if applicable) several influencing factors that can potentially create biases in the data. The influencing factors for SEMS evaluation targets and measurable indicators include:

Building/ district electricity consumption

Gas supply pressure

Water supply flow rate

Heat supply delivery temperature

Heat-pump efficiency

Heat-exchanger efficiency of building/ district and flat (if applicable)

District thermal consumption

Building EPC rating

Energy use (per m2)

Local weather conditions, including: o Temperature, o Humidity, o Wind speed, and o Precipitation, and

SEMS asset inventory Summarizing the data collection methods for all evaluation targets and measurable indicators, the following data are required per SEMS demonstrator:

System logger data

Operational data (including for reliability and capacity measurements)

38

Operator and stakeholders survey 4.2.3 Mobility

Evaluation target Measurable indicator Unit

Data collection method

Distribution of eV user drive style energy efficiency Energy consumption per km miles/ kWh

Vehicle data logger (distance, energy consumed)

Distribution of battery charge level at hire/ drop-off

Battery fullness at hire and drop-off % or kWh

Vehicle data logger/ Station data logger

Easy of hire - Docking station user interface Duration of hire/ drop-off time User survey

Ease of hire - Station location Distance/ Time to station

time (minutes) or distance (km) User survey

Ease of finding a parking spot/ charging/ refuelling station

Time spent/ distance driven in search of charging station

minutes (or km) / trip (or per user)

User destination information (User survey or WP4 platform) & vehicle route from on board GPS

Time spent/ distance driven in search of parking station

minutes (or km) / trip (or per user)

User destination information (User survey or WP4 platform) & vehicle route from on board GPS

Vehicle utilization

Distribution of (or not) use (w.r.t. time) - w.r.t. demand time/time (i.e. %) Station data logger

Duration vehicle is available (not charging) time/time Station data logger

Frequency of vehicle use Users/ Hires per day Station data logger

Range anxiety battery charge @ hire /(over trip) trip distance kWh/km

Vehicle data logger with GPS and charge level from charging station logger/ (+ User survey)

Minimum reliable battery charge at hire

range anxiety metric / average trip distance kWh (%) Usage model

eVs rebalancing (full/empty docking stations) eVs repositioned per day eVs/day

Operator survey (Vehicle data logger?)

Arrival accuracy in deliveries On time delivery success rate % User survey

Performance reliability Frequency of failure Miles drove per failure Operator survey

Maintenance need

Frequency of minor repair Time (or km) between repairs Operator survey

Frequency of major repair Time (or km) between repairs Operator survey

39

Time a vehicle is not available for service for repair purposes % Operator survey

Battery half-life Battery capacity (kWh) w.r.t time

Vehicle data logger/ Station data logger

How frequently do cars run out of battery? Mobility Charging Units calls calls /month Operator survey

Car ownership

Vehicles per citizen (or household)

Number of vehicles Citizen/ User survey

Level/ Amount of mobility Distance travelled km/ user (or day) Citizen/ User survey

Trips generated trips/user/day (or year)

eMobility demand

How frequently potential users log on to the online platform to check vehicle condition

Online platform visitors WP4 Platform data

Distance per trip

Distribution /Average trip distance km

User survey

Data logger & GPS info

Odometer, Docking time, Starting & finishing station, model

Trip purpose Trip intention (commute, leisure, exercise)

Number of trips for each category User survey

Travel mode choice/ Mode replacement survey Modal split

Trips / vehicle type

User survey/ eV usage data

Vehicle occupancy

Distribution /Average number of occupants per vehicle

occupants/ vehicle User survey

Ease of use - Users that include eV in multimodal trips Multimodal trips/ All trips % User survey

Route choice criteria - choice between simpler, faster, shorter route User route choice intention

Number of trips for each category User survey

Route features comparison (directness, travel time, etc.)

GPS & map overlay/ GPS enhanced user survey

Driving style (aggressive / eco-friendly)

Drive cycle (focus on acceleration/ deceleration)

User survey

On-board sensor/ GPS enhanced survey/ model

40

Safety rule compliance

Helmet use % of users wearing helmet

Docking station sensor/ User survey

Number of collisions/ traffic incidents incidents per year Operator data

Tripping hazard from charging cables incidents per year Operator data

Shared eMobility awareness

Awareness of mobility - options available Grade (1-5)

Citizen survey/ User survey

Awareness of environmental friendly mobility benefits Grade (1-5)


Shared eMobility familiarity

User familiarity with eVehicle/ smart mobility features Grade (1-5)


User familiarity with shared mobility features Grade (1-5)


Operator familiarity with shared eVehicle features and performance Grade (1-5) Operator survey

Willingness to use eVehicle

Users registered in online platform

Number of registrations

Operator data/ User survey

How satisfied are people with demonstrator/ service Satisfaction level Grade (1-5) User survey

Policy makers response to eMobility demonstrators

Intention to invest further Grade (1-5) Users registered? /Stakeholder survey

Intention to introduce supportive policies Grade (1-5) Stakeholder survey

Local emissions

Emission free vehicle distance driven km Usage data

Pollutants emitted (NOx, PM) kg Emission model

CO2 kg Emission model

Global emissions

Distance driven now compared to distance driven normally Usage data

Noise pollution Level on street noise dB Noise assessment

Safe mobility Recorded incidents incidents/mile travelled Police data

Distribution of congestion level

Travel time Travel time/ trip Model

Flow veh/h Model

Asset deterioration/ maintenance requirements

Road maintenance budget £ Operator survey

Total distance travelled km Model

To accurately quantify the effects of mobility demonstrators it is required to capture or monitor (if applicable) several influencing factors that can potentially create biases in the data. The influencing factors for mobility evaluation targets and measurable indicators include:

Local traffic congestion

Vehicle performance features, including o Vehicle weight o Vehicle load

41

Terrain flatness

Station specs, including o Charging for batteries, o Density o Proximity to street and other transport modes

eV Noise Vibration Harshness

Travel demand seasonality (per mode), and


Summarizing the data collection methods for all evaluation targets and measurable indicators, the following data are required per mobility demonstrator include:

Vehicle data logger, including o Distance o Energy consumed o Vehicle route (via GPS) o Battery charge o Odometer o Speed

Station data logger o Time of hire/ return o Location o Energy per charge

WP4 platform usage data

Noise assessment

Operator data (including maintenance, and safety)

Users, operators and stakeholders survey 4.2.4 Lamppost

Evaluation target Measurable indicator Unit Data collection method

How much energy is consumed for lighting? Energy use

Lamppost logger data

Is lighting provided sufficient? Luminescence Lux (lm/m2) Light sensor

How accurate are the data collected?

How many modular functionalities are accommodated? Amount of functions


How much maintenance is required?

Frequency of minor repair

Time (or km) between repairs Operator survey

Frequency of major repair

Time (or km) between repairs Operator survey

How satisfied are residents? Grade (1-5) User survey

How satisfied are visitors? Grade (1-5) User survey

Does lighting effect route choice in walk trips?

Path directness i.e. path distance / straight line distance % User survey

42

Does lighting effect route choice in vehicle trips?

Path directness i.e. path distance / straight line distance % User survey

Does operator perception of system functionality change? Grade (1-5) Operator survey

Does operator perception of system control change? Grade (1-5) Operator survey

Does stakeholder willingness to install new smart lamp posts change? Grade (1-5)

Stakeholder survey

Is road safety influenced? Safety incidents Municipality data

Does local criminality change? Criminal incidents Municipality data

Does lighting energy efficiency change?

Energy use per Illuminance provided kWh/Lux


To accurately quantify the effects of lamppost demonstrators it is required to capture or monitor (if applicable) several influencing factors that can potentially create biases in the data. The influencing factors for lampposts evaluation targets and measurable indicators include:

Safety incident severity,

Criminal incident severity, and


Summarizing the data collection methods for all evaluation targets and measurable indicators, the following data are required per mobility demonstrator include:

Lamppost logger data, including o Energy used, and O Active modules

Safety data

User, operator and stakeholder survey

43

5 INITIAL APPRAISAL OF EXISTING AND POTENTIAL DATA SOURCES This chapter provides an initial overview and appraisal of existing and potential data resources. The aim is both to understand what relevant data are currently available and what data require to be collected as part of their normal operation of each demonstrator under ideal circumstances. As the demonstrator features are not finalised, this chapter presents an initial take on data availability and further data requirements that require to be fully described in the local implementation plans. 5.1 Data sources The aim of this section is to associate each measurable indicator and data collection method with detailed information of the data source that is technologically capable of providing them. For example, the trip distance of an eCar can be measured either by using the vehicle odometer and recording the data each time it docks at a charging station, or by using a GPS device. To gather information on the data sources available for each demonstrator, each city was requested to provide specific data on the data collection infrastructure features, the data available and anticipated limitation in the data collection process to follow. 5.1.1 Buildings retrofit

City Demonstrator sub-category

Data sources/ technology of data collection

Data available Limitations in data collection

Lisb

on

Public Housing EDP will collect previous energy consumption data and compare with the new energy consumption after the retrofitting. The new energy consumption will be given by the smart meters

None at the moment Needy people don’t use the energy they need, they use the energy they can afford. Before/After energy consumption comparison may not indicate energy consumption savings.

Public Offices There is no previous energy consumption data to compare with the new energy consumption after the retrofitting. The new energy consumption will be given by the smart meters. The old one will have to be estimated.

None No previous energy consumption data.

Public Offices – Window Replacement

None

No previous energy consumption data. No data can be collected until all the works on the building are finished (see previous task – Lisbon Public Offices) and the people start to use them.

Private Housing

TBD

TBD TBD

44

Lon

do

n

Mixed public Smart Meters, temperature sensors outside blocks, installation of renewable technology

kWh electricity and heat consumption by a whole block; kWh electricity and heat consumption by each household/unit (TBC); Humidity levels in each household/unit in percentage figure (TBC); Measurements of temperature at each housing estate; kWh electricity generation from renewable technologies e.g. solar (Gross and Net figure – if RSHP is installed this will consume electricity); Interruptions to heat supply (hrs or %); Carbon Intensity of Heat Delivered (g CO2/kWh); Seasonal Performance Factor (SPF) – Observed/measured efficiency of RSHP in converting electricity into heat. Possible also to collect qualitative data- E.g. resident comfort/perception, ease/frequency of use of new systems

Heat/gas consumption is not currently collected at household/unit level No existing humidity or temperature measurements – comparison is not possible

Mila

n

Private residential

On site monitoring / smart meters, energy flow meters, environmental sensors, etc. Energy Audit Surveys/Interviews

Social Housing

45

5.1.2 SEMS




Lisb

on

Lon

do

n

Installation of a water source heat pump in the Thames to supply heat to local housing estates

Performance of heat network/ energy output in KWh; Data on the cost of energy production, and the cost to consume energy produced from the heat network

Mila

n

Monet (a Siemens platform) supports local gateway with a different protocol (Modbus, 104, etc.) and a MQTT protocol to connect directly to Monet in cloud environment.

the list is still under definition, based on field availability.

5.1.3 Mobility




Lisb

on

e-bike share Docking stations controllers (via EMEL backend) EMEL backend analytics

Location of docking stations Real-time availability of bikes Aggregated origin-destination matrices Number of rides Number of users

No real-time information on bike location when rented

EDP eLogistics

MDCs to be installed in vehicles collecting data available on the CAN bus

Real-time (among others) -GPS Location; Speed; Odometer; Battery State of Charge Aggregated indicators -Distance Travelled; Energy Consumed; Cost of Energy; CO2 saved

Currently not clear whether individual users shall be identifiable

EMEL eLogistics



Currently not clear whether individual users shall be identifiable

CML eLogistics



Currently not clear whether individual users shall be identifiable.

46

CML corporate eCar sharing

MDCs to be installed in vehicles collecting data available on the CAN bus Remote vehicle control, namely enabling the automatic car-sharing use cases

User profile Real-time (among others) -GPS Location; Speed; Odometer; Battery State of Charge Aggregated indicators -Distance Travelled; Energy Consumed; Cost of Energy; CO2 saved

N/A

EMEL Smart Parking

Parking sensors (Different technologies)

Number of parking spaces available Occupancy time per parking space (when applicable)

EDP Public Charging

Connected public charging stations

User identification (account / vehicle); Energy consumed; Charge time; Electric sector emissions

N/A

EDP Private Changing

Connected charging stations with OCPP – Open Charge Point Protocol version 1.6 (at least)

Available power to charge; Energy consumed; Charge time; User profile; Electric sector emissions

N/A

CML Private Charging

Connected charging stations with OCPP – Open Charge Point Protocol version 1.6 (at least)

Available power to charge; Energy consumed; Charge time; User profile; Electric sector emissions

N/A

CML eBus MDC (or equivalent)


Still to be defined whether the number of users currently onboard shall be considered

Lon

do

n

Foot Tunnels, Parking Sensors, e-Bike and e-Car sensors

Footfall data on usage of the foot tunnel;

Data on the availability of car parking spaces,

and use of spaces; Data on the availability of

e-Vehicles and their status

Mila

n

eBike sharing aggregated data are currently collected by AMAT and will be shared with WP4 platform (CEFRIEL)

info for each pick-up, number of registered users

no data available in real time

47

eCar sharing aggregated data are currently collected by AMAT and will be shared with WP4 platform (CEFRIEL)

info for each pick-up, paths, number of registered users

no data available

in real time

eLogistics data will be

collected by WP4

platform

(CEFRIEL)

paths, number of deliveries for different time slots, number of deliveries for different days, etc.

N/A

Smart parking data will be

managed by

Kiunsys and

shared with WP4

platform

(CEFRIEL)

number of uses for different time slots, number of uses for different days, etc.

N/A

eV charging points

data will be managed by SEMS (Siemens) and shared with WP4 platform (CEFRIEL)

number of uses for different time slots, number of uses for different days, energy provided for different time slots, etc.

N/A

5.1.4 Lamppost




Lisb

on

Lon

do

n

CMS is the

expected data

collection

method

Energy usage. The rest of the data depends on the sensors installed

Energy usage per lamp-post is not

currently metered. RBG is billed by

Npower(?) based on expected usage

per year x number of street lights

Mila

n Lorawan 1/4/17

5.2 Data gap analysis For the gap analysis a direct comparison of data requirements from Section 4 and data sources and data availability from Section 5.1 is undertaken for each demonstrator. The aim is to identify gaps and normalize the data collection process across all cities. In the tables below “X” marks that the data requirements described in section 4.2 are covered by the data provision plans described in section 5.1.

48

5.2.1 Building retrofit

Data collection requirements

Lisb

on

Lon

do

n

Mila

n

Energy monitoring via electricity meters (including amount of locally generated energy) per building function

X X X

Delivered (physically) energy monitoring via gas meters, flow meters, barrels and pellets, per building function

X

Temperature and RH sensors, anemometer X X

Air pollutant sensor X X

Operational data for reliability measurements X X

Tenants, Operators and Stakeholders survey X X

5.2.2 SEMS


Lisb

on

Lon

do

n

Mila

n

System logger data X X

Operational data (including for reliability and capacity measurements)

Operator and stakeholders survey

5.2.3 Mobility


Lisb

on

Lon

do

n

Mila

n

Vehicle data logger: Distance X X

Vehicle data logger: Energy consumed X

Vehicle data logger: Vehicle route (via GPS) X X

Vehicle data logger: Battery charge X X

Vehicle data logger: Odometer X

Vehicle data logger: Speed X

Station data logger: Time of hire/ return X X X

Station data logger: Location X X X

Station data logger: Energy per charge X X

WP4 platform usage data

Noise assessment

Operator data (including maintenance, and safety) X

Users, operators and stakeholders survey

5.2.4 Lamppost


Lisb

on

Lon

do

n

Mila

n

Energy used X

Active modules

Safety data

User, operator and stakeholder survey

49

5.3 Findings discussion The gap analysis undertaken yields two main findings with respect to the data collection process and the performance assessment of the demonstrators. 1. As shown in all demonstrator tables presented in Section 5.2, there is a significant gap between

the CMEF desirable evaluation targets presented in Sections 3 and 4, and the data available presented in Section 5.1. This implies that cities current data collection plans require to be further expanded to cover more evaluation targets, which will enable a comprehensive demonstrator assessment.

2. The demonstrator tables presented in Section 5.1 also reveal that there are differences in the data collection detail for similar demonstrators across cities. Although, each demonstrator is recognized to have a unique nature, it is important to stress the need for “commonality” of the evaluation framework, as the success of each demonstrator will be associated with each city’s specific features. Furthermore, as similar solutions are anticipated to be deployed in other cities, a common monitoring and evaluation framework is required to be replicable itself.

Concluding, it is worth noting that as the demonstrators planning matures, it is becoming increasingly important to specify a sufficiently thorough and comprehensive data collection process aiming to:

Minimise data discrepancies across cities (or allow them where there are reliable methods for converting data to the required form), and

Ensure sufficient data collection equipment is in place w.r.t. a collection of frequent, accurate and complete data.

50

6 CONCLUDING REMARKS AND FUTURE STEPS Based on the core monitoring and evaluation principles described in Section 1.3, this report delivers a “Common Monitoring and Evaluation Framework” (CMEF), establishing a core of evaluation targets, measurable indicators and data collection methods for all partner cities, taking into account the individualities and unique features of each demonstrator. Chapter 2 provides a summary of each of the demonstrator activities for each city, based on key information such as type, location, scale, technologies, etc. which are important for performance assessment. In chapter 3, the evaluation targets for each demonstrator are stated, including both desired outcomes (e.g. improved air quality and car ownership reduction) and collateral or unintended effects. For each of the evaluation targets, quantitative indicators and corresponding measurement quantities are introduced in chapter 4, while covariates potentially influencing them are also considered. Chapter 5 provides an initial overview and appraisal of the existing and potential data resources. According to the data provided by partner cities through the info proforma’s, each city anticipates the monitoring and evaluation of several evaluation targets, through various data sources. The CMEF presented in this report attempts to align the evaluation targets and data sources for all cities, to enable complementary and comparative analysis. It is shown that although there are some commonly anticipated evaluation targets from all cities, there are considerable data availability discrepancies across partner cities. Follow up work will be based on the CMEF presented in this report, to develop specific data collection methods and instruments (“protocols”) for the core and site specific research targets in each city. These specific protocols will take into account considerations of local context and language (including relevant local covariates) and will be in a form that can be deployed directly in the relevant cities.

51

7 REFERENCES Anuendi, M., Woolf, M., Bilton, M. and G. Strbac, 2014. Impact and opportunities for wide-scale electric vehicle deployment, Report B1 for the “Low Carbon London” LCNF project, Imperial College London, UK. Department for Transport (DfT), 2015. Carplus annual survey of car clubs. Report. Leeds, UK. Electric Vehicles in Urban Europe (EVUE), 2012. EVUE Report. London, UK. Joint Research Centre (JRC), 2014. Data collection and reporting guidelines for European electro-mobility projects. JRC Science and policy report. Ispra, Italy. Willmack, D., Schuman, H., Pennell, B.E. and J. Lepkowski, 1995. Effects of a prepaid nonmonetary incentive on response rate and response quality in a face-to-face survey. Public Opinion Quarterly, V. 59: 78-92.

52

8 APPENDIX 8.1 Appendix A The proforma used for data collection process:

53

Retrofit Demonstrator title:

Number of buildings: Type of buildings (mixed use/ residential/ public)/ Type of tenant: Number of dwellings: Retrofit measures to be implemented:

Retrofitting measures Tick if applies Priority thermal insulation – walls and/or ceilings thermal insulation windows air tightness improvements hybrid/mechanical ventilation with heat recovery

thermostatic valves high efficiency generation system solar shading LED lighting photovoltaic panels solar thermal panels

Others (please specify…)

Renewable energy generation measures:

Renewable energy generation measures Tick if applies Priority District heating Low carbon energy heat PV/solar EV charging

Others (please specify) …

Data sources/ technology of data collection: Data available: Limitations in data collection (for measuring demonstrator impact): Please use the table below for any suggested additional evaluation targets/ measurable indicators:

Theme Focus area Evaluation target

Measurable indicator

Units - Data standards

Data source

Timeframe (for the retrofitting demonstration):

54

Smart Energy

Management System Demonstrator title:

Scale (e.g. district based, building based): Describe existing energy system: Factors to be considered in SMES operation: SEMS capability/ (what can actually be managed?): Data sources/ technology of data collection: Data available: Limitations in data collection (for measuring demonstrator impact): Please use the table below for any suggested additional evaluation targets/ measurable indicators:




Data source

Timeframe (for the SEMS demonstration):

55

Mobility Demonstrator title:

New or build on/ replaces an existing mobility system?

If not new, describe baseline system (if applicable)

Data sources/ technology of data collection:

Data available

Limitations in data collection (for measuring demonstrator impact)

Please use the table below for any suggested additional evaluation targets/ measurable indicators:




Data source

Timeframe (for each of the demonstrations in Mobility)

56

Smart Lamppost Demonstrator title:

Confirm demo numbers (from table below):

Functions to be considered:

Function Tick if applies Priority

Wi-Fi, Mobile & Mesh

App based wireless control

Environmental sensing (air quality, noise)

Façade lighting (colours)

ROBA notification

Digital signage

Water level/ flood monitoring

PV, power for lamp, mobile phone

Smart lighting - LED

Smart lighting – Photocell control

Smart lighting – 0-100% dimming

Smart lighting – On-demand lighting

Concealed speakers

Image sensing

Push-to-talk system

eV charging

Other (please describe)… Any smart lighting function in place?

Data sources/ technology of data collection (per module):

Data available (per module)

Type of bulbs currently used:

Limitations in data collection (for measuring demonstrator impact):

Please use the table below for any suggested additional evaluation targets/ measurable indicators:




Data source

Timeframe (for the smart lamppost demonstration):

57

8.2 Appendix B Detailed information on the mobility demonstrators implementation timeframes:

Lisbon London Milan

eV Car Sharing

CML Corporate eCar Sharing

Assess requirements for municipality.

Jan 2016 – Dec 2016

RBG e-Car Club Trial

Feasibility Study Apr 2016- Jun 2016 Milan

Public e-Car

Sharing

2016 – 2018

Preparation of tender documents (post attitude survey)

Oct 2016 - Dec 2016

Identify and technically adapt first 15 EVs.

Apr 2016 – Dec 2016

Set up of e-charging points at locations

Jan 2017 – Mar 2017

Award of tender and start of scheme

Apr 2017- Jun 2017


Test fleet installation Jul 2016 – Dec 2016

Evaluation and analysis Jul 2017- onwards

Autonomous EVs

Pre-trial planning Jan 2016- Sep 2016

Trials of last mile shuttle Jul 2016- Sep 2016

Design Corporate Car-sharing service

Apr 2016 – Dec 2016

E logistics trial Oct 2016- Mar 2017


Extend of eCar Sharing operation to 45 EVs (or more)

Jan 2017 – Jun 2017

Last Mile demonstrator service

Oct 2016- Mar 2017

Evaluation and analysis Oct 2016- Mar 2017

eBikes Launch of public tender for eBike Sharing Operation.

Feasibility Study Apr 2016- Sep 2016

2016-2018

58

Lisbon London Milan

EMEL eBike Sharing

Apr 2016 – Sep 2016

RBG e-Bike Trial

Obtain e-bikes for residents/organisations

Oct 2016- Mar 2017

Milan E-Bike Sharing

Roll-out of scheme Apr 2017- Sep 2017

Soft launch of eBike Sharing System.

Sep 2016 – Feb 2017

Evaluation and analysis Oct 2017- onwards

Design and test of Park & Bike Service.

Sep 2016 – Sep 2017

Widespread launch of eBike Sharing

Mar 2017 – Sep 2017

Deploy of Park and Bike service

Jan 2017 – Dec 2017

Smart Parking

EMEL Smart Parking

Select places for sensor deployment

Apr 2016 – Sep 2016

RBG Smart Parking

Feasibility and research Jul 2016- Dec 2016

Milan Smart Parking

2017-2018

Development Dependent initially on outputs of WP3.4

Jan 2017- Mar 2017

Contract completion and roll out

Apr 2017- Dec 2017

Evaluation and analysis Jan 2018- Dec 2018

eLogistics

EDP eLogistics

Install and connect existing 6 vehicles to the mobi.me system.

Sep 2016 – Oct 2016 RBG

eLogistics

Milan e-Logistics

2017-2018 Assess impact on the target

area and feasibility of trip planning.

Apr 2016- Dec 2016

59

Lisbon London Milan

EMEL eLogistics

Install monitoring equipment on limited number of vehicles

Apr 2016 – Sep 2016

Prepare public procurement procedures for new vehicles

Mar 2016 – Dec 2016

Monitor and evaluate vehicles usage

Sep 2016 – Feb 2017

Have eLogistics fleet in operation

Jan 2017 – Dec 2017

CML eLogistics

Evaluate Fleet technically for monitoring feasibility

Apr 2016 – Sep 2016

Connect logistics fleet to the monitoring system

Oct 2016 – Dec 2016

Configure different logistics profiles

Sep 2016 – Jan 2017

Define Public procurement process for new vehicles

Apr 2016 – Dec 2016

Have eLogistics Fleet in operation

Jan 2016 – Dec 2017

eV Charging

Points

EDP Public and Private

Select locations Jan 2016 – Jun 2016

RBG Bolloré EV

Feasibility Jul 2016- Sep 2016 Milan e-

Logistics

2016-2018

Select Equipment Apr 2016 – Oct 2016

Location identification Oct 2016- Dec 2016

60

Lisbon London Milan

Charging Network Install Equipment

Sep 2016 – Apr 2017

Charging

Rapid EVCP

Stakeholder/provider identification/ roll out

Dec 2017


Select locations Jan 2016 – Sep 2016

Evaluation and analysis Dec 2017 - onwards Install / Upgrade smart home

chargers

Apr 2016 – Apr 2017

Other CML eBus Launch Public procurement process for electric buses

Jul 2017 - Dec 2017


Installation and ground truthing of equipment

Jan 2016- Mar 2016

N/A N/A

Signage going live Jan 2016- Jun 2016

Behavioural change programme

Sep 2016 forwards

Evaluation and analysis Oct 2016 forwards

Smart Square/ neighbourhood

61

Documents

Common monitoring and evaluation framework - Eurocitiesnws.eurocities.eu/.../media/CommonMonitoringandEvaluationFramework.pdf · Common framework: The project will create a common