ETC 2009 Gerard de Jong – Significance and ITS Leeds

Predicting uncertainty of traffic forecasts:

giving the policy-makers a range instead of a single number

November 2014

ETC 2009

Contents of this presentation

■ Background and types of uncertainty affecting traffic forecasts

Uncertainty prediction method

Examples of outcomes (uncertainty margins)

Netherlands national/regional models

Some public transport project in Paris

Fréjus Tunnel

p.2

ETC 2009

Background I

Laplace, Pierre Simon Théorie Analytique des Probabilités, 1812

‘The most important questions of life are indeed, for the most part, really only problems of

probability.’

Godfried Bomans (1913-1971):

‘A statistician waded confidently through a river that on average was one metre deep ….

… He drowned.’

p.3

ETC 2009

Background II

Usually only point estimates for transport volumes and traffic flows, no uncertainty margins

In The Netherlands often 3-4 point estimates: for different scenarios

But for investments and policy-making, it is important to know the range: robust decisions?

p.4

ETC 2009

Background III

p.5

ETC 2009

Types of uncertainty (risk) affecting the predictions

We are predicting Y using a model Y = f(’X , u)

■ Input uncertainty (in X):

Economic/demographic variables, e.g. GDP/capita, population Policy variables: travel time and travel cost:

(Policies of the decision-maker)

Policies of other organisations, e.g. specific taxes, safety measures, or competitors, e.g. competing modes

p.6

ETC 2009

Types of uncertainty (risk)

Model uncertainty, e.g. in the model coefficients such as impact of rail in-vehicle time on modal split

Estimation error (in )

Micro-simulation error (different model runs lead to different choice outcomes)

Specification error (e.g. different functional form f or error distribution for u)

Unexpected discrete events (e.g. fire in the Mont Blanc tunnel, natural disaster, major strike, terrorist attack)

p.7

ETC 2009

Contents of this presentation

■ Background and types of uncertainty affecting traffic forecasts

Uncertainty prediction method

Examples of outcomes (uncertainty margins)

Netherlands national/regional models

Some public transport project in Paris

Fréjus Tunnel

p.8

ETC 2009

Methodology: reviews

■ de Jong et al. (2007) Uncertainty in traffic forecasts: literature review and new results for The Netherlands, Transportation, 34(4), 375-395

■ Rasouli and Timmermans (2012) Uncertainty in travel demand forecasting models: literature review and research agenda, Transportation Letters, 4, 55-73

p.9

ETC 2009

Methodology: reviews

■ de Jong et al. (2007) Uncertainty in traffic forecasts: literature review and new results for The Netherlands, Transportation, 34(4), 375-395

■ Rasouli and Timmermans (2012) Uncertainty in travel demand forecasting models: literature review and research agenda, Transportation Letters, 4, 55-73

PhD thesis of Stefano Manzo (2014) at DTU Copenhagen (supervised by Otto Anker Nielsen and Carlo Prato): Uncertainty calculation in transport models and forecasts

p.9

ETC 2009

Methods for quantifying uncertainty I

The literature on quantifying uncertainty in traffic forecasts is fairly limited (compared to the number of forecasts)

For input uncertainty:

all studies use repeated model simulation

usually with random draws for the inputs

most studies ignore correlation between inputs

some studies use long time series on the past to determine the amount of variation and correlation in the input variables

an alternative for this is a rule-based approach from directed probabilistic graphical models (Petrik et al., IATBR, 2012)

p.10

ETC 2009

Methods for quantifying uncertainty II

For model uncertainty:

variances and covariances of parameters can come from the model estimation

Jackknife and Bootstrap methods to obtain proper variances (some specification error)

some studies use analytic expressions for the output variance (due to using parameter estimates). Not a practical method for complicated models

repeated model simulations with random draws for parameter values

p.11

ETC 2009

Overview of common method for both input and model uncertainty

■ Assume Normal (or triangular) distributions fo each input variable and coefficient, if possible correlated with each other

■ Take ‘random’ draws from multivariate Normal distributions (Monte Carlo simulation)

Insert the values drawn in the transport model and run the model to obtain traffic forecasts

Do this for many draws (e.g. 1000)

Calculate summary statistics on the series of traffic forecasts obtained

p.12

ETC 2009

Contents of this presentation

■ Background and types of uncertainty affecting traffic forecasts

Uncertainty prediction method

Examples of outcomes (uncertainty margins)

Netherlands national/regional models

Some public transport project in Paris

Fréjus Tunnel

p.13

Case study: A16 motorway near Rotterdam

ETC 2009

Method used in Dutch study for input uncertainty

List input variables in tour frequency models, mode-destination models and expansion procedure:

income, car ownership, car cost/km, jobs by sector, population by age group; household size, occupation, education

Use existing time series (1960-2000; 20-year moving averages) as source on variances and covariances

Draw input values from multivariate normal distribution (with correlations; generated using Choleski decomposition)

Run models for many different sets of inputs

p.15

ETC 2009

Method used in Dutch study for model uncertainty

Variances and covariances for parameters from estimation (including Bootstrap) of the tour frequency and mode-destination choice models

Draw parameters from multivariate normal distribution

Run models for many different sets of parameters

Sources of variation that were not included:

Uncertainty in base matrices

Errors in licence holding and car ownership models

Errors in assignment and time of day procedures

Distribution over zones

p.16

ETC 2009

95% confidence intervals for pkm by mode for Reference 2020 (input, model, total uncertainty)

p.17

ETC 2009

Outcomes for vehicle flows on selected links for Reference 2020

p.18

ETC 2009

Contents of this presentation

■ Background and types of uncertainty affecting traffic forecasts

Uncertainty prediction method

Examples of outcomes (uncertainty margins)

Netherlands national/regional models

Some public transport project in Paris

Fréjus Tunnel

p.19

ETC 2009

Main results in Paris

■ New element: input uncertainty in policy variables, such as transport cost and different time components by mode (partly own policy; partly determined by others)

As in the Dutch application, the macro-economic variation (part of input uncertainty) is the most important source of outcome uncertainty

The possible variation in transport time and cost by mode (partly own policy; partly determined by others) also important

Uncertainty of model coefficients relatively more important than in The Netherlands

p.20

ETC 2009

Contents of this presentation

■ Background and types of uncertainty affecting traffic forecasts

Uncertainty prediction method

Examples of outcomes (uncertainty margins)

Netherlands national/regional models

Some public transport project in Paris

Fréjus Tunnel

p.21

ETC 2009

Fréjus tunnel application

Road connection in the Alps between France and Italy

Private operator; toll and subsidies from France and Italy

Part of the TEN-T

Competes with Mont-Blanc tunnel, mountain passes, railway lines and future Lyon-Turin high-speed rail service (passengers, freight)

New: inclusion of time dimension (uncertainty margins as long-term predictions over time)

p.22

ETC 2009

Variables and coefficients that are varied (Fréjus)

■ GDP (distinguishing 3 time periods up to 2050)

When will Lyon-Turin HSR service (passengers, freight) open? And its prices?

When will Fréjus Safety Tunnel open?

Competing conventional and container rail routes: when will increased capacity become available?

EU environmental policies (e.g. volume cap on trucks through tunnels)

Alternative-specific coefficients (for routes)

Other model coefficients (elasticities, mode/route choice)

p.23

ETC 2009

Uncertainty margins passenger forecasts

p.24

Passenger vehicles Frejus + Mont Blanc tunnel corridor

0

50

100

150

200

250

300

350

20

07

20

09

20

11

20

13

20

15

20

17

20

19

20

21

20

23

20

25

20

27

20

29

20

31

20

33

20

35

20

37

20

39

Vo

lum

e in

dex

(20

07 =

100

)

ETC 2009

Uncertainty margins freight forecasts

p.25

Freight vehicles Frejus + Mont Blanc tunnel corridor

0

20

40

60

80

100

120

140

160

180

200

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

2027

2029

2031

2033

2035

2037

2039

Vo

lum

e in

dex

(20

07 =

100

)

ETC 2009

What do we conclude from the Fréjus graphs?

Uncertainty increases over time, …

… but not at a constant rate

Important sources of uncertainty:

opening of Lyon-Turin HSR (passengers: 2018-2024; freight: 2023-2030)

regulatory measures (volume cap for road freight through tunnels): timing (2023-2030) and size

p.26

ETC 2009

Concluding remarks

Most traffic forecasts do not quantify uncertainty

Methods exist for both input and model uncertainty (Monte Carlo simulation, repeated model runs)

Case studies: input uncertainty dominates model uncertainty

Policy variables (actions of other decision-makers) can be included

Time dimension can be included (uncertainty margins over time). Especially for PPP projects one would like to know time path of forecasts and uncertainty

p.27