SUPERQUANTILE REGRESSION WITH …dinamico2.unibg.it/icsp2013/doc/ps/4.Miranda.pdfsuperquantile regression with applications to buffered reliability, uncertainty quantification, and

SUPERQUANTILE REGRESSION WITH APPLICATIONS TO

BUFFERED RELIABILITY, UNCERTAINTY QUANTIFICATION,

AND CONDITIONAL VALUE-AT-RISK

INTERNATIONAL CONFERENCE ON STOCHASTIC PROGRAMMING

Sofia Miranda

Portuguese Navy, Lisbon, Portugal

Naval Postgraduate School, Monterey, CA

with Johannes O. Royset

Naval Postgraduate School, Monterey, CA

and R. T. Rockafellar

University of Washington, Seattle, WA

BERGAMO, 11 JULY 2013

This material is based upon work supported in part by the U.S. Air Force Office of

Scientific Research under grants FA9550-11-1-0206 and F1ATAO1194GOO1.

• Introduction

• Fundamental Risk Quadrangle

• Least-Squares Regression

• Quantile Regression

• Superquantile Regression

• Validation Analysis

• Computational Methods

• Numerical Examples

AGENDA

2

• Superquantile

Also known as…

• conditional value-at-risk

• average value-at-risk

• expected shortfall

Coherent [1], regular [2], convex [3] risk measure

Accounts for tail behavior (risk-averseness)

BACKGROUND

3

[1] P. Artzner, F. Delbaen, J.-M. Eber, and D. Heath. Coherent measures of risk. Mathematical

Finance, 9:201-227, 1999.

[2] R.T. Rockafellar and S. Uryasev, “The fundamental risk quadrangle in risk management,

optimization and statistical estimation,” Surveys in Operations Research and Management

Science, 18:33–53, 2013.

[3] A. Ruszczynski and A. Shapiro. Optimization of convex risk functions. Mathematics of

Operations Research, 31(3):433-452, 2006.

-quantile = min | ( )Yq Y y F y

1

-superquantile =

1 ( )

1

q Y

q Y d

E Y

4

BACKGROUND

Cost / Loss / Damage

Den

sity

max Y

1

• Cost / Loss / Damage random variable Y

Incomplete information about random variable Y

(e.g., unknown probability distribution, mean and

other relevant statistics)

Beneficial to approximate loss random variable Y

in terms of an n-dimensional explanatory random

vector X that is more accessible

MOTIVATION

5

Naturally we think…

… Least-squares regression:

estimate the conditional expectation of the loss

random variable Y.

6

MOTIVATION

Naturally we think…

… Least-squares regression:

estimate the conditional expectation of the loss

random variable Y.

But what if…

… the decision maker is risk averse, i.e., more

concerned about upper-tail realizations of Y

… and sees errors asymetrically, i.e.,

UNDERESTIMATING losses being more

DETRIMENTAL than overestimating

MOTIVATION

6

An alternative…

… Quantile regression:

accomodates risk-averseness and a asymmetric

view of errors by estimating conditional quantiles

at a probability level.

7

MOTIVATION

An alternative…

… Quantile regression:

accomodates risk-averseness and a asymmetric

view of errors by estimating conditional quantiles

at a probability level.

But…

… quantile is not a coherent measure of risk.

… quantiles cause computational challenges

when incorporated into optimization problems.

7

MOTIVATION

8

MOTIVATION

8

MOTIVATION

8

MOTIVATION

8

MOTIVATION

8

MOTIVATION

9

MOTIVATION

9

MOTIVATION

9

MOTIVATION

9

MOTIVATION

9

MOTIVATION

Superquantile tracking

•

•

•

Surrogate estimation

•

•

10

MOTIVATION

Focus on the conditional ( ) ( | ) for

Minimize ( ) by choice of min ( )

Given data of realizations of ( ), build approximating

model of ( ) as function of

n

x

Y x Y X x x

Y x x q Y x

Y x

q Y x x

Explanatory random vector beyond our control

Select function s.t. ( ) serves as surrogate for

X

f f X Y

11

MOTIVATION

Superquantile weighs the data above and below

the given probability level α

(magnitude of the errors are important)

We develop Superquantile regression.

FUNDAMENTAL RISK QUADRANGLE

risk deviation

regret

erro

r

12

[3] R.T. Rockafellar and S. Uryasev, “The fundamental risk quadrangle in risk management,

optimization and statistical estimation,” Surveys in Operations Research and Management

Science, 18:33–53, 2013.

0

0

0 0

0

arg min

statist

( )

( ) arg min ( )

ic C

C

C Y C

YY C

0

0

0 0

0

( ) min ( )

( ) min ( )

C

C

Y C Y C

Y Y C


13

0

0

0 0

0

arg min

statist

( )

( ) arg min ( )

ic C

C

C Y C

YY C

0

0

0 0

0

( ) min ( )

( ) min ( )

C

C

Y C Y C

Y Y C


13

What error measure should we minimize so that

we get a specific statistic?...

… Say expectation?

LEAST-SQUARES REGRESSION

• Risk quadrangle (variance version)

14

2

0 0 2

Y E Y

Y C Y C

LEAST-SQUARES REGRESSION

• Risk quadrangle (variance version)

14

2

0 0 2

Y E Y

Y C Y C

Least-squares regression obtained when we minimize

the corresponding error measure

QUANTILE REGRESSION

• Risk quadrangle

15

0 0 0

1max ,0 max ,0

1

Y

Y C E Y C Y C

q Y

For a given probability level (0,1)

[4] R. Koenker. Quantile Regression. Econometric Society Monographs, 2005.

QUANTILE REGRESSION

• Risk quadrangle

15

0 0 0

1max ,0 max ,0

1

Y

Y C E Y C Y C

q Y


Quantile regression obtained when we minimize

the corresponding error measure

(the Koenker-Bassett error)

SUPERQUANTILE REGRESSION

16

What error measure should we minimize so that

we get the SUPERQUANTILE as the

STATISTIC?...

0 0 0

0 0

1

0

1max 0 ), (

1

Y C Y C C

Y C q Y C

Y q Y

Y q Y q Y

E Y

d

Y q Y

Y q Y E Y

Superquantile-based quadrangle

• Risk quadrangle

Superquantile

error measure

17



• Assumptions on loss random variable Y:

(Y has a finite second moment)

2 2( ) : { : | [ ] }Y Y E Y


18

• Superquantile regression problem

Before…

We want to go beyond class of constant functions

and utilize connection with underlying

explanatory random vector X

Regression functions of the form

for given ‘basis’ functions

0

0 min C

Y C

19


0 0( ) , ( ) , with , ,mf x C C h x C C

: n mh

0

0,

0 0

: min ( , )

where ( : )

, ) , (

mC CP Z C C

Z C C Y C C h X

Superquantile regression problem:

is the error random variable.

: (0,1)For any ,and n mh

1

0

set of optimal solution

( , ) reg

s of

ression ve

ctor

m P

C C


20

1

0 0 0

0

1( , ) max 0, ( , ) ( , )

1Z C C q Z C C d E Z C C

convex

optimization

problem

21


Consistency

Distribution of (X,Y) rarely available

(e.g., empirical distribution generated by sample)

random vector whose joint

distribution approximates that of ,

number observati

( , )

ons

X

Y

Y

X

CONSISTENCY

0 0( , ) : , ( )Z C C Y C C h X 22

• Superquantile regression problem

Approximate superquantile regression problem

(incomplete distributional information)

Recall:

0

0,

: min ( , )mC C

P Z C C

0

0,

: min ( , )mC C

P Z C C

0 0( , ) : , ( )Z C C Y C C h X


23

Suppose ( , ) ( , ) then:dX Y X Y

• Theorem

0 1If {( , )} is sequence of optimal solutions

of , then every accumulation

point of that sequence is a regression vector of .

C C

P

P


24

1

0

1

Fix ( , )

( (·,·)) ( (·,·)) converges

pointwise on

( (·,·)) and ( (·,

abritrarily.

.

are finite and

convex functi

·))

( (·,·)) epiconverges to ( (·,·))

convergence of opti

on

ma

s

l

m

m

C C

Z Z

Z Z

Z Z

solutions

• Proof


25

• Proposition

The set of regression vectors of

is equivalently obtained as

C P

26

0

0,

: min ( , )mC C

P Z C C

0

0 0

: min ( )

and set ( ( ))

mCD Z C

C q Z C

0( ) , ( ) .where Z C Y C h X

ALTERNATIVE PROBLEM

• Proposition

The set of regression vectors of

is equivalently obtained as

C P

26

0

0,

: min ( , )mC C

P Z C C

0

0 0

: min ( )

and set ( ( ))

mCD Z C

C q Z C

0( ) , ( ) .where Z C Y C h X

ALTERNATIVE PROBLEM

1

0

0

0

1max 0, ( , )

1

( , )

q Z C C d

E Z C C

1

0

0

1( )

1

( )

q Z C d

E Z C

• Coefficient of determination

Least-squares regression

2

2 Res

2

T

Res

T

0

1 1 1,

where

= residual sum of squares error measure

= total sum of squares deviation measu e

,

r

ESS

Y C C h

RSS

SS

S

X

S

Y

VALIDATION ANALYSIS

0( ( , ))Z C C

( )Y

27

28

VALIDATION ANALYSIS

• Coefficient of determination

Superquantile regression

0

02 , : 1,Y C C h X

R C CY

0 0

2

0 0, ,

arg min , arg max ,C C C C

Y C C h X R C C

COMPUTATIONAL METHODS

• Optimization problem we need to solve:

29

0

0 0

0

1

0

: min ( )

1

1 min ( ( )) ( )

and set ( ( ))

m

m

C

C

D Z C

q Z C d Z CE

C q Z C

0( ) , ( ) .where Z C Y C h X

Approximate problem with observations: D


• Analytical integration

Numerical integration can be avoided by expense

of additional variables

Partition integral over

in D

We obtain a LP involving

1 1 variables

and 2 inequality constraints,

where .

m

30

• Numerical integration

0 1 1

,

0 0

1

[ ,1] subintervals,

where 1.

0, 0,..., .

Approximation of

Divide interval into

Weigh each o

1: min ( ( )) ( )

1

Larg

ne by it

e-scale LP

s wei

w

ght

i

:

m i

i

iC

i

w i

D

D w q Z C E Z C

th 2 variables

and 2 constraints

m


31

[5] Inc. American Optimal Decisions. Portfolio Safeguard (PSG) in Windows Shell

Environment: Basic Principles. AORDA, Gainesville, FL, 2011.

• Investment analysis (I)

Y is the negative return of the Fidelity Magellan

Fund

Explanatory variables:

RLG (Russel 1000 Growth Index)

RLV (Russel 1000 Value Index)

RUJ (Russel Value Index)

and RUO (Russel 2000 Growth Index)

32

NUMERICAL EXAMPLES

• Investment analysis (I)

Y is the negative return of the Fidelity Magellan

Fund






32

NUMERICAL EXAMPLES

X

[5] Inc. American Optimal Decisions. Portfolio Safeguard (PSG) in Windows Shell

Environment: Basic Principles. AORDA, Gainesville, FL, 2011.

NUMERICAL EXAMPLES

33

2

0.90

0.90

( ) 0.0475 1.0727

0.5702

f x x

R

• Investment analysis (II)






34

NUMERICAL EXAMPLES

1

2

3

4

X

X

X

X

1 2 3 4

2

0.90

( ) 0.0138 0.4837 0.4912 0.0223 0.0019

0.8722

f x x x x x

R

QUESTIONS ?

REFERENCES

[1] R.T. Rockafellar and J.O. Royset, “Random variables, monotone

relations, and convex analysis,” Mathematical Programming B, accepted.

[2] R.T. Rockafellar, J.O. Royset, and S.I. Miranda, “Superquantile

regression with applications to buffered reliability, uncertainty

quantification, and conditional value-at-risk,” in review.

[3] R.T. Rockafellar and J.O. Royset, “Superquantiles and their applications

to risk, random variables, and regression,” INFORMS Tutorials,

INFORMS, 2013.

[4] R.T. Rockafellar and S. Uryasev, “The fundamental risk quadrangle in

risk management, optimization and statistical estimation,” Surveys in

Operations Research and Management Science, 18:33–53, 2013.

[5] R. T. Rockafellar, S. Uryasev, and M. Zabarankin. Risk tuning with

generalized linear regression. Mathematics of Operations Research,

33(3):712–729, 2008.

Documents

SUPERQUANTILE REGRESSION WITH …dinamico2.unibg.it/icsp2013/doc/ps/4.Miranda.pdfsuperquantile regression with applications to buffered reliability, uncertainty quantification, and