Connecting Markets East & West

© Nomura

RiskMinds

December 9, 2015

Standard Initial Margin Model (SIMM) – How to

validate a global regulatory risk model

Eduardo Epperlein*

Risk Methodology Group

* In collaboration with Martin Baxter and James McEwen (GM Quantitative Research)

The analysis and conclusions set forth are those of the author. Nomura is not responsible for any statement or conclusion herein, and opinions or theories presented herein do not

necessarily reflect the position of the institution.

1. Margin Requirements for non-centrally cleared derivatives, September 2013, BCBS 261

Basel regulation1 stipulates that IM should be calculated at a 99% confidence level, with MPoR is set at a minimum of 10 business days

The calculation is repeated daily, thus capturing any change in the portfolio and, hence, any change to its variability

Both counterparties conduct equivalent calculations of IM

The bilateral IM is segregated, such that in the event of a counterparty defaulting its posted collateral provides the necessary protection

Reconciliation and agreement on the amount of posted/called collateral is crucial for this process to work smoothly

Hence, a standardization of the method to calculate IM is vital

An Initial Margin (IM) model is designed to estimate how much collateral we need to post to cover a potential

increase in the value of our derivative contracts over the Margin Period-of-Risk (MPoR) within a netting set

Before embarking on the validation framework it is important to appreciate the

differences between a margin model and a capital model

Regulatory counterparty exposure models, such as the Internal Model Method (IMM), are designed to calculate the EPE of derivative

contracts traded with the counterparty

The credit risk capital is then estimated via the EPE, the PD of the counterparty, and the loss-given-default

Unlike the risk mitigation provided by IM, the credit risk capital model requirement is imposed on the surviving counterparty

The capital calculations need not be reconciled with the counterparty and, hence, don’t require the same level of standardization as IM

(though regulators may think otherwise to promote uniform financial safety)

In a capital model we calculate the Expected Positive Exposure (EPE) to our counterparty in order to estimate

the amount of credit risk capital we need to hold given the counterparty’s probability of default (PD)

1

1. ISDA: International Swaps and Derivatives Association Inc

2. FRTB: Fundamental Review of the Trading Book

The financial industry, through the auspices of ISDA1, agreed on a Standardized

IM Model (SIMMTM) and proposed it to the regulators

SIMM is based on a variant of the “Sensitivity Based Approach” (SBA), which was developed by the regulators

as risk-sensitive yet conservative standard model for market risk capital under FRTB2

Equally important, the financial industry needed to propose a common approach for validating the SIMM and

propose that the national regulators adopt that approach uniformly

The “gold standard” for validating risk models is Backtesting. But, once again, it is important to highlight the

differences between backtesting a capital model and an IM model:

Risk Model Type Backtesting Approach Participation Frequency Corrective Actions

Value-at-Risk – VaR

(market risk capital) Stand-alone All individual firms Daily

Capital multiplier/model

updates

IMM

(counterparty credit

risk capital)

Stand-alone All individual firms Quarterly Capital multiplier/model

updates

SIMM

(Initial Margin)

Global via central

coordination

Systemically important

firms, covering

systemically important

portfolios

Annually SIMM updates via

central coordination

2

1. https://www.bis.org/publ/bcbsc223.pdf

Basel regulation stipulates Red-Amber-Green (RAG) zones for establishing the validly of the VaR model

The regulatory backtesting framework currently used to validate VaR models1

appeared to be the most suitable candidate to validate SIMM

Backtesting is performed by comparing the one-day VaRex-ante(t) against the P&Lex-post(t to t+1) over 250 business days

A VaR exception occurs when P&L < - VaR (i.e. loss exceeds VaR)

RAG: Green (0-4 exceptions) | Amber (5-9 exceptions) | Red (10 or more exceptions) – a.k.a. “Basel Traffic light test”

RAG zones correspond to “type 1 errors” (falsely rejecting an accurate model): Green (<95%), Amber (95%<99.99%), Red (>=99.99%)

-3

-2

-1

0

1

2

3

1

15

29

43

57

71

85

99

113

127

141

155

169

183

197

211

225

239

VaR

P&L

3 Exceptions

Sample VaR backtesting

Exceptions follow a

binomial distribution

3

1. This backtesting exercise was coordinated by ISDA

The adopted SIMM backtesting framework needed to be done globally across

systemically important firms, in a coordinated fashion

The last exercise concluded in July 30th, 2015, involving 16 institutions, across 19 legal entities, generating

280 portfolios, via the following 4 “simple” steps:1

1. Calculate the SIMM (post and call) by taking a snapshot of the portfolios as of April 30th, 2015,

2. Generate about 7 years of historical P&L data, from January 1st, 2008 to April 30th, 2015, by shocking the frozen portfolio with a 10

business day market move, thus generating approximately 1900 (overlapping) P&Ls,

3. Conduct an extensive reconciliation exercise to help minimize operational errors

4. Perform backtesting analysis

4

Before starting the actual backtesting exercise every firm conducted extensive

reconciliations on a bilateral basis

Two sample tests involved calculating:

a) The correlation of pairs of P&L vectors (between two firms) – perfect reconciliation would imply -100% correlation

b) The relative difference between “Own Entity Call IM” and “Counterparty Post IM”

As shown below, results were generally considered successful

5

1. Go live expected September 2016

The risk across the 280 portfolios was primarily driven by delta exposure

In order to make the portfolio more representative of future state of when SIMM goes live1 each selected

portfolio contained uncleared OTC derivative trades executed between June 30th, 2013 (inclusive) and April

30th, 2015 (inclusive) and open as of April 30th, 2015.

6

The ratio of the calculated IM to the 99% and 1% percentile of the historical 10-day

P&L distribution gave the first indication of the validity of the SIMM

The sum of all SIMM values is over 2x larger than the sum of all historical VaR measures.

This indicates that for the actual portfolios the calculated IM is likely to be conservative and pass backtesting.

7

The standard 1-day VaR backtesting had to be modified to for the 10-day SIMM

backtesting using overlapping windows and multiple portfolios

The first modification to the backtesting involved the transition from taking independent samples of 1-day

P&Ls to overlapping samples of 10-day P&Ls

Zone Number of exceedances

1-day 10-day overlapping

Green 0 - 4 0 - 8

Amber 5 - 9 9 – 25

Red 10+ 26+

Zone

Number of exceedances

1-day independent 10-day overlapping,

50% correlation

Green 0 – 11 0 – 19

Amber 12 – 19 20 – 51

Red 20+ 52+

We can illustrate the effect of auto-correlation introduced by the overlapping windows by conducting a Monte Carlo simulation of 250 IID

random variables and generating the overlapping P&Ls to empirically estimate the RAG zones: Please see below:

The second modification involved taking into account the fact that the 280 backtesting portfolios were

conducted across common time slices and therefore were not necessarily independent

We can also illustrate this effect calculating the empirical correlation across the portfolios and repeating the Monte Carlo simulation with

the same correlation structure. Please see below an example with 3 time series with identical pair-wise correlation of 50%:

8

We can now backtest a single portfolio by comparing the Call and Post IM against

the historical time series of 10-day P&Ls

Here we show a sample plot of 782 overlapping 10-day P&Ls (from Feb 29th, 2008 to Apr 8th, 2015) against IM to

Post and IM to Call.

We observe 9 exceedances against IM to Call and 3 against IM to Post, which fall well within the Green zone of up to 18 exceedances

As one might expect, the majority of the exceedances occurred during the 2008-09 crisis period.

8

1. SIMM is designed to be non pro cyclical so it makes sense to backtest annually over a an extended test period, even it involves some level of “in sample” testing

By conducting the modified backtesting at legal entity level the number of

exceedances beyond the IM level where all within the GREEN zone

It should be noted that each legal entity had multiple portfolios with different correlation structure and different

numbers of historical data points. Hence, the RAG zones had to be estimated separately

Legal

Entity

Number of

Observations Green up to Amber up to

Exceedance Count

(to call)

Traffic Light

(to call)

Exceedance Count

(to post)

Traffic Light

(to post)

A 1913 427 594 8 Green 42 Green

B 1903 398 553 38 Green 23 Green

C 1904 415 555 15 Green 42 Green

D 1775 409 625 122 Green 44 Green

E 1832 419 600 39 Green 62 Green

F 1497 323 459 63 Green 40 Green

G 1913 404 522 31 Green 35 Green

H 1911 274 373 76 Green 59 Green

I 1911 115 181 25 Green 17 Green

J 1911 384 505 21 Green 2 Green

K 1903 221 295 35 Green 9 Green

L 1903 322 439 40 Green 8 Green

M 1913 403 569 27 Green 37 Green

N 782 168 255 22 Green 46 Green

O 1211 279 398 37 Green 55 Green

P 1913 394 535 60 Green 48 Green

Q 1852 377 499 9 Green 29 Green

R 1903 418 567 23 Green 115 Green

S 1903 369 543 47 Green 66 Green

9

Backtesting covered “in- and out-of-sample” test periods since SIMM is calibrated with a 1 year stress period and recent 3 year period1

More granular backtesting was also performed at the 280 individual portfolios for both called and posted IM, and only two out 280

portfolios (less than 1%) had exceedances in the Amber zone – Hence, overall, a successful result

1. Submitted to regulators on July 31st, 2015

Summary and conclusions

IM models, unlike capital models, require a much higher level of standardization. Hence, the need to a “Standard IM Model” or SIMM

To preserve the standardization of IM the validation needs to be applied uniformly by national regulators

The standard VaR backtesting framework has been adapted to test the SIMM over a 10-day overlapping window and across multiple

portfolios

A global validation framework has been successfully developed and tested across 16 major financial institution

“How to validate a global regulatory risk model” – In particular, “How to validate SIMM”

10

SIMM successfully passed the global backtesting exercise as of April 30th, 20151

Appendix

Glossary

BCBS: Basel Committee on Banking Supervision. Founded in 1974 by regulators in the G-10 countries, with mandate to strengthen

regulation, supervision and practices of banks worldwide to enhance financial stability, but without any formal supranational authority.

EPE: Expected Positive Exposure. Average positive exposure calculated across a netting set over a 1 year horizon..

FRTB: Fundamental Review of the Trading Book. Fundamental review of the market risk framework introduced under Basel 2.5.

IMM: Internal Model Method. Internal model used for calculating counterparty exposure at netting set level for both OTC derivative and

Securities Financing Transactions (SFT)

MPoR: Margin Period-of-Risk

PD: Probability of Default

SBA: Sensitivity Based Approach

SIMM: Standard Initial Margin Model

VaR: Value at Risk. Trading loss calculate at a given confidence level and time horizon. For regulatory capital calculation we use 99%

confidence level and 10 days.

11