Be aers-fara-modellinginsolvency-nov2010

© 2010 Deloitte

Modelling in Solvency II:from regulation to practice

November 2010

© 2010 Deloitte

Solvency II Processes: What are we talking about?

As depicted hereunder, the Solvency II value chain is built upon

complex processes and information flows:

Life:

Contracts and

claims

Non life / Health:

Contracts and

claims

Asset Data

Accounting Data

Data sourcesData

CollectionCalculation

Conso. and

aggreg.Reporting

SFCR &

RTS Group

SFCR &

RTS Solo

Ris

k C

lass a

gg

reg

ato

r and

co

nso

lid

atio

n

Market

Risk

Life Insurance

Risk

Non-life and

Health Risk

Counterparty

Default Risk

Operational

Risk

Economic

Scenario

Generation

Cash Flow

projection Life

Cash Flow

projection

Non-life /

Health

Cash Flow

projection

Assets

Extr

act Tra

nsfo

rm a

nd

Lo

ad

Co

here

nce c

ontr

ol, v

alid

atio

n and

sig

n o

ff

Valid

atio

n a

nd

sig

n o

ff

Monitor and Control

Results:

SII Balance

sheet, Capital

Requirements

Operational

Losses

Counterparty

information

Life

Liabilities

Model Point

Assets

Model Point

Data

Preparation

Experience

analysis and

assumptions

setting

External Market

Data

Co

here

nce c

ontr

ol, v

alid

atio

n and

sig

n o

ff

MVL & Tech

Provisions Life

MVL & Tech

Provisions Non

life and Health

Asset

Pricing

Internal

Reports

2 Modelling in Solvency II: from regulation to practice

© 2010 Deloitte

5 options can be envisaged which will influence the level of complexity and the

accuracy of the implemented SCR calculations.

What are the Solvency II options for the calculation of

the SCR?

Simplified calculation in the Standard

formula

Standard

Formula

Standard

Formula using own

data to

calibrate the parameters

in the risk modules

Partial

Internal Model

Internal

Model

Complexity,

Accuracy

Complexity,

Accuracy

1

2

3

4

5

And of course, don’t forget Technical Provisions …


© 2010 Deloitte

77%

76%

74%

71%

71%

71%

71%

64%

64%

62%

60%

56%

54%

54%

53%

49%

48%

47%

45%

44%

44%

43%

41%

37%

37%

36%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Liquidity

Regulatory/compliance

Market

Credit

Legal

Budgeting/financial

Tax

Mortality

Liability management

Privacy

Morbidity

Business continuity/IT security

Country/sovereign risk

Fraud

Property and casualty

Reputation

Strategic

Operational

Lapse

Data integrity

Vendor/service provider

Human resource

Model

Systemic

Catastrophe

Geopolitical

How effective do you think your organization is in managing

each of the following types of risks?Percent responding extremely or very effective

7th Edition Global Risk Management Survey Review, Draft Report


© 2010 Deloitte

Avatars of model risk

There are many aspects to model risk. For instance:

• The model design, and the choices made in representing reality, do not correspond

to the objectives of the model. I.e. some essential features of the economic reality or

of the contracts modelled, are not captured.‒ Pricing model for mortgages that ignores prepayment risk; valuing a life insurance company while ignoring

the cost of guarantees, correlation effects in defaults,…

• Somewhere along the lines of the mathematical developments, a mistake was

made. Describing the setting correctly is no guarantee for a correct solution …‒ Using results that only hold for a normal distribution in credit or operational risk modelling

• A model is used for a purpose it was not designed for. For instance, a model may

only work for a specific range of market parameters, but is applied outside that range‒ A model developed for valuing a complex financial instrument for accounting purposes, where there is a

materiality concept and where timeliness is key, may not be the best guide in setting up the hedge, since

it is probably a simplified model.

• Absence of accurate and adequate input data leads to non-sensical output‒ Correlations, length of the liquid market curve, unlisted equity, historical loss data, mortality trends,…

• The model’s implementation in a software environment, does not correspond to the

specifications‒ Numerical instability e.g. Excel has its limits

‒ Software bugs, version control, use of libraries from vendors,…


© 2010 Deloitte

Agenda

Some ideas on model governance2

Adding value through model validation3

Solvency II Internal model requirements1

Model documentation4


© 2010 Deloitte

How will internal models be approved?

Internal model tests (principles-based approach)

To gain sign-off an internal model needs to pass six tests which need to be repeated when

the model is changed to enable on-going appropriateness of the model. Where the

application for that approval relates to a partial internal model, the requirements shall be

adapted to take account of the limited scope of the application of the model.

The tests explicitly refers to economic capital under Pillar I.Use test

(Article 120)

Demonstrate that the internal model is widely used and plays an important role in system governance , in particular in the risk-management system en decision-making processes, the economic and Solvency Capital assessment and allocation processes, including ORSA

Demonstrate that internal model complies with adequate actuarial and statistical techniques and data quality requirements. Statistical qualityVerify it assesses all material risks the company is exposed to and the mitigation actions for these risks taking into account standardspolicyholders and management actions using realistic assumptions.

Calibration Demonstrate calibration details of the internal model and verify the reconciliation to regulatory standard i.e. the level of

standards protection within 1 year being at 99.5% confidence interval.

Profit and loss Demonstrate that the causes and sources of profit and losses for each major business unit are reviewed at least annually attribution and verify how categorisation of risks chosen in the model explains the causes of profits and losses.

Validation Demonstrate that there is a regular cycle of model validation that includes monitoring performance, appropriateness of standards specification and testing results against experience.

Document the design and operational details of the internal model to provide a detailed outline of theory, assumptions and Documentation

mathematically and empirical basis underlying the model and indicate any circumstances where the model does not work standardseffectively.

(Article 121)

(Article 122)

(Article 123)

(Article 124)

(Article 125)


© 2010 Deloitte

Internal Model

Insurers have the choice of:

• Standard model

• Partial internal model

• Full internal model

Internal

Model

Risk Strategy

• Risk appetite

• Understanding risk impacts

• Accountability

Information

• Data

• Assumptions

• Technology

Management

• Decision Structure

• Remuneration Policy

• Responsibilities

Decision Making

• Pricing

• Capital allocation

• Reinsurance

• Planning

• Interaction and

integration

‒ Reserving

‒ Pricing

‒ Planning

‒ Management information

• Parameters

‒ Frequency

‒ Interaction and

consistency

‒ Ownership

‒ Sign-off


© 2010 Deloitte

Internal model

Requirements for Approval

Approvalcriteria

Use test

ModelGovernance

Statistical quality

standards

Calibrationstandards

Profit and loss statement

Validation standards

Documentation standards

Is the internal model

adequately documented?

Is the internal model widely accepted within

the organisation and does it capture the

fundamental risks?

Are the quality of the data,

assumptions and methods

used sufficient?

Does the internal model

produce the same level

of prudence as the

standard formula?Can the realised gains and losses be explained

using the internal model?


produce the proper

results?

Is there an ongoing feedback

loop between the administrative

and management board (AMB)

and the risk management

function?

Solvency II Directive, Articles 120 - 126

CEIOPS Advice on Level 2 Implementing measures (former CP 56)9 Modelling in Solvency II: from regulation to practice

© 2010 Deloitte

Use Test

General Principles

Theinternal model

…

… is well understood by

senior management

… is appropriate in the business

model

… covers risk sufficiently to be

useful in risk management and decision making

… is widely integrated into

the risk management

system

... is integrated into the risk

management system on a

consistent basis for all uses

… is used to support and

verify decision making

… is used to calculate SCR

… is used to improve the risk

management system

… is designed to facilitate the analysis of business decisions

Is the internal model widely accepted within

the organisation and does it capture the

fundamental risks?

Solvency II Directive, Articles 120

CEIOPS Advice on Level 2 Implementing

measures (former CP 56)


© 2010 Deloitte

Use Test

Some examples on the use of the internal model

Capital

Risk

appetite

Strategy

• Business strategy

should be formulated

and implemented in

daily operations.

• The risk strategy has a

direct impact on capital

requirements

• The risk appetite should be determined and be related to the strategy and solvency position. The

appetite depends on available capital and the willingness to take more risks in exchange for a

higher expected return.

• The risk appetite depends on the risk-taking capacity in terms of both capital and organisation

• Volatility of capital is

required to withstand

unexpected losses. The

volatility depends on the

adopted risk management

strategy.

• A high risk profile requires

more capital, with the

possibility of a higher return

on capital. An insurer should

understand their risk

appetite and the resulting

capital requirements.

An important aspect of the use test is the interaction between strategy, capital

and risk appetite


CEIOPS Advice on Level 2 Implementing measures (former CP 56)


© 2010 Deloitte

In which areas do you plan to use your internal model as

part of the decision-making process for Solvency II?

87%

87%

80%

67%

67%

60%

60%

53%

47%

40%

33%

33%

20%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Risk-based performance reporting

Capital management and planning

Management information, e.g., providing information on how the risk position compares …

Decisions on asset mix strategy and the possible effects of investment decisions

Strategy and planning, e.g., as an input to planning and strategy by providing an …

Analysis, design or purchase of reinsurance

Pricing of business

Assessment of the risks, value and impact to the business of potential mergers, …

Product development

Prioritization of risk management activity

Purchase of hedging assets or changes to existing hedges

Excess surplus investigations (with profit funds)

Executive compensation



© 2010 Deloitte

Use Test

Some examples on the use of the internal model

Governance systemRisk management

systemDecision Making

Capital Assessment

and Allocation

• Connection between

the internal model

and technical

provisions


the internal model

output and internal

and external

reporting


the internal model

and the technical

implementation of

management actions

• Measurement of

material risks

• ALM

• External risk

reporting

• Internal risk reporting

• Draft reinsurance

program

• Development of risk

strategies

• Efficient use of

Capital

• Investment Decisions

• Identifying targets for

return on capital and

rewards

• Product development

and pricing

• Business planning

and strategy

• Company policy for

taking on risk

• Capital management

• ORSA

• Calculation of SCR

• Economic capital

allocation for each

entity, business line,

risk or business unit

• Solvency capital

allocation by entity,

business line, risk or

business unit




© 2010 Deloitte

The Statistical Quality Standards philosophy: an internal model is not just a

black box or an expert with good predictive power for the probability

distribution forecast.

Statistical Quality StandardsAre the quality of the

data, assumptions and

methods used sufficient?

Potential discussion areas:

• Probability distribution forecast (does a probability distribution forecast always has to

consist of a full distribution, i.e. whether every quantile must be known?)

• The use of expert judgement related to data.

• Materiality (circumstances under which risks which are in the scope of the internal

model must actually be modelled in the internal model?)

• Aggregation in internal models

The methods used to calculate the probability distribution forecast shall be based

on:

• Adequate actuarial and statistical techniques

• Consistency of calculation methods used for the probability distribution forecast and

technical provisions

• Current and credible information

• Justification of underlying assumptions




© 2010 Deloitte

Statistical Quality StandardsAre the quality of the

data, assumptions and

methods used sufficient?

• Risk ranking & model coverage‒ Relative importance of risks, comparables, relative classification

‒ Link with use test: coverage, resolution, consistency,...

• Recognition of diversification effects‒ What drives dependence? Is there diversification gain?

‒ Account for extreme scenarios and tail dependence

‒ Part of business decisions?

• Recognition of risk mitigation‒ Economic view, but legal effective/enforceable?

‒ How incorporated in model?

• Financial Guarantees and options‒ Identify and document

‒ Influencing factors

‒ Coherence with TP

• Future management actions


© 2010 Deloitte

The Calibration Standards aim to assess whether the SCR derived from the

internal model has the appropriate level of prudence.

Calibration StandardsDoes the internal model

produce the same level of

prudence as the standard

formula?

Article 122.1

Insurance and reinsurance undertakings may use a different time period or risk

measure than that set out in Article 101(3) for internal modelling purposes as long as the

outputs of the internal model can be used by those undertakings to calculate the

Solvency Capital Requirement in a manner that provides policy holders and

beneficiaries with a level of protection equivalent to that set out in Article 101.

Article 101(3)

The Solvency Capital Requirement shall be calibrated so as to ensure that all

quantifiable risks to which an insurance or reinsurance undertaking is exposed are taken

into account. It shall cover existing business, as well as the new business expected to be

written over the following 12 months. With respect to existing business, it shall cover only

unexpected losses.

It shall correspond to the Value-at-Risk of the basic own funds of an insurance or

reinsurance undertaking subject to a confidence level of 99,5 % over a one-year period.




© 2010 Deloitte

Calibration Standards

Different time period and risk measure:

• If the time period used is different from the one set out in Article 101:

‒ Demonstrate that the time effects of the risks are taken into account

‒ Demonstrate that all significant risks over a one-year-period are properly managed

‒ Give special attention to the choice of the data used

‒ Justify the choice of time horizon in view of the average duration of the liabilities of the

undertaking, of the business model and of the uncertainties associated with too far time horizons

• Consider the practical implications if the undertaking uses different time periods and/or risk

measures within the same internal model, especially when aggregating the capital for the different

risks.

• When taking this approach, the undertaking shall show that the level of protection provided by

the SCR is equivalent to that set out in Article 101(3), and specifically that the approach taken to

aggregate the risks is appropriate.

Equivalent protection of policy holders:

• If the SCR cannot be derived directly from the probability distribution:

‒ Explain how risks are rescaled and justify that the bias introduced when doing so is immaterial.

‒ Explain the shortcuts used to reconcile the outputs of its internal model with the distribution of

the Basic Own Funds, if any.

‒ If considering a longer time horizon than that set out in Article 101(3), show due consideration of

the solvency position at the earlier time horizons.

‒ If considering a different time horizon than that set out in Article 101(3), justify the particular

assumptions made in order to properly take into account the dependencies between

consecutive time steps.


produce the same level of

prudence as the standard

formula?




© 2010 Deloitte

The Profit and Loss attribution aims to demonstrate how the categorisation

of risk chosen in the model explains the causes and sources of profits and

losses.

Profit and loss attributionCan the realised gains and

losses be explained using the

internal model?

Profit and loss attribution and the Use test

• The P&L attribution will give the undertaking information relating to the risk profile of the

undertaking, and therefore this information is also used in the ORSA.

• The P&L attribution may also provide an unbiased view on the risks of the portfolio to better

understand the portfolio exposures and assess whether the risk management framework is

appropriate.

• The results of the P&L attribution can also be used for other internal purposes such as

budgeting, forecasting, reinsurance-program testing.

Form of profit to be taken

• The variable may differ from basic own funds, because a different internal definition may be used

for economic capital resources.

• Be aware what causes differences in the profits and losses used in the Profit and loss attribution

and the profits and losses reported in the accounting systems.

Categorisation of risks

• The economic capital requirements resulting from the internal model shall lead directly to a

categorisation of all material risks. The qualitative assessment of non-material risks or non-

quantifiable risks completes the categorisation of risks based upon the internal model results.




© 2010 Deloitte

Agenda






© 2010 Deloitte

Internal Model Governance under SOLVENCY II regulationThe administrative or management body (high level internal model governance):

• Approving the application for approval to use the internal model to calculate the SCR, and the application for approval

for major changes or extensions to the model.

• Deciding roles and responsibilities for the internal model governance.

• Deciding on the strategic direction of the model and hence any changes to the model.

• Agreeing major changes in advance of the change being made.

• Aligning the model design and operations with the undertaking’s risk profile and operations.

• Ensuring there are sufficient resources to develop, monitor and maintain the model.

• Monitoring on-going compliance with the requirements for internal model approval, and informing the

supervisory authorities if the model ceases to comply.

• Ensuring there are adequate independent review procedures in place around the internal model design, operation and

validation.

• Ensuring that outputs are aligned with use – i.e. that the management information produced by the model assists in

decisions made at Board level.

• If the internal model ceases to comply with the requirements for approval, the administrative or management body

must ensure that a plan to restore compliance is developed or assess the non-compliance as immaterial.

The risk management function (detailed internal model governance):

• Design and implementation of the internal model.

• Testing and validation of the internal model.

• Documentation of the internal model and any changes to it.

• Analysing the performance of the internal model, and reporting on the performance to the high-level

governance, including compliance with the internal model approval requirements.

• Suggesting areas for improvement and reporting on the status of efforts to improve previously identified weaknesses

to the high level governance.

• Liaise closely with users of the outputs of the internal model.

• Develop a communication loop with the actuarial function to pass the detailed actuarial perspective to the risk

management function and in return receive the insights on the internal model.

Ongoin

g feedback lo

op

Undertakings may use external validation

review, however the ultimate responsibility

for signing off the appropriate validation

processes shall fall on the board, and may

not be delegated to any third party.

The administrative,

management or

supervisory body may

set up an internal

control committee


© 2010 Deloitte

Conceptual considerations

• Model Governance should support the

achievement of at least two objectives:

‒ The orderly development and usage of models

‒ The containment of model risk in the organisation

• Some criteria that can be used to prioritise the

need for model governance in the universe of

models discussed before:

‒ Regulatory expectations (explicit and implied)

‒ Stakeholder expectations

• Counterparts like reinsurers, dealing rooms,...

• Credit rating agencies

• Clients (e.g. Investment models)

‒ Risks for the group

• Financial risks; e.g. due to mispricing,

mishedging

• Reputational risks

• Materiality concept to be incorporated!

‒ The assessment of model risk, as hinted at in the

previous section

‒ Cost of installing a rigorous process vs. the

expected benefitsFDIC, Supervisory Insights, Winter 2005, Vol. 2, Issue 2.


© 2010 Deloitte

A bit more pragmatic thinking

• Any fair value or risk model is composed of three

ingredients:

1. A mathematical recipe, the model

2. Position data/contract data

3. Model data (e.g. market data such as interest rates; lapse

rates; correlations, behavioural parameters,...)

Similar set-up seems possible for most models.

• Of these three ingredients, only the mathematical part

can be legitimately be claimed to require dedicated

expert skills.

• Different departments have different skills/roles relevant

for each of the above ingredients:

‒ Risk department often is a knowledge centre for

mathematical modelling within the organisation (so, it can

act as internal consultant, even if it is not the

“owner”/”responsible” for the model)

‒ Under Solvency II, the risk department gets the model

validation role explicitly

‒ Setting of all sorts of parameters is a management decision,

and should remain their ultimate responsibility

‒ The gold copy of any contractual data will mostly be the

financial accounts (since audited, and rigorous internal

control procedures mostly apply to them)

Fair Value Or

Risk Measure

Contract data Market & behavioural data

A mathematical recipe

Reasonably easily

verifiable

Least accessible to broad user base

Ease of control depends on

market or risk type

A tentative role for Finance Department:

• In Risk Models

‒ Ensuring accurate and complete

capture of exposure data

‒ Back-testing any performance

predictions coming from the risk

models

• In Fair Value models

‒ Validate consistency of parameters

across the group

‒ Understand sensitivity w.r.t. to

parameters


© 2010 Deloitte

Conceptual considerations

• It would be best practice to ensure that there is

always a 4-eyes principle applied. The rigour with

which this is applied should be function of the

regulatory expectations, the risk the model presents

to the company,...

• In terms of controlling model risk, there are at least

two control layers:

‒ MODEL VALIDATION: model validation is a formal

process, that challenges the assumptions, mathematical

algorithm, the implementation in a tool of a model.

‒ MODEL REVIEW: a set of periodic activities, not

necessarily requiring in-depth mathematical skills, but

that are confronting the “predictions” of the model with

reality. E.g. realised losses or earnings over the first

quarter, prices realised in actual transactions, excess

losses beyond the VaR amount,... Such controls can

provide indications that the model is not reliable.

• The question of scope. Which models will you include

in such a formal process?

Often performed by professionals close to

the business, like performance

controllers,….

Requires in-depth technical modelling

skills, so mostly concentrated in an expert

centre, often within the risk department


© 2010 Deloitte

An example: ALM model

Potential allocation of responsibilities related to ALM modelling and ALM

management

What Who Performs Who Controls (second line of defense)

Development of the model Risk departmentIndependent validation by a model validation team or

external third party

Running the model (including

setting of model data and

uploading contract data)

Balance sheet management/ALM

department

• Model data subject to challenging validation of the

Risk department (formal sign off)

• Position data subject to formal reconciliation

process (documented checklist) at each run,

possibly in collaboration with finance department

Periodically reviewed by internal audit

Analysis of the results and

suggestions for management

action


department

• Formally reviewed by Risk department before

being presented to ALCO/MT Finance

Analysis of new balance sheet

management

proposals/instruments


departmentValidation and challenge by Risk department

Back testing of earnings

forecasts by reconciliation with

realised

Finance department (financial

controlling)NA


© 2010 Deloitte

Agenda






© 2010 Deloitte

In designing the internal model, the Level 1 Text allows a large amount of freedom. Two examples of

this are given below:a. Article 121(4) sets out that “No particular method for the calculation of the probability distribution forecast shall be

prescribed.”

b. Article 122(1) sets out that “Insurance and reinsurance undertakings may use a different time period or risk

measure than that set out in Article 101(3) for internal modelling purposes as long as the outputs of the internal

model can be used by those undertakings to calculate the Solvency Capital Requirement in a manner that provides

policy holders and beneficiaries with a level of protection equivalent to that set out in Article 101.”

In addition to being used for the undertaking’s own risk management purposes, the internal model is

also used to calculate the regulatory capital. Materially misstating the regulatory capital, especially

holding regulatory capital that is not high enough, will result in a decrease in the level of the policy holder

protection provided by the undertaking.

In addition, as the internal model should also be widely used in and play an important role in the system

of governance, any material misestimation within the internal model will affect not only policy holder

protection, but also the whole risk management and decision making processes of the undertaking.

Therefore, supervisory authorities will require undertakings to take appropriate steps to validate

that the internal model is appropriate for the calculation of regulatory capital to ensure that the

level of regulatory capital is not materially misstated so as to decrease the level of the policy

holder protection, for use within the undertaking’s risk management and decision making processes.

It is the undertaking which has the primary role in this validation process, not the supervisory

authority. The validation process ≠ the approval process, in respect of which the supervisory authority

needs to take a decision. As part of the approval process, the supervisor will need to evaluate the

validation processes which the undertaking has in place.

Introduction to the validation cycle


© 2010 Deloitte

Model validation function

Model validation is a key activity to help make sure that models function as

intended, both when they are implemented and over time. Ongoing monitoring

and validation of risk management models are important in order to assess a

model’s sensitivity to structural changes and to changes in parameters and

assumptions. Fifty-nine percent of institutions reported having a model

validation function, an increase from 53 percent in 2008 . Larger

institutions were more likely to have a model validation function, with 79

percent institutions with more than $100 billion in assets have such a

function, up from 66% in 2008.

Model validation is most often placed in an independent risk management

function. Among institutions with a model validation function, 65 percent reported

that this model validation resides in independent risk management, while 19

percent placed it in Internal Audit and 8 percent in Actuarial. Larger institutions

were even more likely to have risk management handle model validation. Among

institutions with more than $100 billion in assets, 77 percent said that model

validation was placed in independent risk management.



© 2010 Deloitte

The validation policy which sets out the way in which undertakings will validate

their own internal model and why that way is appropriate.

How to demonstrate compliance: The validation policy shall set out at least the following items:

• Purpose and Scope of Validation: the areas of the internal model that need to be validated shall

include at least:‒ Data

‒ Methods

‒ Assumptions

‒ Expert judgement

‒ Documentation

‒ Systems/IT

‒ Model governance

‒ Use test

• Tools used: may be a mathematically well defined test, a qualitative judgement or any other

process designed to gain comfort that the internal model is appropriate and reliable. ‒ Some validation tools will need to be used by all undertakings using an internal model to calculate the SCR:

• Testing results against experience

• Testing the robustness of the internal model (shall include at least sensitivity testing)

• Stress and scenario testing

• Profit and loss attribution

‒ Examples of what further validation tools undertakings may want to use:• Benchmarking

• Analysis of change

• Hypothetical portfolio

• Qualitative reviews

The validation policy

Modelling in Solvency II: from regulation to practice28

© 2010 Deloitte

• Frequency of validation process

• Governance of validation results‒ The risk management function shall be tasked with the validation of the internal model.

‒ Nevertheless, certain parts of the validation process may be carried out by other parts of the undertaking, as

long as there are clear lines of reporting and the risk management function retains overall responsibility for the

validation process.

‒ Set out how the senior management is involved in the validation processes

‒ Consideration needs to be given to what is validated at group level and what is validated at the related

undertaking level

• Limitations and future developments: expect that the model and its associated validation

processes will constantly develop as the risk profile of the undertaking develops and as new

validation tools become available

• Documentation of the validation policy

• Independent review: set out how independent review, external or internal, is being used within the

validation process

The validation policy (cont.)

Modelling in Solvency II: from regulation to practice29

© 2010 Deloitte

Internal Model Change Policy

Model Change Policy: The development of a model change policy is a requirement on

all undertakings applying for approval to use the internal model to calculate the SCR.

This is part of the application and must be approved by the supervisory authorities.

Model changes

• Major changes to the internal model are subject to prior supervisory approval

• The amended internal model should not be used to calculate the SCR until approval is granted

• Undertakings may further assign model changes to the following subcategories:

‒ The calculation kernel of the internal model

‒ The risk management

‒ The internal model governance

‒ The existing and regulatory approved internal model change policy of the undertaking

‒ Other aspects of the internal model

The policy for model changes shall contain an indication of the undertaking internal governance for

changes to the model (e.g. internal approval of changes, escalation path, internal communication,

documentation and validation of changes, etc.).


© 2010 Deloitte

Model risk and an extra capital buffer under ORSA?

CEIOPS Level 2: Technical Provisions – Article 86 h Simplified methods and techniques to

calculate technical provisions

3.109 Therefore the undertaking shall assess the model error that results from the use of a given

valuation method, having regard to the nature, scale and complexity of the underlying risks The

valuation method should be regarded as proportionate if the model error is expected to be non-

material.

3.110 For this purpose the undertaking should define a concept on materiality which should lay down

the criteria on basis of which a decision on the materiality of a potential misstatement of technical

provisions is made. This materiality concept and should be reflected in the undertaking’s own risk and

solvency assessment (ORSA).

3.112 An assessment of the model error may be carried out, by:

• Sensitivity analysis in the framework of the applied model: this means to vary the parameters

and/or the data thereby observing the range where a best estimate might be located.

• Comparison with the results of other methods: applying different methods gives insight in

potential model errors. These methods would not necessarily need to be more complex.

• Descriptive statistics: in some cases the applied model allows the derivation of descriptive

statistics on the estimation error contained in the estimation. Such information may assist in

quantitatively describing the sources of uncertainty.

• Back-testing: comparing the results of the estimation against experience may help to identify

systemic deviations which are due to deficiencies in the modelling.


© 2010 Deloitte

Making model validation concrete and tangible


• The following areas related to an internal model need to be

considered and addressed in the context of a model validation:

• The level II CEIOPS also suggests the following validation tools:

Use TestExpert judgment

AssumptionsModel

Governance

Methods Systems/IT

Data Documentation

MUST CONSIDER MAY CONSIDER

Testing against experience Benchmarking

Robustness Analysis of change

Stress & Scenario Testing Hypothetical portfolios

P&L attribution Qualitative review

© 2010 Deloitte

Model Review Questions to address

Reading documentation of the

ESG, and assessment of some

content questions

• Is the objective of what needs to be modelled clearly stated?

• Does the documentation justify the choice of approach?

• Are the strengths and weaknesses of the chosen approach explained?

• Is part of the documentation readable by an interested, non-technical user,

and will it allow that person to assess strengths and weaknesses of the

approach? I.e. roughly speaking, there would be three levels of

documentation, one for non-technical users, one providing the full

mathematical detail, and one describing the implementation in a tool and the

process to be used to run the model.

• Is the calibration process well-described (market data to be used,

formulae,...) both theoretically and operationally?

• How is the choice for other asset classes (real estate, shares) modelled

based on the interest rate model, justified? How are correlations accounted

for/incorporated?

Review the mathematical

model

• Check all formulae, either by reference to the literature (include in review file)

or by recomputation (document in file). This includes the stochastic

simulation engine, the closed form formulae e.g. used for calibration, the

numerical discretisation schemes,...

Acceptability of the model for

the stated aim

• Benchmarking with approach/model chosen in the literature, by peers or in

vendor packages. Collect and store benchmark information.

• How was the choice of model argued in the past? Who approved its use?

Has it been subject to review/challenge by (senior) users?

• Where is the ESG used in practice in the organisation (as compared to

stated objectives)? Is this acceptable given its characteristics? (USE test)

ESG Validation Workplan along the above lines (1/4)


© 2010 Deloitte

Input Testing Questions to address

Input Data • Review of the procedures used to extract input parameters (e.g. implied or

historical volatilities, zero coupon curve, risk premia) from market data.

• Review of interpretation and acceptance controls of such data when

received from vendors.

Stability of the calibration • Hull-White type models are calibrated using a minimization on a set of

chosen derivatives for which the price is known (swaptions, caplets).

Impact of choosing a different set of instruments? Impact of choosing a

different seed point for the optimisation?

• How is correlation between stochastic processes linked to observable

parameter of correlation between asset classes?



© 2010 Deloitte

Computational Process Questions to address

Review of the code • Does the code correspond to what was documented in the model?

• Have pre-coded subroutines been tested/assessed or is there general

documentation available? (e.g. random number generation for a given

distribution, numerical minimizers)

Assessment of the numerical

procedures

• Impact of the discretisation, no creation of fictitious value ("1=1" test,

martingale test)?

• Is the starting yield curve found back as E[Product(exp(-r.dt))]?

• Are the original parameters such as volatilities and correlations that one

calibrated again, found back in the runs?

• Stability for extreme values of parameters (inverted curve, high volatility,

no mean reversion/strong mean reversion, …)?

• Sensitivity relative to the parametrisation/interpolation/extrapolation for the

yield curve to which the mode is calibrated?

• Comparison of the simulated value of an instrument with the closed form

formula (in the chose model) if available?

• Convergence enhancing tricks, such as control variates or anti-thetic

variables.

• Numerical stability of the simulated variables in particular for longer time

horizons or extreme environments (high or low rates, ...)?

• Estimation error in function of the number of simulation runs (e.g. for

known instruments)?



© 2010 Deloitte

Outcome Analysis Questions to address

Back Test • Review the implemented back testing procedures. If incomplete, design

and possibly perform some back testing procedures: shape of the

distributions for rates that are generated e.g. the 10y swap rate in 20years;

standard deviation of these distributions,…

Sensitivity analysis • Controlled assessment of the impact on the output of changing input

parameters. The output here should primarily be interpreted as being the

value or risk measures that one wants to compute with the ESG. This step

may therefore require considering the ESG and the applications that rely

on it, jointly. The purpose is to indentify and separate the important from

the less important input parameters, in order to focus the calibration and

expert judgement controls.

Use of scenarios • How are the generated scenarios stored? Are the labels of the data stored

aligned with their mathematical meaning and definition?



© 2010 Deloitte

Agenda






© 2010 Deloitte

The Documentation standards aim to instill a

culture of good documentation praxis leading to

all model developments being documented.

38

Documentation

standards

The documentation shall demonstrate compliance with the internal model tests.• Detailed outline of the theory, assumptions, and mathematical and empirical basis underlying the internal model.

• Describe the drawbacks and weaknesses of the model, including the circumstances under which the model does

not work effectively, such as:‒ Any specific features of the internal model or circumstances or limitations that present potential concerns or would significantly

increase the uncertainty of the results of the internal model beyond what would reasonably be expected. For instance:

• Limitations in risk modelling and the cover of risk captured

• The nature, degree and sources of uncertainty surrounding the results of the internal model and sensitivity to key assumptions

• Shortcoming and/or deficiencies in input data

‒ Insufficiencies in IT-systems, governance and related controls surrounding the internal model.

• Tailored for key bodies and key personnel

• Include evidence such as training that all levels and functions of management, for example the board, senior

management, and the internal audit, of the undertaking understand the relevant aspects of the internal model

• Include a list of all documents that the undertaking considers relevant to the internal model, and where and how

these documents can be accessed.

• Identify those responsible for pulling together and/or updating documents.

The documentation does not have to be one single document, provided there is a list or a mapping

process that brings it all together.

The documentation of an internal model shall be thorough, sufficiently detailed and sufficiently complete

to satisfy the criterion that an independent knowledgeable third party could form a sound judgment as

to the reliability of the internal model and the compliance with Articles 120 to 126 and could understand

the reasoning and the underlying design and operational details of the internal model.

Modelling in Solvency II: from regulation to practice

© 2010 Deloitte

Documentation: Qualitative Questions from the QIS 5


1. Model description and overview

General Model Governance:

2. Policies, controls and procedures for the management of the internal model

3. Model change policy

4. Evidence of Use Test

5. Training Manual for Management and staff training in use and understanding of

Internal Model

Model Changes:

6. Record of Major and Minor changes of the model

Technological Specifications:

7. Description of the Information Technology platform(s) used in the internal model

8. Description of Contingency plans relating to the technology platform(s) used

9. User guide

10. Source code

© 2010 Deloitte



Data:

11. Data policy

12. Documentation evidencing or justifying the accuracy, completeness and

appropriateness of the data

13. Data directory

Statistical Quality Standards:

14. Detailed description of Internal Model Methodology

15. Description of underlying assumptions

Expert Judgement:

16. Description of where Expert Judgement is applied in the model

17. Validation of Expert Judgement as applied in the model

Calibration:

18. If your model output uses a risk measure other than 1-year VaR at 99.5%, do you

have documentation evidencing or verifying that the chosen risk measure is at least as

strong as 1 year VaR at 99.5%?

© 2010 Deloitte



Profit and Loss Attribution:

19. Results of Profit and Loss Attribution

Validation:

20. Description and report/results of Validation Tests

Scope:

21. Partial Internal Model Scope

22. Qualitative and Quantitative indicators for the coverage of risk

Other:

23. Description of risk mitigation techniques accounted for in the internal model

24. Description of Future Management Actions accounted for in the internal model

25. Description of known internal model shortcomings / weaknesses, including

circumstances under which the internal model does not work effectively

© 2010 Deloitte

Our contact details

Email: [email protected]

Phone: +32 2 800 20 45

Yves Dehogne

Audit Partner

Partner, Bedrijfsrevisoren


Phone: + 32 2 800 24 73

Arno De Groote

Director, Enterprise Risk Services

Director, Actuarial & Financial Risk Advisory


Phone: + 32 2 800 21 46

Dirk Vlaminckx

Audit Director

Director, Bedrijfsrevisoren


Phone: + 32 2 800 24 94

Christophe Vandeweghe

Manager, Enterprise Risk

Services

Manager, Actuarial & Financial Risk Advisory


mailto:[email protected]




About Deloitte

Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private

company limited by guarantee, and its network of member firms, each of which is a

legally separate and independent entity. Please see www.deloitte.com/about

detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and

its member firms.

Deloitte provides audit, tax, consulting, and financial advisory services to public

and private clients spanning multiple industries. With a globally connected network

of member firms in more than 140 countries, Deloitte brings world-class

capabilities and deep local expertise to help clients succeed wherever they

operate. Deloitte's approximately 170,000 professionals are committed to

becoming the standard of excellence.

Deloitte in Belgium

A leading audit and consulting practice in Belgium, Deloitte offers value added

services in audit, accounting, tax, consulting and financial advisory.

In Belgium, Deloitte has more than 2,240 employees in over 12 offices across the

country, serving national and international companies, from small and middle-sized

enterprises, to public sector and non-profit organisations. The turnover reached 317

million euros in the financial year 2009.

The Belgian practice is part of the international firm Deloitte Touche Tohmatsu

Limited. For the fiscal year ended 31 May 2009, DDTL realised a turnover of

US$26.1 billion worldwide, with approximately 170,000 members of staff in more

than 140 countries.

This document is confidential and prepared solely for your information.

Therefore you should not, without our prior written consent, refer to or use our

name or this document for any other purpose, disclose them or refer to them

in any prospectus or other document, or make them available or communicate

them to any other party. No other party is entitled to rely on our document for

any purpose whatsoever and thus we accept no liability to any other party who

is shown or gains access to this document.