View
86
Download
0
Category
Preview:
Citation preview
W W W . C H I C A G O L A N D R I S K F O R U M . O R GW W W . C H I C A G O L A N D R I S K F O R U M . O R G
What’s New in Risk Assessment?
22
33
Risk Management Depends on Risk AssessmentThe simplest definition of Risk Management involves 3 steps:
55
Risk Assessment Has Many Moving Parts!
66
Strategic Risk Assessment: What is Important to Achieving Organizational Objectives andNot Under [Complete] Control?• Identifying threats and exposures without measurement only
generates lists -- that may or may not be applicable or important to the organization.
• Some ERM projects create spreadsheets full of “Critical Risks” that frustrate management and fail to provide a blueprint for action.
• Instead of identification run rampant, Strategic Risk Assessment starts with corporate objectives and considers what is at risk, identifies potential threats, and assesses the impact and the effectiveness of current controls to counter those threats – and points to controls where objectives are threatened.
77
Strategic Risk Assessment IssuesTo be effective, risk assessment cannot be merely checklists or a process that is disconnected from business strategy. • Risk Assessment must be integrated in a way that provides
timely and relevant risk information to management. • For risk management to be a strategic process, risk assessment
must be owned by the business units and be embedded within the business cycle, starting with strategic planning.
• And: Risk assessment begins and ends with the organization’s specific objectives.
8
Strategic Risk Assessment
Qualitative AnalysisRisk Register ◄
Risk Map ◄Risk Categorization ◄
Quantitative Analysis► Decision Tree Analysis► Scenario Analysis► FMEA► Simulation & Modeling
99
Risk Assessment Basics• It is a matter of widespread understanding that risks should be
assessed in terms of the likelihood (probability) that an uncontrolled event will occur and the consequences (impact) to achieving one or more organizational objectives.– Applicable to both qualitative and quantitative methods of
assessment.
• Strategic Risk Assessment requires pursuing a systematic, logical set of actions to identify the magnitude of hazards and exposures, assess threats, and implement controls to mitigate, eliminate or control high-risk conditions.
Risk Maps Are Primarily Qualitative Assessments
Qualitative Methods & Risk Maps Highlight Critical Threats
Data gathering & representation
Select appropriate techniques(s)
Risk analysis & modeling Expert judgment
RISK
But Quantitative Methods are Often Required to Identify Corrective Actions
1313
Risk Assessment Tools & Techniques Are Rapidly EvolvingRisk Assessment needs to move beyond Probability x Severity and Risk Maps to evaluate emerging issues, warning & detectability, and other key threats to strategic objectives.• Over the past decade, developments in economic and
financial theory -- plus computing and data advancements –are providing new methods for quantitative risk assessment, as well as improvements to existing techniques.
• Risk Managers should understand available risk assessment techniques and adopt a set of tools they can apply to their organization's unique Risk Management requirements.
1414
Three Basic Types of Quantitative Assessment Tools – In Order of Complexity
1.Comparative methods; 2.Temporal methods; and,3.Functional methods.
1515
Comparative Assessment MethodsA Comparative Analysis takes an explicit standard – eg., “Best Practices”– and compares a system, process and/or set of procedures to that standard, resulting in a “Gap Analysis”. • A “good standard” is prepared and maintained as “the distillation of
continually developing expert opinion and experience in the face of a continually changing environment”.
• One of the strengths of this approach is its simplicity. Comparative methods can be ideal for organizations as they begin to focus attention on specific systems, processes or threats.
• A weakness is that there is no explicit list of threats as there is in other approaches.
1616
Sample “Best Practices” Matrix –Claims Handling
O Managerial OversightP PrimaryS SecondaryC Consultative InputD Data Resource
Dire
ctor o
f Ins
uran
ce
Dire
ctor o
f Leg
al Su
ppor
t &
Claim
s
Exec
utive
Vice
Pres
ident
, Aon
Senio
r Vice
Pres
ident
, Clai
ms
Vice
Pres
ident
, Clai
ms
Assis
tant
VP,
Claim
s
Senio
r Con
sulta
nt, C
laim
s
Senio
r Clie
nt Sp
ecial
ist,
Claim
s (M
egan
)
Senio
r Clie
nt Sp
ecial
ist,
Claim
s (M
arth
a)
Claim
Ass
istan
t
CLAIM MANAGEMENT PROCEDURES
1)Establish formal claims service standards for TPA's, carriers and other vendors C P C P C C
2)Develop annual written service plan for TPA's and other vendors and monitor performance C P O P C C
3) Develop written Claims Procedures or Manual C O C P C C
4)Establish internal claims reporting and management procedures and monitor compliance C O C O P S
5)Develop claim reports, distribute and review with business units as necessary C O C O C P S S S D
6) Maintain listing of all insured claims O O O C P S S S D7) Maintain listing of all self-insured claims O O O C P S S S D8) Establish and monitor WC post-injury management program C O O O D P S9) Manage claims litigation process C O O P D S D10) Administer OCIP claims C O O O P S11) Administer non-litigated GL claims O O O P S S D D12) Administer auto claims O O O P S S D D13) Administer D&O, fidelity, fiduciary, EPL C P C14) Administer Litigated GL claims O O P D S D15) Administer Property claims O O O P D S D16) Pursue subrogation activities O O O P S S S D17) Review losses and identify trends C C C O C P S S S D18) Conduct/coordinate periodic claims audits D D19) Monitor large loss activity C O O P D D
20)
Review and adjust safety/loss control initiatives as needed to proactively treat risk and address trends observed in claims management activities
O C C C D C D
Management assures activ ity is addressed
Principally responsible for driv ing the activ ity
Responsible to perform or drive certain aspects of the activ ity , but is not the leader
Can prov ide guidance or feedback at a high level for activ ity
Prov ides data or information that is used in the activ ity
1717
Sample “Best Practices” Gap Analysis –RM Strategy
1818
Temporal Analysis MethodsA Temporal Assessment applies quantitative tests to a system, process or set of procedures. These “tests” involve analyzing the results of specific threats or attacks against actual protections and controls, subject to some constraints.• Since it is often impractical to test a system directly, a model of the
system is generally used instead. – However, a model introduces the question of fidelity: an
inaccurate model may not only confuse matters; it may provide a false sense of security that is even worse than confusion.
• A key weakness of a temporal method is that it is not possible to model all possible threats; it is not even possible to list them all.
1919
Temporal Method: Scenario AnalysisScenario analysis considers the questions ‘what might happen and what should/would we do?’ It can not only highlight risks and opportunities in the short and long term; but also test the effectiveness and efficiency of specific controls and plans.• The central idea is to consider a variety of possible futures that
include many of the important uncertainties in the system, rather than to focus on the accurate prediction of any particular outcome.
• A strength of scenario analysis is that it can consider “existential threats” that involve large swaths of the organization.
2020
Four Critical Components of Scenario Analysis1. Determining which factors the scenarios will be built around. In
general, analysts should focus on the two or three most critical factors.
2. Determining the number of scenarios to analyze for each factor. Depends upon how different the scenarios are, and how well the results of each scenario can be forecast.
3. Estimating results – e.g., asset cash flows, control failures, unexpected breakdowns, etc. -- under each scenario.
4. Assigning probabilities to each scenario. Note that this makes sense only if the scenarios cover the full spectrum of possibilities; otherwise, the probabilities will not add up to 100%
2121
Sample ScenarioA Scenario Analysis can be used to ensure effective and reliable insurance coverage. • It typically involves sitting down with brokers, underwriters, lawyers,
adjusters and managers to analyze and talk through how each insurance policy would respond to different circumstances.
• The results are compiled in systematic tables and charts that point out problem areas and suggest solutions.
• One of the strengths of Scenario Analysis is that it tests the system itself (or a model), clearing away misconceptions and uncovering specific elements or issues needing attention.
2222
Other Temporal Analysis MethodsThe most important Temporal Assessment methods use Predictive Analytics to not only determine What might happen, but How Much it could impact objectives.• Two useful tools are:
– Decision Tree Analysis; and,– Modeling & Simulation.
2323
Decision Tree AnalysisA Decision Tree is a structure in which each internal node represents a "test" on an attribute; each “branch” represents the outcome of the test; and each “leaf” represents a decision taken after computing all attributes. • The paths from root to leaf represent classification rules:
– A Root node represents the start of the decision tree, where a decision maker is faced with an uncertain outcome. The objective is to evaluate the overall net positive or negative outcomes at this node.
– Event nodes represent outcomes based upon the probable occurrence of various events.
– Decision branches represent choices that are made by the decision maker.
– End nodes represent final outcomes where a payoff value is identified.
2424
Sample Decision Tree: Jenny Lind• Jenny Lind is a writer of romance novels. A movie company
and a TV network both want exclusive rights to one of her more popular works.
• If she signs with the network, she will receive a single lump sum, but if she signs with the movie company, the amount she will receive depends on the market response to her movie.
• What should she do?
Jenny Lind Decision TreeSmall Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Estimated OutcomesEstimated
Likelihood
Root Node
Event Nodes
Jenny Lind Decision Tree - SolvedSmall Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Expected$900,000
Expected$960,000
Best Result$960,000
2727
Modeling & SimulationWhere Scenario Analysis and Decision Tree Analysis are techniques to assess discrete risk events, simulation methods measure continuous risk exposures and outcomes.• Simulations yield a distribution of outcomes rather than a single point
estimate. • One simulation tool is an “Exceedance Probability Curve” that
measures whether an outcome will exceed a specific estimate, based upon predetermined probabilities.
• Simulation has few limitations in terms of events, probabilities and outcomes – very robust models may be constructed, evaluated and displayed graphically.
2828
Simulation Example: Quantifying the Risk of Natural CatastrophesHow do companies prepare for the financial impact of natural catastrophes? How can they possibly have an idea of what the potential cost can be for events that haven't yet happened?Catastrophe Modeling provides the answers. A catastrophe model can be roughly divided into three modules:• The Hazard Module looks at the physical characteristics of potential
disasters and their frequency.• The Vulnerability Module assesses the vulnerability (or
“damageability”) of buildings and their contents.• The Damage Module determines the overall loss distribution for a
specific event by multiplying building values by potential damage.
Sample Catastrophe Model Results
3030
Functional Assessment MethodsA Functional Analysis focuses on specific threats and protections. • A threat model -- a list of system vulnerabilities, and the likelihood of
successful threats against those vulnerabilities -- is weighed against organizational objectives, assets, protections, and the likelihood of available protections successfully defending those assets against specified threats.
• Temporal Assessment methods, such as statistical modeling; and Comparative Assessment techniques, such as expert systems, are often employed jointly.
• The key strength of a Functional Assessment is its ability to consider a wide range of threats, vulnerabilities, assets and countermeasures.
3131
Failure Mode & Effects Analysis (FMEA)FMEA identifies where & how failures can occur within processes and measures the impact of those failures.• The FMEA Process has 4 basic steps:
1. Determine the failure modes of specific process elements; 2. Analyze the effects on other elements and the overall system; 3. Rank criticality; and,4. Identify existing and potential controls.
• FMEA is particularly useful for evaluating critical risks in very complex systems.
FMEA Thought Process
3333
Sample FMEA TemplateAction Results
Item / FunctionPotential Failure
Mode(s)
Potential Effect(s) of Failure
Sev
Potential Cause(s)/
Mechanism(s) of Failure
Prob
Current Design Controls
Det
RPN
Recommended Action(s)
Responsibility & Target
Completion DateActions Taken
New
Sev
New
Occ
New
Det
New
RP
N
Coolant containment. Hose connection. Coolant fill. M
Crack/break. Burst. Side wall flex. Bad seal. Poor hose rete
Leak 8 Over pressure 8 Burst, validation pressure cycle.
1 64 Test included in prototype and production validation testing.
J.P. Aguire 11/1/95 E. Eglin 8/1/96
Response Plans and Tracking
Risk Priority Number - The combined weighting of Severity, Likelihood, and Detectability.RPN = Sev X Occ X Det
Likelihood - Write down the potential cause(s), and on a scale of 1-10, rate the Likelihood of each failure (10= most likely). See
Severity - On a scale of 1-10, rate the Severity of each failure (10= most severe). See Severity
Detectability - Examine the current design, then, on a scale of 1-10, rate the Detectability of each failure(10 = least detectable). See Detectability sheet.
Write down each failure mode and potential consequence(s) of that
FMEA Path Model Example
3535
FMEA Technique: Fault Tree Analysis• A Fault Tree is a logical diagram that starts with an actual or
potential failure and works backward to identify all of the possible causes or origins of that failure.
• Made up of branches connected by AND nodes and ORnodes.– ALL of the branches below an AND node must occur for the
event above the node to occur.– Only ONE of the branches below an OR node needs to occur for
the event above the node to occur
3636
Fault Tree Example
Identified “Fault”
Both Required
Any of These
Required
3737
FMEA Technique: Event Tree Analysis• An Event Tree is a logical diagram that starts with an actual or
potential event and works forward to identify all of the possible corrective actions -- and failures that could result.
• Essentially the reverse of a Fault Tree; in an analysis, one Event Tree may lead to multiple Fault Trees and vice-versa.
• Although initially developed by engineers to determine vulnerabilities in nuclear power generators; it is applicable, and has been applied, to assess many complex processes.
.302
.034
.084
.180
.400
3939
Summary – Strategic Risk AssessmentVarious strategic risk assessment methods view the landscape from different heights, so to speak -- altitude is a tradeoff between scope and detail.• The more abstract the method, the greater the scope but the
coarser the detail; the more concrete the method, the smaller the scope and the finer the detail.
• Different objectives, systems, threats, perils, hazards, controls, etc. dictate the use of different assessment tools and methods.
• Identifying the appropriate technique should be the first – and most important – step in risk assessment.
4040
And, Don’t Forget – the Real Objective is to Manage Risk
• The techniques examined in this discussion should only be used when you need to identify exposures, risks, perils and/or hazards that can be eliminated, mitigated or otherwise managed.
• NO measurement is necessary when you KNOW what to DO – and everyone AGREES!
QUESTIONS?
Thank you very much for listening!
Backup
4444
Categorizing Risk Assessment Techniques• Three basic types of assessment tools are:
1. Temporal methods;2. Comparative methods; and,3. Functional methods.
• Assessment techniques and tools can be classified on three axes:1. by their level of formality on a continuum from abstract to
concrete; 2. the type of analysis performed; and 3. the threats they are attempting to find and address.
4545
Types of Temporal Assessment Methods
• An Engagement consists of experts looking for any way, within given bounds, to compromise assets.
• An Exercise links experts and owners together in order to test the protection on assets particular to a particular system.
• Compliance Testing includes methods that the owner can execute them himself without the aid of an expert.
4646
Types of Comparative Assessment Methods• A Principles Method type, like all of the Comparative types, is a list.
This type asks the user to apply the principles to their system. • A Best Practices list consists of directives: Do this, Don’t do that. This
method type asks the user to compare what they do—their current practice—with the best practice list: the list of differences represents the “Gaps” between actual practices and ideal.
• An Audit is based on an explicit standard, such as a Best Practice list or a Principles list. This type asks the user to evaluate the effectiveness of the controls in place in fulfilling each item in the standard.
4747
Types of Functional Assessment Methods• Sequence Methods are the epitome of abstract methods. A simple
sequence method asks the questions:1. What can happen? (i.e., What can go wrong?)2. How likely is [it] that that will happen?3. If it does happen, what are the consequences?
• An Assistant Method type keeps track of details; best instances of this type “walk” the user through the process, prompting for the input needed to populate and rank lists of threats, vulnerabilities and remedial actions.
• A Matrix Method asks the user to select ranges for n dimensions – assets, threats, vulnerabilities and protections. The information in the cells of the corresponding n-dimensional subspace is the result of analysis.
• An Expert System is one implementation that is representative of the functional approach.
Recommended