Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 1
Improving Metrics in Cyber Resiliency
(With ETIF/ETIT)
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 2
© 2017 Cloud Security Alliance – All Rights Reserved All rights reserved. You may download, store, display on your computer, view, print, and link to the Improving Metrics in Cyber Resiliency (ETIF/ETIT) white paper at https://cloudsecurityalliance.org/download/improving-metrics-in-cyber-reciliency subject to the following: (a) the Report may be used solely for your personal, informational, non-commercial use; (b) the Report may not be modified or altered in any way;(c) the Report may not be redistributed; and (d) the trademark, copyright or other notices may not be removed. You may quote portions of the Report as permitted by the Fair Use provisions of the United States Copyright Act, provided that you attribute the portions to the Improving Metrics in Cyber Resiliency (ETIF/ETIT) white paper.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 3
Acknowledgements
Lead Authors
Dr. Senthil Arul
Dr. Shimon Modi
Contributors
Josep Bardallo
Ramon Codina
Bernd Jaeger
Courtney Keogh
Paul Lanois
Daniel Miller
Michael Roza
Duncan Sparrell
John Yeoh
Stephen Lumpe (design)
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 4
Table of Contents
Improving Cyber Resiliency by Defining, Measuring and Reducing Elapsed Time to Identify
Failure (ETIF) and Elapsed Time to Identify Threat (ETIT)
Abstract
Introduction
Modified Cyber Resiliency Model
Conclusions and future work
References
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 5
Improving Cyber Resiliency by Defining, Measuring and Reducing Elapsed Time to Identify Failure (ETIF) and Elapsed Time to Identify Threat (ETIT)
AbstractCyber resiliency is important as it gives us “the ability to prepare and plan for, absorb, recover
from, or more successfully adapt to actual or potential adverse effects.” Despite billions of
dollars being spent on cybersecurity, information systems data breaches are increasing year
after year. To reverse this trend, it is essential to develop metrics and processes to measure (1)
threats before they become cyberattacks, (2) recovery of lost functionality after a cyberattack.
This paper introduces two essential metrics: Elapsed Time to Identify Failure (ETIF) and Elapsed
Time to Identify Threat (ETIT). Measuring them and developing processes to lower the values of
ETIF and ETIT would improve the resiliency of an information system. The paper also discusses
challenges associated with measuring ETIF and ETIT and proposes that the measurement and
reporting of ETIF and ETIT could be transferred from companies whose systems encounter
cyberattacks to companies who are in the Intrusion Detection System (IDS) space. Transferring
the responsibility for measuring and detecting cyber intrusions to the IDS community would
bring superior algorithms that are needed to detect anomalies and improve cyber resiliency.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 6
IntroductionPreventing cyberattacks is an important objective. However, achieving such a lofty objective
is a challenge. Case in point, cyberattacks, data breaches, and spending on cybersecurity are
increasing year over year. According to ITRC survey study1, a 27.5% increase in cyberattacks
occurred in 2014 compared to 2013. These cyberattacks resulted in the exposure of over
80 million components of personally identifiable information (PII) including social security
numbers, driver’s license numbers, medical records, etc.1. In 2014, McAfee2 estimated that
the annual cost of cybercrime could be as high as $575 billion globally. Target Corporation’s
(NYSE:TGT) data breach in 2014 resulted in $1 billion in losses and the company incurred $88
million in breach related expenses3. According to Ponemon Institute4, the average cost of a
data breach for an American company is $5.4 million and the average per capita cost is $188.
In the USA, spending on cybersecurity reached more than $70 billion in 20145. The Yahoo data
breach, which is considered to be the largest discovered in history with over 500 million users
affected, has reduced the acquisition price by $350 million.6
The flexibility, scalability, availability, lower cost, etc., are driving the corporate information
infrastructure to migrate to the cloud from local servers and the operational assets information
such as financial, human resources, production, logistics, etc., are being stored remotely. The
cloud services could introduce new cyberattack vectors due to (1) lack of direct control of
cloud service providers on the shared infrastructure and dependency on third party developed
capabilities7, (2) ease in procuring and accessing cloud services allows nefarious users to hack
into data of other users in a multi-tenant cloud architecture8, and (3) Man In Cloud Attack –
involves the theft of user tokens which cloud platforms use to verify individual devices without
requiring logins during each update and sync9., etc. These new attack vectors would not only
compromise the information assets but also result in a detrimental impact to the operations
of the whole enterprise. Therefore, the discussions presented in this paper on improving
information system resiliency could be extended to the operational systems’ resiliency as well.
The cyber ecosystem is non-discriminant in the sense that good and harmful information
coexists harmoniously. The internet and transmission protocols by which the information
travels within the cyber ecosystem is also non-discriminant in the sense that good and
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 7
harmful information is transmitted across
cyberspace without discrimination as to
priority or hierarchy. Thus, the opportunity
to discriminate between good and harmful
information would not occur in the cyber
ecosystem. Discrimination must occur within
one’s own Information Technology/Operational
Technology (IT/OT) infrastructure, and must
occur in the security infrastructure layers.
As shown in Figure 1, the first opportunity
to discriminate between good and harmful
information occurs at the security layers
between the cyber ecosystem and IT/OT
infrastructure. The second opportunity occurs
at the security infrastructure layers part of the
information system.
If a cyberattack is identified and captured at
the security layers in the IT/OT infrastructure,
then the information system is protected.
However, if a cyberattack is identified by the security layers residing in the information system
(i.e., harmful information already past the IT/OT infrastructure), then the information system
is compromised. The loss of function resulting from a cyberattack depends on the kind of
function that the attack targets (logistics, tactics, etc.) and on the type of information that is
available and exposed (trade secrets, personally identifiable information, etc.). For example, a
2014 cyberattack on Sony Pictures (NYSE:SNE)6 disabled computers, leaked upcoming movies,
and completely paralyzed the operation for a short period of time whereas a cyberattack7 on
the Pentagon’s Joint Forces e-mail system resulted in the shutdown of the e-mail system for a
short period of time but only locally, without affecting rest of the Pentagon’s e-mail systems.
IT/OT Infrastructure
Cyber Ecosystem
Information System
Application 1
Layered security such as Intrusion Detection, Authentication, and Malware Detection.
Information System
Application 2
Information System
Application 3
Layered Security Layered Security Layered Security
Figure 1Depiction of various layered security in
the IT/OT infrastructure
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 8
The National Academy of Sciences8 (NAS) defines resiliency as “the ability to prepare and
plan for, absorb, recover from, or more successfully adapt to actual or potential adverse
effects”. Bruneau et al.9 developed a framework to quantitatively assess the resiliency of a
community after an earthquake by measuring Quality of Infrastructure, Q(t), over a period of
time. The value of Q(t) ranges from 0% to 100% where 0% means no service available and
100% means no degradation
in service. At time t0, an
earthquake event occurred
and dropped Q(t) from 100%
to 50% instantaneously. It took
time t1 to fully recover, Figure
2. This approach is based
on the notion that Quality
of Infrastructure, Q(t), of a
community is affected after
an earthquake and it takes
a certain amount of time to
fully recover from it. Zobel et
al.10 applied the resiliency of
community after an earthquake framework to quantify cyber resiliency after a cyberattack.
They analyzed many types of cyberattacks: (1) slow-onset single event, (2) sudden-onset single
event (3) slow-onset multi event, (4) sudden-onset multi event. An information system performs
many functions; they were all combined into a single function defined as Quality of Service, Q(t). The
functional value for Q(t) ranged between 0 and 1; value of zero (0) represents unable to perform
the intended function and value of one (1) represents fully performing the intended functions.
This paper expands the cyber resiliency model presented by Zobel et al. 10. It introduces two
new variables, namely Elapsed Time to Identify Failure (ETIF) and Elapsed Time to Identify
Threat (ETIT) and qualitatively presents a compelling case that the measurement and
publication of ETIF and ETIT would spur innovation in the IDS space, and aid in the overall
improvement of the resiliency of information systems.
100
t0 t1
0.0
Qua
lity
of S
ervi
ce, Q
(t)
Time, t
Figure 2Resiliency of community after an earthquake (adopted from Bruneau et al.)
Q’(t)
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 9
Modified Cyber Resiliency ModelFigure 3a graphically represents the Zobel et al.10 characterization of a resiliency profile
for a slow-onset single-event cyberattack. In this case, the loss of quality of service occurs
gradually over time, a virus gradually propagating across the system unnoticed. The start of the
cyberattack in the timescale is represented as t0start, the end of the cyberattack is represented
as t0end, and the corresponding of loss of functionality is Q’(t). The time to recover the full
functionality from t0start is T. The triangular area with the base as T and the height as Q’(t)
represent the loss of resiliency. The smaller the area under the resiliency curve the system
is more resilient. That is, the cyberattack did not impact the functional performance, or the
system recovered quickly and/or a combination of both.
Figure 3b represents the modified cyber resiliency model. In a typical cyberattack, the failure
is identified by studying the anomaly/degradation of intended functions at the IDS layer.
Therefore, the modified resiliency model starts with the time at which the failure was identified
ti (in Zobel et al. model is t0end) and the corresponding loss of function is Q’(t). Immediately
after the identification of a failure, an investigation would start to identify the root causes of
failure and to implement recovery actions to restore full functionality. The investigation would
1.0
T
0.0t0
start ts ti tet0end
1.0
0.0
Qua
lity
of S
ervi
ce, Q
(t)
Loss of Function
Recovery of Function
Qua
lity
of S
ervi
ce, Q
(t)
Loss of Reliliency
Time, t Time, t
Figure 3a represents the original cyber resiliency model for slow-onset single-event (adopted from Zobel
et al. slow-onset multi-event resiliency profile) and Figure 3b represents modified cyber resiliency model.
Please note - ETIF and Recovery time may vary
Figure 3a Figure 3b
ETIF
T
Q’(t)Q’(t)
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 10
identify the time at which the cyberattack started, ts (in Zobel et al.’s model is t0start). The line
connecting ti to ts represent the loss of function. The duration between the start of the failure
and the identification of the failure is defined as the Elapsed Time to Identify Failure (ETIF). The
recovery actions would fix the failure, necessary to achieve full functionality. The time at which
full functionality is achieved is represented as te. The line connecting ti and te represents the
recovery function. Therefore, the modified resiliency model includes ETIF and the slopes of loss
and recovery of functions, Figure 3b.
Cyberattacks occur when the information system vulnerabilities are exposed and exploited.
For a given information system, these vulnerabilities are embedded into the architecture of the
software and the hardware and characterize the attack surface15.
As noted earlier, resilience of a system is characterized by its ability to achieve its intended
functions despite disruptions. From that perspective, the bounding function on how a system
is attacked and recovers is dependent on the attack surface. Thus, the functions representing
resiliency, in this case ETIF, are bounded by the attack surface. These functions can be linear or
non-linear; for illustrative purposes, the graphical information presented in this paper show linear
relationships. However, discussions are certainly valid for both linear and non-linear functions.
100
D
A E C G
B
F
0.0
Q”(t)ETIF”
ETIF’
ETIF’”
Q’(t)
Q*(t)
ts ti” te” te’ te’”ti’
Qua
lity
of S
ervi
ce, Q
(t)
Time, t
Figure 4Illustrating the role of ETIF in cyber resiliency
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 11
ETIF measures the loss of resiliency of an information system. If ETIF moves from ETIF’ to ETIF’’,
Figure 4, then the loss of resiliency area would reduce from triangle ∆ABC to triangle ∆ADE,
the loss of quality of service, Q(t), would reduce from Q’(t) to Q’’(t), and the recovery time (te)
would reduce from te’ to te’’. Extending this argument, the loss of resiliency would reduce to
zero when ETIF approaches zero. That is, the start of the cyberattack (t0) and the identification
of the cyberattack (ti) occurred at the same time (i.e., the cyberattack was nullified at the IT/OT
infrastructure.)
If ETIF moves from ETIF’ to ETIF’’’, Figure 4, then the loss of resiliency area would increase from
∆ABC to triangle ∆AFG, the loss of quality of service, Q(t), would increase from Q’(t) to Q’’’(t), and
the recovery time (te) would increase from te’ to te’’’. However, if ETIF continue to increase, at
some point the loss of resiliency reaches a level beyond which the system is not recoverable.
This condition would be reached either when the quality of service, (Q(t), reached a point of
no recovery, Q*(t), or when the cost of recovery is more than that to build a new information
system, Figure 4. An example of this condition would be a security breach at a power
generation company causing an electric generator to have a catastrophic failure resulting in a
total loss of functionality. In this case recovery may not be possible.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 12
No
1
2
3
May-14
Dec-14
as early as Nov 2013
29-Jan-15
29-Jan-15
24-Nov-14
> 7 months
> 1 month
1 year
Premera Blue Cross11
Anthem Blue Cross12
Sony Pictures6
Company/Organization
Cyberattack Identified Date
Elapsed Time to Identify Failure (ETIF)Cyberattack Date
4
5
6
as early as July 20, 2014
as early as April 2014
as early as mid-June 2014
20-Oct-14
2-Sep-14
Mid-August 2014
> 3 months
> 4 month
> 2 months
Staples13
Home Depot14
JP Morgan15
7 Apr-14 1-Jul-14 3 monthsCommunity Health16
8
9
10
28-Jan-15
17-Feb-15
May-14
30-Jan-15
17-Feb-15
1-May-15
3 days
3 hours
1 Year
Toys R Us17
University of Maryland18
Office of Personnel Management (OPM)19
Table 1Calculated ETIF based on recent cyberattacks
DiscussionTable 1 shows some of the recent cyberattacks and the ETIF for each of the attacks. The ETIF
ranged from as low as three (3) hours (University of Maryland) to as long as one (1) year (Sony
Pictures & the US Government’s Office of Personnel Management). The actual cyberattack date
and time, as mentioned before, is determined by forensic analysis. The forensic analysis process
is not standard across all industries and as such the data in the public domain is not easily
comparable. As of today, there is no agreed definition of what activities would characterize the
start of a cyberattack. For example, in some cases, certain viruses are dormant in the system
before becoming malignant and in other cases there are multiple attacks over a period of time.
Corporations are concerned about confidentiality, competitive pressures, litigation, image, etc.,
and are therefore reluctant to disclose the specific details of a cyberattack including the date of
the attack and the date of the identification. Given the lack of standardization in publicly available
data on start of a cyberattack, ts, and on the identification of a cyberattack ti, it is currently difficult
to consistently and specifically calculate ETIF and hence cyber resiliency.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 13
If skilled companies in the IDS calculate and report ETIF instead of the corporations
experiencing the cyberattack , it would enable standardization of the forensic process and
development of the tools necessary to clearly define, measure and calculate the start of a
cyberattack. By reporting ETIF instead of the start of cyberattack, ts, and the identification
of cyberattack, ti, corporations may feel less exposed to litigation and negative publicity.
Publishing ETIF and reducing the value of ETIF could spur competition in IDS space to
develop advanced algorithms to identify anomalies in the quality of service data. However,
for ETIF to be meaningful and to be an effective metric for improving cyber resiliency, clear
definitions for ti and te, and agreed upon processes and tools need to be established to
measure ti and te.
The discussion until now has been focused on the loss of resiliency of an information system.
However, an ideal state would be to have no loss of resiliency. That is, the threat of a failure is
identified early enough before it becomes an attack. Such an outcome is possible if there was
a mechanism to disseminate information about the adverse events and the threats among
various entities on a real-time basis. Ideally, if an entity that experienced failure could share the
system’s vulnerability with other entities, analysis of such shared failure event would be critical
in detecting the presence of a threat or recover quickly from the failure. We define this metric
as Elapsed Time to Identify Threat (ETIT). The effectiveness of such a metric can be measured
by its ability to move ti (time to identify failure) closer to ts (start time of the failure). ETIT is
critical in changing limits on the loss and recovery functions and thus impacting the quality of
service. If there is an ability for early identification of the threat that is causing the failure, then
the overall time to recovery and hence the loss of resiliency could be reduced.
It is conceivable that different information systems could have the same loss of resiliency
(i.e., same area of the triangle) but with different slopes for loss and recovery functions. This
is illustrated in Figure 5 where System 1 and System 2 have the same loss of resiliency but
System 1 has much steeper loss and recovery functions and it fully recovers to perform
its full intended function much sooner than System 2. From a user/customer perspective,
System 1 would be preferred in the sense that it recovered to perform its fully intended
function in a much shorter duration than System 2. Therefore, in addition to the loss of
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 14
resiliency, the slope of the loss and recovery functions would become important. The steeper
the slope for the loss and recovery functions and shorter the duration between ts and ti (i.e.,
ETIF), the smaller the loss of resiliency.
100
0.0
Qua
lity
of S
ervi
ce, Q
(t)
Time, t
System 1
System 2
Figure 5Comparing two different information systems with the same loss of resiliency with
different Elapsed Time to Identify Failure (ETIF), loss and recovery functions
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 15
Conclusions and Future WorkA modified information system resiliency model was presented and introduced two new
variables: Elapsed Time to Identify Failure (ETIF) and Elapsed Time to Identify Threat (ETIT). The
model qualitatively demonstrated that the lower the ETIF, the smaller the loss of resiliency for
an information system. Challenges in calculating ETIF in recent cyber attacks showed difficulty
in measuring the start of the cyberattack (ts) and the identification of cyberattack (ti).
By transferring the responsibility of calculation and publication of ETIF from the companies that
experience cyberattacks to the companies that are in IDS space could bring standardization
and continuous improvement in measuring and reporting ETIF. In addition, it could encourage
competition in the IDS space and would bring innovative and superior algorithms to find
anomalies in the data quality thus improving cyber resiliency.
With the growth in cloud computing, information about the operational assets is being stored
away from the local servers. Although the benefits of cloud computing are obvious, such
decoupling could result in poor operational resiliency if the information asset is compromised.
Therefore, to keep the operational resiliency unaffected, it is essential to bolster information
asset resiliency in the cloud. Hence, a technical framework, along with appropriate regulatory
framework, needs to be created to enable the measurement and reporting of ETIF and ETIT.
The Cybersecurity Act of 201516 passed by the US Congress, aims at setting a regulatory
framework for inter-government and private company to government exchanges. NIST has
released a special publication outlining the building blocks of an incident exchange program.17
Future work would entail architecting a framework that allows a set of cooperative systems to
aggregate, summarize, and utilize ETIF and ETIT to improve resiliency.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 16
References
1. http://www.idtheftcenter.org/ITRC-Surveys-Studies/2014databreaches.html, 2014
2. “Net losses: Estimating the global cost of cybercrime Economic impact of cybercrime II”,
Center for strategic and international studies, June 2014
3. Perlroth, N., Harris, E. A., “Cyberattack insurance a challenge for business”, http://www.
nytimes.com/2014/06/09/business/cyberattack-insurance-a-challenge-for-business.html?_
r=0 , June 2014
4. “2013 cost of data breach study: global analysis”, Research sponsored by Symantec and
independently conducted by Ponemon Institute LLC, May 2013
5. http://www.gartner.com/newsroom/id/2828722, August, 2014
6. http://money.cnn.com/2017/02/21/technology/yahoo-verizon-deal/
7. https://blogs.microsoft.com/microsoftsecure/2013/01/17/cloud-services-building-
resiliency-and-business-continuity/
8. http://www.levelcloud.net/why-levelcloud/cloud-education-center/advantages-and-
disadvantages-of-cloud-computing/
9. https://www.incapsula.com/blog/top-10-cloud-security-concerns.html
10. https://en.wikipedia.org/wiki/Sony_Pictures_Entertainment_hack
11. http://www.cnn.com/2015/08/05/politics/joint-staff-email-hack-vulnerability/
12. “Disaster resiliency: A national imperative”, The National Academies Press, 2012
13. Bruneau M., et al., “A framework to quantitatively assess and enhance the seismic resiliency
of communities”, Earthquake Spectra, 19(4), 733-752, 2003
14. Zobel C.W., et al. “Quantifying cyberinfrastructure resilience against multi-event attacks”,
Decision Sciences, Volume 43, No. 4, 687-709, August, 2012
15. Mandhata, P., Wing, J., “Measuring a System’s Attack Surface”, CMU-DS-04-102, January
2014
16. https://www.congress.gov/bill/114th-congress/senate-bill/754
17. Johnson, C, et al., “Guide to cyber threat information sharing”, NIST special publication (800-
150), April, 2016.
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 17
18. http://www.npr.org/sections/alltechconsidered/2015/03/18/393868160/premera-blue-
cross-cyberattack-exposed-millions-of-customer-records
19. https://www.anthemfacts.com/
20. http://www.technobuffalo.com/2014/12/22/staples-1-16-million-credit-cards-affected-in-
cyber-attacks/
21. http://www.huffingtonpost.com/2014/09/18/home-depot-hack_n_5845378.html
22. http://www.wsj.com/articles/j-p-morgan-says-about-76-million-households-affected-by-
cyber-breach-1412283372
23. http://www.forbes.com/sites/danmunro/2014/08/18/cyber-attack-nets-4-5-million-records-
from-large-hospital-system/
24. http://www.scmagazine.com/attacks-attempt-to-access-rewardsrus-accounts/
article/401160/
25. http://www.diamondbackonline.com/news/umd-social-security-numbers-compromised/
article_b8236dea-99b6-11e3-92eb-0017a43b2370.html
26. http://www.politico.com/story/2015/07/federal-government-cyber-attack-breach-21-million-
people-affect-119918