Embed Size (px)
Citation preview
Modeling the Risk of Data Breach Incidents at the Firm Level
2020. 06. 25.IMIS2020
Kazuki ikegami, Hiroaki KikuchiGraduate School of Advanced Mathematical Sciences Meiji University
1
Backgroundn Increasing of cyber incident
• In 2018 alone, 443 data breaches in Japan, 5.61 million records of personal information
nThe Ministry of Economy, Trade and Industry (METI) published Cybersecurity Management Guidelines as a way to help address the cyber incident.• Risk identification and implementation of countermeasures• Necessity of measures to prepare for incidents
nOrganizations need to know proper risk
2
Goal of our study
n To reveal the risk of cyber incidents specific to a given organization and to quantify the effect of
security management in reducing this risk
3
Cost of security managements
Risk of Cyber incident
Previous study
4
In this study yamada [1] Edword [2]Purpose Risk assessment for
each organizationQuantification of management effect
Investigation of incident tendency
Technique Negative binomial distribution
Logistic regression Bayes generalized liner model
Data Data Breach incident in Japan from 2005 to 2018
Data Breach incident in Japan from 2005 to 2016
Data Breach incident in the US from 2005 to 2015
Target Single organization in Japan
All organizationsin 17 industries in Japan
All organizations in the US
[1] Yamada M, Ikegami K, Kikuchi H, Inui K (2018), Assessment of the effect of decreasing data breach by the management situation (2). Computer Security Symposium (CSS2018), 376--384[2] B.~Edwards, S.~Hofmeyr, and S.~Forrest, Hype and heavy tails: A closer look at data breaches, Journal of Cybersecurity, 2(1):3--14, 2016.
Research Question1. What is the probability that an incident will occur at an
organization in one year?• 10% or 20% ?
2. How long does it take before the next incident will occur at the organization?• 1 year or 3 year ?
3. How much is the inter-arrival time of incidents reduced by security management?• ISMS is good ?
5
Inter-arrival time
6
2017-09-01 2018-02-162018-01-09
2018-01-15
!1 !2 !3 !4
Inter-arrival time(day)
time
&'Inter-arrival time(day)
time
Two DatasetsnJNSA dataset(2005-2018)
• Data Breach dataset• The JNSA collects Data Breach incident information from Internet news
sites and major press releases officially published each year since 2005.
nCSR dataset• Management dataset• Toyo Keizai Inc. conduct a survey about corporate social responsibility
(CSR) for listed firms and major unlisted firms every year
8
period total Incident total organization2005-2018 16,392 9,358
period total question total organization2017 800 1,574
Research Question
1. What is the probability that an incident will occur at an organization in one year?
2. How long does it take before the next incident will occur at the organization?
3. How much is the inter-arrival time of incidents reduced by security management?
9
Method 1-1︓Fitting
nModeling the inter-arrival time by probability distribution(Normal, Poisson, Negative binomial)
nEstimate parameters for given inter-arrival time by the maximum likelihood estimation.
10
●
●
●
●
●
●
●
●
●
●●
0.00
0.25
0.50
0.75
1.00
0 1 2 3 4 5 6 7 8 9 10try_times
Pr
size●
●
●
●
1.4001.4251.4501.475
colour● 5
day
2018-01-262018-11-282018-05-16
2018-11-26
110day 194day 2dayInter-arrival time
example CDF
Method 1-2 ︓Kolmogorov-Smirnov Test
nPurpose : Test if a reference probability distribution is correctly modeled for a given sample
nH0 : The estimated distribution is identical to a given sample
11
The Estimated distribution function
The empirical distribution function
0 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)
Pr
Experiment
1. Get inter-arrival time 𝑑𝑖 for each firm from JNSA dataset• At least 4 incidents required• 3,789 incidents of 391 firms were extracted.
2. Estimate parameters of distribution by the maximum likelihood estimation.
3. Confirm the accuracy of the model by KS test
12
391 Results of fitting to the organization (partial)
13
オリエントコーポレーション
day
WOWOW
day
Pr
らでぃっしゅぼーや
day
Pr
GEコンシューマー・ファイナンス
day
Pr
テレビ朝日
day
Pr
愛知県
day
北海道電力
day
Pr
住友信託銀行
day
Pr
NTTドコモ
day
Pr
京都大学医学部付属病院
day
Pr
東京ガス
day
神奈川県
day
Pr
中部電力
day
Pr
東京電力
day
Pr
JR東日本
day
Pr
住友生命保険
day
JCB
day
Pr
高島屋
day
Pr
みずほ銀行
day
Pr
愛知社会保険事務局
day
Pr
北海道銀行 阪急交通社
Pr
北陸ガス
Pr
横浜市水道局
Pr
静岡市
Pr
名古屋市
day
横浜市保土ケ谷区
day
Pr
TBS
day
Pr
中国電力
day
Pr
沖縄電力
day
Pr
りそな銀行
day
住宅金融公庫
day
Pr
ブラザー販売
day
Pr
横浜市都筑区
day
Pr
東京都水道局
day
Pr
琉球銀行
day
福島県
day
Pr
九州電力
day
Pr
東邦銀行
day
Pr
八十二銀行
day
Pr
北越銀行
day
関西アーバン銀行
day
Pr
京都中央信用金庫
day
Pr
近畿大阪銀行
day
Pr
熊本ファミリー銀行
day
Pr
横浜銀行 大分銀行
Pr
豊和銀行
Pr
佐賀銀行
Pr
三重銀行
Pr
Tokyo Gas Corporation
Result of Fitting and KS test
nComparison of results fitted to three different probability distributions (Tokyo gas Co., Ltd)
nRate of organizations rejecting the null hypothesis at the 5% level
14
Negative Binomial Poisson Normal0.02(9/391) 0.39(155/391) 0.08(31/391)
P-value=0.09 P-value=0.00 P-value=0.00
Negative Binomial Poisson Normal
0 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)
Pr
0 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)
Pr
0 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)
PrFitting
KS test
Answer to Research Question 1Q) What is the probability that an incident will occur at an organization in one year?
A) Pr 𝐷 ≤ 365 = 0.92• Tokyo gas Co., Ltd
• Parameter 𝜇 = 111, 𝑟 = 0.92
nIncident probability statisticsafter one year
15
0.92
0 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)
Pr
Average Max Minimum deviation 0.11 1 0 0.27
365
Answer to Research Question 2Q) How long does it take before the next incident will occur at the organization?
A) 133 day• Tokyo gas Co., Ltd • Use Pr 𝐷 ≤ 365 = 0.7 as a threshold
160 100 200 300 400 500 600
0.0
0.2
0.4
0.6
0.8
1.0
d(day)Pr
0.7
133
How correct is our prediction ?
17
-𝒅𝒊" :不正解
2017-09-01 2018-01-09
time
実到着間隔𝑑#
-𝒅𝒊 : 正解
誤差
misscorrect
Real inter-arrival time Error
Prediction accuracynPrediction accuracy (n=391)
nHistogram of Error
18
Recall Average ofPredicted Inter-arrival time
0.55(214/391) 426
0
50
100
0 1000 2000 3000 4000error(day)
Frequency
Error(day)
Most of them are within 1 years. The fraction is 60%.
Research Question
1. What is the probability that an incident will occur at an organization in one year?
2. How long does it take before the next incident will occur at the organization?
3. How much is the inter-arrival time of incidents reduced by security management?
19
Method 2 ︓Generalized linear model
20
n Inter-arrival time 𝜇! when organization 𝑖 implements management 𝑚• 𝜇! = 𝑒"#$!%!#$"%"#$#%##⋯#$$!%%!%
• 𝑥!︓Industry，𝑥"︓Number of employees• 𝑥#︓1 if security management 𝑚 is deployed, or otherwise.
nEffect of management 𝑥!• Let 𝜇$%+ and 𝜇$& be the mean inter-arrival time with/without security management 𝑥$
• Ration of two of inter-arrival time = &!"
&!#
= '$"%&'&"%('("%)')"⋯"%!#&'!#&"%!'!'$"%&'&"%('("%)')"⋯"%!#&'!#&
= 𝑒(!
Result of Management effects
21
Management Estimate 𝜇!"/ 𝜇!#Pr(>|t|)ISMS 0.04 1.04 0.18CIO −0.07 0.93 0.01**CFO 0.01 1.01 0.64External Report Help Line 0.01 1.01 0.70Internal Report Help Line −0.07 0.93 0.14Whistleblower Rights Protection 0.06 1.06 0.24Establishment of Internal Control Committe −0.01 0.99 0.65Privacy Policy 0.00 1.00 0.98Security Policy −0.01 0.99 0.79Internal Auditing 0.01 1.01 0.75External Auditing −0.07 0.93 0.00**Independent Internal Audit Department 0.02 1.02 0.61Establish a Risk Management/Crisis Management System 0.03 1.03 0.42Basic Risk and Crisis Management Policy −0.08 0.92 0.03*Conduct Environmental Audits −0.03 0.97 0.33Establish Environment Management 0.10 1.10 0.01**Building an Occupational Health and Safety Management System 0.00 1.00 0.98
How much is the inter-arrival time of incidents reduced by security management?
ConclusionsnThe inter-arrival time in391 organizations was applied to three
different probability distributions, and it was shown that the negative binomial is the best.
n What is the probability that an incident will occur at an organization in one year?- For example, In Tokyo gas Co., Ltd, it’s 0.92.- An average of probability is 0.11.
n How long does it take before the next incident will occur at the organization?- For example, In Tokyo gas Co., Ltd, it’s 133 days.- An average of time is 426 days.
n How much is the inter-arrival time of incidents reduced by security management?- The effect of security management to inter-arrival time is that the inter-arrival time is 1.04
times longer when the ISMS is conducted.
22
