Embed Size (px)
Citation preview
CONFIDENCE INTERVAL MEASUREMENTS
Daniel Busch
NSS Labs
BEGINNING IN CERTAINTIESWILL END IN DOUBTS
BEGINNING WITH DOUBTSWILL END IN CERTAINTIES
Sir Francis Bacon
Vulnerabilities Disclosed by Vendor 2012
Evasion - Malware Development Process
Quality Assurance
3
Evasion2
Development1
Deployment4
① Create malicious tool(write, buy, lease, or steal it)
② Obfuscate malware,create permutations
③ Test againstdetection engines
④ Deploy undetected samples only
1 x
100,000 x
20,000 x
Layered Security
server desktop laptop
Peri
mete
rH
ost
ba
sed
direct attack
Firewall
IPS
indirect attack indirect attack
Firewall
IPS
Anti Virus
BrowserURL Block
Anti Virus
BrowserURL Block
sidechannel
attack
on premise off premise
Ressources are limited
Security can be measured
Justus von Liebig - "Laws of Minimum"
• Securitylevel
Which Security Technologies are tested
• Next Generation Firewall und Firewall
• Web Application Firewall
• Antivirus und Breach Detection
• Intrusion Prevention Systems
• Mobile Device Management
• Sandbox / Isolations Technologies
NSS Labs Core Capabilities
• Static Analysis, Reverse Engineering and Decompilation• Strategic Advisory Services by gobally recognized Analysts • Product Testing
(Multiple 1Tb swim lanes / Cloud and virtualization test infrastructure)
• Threat Modeling(Data mining of test results to provide Analyst advice / run what if scenarios)
• Threat Forecasting
NSS Labs, Inc. is the world's leading information security research and advisorycompany, with unparalleled expertise in the complex aspects of informationsecurity across a wide range of technologies.
NSS Reports & Results
NSS Labs Data and Intelligence
CIO / CSO / CISO
IT Security
Risk Manager und Officer
Operations Manager und Officer
What We Provide to our Clients
Effectiveness of Layered SecurityOur tests demonstrate that defense layers act like well aligned layers of Swiss Cheese – with many holes aligned
IPS exploit block rate varies between 77% and 98%
Tuning of the IPS policy makes a difference, up to 50% less protection with default policy
A small set of exploits to bypass all tested IPS
Testing 15 IPS devices from 10 vendors against 1486 exploits
2012/Q4 IPS Group Test Results
What we provide to our clients:
BDS Throughput HTTP Email Exploits EvasionFalse Positive Rate
Breach Detection TCO / Detected Mbps
Vendor 1 1Gbps 100,00% 94,00% 90,00% 98,00% 7,00% 94,30% $ 360
Vendor 2 1Gbps 98,00% 98,00% 100,00% 100,00% 0,00% 98.4% $ 267
Vendor 3 667Mbps 95,00% 96,00% 93,00% 92,00% 0,00% 94,50% $ 374
Breach DetectionTests
Take away
• Trust, but verify.
Ronald Reagan
Referenzen
• Methodologien:
https://www.nsslabs.com/reports/categories/methodologies
