Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
OSINTNot Just For Attackers
© 2019 David Mashburn | All Rights Reserved
Applied Open Source Intelligence
© 2019 David Mashburn Defensive OSINT
David Mashburn, @d_mashburn
• SANS Certified Instructor
• GSE #157• Security mule• Family guy
2
© 2019 David Mashburn Defensive OSINT
What is OSINT?
Open-Source Intelligence (OSINT)• Harvesting and analysis of data from publicly available
resources
OSINT typically performed without directly interacting with the target
Going beyond the search• Key is in the analysis to transform from data to information to
intelligence
3
© 2019 David Mashburn Defensive OSINT
OSINT Resources
Numerous OSINT resources on the web
Web-based resources are often free,
but may be rate limited
Many offer APIs, which facilitate scripting & automation
May need to leverage a paid service to unlock features or to access more detailed information
4
© 2019 David Mashburn Defensive OSINT
OSINT as attack reconnaissance
Penetration testing frameworks have dedicated phases for reconnaissance activities
• PTES (Pentest Execution Standard)1
Attack models such the Lockheed-Martin Cyber Kill Chain2
include recon phase
Image source: https://www.oreilly.com/library/view/practical-cyber-intelligence/9781788625562/37a5852b-ef31-4b1e-a184-93ea7cf5cd75.xhtml
5
© 2019 David Mashburn Defensive OSINT
OSINT as a defensive resource
Blue needs to know what Red sees
The same OSINT tools used by pen testers, network admins, and security researchers can be leveraged by defenders
Let’s explore the application of OSINT by looking at a few defensive scenarios
6
© 2019 David Mashburn Defensive OSINT
Defensive OSINT use cases
Compromised Credentials
• Identities involved in some sort of data breach
Data Leakage
• Sensitive company information posted to the web
Encryption Certificates
• Track certificates without having network visibility
Social Media
• Basis for social engineering attacks, map relationships
7
© 2019 David Mashburn Defensive OSINT
Compromised credentials
Password re-use is an unfortunate reality
Our user communities may will reuse passwords across different sites
May lead to compromise even though the source was a third-party
8
© 2019 David Mashburn Defensive OSINT
Keeping Things Simple
9
A simple web lookup is a good start, but will not scale effectively
https://haveibeenpwned.com/
© 2019 David Mashburn Defensive OSINT
HIBP API access via curl
10
$ curl --insecure
https://haveibeenpwned.com/api/v2/breachedaccount/[email protected] | json_pp
[{
"Description" : "In August 2017, a spambot by the name of <a
href=\"https://benkowlab.blogspot.com.au/2017/08/from-onliner-spambot-to-
millions-of.html\" target=\"_blank\" rel=\"noopener\">Onliner Spambot was ...
<a href=\"https://www.troyhunt.com/inside-the-massive-711-million-record-
onliner-spambot-dump\" target=\"_blank\" rel=\"noopener\">Inside the Massive
711 Million Record Onliner Spambot Dump</a>.",
"Domain" : "",
"Name" : "OnlinerSpambot", ...
},{
"Domain" : "data4marketers.com",
"Description" : "In early 2015, a spam list known as <a
href=\"http://www.data4marketers.com/2015APRspecials.html\" target=\"_blank\"
rel=\"noopener\">SC Daily Phone</a> emerged containing almost 33M identities.
The data includes personal attributes such as names, ...
© 2019 David Mashburn Defensive OSINT
An Open Source OSINT Framework
12
Recon-ng by Tim Tomes
Python-based framework
Performs data transformation, using input data ‘seeds’ to harvest new information
http://recon-ng.com
© 2019 David Mashburn Defensive OSINT
Automating the Search
13
Framework lookup for breached credentials (slow, rate limited)• recon/contacts-credentials/hibp_breach
© 2019 David Mashburn Defensive OSINT
Operational Efficiencies
14
How can we further automate OSINT?• What if we could script the framework?
The recon-ng framework has two scripting options• Provide a resource file via the –r option• Can use an interactive session to create the resource file using the record command
• Use the scripting interface recon-cli.py• Supports shell scripting
Merges automation with structured data collection and storage
© 2019 David Mashburn Defensive OSINT
Leveraging the Findings
15
HIBP service allows you to do regular monitoring of something that you can’t control
Automation and different output formats make it possible to ingest this data into your SIEM
HIBP will not provide the actual passwords
© 2019 David Mashburn Defensive OSINT
A slightly grayer path
16
Choose your own adventure, but you can find resources beyond the clearly white hat arena
Paid service with tiers, API access is only via paid service
© 2019 David Mashburn Defensive OSINT
Staying on top of things
HIBP is focused on credentials
Checks for potential credential dumps by consuming the @dumpmon Twitter feed
Open source project
https://github.com/jordan-wright/dumpmon
What exactly is being monitored?
17
© 2019 David Mashburn Defensive OSINT
Beyond credential dumps
Data loss is more significant than password loss
Challenging to track and control data in authorized systems
What about our data where it doesn’t belong?
18
© 2019 David Mashburn Defensive OSINT
Pastebin
Pastebin is one of the better-known paste sites
Provides API access and email alerts to monitor for keywords
Pro version (paid)
19
© 2019 David Mashburn Defensive OSINT
Monitoring across multiple paste sites
Michael Bazzell runs a fantastic website for OSINT
IntelTechniques.com
One of the tools is a search across 57 paste sites
Leverages Google Custom Search
20
© 2019 David Mashburn Defensive OSINT
Google Custom Search Engines
Create your own CSE at https://cse.google.com/cse/
OpSec may be a significant consideration
How sensitive are the keywords you are monitoring?
21
© 2019 David Mashburn Defensive OSINT
Google Alerts
Create alerts for specific searches
Alerts delivered via email
How much do you want to entrust to any third party?
22
© 2019 David Mashburn Defensive OSINT
Certificate Tracking
Encrypted traffic is the norm
Certificate issuance is free and requires only proof of domain control
Can you identify certificates in use that aren’t hitting your sensors that use your domains?
23
© 2019 David Mashburn Defensive OSINT
Certificate Transparency
“… provides a way for every certificate issued by any publicly trusted CA to be publicly logged, monitored, and audited”
This means that we can monitor in near real-time certificates that have been issued for your domains
24
© 2019 David Mashburn Defensive OSINT
Certificate Transparency Sources
Many options for online searching of CT logs
• Digicert SSL tools, Entrust Datacard, Cas generally
Can also download the CT logs directly and ingest
25
© 2019 David Mashburn Defensive OSINT
Other Certificate Monitoring options
Censys.io and crt.sh are 0ther options for tracking certificates
Web interface for simple searching
Censys.io API for automation
26
© 2019 David Mashburn Defensive OSINT
Social media mining
Oversharing on social media has significant personal implications
Social media goes far beyond the personal realm
Who is claiming to be part of our organization?
29
© 2019 David Mashburn Defensive OSINT
Understanding the Social Media surface area
Who looks interesting in this list?
Social media helps make the connections with suggestions for who might be of interest to you
Profile information allows relationships and lines of reporting to be determined
Who needs that extra awareness training?
30
© 2019 David Mashburn Defensive OSINT
Nice to meet you
So about what do we know about Roma Blaser?
Reverse image search may be a good starting point*
31
© 2019 David Mashburn Defensive OSINT
Conclusions
OSINT has wide applicability for defenders
Helps find things where they don’t belong
Provides us with the same knowledge as an external party
Can help deal with some issues that are difficult for our typical logging and visibility setups
32
© 2019 David Mashburn Defensive OSINT
Domain Shadowing
Domain registrar accounts compromised
Attacker creates subdomains to support exploit kits
Excellent blog from Cisco Talos on domain shadowing
34
© 2019 David Mashburn Defensive OSINT
OSINT unshadows?
GoDaddy specifically named as a targeted environment
Blog post focused on detecting at scale
What about focusing on your domains?
35
© 2019 David Mashburn Defensive OSINT
GoDaddy DNS behavior
Automatically updates the zone serial number based on the date of the last change to the name server
Simple detect for zone changes
36