Upload
open-analytics
View
1.267
Download
8
Embed Size (px)
Citation preview
MOLOCH:SEARCH FOR FULL PACKET
CAPTURE
It is a Great Horned Owl
Project Logo
WHY THE OWL?
Owls are silent hunters that go after RATs. We think that’s pretty
cool.
3
HISTORYLESSONWHY AOL BUILT MOLOCH
WHAT IS MOLOCH?
5
Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies.
• A simple web GUI is provided for browsing,
searching, viewing and exporting PCAP data.
• Web APIs are accessible if you wish to design your
own GUI or directly grab PCAP with various
command line tools for further analysis or
processing.
• Find it on AOL’s GitHub page:
https://github.com/aol/moloch
It’s like AOL Search for PCAP repositories!
WHAT IS MOLOCH NOT?
6
NOT IDS: NO ALERTS
NOT IPV6 (Today)
NOT SLOW
NOT CLOSED
NOT EXPENSIVE
WHY USE MOLOCH?
7
Real-time capture of network traffic for forensic and investigative purposes
• Combine the power of Moloch with other indicators (intelligence
feeds, alerting from IDS/anti-virus) to empower your analysts to
quickly and effectively review actions on the network to
determine the validity/threat.
• Review past network traffic for post compromise investigations.
Static PCAP repository
• Import large collections of PCAP that were created by malware.
• Import collections of PCAP from Capture The Flag events.
• Custom tagging of data at time of import.
THE PIECES OF MOLOCH
8
CAPTURE
• A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk.
DATABASE
• Elasticsearch is used for storing and searching through the SPI data generated by the capture component.
VIEWER
• A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
THE PIECES OF MOLOCH:CAPTURE
9
Libnids based daemon written in C
Can be used to sniff network interface for live capture
Can be called from CLI to do manual imports
Parses layers 3-7 to create SPI data
• Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
THE PIECES OF MOLOCH:DATABASE
11
Elasticsearch (http://www.elasticsearch.org)
• Powered by Apache Lucene (http://lucene.apache.org)
• Requests over HTTP(s)
• Results returned in JSON
Nosql
• Network traffic doesn’t fit the mold for relational DBs.
Documented oriented
• Great for lots and lots of network sessions.
Automatic sharding across multiple hosts
• At the time, we skipped SOLR because it couldn’t run distributed.
Fast, scalable, all that goodness
THE PIECES OF MOLOCH:VIEWER
12
Node.js based application
• Event driven server side JavaScript platform.
• Based on Chrome’s JavaScript runtime.
• Comes with its own HTTP server and easy JSON for communication.
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP.
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy.
• Easy automation to retrieve PCAP or sessions of interest.
THE PIECES OF MOLOCH:VIEWER
13
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
THE PIECES OF MOLOCH:VIEWER
14
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
ARCHITECTURE OF MOLOCH:DATA FLOW
15
ARCHITECTURE OF MOLOCH:MULTINODE WITH CLUSTER
16
ARCHITECTURE OF MOLOCH:SCALE
17
Packets Captured Kilobytes Saved Sessions Saved
Example: Moloch Capture
Documents Disk Storage (MB)
Example: Elasticsearch
MOLOCH: SPI-DATATYPESSESSION PROFILE INFORMATION
18
IP
• Source
• Destination
• Ports
• Protocol
HTTP
• Method
• Status Codes
• Headers
• Content Type
DNS
• IP Address
• Hostnames
MOLOCH: SPI-DATATYPESSESSION PROFILE INFORMATION
19
SSL/TLS
• Cert Elements:
• Common Name
• Serial Number
• Alt Names
SSH
• Client Name
• Public Key
• Port
IRC
• Channel Name
• Hostname
MOLOCH: CAPTURECREATING SPI DATA
20
MOLOCH: CAPTURECREATING SPI DATA
21
MOLOCH: CAPTURECREATING SPI DATA
22
MOLOCH: CAPTURECREATING SPI DATA
23
MOLOCH: CAPTURECREATING SPI DATA
24
MOLOCH: DEMO
25
MOLOCH: QUESTIONS?
26