Cisco Cyber Threat Defense: Detecting and Protecting Against Insider Threat

Cisco Cyber Threat Defense Chad Mitchell – CCIE# 44090 Consulting Systems Engineer

[email protected]

Detecting and Protecting Against Insider Threat

Cisco Public 2 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Consider these guys…

All were smart. All had security. All were seriously compromised.


Increased Attack Surface

APTS Cyberware

Spyware and Rootkits Worms

Antivirus (Host-Based)

IDS/IPS (Network

Perimeter)

Reputation (Global) and Sandboxing

Intelligence and Analytics

(Cloud)

Enterprise Response

2010 2000 2005 Tomorrow

The Threat Landscape is evolving


DEFEND Assess Environment & Threat

Visibility & Investigation Contain

Fix

Advanced Content Analysis Behavior Anomaly Detection

Policy & Access Control Blocking Quarantine Re-Routing Traffic

Re-Think Security Process and Technology

Advanced Targeted Attacks Inside the Network


The Advanced Malware Attack Lifecycle PLAN EXPLOIT / ATTACK INFECT / SPREAD STEAL / DISRUPT

Attacker determines possible entry points,

formulates a plan of attack

Attacker exploits vulnerabilities and delivers

its weapon

Malware moves laterally through the internal network in search of

additional resources and data

Attacker takes action on its objectives and

exfiltrates data or disrupts systems

HACKER


NetFlow Generator

Source IP Address Destination IP Address Source Port Destination Port

Layer 3 Protocol TOS byte (DSCP) Input Interface

NetFlow Key Fields

Flow Information Packets Bytes/packet

Address, ports... 11000 1528

...

NetFlow Cache

StealthWatch FlowCollector

1

2

3

Source Des5na5on

Introduction to NetFlow


Internet

Internal Network

NetFlow Data

NetFlow Collector

Key NetFlow Fields

•  Packet count •  Byte count

•  Source IP address •  Destination IP address

•  Start sysUpTime •  End sysUpTime

•  Packet count •  Byte count

•  Input ifIndex •  Output ifIndex

•  Type of Service •  TCP flags •  Protocol

•  Next hop address •  Source AS number •  Dest. AS number •  Source prefix mask •  Dest. prefix mask

Usage

Time

Port Utilization

QoS

From/To

Application

Routing and Peering

The Network as a Scalable Source of Truth


Cisco Infrastructure Unsampled NetFlow is key to internal network visibility

Sampled = Partial •  Subset of traffic, usually less than 5%, •  Gives a snapshot view into network activity •  Similar to reading every 20th page of a book

Unsampled = All •  All traffic is collected •  Provides a comprehensive view into all activity

on the network •  Equivalent to reading every word on every

page of a book

Sampling is sufficient for network performance monitoring, not security


Versions of NetFlow Version Major Advantage Limits/Weaknesses

V5 Defines 18 exported fields Simple and compact format Most commonly used format

IPv4 only Fixed fields, fixed length fields only Single flow cache

V9 Template-based IPv6 flows transported in IPv4 packets MPLS and BGP nexthop supported Defines 104 fields, including L2 fields Reports flow direction

IPv6 flows transported in IPv4 packets Fixed length fields only Uses more memory Slower performance Single flow cache

Flexible NetFlow (FNF) Template-based flow format (built on V9 protocol) Supports flow monitors (discrete caches) Supports selectable key fields and IPv6 Supports NBAR data fields

Less common Requires more sophisticated platform to produce Requires more sophisticated system to consume

IP Flow Information Export (IPFIX) AKA NetFlow V10

Standardized – RFC 5101, 5102, 6313 Supports variable length fields, NBAR2 Can export flows via IPv4 and IPv6 packets

Even less common Only supported on a few Cisco platforms

NSEL (ASA only) Built on NetFlow v9 protocol State-based flow logging (context) Pre and Post NAT reporting

Missing many standard fields Limited support by collectors


NetFlow Version 5 (Common Record) Fixed format


How do I want to cache information

Which interface do I want to monitor?

What data do I want to meter? Router(config)# flow record my-record Router(config-flow-record)# match ipv4 destination address Router(config-flow-record)# match ipv4 source address Router(config-flow-record)# collect counter bytes

Where do I want my data sent? Router(config)# flow exporter my-exporter

Router(config-flow-exporter)# destination 1.1.1.1

Router(config)# flow monitor my-monitor

Router(config-flow-monitor)# exporter my-exporter

Router(config-flow-monitor)# record my-record

Router(config)# interface s3/0

Router(config-if)# ip flow monitor my-monitor input

1. Configure the Exporter

2. Configure the Flow Record

3. Configure the Flow Monitor

4. Apply to an Interface

Configuring Flexible NetFlow

Best Practice: include all v5 fields


NetFlow Challenges: Flow Stitching

12

10.2.2.2 port 1024

10.1.1.1 port 80

eth0

/1

eth0

/2

Start Time Interface Src IP Src Port Dest IP Dest Port Proto Pkts Sent Bytes Sent

10:20:12.221 eth0/1 10.2.2.2 1024 10.1.1.1 80 TCP 5 1025

10:20:12.871 eth0/2 10.1.1.1 80 10.2.2.2 1024 TCP 17 28712

Start Time Client IP Client Port

Server IP Server Port Proto Client Bytes

Client Pkts Server Bytes

Server Pkts Interfaces

10:20:12.221 10.2.2.2 1024 10.1.1.1 80 TCP 1025 5 28712 17 eth0/1 eth0/2

Uni-directional flow records

Bi-directional: •  Conversation flow record •  Allows easy visualization and analysis


NetFlow Challenges: De-duplication

13

Router A

Router B

Router C

10.2.2.2 port 1024

10.1.1.1 port 80

Router A: 10.2.2.2:1024 -> 10.1.1.1:80 Router B: 10.2.2.2:1024 -> 10.1.1.1:80 Router C: 10.1.1.1:80 -> 10.2.2.2:1024

Duplicates

•  Without de-duplication: •  Traffic volume can be misreported •  False positives would occur

•  Allows for the efficient storage of flow data •  Necessary for accurate host-level reporting •  Does not discard data •  Includes NAT


CTD Solution Components

Cisco ISE

Flow Sensor

Flow Collector

StealthWatch Management

Console

Catalyst 3500-X

10G-Servicemodule Catalyst 3850* Catalyst 4500

Supervisor 7E/L Catalyst 6500 Supervisor 2T

ASA-5500-X(NSEL) Nexus 1k*

Nexus 7k M Series* Nexus 7k F2 Series*

Nexus 6k* Nexus 2k on 7k*

NGA-3240 ASR 1000(NBAR)

ISR-G2(incl. NBAR) WLC (incl. NBAR)**

Catalyst 2960X LanBase* Net

flow

Ena

bled

Dev

ice

LanC

ope

Ste

alth

Wat

ch

Cis

co Id

entit

y S

ervi

ce E

ngin

e

Connection Information

Monitoring

Identity Information

Device User

Collection/Analysis

Presentation


Cisco Network

StealthWatch FlowCollector


Console

NetFlow

Users/Devices

Cisco ISE

NBAR NSEL

StealthWatch Solution Design

StealthWatch FlowSensor

StealthWatch FlowSensor

VE

NetFlow

StealthWatch FlowReplicator

Other tools/collectors


Devices Internal Network

Use NetFlow Data to Extend Visibility to the Access Layer

Unify Into a Single Pane of Glass for Detection, Investigation and

Reporting

Enrich Flow Data With Identity, Events and Application to Create Context

WHO

WHAT WHERE

WHEN

HOW Hardware-enabled NetFlow Switch

Cisco ISE

Cisco ISR G2 + NBAR

Cisco ASA + NSEL

Context

Visibility, Context, and Control


Conversational Flow Record - Visibility Who Who What

When

How

Where More context

•  Highly scalable (enterprise class) collection •  High compression => long term storage

•  Months of data retention


Situational Awareness

Shopping cart? “it's knowing what is going on around you …”

www.sans.edu/research/management-laboratory/article


Adding Context and Situation Awareness

NAT Events

Known Command & Control Servers

User Identity

Application Application & URL


LIMIT ACCESS Switch port control

INCREASE SCRUTINY Inform IPS, FW to selectively use stringent policy

ROUTE DIFFERENT PATH Route traffic through advanced security stack

ROUTE COPY OF TRAFFIC Selectively archive all packets of suspicious users / devices

Close the loop from detection to mitigation

•  Save money by leveraging network, itself, to enforce policy •  Reduce risk / latency by focusing resource intensive analysis •  Protection policy will follow the user and/or device •  Takes advantage of ISE, TrustSec, and pxGrid

DEFEND

Network-Integrated Mitigation - Control


Behavioral Analysis & Anomaly Detection

Behavioral Analysis •  Leverages knowledge of known bad

behaviour

Anomaly Detection •  Identify a change from

“normal”


DEFEND

Today – Advanced Visibility & Investigation

•  Partner with Lancope to deliver NetFlow visibility and security intelligence •  Enhance with Identity, device, application awareness

Cisco ISE

Cisco ISR G2 + NBAR

Firewall

IPS

Web Sec

N-AV

Email Sec

Threat Detection

Routers

Switches

Firewall

NetFlow

Visibility

Scalable Network Defense


DEFEND

Firewall

IPS

Web Sec

N-AV

Email Sec

Threat Detection

Routers

Switches

Firewall

NetFlow

Visibility

Advanced

Analysis

Next – Advanced Discovery

•  Next-gen analysis utilizing Artificial Intelligence, Game Theory and Predictive Algorithms

•  Leverage NetFlow, Local Web Data, DNS, Identity and SIO Context

•  Reduce human analysis •  Mitigation – TrustSec with ISE, Cisco ONE

SIO

Local Web Data, DNS,

Identity Global Reputation

URL, IP, File, Domain

Scalable Network Defense


Collect & Analyze Flows

1 2 •  # Concurrent flows • Packets per second • Bits per second • New flows created • Number of SYNs sent • Time of day

• Number of SYNs received

• Rate of connection resets

• Duration of the flow • Over 80+ other

attributes

Establish Baseline of Behaviors

Alarm on Anomalies & Changes in Behavior

threshold

threshold

threshold threshold

Critical Servers Exchange Server Web Servers Marketing

Anomaly detected in host behavior

3

Flow-based Anomaly Detection


Detailed Flow Information – StealthWatch 6.6


Attribute Flows and Behaviors to a User and Device

26

Policy Start Active Time

Alarm Source Source Host Groups

Source User Name

Device Type

Target

Desktops & Trusted Wireless

Jan 3, 2013 Suspect Data Loss

10.10.101.89 Desktops, San Jose

jchambers Windows7-Workstation

Multiple Hosts


A Note on StealthWatch and NSEL

•  Flow Action field can provide additional context

•  State-based NSEL reporting is taken into consideration in StealthWatch’s behavioral analysis •  Concern Index points accumulated for Flow Denied events

•  NAT stitching


Behavior-Based Attack Detection

High Concern Index indicates a significant number of suspicious events that deviate from

established baselines

Host Groups Host CI CI% Alarms Alerts

Desktops 10.10.101.118 865,645,669 8,656% High Concern Index

Ping, Ping_Scan, TCP_Scan


Identifying Reconnaissance Activity

NetFlow Capable

Internal Network

Devices

Management StealthWatch FlowCollector


Console

1. Infected host performs random pings and sweeps in the internal network

2. Infrastructure generates records of the activity using NetFlow

3. Collection and analysis of NetFlow data

4. Contextual information added to NetFlow analysis

5. Concern index increased Suspicious network scanning activity alarms generated

Cisco ISE

29


Detecting Internally Spreading Malware

NetFlow Capable

Devices



Console 3. Collection and analysis of NetFlow data


5. Concern index increased Worm propagation Alarm generated

Cisco ISE

Initial Infection

Secondary Infection

1. Infection propagates throughout the internal network as attacker executes their objective


Internal Network

30


Detecting Internally Spreading Malware

Devices



Console 3. Collection and analysis of NetFlow data


5. Concern index increased Worm propagation Alarm generated

Cisco ISE

Tertiary Infection

Initial Infection

Secondary Infection


Internal Network

NetFlow Capable

1. Infection propagates throughout the internal network as attacker executes their objective

31


Infection Tracking

Tertiary Infection

Secondary Infection

Initial Infection

32


Attack Detection without Signatures

High Concern Index indicates a significant number of suspicious events that deviate from

established baselines

Host Groups Host CI CI% Alarms Alerts

Desktops 10.10.101.118 338,137,280 8656% High Concern index Ping, Ping_Scan, TCP_Scan

Monitor and baseline activity for a host and within host groups.


Detecting Botnet Command and Control SLIC Feed

Alarm indicating communication with known

BotNet Controllers

IP Address Source user name

Policy that triggered alarm


Alarms Source Source Host Groups

Source User Name

Target Target Host Group

Inside Hosts

Jan 27, 2014

Host Lock Violation

10.35.88.171 Remote VPN Bob ZeusCCServer.com Zeus BotNet Controllers

34


Detecting Suspect Data Loss


Alarm Source Source Host

Group

Source Username

Target Details

Inside Hosts 8-Feb-2014 Suspect Data Loss

10.34.74.123 Wired Data

Bob Multiple Hosts

Observed 4.08G bytes. Policy Maximum allows up to 81.92M

bytes.

35


Detecting attacks like Heartbleed

•  Attack payload size and response is consistent

•  Application & •  Ports and Protocol are well

known •  Connections from attacker

are typically very long (hours-days)


What Heartbleed looks like in StealthWatch

Secure HTTP Server App = SSL Client Ratio ~ 5%

Duration is typically long – multiple hours


What can I do? What has been done?


§  Preparation §  Scanned 1.2M vulnerable servers – 300 in need of repair §  Developed signatures for Cisco IPS and Cisco NGIPS (SourceFire) §  Deployed Signatures to IPS/IDS

§  Monitoring and Response §  Discovered 25 Attacks: 21 Benign, 4 Malicious §  Researched attack via StealthWatch to discern normal connections vs

anomalous and malicious.

Cisco CSIRT response to Heartbleed


Combine Network and Security Management


Combine Network and Security Management

Top Conversations:


StealthWatch: Visibility into the Network


“1-touch” network mitigation action – from 3rd party partner console

pxGrid ANC API

ISE as unified policy point

User/Device Quarantine

Dynamic ACLs, Increase Inspection

Adaptive Network Control for ISE. It provides the ability to: •  Quarantine user devices from 3rd party products, such as StealthWatch & Threat Identification systems

•  Enlist other Cisco infrastructure in the network response – such as dynamic ACLs or SGTs on switches and ASA or increase IPS inspection levels

pxGrid: Adaptive Network Control Transforms the Cisco Infrastructure into a Unified Event Response Network


StealthWatch 6.6 – Quarantine w/ Cisco ISE ANC

•  Utilizes pxGrid to initiate action

•  Actions can be initiated Manually from the StealthWatch UI or Automatically with StealthWatch rules


Manipulating User Traffic using ANC & SGT

Classified Data Server

ISE

RADIUS (Access Request)

SGT = Classified_Operator

Security Group Filtering

Classified Operator

Allow access to Data Center

IP 10.45.1.70 = Classified_Operator


Manipulating User Traffic using ANC & SGT

Classified Data Server

ISE

RADIUS (Access Request)

SGT = Quarantine

Security Group Filtering

Quarantine

QoS = Bronze

Filter Traffic Block access to DC Allow remediation Full Packet Capture

Malware Traffic

Re-Route

Punt to IPS for further inspection


Walking the Talk - CSIRT NetFlow Collection at Cisco

250,000 Hosts 180 Flow Exporters

180,000 FPS

16 Billion Flows Daily

90+ Days Flow Retention

Cisco Public 48 © 2013-2014 Cisco and/or its affiliates. All rights reserved. 48

RTP San Jose Amsterdam

Bangalore

Sydney

Tokyo

15.6 billion flows / day 90 day retention

Walking the Talk - CSIRT NetFlow Collection at Cisco


Summary

Provides Rich Context Unites NetFlow data with identity and application ID to provide security context

Leverages Cisco Network for Security Telemetry

NetFlow-enabled Cisco switches and routers become security telemetry sources Cisco is the undisputed market leader in Hardware-enabled NetFlow devices

Cisco ISE

Cisco Network

Provides Threat Visibility and Context

Single pane of glass that unifies threat detection, visibility, forensics analysis, and reporting

Cisco ISR G2 + NBAR

+ +

+ NetFlow

FlowSensor

FlowCollector StealthWatch Management

Console

Cisco ASA Who What Where When How