Network Dialog Minimization and Network Dialog

Diffing: Two Novel Primitives of Network Security

M. Zubair Rafiquezubair.rafique@cs.kuleuven.be

Juan Caballero (IMDEA Software Institute)

Christophe Huygens (iMinds-Distrinet, KU Leuven)

Wouter Joosen (iMinds-Distrinet, KU Leuven)

Network

Trace

Malicious SIP INIVTE Request

VoIP PhonesPCs

SIP Servers

Network

Switch

Gateway

Router

Internet

Server Crashed

Attack traffic?

Drive-by Download Milkers

Downloads a malware sample

Browser plugin

detected and vulnerabilities

exploited

Redirects to exploit kit

landing page

Navigate to given URL

HoneyClient

• Grier et al. “Manufacturing Compromise: The Emergence of Exploit-as-a-Service”,

CCS 2012

• Nappa et al. “Driving in the Cloud: An Analysis of Drive-by Download Operations

and Abuse Reporting”, DIMVA 2013

Downloads a malware

sample

Minimized Dialog,

IPs, Time

Milker

PCAP

PCAP

PCAP

PCAP

PCAP

Unlabeled Malware

SamplesMalware Network Dialogs Compare Dialogs

PCAP

PCAP

PCAP

PCAP

PCAP

Cluster 1

Cluster 2

Cluster 3

• Perdisci et al. “Behavioral Clustering of HTTP-Based Malware and

Signature Generation Using Malicious Network Traces”, Computer

Networks

• Rafique et al. “Firma: Malware clustering and network signature

generation with mixed network behaviors”, RAID 2013

Dialog Clustering

In a nutshell …

● Problem

- Network Dialog Minimization

- Network Dialog Diffing

● Applications

- Building drive-by download milkers

- Cookie expiration validation

- Simplifying user interfaces

- Vulnerability analysis

- Dialog clustering

● Outcomes

- Reduction in time and bandwidth

- Perfect precision and high recall

Outline

● Network Dialog Minimization

● Network Dialog Diffing

● Evaluation and Findings

- Milkers for 9 exploit kits (14000 malware samples)

- 17% top websites allow cookie replay >1 month

- Savings of time per year and employee

- New vulnerability in SIP server

- Clustering 6 malware families (F-Meausre = 87.6%)

● Limitations and Future Improvements

Network Dialog Minimization: “Given an original dialog that satisfies

a goal, can we produce a minimized dialog comprising the smallest

subset of the original dialog that when replayed still achieves the

same goal as the original dialog?”

Network Dialog Minimization

● Encode network dialog as dialog tree.

Dialog Generation

C2

C1

C3

M1

M2

M3

M4

Exploit

kitPre-filtering Filtered

Nodes C:M:F C:M:F IPs

Blackhole 1.x 73 6:6:60 5:5:50 2

CoolExploit 646 18:58:569 5:5:49 2

CritiXPack 192 4:19:168 2:7:62 2

Eleonore 936 12:76:848 8:66:736 2

Phoenix 132 12:12:107 7:7:73 1

ProPack 137 10:12:114 6:6:57 2

RedKit 154 8:17:128 2:6:57 1

Serenity 54 5:5:43 5:5:43 1

Unknown 79 5:7:66 5:7:66 2

Dialog Generation

Building Drive-by Download Milkers

Architecture

Network Delta Debugging

Test Dialog Replay

Remove

Dialog

Yes

No

Original Dialog

Minimized Dialog

Keep

DialogGoal

C2

C1

C3

M1

M2

M3

M4

C2 C3

M2 M4

Network Delta Debugging

Network Delta Debugging

● Generalized version of delta debugging

- Reset Button

- Goal beyond crashing the program

- Hierarchical structure of dialog tree

Zeller et al. “Simplifying and isolating failure-inducing input”, IEEE Transactions in

Software Engineering.

• NDM deals with remote networked applications.

- commercial Virtual Network (VPN) that offers exit

points in more than 50 countries (4500 IPs)

Incorrect

Minimization

L1 L2 L3 Tree IPs GDT Time

C:M:F C:M:F C:M:F Nodes used Pref. (sec.)

2:2:22 2:2:22* 2:2:6 11 33 157.0

1:1:7 1:1:7* 1:1:3 6 15 X 42.5

1:4:33 1:1:7 1:1:3 6 17 X 49.0

1:1:8 1:1:8* 1:1:4 7 27 X 215.8

1:1:7 1:1:7* 1:1:3 6 15 X 24.2

1:1:7 1:1:7* 1:1:3 6 15 X 37.3

2:6:57 2:2:19 2:2:10 15 71 250.4

2:2:15 2:2:15* 2:2:6 11 28 X 79.7

1:2:14 1:1:7 1:1:3 6 18 X 51.0

Exploit

kit

Blackhole 1.x

CoolExploit

CritiXPack

Eleonore

Phoenix

ProPack

RedKit

Serenity

Unknown

Network Delta Debugging

Building Drive-by Download Milkers

Network Dialog Diffing

Network Dialog Diffing: “Given two dialogs, identifying

how similar they are, how to align them, and how to

identify their common and different parts?”

Network Dialog Diffing

Rock.in

Rock.in

Dialog 1 Dialog 24 RRP 3 RRP

sim(D1, D2) = (1/N) * ∑ wi

sim(D1, D2) = (0.9+1+1+0)/4= 2.9/4 = 0.725

i=1

N

Dialog Similarity

Evaluation and Findings

34 times faster than honey

client.

14000 malware

downloaded from single

machine.

Drive-by Download

Milkers

Results Summary

Cookie Expiration

Validation

71 times reduction in

replay time. Savings of 20

hours of processing/day.

31% of websites allows

cookie replay (on

logout). 17% cookies

live over a month.

Simplifying User Interface Savings of 3 hours per

employee per year.

Command line tool to

perform building task.

Vulnerability Analysis Finding new vulnerability in OpenSBC Server

OSVDB 86607 (See details in the paper).

Dialog ClusteringBenign Dialogs (F-Measure = 100%), Malware

Dialogs (F-Measure = 87.6%)

Results Summary

34 times faster than honey

client.

14000 malware

downloaded from single

machine.

Drive-by Download

Milkers

Cookie Expiration

Validation

71 times reduction in

replay time. Savings of 20

hours of processing/day.

31% of websites allows

cookie replay (on

logout). 17% cookies

live over a month.

Simplifying User Interface Savings of 3 hours per

employee per year.

Command line tool to

perform building task.

Vulnerability Analysis Finding new vulnerability in OpenSBC Server

OSVDB 86607 (See details in the paper).

Dialog ClusteringBenign Dialogs (F-Measure = 100%), Malware

Dialogs (F-Measure = 87.6%)

OSVDB: 86607

34 times faster than honey

client.

14000 malware

downloaded from single

machine.

Drive-by Download

Milkers

Results Summary

Cookie Expiration

Validation

71 times reduction in

replay time. Savings of 20

hours of processing/day.

31% of websites allows

cookie replay (on

logout). 17% cookies

live over a month.

Simplifying User Interface Savings of 3 hours per

employee per year.

Command line tool to

perform building task.

Vulnerability Analysis Finding new vulnerability in OpenSBC Server

OSVDB 86607 (See details in the paper).

Dialog ClusteringBenign Dialogs (F-Measure = 100%), Malware

Dialogs (F-Measure = 87.6%)

Clustering Results

Dataset Algor. Clusters Precision Recall F-Measure

Alexa PAM 30 100% 100% 100%

Malware PAM 10 100% 64.8% 78.6%

Alexa Agg. 30 100% 100% 100%

Malware Agg. 12 100% 78.0% 87.6%

Limitations and Future Improvements

● Minimized dialog may look suspicious

● Dynamically generated requests

● Achieving global minimum

● Diffing of dialogs beyond HTTP

Conclusion

● Introduce the problem of network dialog minimization

and present novel network delta debugging technique.

● Propose a novel dialog diffing technique.

● Applied our techniques to 5 different applications.

- building drive-by download milkers

- cookie expiration validation

- simplifying user interfaces

- vulnerability analysis

- dialog clustering

Questions?