VIOLIN: A Network Virtualization Middleware for Virtual Networked Computing

Dongyan XuLab FRIENDS

(For Research In Emerging Network and Distributed Services)Department of Computer Sciences

Center for Education and Research in Information Assurance and Security (CERIAS)

Purdue University

The Team

Lab FRIENDS Xuxian Jiang

(Ph.D. student) Paul Ruth (Ph.D.

student) Dongyan Xu

(faculty)

RCAC, ITaP Dr. Sebastien

Goasguen

Outline

Motivations and goals Architecture of VIOLIN Applications of VIOLIN

Network system emulation Scientific computing Honeyfarm (network attack aggregation)

On-going work

Motivations

Formation of wide-area shared cyber-infrastructure Multiple domains Heterogeneous platforms Large number of users

Need for mutually isolated distributed environments Customized system administration and configuration Consistent and binary-compatible runtime support Un-trusted or malfunctioning applications

Known vulnerabilities in SETI@Home, KaZaa, and Condor Un-trusted network traffic control

Potential Applications

Multi-institutional collaboratories Large-scale distributed emulations

Cyber-systems Real-world systems

Parallel/distributed scientific applications Philanthropic (volunteer) computing

services Content distribution networks

VM (Virtual Machine): a Solution?

Achieves single node isolation (SODA*) Administration Resource Runtime services/libraries Fault/attack impact

However, does not achieve network isolation VMs addressable from/to any Internet hosts Cannot control traffic volume between VMs Cannot have overlapping address spaces* X. Jiang, D. Xu, “SODA: Service-on-Demand Architecture for Service

Hosting Utility Platforms”, IEEE HPDC-12, 2003.

VIOLIN: Proposed Solution

VIOLIN: A VN (Virtual Network) for VMs * Independent IP address space Invisible from Internet and vice versa Un-tamperable topology and traffic control Value-added network services (e.g., IP

multicast) Binary and IP compatible runtime

environment

* X. Jiang, D. Xu, “VIOLIN: Virtual Internetworking on OverLay INfrastructure”, Springer LNCS Vol. 3358 (ISPA 2004).

VIOLIN: the Big Picture

Internet

NMI

NMI

NMI N

MI

NMIN

MI

NMI

Physicalinfrastructure

NMI-based Gridinfrastructure

Two mutuallyIsolated VIOLINs VM

Key Ideas in VIOLIN

One level of indirection between VIOLIN and real Internet “All problems in Computer Science can be

solved by another level of indirection ” – Butler Lampson

A middleware-level underlay network serving as “intelligent carrier” of a VIOLIN Traffic tunneling Topology control Traffic volume control Traffic encryption Network service virtualization

VIOLIN Architecture

Host OS

Guest OS

App1

Guest OS

App2

VIOLIN daemon

…

ExistingNMI

Middleware

VMs

Physical host

VIOLIN Architecture

Host OS

Guest OS

App1

VIOLIN daemon

Virtual NIC

Host OS

Guest OS

App1

VIOLIN daemon

Virtual NIC

Message (e.g.,MPI)

TCP, UDP, …

IP

Ethernet frame via UDP tunneling

Between two VIOLIN nodes (VMs)

planetlab8.lcs.mit.edu planetlab6.cs.berkeley.edu

196.128.1.2 196.128.1.3

VIOLIN Network Performance

0

0.5

1

1.5

2

2.5

3

2 12 22 32 42 52

TCP Window Size (KBytes)

Th

rou

gh

pu

t (M

bp

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

0

0.5

1

1.5

2

2.5

3

2 12 22 32 42 52

TCP Window Size (KBytes)

Th

rou

gh

pu

t (M

bp

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

TCP throughput measurement on PlanetLabplanetlab8.lcs.mit.edu → planetlab6.cs.berkeley.edu

VIOLIN Network Performance

ICMP latency measurement on PlanetLabplanetlab8.lcs.mit.edu → planetlab6.cs.berkeley.edu

60

70

80

90

100

110

120

130

30 530 1030 1530 2030 2530

ICMP Data Length (bytes)

La

ten

cy

(m

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

60

70

80

90

100

110

120

130

30 530 1030 1530 2030 2530

ICMP Data Length (bytes)

La

ten

cy

(m

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

Application I: Network System Emulation

vBET: an education toolkit for network emulation * “Create your own IP network ” on a shared platform

IP address space and network topology Routers, switches, firewalls, end-hosts, links Real-world network software (OSPF, BGP…)

Strict confinement (network security experiments) Flexible configuration

Not constrained by device/port availability No manual cable re-wiring or hardware setup

* X. Jiang, D. Xu, “vBET: a VM-Based Emulation Testbed”, ACM SIGCOMM Workshop on Models, Methods, and Tools for Reproducible Network Research (ACM MoMeTools), 2003

vBET GUI

Sample Emulation: OSPF Routing

Emulation of OSPF RoutingDemo video clip:

Sample Emulation: Critical Server Protection

Screenshot: Distributed Firewall

Sample Emulation: Chord P2P Network

Screenshot

Sample Emulation: Internet Worms

Application II: Scientific Computing

Virtual clusters leveraging idle CPU cycles Long running parallel/distributed jobs Complicated communication patterns

between nodes (different from SETI@Home, Condor)

Runtime adaptation Resource re-allocation Migration/re-location Scale adjustment

Experiment Setup

Physical Cluster(ITaP)

Two mutuallyisolated virtual clusters

VM

Physical Switch

VS VS

VIOLIN vs. Physical Hosts

01234567

GFl

ops

1 2 4 8 16

# of Nodes

N = 8000, one VM per physical hostHPL sub-benchmark WROOL2L2

Physical hostsVIOLIN

Physical host: dual processor 1.2 GHz Athlon, 1GB memory VM: running one per host, 512MB memory

Multiple VIOLINs Sharing Physical Hosts

0

0.5

1

1.5

2

2.5

Average GFlops for

each VIOLIN

1 2 4 8

# of VIOLINs

N = 8000, sharing 4 physical hostsHPL sub-benchmark WROOL2L2

Doubling number of VMs on a host does not half the GFlops In this example, 8 VIOLINs exhaust memory

VM Communication Pattern

7MB/s

6MB/s

172

3640

5

6MB/s

4MB/s

6MB/s

6MB/s

5MB/s

5MB/s

7MB/s

5MB/s

3MB/s

Application III: Honeyfarm

Collapsar: a network attack aggregation center * Achieving two (seemingly) conflicting goals

Distributed honeypot presence Centralized honeypot operation

Key ideas Leveraging unused IP addresses in each network Diverting corresponding traffic to a “detention”

center (transparently), by VIOLIN Creating VM-based honeypots in the center

* X. Jiang, D. Xu, “Collapsar: a VM-Based Architecture for Network Attack Detention Center”, 13th USENIX Security Symposium (Security’04), 2004.

Collapsar Architecture

VM-based Honeypot

Collapsar Architecture

Redirector

Redirector Redirector

Correlation Engine

Management Station

Production Network

Production Network

Production Network

Collapsar Center

Attacker

Front-End

Real-Time Worm Alert

* X. Jiang, D. Xu, R. Eigenmann, “Protection Mechanisms for Application Service Hosting Platforms”, IEEE/ACM CCGrid’04, 2004.

Log Correlation: Stepping StoneLog Correlation: Stepping Stoneiii.jjj.kkk.11 compromised a honeypot & installed a rootkit, which contained

an ssh backdoor

iii.jjj.kkk.11 compromised a honeypot & installed a rootkit, which contained

an ssh backdoor

xx.yyy.zzz.3 connected to the ssh backdoor using the same passwd

xx.yyy.zzz.3 connected to the ssh backdoor using the same passwd

Log Correlation: Network ScanningLog Correlation: Network Scanning

On-going Work

VIOLIN-based virtual distributed environments (VP-Grids) on shared cyber-infrastructure Self-management (making them smart

entities) Missing role of VIOLIN administrator Automatic customization and bootstrapping Enforcement of application-specific policies

Self-orchestration (application-driven) Resource scaling Scale adaptation Topology evolution

Thank you.

For more information:

Email: dxu@cs.purdue.eduURL: http://www.cs.purdue.edu/~dxuGoogle: “Purdue SODA Friends”