M. S. Thesis Defense, 09/24/20011 Migratory TCP (MTCP) Transport Layer Support for Highly- Available Network Services Kiran Srinivasan DisCoLab Division

M. S. Thesis Defense, 09/24/2001 1

Migratory TCP (MTCP)Transport Layer Support for Highly-

Available Network Services

Kiran SrinivasanDisCoLab

Division of Computer and Information SciencesRutgers University

Advisor: Liviu Iftode


Services Rather Than Servers

Client oblivious to a given server’s location Client is interested only in the content

provided and QoS A Service -> set of equivalent servers Internet services

Most services are based on TCP Clients demand both high-availability and

performance


TCP-based Internet Services Adverse conditions

Core network congestion, overloaded servers, failed servers or under DoS attack

TCP response Network delays => packet loss => retransmission

TCP limitations Creates an implicit binding of service to a server ! Clients cannot change to another server for

sustained service


Migratory TCP

Transport layer protocol that supports dynamic connection migration

Extension to TCP A client connection can transparently migrate to

different servers during its lifetime Modified server applications cooperate to support the

handoff Client applications do not change

Servers can be geographically distributed Does not depend on application-specific info


Contributions

Designed a transport layer protocol that enables dynamic connection migration

Implemented a prototype on FreeBSD Demonstrated the utilization of our protocol

in building highly-available services Conducted preliminary performance

evaluation F. Sultan, K. Srinivasan, L. Iftode - “Transport

Layer Support for Highly-Available Network Services ”, HotOS 2001


Outline

Motivation Protocol Design Prototype Implementation Preliminary Performance Evaluation Protocol Utilization Related Work Conclusions and Future Work


The Migration Model

Client

Server 1 - overloaded

Server 2

congestion


Triggers and Initiators

Client

Server 1

Server 2

MIGRATE_TRIGGE

R

MIGRATE_TRIGGER

MIGRATE_TRIGGERMIGRATE_INITIATE


Design Issues

How to resume the service at the new server? Without the involvement of the client application Without full migration of the server application With low overhead

What is the state that needs to be transferred? Application state Protocol state


Contract Between Server Application and MTCP

Application Define fine-grained per-connection application state Export per-connection application state snapshot

periodically to the kernel After migration, import per-connection state to

resume service MTCP

Transfer the per-connection state Synchronize the application state with the protocol

state


Server Application Interface export_state

Export the state snapshot at the origin server Many times periodically

import_state Import the state snapshot at the destination

server Enables the destination server to resume from the

last snapshot Only once at the destination server

Light-weight connection migration


Server Application Examplewhile ((s = accept(ssock)) != -1){ if (import_state(s, &state))

num = state else

num = 0recv(s, &buf)

state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num

export_state(s, &state) . . .

}


Contract Between Server Application and MTCP

Application Define fine-grained per-connection application

state Export per-connection application state snapshot

periodically After migration, import per-connection state to

resume service MTCP

Transfer the per-connection state Synchronize the application state with the

protocol state


MTCP ProtocolServer 1 Server

2

Application

Protocol

Connections Connections


State Synchronization Examplewhile ((s = accept(ssock)) != -1){ if (import_state(s, &state))

num = state else


state = ++ num export_state(s, &state) recv(s, &buf) state = ++ num


}

1

13 2

Export

Application

MTCP


State Synchronization Example

while ((s=accept(ssock)) != -1){ if (import_state(s, &state))

num = state else


state = ++num export_state(s, &state) recv(s, &buf) state = ++num


}

2

14 3

RE

CV

Out of Sync

Application

MTCP


State Synchronization Example

1

13 2

RE

CV

MTCP

2

13 2

Application

MTCP

2

23 2

ApplicationApplication

EXPORT MTCP


Log-Based State Synchronization

Logs are maintained in the protocol At export time, protocol discards logs

(sync-ed) Logs form part of the state to be

transferred during migration Resumption of service starts from the last

state snapshot


Implementation

Modified the TCP/IP stack in FreeBSD kernel

Lazy connection migration mechanism Evaluation on synthetic benchmarks Target applications

HTTP server PostgreSQL front-end


Lazy Connection Migration

C (0)

C’

< S

tat e

Requ

est

> (

2)

< S

tate

Reply

> (

3)

Client

Server 1

Server 2

<SYN C,…> (1)

<SYN + ACK> (4)


Implementation: Per-Connection State

snap

not ack’ed

Write Buffer

Application Write

Application

Protocol


Implementation: Per-Connection State

X

From network

Read pointer

Read since snap

Read before snap

ack’ed

Application

ProtocolRead Buffer

snap


Preliminary Performance Evaluation

Experimental setup Two servers and a client: P III 233MHz, 256 MB

RAM Servers are connected by a dedicated network

link Benchmarks

Microbenchmark Simple web server like application


Microbenchmark

• One migration averaged over 200 runs for each state size

• All times are in microseconds

• The state size is in bytes


Microbenchmark

The time for migration varies linearly with state size


Stream Server Experiment

Server application sends a stream of 256K bytes in chunks of 1K each

Server performance degrades after a threshold Degradation is emulated by means of delays in

the server Migration policy module inserted into kernel at the

client Smoothed rate estimator Migration occurs when the estimated rate drops to 70%

of max. observed rate


Stream Server Experiment Results

Effective rate is almost equal to the best rate from the server


Implementation Summary

Addition of ~5000 lines of kernel code Changes to TCP finite state machine

Addition of two logical TCP states: Migrating and Blackhole

Time spent on the design and implementation was 9 months


Protocol Utilization

For high availability For performance through load balancing

schemes Servers trigger migrations based on a load

balancing policy Fault tolerance

Eager transfer of connection state information


Related Work

Classes of related work TCP Fault tolerance Process migration


TCP Related Works

HTTP server fail-over by connection migration [Snoeren ‘00]

soft TCP and HTTP state maintained at back-up server Fault-tolerant TCP [Alvisi ‘00]

failure masking persistent connections across server crashes

Stream Control Transmission Protocol (SCTP) - RFC 2960

multi-homing (many IP addresses / endpoint) ensures connectivity in the face of network failure


TCP Related Works (cont’d)

MSOCKS [Bhagwat `98] Proxy based “TCP splicing” for mobile clients

with multiple interfaces Indirect TCP [Bakre ‘95]

Connection handoffs between MSRs allow mobile hosts to maintain open connections with a fixed host


Fault Tolerance Related Works

Log-based rollback recovery The NonStop kernel [Bartlett ’81]


Process Migration Related Works

Condor [Litzkow ‘88] Locus [Popek ‘81] MOSIX [Barak ‘85] Sprite [Ousterhout ‘88]


Conclusions

We provide a transport layer protocol that enables building of highly-available services

The transport layer protocol allows dynamic server endpoint connection migration

We implemented a prototype in a FreeBSD kernel We designed an API for the server applications Preliminary performance evaluation on a web

server-like application achieved constant rate in the presence of server degradation

http://discolab.rutgers.edu/mtcp


Future Work

Investigate migration policies Alternate state transfer

implementations Recursive connection migration

mechanism Future applications


Acknowledgment

Florin – for contributing to the design and implementation of the project

Other members of DisCo Lab


Thank You

Demo in CoRE 336 !

Documents

M. S. Thesis Defense, 09/24/20011 Migratory TCP (MTCP) Transport Layer Support for Highly- Available Network Services Kiran Srinivasan DisCoLab Division