The DiPS+ software architecture for self-healing protocol stacks [email protected] Sam Michiels, Lieven Desmet, Wouter Joosen and Pierre Verbaeten

The DiPS+ software The DiPS+ software architecture for self-architecture for self-

healing protocol healing protocol stacksstacks

[email protected]@cs.kuleuven.ac.bec.be

Sam Michiels, Lieven Desmet, Wouter Joosen and Pierre VerbaetenDistriNet research group, K.U.Leuven, Belgium

WICSA-2004 2The DiPS+ software architecture

OverviewOverview Software architectures…Software architectures…

Software contextSoftware context Self-healing Self-healing The DiPS+ software architectureThe DiPS+ software architecture Modeling concurrencyModeling concurrency

A realistic case study on load A realistic case study on load managementmanagement Business objectiveBusiness objective ApproachApproach ResultsResults

Conclusion & future workConclusion & future work


Software contextSoftware context System softwareSystem software

Protocol stacks, web services, file systemsProtocol stacks, web services, file systems Strict functional & non-functional requirementsStrict functional & non-functional requirements

F: bug fix, security patch, version update, …F: bug fix, security patch, version update, … NF: concurrency, performance, security, …NF: concurrency, performance, security, …

Run-time adaptabilityRun-time adaptability Software architecture, separation of concerns, Software architecture, separation of concerns,

component frameworkcomponent framework Our vision: Our vision:

A tailored software architecture and an A tailored software architecture and an associated component framework can support associated component framework can support the development of run-time adaptable system the development of run-time adaptable system softwaresoftware


Software contextSoftware context The case on protocol stacksThe case on protocol stacks

Clear trend to more flexible, adaptive networksClear trend to more flexible, adaptive networks Active, ad-hoc, ubiquitous, pervasive, sensor networksActive, ad-hoc, ubiquitous, pervasive, sensor networks

Network nodes become self-healingNetwork nodes become self-healing Trends visible…Trends visible…

In research…In research… x-kernel, Click, Scout, SEDA, …x-kernel, Click, Scout, SEDA, …

In the market…In the market… More & more application-specific knowledge More & more application-specific knowledge insideinside

the protocol stack the protocol stack Cisco, Alcatel, NortelCisco, Alcatel, Nortel


Self-healingSelf-healing

Our view of self-healingOur view of self-healing Autonomously customizing system Autonomously customizing system

behaviorbehavior To handle specific overload situationsTo handle specific overload situations By injecting anticipated behavior to By injecting anticipated behavior to

fulfill a specific business policyfulfill a specific business policy Feedback control loopFeedback control loop Intercept data, process, propose solution, Intercept data, process, propose solution,

deploy it, activatedeploy it, activate


The DiPS+ research The DiPS+ research picturepicture

Prototype of self-healing management plane for load management [WoSS’02]

Real-life business case on healing strategies [WICSA’04]

Self-healing feedback control for load managementSoftware architecture

for adaptable protocol stacks (DiPS+)


The DiPS+ software The DiPS+ software architecturearchitecture

Helicopter view…Helicopter view… Pipe-and-filter stylePipe-and-filter style

Pipelines of processing componentsPipelines of processing components Blackboard style Blackboard style

In one pipeline (meta-information)In one pipeline (meta-information) Across pipelines (shared resources)Across pipelines (shared resources)

Layered styleLayered style Explicit support for layers and layer gluesExplicit support for layers and layer glues

PacketsPackets First class entities, transported along the First class entities, transported along the

pipespipes


TCP

IPv4

Eth

Layer


Packetflow

Component

Packet

Concurrencycomponent


A DiPS+ component:A DiPS+ component:

Explicit packet receivers (PR) and Explicit packet receivers (PR) and forwarders (PF)forwarders (PF)

Several types of units:Several types of units: Functional unitsFunctional units Dispatching unitsDispatching units Concurrency unitsConcurrency units

Connector binds forwarder and receiverConnector binds forwarder and receiver

Component


Connector

Unit PFPR UnitPRUnit PF



Extensible architecture: pluggable Extensible architecture: pluggable managementmanagement

Unit PFPR UnitPRUnit PF

Data plane

Management plane

policy

Collectorsensors

ManagementEngine


Traditional concurrency modelsTraditional concurrency models Single threaded, thread per request, (bounded) Single threaded, thread per request, (bounded)

thread poolthread pool Hidden in the systemHidden in the system

Need for application-specific customizationNeed for application-specific customization Difficult to adapt scheduling strategies once a Difficult to adapt scheduling strategies once a

thread starts executingthread starts executing Some information comes only available while Some information comes only available while

processingprocessing Difficult to differentiate between fine-grained Difficult to differentiate between fine-grained

taskstasks

Modeling concurrency in Modeling concurrency in DiPS+DiPS+



Our goalOur goal Mapping of application-specific Mapping of application-specific

characteristics or business policy into characteristics or business policy into concurrency modelconcurrency model

Provide the concurrency model as a Provide the concurrency model as a modular, adaptable, separate concernmodular, adaptable, separate concern

Our techniqueOur technique Concurrency components…Concurrency components…



Concurrency component, Concurrency component, responsible for…responsible for… Injecting active behaviorInjecting active behavior

Temporal buffering in the component pipelineTemporal buffering in the component pipeline Packet handler thread transport packets through Packet handler thread transport packets through

component areacomponent area Request schedulingRequest scheduling

Customizable scheduling strategyCustomizable scheduling strategy

Component Areas

Concurrency component



Advantages:Advantages: Reusability Reusability

Functional components without concurrencyFunctional components without concurrency Concurrency componentsConcurrency components

More fine-grained processing:More fine-grained processing: By classifying packets:By classifying packets:

Fine-grained and distributed control of schedulingFine-grained and distributed control of scheduling By classifying tasksBy classifying tasks

Prioritize between component areasPrioritize between component areas



Prototypes realized…Prototypes realized… RADIUS authentication protocolRADIUS authentication protocol Network protocolsNetwork protocols

TCP, UDP, ICMP, IP(v6), IPSec, SIP, ARP, …TCP, UDP, ICMP, IP(v6), IPSec, SIP, ARP, … RIP dynamic routing protocolRIP dynamic routing protocol Firewall customizationFirewall customization Safe & transparent component hot-Safe & transparent component hot-

swappingswapping







Business objectiveBusiness objective ContextContext

RADIUS authentication & accounting protocolRADIUS authentication & accounting protocol e.g. used for ADSL dial-ine.g. used for ADSL dial-in

RADIUS server overloadRADIUS server overload peak load ADSL dial-in from 18h00 till 20h00 peak load ADSL dial-in from 18h00 till 20h00

RADIUS sessionRADIUS session authenticate userauthenticate user session starts with “accounting start” requestsession starts with “accounting start” request during the session: accounting interim requestsduring the session: accounting interim requests session ends with “accounting stop” requestsession ends with “accounting stop” request


Business objectiveBusiness objective ObjectiveObjective

Compensate for the ADSL authentication Compensate for the ADSL authentication peak between 18h and 21hpeak between 18h and 21h

Business policyBusiness policy Differentiate between user typesDifferentiate between user types

Gold/silver/bronze usersGold/silver/bronze users Deny new authentication requests when Deny new authentication requests when

server reaches processing limit server reaches processing limit Prevent server from getting overloaded…Prevent server from getting overloaded…


ApproachApproach

OO RADIUS implementation OO RADIUS implementation (www.axlradius.com)(www.axlradius.com) Concurrency code is cross-cutting [Kiczales, Concurrency code is cross-cutting [Kiczales,

AOSD]AOSD]/** Current 64 byte block to process */ private byte[] currentBlock = new byte[64] ;

/** Constructor. */ public MD5(){ super("MD5") ; engineReset() ; }

// *********************** // JCA JAVA ENGINE METHODS // *********************** /** Method to reset the MD5 engine. */ public void engineReset(){ count = 0 ; state[0] = 0x67452301 ; state[1] = 0xefcdab89 ; state[2] = 0x98badcfe ; state[3] = 0x10325476 ; }

/** Method to add a byte to the current message. *@param input : the byte to append to the current message. */ public void engineUpdate(byte input){ //append byte to currentBlock currentBlock[(int)(count & 63)] = input ; //count&63 = count%64 //if currentBlock full => process if ((int)(count & 63) == 63){ //whole block => process MD5Transform() ; }

//and update internal state (count) count++ ; }

/** Method to add a byte array to the current message. *@param buf : the bytearray to append to the current message. *@param offset : the offset to start from appending the bytearray to the current message. *@param len : the length of the message to append to the current message. */ public void engineUpdate(byte[] buf, int offset, int len){ //FIRST : process first part of buffer until no more or full block //calculate number of bytes that fit in current block int no = java.lang.Math.min(len, 64 - (int)(count&63)) ; System.arraycopy(buf, offset, currentBlock, (int)(count&63), no) ; count += no ; len -= no ; offset += no ;

}

/** Method to calculate the digest of the current message. *After calculation, the engine is reset. *@return returns the message digest in a bytearray. */ public byte[] engineDigest(){ //calculate correct number of bits in total message long origMsgCount = count << 3 ; //append padding bits engineUpdate((byte)128) ; // append byte "10000000" while (((int)(count & 63)) != 56){ engineUpdate((byte)0) ; //append byte 0 until 56 mod 64 }

//append length (big endian) int[] cnt = new int[2] ; cnt[0] = (int) (origMsgCount & 0xffffffff) ; cnt[1] = (int) (origMsgCount >> 32) ; intToByte(currentBlock, 56, cnt, 0, 8) ; //process last block MD5Transform() ;

//return digest byte[] result = new byte[16] ; intToByte(result, 0, state, 0 , 16) ; //reset the engine for JCA compatibility engineReset() ;

return result ; }

/** Method to calculate the digest of the current message. *After calculation, the engine is reset. *@param buf : the byte array in which the digest is put. *@param offset : the offset from where the digest is put in the bytearray. *@param len : the length of free space in the bytearray. *@return returns the length of the messagedigest. */ public int engineDigest(byte[] buf, int offset, int len) throws DigestException { //if not enough space in buf, return if (len < 16) throw new DigestException("Buffer too small.") ;

//calculate digest, copy into buf and return byte[] result = engineDigest() ; System.arraycopy(result, 0, buf, offset, result.length) ; return result.length ; }

/* Method to get the length of a digest.}

}

public class MD5Test { public static void main(String[] args){ MessageDigest digest = null ; Security.addProvider(new DistriNet()) ; try{ digest = MessageDigest.getInstance("MD5", "Distrinet") ; } catch(Exception e){ e.printStackTrace() ; System.exit(1) ; } digest.update(args[0].getBytes()) ; System.out.println("Input : " + formatBin2Hex(args[0].getBytes(), 16, 2) +"\n" ) ; System.out.println("Digest : " + formatBin2Hex(digest.digest(), 16, 2) +"\n" ) ; int m = ( n % s1 ); for ( int i = m ; i < s1 ; i++ ) { if ( ( i % s2 ) == 0 ) result += " "; result += " "; ascii += " "; } if ( m > 0 ) { result += " [" + ascii + "]\r\n"; } return result; }

private static String _hexmap = "0123456789abcdef"; private static String _int2hex ( long i , int n ) { String result = ""; for ( int j = 0 ; j < n ; j++ ) { int m = (int)(i & (long)0xf);}}

concurrency


ApproachApproach DiPS+ modularized designDiPS+ modularized design

Authentication

UDP/IP stack

HdrParser

AttributeParser

NAS CheckerAuthenticatorAcceptReplyPreparator

AccessRequestAuthCalculator

HdrConstructor

AttributeConstructor

RejectReplyPreparator

HdrParser

AttributeParser

NAS CheckerAcctReplyPreparator

Accounter

HdrConstructor

AttributeConstructor

AcctRequestAuthChecker

UDP/IP stack

Accounting


ApproachApproach

Case1: Gold/Silver/Bronze user Case1: Gold/Silver/Bronze user authenticationauthentication• Idea: Idea: don’t load the protocol stack with low-don’t load the protocol stack with low-

priority requestspriority requests

UDP/IP stackUDP/IP stack

RADIUS authentication

Attributeparser

HdrParser

NASChecker

AuthenticatorAccesRequestAuthCalculator

AcceptReplyPreparator


Attributeconstructor

HdrConstructor


ApproachApproach





Attributeparser

HdrParser

NASChecker





HdrConstructor


ApproachApproach





Attributeparser

HdrParser

NASChecker





HdrConstructor

G S B

Concurrencycomponent


ApproachApproach

Case2: Case2: Authentication versus Authentication versus accountingaccounting Idea: Idea: maximize number of established maximize number of established

sessions, control creation of new sessionssessions, control creation of new sessions

UDP/IP stack

Authentication Accounting


ApproachApproach Case3: combinationCase3: combination

Idea: Idea: maximize number of established sessions, control maximize number of established sessions, control creation of new sessions, while prioritizing between userscreation of new sessions, while prioritizing between users

UDP/IP stack

Authentication AccountingG S B


Some results…Some results…


Non-DiPS+ implementation

Gold users

Silver users

Bronze users

DiPS+ implementation, without load management

Gold users

Silver users

Bronze users

< 5% performance loss


DiPS+ implementation with load management

(gold/silver/bronze)

Gold users

Silver users Bronze

users

I II III

I II III

I II III

In = 30 r/sOut = 30 r/s

In = 30 r/sOut = 30 r/s

In = 30 r/sOut = +/- 0 r/s

In = 30 r/sOut = 30 r/s

In = 60 r/sOut = +/- 30 r/s

In = 30 r/sOut = 0 r/s

In = 60 r/sOut = 60 r/s




Non DiPS+ RADIUS server accepts every authentication request, but 100 accounting requests per sec are dropped!

Non-DiPS+ throughput = 30 authentication requests/sec

In = 30 r/sOut = 30 r/s

In = 300 r/sOut = 200 r/s

DiPS+ server protects itself against overload by rejecting session initiations

when processing limit has been reached

DiPS+/DMonA applied. Throughput =

20 authentication requests/sec

In = 30 r/sOut = 20 r/s

In = 200 r/sOut = 200 r/s


Load managementLoad management Summary Summary

Cost of modularized software architecture Cost of modularized software architecture support is minimalsupport is minimal

Flexibility by using concurrency componentsFlexibility by using concurrency components Using various application-specific concurrency Using various application-specific concurrency

strategiesstrategies Request control can be injected at arbitrary Request control can be injected at arbitrary

placesplaces While reusing functional componentsWhile reusing functional components







ConclusionConclusion Few implementations of protocol stacks Few implementations of protocol stacks

define and apply an explicit software define and apply an explicit software architecturearchitecture

Related protocol stack architectures often Related protocol stack architectures often lack a clear separation of concurrency and/or lack a clear separation of concurrency and/or management aspects from functional codemanagement aspects from functional code

DiPS+ offers flexibility by its modularized DiPS+ offers flexibility by its modularized design and concurrency componentsdesign and concurrency components

Overhead of the architecture is compensated Overhead of the architecture is compensated by deploying application-specific strategies by deploying application-specific strategies in the protocol stackin the protocol stack Providing software architecture support pays offProviding software architecture support pays off


Future workFuture work Combining feedback control loop with Combining feedback control loop with

specific load management strategiesspecific load management strategies Prototype management plane [WoSS’02]Prototype management plane [WoSS’02] Business case [WICSA’04]Business case [WICSA’04]

Load management in a distributed Load management in a distributed contextcontext Coordinating multiple self-management Coordinating multiple self-management

systemssystems Combining management planes Combining management planes

load management load management hot-swappinghot-swapping


Q&AQ&A

Documents

The DiPS+ software architecture for self-healing protocol stacks [email protected] Sam Michiels, Lieven Desmet, Wouter Joosen and Pierre Verbaeten