Real-Time Switched Networks - comp.polyu.edu.hkcsqwang/research/RealTimeSwitchedNet… · Real-Time Switched Networks Qixin Wang Department of Computing. The Hong Kong Polytechnic

Real-Time Switched Networks

Qixin WangDepartment of Computing

The Hong Kong Polytechnic University2012

Part I. What is Real-Time?

Robotic Surgery: each task is a continuous loop of sensing (or actuating) jobs

Each job:

1. Must catch deadline

2. Does not have to be fast

Aviation and Industrial Control: each task is a continuous loop of sensing (or actuating) jobs

Each job:

1. Must catch deadline

2. Does not have to be fast

A typical real-time task is a continuous loop of periodic jobs.

TimePeriod

Exe. Time Exe. Time Exe. Time

Job: (Period, Exe. Time, Deadline)A Job

Deadline

A typical real-time task is a continuous loop of periodic jobs.

TimePeriod

Exe. Time Exe. Time Exe. Time

Job: (Period, Exe. Time, Deadline)

Real-time = each job catches deadline

Real-time ≠ running fast

A Job

Deadline

Part II. Adapting a Main-Stream Internet Switch Architecture for Multi-Hop Real-Time

Networks

Content

Demand

Background

Solution

Evaluation

Real-Time Switch is the key component for many infrastructures

Real-Time WAN, Real-Time Internet Subnet

Real-Time Fieldbus

RT High Performance Computing Architecture, e.g., InfiniBand



Real-Time Fieldbus



Telepresence


Tele-robotic underground mining saves lives

2000ft (609.6m)

2000ft (609.6m)

> 5000 annual death toll



Real-Time Fieldbus


Real-Time Fieldbus: Scaling from Shared Medium to Multi-Hop

Robotic Manufacturing



Real-Time Fieldbus


RT High Performance Computing Architecture (e.g., InfiniBand): Scaling from Shared Medium to Multi-Hop

Concord

1976

RT High Performance Computing Architecture (e.g., InfiniBand): Scaling from Shared Medium to Multi-Hop

A380

2007,

Several Hundreds of computing nodes



Real-Time Fieldbus


Designing real-time switch is not easy1.

A. Demers, S. Keshav, and S. Shenker, “Analysis and simulation of a fair queueing

algorithm,”

in SIGCOMM’89, 1989. pp. 1-12.

2.

D. Ferari, and D. Verma, “A scheme for real-time channel establishment in wide-area networks,”

in IEEE JSAC, v8, n3, April, 1990

3.

D. C. Verma, H. Zhang, and D. Ferrari, “Delay jitter control for real-time communication in packet switching network,”

in Proceedings of Tricomm’91, pp. 35-46, April 1991.

4.

S. Golestani, “A framing strategy for congestion management,”

in IEEE JSAC, v9, n7, pp. 1064-1077, Sep., 1991

5.

L. Zhang, "VirtualClock: a new traffic control algorithm for packet-switched networks," ACM Trans. on Computer Systems, Vol. 9, No. 2, May 1991

6.

A. K. Parekh, R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Single-Node Case,”

in IEEE/ACM ToN, v1, n3, June, 1993.

7.

A. K. Parekh, R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Multiple Node Case,”

in IEEE/ACM ToN, v2, n2, April, 1994.

8.

S. Jamaloddin

Golestani, ``A self-clocked fair queuing scheme for broadband applications,'' Proc. IEEE INFOCOM'94, pages 636--646, IEEE, 1994.

9.

H. Zhang, "Service Disciplines For Guaranteed Performance Service in Packet-Switching Networks," Proceedings of the IEEE, 83(10), Oct 1995

10.

Jon C. R. Bennett and H. Zhang, ``WF2Q: worst-case fair weighted fair queueing,'' IEEE INFOCOM'96, March, 1996

11.

Pawan

Goyal, Harrick

M. Vin, and Haichen

Cheng, "Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks," ACM SIGCOMM'96, pages 157--168, ACM press, August, 1996

12.

I. R. Philp, “Scheduling real-time messages in packet-switched networks,”

Ph.D. Thesis, Technical Report No. UIUCDCS-R-96-1977, CS Dept., UIUC, Oct., 1996.

13.

Subhash

Suri, George Varghese, and Girish

Chandranmenon, ``Leap forward virtual clock: A new fair queuing scheme with guaranteed delays and throughput fairness,'' IEEE Proc. INFOCOM'97, 1997

14.

I. Stoica, S. Shenker, and H. Zhang, ``Core-stateless fair queuing: achieving approximation fair bandwidth allocations in high speed networks,'' Proc. of ACM SIGCOMM'98, October 1998, pp. 118-130.

15.

C. R. Kalmanek, H. Kanakia, and S. Keshav, “Rate controlled servers for very high speed networks,”

in IEEE Global Telecommunications Conference, Dec., 1999.

16.

W. Sun, and K. G. Shin, “End-to-end delay bounds for traffic aggregates under guaranteed-rate scheduling algorithms,”

in IEEE ToN, 13(5): 1188-1201, Oct., 2005.

Huge Volume of Literature since 1989

Today’s Internet is still NOT real-time.

Measured delay btw UK and Spain [Marsh06]

Today’s Internet is still NOT real-time.

Measured delay btw UK and Spain [Marsh06]

Why?

Industry acceptance has the final say. Two things rules: simplicity and switch hardware features

Output

Queueing

+

QoS

Crossbar

+

Input

Queueing

Crossbar

+

VOQ

+

iSLIP

Combined input-output queueing, or

more complicated switch hardware,

to emulate output queueing

+

QoS

(e.g. WFQ)

N Speed-Up

Problem

HOL

Problem

Past lessons tell us to design the real-time switch compatible to “crossbar + VOQ + iSLIP”

Output

Queueing

+

QoS

Crossbar

+

Input

Queueing

Crossbar

+

VOQ

+

iSLIP

Combined input-output queueing, or

more complicated switch hardware,

to emulate output queueing+

QoS

(e.g., WFQ)

N Speed-Up

Problem

HOL

Problem

What is “Crossbar + VOQ + iSLIP”?

A widely implemented switch architecture

Simple

High switch utilization

Fast adaptation to random traffic (Internet traffic)

Advantages

I1

I2

I3


Input Ports

I1

I2

I3

O1

O2

O3


Output Ports

I1

I2

I3

O1

O2

O3


Virtual Output Queue (VOQ)

I1

I2

I3

O1

O2

O3


cells

I1 O1

O2

O3


I2

I3

cell cell cell

cell cell

cell

2012/8/2Qixin

Wang, UIUC,

I1

I2

I3

O1

O2

O3


Synchronous periodic cell forwarding Cell-Time

I1

I2

I3

O1

O2

O3


Matching

I1

I2

I3

O1

O2

O3


Why Matching? An input/output can only send/receive one cell per cell-time

I1

I2

I3

O1

O2

O3


I1

I2

I3

O1

O2

O3


I1

I2

I3

O1

O2

O3


I1

I2

I3

O1

O2

O3

Request (apply job)

What is “Crossbar + VOQ + iSLIP”? iSLIP

is like job hunting

I1

I2

I3

O1

O2

O3I1

I2

I3

O1

O2

O3

Request (apply job)

Grant (offer job)


is like job hunting

I1

I2

I3

O1

O2

O3I1

I2

I3

O1

O2

O3

I1

I2

I3

O1

O2

O3

Request (apply job)

Grant (offer job)

Accept (take offer)


is like job hunting

But we cannot directly adopt “crossbar + VOQ + iSLIP”

Huge per hop delay bound because of poor isolation;

E2E delay bound is an open problem [Gopalakrishnan06].




What to do?




What to do?

Look at changed assumptions and

new features.

Changed assumption: traffic predictability

iSLIP

is for non-real-time traffic: Unpredictable


iSLIP

is for non-real-time traffic: UnpredictableFlows change rapidlyUnpredictable message size and arrival

Unpredictable traffic forces

iSLIP

to negotiate for each cell-time

I1

I2

I3

O1

O2

O3I1

I2

I3

O1

O2

O3

I1

I2

I3

O1

O2

O3

Request (apply job)

Grant (offer job)

Accept (take offer)

Just like job application, iSLIP’s

negotiate is dynamic, hard to give tight delay upper bound.


iSLIP


RT-Switch for real-time traffic: Deterministic


iSLIP


RT-Switch for real-time traffic: DeterministicFlows rarely changeRegular message size and arrivalNon-real-time traffic real-time traffic

Deterministic traffic allows RT-Switch to forward cells like clockwork, no need to negotiate!

I1

I2

I3

O1

O2

O3

Deterministic Grant is enough

No need for Request and Accept

Simplifies instead of extends iSLIP

New feature: Large message arrival period allows clock-driven time slicing

Real-time traffic


Real-time trafficSmall msg

size (sensing: 1cell/msg, TV quality video: 480cell/msg)



size (sensing: 1cell/msg, TV quality video: 480cell/msg)Short per msg

transmission time (s: 0.5us, v: 240us)




transmission time (s: 0.5us, v: 240us) Large msg

arrival period (s: 10ms, v: 30ms)






Reminds us of clock-driven time slicing OS






Reminds us of clock-driven time slicing OS

RT-Switch runs fine time grain clock period (1ms), serving time slices (cell-time) to coarse time grain periodic RT flows

Solution: crossbar RT-Switch runs deterministic clock-driven time slicing


Clock period of M cell-time, e.g., M = 5

Cell time: 1 2 3 4 5

I1:

I2:

I3:

I4:



I1:

I2:

I3:

I4:

Fit original real-time flow-forwarding tasks into clock period, e.g., (11, 3) (5, 2), i.e., (10, 4)



Fit original real-time tasks into clock period, e.g., (11, 3) (5, 2), i.e., (10, 4)


I1:

I2:

I3:

I4:

Demand

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


Demand


I1:

I2:

I3:

I4:

Clock period of M cell-time


Schedule

Scheduling

Algorithm

Fit original RT tasks into clk

period

Config. time scheduling (O(N4))


I1:

I2:

I3:

I4:

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:Forward cells according to the schedule during runtime



period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:


Forward cells according to the schedule during runtime


period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:




period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:




period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:




period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:




period



Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


I1:

I2:

I3:

I4:

Satisfies demands:

•

Real-Time Guarantee•

Backward Compatibility

•

Simplicity•

Performance

•

Isolation


Schedule

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4

Evaluation: Schedulabililty

Randomized traffic demand based on typical real-time industrial/medical application models

Total demanded switch utilization is U

Given U, what is the schedulable ratio?

Evaluation: Schedulability

Evaluation: Schedulability

Evaluation: E2E Delay Bound

Randomized traffic demand based on typical real-time industrial/medical application models

Max hop count is 15

Compare E2E Delay Bound provided by Real-Time Switch and iSLIP



Conclusion

Short E2E delay guarantee

Good schedulability

Compatible to, and simpler than the widely implemented iSLIP

crossbar switch.

O(1) runtime computation

Polynomial configuration time computation

References[Bharghavan94] Vaduvur

Bharghavan, et al., “MACAW: A Media Access Protocol for Wireless LANs.”

In SIGCOMM 1994: 212-225.[Gopalakrishnan06] S. Gopalakrishnan, M. Caccamo, and L. Sha, Switch Scheduling and Network Design for Real-Time Systems. In Proc. Of IEEE Real-Time and Embedded Technology and Applications (RTAS), Apr. 2006.[March06] Marsh J., Glencross

M., Pettiffer

S., Hubbold

R. J., “A robust network architecture supporting rich behaviour

in collaborative interactive applications.”

In IEEE Transactions on Visualization and Computer Graphics (TVCG), 12, 3 (May 2006), 405=416.[Mckeown99] Nick McKeown, “The iSLIP

Scheduling Algorithm for Input-Queued Switches”, in IEEE/ACM Transactions on Networking, vol. 7, no. 2, April 1999. [Ploplys04] N.J. Ploplys, et al., “Closed-Loop Control over Wireless Networks.”

In IEEE Control Systems Magazine, vol. 24, June, 2004.[wang08] Qixin

Wang, Sathish

Gopalakrishnan, Xue

Liu, and Lui

Sha, “A switch design for real-time industrial networks,”

in Proc. of IEEE RTAS 2008, Apr. 2008, pp. 367–376.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010. [Willig02] Willig

A., et al., “Measurements of a wireless link in an industrial environment using an IEEE 802.11-compliant physical layer.”

In IEEE Transactions on Industrial Electronics, v43, n6, pp1265-1282, Dec., 2002.

Contents of Part II are published in [wang08] and [wang10].

Part III. A Multicast Routing Scheme for Multi-Hop Real-Time Switched Networks

ContentDemand

Background

Problem Definition and Complexity

Heuristic Algorithm

Evaluation

Related Work

Real-Time Switched Networks (RTSN)

Specialized networks used in industrial, mining, medical, vehicular, avionic environments


RTSN is fundamentally different from the Internet

Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)



RTSN: The Internet:




RTSN:(Hard) Real-Time

The Internet:Best Effort





Periodic Stable Traffic


Random Bursty Traffic






Bounded w/ Global Info



Unbounded w/o Global Info



RTSN is fundamentally different from the Internet, hence requires different solutions.



Bounded w/ Global Info



Unbounded w/o Global Info

Shared medium multi-hop switched: real-time multicast becomes a problem.

11

111

)()()()()(

nnmm

mmnnnnn

txKtutuBtxAtx

Modern control assumes MIMO Real-Time Multicast btw

Sensors-Controllers-Actuators/Observers

Shared medium multi-hop switched: real-time multicast becomes a problem.

11

111

)()()()()(

nnmm

mmnnnnn

txKtutuBtxAtx

Shared Medium Real-Time Multicast: Easy

Multi-Hop Switched Real-Time Multicast: ?

Recap of our real-time switch architecture: crossbar per-flow-q TDMA

I1

I2

I3

Input Ports


I1

I2

I3

O1

O2

O3

Output Ports


I1

I2

I3

O1

O2

O3

Per-Flow-Queueing


I1

I2

I3

O1

O2

O3

cells


I1 O1

O2

O3

I2

I3

cell cell cell

cell cell

cell


I1

I2

I3

O1

O2

O3

Synchronous periodic cell forwarding Cell-Time


I1

I2

I3

O1

O2

O3

Matching


I1

I2

I3

O1

O2

O3

Why Matching? An input/output can only send/receive one cell per cell-time


I1

I2

I3

O1

O2

O3

Internal Matching: if an input has multiple per-flow-q for the same output, only one is picked every cell-time.


I1

I2

I3

O1

O2

O3


I1

I2

I3

O1

O2

O3


I1

I2

I3

O1

O2

O3


Fit all real-time flows’

periods into frame, e.g., (11, 3) (5, 2), i.e., (10, 4)


I1:

I2:

I3:

I4:

Demand

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4

TDMA scheduling frame

of M cell-time, e.g., M = 5


Demand


I1:

I2:

I3:

I4:

Schedule

Scheduling

Algorithm

Theorem 1: If demand matrix’ every color ≤ M cell, then

have config. time scheduler with O(N4) time cost [wang10].


I1:

I2:

I3:

I4:

a cell to send to

O1

a cell to send to

O2

a cell to send to

O3

a cell to send to

O4


Simple extension to support multicast: only need to change network layer.

I1

I2

I3

O1

O2

O3

Support for Multicast

c

I1

I2

I3

O1

O2

O3c

cc

Support for Multicast

Simple extension to support multicast: only need to change network layer.

Real-Time Multicast Scheduling (RTMS) problem in RTSN

),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

Graph of the RTSN


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

Switches (nodes) of

the network

Graph of the RTSN


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

Switches (nodes) of

the network

Edges of the network

Graph of the RTSN


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)

multicast group


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)

multicast groupSource

End


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)


End

Destination Ends


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)


End

Destination Ends

Cells to multicast

every period


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)


End

Destination Ends

Cells to multicast

every period

Period (unit: cell-time)


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

A (real-time)


End

Destination Ends

Cells to multicast

every period

Period (unit: cell-time)

Deadline (relative, unit: cell-time)


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

The set of all real-time multicast groups


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

The set of all real-time multicast groups

The ith

real-time multicast group


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

An instance of RTMS problem


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


The RTSN topology


),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq


The RTSN topology

The set of RT multicast groups

that user demands

RTMS Problem: Given a q, how to schedule each switch s.t. every mi meets its needs?

),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

RTMS Problem: Given a q, how to schedule each switch s.t. every mi meets its needs?

),( EVG

),,,,( HTwDsm

}{ imM

)),,(( MEVGq

Theorem: RTMS Problem is NP-Hard.

M-slot Periodic RTMS: a subset of RTMS problems.

}|{ instance RTMS an is qqQ

},),,,,,(},{

,),('|'{

MTiHTwDsmm

Gqq'

i

iiiiiii

andM

QMQ



},),,,,,(},{

,),('|'{

MTiHTwDsmm

Gqq'

i

iiiiiii

andM

QMQProposition: M-slot Periodic

RTMS is NP-Hard.



},),,,,,(},{

,),('|'{

MTiHTwDsmm

Gqq'

i

iiiiiii

andM

QMQProposition: M-slot Periodic

RTMS is NP-Hard.

Search for Heuristic Solutions.

Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.

.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM


.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM

M-slot Periodic RTMS instance


.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM


RTMR instance


.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM


Max Multicast Tree HeightRTMR

instance


.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM

).,('

)~

,(~

MM

Gq

Gq

to solution a

is to solutionA


.0,1

max~

)},~),(,,,{(}~{~

),~

,(~

)},),(,,,{(}{

,'),('

MMHH

HMTwDsm

Gq

HMTwDsm

Gq

ii

iiiiii

iiiiii

and where

define where

Given

M

M

M

QM

).,('

)~

,(~

MM

Gq

Gq

to solution a

is to solutionA

RTMScheduling

problem becomes a Routing problem.

Heuristic Routing Algorithm

Existing mainstream internet multicast routing algorithms become Dijkstra

when network is static

and global info is available

Dijkstra’s

short-coming: only cares about # of hops, ignores congestion.

s

s d





Dijkstra’s

short-coming: only cares about # of hops, ignores congestion.

s

s d





We want a heuristic routing algorithm that considers both (hops and congestion).

s

s d





We want a heuristic routing algorithm that considers both (hops and congestion).

s

s d


Grow the multiple trees simultaneously with multiple iterations.



In each iteration, ≤1

link is added to each tree.





When multiple trees contend in a same switch, we carry out a job-hunting-like negotiation to let only ≤1 contending tree grow through an output in this iteration.

The job-hunting-like contention negotiation is itself an iterative algorithm.

I1

I2

I3

O1

O2

O3

different colors for different trees


I1

I2

I3

O1

O2

O3Each tree ranks all outputs and only apply to

its favorite

output



I1

I2

I3

O1

O2

O3I1

I2

I3

O1

O2

O3

Each tree ranks all outputs and only apply to

its favorite

output

Each output offers job

to the most loyal

applicant



I1

I2

I3

O1

O2

O3I1

I2

I3

O1

O2

O3

I1

I2

I3

O1

O2

O3Each tree ranks all outputs and only apply to

its favorite

output

Each output offers job

to the most loyal

applicant

Accept job

by reserving corresponding frame slots.


updated demand matrix

Ranking function design: considers both E2E delay (hops) and congestion.


rank of o to tree t


rank of o to tree t

routing flexibility: slack to reaching max hop (max e2e delay) bound


rank of o to tree t


shortest distance to target end


rank of o to tree t

congestion


shortest distance to target end

Definition of “favorite”, a.k.a., “loyalty”.

t ’s loyalty to o(t,1)

The proposed heuristic algorithm terminates within polynomial time.




When multiple trees contend in a same switch, we carry out a job-hunting-like negotiation to let only ≤1 contending tree grow through an output in this iteration.

Evaluation Setup.

4x4 port real-time switches

15x15 square grid network topology

Per port capacity: 1Gbps,

M

= 2000 (cell/frame), 1 cell = 500 bit

10000 Trials, in each trial:

Random number of multicast groups,

wi

= 1~20 cell/frame (500K~10Mbps)

Evaluation Setup.

Evaluation Setup.

Allowed Tree

Height

Evaluation Setup.

Allowed Tree

Height

Minimum Tree

Height

Evaluation Setup.

Network Demanded Utilization (Application Layer E2E Utilization)

Allowed Tree

Height

Minimum Tree

Height

Evaluation Results: =3

Evaluation Results: =9

Mainstream Internet multicast routing algorithms mainly concern about dynamic distributed group management.

Reverse Path Broadcasting/Multicasting (RPB/RPM) [semeria97]

Truncated Reverse Path Broadcasting (TRPB) [semeria97]

Distance Vector Multicast Routing Protocol (DVMRP) [waitzman88]

Multicast Extension to Open Shortest Path First (MOSPF) [moy94a][moy94b]

Protocol Independent Multicast (PIM) [fenner06][adams05]

Core-Based Tree Multicast Routing (CBT) [ballardie97]

Mainstream Internet multicast routing algorithms become Dijkstra

for static network with global info.

Reverse Path Broadcasting/Multicasting (RPB/RPM) [semeria97]

Truncated Reverse Path Broadcasting (TRPB) [semeria97]

Distance Vector Multicast Routing Protocol (DVMRP) [waitzman88]

Multicast Extension to Open Shortest Path First (MOSPF) [moy94a][moy94b]

Protocol Independent Multicast (PIM) [fenner06][adams05]

Core-Based Tree Multicast Routing (CBT) [ballardie97]

OthersP2P and Overlay Network Multicast are concerned with statistical performance instead of hard real-time E2E delay bound.

De facto real-time switch architecture [wang10][wang08] [dopatka07][leung97]

Conclusion

Multicast routing in RTSN is NP-hard.

Heuristic algorithm exists that performs much better than shortest path (Dijkstra).

References[adams05] A. Adams, J. Nicholas, and W. Siadak, Protocol Independent Multicast–

Dense Mode (PIM-

DM): Protocol Specification (Revised), RFC 3973, Jan., 2005.[ballardie97] A. Ballardie, Core Based Tees (CBT) Multicast Routing Architectre, RF 2201, Spe., 1997.[chen11] Lixiong

Chen, Xue

Liu, Qixin

Wang, Yufei

Wang, "A Real-Time Multicast Routing Scheme for Multi-Hop Switched Fieldbuses," in Proc. of the 30th IEEE International Conference on Computer Communications (IEEE INFOCOM 2011), Shanghai, China, April, 2011. pp.3209-3217. [dopatka07] F. Dopatka

and R. Wismuller, “Design of a realtime

industrial ethernet

network including hot-pluggable asynchronous devices,”

IEEE International Symposium on Industrial Electronics (ISIE 2007), 2007.[fenner06] B. Fenner, M. handley, H. Holbrook, and I. Kouvelas, Protocol Independent Multicast –

Sparse Mode (PIM-SM): Protocol Specification (Revised), RFC 4601, Aug., 2006.[leung97] Yiu-Wing Leung and Tak-Shing

Yum, “A TDM-based multibus

packet switch,”

IEEE Trans. on Communications, vol. 45, no. 7, pp. 859–866, Jul. 1997.[moy94a] J. Moy, Multicast Extensions to OSPF, RFC 1584, Mar., 1994.[moy94b] J. Moy, MOSPF: Analysis and Experience, RFC 1585, Mar., 1994.[semeria97] C. Semeria

and T. Maufer, Introduction to IP Multicast Routing, IETF draft-ietf-mboned-

intro-multicast-03.txt, Jul., 1997.[waitzman88] D. Waitzman, C. Partridge, and S. Deering, Distance Vector Multicast Routing Protocol, RFC 1075, Nov., 1988.[wang08] Qixin Wang, Sathish

Gopalakrishnan, Xue

Liu, and Lui

Sha, “A switch design for real-time industrial networks,”

in Proc. of IEEE RTAS 2008, Apr. 2008, pp. 367–376.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010.

Contents of Part III are published in [chen11].

Part IV. Analysis of the TDMA Crossbar Real-Time Switch Design for AFDX Networks

Content

Compliance Analysis

Resource Planning

Related Work

Demand

AFDX is a data network for safety-critical applications that utilizes dedicated bandwidth while providing deterministic

Quality of Service (QoS). –

from wikipedia

AFDX Network

10/100Mbit switched Ethernet

Based upon IEEE 802.3 and ARINC 664

Bridges the gap on reliability of guaranteed bandwidth

in ARINC 664

Adopted by Airbus A380, Boeing 787 Dreamliner

etc

Background: Avionics Full DupleX

(AFDX) Switched Ethernet

AFDX Concepts

Elements in an AFDX network:AFDX End-systemAFDX SwitchAFDX Links

AFDX FeaturesReal-time end-to-end delay guarantee through traffic shapingOpen switch architecture: AFDX standard leaves the switch

architecture design open, so that vendors can reuse their legacy

switch architectures.

AFDX compliant switch: as long as the switch can support virtual link (VL) traffic shaping.

Each VL conducts one unicast

flow from a source-end to a destination-end (e.g. E1 to E5) ;Along the VL’s

route, before entering each AFDX switch, the VL flow must behave as if policed by a token bucket;With the per hop token bucket policing and proper switch architecture design, we can guarantee end-to-end real-time for each VL.

Demand

AFDX standard leaves the switch architecture design open, so that vendors

can reuse their legacy switch architectures.

Demand: build AFDX networks using our real-time switch architecture, the TDMA crossbar real-time switch

AFDX Compliance

AFDX complianceAlong the VL’s

route, before entering each AFDX switch, the VL flow must behave as if policed by a token bucket;With the per hop token bucket policing and proper switch architecture design, we can guarantee end-to-end real-time for each VL

Analysis: AFDX Compliance

•

Theorem 2 (AFDX Compliance)–

Per flow analysis with network calculus–

Giving end to end delay

src

end: source a: arrival curve for each flow at a switchs: service curve for each flow at a switchv: TDMA crossbar real-time switchH: total number of hops for a flowdes end: destination Lf

: flow f’s

in-network maximum packet lengthPf

: flow f’s

in-network periodCf

: per-frame allocated slots

Resource Planning Problem

P(G(V,E),F): TDMA crossbar real-time switch AFDX network resource planning problem

•

AFDX network G(V,E), where V

is the set of all switches and E

is the set of links between the switches• F

is the set of flow in the AFDX network

Objectivenetwork utility maximization

Constraintsswitch schedulability

Constraintsend-to-end delay guarantee

Decision variablesCf


Resource Planning Problem

P(G(V,E),F): TDMA crossbar real-time switch AFDX network resource planning problem

•

AFDX network G(V,E), where V

is the set of all switches and E

is the set of links between the switches• F

is the set of flow in the AFDX network

Objectivenetwork utility maximization





Alternatives for flow f

Resource Planning Problem: NP-HardObjectivenetwork utility maximization





•

Knapsack problem has been known as NP-Hard•

An instance of knapsack problem ҡ(Ξ, size, value, Өs ,Өv

) can be reduced to an instance of TDMA crossbar real-time switched AFDX network resource planning problem:–

Construct an AFDX network of three nodes: one source-end, connected by one TDMA crossbar switch to one destination end.

–

becomes equivalent to asking ‘is the constructed resource planning problem results in a maximum ≥ Өv

’

Why NP-Hard

Approximation Algorithm

To address the challenge that resource planning problem P(G(V,E),F) is NP-Hard, we propose a re-modeling approach, with which, we propose an approximation algorithm for P(G(V,E),F)

Let U~

and U*

be the total utility achieved by approximation algorithm and the optimum total utility respectively. We have

Related Work

•

Analysis of real-time behavior of AFDX networks upon switches [scharbarg09][charara06][chen11]

•

Industrial fieldbus

designs [santos09]•

TDMA Crossbar Switch Design [wang10]

•

Knapsack problem approximation algorithm

Conclusion

We proved that TDMA crossbar real-time switched network is AFDX compliant.

We proved the corresponding AFDX network’s resource planning problem is NP-Hard.

We proposed an approximation algorithm for the resource planning problem.

References[charara06] H. Charara, J. luc

Scharbarg, J. Ermont, and C. Fraboul, “Methods for bounding end-to-end delays on an afdx

network,”

in Euromicro

Conference on Real-Time Systems, 2006.[chen11] Lixiong

Chen, Xue

Liu, Qixin

Wang, Yufei

Wang, "A Real-Time Multicast Routing Scheme for Multi-Hop Switched Fieldbuses," in Proc. of the 30th IEEE International Conference on Computer Communications (IEEE INFOCOM 2011), Shanghai, China, April, 2011. pp.3209-3217. [rao12] Lei Rao, Qixin

Wang, Xue

Liu, Yufei

Wang, "Analysis of TDMA Crossbar Real-Time Switch Design for AFDX Networks," in Proc. of IEEE INFOCOM'12, March, 2012. pp.2462-2470. [santos09] R. Santos, R. Marau, A. Vieira, P. Pedreiras, A. Oliveira, and L. Almeida, “A synthesizable ethernet

switch with enhanced real-time features,”

in Proc. of IECON’09, pp. 2817–2824, 2009.[scharbarg09] J.-L. Scharbarg, F. Ridouard, and C. Fraboul, “A probabilistic analysis of end-to-end delays on an afdx

avionic network,”

in IEEE Transactions on Industrial Informatics, v5, n1, pp.38-49, 2009.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010.

Contents of Part IV are published in [rao12].

http://www4.comp.polyu.edu.hk/~csqwang/research/INFOCOM12.AFDX.full.pdf

http://www4.comp.polyu.edu.hk/~csqwang/research/INFOCOM12.AFDX.full.pdf

Thank You!

Supplementary Slides




Support for multicast: only need to change network layer

Support for multicast: only need to change network layer

De facto standard (real-time) fieldbus switch architecture: crossbar per-flow-q TDMA

Background: Token Bucket

A token bucket flow is defined by (ρ,σ)

ρ

denotes the bucket refilling rate at which tokens(credits) are accumulated

ρf

= Lmaxf

(1+Jf

/ BAGf

)

σ

is the bucket size

σf

= Lmaxf

/ BAGf

Lmax

f

: the maximum packet bit length

BAGf

:

the bandwidth allocation gap

Jf

: the maximum admissible jitter

Background: BAG

•

How to affect QoS? --

Bandwidth•

BAG (bandwidth allocation gap)

•

Primary bandwidth control scheme•

Minimum time interval between two successive frames

Documents

Real-Time Switched Networks - comp.polyu.edu.hkcsqwang/research/RealTimeSwitchedNet… · Real-Time Switched Networks Qixin Wang Department of Computing. The Hong Kong Polytechnic