The Process Group Approach to Reliable Distributed Computing

Kenneth P. Birman, Cornell University,CSdept

Presented by Constantin Serban, R.U.

Assumptions

ProblemThe traditional communication primitives are generally reliable, but exhibit problematic semantics during transient failures and system configuration changes

ApproachUse a “glue” software layer with predictable, fault-tolerant, flexible, and reliable behavior

SolutionApproach: use process groups along with group programming tools to achieve this goal

Paper Structure

1. Desirable system example2. Process group models and requirements3. Conventional communication primitives:

common pitfalls4. Failure assumptions5. Group support requirements6. Close synchrony versus virtual

synchrony7. The ISIS toolkit and ISIS based utilities8. Potential application of ISIS9. Conclusion

System example

Brokerage and trading systems are required to integrate large numbers of demanding applications, timely reaction to high volumes of information.

Goal: reliability, security, flexibility, availability, uniformity, etcInformation backplane requirements

1. Publish/subscribe model2. Naming structure3. Communication interface

1. Access restrictions 2. Selective history mechanism

Customization1. Systematic organization2. Flexible stream connection at runtime

Hierarchical structureSystem must checkpoint, replicate on

independent machines, activate backup on failure

Solution: distributed group of cooperating programs that adapt transparent to failures and recoveries

Process Group Types: Anonymous

Publish/subscriber model. Properties:Interaction exclusive through group address (no membership knowledge)Exactly once delivery to all or none of the subscribersMessage delivered in an order consistent with casual dependenciesLogging of key events for history

Process Group Types: Explicit group

Members cooperate directly, explicit membership knowledge. Additional needs:Support for group communication, addressing, failure atomicity, message delivery orderingUse of membership as inputSynchronization of shared information

Common communication primitives

Datagrams: unreliable, message loss, duplicates, out-of-order messagesRPC:reliable, sequenced message delivery. Fuzzy during failure: unable to distinguish between failure and delays or the precise moment of failureReliable data streams: outperforms RPC b/c of pipelining. Still no guarantees regarding consistent channel break and subsequent handling.

Failure Model Assumptions

Fail-Stop model:Processes or processors fail by halting, no erroneous actionsIntegration of transport layer (TCP-like) with the failure detection layer May trigger false fails on timeout

Maintain a system membership list of processes, a non-responsive process is dropped from the list, and forced to shut-down or reconnectRejoining processes are treated like completely new entities

Group Support: Addressing