1
Phoenix Project:Phoenix Project:Making Applications Persistent Making Applications Persistent
David Lomet, Roger Barga
Microsoft Research, Redmond
Gerhard Weikum
University of Saarlands
Mark Tuttle
Compaq Cambridge Research Lab
Interns: Sanjay Agrawal, Thomas Baby,
Sirish Chandrasekaran, Stylianos Paparizos
2
Ex: An E-Commerce ServerEx: An E-Commerce Server
TP App Server usestransactions to accessthe DBMSDBMS
Web Server
TP App Server…
IP Sprayer
Web Server
TP App Server
Internet
3
ProblemProblem Half+ of TP system is outside the transactionHalf+ of TP system is outside the transaction
Much of it is distributed computingMuch of it is distributed computing Some is TP-specific (e.g. persistent queues, workflow)Some is TP-specific (e.g. persistent queues, workflow)
CurrentCurrent “Fact of Life”: “Fact of Life”: after a crashafter a crash DatabasesDatabases recover – to last committed transaction recover – to last committed transaction But But applicationsapplications “disappear” “disappear”
They crash themselves and cannot recoverThey crash themselves and cannot recover Or the DBMS crashes, and they cannot continue executionOr the DBMS crashes, and they cannot continue execution
Availability requires programmer actionsAvailability requires programmer actions ““Stateless” applicationsStateless” applications Explicit storing of state in queues or databaseExplicit storing of state in queues or database
4
Network
Traditional TP ApproachTraditional TP ApproachStateless ComponentsStateless Components
DCOM
Client Machine
ComponentComponent
Server Process
Request Q
Each method call is a transaction1.Read state from a transactional
queue 2.Invoke the method3.Write state to the transactional queue
Each method call is a transaction1.Read state from a transactional
queue 2.Invoke the method3.Write state to the transactional queue
Cmpnt1Cmpnt1 Reply Q
DCOM
Cmpnt1Cmpnt1
ComponentComponent
Server Process
5
Stateful Applications NowStateful Applications Now
Local Servicesand Data
ComponentBased
App Code
Network
Application programmers tend to write stateful applications that retain state across method calls
Application programmers tend to write stateful applications that retain state across method calls
DCOM
ComponentComponent
ComponentComponent
ComponentComponent
DCOM
Client Machine
Server Machine
Server MachineServer Machine
Server MachineServer Machine
The problem with stateful apps is the risk of losing state as a result of a failure
6
Transparent PersistenceTransparent Persistence
Programmer simply writes his programProgrammer simply writes his program System ensures persistence– System ensures persistence– HOW?HOW?
Virtual ComponentsVirtual Components Virtual component mapped to physicalVirtual component mapped to physical
Mapping can be changedMapping can be changed Extract and restore state of virtual componentExtract and restore state of virtual component
Into physical componentInto physical component Under system discretion for Under system discretion for scalabilityscalability As As recovery techniquerecovery technique after system crash after system crash
Efficiency byEfficiency by Replaying to rebuild stateReplaying to rebuild state Not by simply frequently saving entire component stateNot by simply frequently saving entire component state Rationale: modest length of execution histories Rationale: modest length of execution histories
7
Important Benefits forImportant Benefits for Application programmerApplication programmer
Program does not have to deal with crashesProgram does not have to deal with crashes Greatly simplifying programGreatly simplifying program
Natural stateful applicationsNatural stateful applications More focus on correctness/functionalityMore focus on correctness/functionality
Operational application Operational application No manual restore of system of application stateNo manual restore of system of application state
Hence shorter outagesHence shorter outages Users/customersUsers/customers
Less lost work, fewer frustrationsLess lost work, fewer frustrations
BusinessBusiness Increased availability means Increased availability means
Less lost revenue (e.g. Ebay lost $1M/hr in revenue)Less lost revenue (e.g. Ebay lost $1M/hr in revenue) Happier customersHappier customers
8
Outline of TalkOutline of Talk BackgroundBackground
Application Availability ProblemApplication Availability Problem Phoenix goal: transparent application persistencePhoenix goal: transparent application persistence
Persistent Stateful ApplicationsPersistent Stateful Applications Fundamentals of multi-component protocolsFundamentals of multi-component protocols
Three forms of componentsThree forms of components Three interaction contractsThree interaction contracts
Two sub-projectsTwo sub-projects Phoenix/COM for persistent componentsPhoenix/COM for persistent components Phoenix/ODBC for transactional componentsPhoenix/ODBC for transactional components
External components need no system supportExternal components need no system support
9
Persistent Stateful Persistent Stateful ApplicationsApplications
For general multi-tier applications
including web/app servers
and database backends
10
Component TypesComponent Types Persistent(Persistent(PCOMPCOM)): : e.g.e.g. web/application serversweb/application servers
State survives system failuresState survives system failures Phoenix/COMPhoenix/COM component acts as PCOM component acts as PCOM
Via logging interactions with other system componentsVia logging interactions with other system components
Transactional(Transactional(TCOMTCOM)): : e.g.e.g. SQL DB sessionsSQL DB sessions Transactions can abort- aborted state may “die”Transactions can abort- aborted state may “die”
But But committedcommitted states, messages survive system failures states, messages survive system failures Phoenix/ODBCPhoenix/ODBC session acts as TCOM session acts as TCOM
Via Via server reply loggingserver reply logging
External(External(XCOMXCOM)): : e.g.e.g. users, autonomous sitesusers, autonomous sites,,etc.etc. Not under our control (Not under our control (need do nothingneed do nothing)) Prompt logging by PCOM limits exposurePrompt logging by PCOM limits exposure
Only failure during interaction is “unmasked”Only failure during interaction is “unmasked”
11
Interaction ContractsInteraction Contracts Committed interaction contractCommitted interaction contract((CICCIC) )
PPCOMCOMPCOMPCOM
Guarantees that interaction persists across failuresGuarantees that interaction persists across failures Transaction’l interaction contractTransaction’l interaction contract ( (TICTIC) )
PPCOMCOMTCOMTCOM Permits transactional component to abortPermits transactional component to abort But final commit and its result msg is persistent But final commit and its result msg is persistent
External interaction contractExternal interaction contract ( (XICXIC) ) PPCOMCOMXCOMXCOM Permits interaction with external world, which does not Permits interaction with external world, which does not
play by our rules play by our rules Immediate forced logging by PCOM to limit failure Immediate forced logging by PCOM to limit failure
windowwindow
12
System SchematicSystem Schematic Replay needs captured at component interactionsReplay needs captured at component interactions
Actions depend on component types involvedActions depend on component types involved Interaction contracts provide persistence guaranteesInteraction contracts provide persistence guarantees
Forms of Components and Interaction ContractsForms of Components and Interaction Contracts Persistent [PCOM]-Persistent [PCOM]- pervasive within system pervasive within system External [XCOM]-External [XCOM]- at system edges (can initiate work) at system edges (can initiate work) Transactional [TCOM]-Transactional [TCOM]- at system edges (receives work) at system edges (receives work)
CICXIC
TICClient
“SQLServer”
AppServer
Web Server
13
Phoenix/COM: Phoenix/COM:
Persistent Components– Persistent Components–
for multi-tier applicationsfor multi-tier applications
TransparentlyTransparently
14
Phoenix/COM ApproachPhoenix/COM Approach
Persistence as a runtime servicePersistence as a runtime service Component Component registeredregistered as persistent [I.e. a PCOM] as persistent [I.e. a PCOM]
Similar to COM+ transaction property registration Similar to COM+ transaction property registration State persistence is transparent and recovery automaticState persistence is transparent and recovery automatic
Phoenix/COM runtime provides Phoenix/COM runtime provides interaction contractsinteraction contracts Programmers write application logic Programmers write application logic onlyonly..
Via application replayVia application replay Piece-wise deterministic assumptionPiece-wise deterministic assumption
Between non-deterministic events, program execution is Between non-deterministic events, program execution is repeatablerepeatable
Non-determinism removed by logging, e.g.Non-determinism removed by logging, e.g. User interactions, Procedure calls, Reading sys clock, etc. User interactions, Procedure calls, Reading sys clock, etc.
Efficiency by minimizing log forces & logged dataEfficiency by minimizing log forces & logged data Via efficient interaction protocols & “logical” log operationsVia efficient interaction protocols & “logical” log operations
15
Recovery InfrastructureRecovery Infrastructure
Phoenix/COM Phoenix/COM
LogLog interactions, checkpoint state IF IF REQUIREDREQUIRED to fulfill CIC requirements
Log
Com Object App 1
Com ObjectApp 2
CIC
App see COM interface
CIC: keeps App’s recoverable
16
Phoenix/COM ElementsPhoenix/COM Elements Virtual ComponentsVirtual Components
Isolate application from physical component failureIsolate application from physical component failure Permit another component to “continue” executionPermit another component to “continue” execution
Normal operation: interception and loggingNormal operation: interception and logging Log activation, call, etc. to capture object stateLog activation, call, etc. to capture object state
Recovery: detect failure & recoverRecovery: detect failure & recover Install virtual object state in new physical objectInstall virtual object state in new physical object Error handler reconnects to recovered virtual objectError handler reconnects to recovered virtual object
Implementation on Implementation on contextcontext infrastructure infrastructure Feature of COM+, .NET CLR, COMAppsFeature of COM+, .NET CLR, COMApps
17
Committed Interaction ContractCommitted Interaction ContractPhoenix/Com ResponsibilityPhoenix/Com Responsibility
Sender guarantees:Sender guarantees: S1: Persistent state at least up to sendS1: Persistent state at least up to send S2: Persistent messageS2: Persistent message
S2A: Repeated transmission retry S2A: Repeated transmission retry S2B: Transmission on requestS2B: Transmission on request
Receiver guarantees:Receiver guarantees: Duplicate message eliminationDuplicate message elimination Persistent state with msg receive: S2A releasedPersistent state with msg receive: S2A released
Subsequently, receiver must request messageSubsequently, receiver must request message Message content stable: S2B releasedMessage content stable: S2B released
Sender can now discard message contentsSender can now discard message contents
18
Infrastructure:Infrastructure: ContextContext Boundary definitionBoundary definition around components around components
Cross-context boundary events are raisedCross-context boundary events are raised Permitting cross context call interceptionPermitting cross context call interception
Every component lives in a contextEvery component lives in a context Subset of an apartmentSubset of an apartment
Context
Component
Context Properties
Context Properties
Contexts specify interception Contexts specify interception
19
Infrastructure:Infrastructure: PolicyPolicy HandlerHandler for context boundary events for context boundary events
Cancel or repeat the callCancel or repeat the call Examine the call state (IID, method, Examine the call state (IID, method,
parameters, etc.) parameters, etc.) Take action as requiredTake action as required
Example: Synchronization PolicyExample: Synchronization Policy onEnter:onEnter: acquire mutex acquire mutex onLeave:onLeave: release mutex release mutex
Policies enable runtime extension… as Policies enable runtime extension… as
interception handlersinterception handlers
20
Context CContext C11 Context CContext C22
ClientSide
Policies
ServerSide Policies
Call,Return
Enter,Leave
Pass Buffer
Pass Buffer
ProxyProxy StubStub
A B
Infrastructure:Infrastructure: SwitcherSwitcher
SW
ITC
HER
The The Interception MechanismInterception Mechanism
21
Interception mechanismInterception mechanism for method for method executionexecution
Detects errors during processingDetects errors during processing
Resolves reference to physical OIDResolves reference to physical OID
Calls a registered error handlerIf R_OK it retries method
call/response
API to rebind “virtual” object To new physical OIDEnhanced switcher detects errors, provides Enhanced switcher detects errors, provides
automatic recovery, and automatic recovery, and virtualizes componentsvirtualizes components……
Role for SwitcherRole for Switcher
22
Infrastructure SummaryInfrastructure SummaryHow Phoenix/COM leverages it…How Phoenix/COM leverages it… ContextContext
Registers interception boundaryRegisters interception boundary
PoliciesPolicies Log Component CreationLog Component Creation Log Method Call/ResponseLog Method Call/Response
SwitcherSwitcher Detect Errors from Component FailureDetect Errors from Component Failure Remap Virtual Component ReferencesRemap Virtual Component References Retry Method Call/ResponseRetry Method Call/Response
25
Phoenix/ODBCPhoenix/ODBC
Transactional components for Database servers
26
Phoenix/ODBC ObjectivesPhoenix/ODBC Objectives Persistent Database SessionsPersistent Database Sessions
Mask network and SQL Server failuresMask network and SQL Server failures Leverage database recovery mechanisms Leverage database recovery mechanisms Recovery is Recovery is transparent to clienttransparent to client Originally:stand-alone session persistenceOriginally:stand-alone session persistence
No protection from client crashesNo protection from client crashes Realize Transactional ComponentsRealize Transactional Components
Within Phoenix/COM Within Phoenix/COM Phoenix/ODBC wrapped DB becomes TCOMPhoenix/ODBC wrapped DB becomes TCOM Client becomes a PCOMClient becomes a PCOM
Component registered as transactional (TCOM)Component registered as transactional (TCOM)
27
ODBC Database SessionODBC Database Session Allocate ODBC handlesAllocate ODBC handles Connect to SQL ServerConnect to SQL Server Set session attributesSet session attributes Send SQL statements to SQL ServerSend SQL statements to SQL Server Fetch records from result setFetch records from result set
Client
SQLServer
Interconnect
SQL
28
Phoenix/ODBC ElementsPhoenix/ODBC Elements Phoenix-enabled ODBC connectionPhoenix-enabled ODBC connection
Virtual session & connection from client to serverVirtual session & connection from client to server
Intercept and Intercept and “wrap”“wrap” Log actions that affect ODBC session contextLog actions that affect ODBC session context
Connection, logon, database and userid, set optionsConnection, logon, database and userid, set options Make temporary tables/cursors on server persistentMake temporary tables/cursors on server persistent Rewrite SQL statement(s) to Rewrite SQL statement(s) to persist SQL resultspersist SQL results Receive server responses and error messagesReceive server responses and error messages Fetch results from persistent result tablesFetch results from persistent result tables
Recover sessionRecover session Remapping virtual ODBC session to new real sessionRemapping virtual ODBC session to new real session
29
Phoenix/ODBC InfrastructurePhoenix/ODBC Infrastructure Currently implemented in ODBC Driver ManagerCurrently implemented in ODBC Driver Manager
Which wraps native ODBC driversWhich wraps native ODBC drivers NoNo changes required for ODBC driver(s) changes required for ODBC driver(s) NoNo changes required for database systems changes required for database systems NoNo special client programming special client programming
SQLServer
Oracle
Informix
Client
SQL SRVR 3.0
Oracle 3.0
SQL SRVR 2.5
Informix 3.0
Phoenix ODBCDriver
Manager TCOMTCOM
30
Transaction Interaction ContractTransaction Interaction Contract Phoenix/ODBC ResponsibilityPhoenix/ODBC Responsibility
Before PCOM issued “Commit”Before PCOM issued “Commit” TCOM can abort, forgetting everythingTCOM can abort, forgetting everything PCOM can forget some/all state/messagesPCOM can forget some/all state/messages
At PCOM issued “Commit”At PCOM issued “Commit” PCOM sender promises are CIC promisesPCOM sender promises are CIC promises CommittingCommitting TCOM receiver guarantees: TCOM receiver guarantees:
Duplicate message eliminationDuplicate message elimination Persistent state with message receive/commit:Persistent state with message receive/commit:
S2A, “S2B” released when TCOM commits & sends replyS2A, “S2B” released when TCOM commits & sends reply PCOM can discard msg contents when reply receivedPCOM can discard msg contents when reply received
Persistent reply messagePersistent reply message A persistent result set committed by transactionA persistent result set committed by transaction
31
Default Result Set (stable reply)Default Result Set (stable reply)
ClientSQL
ServerODBCSELECT … WHERE FALSECREATE TABLE phtableSELECT … INTO phtableSELECT * FROM phtable
Determine format of result setDetermine format of result set
Create persistent table on SQL ServerCreate persistent table on SQL Server
Rewrite SQL stmt to load persistent tableRewrite SQL stmt to load persistent table
Open access to persistent tableOpen access to persistent table
Select cust_name,cust_address from customers
where acct_balance > max_allowed_balance
32
Database Session RecoveryDatabase Session Recovery Intercept errors and detect Intercept errors and detect possiblepossible system failure system failure Ping database server (interval and timeout)Ping database server (interval and timeout) Reconnect to database serverReconnect to database server
Determines if server really failedDetermines if server really failed Verifies persistent structures were recoveredVerifies persistent structures were recovered
Reinstall session context from client logReinstall session context from client log Advance through result set(s) to last result seenAdvance through result set(s) to last result seen
Application still sees aborted transactionsApplication still sees aborted transactions
SQLServer
ODBC Session Context
33
SummarySummary Phoenix goal:Phoenix goal: transparent app persistence transparent app persistence
No application programmingNo application programming Higher availability and scalabilityHigher availability and scalability
PCOMPCOM by Phoenix/COM via by Phoenix/COM via Logging non-deterministic eventsLogging non-deterministic events
By infrastructure via wrappingBy infrastructure via wrapping Component replay if system crashesComponent replay if system crashes
TCOM TCOM by Phoenix/ODBC viaby Phoenix/ODBC via Inducing DBMS logging for database recoveryInducing DBMS logging for database recovery
Via wrapping ODBC and modifying SQL requestsVia wrapping ODBC and modifying SQL requests Inducing DBMS recovery for session stateInducing DBMS recovery for session state
34
Phoenix/COM Project URLPhoenix/COM Project URL
1.1. Barga, Lomet, Weikum: Recovery Guarantees for General Barga, Lomet, Weikum: Recovery Guarantees for General Multi-tier Applications. (submitted for publication) Multi-tier Applications. (submitted for publication)
2.2. Barga, Lomet: Phoenix/COM: Barga, Lomet: Phoenix/COM: Enabling persistent component-Enabling persistent component-based applications. (Internal white paper)based applications. (Internal white paper)
http://www.research.microsoft.com/research/db/phoenixhttp://www.research.microsoft.com/research/db/phoenix
Phoenix/COM ReferencesPhoenix/COM References
35
Phoenix/ODBC ReferencesPhoenix/ODBC References
1.1. Barga, Lomet: Measuring and Optimizing a System for Barga, Lomet: Measuring and Optimizing a System for Persistent Database Sessions. Persistent Database Sessions. Proceedings of ICDE,Proceedings of ICDE, Heidelberg, Germany (April 2001) Heidelberg, Germany (April 2001)
2.2. Barga, Lomet, Baby, Agrawal: Persistent Client-Server Barga, Lomet, Baby, Agrawal: Persistent Client-Server Database Sessions. Database Sessions. Proceedings of EDBT Conference, Proceedings of EDBT Conference, Lake Lake Constance, Germany (Mar. 2000) 462-477. Constance, Germany (Mar. 2000) 462-477.
3.3. Barga, Lomet:Barga, Lomet: Phoenix: Making Applications Robust. (demo Phoenix: Making Applications Robust. (demo paper) paper) Proceedings of ACM SIGMOD ConferenceProceedings of ACM SIGMOD Conference, , Philadelphia, PA (June, 1999) 562-564.Philadelphia, PA (June, 1999) 562-564.
4.4. Lomet, Weikum: Efficient Transparent Application Recovery Lomet, Weikum: Efficient Transparent Application Recovery in Client-Server Information Systems. (Best Paper Award) in Client-Server Information Systems. (Best Paper Award) Proceedings of 1998 ACM SIGMOD ConferenceProceedings of 1998 ACM SIGMOD Conference, Seattle, WA , Seattle, WA (June 1998) 460-471. (June 1998) 460-471.