Dynamo: Amazons Highly Available Key-value StoreDeCandia, Hastorun, Jampani, Kakulapati, Lakshman, Pilchin, Sivasubramanian, Vosshall, VogelsPRESENTED BY: KIMIISA OSHIKOJI

OUTLINEAmazonDynamoArchitecturePerformance

AMAZONHuge InfrastructureCustomer oriented businessReliability is key

DYNAMOData storage systemFlexibleAutomated addition and removal of storage nodes

DYNAMO-REQUIREMENTS

RequirementEffectQuery ModelRead and write operations that are associated with a keyACID PropertiesProperties for database transactionsEfficiencySystems must achieve latency and throughput requirementsOther AssumptionsWhat Dynamo assumes

DYNAMO-QUERY MODELKey identifies operationsOperations dont require multiple data itemsData to be stored is relatively small

DYNAMO-ACID PROPERTIES

PropertyEffectAtomicityTransactions happen or dontConsistencyTransactions consistent across statesIsolationData cannot be accessed by external operations while its in an intermediate stageDurabilityAfter transaction concluded it will never be undone

DYNAMO-EFFICINCY

Chart1

99.9

% of Requests that are Answered

Sheet1

% of Requests that are Answered

99.9th Percentile99.9

To resize chart data range, drag lower right corner of range.

DYNAMO-ASSUMPTIONSOnly used by internal Amazon systemsNo security considerationsLimited scalability

DYNAMO-SLAService Level Agreement: contract between client and service about their relationshipIn Amazon a typical client request involves over 100 services who might have dependenciesSLA are governed by 99.9th percentile

DYNAMO-DESIGNFocus on correctness of an answer rather than how quickly it can be availableEventually consistent data storeWrites can never be rejected99.9th percentileZero-hop DHT

DYNAMO-PRINCIPLES

PrincipleEffectIncremental scalabilityA storage host can be scaled without undue impact to the systemSymmetryAll nodes are the sameDecentralizationFocus on peer to peer techniquesHeterogeneityWork must be distributed according to capabilities of the nodes

ARCHITECTURE-STORAGEObjects stored with a key using:Get(key): locates object with key and returns object or list of objects with a contextPut(key, context): places an object at a replica along with the key and contextContext: metadata about object

ARCHITECTURE-HASHING

ARCHITECTURE-REPLICATIONData is replicated on N hosts (N is determined by user)Coordinator nodes replicate the data for nodes they are responsible for coordinating

ARCHITECTURE-VERSIONINGMultiple versions can existVector clock is used for version controlVector clock size issue

ARCHITECTURE-FAILURE*Failure are not automatically detected by a central node

Failure TypeDescriptionTemporary failure of nodeReplica that would have been on failed node is sent to another with a hint as to original destinationPermanent failure of nodeReplica synchronization to insure no information is lost

ARCHITECTURE-ADDING

Discovery TypeDescriptionInternalGossip based protocol which leads to eventual consistent membership listExternalSeed nodes, known by all nodes in system

PERFORMANCE-BUFFERSystem can be optimized without sacrificing the 99.9th percentileBuffer usage can decrease latency by a factor of 5 during peak traffic times

PERFORMANCE-LOAD DISTRIBUTION*Third strategy is the best in terms of balancing

Partitioning schemeDescriptionPartition by Token and T Tokens per nodeRange of nodes vary b/c of random selection of tokensPartition into equal slices and T Tokens per nodeTokens used to map values in hash space to nodesPartition into equal slices and Q/S Tokens per nodeEach node in system must always have Q/S Tokens assigned to it

QUESTIONS?

*