Patterns for Scalability in Windows Azure Applications (Alex Mang)

Premium community conference on Microsoft technologies itcampro@ itcamp14#

Patterns for Scalability in Microsoft Azure Applications

Alex Mang

http://alexmang.ro

@mangalexandru

http://alexmang.ro/


Huge thanks to our sponsors & partners!



• Alex Mang

• CEO @ KeyTicket Solutions

–Microsoft BizSpark Plus

• Azure Advisor

• MS, MCP, MCSD

Speaker.Bio.ToString()


• Common requirements for cloud apps:–Availability

–Data management

–Design and implementation

–Messaging

–Management and monitoring

–Performance and scalability

–Resiliency

– Security

What are patterns?


• Performance

– ‘indication of responsiveness of a system to execute any action within a given time interval’

• Scalability

– ‘ability of a system to handle increases in load without impact on performance’

Performance and Scalability Patterns

= ???

= ???


Performance and Scalability Patterns

Cache-aside

Competing consumersCQRS

Event sourcing

Index table

Materialized view

Priority Queue

Queue based load leveling

Sharding

Static content

Throttling


QUEUE-BASED LOAD LEVELING PATTERN


Queue-Based Load Leveling Pattern (Context)

• Cloud app require external services

• High load on cloud app means high load on services

• External services may be less scalable

• High load on cloud app could result in failing external services

• Possible self-throttling


Queue-Based Load Leveling Pattern (Solution)

• Force the processing of request inside a queue

• Thus, load-leveled service requests

• Additional advantage: queue also works as a buffer


Queue-Based Load Leveling Pattern (Solution)


Queue-Based Load Leveling Pattern (Consid.)

• Make sure services are scaled correctly

• Task senders may wait service replies


COMPETING CONSUMERS PATTERN


• Asynchronously process requests

• The number of concurrent requests over time varies

• The time required for processing varies

Competing Consumers Pattern (Context)


Competing Consumers Pattern (Solution)


Competing Consumers Pattern (Considerations)

• Ordering

• Poisoned messages

• Result handling

• Message queue scaling

• Reliability


PRIORITY QUEUE PATTERN


• Asynchronous processing via queues

–Queues can’t sort messages (most of the times)

• Push notification (15K) vs. e-mail (15K)

Priority Queue Pattern (Context)


• Queues with different priorities

• Consumers based on queue priority

Priority Queue Pattern (Solution)


Priority Queue Pattern (Solution)


Priority Queue Pattern (Considerations)

• What is ‘high priority’ vs ‘low priority’

• (Single pool consumers) high first, low after

• (Single pool consumers) elevate old messages

• Multiple queues work best for less priority definitions


Priority Queue Pattern (When To Use)

Push first, send after example

Multi-tenant applications

Different SLAs / customers


Priority Queue Pattern (When NOT To Use)

Messages have similar priority

No burst of messages in the queue ever exists

Costs must be kept down


DEMO

PRIORITY QUEUE PATTERN


THROTTLING PATTERN


Throttling Pattern (Context)

• Cloud application load varies

–# active users (mostly during work hours)

– Type of activities (analysis at end of month)

• Sudden unanticipated bursts

–Poor performance

– Eventual failures

• SLA requirements


Throttling Pattern (Solution)

• Auto-scaling, for starters…

• Define resource soft limits

• Monitor resource usage

• Throttle users –Based on business impact (tiers / plans)

–Based on users’ concurrent requests

• Degrade functionality

• Load-leveling pattern / priority-queue pattern






Throttling Pattern (Considerations)

• Architectural decision: consider it while designing

• Quick monitoring technique

• Notify accordingly


Throttling Pattern (When To Use)

• Meet SLA

• Prevent single user monopolize everything

• Gracefully handle activity bursts

• Control costs by limiting max. resource usage


DEMO

THROTTLING PATTERN


Throttling Pattern (Demo)


MATERIALIZED VIEW PATTERN


• Most developers think about how data is stored

• In NoSQL, we usually store everything in a single entity

• In SQL, we have size constraints

• End-up in:

–Performance impact

–High prices

Materialized View Pattern (Context)


• Generate views in advance, containing data on a per-requirement basis

• Only contain data required by query

• Include current values of calculated columns or data items

• May be optimized for a single query

• Updated a.s.a.p. (schedule / triggered)

Materialized View Pattern (Solution)


Materialized View Pattern (Solution)


Materialized View Pattern (When To Use)

Queries are complex

Data difficult to query directly

Temporary views dramatically improve perf.

Temporary views act as DTOs for UI, reporting etc.

Data store not always available

Security or privacy reasons


Materialized View Pattern (When NOT To Use)

Data source is simple to query

Data changes quickly

Consistency is most important


DEMO

MATERIALIZED VIEW PATTERN


Materialized View Pattern (Demo)


COMMAND AND QUERY RESPONSIBILITY SEGREGATION PATTERN


CQRS Pattern (Context)

• Traditional CRUD system do everything over the same data store

• Typically, same entity for DB <--> UI <--> DB





• Many concurrent connections FAIL

• Complex business logic FAIL

• Too much data passed around

• Performance impact @ high load, due to complex querying

• Security issues may arise


CQRS Pattern (Solution)

• Segregate read (queries) from write (commands)

• Models for querying and for updating are different

• Possible to access same store, better not to


CQRS Pattern (Solution)


CQRS Pattern (Considerations)

• Additional complexity

• Consistency considerations

• CQRS for parts of the application

• Use in conjuction with Event Source pattern


CQRS Pattern (When To Use)

Multiple concurrent operations

Already familiar with Domain-Driven-Design techniques

Read performance ≠ write performance

Different teams (read vs. write)

App. lifecycle: model update, business logic update


CQRS Pattern (When NOT To Use)

Simple business rules

Simple CRUD-style UI are enough

Across the whole system


SHARDING PATTERN


Sharding Pattern (Context)

• Why scale out compute, and not scale out data?

• Must scale out data because:

– Storage limitations

–Concurrent requests

–Network bandwidth

–Geography


Sharding Pattern (Solution)

• Horizontal partitions of data – (a.k.a. shards)

• Same schema, different data

• Runs on its own server

• Benefits:– Scale out data service

–Use commodity hardware

–Better performance

–Closely located geographically



• Lookup strategy• Range strategy• Hash strategy


Strategy Advantages Considerations

Lookup • More control• Easy shard rebalance

• Shard lookup may create additional overhead

Range • Easy to implement• Works well on range

queries• Easy management

• Suboptimal balance• Shard rebalance is

difficult

Hash • Best balance• Request routing directly via

hashing alg.

• Calculating hash may create additional overhead

• Rebalance is difficult



THANK YOU!


Q & A

Technology

Patterns for Scalability in Windows Azure Applications (Alex Mang)