Embed Size (px)
Citation preview
Architecture and Design of MySQL Powered Applications
Peter Zaitsev CEO, Percona Highload++ 2014 Moscow, Russia 31 Oct 2014
About Percona
Open Source Software for MySQL
Ecosystem
• Percona Server
• Percona XtraDB Cluster (PXC)
• Percona Xtrabackup
• Percona Toolkit
Solutions to succeed with MySQL
• Consulting
• Support
• Managed Services
2
About Presentation
What's in your Toolbox ?
What Questions you should ask ?
Common MySQL Architecture
Patterns
3
LiveJournal Architecture 10 Years Ago
4
10 Years ago we see
Load Balancers
FrontEnd vs BackEnd Web Server Separation
MemcacheD (Caching)
Global Database for Shared Data
Read Write Splitting
Sharding “Database Clusters”
5
History Lesson
MySQL is quite Mature
Battle tested Architecture Patterns
Most Problems have proven solutions
6
YOUR TOOLBOX NOW
Well… certain things have changed
7
Very Powerful Hardware
• 16GB to 512GB Lots of
Memory
• 1000 IOPS to 100.000 IOPS
• 10ms latency to 0.1ms IO Latency
Very Fast Storage
• 4 Cores to 32 Cores Many CPU
Cores
8
Reasonable Software
•MySQL
•Linux Kernel
•File Systems
•Etc..
Can use this
Hardware Pretty Well
9
Cloud
Dynamic Scalability
Throw Away Server Mentality
Agility
Less involved Operations
10
Automation
Development Testing Deployment Operations
11
New Languages and Frameworks
More Agile
Less “system” code
More Client Side Development
Less Database Dependence
ORM
12
Multiple “Clients”
Desktop Web Browsers
Mobile Web Browsers
“Apps”
API
13
Not Only “MySQL”
• Memcache, Redis
• RabbitMQ
• Cassandra, MongoDB
• Hadoop
• Column Store Engines, Vertica
Right Tools
for Right Job!
14
More HA Options
Mature MySQL Replication
Percona XtraDB Cluster, Galera
MySQL Cluster
Proprietary cloud HA
15
Ready solutions for Replication Management
MHA
PRM
Continuent Tungsten
16
Making Sharding Better
• Clustrix
• MemSQL
• ScaleDB
Not Quite MySQL
• ScaleArc, ScaleBase, Tesora
• MySQL Proxy, MaxScale, ProxySQL
Proxy Solutions
• Vitess
• JetPants
• MySQL Fabric
OpenSource Frameworks
17
ARCHITECTURE QUESTIONS
18
Basics
Make some Right decisions Early
Your first Architecture will not be perfect
Some poor choices are very expensive to fix
19
Consultants are like Lawyers
You can waste a lot of money with them
Getting right advice on the right topics will save you a lot of trouble in the end
20
Dimensions to consider
Application Scale
High Availability
Team Experience
21
Approach to Scaling
Avoid Over-Engineering
Give yourself some
runway
Do Some Capacity Planning
22
High Availability
Real High Availability is Really Expensive
Medium Availability is getting more affordable
People are often leading cause of downtime
23
Team Experience
Chose Solutions which your Team can Support
Often Developers know little about Databases
Right Development, Operational Process Is a Key
Tools to Support your Process
• Percona Toolkit, Percona Cloud Tools
• Systematically find and review new queries
24
Simplicity
• Less complexity lower costs Use fewer
components when possible
• Monitor
• Troubleshoot
Each components needs experience
to
• “Follow the Crowd”
• Unless you have a reason not to There is safety in
numbers
25
World of “Small” Applications
• Really
Majority of Applications are
Small
• Especially with support of caching
May well run on single database
instance
• Active intranet site for company with 200K employees
• Ecommerce site supporting $10M+/month in Sales
Such “Small” Applications can
be significant
26
Lets do Some Math
Modern MySQL
Instance Handle 100K
QPS
20 Queries per User
Interaction
5K User Interactions Per Second
144M User Interactions a
Day (Considering
3x peak to average)
10M Daily Active users
(with Medium
Engagement)
27
PRACTICAL CHOICES
Basic Architecture Patterns
28
Baseline
Single MySQL Instance
No Caching
No HA
No Supplemental Technologies
29
Simple High Availability
HA with DBaaS (Amazon RDS)
HA with MySQL Replication
HA with Percona XtraDB Cluster
30
What do you need to Scale ?
Reads
Writes Data Size
31
Load Management Tricks
Move the load
In Time (Doing Batch Processing
at Night)
In Space (Use Slave for Reports)
32
Queueing
Used by Most high volume architectures
Deal with Load Spikes
Increase Reliability
Scale parts of the system which need to
Many solutions
• RabbitMQ
• Redis
• Gearman
33
Scaling Reads
Replication (and Read Write Splitting)
Caching (Memcache, Redis etc)
Percona XtraDB Cluster (Read Anywhere)
34
Scaling Writes
• Using Different Servers for Different Parts of Application
Functional Partitioning
• Horizontal Partitioning over many servers Sharding
35
Scaling Data Size
•Depends on the Application and Needs
•10TB good rule of thumb
How Much We Can
Practically Store in Single
MySQL Instance
these days
36
How Is it Possible
“Online” Schema Change
MySQL 5.6
Pt-online-schema-change (Percona
Toolkit)
Fast Backup
Percona Xtrabackup
MySQL Enterprise Backup
Modern Filesystem/Volume
Managers
Fast Network
10Gbps affordable
High Performance
Storage
Flash/SSD
Compression
Innodb
TokuDB
37
Summary
MySQL is Mature. Solutions Available for Many Problems
Make few choices right in the beginning
Hardware and Software advances allow to go a long way with simple architectures
38
