Как сэкономить$1 000 000
набазах данных
Денис Аникин,ТехДир Почты/Облака Mail.Ru
Why use a DBMS?
Why use a DBMS?
+ QUERIES+ ACID+ INDEXING+ TABLES+ DURABILITY
Database File
Heavy read workload
APP
DB
APP
APP APP
Overloaded
Solution: replication
APP DB Master
Update
DB Slave
Replication
Select
DB SlaveDB Slave
DB SlaveDB Slave
Heavy write workload
APP
DB
APP
APP APP
Replication does not scale writes
Solution: sharding
DB DB DB
DB DB DB
DB DB DB
Replication
Sharding
Who is the next problem?…
The boss
The boss
Technology
We are loosing money
Close down
What should we do?
Cache!
DB DB
DB DB
DB DB
Cache
App
Goodlatency
But …
Inconsistency
DB DB
DB DB
DB DB
Cache
App
InconsistencyStill have sharding
DB DB
DB DB
DB DB
Cache
App
InconsistencyCache does not optimize updates out!
DB DB
DB DB
DB DB
Cache
App
InconsistencyApp
Cache DB
Cache rebooted
Data’s lost
InconsistencyApp
CacheDB
Cache rebooted
Data’s lost
Cache- Inconsistency- Still need sharding- What else?
Cache is not a DB
File
Inconsistency
WHATTO
DO?
Smart cache
DB DB
DB DB
DB DB
SmartCache App
Goodlatency
AppSmartCache DB
Cache rebooted
OK
Well it’s better
Well it’s better, but …
Still inconsistency
Cache rebooted
Data lost
AppSmartCache DB
Still sharding
DB DB
DB DB
DB DB
SmartCache App
Remember, your boss …
… doesn’t like sharding
… because it’s way too expensive
Cache- Inconsistency- Still need sharding- No database properties- What else?
Cold start
Cold start
DB DB
DB DB
DB DB
Cache
ColdStartKillsThe
Cache
Cold start
DB
DB
DB
Cache
DB
DB
DB
DB
DB
DB
Only to warmup
cache
Cache- Inconsistency- Still need sharding- No database
properties- Cold start
How to warm it up?
How to warm it up?
Persistent cache
DiskMemory
Dump
Do you like it?
Persistent cache
DiskMemory
Dump
- IOPS- STALE DATA+ FAST WARMUP
How to fix it?
Persistent cache
DiskMemory
Log
+ IOPS+ FRESH DATA+ FAST WARMUP- BIGGER SIZE
100Mb/sec
There’s still a problem
Persistent cache
DiskMemory
Dump & Log
+ IOPS+ FRESH DATA+ FAST WARMUP+ OPTIMAL SIZE
Solved!
Cache is warmed up
Cache
Cache
Why use a DBMS at all?
Why use a DBMS at all?• Durability• Replication
Why use a DBMS at all?• Durability. What does a
DB store?• Replication
Why use a DBMS at all?• Durability. What does a
DB store? Hot & Cold data
• Replication
Temperature of data
9.9Tb100Gb
Temperature of data
9.9Tb
100Gb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
Shard
Replicate
Why do we copy a DB?
Temperature of data
9.9Tb
100Gb
Shard
Replicate
100Gb100Gb
100Gb100Gb
100Gb100Gb
The boss’s still angry
Ideas?
Why use a DBMS at all?• Durability. What does a
DB store? Well, cache is also durable now
• Replication
An idea!CACHEIS THE
PRIMARYDATA SOURCE
An idea!CACHEIS THE
PRIMARYDATA SOURCE
With replication
An idea!CACHEIS THE
PRIMARYDATA SOURCE
With replicationAnd with all other DB features
Cache
Cache
Cache is not a DB
File
Cache can be a DB!
Cache
CanBeA
DATABASE
+ CACHE+ PERSISTENCE+ ACID TRANSACTIONS+ REPLICATION+ STORED PROCEDURES
Database
Tarantool – for hot data+ CACHE+ PERSISTENCE+ ACID TRANSACTIONS+ REPLICATION+ STORED PROCEDURES (LUA)
Database
Tarantool
We’ve done it!
9.9Tb
100Gb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
9.9Tb9.9Tb
Shard
Replicate
We’ve done it!
9.9Tb
100Gb
SQL DBMS
Tarantool
Look back• Shard, replica, cache, smart
cache• Shard exists, replica back,• Persistent cache, replication,
acid. Now cache is a DB• 2 instances could be
enough
Cache. Why not?• Dependable, durable• You’re on the road to cache• No ACID, no consistency, no
durability• Leave cozy DB for cache• Tarantool helps come over• No magic, just a new tool
Tarantool in Mail.Ru
Mail.Ru Users’ Profiles
Profiles MySQLMySQL
MySQLMySQL
MySQLMySQL
16x1/8 of workload
Mail.Ru Users’ Profiles
$1.2M upfront$300K each year
128x MySQL 4x TarantoolVS
$20K upfront$5K each year
60 times!!!
Saved more than $1 000 000
Mail.Ru Email & Cloud
$100M+
Tarantool120+ instances of
The bottom line is …
Right tool for the job
Saved $1 000 000 on a feature
Summary
Questions?Денис Аникин,техдир Почты/Облака Mail.Ru
[email protected]: danikin2Whatsapp/Viber: +79165129650