Upload
kelly-kirk
View
24
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Sam Ghods VP Technology. The simplest way for businesses to share and access data, anywhere. 12 months ago… 4 00M files 4 0M folders One MySQL database. Need to scale!. NoSQL!. “NoSQL” goodies. Easy to scale Just add machines! Sharding handled by the database - PowerPoint PPT Presentation
Citation preview
“NoSQL” goodies
• Easy to scale– Just add machines!– Sharding handled by the database– Linearly scales, shared-nothing, no serious SPOF
• Fast, fairly simple CRUD operations• Schema-less
If you use a NoSQL store, but need any advanced features in your data
store, you have to rebuild them from scratch yourself.
Inter-Row ConsistencyFile trees must remain consistent
• Folder A– Test File– Test File
• Solution: unique index• Solution: lock folder A
Inter-Row ConsistencyModify data structure and log event
• Folder A– Test File 2
• Solution: Use transactions rename event
Inter-Row ConsistencyDenormalizations
• Folder A– Test File 1
• Solution: transactions
delete this must be deleted too
Indexes
• Indexes are way more awesome than people give them credit for– Guaranteed to be consistent– Extremely fast– Data locality – Only access and pull the data you
need– No maintenance required except initial ALTER cost
• SELECT files ORDER BY name (or updated time, or size, etc…)
Tools
• How do you know what’s happening in your data store?– SHOW FULL PROCESSLIST– innotop
• Benchmarking tools– mysqlslap
• pt-query-digest– github.com/box/anemometer
Maturity/Reliability
• Biggest companies in the world have been using MySQL for primary data storage for over a decade– Facebook, Google, Twitter, every other
company ever• When you’re dealing with the crown jewels
of your company, you can’t experiment
HBase
• Currently using it as a massive event-propagation store (which can be recreated from MySQL data)
• Started a 3-person task force to learn and productionalize it
• Considering moving more to it in the future but likely need few more years of production experience
Final Thoughts
• Don’t choose a database just because “it scales”
• “Wade, don’t jump into new technologies.”• If you go with new technology, be aware that
crazy things might happen• Make sure you’re not rebuilding MySQL