9
I have a good shard key now what? David Murphy , Mongo Master Lead DBA, ObjectRocket @dmurphy_data @objectrocket

Lightning Talk: MongoDB Sharding

  • Upload
    mongodb

  • View
    488

  • Download
    11

Embed Size (px)

DESCRIPTION

 

Citation preview

  • 1. I have a good shard key now what?David Murphy , Mongo MasterLead DBA, ObjectRocket@dmurphy_data @objectrocket

2. Background 16 yrs in databases, development, & system engineering MySQL and NoSQL Architect for largest game company Lead DBA @ ObjectRocket Mongo Master with a focus on sharding, chunks, andscaling mongo beyond normal means. 3. The balancer is great but you can also perform some proactive checks to predict issues.Understanding the reasons why not all chunks are equalCounting Chunks per shard without sh.status()Finding out how many chunkMoves or splits occur at a timeGetting the size of all chunks for a collection 4. Sharding, the balancer, and chunks 5. Ways to Count ChunksMany functions for this: sh.status() db.collection.stats()Simple query for counting chunks: db.chunks.count({ns:"database.collection"});Why I prefer using Aggregation: Faster as you only look at one type of data Provides chunk count per shard Can programmatically use the output 6. Tracking moveChunks and SplitsThe config.changelog tracks chunk changes.The types of changes to chunks are split moveChunk moveChunk.start moveChunk.from moveChunk.to moveChunk.commit*Bold means I track with aggregations by hour/day 7. Sizing ChunksdataSize command - Very expensiveScans all documents to be 100% accuratefind().count() * avgObjSize on chunk rangesRough size estimate based on statsChunkHunter.pyNew script from ObjectRocket to:a) copy chunks meta data to new collectionb) scan each chunk there with dataSize or countc) output summary 8. Anatomy of a ChunkHunter 9. Contact@dmurphy_data@[email protected]://www.objectrocket.comWE ARE HIRING! (DBA,DEVOPS, and more)https://www.objectrocket.com/careersGitHub for ChunkHunter (later today):https://github.com/objectrocket/Utils/blob/master/DBA/ChunkHunter.py