Upload
mongodb
View
1.691
Download
3
Embed Size (px)
DESCRIPTION
Are you in the process of evaluating or migrating to MongoDB? We will cover key aspects of migrating to MongoDB from a RDBMS, including Schema design, Indexing strategies, Data migration approaches as your implementation reaches various SDLC stages, Achieving operational agility through MongoDB Management Services (MMS).
Citation preview
Migrating to MongoDBBest Practices
Muthu ChinnasamySenior Solutions Architect
Agenda
• Project Team
• Schema Design
• Application Integration
• Data Migration Options
• Ops considerations
Why MongoDB
• Rich documents
• Promote business agility
• Achieve higher scalability
• Lower budget strain compared to RDBMS
RDBMS to MongoDB – Success stories
Project Team
Organizing for Success - Stakeholders
• Key to success: Involve all key stakeholders for the applicationo Line of businesso Developerso Data Architectso DBAso Systems Administratorso Security
Organizing for Success – Project Charter
• Develop project chartero Define business and technical objectiveso Define timeliness and responsibilitieso Monitor progress and address any issues
Organizing for Success – Help needed?
• Partner services and resources available from MongoDBo Community supporto Build skills and proficiency through web based trainingo Support and consulting services
Schema Design
Definitions
RDBMS MongoDB
Table Collection
Row Document
Column Field
Index Index
JOIN Embedded Document or Reference
Document Model Benefits
RDBMS MongoDB
{
_id : ObjectId("4c4ba5e5e8aabf3"),
employee_name: "Dunham, Justin",
department : "Marketing",
title : "Product Manager, Web",
report_up: "Neray, Graham",
pay_band: “C",
benefits : [
{ type : "Health",
plan : "PPO Plus" },
{ type : "Dental",
plan : "Standard" }
]
}
Schema Design – Blogging Platform
Schema Design - Indexing
• Compound Indexes• Unique indexes• Array Indexes• Text Indexes• Geospatial indexes• Sparse Indexes
Schema Design – For more details
• Afternoon session: Data modeling deep dive
• Google: "6 rules of thumb for MongoDB schema design"
• Google: "MongoDB compound index optimization"
Application Integration
Morphia
MEAN Stack
Java Python PerlRuby
MongoDB API is implemented as methods
Drivers & Ecosystem
Developer Efficiency
Object-Relational impedance?
Alter 2TB table to modify a column?
MongoDB APIs are classes and packages
Dynamic schema
Rigid schema
Modify code to use MongoDB APIs
RDBMS
MongoDB
Data Migration
Data Migration
Source DatabaseSource
Database
ApplicationApplication
Data Migration – Can you have downtime?
Source DatabaseSource
DatabaseMasterMaster ExportingExporting
ImportingImporting MasterMaster
AvailableAvailable DegradedDegraded DownDown AvailableAvailableApplication View
Time
T1
T2
T3
Data Migration – mongoimport$ mongoimport --db test --collection customers < customers.json connected to: 127.0.0.12014-11-26T08:36:47.509-0800 imported 1000 objects$ mongoMongoDB shell version: 2.6.5connecting to: test> db.customers.findOne(){
"_id" : 363862536,"first_name" : "Landon”,"last_name" : "Moore","created_date" : ISODate("2010-03-02T22:48:35Z"),"is_active" : true,"phone" : [
{ "type" : "Work”, "number" : "683-560-1311” },{ "type" : "Other”, "number" : "437-849-4219” }
],"address" : {
"street_number" : 14,"street" : "Granite","street_type" : "Way","city" : "New Jersey","zip_code" : 96881
},"company" : ”Example"
}
Data Migration – ETL tools
Source DatabaseSource Database ETLETL
Data Migration – Hadoop
Source DatabaseSource Database
jobjobjobjob
jobjobjobjob
App Driven Migration
Source DatabaseSource Database
ApplicationApplication
Data Migration - Options
Source DatabaseSource Database
SnapshotSnapshot
Continuous Sync
Batch Migration
Batch Migration
ApplicationApplication
Application Managed
Case Study
Case Study
Uses MongoDB to safeguard over 6 billion images served to millions of customers
Problem Why MongoDB Results
• 6B images, 20TB of data
• Brittle code base on top of Oracle database – hard to scale, add features
• High SW and HW costs
• JSON-based data model
• Agile, high performance, scalable
• Alignment with Shutterfly’s services-based architecture
• 80% cost reduction
• 900% performance improvement
• Faster time-to-market
• Dev. cycles in weeks vs. tens of months
• Meta data stored in XML Blobs• App responsible for content of blob
Shutterfly – Original Data store
OracleOracle
Photo ID XML Blob
1 <xml><meta-data>…</xml>
2 <xml><meta-data>…</xml>
3 <xml><meta-data>…</xml>
Schema Migration – Initial
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
Schema Migration – Phase 1
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
Schema Migration – Phase 2
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "votes" : { 000000000:1, 111111111:1, 222222222:1 }}
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "votes" : { 000000000:1, 111111111:1, 222222222:1 }}
Data Migration – Application driven
1. Request for photo
2. Try to read from MongoDB
3. If cache miss, read from Oracle
4. Translate document & write to MongoDB
5. Return to clientSource DatabaseSource
Database
ApplicationApplication
11
2233
44
55
Ops Considerations
Replica Sets - No downtime maintenances
Replica set provides ops agility & HA
•Database upgrades
•Hardware swaps/maintenance
•Maintenance operations
•Automatic failover
MongoDB Management Service (MMS)
Provision
Upgrade
Scale
Continuous Backup
Point-in-Time Recovery
Alerts
1. Automation 2. Backups 3. Monitoring
Cloud Managed MongoDB
36
Defense in Depth Security Architecture
Authentication•Database •LDAP •Kerberos•x.509 Certificates
Authorization•Built-in Roles•User-Defined Roles•Field-Level Redaction
Auditing•Admin operations•Queries
Encryption•Network : SSL•Disk: Partner solutions
Help available from MongoDBMongoDB Enterprise AdvancedThe best way to run MongoDB in your data center
MongoDB Management Service (MMS)The easiest way to run MongoDB in the cloud
Production SupportIn production and under control
Development SupportLet’s get you running
ConsultingWe solve problems
TrainingGet your teams up to speed.