Embed Size (px)
DESCRIPTION
This webinar will introduce how to build your first Java application with MongoDB by walking you through how one can build a simple location based application. The talk will cover the basics of MongoDB's document model, query language, aggregation framework and deployment architecture. New features, fixes and improvements in the latest release will also be covered.
Citation preview
High performance Highly available Easily scalable Easy to use Feature rich
Document store
©2012 Jaspersoft Corporation. Proprietary and Confidential 2
©2012 Jaspersoft Corporation. Proprietary and Confidential
3
Data Model
A Mongo system holds a set of databases A database holds a set of collections A collection holds a set of documents A document is a set of fields A field is a key-value pair A key is a name (string) A value is a
basic type like string, integer, float, timestamp, binary, etc.,
a document, or
an array of values
High Availability: Replica Sets
Initialize -> Election Primary + data replication from primary to secondary
©2012 Jaspersoft Corporation. Proprietary and Confidential 4
Node 1Secondar
y
Node 2Secondar
y
Node 3Primary
ReplicationReplication
Heartbeat
High Availability: Failure
Primary down/network failure Automatic election of new primary if majority exists
©2012 Jaspersoft Corporation. Proprietary and Confidential 5
Node 1Secondar
y
Node 2Secondar
y
Node 3Primary
Heartbeat
Primary Election
High Availability: Failover
New primary elected Replication established from new primary
©2012 Jaspersoft Corporation. Proprietary and Confidential 6
Node 1Secondar
y
Node 2Secondar
y
Node 3Primary
Heartbeat
Durability
Fire and forget Wait for error Wait for journal sync Wait for fsync Wait for replication
©2012 Jaspersoft Corporation. Proprietary and Confidential 7
Read Preferences
PRIMARY PRIMARY PREFERRED SECONDARY SECONDARY PREFERRED NEAREST
©2012 Jaspersoft Corporation. Proprietary and Confidential 8
Let’s build a location based surf reporting app!
©2012 Jaspersoft Corporation. Proprietary and Confidential 9
Let’s build a location based surf reporting app!
• Report current conditions
Let’s build a location based surf reporting app!
• Report current conditions• Get current local conditions
Let’s build a location based surf reporting app!
• Report current conditions• Get current local conditions • Determine best conditions per beach
Document Structure
{"_id" : ObjectId("504ceb3d30042d707af96fef"),"reporter" : "test","location" : {
"coordinates" : [-122.477222,37.810556
],"name" : "Fort Point"
},"conditions" : {
"height" : 0,"period" : 9,"rating" : 1
},"date" : ISODate("2011-11-16T20:17:17.277Z")
}
Document Structure
{"_id" : ObjectId("504ceb3d30042d707af96fef"),"reporter" : "test","location" : {
"coordinates" : [-122.477222,37.810556
],"name" : "Fort Point"
},"conditions" : {
"height" : 0,"period" : 9,"rating" : 1
},"date" : ISODate("2011-11-16T20:17:17.277Z")
}
Primary Key, Unique, Auto-indexed
Document Structure
{"_id" : ObjectId("504ceb3d30042d707af96fef"),"reporter" : "test","location" : {
"coordinates" : [-122.477222,37.810556
],"name" : "Fort Point"
},"conditions" : {
"height" : 0,"period" : 9,"rating" : 1
},"date" : ISODate("2011-11-16T20:17:17.277Z")
}
Primary Key, Unique, Autoindexed
Compound Index,Geospacial
Document Structure
{"_id" : ObjectId("504ceb3d30042d707af96fef"),"reporter" : "test","location" : {
"coordinates" : [-122.477222,37.810556
],"name" : "Fort Point"
},"conditions" : {
"height" : 0,"period" : 9,"rating" : 1
},"date" : ISODate("2011-11-16T20:17:17.277Z")
}
Primary Key, Unique, Autoindexed
Compound Index,Geospacial
Indexed forTime-To-Live
Get local surf conditions
db.reports.find( {"location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
Get local surf conditions
db.reports.find( {"location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
Get local surf conditions
db.reports.find( {"location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"date" : 1, "location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports• Get today’s reports
Get local surf conditions
db.reports.find( {"location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports• Get today’s reports• Return only the relevant info
Get local surf conditions
db.reports.find( {"location.coordinates" : { $near : [-122, 37] , $maxDistance : 0.9}, date : { $gte : new Date(2012, 8, 9)} }, {"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports• Get today’s reports• Return only the relevant info• Show me the best surf first
Results
{ "location" : { "name" : "Montara" }, "conditions" : { "height" : 6, "period" : 20, "rating" : 5 } }{ "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 5, "period" : 13, "rating" : 3 } }{ "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } }{ "location" : { "name" : "Maverick's" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } }{ "location" : { "name" : "Montara" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } }{ "location" : { "name" : "Linda Mar" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } }{ "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } }{ "location" : { "name" : "Sharp Park" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } }{ "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } }{ "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } }{ "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 4, "period" : 6, "rating" : 1 } }{ "location" : { "name" : "South Ocean Beach" }, "conditions" : { "height" : 0, "period" : 14, "rating" : 1 } }
Sharding is the partitioning of data among multiple machines
Balancing occurs when the load on any one node grows out of proportion
©2012 Jaspersoft Corporation. Proprietary and Confidential 23
Scaling
Scaling MongoDB
Client Application
Single InstanceOr
Replica Set
MongoDB
Sharded cluster
The Mechanism of Sharding
Complete Data Set
Maverick’s RockawayFort Point Ocean BeachLinda Mar
Define Shard Key on Location Name
The Mechanism of Sharding
Chunk
Define Shard Key on Location Name
Chunk
Maverick’s RockawayFort Point Ocean BeachLinda Mar
The Mechanism of Sharding
Chunk Chunk ChunkChunk
Maverick’s RockawayFort Point Ocean BeachLinda Mar
The Mechanism of Sharding
Chunk
Maverick’s RockawayFort Point Ocean BeachLinda Mar
Chunk ChunkChunk
Shard 1 Shard 2 Shard 3 Shard 4
The Mechanism of Sharding
©2012 Jaspersoft Corporation. Proprietary and Confidential 29
Shard 1 Shard 2 Shard 3 Shard 4
Chunkc Chunkc Chunkc ChunkcChunkc Chunkc
Chunkc
Chunkc Chunkc
Chunkc
The Mechanism of Sharding
©2012 Jaspersoft Corporation. Proprietary and Confidential 30
Shard 1 Shard 2 Shard 3 Shard 4
Chunkc Chunkc Chunkc ChunkcChunkc Chunkc
Chunkc
Chunkc Chunkc
Chunkc
Client ApplicationQuery: Linda Mar
The Mechanism of Sharding
©2012 Jaspersoft Corporation. Proprietary and Confidential 31
Shard 1 Shard 2 Shard 3 Shard 4
Chunkc Chunkc Chunkc ChunkcChunkc Chunkc
Chunkc
Chunkc Chunkc
Chunkc
Client ApplicationQuery: Maverick’s
Analysis Features:Aggregation Framework
What are the best conditions for my local beach?
Pipelining Operations
$match
$project
$group
Match “Linda Mar”
Only interested in conditions
Group by rating, averagingwave height and wave period
$sort Order by best conditions
Aggregation Framework
{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
Aggregation Framework
{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
Match “Linda Mar”
Aggregation Framework
{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
Only interested in conditions
Aggregation Framework
{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
Group by rating & average conditions
Aggregation Framework
{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : "Linda Mar"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
Show me best conditions first
Other Features…
Native MapReduce Hadoop Connector Tagging Drivers for all major languages
©2012 Jaspersoft Corporation. Proprietary and Confidential 39
