MongoDB for mobile appbackends

sf android meetup08.31.2010

I’ve written my mobile app…where should I store my data ?

?

Mobile apps do not existin a vacuum

• user profile data• logging• location-based data collection• game data (stats, high scores)• polls and surveys• user feedback

Do I want to host thismyself ?

Java + Spring + Hibernate ?Ruby on Rails ?

Hmmm… I will probablyneed to build a REST API…

GET, POST, PUT, DELETE

•EC2•Rackspace•App Engine*

•MySQL•SQLServer•Postgres

•RoR•J2EE

General approach

REST API

How can this be even simpler andmore awesome?

• “Scalable, open-source, high-performance, document-orienteddatabase” - 10gen

• “The database your database couldsmell like” - old spice guy

MongoDB is designed to addresstwo modern DB problems

1. object / relational “impedance mismatch”

2. horizontal scalability

Native drivers in almost everylanguage

• Java• Javascript• Python• Ruby• C#• PHP• and more…

A paradigm shift…

• Traditional RDMS– database– table– row

• MongoDB– database– collection– document / object

Documents (a.k.a. Objects){ _id: 1234, author: { name: “Bob Jones”, email: “b@b.com” }, post: “In these troubled times I like to …“, date: { $date: “2010-07-12 13:23UTC” }, location: [ -121.2322, 42.1223222 ], rating: 2.2, comments: [ { user: “jgs32@hotmail.com”, upVotes: 22, downVotes: 14,

text: “Great point! I agree” }, { user: “holly.davidson@gmail.com”, upVotes: 421, downVotes: 22,

text: “You are a moron” } ]}

{ _id: 1234, author: { name: “Bob Jones”, email: “b@b.com” }, post: “In these troubled times I like to …“, date: { $date: “2010-07-12 13:23UTC” }, location: [ -121.2322, 42.1223222 ], rating: 2.2, comments: [ { user: “jgs32@hotmail.com”, upVotes: 22, downVotes: 14,

text: “Great point! I agree” }, { user: “holly.davidson@gmail.com”, upVotes: 421, downVotes: 22,

text: “You are a moron” } ], tags: [ “politics”, “Virginia” ]}

Flexible “schemas”

db.posts.find({ author.name: “mike” })

Dynamic queries

db.posts.find({ rating: { $gt: 2 }})

db.posts.find({ tags: “software” })

db.posts.find().sort({date: -1}).limit(10)

Comment c = {author: “will”, date: new Date(), text: “great post!”}

db.posts.update({_id: post._id}, {$push: {comments: c}})

Atomic update operators

Aggregation and Map/Reduce

Focus on scalability andperformance

depth of functionality

sca

lab

ility

an

d p

erf

orm

an

ce •memcached

•key / value

•RDBMS

•

High performance, fault tolerantclusters made easy

• master / slave replication

• replica sets

• auto-sharding

MongoDB is great for mobile

• everything is JSON!– storage model is JSON– queries are in JSON– update operators are in JSON

• geospatial indexing built-in

• GridFS

Geospatial indexing

• index on geo coordinate pairs

• search by– bounding box– bounding circle (point, radius)

Geospatial indexing

db.places.ensureIndex( { loc: “2d” } )

db.places.find({ loc: { $near : [50, 50] } }).limit(10)

db.places.find({loc: {$within : {$box : [[40,40],[60,60]]}}})

db.places.find({loc: {$within : {$center : [[40,40],10]}}})

Geospatial indexing atFoursquare

• ~1.3M registered users• ~615k check-ins a day• “Who’s here” service tracks last 3 hours for

every user; uses mongo exclusively• all checkins, tips and venues written to

mongo; reads a mix of mongo / legacypostgres

GridFS

• distributed filesystem inside MongoDB

• stores large files by splitting them intosmaller documents

• leverages existing sharding andreplication configuration

• stores metadata along with files

• works well behind a CDN

Street cred

• Shutterfly• Foursquare• bit.ly• Sourceforge• Etsy• The New York Times• Business Insider• Github• Gilt Groupe

• Sugar CRM• Electronic Arts• Evite• CollegeHumor• Disqus• Justin.tv• Chartbeat• Hot potato• Eventbrite

Not so good for

• Complex / multi-operationtransactions

• When you (really) need joins

How can this be even simpler andmore awesome?

• MongoLab provides cloud-hostedMongoDB

• It’s EC2 hosting + MongoDB + RESTAPI + awesome admin tools

• http://mongolab.com

GET, POST, PUT, DELETE

EC2

MongoDB

Two ways to use MongoLab

REST API

method 1: REST API

method 2: MongoDB driver

Your application

/databases/<d>/collections GET

/databases/<d>/collections/<c> GET POST

/databases/<d>/collections/<c>/<_id> GET PUT DELETE

/databases/<d>/collections/<c>?[q=<query>] [&f=<fields>] [&s=<order>] [&sk=<skip>] [&l=<limit>] GET

MongoLab API

Example app:GeoPost

• add a GeoPost (email, date, message, location)

• list nearby GeoPosts

Example app:GeoPost

• POSTURL: https://mongolab.com/api/1/databases/demo/collections/geoposts

data: { “email” : “will@mongolab.com”, “date” : { “$date” : “2010-02-03 10:21:21 UTC” },

“location” : { “lat” : 41, “long” : -82 }, “message” : “Hello Cleveland!” }

• GETURL: https://…/geoposts?q={"location":{"$near":{"lat":38,"long":-122}}}

Android SDK makes client-sideeasy

• JSON: org.json.*• REST: org.apache.http.*

HttpClient client = new DefaultHttpClient()HttpGet request = new HttpGet(url);HttpResponse response = client.execute(request);HttpEntity entity = response.getEntity();InputStream in = entity.getContent();String json = convertStreamToString(in);JSONObject o = new JSONObject(json);

For more info…

• MongoDB / 10Gen– http://mongodb.org– http://10gen.com

• MongoLab– http://mongolab.com– will@mongolab.com