Code-First.NET and MongoDB The Ubiquitous Blog Example Philly ALT.NET User Group 2010-11-16 John C....

Code-First .NET and MongoDBCode-First .NET and MongoDB

The Ubiquitous Blog ExampleThe Ubiquitous Blog ExamplePhilly ALT.NET User GroupPhilly ALT.NET User Group

2010-11-162010-11-16

John C. ZablockiJohn C. ZablockiDevelopment Lead, MagazineRadarDevelopment Lead, MagazineRadar

Adjunct, Fairfield UniversityAdjunct, Fairfield University

Agenda

MongoDB Basic Concepts MongoDB Shell NoRM MongoDB C# Driver MongoDB Design Considerations In Depth: Meringue Case Study: RateMySnippet Questions?

MongoDB Concepts

Schema-less documents stored in collectionsSchema-less documents stored in collections Documents are stored as BSON (Binary JSON)Documents are stored as BSON (Binary JSON) JavaScript used to query and manipulate JavaScript used to query and manipulate

documents and collectionsdocuments and collections Each document in a collection has a unique Each document in a collection has a unique

BSON ObjectId field named _idBSON ObjectId field named _id Collections belong to a databaseCollections belong to a database

Installing MongoDB on WindowsInstalling MongoDB on Windows

Download the binaries from mongodb.orgDownload the binaries from mongodb.org Extract to Program Files directory (or wherever)Extract to Program Files directory (or wherever) Create a directory c:\data\dbCreate a directory c:\data\db Run mongod.exe from the command line with the --Run mongod.exe from the command line with the --

install switchinstall switch

See See http://bit.ly/aed1RW for some gotchas for some gotchas To run the daemon without installing, simply run To run the daemon without installing, simply run

mongod.exe without argumentsmongod.exe without arguments Run mongo.exe to verify the daemon is runningRun mongo.exe to verify the daemon is running

MongoDB and .NET

A funny thing happened on the way to Philadelphia...

MongoDB - ShellMongoDB - Shell

The MongoDB interactive JavaScript shell The MongoDB interactive JavaScript shell (mongo.exe) is a command line utility for (mongo.exe) is a command line utility for working with MongoDB serversworking with MongoDB servers

Allows for CRUD operations on collectionsAllows for CRUD operations on collections May be used for basic administrationMay be used for basic administration

Creating indexesCreating indexes Cloning databasesCloning databases

Also useful as a test-bed while building appsAlso useful as a test-bed while building apps

MongoDB - Shell

/*Connect to a server:port/database (defaults are localhost:27017/test ):*/mongo.exe localhost:27017/AltNetGroup

//Switch database:use CodeCamp

//View collections in a database:show collections

//create an index on Name fielddb.Posts.ensureIndex({ Name : 1 });

//copy one database to anotherdb.copyDatabase("CodeCamp", "AltNetGroup")

MongoDB - Shell

//create a documentvar post = { Title: "On Installing MongoDB as a Service on Windows" }

//insert a document, if the collection doesn't exist it's createddb.Posts.insert(post);

//verify that the document was createddb.Posts.find();

//write a query to find a post with a valid titlevar query = { Title: { $ne: null} }

//use that query to find the postvar post = db.Posts.findOne(query);

//this line will actually set the content after pressing enterpost.Content = "When installing MongoDB as a service on Windows..."

MongoDB - Shell

//update the content to include an author using collection update methoddb.Posts.update( { Title : "On Installing MongoDB as a Service on Windows" },

{ Author : "John Zablocki" } )

//check that the post was updateddb.Posts.findOne()

//where'd my document go? updates are in place, replacing entire documents!//need to use the $set operator to update partial documents - let's start over//first remove the new document. Notice how remove takes a func as an argument.//find and findOne also accept functions as argumentsdb.Posts.remove(function (e) { return this.Author == "John Zablocki" })

//rerun the first statements up to but not including the db.Posts.update(...db.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $set: { Author: "John Zablocki" } })

//verify that the update workeddb.Posts.findOne()

MongoDB - Shell

//add a tag to the postdb.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $push: { Tags: "mongodb" } })

//look for that tagdb.Posts.findOne()

//add two more tagsdb.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $pushAll: { Tags: ["windows", "nosql"] } })

//add another postdb.Posts.insert( { Author : "John Zablocki", Title : "On MapReduce in MongoDB", Tags: ["mongodb", "nosql"]})

//verify that last insert workeddb.Posts.findOne(function (e) { return this.Title.indexOf("MapReduce") != -1; })

MongoDB - Shell

//add a "like" counter to the post. The booleans arguments tell update//not to insert if the document doesn't exist and to update all documents, //not just one respectivelydb.Posts.update({ Author: "John Zablocki" }, { $set: { Likes: 0} }, false, true)

//increment the likes counter for the mapreduce articledb.Posts.update({ Title: /mapreduce/i }, { $inc: { Likes: 1} })

//check that the counter was incrementeddb.Posts.findOne({ Title: /mapreduce/i }).Likes

MongoDB – ShellMongoDB – Shell

/*Create the map functionThis function creates a view that looks like{ “mvc”, [1, 1] },{ “norm”, [1] }*/ var map = function() {

if (!this.Tags) { return; } for (var index in this.Tags) {

emit(this.Tags[index], 1); } };

MongoDB – Shell MapReduceMongoDB – Shell MapReduce

/*Create the reduce functionconceptually, reduce gets called like:reduce("mvc", [1, 1]);reduce("norm", [1]/*var reduce = function(key, vals) { var count = 0; for (var index in vals) { count += vals[index]; } return count; };

MongoDB – Shell MapReduceMongoDB – Shell MapReduce

/*Run the mapreduce command on the Posts collectionusing the map and reduce functions defined abovestore the results in a collection named Tags*/var result = db.runCommand( { mapreduce : "Posts", map : map, reduce : reduce, out : "Tags" });

db.Tags.find()

Something Completely Different

NoRM

NoRMNoRM

Developed by Andrew Theken among othersDeveloped by Andrew Theken among others Active community with reliable supportActive community with reliable support

I received help even as I prepared this slide!I received help even as I prepared this slide! Support for typed and untyped collections, Support for typed and untyped collections,

MapReduce, Property Mapping, LINQ, MapReduce, Property Mapping, LINQ, Expando, nearly all CRUD operationsExpando, nearly all CRUD operations

Will eventually sit on top of the 10gen officially Will eventually sit on top of the 10gen officially supported driversupported driver

NoRM – The BasicsNoRM – The Basics

//Connections managed with IDisposable pattern

//Connect to test database on localhostusing (IMongo mongo = Mongo.Create("mongodb://localhost/test")) {

Console.WriteLine(mongo.Database.DatabaseName);}

//Mongo instance has MongoDatabase property

/*MongoDatabase has GetCollection<T> methods for accessing MongoCollection instances*/

//CRUD operations performed on collections

NoRM - CRUDNoRM - CRUD

//Class properties map to document structure public class Artist { //ObjectId property mapped to _id field in document public ObjectId Id { get; set; } public string Name { get; set; }

//IList property will be mapped to JavaScript Array private IList<string> _albums = new List<string>(0); public IList<string> Albums { get { return _albums ; } set { _albums = value; } } }

NoRM - CRUDNoRM - CRUD

var artist = new Artist() { Name = "The Decembrists" };

//Inserting a document into a typed collection mongo.Database.GetCollection<Artist>("Artists").Insert(artist);

//Updating (replacing) a document in a typed collectionartist.Name = "The Decemberists"; mongo.Database.GetCollection<Artist>("Artists").Save(artist);

//Updating a nested collectionmongo.Database.GetCollection<Artist>("Artists").UpdateOne(

new { Name = "The Decemberists"}, new { Albums = M.Push("Picaresque") }

);

NoRM - CRUDNoRM - CRUD

//Find all documents in a typed collectionvar artists = mongo.GetCollection<Artist>("Artists").Find();Console.WriteLine(artists.FirstOrDefault().Name);

//Query with a document specvar artist = mongo.GetCollection<Artist>("Artists")

.FindOne( new { Name = "The Decemberists" });Console.WriteLine(artist.Albums.Count);

//Count the documents in a collectionlong count = mongo.GetCollection<Artist>("Artists").Count();Console.WriteLine(count);

NoRM - MapReduceNoRM - MapReduce

//Add a Tags collection added to Artist classprivate IList<string> _tags; public IList<string> Tags {

get { return _tags; } set { _tags = value; }}

//Add some tags – use Set not PushAll, since Tags was null mongo.Database.GetCollection<Artist>("Artists").UpdateOne(

new { Name = "The Decemberists" }, new { Tags = M.Set(new List<string>()

{ "Alternative", "Folk Rock }) });

NoRM - MapReduceNoRM - MapReduce

//Create map and reduce functonsstring map = @"function() {

if (! this.Tags ) { return; } for (index in this.Tags) {

emit(this.Tags[index], 1); }

}";

string reduce = @"function(previous, current) {var count = 0;

for (index in current) { count += current[index];

} return count; }";

NoRM - MapReduceNoRM - MapReduce

//MapReduce class is responsible for calling mapreduce commandMapReduce mr = mongo.Database.CreateMapReduce(); //Represents the document passed to the //db.runCommand in the shell exampleMapReduceOptions options = new MapReduceOptions("Artists") {

Map = map, Reduce = reduce, OutputCollectionName = "Tags"

}; MapReduceResponse response = mr.Execute(options);var collection = mongo.Database.GetCollection<Tag>("Tags");Console.WriteLine(collection.Count());

NoRM - LINQNoRM - LINQ

//LINQ provider exposed via AsQueryable method of MongoCollectionvar artists = mongo.Database.GetCollection<Artist>("Artists")

.AsQueryable(); //Find items in typed collectionvar artistsWithDecInName =

from a in mongo.Database.GetCollection<Artist>().AsQueryable() where a.Name.Contains("Dec") select a;

Console.WriteLine("First containing Dec in name: " + artistsWithDecInName.First().Name);

NoRM - LINQNoRM - LINQ

//Find artists without pulling back nested collections//Note use of Regex name searchvar artistsWithoutAlbums =

from a in mongo.Database.GetCollection<Artist>().AsQueryable() where Regex.IsMatch(a.Name, "ber", RegexOptions.IgnoreCase) select new { Name = a.Name };Console.WriteLine(artistsWithoutAlbums.First().Name);

//Find artists with a given tagvar artistsWithFolkRockTag = mongo.Database.GetCollection<Artist>("Artists")

.AsQueryable().Where(a => a.Tags.Any(s => s == "Folk Rock"));Console.WriteLine(artistsWithFolkRockTag.First().Name);

MongoDB C# Driver

10gen developed and supported Consists of two primary components, a BSON

serializer and the MongoDB driver Support for typed and untyped collections, Support for typed and untyped collections,

MapReduce, and all CRUD operationsMapReduce, and all CRUD operations Currently lacking a LINQ providerCurrently lacking a LINQ provider Released about two weeks ago as version 0.7!Released about two weeks ago as version 0.7!

MongoDB C# Driver – The Basics

private static MongoDatabase _mongoDatabase = null;

static Program() {

//MongoConnectioSettings used to create connection strings

MongoConnectionSettings settings = new MongoConnectionSettings();

settings.Address = new MongoServerAddress("localhost", 27017);

//MongoServer manages access to MongoDatabase

MongoServer mongoServer = new MongoServer(settings);

//MongoDatabase used to access MongoCollection instances

_mongoDatabase = mongoServer.GetDatabase("AltNet");

}

MongoDB C# Driver - CRUD

var artist = new Artist() { Name = "The Decembrists" };

//Inserting a document into a typed collection _mongoDatabase.GetCollection<Artist>(COLLECTION).Insert(artist);

//Updating (replacing) a document in a typed collectionartist.Name = "The Decemberists";

_mongoDatabase.GetCollection<Artist>(COLLECTION).Save(artist);

//Updating a nested collection_mongoDatabase.GetCollection<Artist>(COLLECTION).Update(

Query.EQ("Name", "The Decemberists"), new BsonDocument("$pushAll", new BsonDocument("Albums", new BsonArray() { "Castaways

and Cutouts", "Picaresque", "Hazards of Love", "The Crane Wife" }))

);

MongoDB C# Driver - CRUD

//Find all documents in a typed collection

var artists = _mongoDatabase.GetCollection<Artist>(COLLECTION).FindAll();

Console.WriteLine("Artist name: " + artists.FirstOrDefault().Name);

//Query with a document specvar artist = _mongoDatabase.GetCollection<Artist>(COLLECTION).FindOne(

new BsonDocument { { "Name", "The Decemberists" } });

Console.WriteLine("Album count: " + artist.Albums.Count);

//Count the documents in a collectionlong count = _mongoDatabase.GetCollection<Artist>(COLLECTION).Count();Console.WriteLine("Document count: " + count);

MongoDB C# Driver - CRUD

var artists = _mongoDatabase.GetCollection<Artist>(COLLECTION);

//Find items in typed collection

var artistsStartingWithThe = artists.Find(Query.Matches("Name", new Regex("the", RegexOptions.IgnoreCase)));

Console.WriteLine(artistsStartingWithThe.First().Name);

//Find artists without pulling back nested collectionsvar artistsWithDecInTheName = artists.Find(

Query.Matches("Name", "Dec")).SetFields("Name");

Console.WriteLine(artistsWithDecInTheName.First().Name);

////Find artists with a given tag

var artistsWithIndieTag = artists.Find(Query.In("Tags", "Indie"));

Console.WriteLine( artistsWithIndieTag.First().Name);

MongoDB C# Driver - MapReduce

//Add some tags

_mongoDatabase.GetCollection<Artist>(COLLECTION).Update(Query.EQ("Name", "The Decemberists"),

new BsonDocument("$pushAll", new BsonDocument("Tags", new BsonArray()

{ "Folk rock", "Indie" })) );

var artist = new Artist() { Name = "Sunny Day Real Estate",Albums = new List<string>()

{ "How it Feels to be Something On", "Diary" },Tags = new List<string>() { "Indie", "Emo" }};

_mongoDatabase.GetCollection<Artist>(COLLECTION).Save(artist);

MongoDB C# Driver - MapReduce

_mongoDatabase.GetCollection<Artist>(COLLECTION).Save(artist);

//Create map and reduce functonsBsonJavaScript map = @"function() {...}";

BsonJavaScript reduce = @"function(previous, current) { ... }";

var result = _mongoDatabase.GetCollection<Artist>(COLLECTION).MapReduce(map, reduce, MapReduceOptions.SetKeepTemp(true).SetOutput("Tags"));

In Depth: Meringue

Design Considerations

Your object graph is your data model Don't be afraid to store data redundantly

Your graph might be redundant Not everything has to fit in 1 document Don't be afraid to store aggregate statistics with

a document.

Object Graph as Data Model

Generally speaking, most MongoDB drivers will serialize an object graph as a single document

The relationships of your classes creates an implied schema!

Migrating this schema is not trivial if you are trying to deserialize properties that did not or no longer exist

Consider use cases carefully to avoid inefficiently structured documents

Projection queries will be your friend

Redundant Data is OK

Optimize documents for quick reads and writes Your application layer will have to maintain

referential integrity! If every time you access a Post document, you

need some of an Author document's data, store that data with Post

Design simple classes for this redundant data for reusability (see AuthorInfo in Meringue)

Do Not Stuff it In 1 Document

Nothaving formal relationships does not mean throwing away relationships

Consider a user and his or her logged actions The user would likely have a User class/doc

with properties for name, email, etc. User actions are generally write heavy and

read out of band. Don't clutter user documents - create a

separate collection for user actions

Don't Be Afraid of Extra Data

The schema-less nature of documents makes it easy to store meta data about that document – particularly aggregate data

Consider a blog post with a rating feature Each rating would be stored as a nested

document of the post Rather than compute vote totals and averages

real time, simply add these properties to the document and update on writes

Case Study: RateMySnippet.com

Final Thoughts

Eat food. Not too much. Mostly plants.- Michael Pollan

Final Thoughts

Write Code. Not too much. Mostly C#. - John Zablocki

The Meringue Stack

ASP.NET MVC

MVCContrib

MongoDB

Mongo C# Driver

Spring.NET

Recoil

AutoMapper

Links

http://dllHell.net - my blog http://www.CodeVoyeur.com - my code http://www.linkedin.com/in/johnzablocki http://twitter.com/codevoyeur http://mongodb.org - Official MongoDB site http://bitbucket.org/johnzablocki/meringue http://bitbucket.org/johnzablocki/codevoyeur-

samples