Visualizing MongoDB Objectsin Concept and Practice

https://github.com/cvitter/ikanow.mongodc2013.presentation

Washington DC 2013

Introduction

• Do you have a MongoDB database full of BSONdocuments crying out for visual analysis?

Agenda

• Visualizing Objects vs. Records• Getting Your Data from MongoDB• JSON 101• Free & Open Source Visualization Libraries

– Google Maps JavaScript API– D3.js

• Other Visualization Libraries You Should Know• Questions?

Objects vs. Records

• Document oriented data stores support dynamic and complex schemas vs. the simple, static structures in RDBMs, e.g.:

Is There Really a Difference?

• Yes and No

• Yes• Obviously, document oriented formats are different from

record oriented formats;• Few common visualizations tools designed for

traditional record based formats support document based formats

• No• The basic visualizations are the same even if the data

format feeding them are different

Getting your Data from MongoDB

• mongoexportExport data from MongoDB as JSON or CSV

• MongoDB’s Simple REST Interface• Read only access to data• Start your server with the --rest option• Simple queries:

http://127.0.0.1:28017/databaseName/collectionName/

• Other options:• DrowsyDromedary, MongoDB Rest, etc.

> mongoexport --db dbname --collection collectionname --out file.json

JSON 101

• JSON (JavaScript Object Notation) documentsare built using two types of common data structures:

• Name/value pairs (objects) and;

• Ordered lists of values (arrays).

• JSON is a subset of the object literal notation of JavaScript so:

• Converting the string representation of JSON is as easy as:

{ “string” : value }

{ “string” : [value, value, value] }

var jsonObj = {"numbers" : [1, 2, 3] };var numberOne = jsonObj.numbers[0];var numberTwo = jsonObj.numbers[1];

var jsonObj = JSON.parse("{\"numbers\":[1,2,3]}");

A Word About the Sample Code

• All of the code used in this presentation is available online via GitHub at:https://github.com/cvitter/ikanow.mongodc2013.presentation

• The project includes code from the following Open Source Projects:

• Bootstrap: http://twitter.github.com/bootstrap/• JQuery: http://jquery.com/• D3.js: http://d3js.org/

• The data samples used are simple JSON files read in using JQuery’a .ajax method

Google Maps JavaScript API

• Requirements:• Get an API Key (its free*)• Have an Internet Connection

• Build a simple example that:1. Creates a place holder DIV for the map object2. Loads the map script3. Initializes the map4. Draws markers on the map

* Up to 25,000 map loads per day for commercial applications.

• Use a DIV to create a placeholder for the map:

• Load the map script:

• Initialize the map:

Creating the Map

<div id="map_canvas" style="height:450px; width:870px;"></div>

function loadScript() { var script = document.createElement("script"); script.type = "text/javascript"; script.src = "http://maps.googleapis.com/maps/api/js?

key=YOUR_API_KEY&sensor=TRUE_OR_FALSE&callback=initialize"; document.body.appendChild(script);}

var mapOptions = {zoom: 11,center: new google.maps.LatLng( 38.8964, -77.0262 ),mapTypeId: google.maps.MapTypeId.ROADMAP

};map = new google.maps.Map(document.getElementById('map_canvas'),

mapOptions);

• Sample documents:

• Add markers to the map:

Drawing Markers on the Map

for (var i=0; i < locations.length; i++) {var marker = new google.maps.Marker({

map: map,position: new google.maps.LatLng(

locations[i].latitude , locations[i].longitude ),

title: convertToTitleCase(locations[i].search_field)});

}

{...},{

search_field: "adams morgan",country: "United States",country_code: "US",

region: "District of Columbia",latitude: "38.9213889",longitude: "-77.0425000”}

},{...},

The Finished Product

• Requires the Visualization Library:

• Create the Heat Map Data and Layer:

Creating Heat Maps

var heatMapData = new Array();for (var i=0; i < locations.length; i++) {

var newRecord = {location: newgoogle.maps.LatLng( locations[i].geoindex.lat ,

locations[i].geoindex.lon), weight: 1};heatMapData.push(newRecord);

}

var heatmap = new google.maps.visualization.HeatmapLayer({data: heatMapData,dissipating: true,radius: 10,map: map

});

script.src = "http://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&sensor=TRUE_OR_FALSE&libraries=visualization&callback=initialize";

The Finished Product

• D3.js (d3js.org) is:“a JavaScript library for manipulating documents based on data”

• Convert data into visualizations in the following formats: HTML, CSS, SVG

• Download the library or include it via:

D3.js

<script src="http://d3js.org/d3.v3.min.js"></script>

• Select the DIV to write the SVG image to:

• Draw the bars (rectangles):

The (Very) Basics

var chart = d3.select("#d3_canvas").append("svg").attr("class", "chart")

.attr("width", chartWidth)

.attr("height", chartHeight);

chart.selectAll("rect").data(inputData)

.enter().append("rect")

.attr("y", function(d, i) {return i * (lineHeight + 3); })

.attr("width", function(d, i) { return chartWidth * (d.chance_of_rain *

0.01); }).attr("height", function(d) return lineHeight; });

• Making things scale on the chart:

• Drawing the rules:

Adding Rules

var x = d3.scale.linear().domain([0, 100]).range([0, chartWidth]);

chart.selectAll("line").data(x.ticks(10))

.enter().append("line")

.attr("x1", x)

.attr("x2", x)

.attr("y1", 0)

.attr("y2", chartHeight)

.style("stroke", "#ccc");

• Labeling the X and Y axes:

Adding Text

chart.selectAll(".rule").data(x.ticks(10)).enter().append("text").attr("class", "rule").attr("x", x).attr("y", 0).attr("dy", -3).attr("text-anchor", "middle").text(String);

chart.selectAll("text").data(inputData).enter().append("text").attr("x", 0).attr("y", function(d, i) {

return i * (lineHeight + 3) + (lineHeight / 2); }).attr("dx", -3) // padding-right.attr("dy", ".35em") // vertical-align: middle.attr("text-anchor", "end") // text-align: right.text(function(d) { return d.date; });

The Finished Product

Treemaps, Bubble Charts, and More

• D3.js can accept an array of values, objects, or a function that returns an array of values

• Some of the D3.js visualizations allow you to pass data in a wide variety of formats, others expect a specific format

• The Treemap and Bubble Chart samples use a really simple JSON tree structure representing multi-level parent child relationships

Treemap

Treemaps Layout

• D3.js features different layout packs that help you build complex charts including .treemap

• In this example the layout pack is creating a properly sized div for each node in our JSON file as opposed to creating an SVG imagevar treemap = d3.layout.treemap()

.size([width, height])

.sticky(true)

.value(function(d) { return d.size; });

var node = div.datum(data).selectAll(".node").data(treemap.nodes).enter().append("div").attr("class", "node").call(position).style("background", function(d) {

return d.children ? color(d.name) : null; }).text(function(d) { return d.children ? null : d.name; });

Bubble Chart

Other D3js.org Examples

• NVD3 (nvd3.org)Re-usable charts and chart components for d3.js

• Raphael JS (raphaeljs.com)JavaScript library designed to simplify working with vector graphics and build data visualizations

• TimelineJS (timeline.verite.co)“Beautifully crafted timelines that are easy, and intuitive to use.”

More Cool Visualization Libraries

Reminder: Get the Example Code

https://github.com/cvitter/ikanow.mongodc2013.presentation

• All of the sample code used in this presentation is available online via GitHub at:

Thank You!

Craig Vitter

www.ikanow.comdeveloper.ikanow.comcvitter@ikanow.com

@IkanowDev

github.com/ikanow/Infinit.e