Deploying the Graph

Infrastructure Coders - Melbourne - November 2013Kevin O’Neill - CTO PlayUp - @kevinoneill

Kevin O’Neill CTO PlayUp

Deploying the Graph

@kevinoneill

November 2013

Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013

A little History

Engage & Entertain Around Live Sport

Mobile Games

SMS Based

Number of Runs

Complex

Carrier Integration

Smart Phone Revolution

Move from a Content Creator to a Content

Enabler

Top 10 in Engagement

Over 3 Million Likes

30% - 40% Active

Manually Curated

We Know there's an Opportunity

The Graph

• A Scalable language

• Object-Oriented

• Functional

• Seamless Java Interoperability

• Functions are Objects

• Actors

• Fault Tolerance

• Location Transparency

• Transactors

• Graph Database

• Vertices and Edges

• Properties

• Full Text Index

WhiteboardOutlining the bits

Runtime

• us-west

• m2.2xlarge

• ubuntu 13.04

• jre 7

• Graph - 8G of Heap

• Others 128M - 1G

• Parallel GC

The remaining memory is reserved for file system cache and

memory mapped buffers

Building the Components

Builds the jars

Debian Packages

• Places binaries in opt

• Adds an upstart file for the

• Permissions on data and log directories

graph-server.upstart

setuid ubuntu setgid ubuntu !console log !chdir /var/lib/playup/spot/graph-‐server !exec java -‐server -‐Xms4096M -‐Xmx8192M -‐Xss1M \ -‐XX:MaxPermSize=256M -‐XX:+UseParallelGC \ -‐jar /opt/playup/spot/graph-‐server/run.jar \ com.playup.spot.graph.service.NeoKernel

Base configuration is for production

Baking

Packer

Create machine images for multiple platforms from a single config

Images have lots of benefits

Fast Deployment

Stability

Testability

Packer makes creating images a breeze

Example

Builders

"builders": [ { "type": "amazon-‐ebs", "instance_type": "t1.micro", "region": "ap-‐southeast-‐2", "ami_name": "spot-‐graph-‐{{user `version`}}-‐{{timestamp}}", "source_ami": "ami-‐4bf26f71", "ssh_username": "ubuntu", "tags": { "Name": "spot-‐graph-‐{{user `version`}} ({{isotime}})", "OS": "Ubuntu", "Version": "{{user `version`}}-‐{{timestamp}}" }, "ami_regions":["us-‐west-‐2"] } …

Builders

Variables

"variables": { "version": "development" }

Provisioners "provisioners": [ { "script": "scripts/update-‐packages.sh", "type": "shell" }, { "script": "scripts/install-‐puppet.sh", "type": "shell" }, { "type": "file", "source": "packages", "destination": "/tmp" }, { "script": "scripts/install-‐spot-‐packages.sh", "type": "shell" }, { "type": "puppet-‐masterless", "manifest_file": "puppet/spot-‐graph.pp", "module_paths": ["puppet/local", "puppet/common"] }, { "script": "scripts/build-‐date.sh", "type": "shell" }

Builder Provisioners

"provisioners": [ ! … ! { "type": "shell", "script": "scripts/vagrant.sh", "only": ["virtualbox"] }, { "type": "shell", "script": "scripts/guest-‐additions.sh", "only": ["virtualbox"] }, ! …

$ packer build -‐-‐only "amazon-‐ebs" template.json

In a few minutes you have a shiny new image

• Variables

• source ami

• region

• default build is for local development

Starting an instance

Remember

Backed config is for Production*

We have staging plus multiple developer environments

CloudInit

Configure values in /etc/playup/…

Configure the puppet master*

Cloud Formation

Which images

What CloudInit values

Challenges

Too many to list …

Persistant Data

Service Registration

Cluster Membership

Investigating

Docker

Apache Mesos

Come Join Us

Disrupting

Hiring Selectively

Talk to Me or Andrea

You miss 100 percent of the the shots you don't take

– Wayne Gretzky

Kevin O’NeillCTO - PlayUp

kevin@playup.com@kevinoneill

Questions?

Deploying the Graph

Technology

Eindhoven University of Technology MASTER The …Graph I Graph 2 Graph 3 Graph 4 Graph 5 Graph 6 Graph 7 Graph 8 Graph 9 Population and annual growth rate of Tanzania, 1955-1990 Annual

Deploying a Microsoft SQL Server 2008 Database on the … · Deploying a Microsoft SQL Server 2008 Database on the Sun ZFS ... Microsoft SQL Server databases use a ... Deploying a

Graph Isomorphism Algorithms: Investigation Of The Graph

Deploying the BIG-IP System for DNS traffic management · Deploying the BIG-IP System for DNS Traffic Management ... question, as shown in the ... Deploying the BIG-IP System for

Deploying and updating Microsoft Office 365 ProPlus · ... Deploying and updating Microsoft ... Deploying and updating Microsoft Office ... Configuration Manager downloads the updates

Deploying CRM in the Cloud

Graphs of Quadratic Functions Graph the function. Compare the graph with the graph of Example 1

Deploying Apps to the Cloud

Best practices: Deploying the IBM Banking Data · PDF fileBest practices: Deploying the IBM Banking Data ... Deploying the IBM Banking Data Warehouse to IBM InfoSphere BigInsights

Deploying DriveWorks Throughout the Organization

Do Now: Graph Analysis Answer the questions about the graph

Deploying the Net Sweeper Solution

Deploying the image Fritz Ohman Alphageek

The iSMART Suite: Developing and Deploying …hosteddocs.ittoolbox.com/EW030504.pdf · Developing and Deploying Embedded Spatial Applications and ... for developing and deploying

Options for Deploying Apps / Add-Ins Deploying to the Store Deploying To Exchange Deploying to The Corporate Catalog Additional Approaches

Pulley Lab The Graph. Graph 1.Get a blank sheet of graph paper

RecoverPoint Deploying With the Brocade Splitter

Configuring and Deploying the ODI Console

Deploying SSD in the Data Center

Network Configuration Example Deploying the SRX … • ... Network Configuration Example Deploying the SRX Series for Enterprise Security Created Date: 20180227043519Z