Code but now as we know it - Devoxx edition

Infrastructure as Code Patrick Deboishttp://www.tumblr.com/tagged/star-trek-quotes

Thursday 15 November 12

Blog: http://jedi.be/blog - Twitter : @patrickdebois - #devops

Technical ReviewerMonitoring Chapter Co-Author Veewee / Sahara / Mccloud

Organizer Since 2009 Europe Organizer 2010

Engineer

First Europe Training 2010

Libvirt - Fog

Speaker

Freelance consultant http://github.com/jedi4ever

CONFIGURATION MANAGEMENT SYSTEMS

http://ansible.cc/

http://opscode.com/chef/

http://cfengine.com/http://palletops.com/http://puppetlabs.com/

http://www.nerdnirvana.org/wp-content/uploads/2010/12/apology-form.jpg

Chef & Puppet focusedbased on my experience

SERVICE

FILE1PACKAGE

DECLARATIVE LANGUAGE

“KNOWN STATE”

STATE X STATE YCODE RUN

FILE1SERVICE

FILE1PACKAGE

convergence

“IDEMPOTENCE”

STATE Y STATE YCODE RUN

SERVICE

FILE1PACKAGE

SERVICE

FILE1PACKAGE

GIVE ME THE CODE:“LANGUAGE”

Puppet Chef

class myapache {

package “apache2”

service “apache2”: ensure => “running”, require => Package[“apache2”]

package “apache2”

service “apache2 do action [:enable , :start]end

EXAMPLE SYNTAX

TERMINOLOGY

Java Puppet Chef

*.javamanifests

*.pprecipes

package module cookbook

TERMINOLOGY (2)

Java Puppet Chef

singletonparametrized

classesrecipes with attributes

classdefines, classes

definitions, libraries

SYNTAXPuppet Chef

Limited DSL(by design) DSL + full ruby

Extended by Definitions

Light Weight Resource Providers /

DefinitionsCustom Resource providers

Providers, Light Weight Resource

ProvidersERB & Custom Libraries Libraries

LOOPING CONSTRUCTS

Puppet Chef

pass array of elementserb template for loops

create_resource (:type, hash)

do while , repeat etc. available through

native ruby

VARIABLES

Puppet Chef

special syntax$bla = “aa”

string interpolation$bla_string = “${bla}”

native rubybla = “aa”

string interpolationbla_string = #{bla}

SINGLE CLASSINHERITANCE

Puppet Chef

override via ‘module path’

override via ‘cookbook’ path

+> operator “cookbook” attribute

Puppet Module Chef Cookbook

files/lib/

README.rdocmanifests/templates/

attributes/definitions/files/libraries/metadata.rbproviders/README.rdocrecipes/resources/templates/

PROJECT STRUCTURE

EXECUTION MODEL

Puppet Chef

Compile catalog(directed graph

model)Then Execute

Top Down execution

PLUGINSPuppet Chef

FACES KNIFE PLUGINS

REPORTS REPORT HANDLERS

http://puppetlabs.com/blog/puppet-faces-what-the-heck-are-faces/http://wiki.opscode.com/display/chef/Knife+Plugins

CODE EDITOR

https://github.com/rodjek/puppet-pygments-lexerhttp://redmine.djagios.org/projects/puppet-lexer

LEXERS

VIM SYNTAX HIGHLIGHTIING

https://github.com/rodjek/vim-puppethttps://github.com/t9md/vim-chef

https://github.com/vim-scripts/Cfengine-version-3-syntax

http://cloudsmith.github.com/geppetto/https://github.com/cloudsmith/geppetto

GEPPETTO - ECLIPSE IDE

JETBRAINS RUBYMINE (PUPPET)http://youtrack.jetbrains.com/issue/RUBY-10832

REACTOR8http://beta.r8network.com/

COMING SOON

https://github.com/janschumann/intellij-lang-puppet

REVERSE ENGINEERING

https://github.com/devstructure/blueprint

Existing Server -> Manifest/Recipes

CODING STYLE

Puppet Chef

http://puppet-lint.com/ http://acrmp.github.com/foodcritic/

• Must use two-space soft tabs.• Must not use literal tab characters.• Must not contain trailing white space.• Should not exceed an 80 character line width

An exception has been made for source => 'puppet://...' lines as splitting these over multiple lines decreases the readability of the manifests.

• Should align arrows (=>) within blocks of attributes.• ....

• FC001: Use strings in preference to symbols to access node attributes• FC002: Avoid string interpolation where not required• FC003: Check whether you are running with chef server before using server-

specific features• FC004: Use a service resource to start and stop services• FC005: Avoid repetition of resource declarations• FC006: Mode should be quoted or fully specified when setting file

permissions• FC007: Ensure recipe dependencies are reflected in cookbook metadata• FC008: Generated cookbook metadata needs updating• FC009: Resource attribute not recognised• FC010: Invalid search syntax• FC011: Missing README in markdown format• FC012: Use Markdown for README rather than RDoc• FC013: Use file_cache_path rather than hard-coding tmp paths•

http://docs.puppetlabs.com/guides/style_guide.htmlhttps://github.com/ampledata/cookbook-style-guide

http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-puppet-camp

CODE VS CONFIG(THINK “PROFILES”)

http://wiki.opscode.com/display/chef/Data+Bags

http://projects.puppetlabs.com/projects/hierahttps://github.com/crayfishx/hiera-gpg

https://github.com/jedi4ever/stop-the-forkhttp://realityforge.org/code/2012/11/12/reusable-cookbooks-revisited.htmlhttp://devopsanywhere.blogspot.it/2012/11/how-to-write-reusable-chef-cookbooks.html

“STOP THE FORK(ING)”

PRE-COMMIT CHECKS

DEPENDENCY MGMT

GitSubmodules

Subtree à la Pom File

UPSTREAM REPOS

http://forge.puppetlabs.com/http://community.opscode.com/

Puppet Chef

https://github.com/rodjek/librarian-puppet

https://github.com/applicationsonline/librarian

http://berkshelf.com/

site "http://community.opscode.com/api/v1"

cookbook "ntp"cookbook "timezone", "0.0.1"

cookbook "rvm", :git => "https://github.com/fnichol/chef-‐rvm", :ref => "v0.7.1"

cookbook "cloudera", :path => "vendor/cookbooks/cloudera-‐cookbook"

CHEF METADATA

http://bitfieldconsulting.com/puppet-dependency-graphs

Dependency Graphs

https://github.com/miketheman/knife-role-spaghettiThursday 15 November 12

http://wham.cloudsmith.com

Online Service

DEBUGGING

LOGGING WITH DIFFERENT LEVELS

Info only, no exceptionChef::Log.info('Some useful info')

Fatal, raising exceptionChef::Log.fatal('Something bad')

REPL - STYLE(READ EVAL PRINT)

http://wiki.opscode.com/display/chef/Shefhttps://github.com/pry/pry

https://github.com/vimeo/simple-black-boxThursday 15 November 12

DEV ENVIRONMENT

SERVICE

FILE1PACKAGE

Virtual Machine (1)

Just Enough Operating System+ Config Management Tool

Virtual Machine ... N

Your laptop (Linux, Wind

virtualization

SERVICE

FILE1PACKAGE

Virtual Machine (1)

Your Laptopvirtualization

http://vagrantup.com/

Virtualbox + (Fusion Soon)

Linux & WindowsThursday 15 November 12

https://github.com/BIAINC/vagrant-windows

https://github.com/dotless-de/vagrant-vbguesthttps://github.com/folken-laeneck/vagrant-bindfs

https://github.com/BerlinVagrant/vagrant-dnshttps://github.com/dwt/vagrant-hosts

USEFULVAGRANT PLUGINS

http://marketplace.eclipse.org/node/519961#.UKISB-Oe-d4

VAGRANT ECLIPSE INTEGRATION

https://github.com/rtyler/blimpy

https://github.com/jedi4ever/mccloud/

Vagrant ‘clones’

EC2, KVM, FUSION,HOSTS, Vagrant

https://github.com/neerolyte/vagueanthttps://github.com/lstoll/tenderloin FUSION

VAGRANT - CLOJURE

https://github.com/tbatchelli/vmfestThursday 15 November 12

VAGRANT - JAVA

https://github.com/guigarage/vagrant-bindingThursday 15 November 12

JEOS CREATION

Your Laptopvirtualization

Virtualbox , FusionKVM, Parallels

Linux & Windows

https://github.com/jedi4ever/veewee/

Centos, Ubuntu, Debian, Solaris, Gentoo, Suse, Oracle Linux, ....

And windows now too!

https://github.com/opscode/test-kitchenhttps://github.com/opscode/bento

uses #veewee

CREATING BASEBOX ‘EN MASS’

https://github.com/atkm/seisan-linehttps://github.com/atkm/origami

http://www.vagrantbox.es/

http://www.morethanseven.net/2011/05/08/Vagrant-plugin-for-interacting-with-vagrantboxes/

http://vagrantfil.es/

http://www.nodescription.net/ideation-into-the-void-vagrant-plugin-for-vag

SHARE YOUR SETUP

UNIT TESTING

https://github.com/nistude/cucumber-puppetThursday 15 November 12

https://github.com/rodjek/rspec-puppet

https://github.com/calavera/rspec-chefhttps://github.com/acrmp/chefspec

DON’T TEST YOUR CONFIG MANAGEMENT

TEST YOUR LOGIC

Was the package installed

If (OS=’X’ ) and (file exists) ...

did it give an error?

FAKE THE FACTSThursday 15 November 12

https://github.com/customink/fauxhaiThursday 15 November 12

AUTO-TRIGGER TESTS RUN

https://github.com/guard/guard-chefhttps://github.com/alister/guard-puppet-linthttps://github.com/johnbintz/guard-puppet

https://github.com/guard/guard-rspec

CODE EDIT

AUTO RUNFEEDBACK

LOCAL VM TESTING

http://www.cucumber-chef.org/

https://github.com/exceedhl/toft

Faster testing via LXC

spinning up a vm is expensiveunless in container

EC2+LXC

Vagrant+LXCThursday 15 November 12

CI INTEGRATIONThursday 15 November 12

https://github.com/nicksieger/ci_reporter

Junit processor for Ruby

SYNTAX CHECKSCOMPILE CHECKS

UNIT TESTS

SMOKE TESTS

NO VM NEEDED

REAL VM

SMOKE TESTING(ON REAL MACHINE)

Code Applied

CLEAN VM

STATE A STATE BCode Run Run TestsCreate VM

SNAPSHOT

http://auxesis.github.com/cucumber-nagios/Thursday 15 November 12

RE-USABLE CUCUMBER STEPS

https://github.com/hedgehog/cukenhttps://github.com/cucumber/aruba

http://www.cucumber-chef.org/

https://github.com/calavera/minitest-chef-handlerhttps://github.com/jedi4ever/puppet-assert

Validate “inside” the provision-run cycle

VM CREATION IS EXPENSIVE

https://github.com/jedi4ever/saharahttps://github.com/t9md/vagrant-snap

Code Applied

CLEAN VM

STATE A STATE BCode Run Run TestsCreate VM

SNAPSHOT

ROLLBACK

https://github.com/rtyler/vagrant-pluginThursday 15 November 12

https://travis-ci.org/

Uses vagrant & veewee

https://github.com/tknerr/bills-kitchen

Windows

TARGETED TESTINGIMPACT OF CHANGE

https://github.com/jedi4ever/puppet-cichttps://github.com/jonlives/knife-preflight

MODULEA

MODULEB

MODULEC

MODULE...

Real VM testingis expensive

Only test what’s needed

PERFORMANCE/METRICS

https://github.com/rodjek/puppet-profilerThursday 15 November 12

https://github.com/joemiller/puppet-graphite_eventhttps://github.com/imeyer/chef-handler-graphite

https://github.com/krux/puppet-module-graphite-report

execution timenumber of classes

Infrastructure as Code Patrick Deboishttp://www.tumblr.com/tagged/star-trek-quotes

THANKYOU!

QUESTIONS?

http://foodfightshow.org/

http://wiki.opscode.com/display/chef/Whyrun+TestingThursday 15 November 12

https://github.com/lak/puppet-memcachedThursday 15 November 12

https://github.com/lak/puppet-static-compilerThursday 15 November 12

http://puppetlabs.com/blog/what-is-user-experience-in-puppet/

AUDITING

http://www.normation.com/en/solutions/rudderhttp://www.fusioninventory.org/

http://comodit.github.com/synapse-agent/Thursday 15 November 12

http://saltstack.org/Thursday 15 November 12

https://github.com/tobami/littlechefThursday 15 November 12

https://github.com/dwt/vagrant-hostsThursday 15 November 12

https://github.com/opscode/omnibusThursday 15 November 12

https://github.com/puppetlabs/puppet-module-toolThursday 15 November 12

http://wiki.opscode.com/display/chef/Recipes#Recipes-ExceptionsandLogging

http://projects.puppetlabs.com/projects/puppet/wiki/Anchor_Pattern

VENDORING

https://github.com/railsmachine/rumpThursday 15 November 12

http://www.example42.com/?q=Puppi_A_Puppet_module_for_Deployment_AutomationThursday 15 November 12

http://rundeck.org/Thursday 15 November 12

http://www.sonian.com/cloud-monitoring-sensu/Thursday 15 November 12

http://docs.puppetlabs.com/guides/parameterized_classes.htmlThursday 15 November 12

https://github.com/rerun/rerunThursday 15 November 12

AUTHENTICATION

DASHBOARDS

http://puppetlabs.com/puppet/related-projects/dashboard/Thursday 15 November 12

http://blog.mornati.net/2012/01/29/kermit-a-webui-for-mcollective/

http://www.opscode.com/hosted-chef/Thursday 15 November 12

http://theforeman.org/Thursday 15 November 12

https://github.com/chapmanb/cloudbiolinuxThursday 15 November 12

https://github.com/aetherical/nimblestratusThursday 15 November 12

https://github.com/fiddyspence/puppet-mconotify

Code but now as we know it - Devoxx edition

Technology

KNOW ME NOW, DIGITALLY

Devoxx 2013 - Hazelcast

Devoxx Real-time Learning

Gaddis We Now Know

Spring Actionscript at Devoxx

Keynote Devoxx 2016

Devoxx Devops University 2012

Devoxx - The Groovy Ecosystem

Vert.x devoxx london 2013

Devoxx devops university_2012

Année N° RUBRIQUE TITRE ARTICLE - … 208 Devoxx Retours sur Devoxx FR 2017 2017 208 Devoxx Loan Tricot et le Machine learning 2017 208 Devoxx Mon expérience sur Devoxx 2017

Vaadin roadmap-devoxx-2013

JSR363 - Devoxx US

Devoxx UK BOF session

211info: now you know

Devoxx Activiti in Action

IzPack at Devoxx 2010

openHAB @ Devoxx 2012

Devoxx 2014 presentation

Kotlin @ Devoxx 2011