Puppet OpenStack<3 Fest

by Dan Bodebodepd <at > [‘freenode.net’, ‘twitter’, ‘github’]

OpenStack?

What is OpenStack?

Open source project

Self service API for your infrastructure

OpenStack Components

•Compute (Nova)•Image (Glance)•Identity (Keystone)•Dashboard (Horizon)•Object Store (Swift)

NovaOpenStack Compute

Schedules allocation of compute resources.

Nova API RabbitMQ

Scheduler

Network

ComputeComputeCompute

VolumesVolumes

NovaSupports management of VM instances, volumes, and networks

Nova API RabbitMQ

Scheduler

Network

ComputeComputeCompute

VolumesVolumes

list instancecreate instance

terminate instance

create network

create volumeattach volume

Glance

GlanceAPI

GlanceRegistry

Store

Swift

FileSystem

Storage service for Virtual machine images

Glance

GlanceAPI

GlanceRegistry

Store

Swift

FileSystem

Storage service for Virtual machine images

AMI Store

AMI

Retrieve

Keystone

Manages authentication/authorization

Keystone API

Authenticate

Token

Tenant

User

Role

Service

Endpoint

HorizonProvides a dashboard for the OpenStack components.

Why Integrate?

OpenStack is hard to build reliably, repeatedly

OpenStack is hard

Hardware

Self Service API

Configuration Management

OpenStack makes it easier to automate the management of compute resources

OpenStack

PuppetYour Business

Puppet can be used to codify best practices for OpenStack

OpenStackiosNow with best practices by:CERN,Enovance,CiscoRedhat,Cybera

BuildingOpenStackModules

Part One:

Creating a community

Because you’re the experts

“The first step towards starting an open source community is to write some code”

User[keystone]

Package[keystone]

File[/etc/keystone/keystone.conf]

Class[keystone]

Interface - bind_host - public_port - admin_port - admin_token - compute_port

Group[keystone]

Service[keystone]

Create classes that wrap all of the OpenStack components

All of them:

• nova

• swift

• glance

• keystone

• horizon

• openstack

• rabbitmq

• mysql

• memcache

• apt

• concat

• ntp

• rsync

• stdlib

• xinetd

Class[openstack::controller]

keystone

mysql::serverglance::api

rabbit::queue

glance::registry

nova::api

nova::scheduler

nova::network::flatdhcp

memcached

horizon

Class[openstack::compute]

nova::compute::libvirt

nova::volume::iscsi

Create Higher level ‘role’ classes

“Design with community in mind”

repos:  repo_paths:    # openstack git repos    git://github.com/puppetlabs/puppetlabs-nova: nova    git://github.com/puppetlabs/puppetlabs-glance: glance    git://github.com/puppetlabs/puppetlabs-swift: swift    git://github.com/puppetlabs/puppetlabs-keystone: keystone    git://github.com/puppetlabs/puppetlabs-horizon: horizon    # openstack middleware    git://github.com/puppetlabs/puppetlabs-rabbitmq: rabbitmq    git://github.com/puppetlabs/puppetlabs-mysql: mysql    git://github.com/puppetlabs/puppetlabs-git: git    git://github.com/puppetlabs/puppetlabs-vcsrepo: vcsrepo    git://github.com/saz/puppet-memcached: memcached    git://github.com/puppetlabs/puppetlabs-rsync: rsync    # other deps    git://github.com/ghoneycutt/puppet-xinetd: xinetd    git://github.com/saz/puppet-ssh: ssh    git://github.com/puppetlabs/puppetlabs-stdlib: stdlib    git://github.com/puppetlabs/puppetlabs-apt: apt    git://github.com/ripienaar/puppet-concat: concat    git://github.com/duritong/puppet-sysctl.git: sysctl

Base everything on small reusable components

class nova(  $nova_cluster_id='localcluster',  $sql_connection = false,  $image_service = 'nova.image.glance.GlanceImageService',  $glance_api_servers = 'localhost:9292',  $rabbit_host = 'localhost',  $rabbit_password='guest',  $rabbit_port='5672',  $rabbit_userid='guest',  $rabbit_virtual_host='/',  $auth_strategy = 'keystone',  $service_down_time = 60,  $logdir = '/var/log/nova',  $state_path = '/var/lib/nova',  $lock_path = $::nova::params::lock_path,  $verbose = false,  $periodic_interval = '60',  $report_interval = '10',  $root_helper = $::nova::params::root_helper){...}

Make them flexible

Make it obvious where extensions go

glance|-- manifests| |-- db| | |-- mysql.pp| | `--postgresql.pp| |-- backend| | |-- file.pp| | `-- swift.pp

Make it obvious how to add additional platforms

class nova::params {

  case $::osfamily {    'RedHat': { # package names      $api_package_name = false      $cert_package_name = false      $common_package_name = 'openstack-nova'      $compute_package_name = false      ...     }    'Debian': { # package names      $api_package_name = 'nova-api'      $cert_package_name = 'nova-cert'      $common_package_name = 'nova-common'      $compute_package_name = 'nova-compute'      ...

“Fear the Fork!”

“Git those forkers!”

300+ merged pull requests

Whoa, we need tests

Rspec Puppet

Puppet

Facts

Data

Model

Emulate Facter

Model

Puppet

Facts

Datalet :facts do {:operatingsystem => ‘RedHat’}end

Specify Data

Puppet

Facts

Datalet :params do {:rabbit_host => ‘10.0.0.42’}end

Model

Inspect catalog

Model

Puppet

Facts

Datait do should contain_file (‘/etc/nova.conf’).\ with_content (‘rabbit_host = 10.0.0.42’)end

Automated unit tests

Integration Tests

Tempest

GitHub

Puppet

Jenkins

Next Steps

• Start connecting the community• get more collaboration on the mailing list• grant community members merge rights

• Single set of “preferred community modules” for• monitoring• HA

Part 2: Bare Metal

Gotta hit bare metal at some point

Razor

• automatically boots a machine in a micro-OS with facter running

• Machines are automatically discovered and turned into an inventory service that can be programmatically installed.

Moving towards Reference Architectures

Puppet

OpenStack

Razor/Puppet/Facter

Hardware

You Business, Automated

• http://puppetlabs.com/blog/module-of-the-week-puppetlabs-openstack/

• http://forge.puppetlabs.com/puppetlabs/openstack

• https://github.com/puppetlabs/puppetlabs-openstack

Resources: