Upload
martin-alfke
View
2.623
Download
0
Embed Size (px)
Citation preview
PuppetAutomated System Configuration Management
Martin Alfke <[email protected]>
1
Wednesday, December 8, 2010
Agenda• Part I - Puppet Basics
• General + Communication
• Manifests, Modules, Templates + Functions
• Part II - Puppet Workshop
• Part III - Working with Puppet
• GIT/SVN for Puppet
• Production / Test / Development
• Monitoring
2
Wednesday, December 8, 2010
General• “Put simply, Puppet is a system for automating
system administration tasks”
• Puppet...
• is a declarative language for expressing system configuration
• is a client-server distribution
• Requirements:
• Ruby > 1.8.1 < 1.9
• Facter
3
Wednesday, December 8, 2010
Communication• Security
• SSL certificate based authentication
• manual signing of certificate requests
• Layers:
• Configuration Language
• Transaction layer
• Resource Abstraction Layer
4
Wednesday, December 8, 2010
Supported Platforms• Linux
• Debian / Ubuntu / Fedora / CentOS / RHEL / OEL / Mandriva / SuSE / Gentoo
• BSD
• FreeBSD / OpenBSD
• Other Unix
• OS X / Solaris / HP-UX
• Windows - coming in 2010
5
Wednesday, December 8, 2010
Functional Overview• Clients connect to
Puppet Master
• Puppet Master send clients description of tasks
• Puppet Master stores Clients reports
• Reports can be imported into dashboard database
• Dashboard web interface to reports
6
Wednesday, December 8, 2010
Facter
7
/usr/bin/facterarchitecture => amd64domain => buero20.localfacterversion => 1.5.7fqdn => puppet.buero20.local...interfaces => eth0,eth1ipaddress => 10.0.2.15...operatingsystem => Debianprocessorcount => 1
Wednesday, December 8, 2010
Puppet Configuration Language - 1-6
8
• manifests/site.pp
• Global file with node definitions
• modules/<name>/manifests/init.pp
• Module initialization
• Use lower case for names (modules, templates, functions, defines, exec, resources,...)
Wednesday, December 8, 2010
Puppet Configuration Language - 2-6
9
• Resources
• user - create or remove users
• group - create or remove groups
• package install or remove distribution packages
• file - create directories, symlinks, copy files
• cron - add cron jobs
• service - run or stop services like daemons
Wednesday, December 8, 2010
Puppet Configuration Language - 3-6
10
• Classes
• aggregate resources for easier use
• subclasses (=nested classes) for modularity
• parameterised classes for more flexible handling
• classes support inheritance
Wednesday, December 8, 2010
Puppet Configuration Language - 4-6
11
• Definitions
• reusable objects
• Modules
• combine collections of resources, classes and definitions
Wednesday, December 8, 2010
Puppet Configuration Language - 5-6
12
• Chaining resources
• make sure that a service is restarted after filechange
• make sure that config file is copied prior starting a service
• make sure that a package is installed prior starting the service
Wednesday, December 8, 2010
Puppet Configuration Language - 6-6
13
• Nodes
• connect modules and clases to systems
• nodenames are short hostname, fqdn or “default”
Wednesday, December 8, 2010
Manifests
• Define static resourcesfile { “/etc/passwd”:
owner => root,group => root,mode => 644,
}
• Static resources have full path and name.
14
Wednesday, December 8, 2010
Manifests with facter Variables
file { “sshconfig”:name => $operatingsystem ? {
solaris => “/usr/local/etc/ssh/sshd_config”,default => “/etc/ssh/sshd_config”,
},owner => root,group => root,mode => 644,
}
• Using facter variables inside a definition
15
Wednesday, December 8, 2010
Manifest with Sub-Classesclass mysql {
class client { class packages { package { "mysql-client": ensure => installed } } } class server { class packages { package { "mysql-server": ensure => installed } package { "mysql-common": ensure => installed }
16
Wednesday, December 8, 2010
Manifests with Exec
file {"/etc/apt/keys/pgp_key.asc": owner => root, group => root, mode => 640, source => "puppet://$server/files/etc/apt/keys/pgp_key.asc"}exec { "/usr/bin/apt-key add /etc/apt/keys/pgp_key.asc": unless => "/bin/sh -c '[ `/usr/bin/apt-key list | grep buildd | \ wc -l` -eq 1 ]'"}
17
Wednesday, December 8, 2010
Manifests with Subscriptionfile {"/etc/apt/keys/puppet.key": owner => root, group => root, mode => 640, source => "puppet:///files/etc/apt/keys/puppet.key"}exec { subscribe-base-config-puppet-key: command => "/usr/bin/apt-key add /etc/apt/keys/puppet.key; \ /usr/bin/apt-get update", logoutput => false, refreshonly => true, subscribe => File["/etc/apt/keys/puppet.key"]}
18
Wednesday, December 8, 2010
Modules - Directory structure
• Directory structure - e.g. /etc/ssh/sshd_configmodule/sshd/
manifests/init.pp
files/etc/
ssh/sshd_config
• Modules require strict directories naming.
19
Wednesday, December 8, 2010
Modules - Initialization Manifest
• modules/manifests/sshd/init.ppclass sshd {
file { “/etc/ssh/sshd_config”:mode => 644,source => “puppet:///modules/sshd/etc/ssh/sshd_config”,
}}
• init.pp manifest will be integrated automatically when class name is equal to module name
20
Wednesday, December 8, 2010
Templates - Directory Structure
• Directory structure + content - e.g. Network settingsnetwork/
manifests/init.pp
templates/network.erb
• Templates require strict directory naming (like modules)
21
Wednesday, December 8, 2010
Templates - Initialization Manifest
• Manifests - init.ppfile { “/etc/sysconfig/network”:
content => template(“templates/network.erb”),}
• Templatess - network.erbNETWORKING=yesHOSTNAME=<%= hostname %>NOZEROCONF=yes
• Templates may use facter variables
22
Wednesday, December 8, 2010
Functions• Directory structure e.g. read parameter
from configuration file using facter:lib/
facter/function.rb
• Content of library functions function.rb:require ‘facter’ Facter.add(“PUPPET_FUNCTION”) do %x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/puppet_function | sed -e ‘s/*.=//’ } .chomp endend
23
Wednesday, December 8, 2010
Agenda• Part I - Puppet Basics
• General + Communication
• Manifests, Modules, Templates + Functions
• Part II - Puppet Workshop
• Part III - Working with Puppet
• GIT/SVN for Puppet
• Production / Test / Development
• Monitoring
24
Wednesday, December 8, 2010
Puppet Workshop• Installation - Puppet master and client on puppet master only
• Initialization
• Installation - Puppet client on puppet client only
• Modules
• User Management
• Apache sites configuration
• Templating for /etc/hosts
• Setup Reporting and Dashboard
25
Wednesday, December 8, 2010
Puppet Workshop - Installation - 1-5
•check requirements:
• ruby --version
• ruby -rshadow -e’print “OK\n”’
26
Wednesday, December 8, 2010
Puppet Workshop - Installation - 2-5
•from source
• fetch and extract source
• wget http://puppetlabs.com/downloads/facter/facter-1.5.8.tar.gz
• wget http://puppetlabs.com/downloads/puppet/puppet-2.6.2.tar.gz
27
Wednesday, December 8, 2010
Puppet Workshop - Installation - 3-5
• install
• ruby install.rb
• mkdir /etc/puppet
28
Wednesday, December 8, 2010
Puppet Workshop - Installation - 4-5
•configuration
• puppet --mkuser
• puppet --genconfig > /etc/puppet/puppet.conf
• vi /etc/hosts - add entry for nodename puppet if not existing
29
Wednesday, December 8, 2010
Puppet Workshop - Installation - 5-5
•manifests/site.pp
• add empty section for default nodenode default {
notice(“default node”)}
30
Wednesday, December 8, 2010
Puppet Workshop - Initialization
•first start of puppet:
• puppetd --test
•puppet CA
• check client certificate
• puppetca --list
• puppetca --list --all
31
Wednesday, December 8, 2010
Puppet Workshop - Modules - 1-2
•File Structure
• mkdir -p modules/<name>/{manifests,files}
•modules/<name>/manifests/init.ppclass <name> {
notice(“module <name>”)}
32
Wednesday, December 8, 2010
Puppet Workshop - Modules - 2-2
•including modules in manifests/site.ppnode default {
include <name>}
33
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 1-6
• User Management
• create your personal login
• create home directory
1. Module directoriesmkdir -p modules/users/{manifests,files}
34
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 2-62. Module init.pp
vi modules/users/manifests/init.pp
class users {user{ "martin":!home!! ! ! ! ! => "/home/martin",!managehome! => true,!shell! ! ! ! ! ! => "/bin/bash",!comment!! ! ! => "Martin Alfke",!ensure! ! ! ! ! => present,
35
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 3-6
#!uid! ! ! ! ! ! => 0,#!gid !! ! ! ! ! => 0,# password ! ! => '0OfNn.f5krlF2',#!allowdupe ! ! => true,
}}
36
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 4-6
3. modify site.pp
vi manifests/site.pp
node default {! include users}
37
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 5-61. create new filemkdir -p modules/users/files/home/martin/www/
cat > modules/users/files/home/martin/www/index.html << EOF<html><head><title>My testsite</title></head><body>foo</body></html>EOF
38
Wednesday, December 8, 2010
Puppet Workshop - Account Module - 6-62. Module init.ppadd to modules/users/manifests/init.ppclass users {
......file {“/home/martin/www”:! ensure => directory,}file{“home/martin/www/index.html”:! source => “puppet:///modules/users/home/martin/www/index.html”,}
}
39
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 1-6• Apache sites Management
• packages
• your own vhost config
1. Module directoriesmkdir -p modules/apache/{manifests,files}mkdir -p modules/apache/files/etc/apache2/sites-available/
40
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 2-62. your vhost definition
cat > modules/apache/files/etc/apache2/sites-available/blit-test << EOFListen 88NameVirtualHost *:88<VirtualHost *:88>! DocumentRoot /home/martin/www</VirtualHost>EOF
41
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 3-62. Module init.ppvi modules/apache/manifests/init.pp
class apache {! package{“apache2”: ensure!! ! => present }! package{“php5-mysql”: ensure! => present }! file{“/etc/apache2/sites-available/blit-test”:! ! source => “puppet:///modules/apache/etc/apache2/sites-available/blit-test”,! }}
42
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 4-6
3. Add to node default manifest site.pp
include apache
43
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 5-6• Apache sites Management
• enabling sites with function1. Add to apache init.ppclass apache {...define vhost ($ensure = ʻpresentʼ) {! case $ensure {! ! ʻpresentʼ: {! ! exec { “/usr/sbin/a2ensite $name”:! ! ! unless => “/bin/readlink -e /etc/apache2/sites-enabled/$name”! ! }! ! }
44
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 5-6
! ! ʻabsentʼ: {! ! ! exec { “/usr/sbin/a2dissite $name”:! ! ! ! onlyif => “/bin/readlink -e /etc/apache2/sites-enabled/$name”! ! ! }! ! }! ! default: { err (“Unknown ensure value: $ensure) }! }
45
Wednesday, December 8, 2010
Puppet Workshop - Apache Module - 6-6
! vhost {“blit-test”:! ! ensure => “present”,! }! vhost{“000-default”:! ! ensure => absent,! }}
46
Wednesday, December 8, 2010
Puppet Workshop - Templates - 1-2
• File Structure
• mkdir -p modules/<name>/{manifests,templates}
• modules/<name>/manifests/init.ppclass <name> {
notice(“module <name>”)}
47
Wednesday, December 8, 2010
Puppet Workshop - Templates - 2-2
• including modules in manifests/site.ppnode default {
include <name>}
48
Wednesday, December 8, 2010
Puppet Workshop - Hosts Template - 1-3
• Hosts Template
• configure entries in /etc/host
1. Module directoriesmkdir -p modules/hosts/{manifests,templates}
49
Wednesday, December 8, 2010
Puppet Workshop - Hosts Template - 2-3
2. Module init.pp
vi modules/hosts/manifests/init.pp
class hosts {! file{“/etc/hosts”:! ! owner! => root,! ! group!! => root,! ! content!=> template(hosts.erb),! }}
50
Wednesday, December 8, 2010
Puppet Workshop - Hosts Template - 3-3
3. template hosts.erb
vi templates/hosts.erb
127.0.0.1!localhost<%= ipaddress %>!<%= fqdn %> <%= hostname %>192.168.0.2! puppet192.168.0.4! mysql! mysqlmaster
51
Wednesday, December 8, 2010
Puppet Workshop - Functions - 1-2
• File Structure
• mkdir -p modules/<name>/lib/
52
Wednesday, December 8, 2010
Puppet Workshop - Functions - 2-2
• including modules in manifests/site.ppnode default {
include <name>}
53
Wednesday, December 8, 2010
Puppet Workshop - Facter Function - 1-4
• Facter Function
• provide additional fact
1. Module directoriesmkdir -p modules/facter/lib/facter
54
Wednesday, December 8, 2010
Puppet Workshop - Facter Function - 2-4
2. function.rb
vi modules/facter/lib/facter/function.rb
require ʻfacterʼFacter.add(“PUPPET_FUNCTION”) do! setcode do! ! %x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/puppet_function | sed -e ʻs/.*=//ʼ}.chomp! endend
55
Wednesday, December 8, 2010
Puppet Workshop - Facter Function - 3-4
3. puppet.confsection [main]pluginsync = true
4. puppet runpuppetd --test
5. call facter puppet functionfacter --puppet | grep puppet_function
56
Wednesday, December 8, 2010
Puppet Workshop - Facter Function - 4-4
3. use custom facts in manifestscase $puppet_function {! “MYSQL”:! ! { include mysql }! “APACHE”:!! { include apache }! “PROXY”:! ! { include proxy }...}
57
Wednesday, December 8, 2010
Puppet Workshop - Dashboard - 1-5
• Installation
• fetch and extract sourc
• wget http://puppetlabs.com/downloads/dashboard/puppet-dashboard-1.0.4.tgz
• install mysql-server
58
Wednesday, December 8, 2010
Puppet Workshop - Dashboard - 2-5
• Configuration
• edit /usr/share/puppet-dashboard/config/database.yaml
• Create Database
• cd /usr/share/puppet-dashboard; rake RAILS_ENV db:create or
• mysql -Ne ‘create database dashboard;’
59
Wednesday, December 8, 2010
Puppet Workshop - Dashboard - 3-5
• Initialize Database
• cd /usr/share/puppet-dashboard; rake RAILS_ENV db:migrate
• Import Reports
• cd /usr/share/puppet-dashboard; rake RAILS_ENV=production reports:import
60
Wednesday, December 8, 2010
Puppet Workshop - Dashboard - 4-5
• Start service
• cd /usr/share/puppet-dashboard; ./bin/server -e production -d
• Review your Dashboard in browser
• http://<your puppetmaster ip>:3000/
61
Wednesday, December 8, 2010
Puppet Workshop - Dashboard - 5-5
• add error to manifest (e.g. point source to a non existing file)
• run puppetd
• puppetd --test
• import data
• cd /usr/share/puppet-dashboard; rake RAILS_ENV=production reports:import
• review dashboard
62
Wednesday, December 8, 2010
Agenda• Part I - Puppet Basics
• General + Communication
• Manifests, Modules, Templates + Functions
• Part II - Puppet Workshop
• Part III - Working with Puppet
• GIT/SVN for Puppet
• Production / Test / Development
• Monitoring
63
Wednesday, December 8, 2010
Puppet into GIT/SVN
• Why revision control system?
• Co-working
• Branches
• Which RCS System?
• Which ever you prefer
64
Wednesday, December 8, 2010
Puppet Staging• Production, Test and Development
• /etc/puppet/puppet.conf
• [main] - environment = ...
• [development] - modulepath=/etc/puppet/development/modules
• [testing] - modulepath=/etc/puppet/testing/modules
• [production] - modulepath=/etc/puppet/production/modules
65
Wednesday, December 8, 2010
Puppet Monitoring
• Puppet Dashboard
• Configure puppet to store results
• [master] section: reports=http, store
• [agent] (v2.6) or [puppetd] section: report=true
• Configure Database (e.g. MySQL)
66
Wednesday, December 8, 2010
Puppet Dashboard
67
Wednesday, December 8, 2010
PuppetAutomated System Configuration Management
Martin Alfke <[email protected]>
Thank you !Questions ?
68
Wednesday, December 8, 2010