Declare Independence from your IT department: SysAdmin skills for PHP developers
Pablo Godel @pgodel - June 29th, 2013Lone Star PHP Conference
http://joind.in/8696
Saturday, June 29, 13
- Why SysAdmin Skills- Introduction to LAMP- Hosting options- Linux distributions- Installing PHP options- Deploying PHP- Various production tips- Monitoring- Backups
Agenda
Saturday, June 29, 13
Who Am I?
⁃ Born in Argentina, living in the US since 1999⁃ PHP & Symfony developer
⁃ Founder of the original PHP mailing list in spanish ⁃ Parrilla Lover
⁃ Co-founder of ServerGrove
Saturday, June 29, 13
Saturday, June 29, 13
Saturday, June 29, 13
⁃ Founded ServerGrove Networks in 2005
⁃ Provider of web hosting specialized in PHP, Symfony, ZendFramework, MongoDB and others
⁃ Servers in USA and Europe!
ServerGrove!
Saturday, June 29, 13
Very active open source supporter through codecontributions and usergroups/conference sponsoring
Community is our teacher
Saturday, June 29, 13
Sysadmin Skills
Who needs them?
Saturday, June 29, 13
Text
IT is busy / Different PrioritiesSaturday, June 29, 13
!Rocket ScienceSaturday, June 29, 13
Lack of controlSaturday, June 29, 13
Less downtimeSaturday, June 29, 13
Better PerformanceSaturday, June 29, 13
What do you prefer?Saturday, June 29, 13
Or this?Saturday, June 29, 13
An example...
I s. I ’ s L. E Cs , .
Saturday, June 29, 13
LAMPWhat is
Saturday, June 29, 13
Saturday, June 29, 13
Saturday, June 29, 13
LAMPSaturday, June 29, 13
LAMPSaturday, June 29, 13
LAMPSaturday, June 29, 13
LAMP Scott Beale
Saturday, June 29, 13
LNMPSaturday, June 29, 13
LNMPSaturday, June 29, 13
LNPPSaturday, June 29, 13
LANMMPPSaturday, June 29, 13
LANMMPPWhy?
Saturday, June 29, 13
Why LANMMPP
Proven
Saturday, June 29, 13
Stable
Why LANMMPP
Saturday, June 29, 13
Secure
Why LANMMPP
Saturday, June 29, 13
Why LANMMPP
Free
Saturday, June 29, 13
Free*
Why LANMMPP
Saturday, June 29, 13
Inexpensive
Why LANMMPP
Saturday, June 29, 13
Easily Available
Why LANMMPP
Saturday, June 29, 13
Hosting Options
Saturday, June 29, 13
Hosting OptionsShared Hosting
- Inexpensive- Widely available- Easy to use- No maintenance- No updates- Instant provisioning- DNS/Email services
- Shared resources- No/Limited Shell- Many restrictions- No control- Fixed versions - Multitude of CPs
+ -
Saturday, June 29, 13
Hosting OptionsDedicated Servers
- Full control- No sharing- Best performance- Software / Hardware to specs
- Hard to plan- Software updates- SysAdmin skills- Expensive to start- Parts replacements- Control Panel licenses
+ -
Saturday, June 29, 13
Hosting OptionsVirtual Servers / VPS
- Shared resources- Software updates- SysAdmin skills- Limited/No CP- Lack of email
-
Saturday, June 29, 13
Hosting OptionsVirtual Servers / VPS
- Inexpensive- Widely available- Choice of OS- Root access- Full shell- Fast provisioning- Custom software
- Custom configurations- Snapshots- Multiple Envs
+ +
Saturday, June 29, 13
Hosting OptionsCloud Hosting
Depends on implementation
+ -
Saturday, June 29, 13
Hosting OptionsCloud Hosting
Some are a shared hosting with a fancy new name
Saturday, June 29, 13
Hosting OptionsCloud Hosting
Some are a shared hosting with a fancy new nameOthers let you start small and scale easily
Saturday, June 29, 13
Hosting OptionsCloud Hosting
Some are a shared hosting with a fancy new nameOthers let you start small and scale easilyGets really expensive at scale
Saturday, June 29, 13
Hosting OptionsCloud Hosting
Most suffer from some of the shared hosting problems:- vendor lock-in- lack of control- high learning curve- limited environment
Saturday, June 29, 13
Hosting Options
Which one to choose?
Saturday, June 29, 13
Hosting OptionsWhich one to choose?
I T D E P E N D S
Saturday, June 29, 13
Linux Distributions
Saturday, June 29, 13
DistributionsRedHat based
- Fedora- RedHat Enterprise Linux (RHEL)- CentOS- Scientific Linux
Saturday, June 29, 13
DistributionsDebian based
- Debian- Ubuntu
Saturday, June 29, 13
DistributionsOthers
- Gentoo- Slackware- OpenSuse- Archlinux- any many many more!
http://distrowatch.com/http://en.wikipedia.org/wiki/Linux_distribution
Saturday, June 29, 13
DistributionsKey differences- Configuration files- Package Manager- Managing Services- Software versions- Updates
Saturday, June 29, 13
DistributionsKey differences
- Configuration files
CentOS
Ubuntu
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/network/interfaces
Saturday, June 29, 13
DistributionsKey differences
- Package Manager
CentOS
Ubuntu
yum install php-cli
apt-get install php5
Saturday, June 29, 13
DistributionsKey differences
- Package Manager
CentOS
Ubuntu
yum search php-cli
apt-cache search php
Saturday, June 29, 13
DistributionsKey differences
- Managing Services
CentOS
Ubuntu
service restart httpdchkconfig --list httpdchkconfig httpd on
service restart apache2sudo update-rc.d apache2 enable
Saturday, June 29, 13
DistributionsKey differences
- Software versions
CentOS
Ubuntu
httpd-2.2.3
apache-2.2.17
Saturday, June 29, 13
DistributionsKey differences
- Updates
CentOS 5.x
Ubuntu 11
PHP 5.1.6
PHP 5.3.5
Saturday, June 29, 13
DistributionsKey similarities
- Based on Linux Kernel
- Screwed up or ancient PHP- Lack of PHP extensions
Saturday, June 29, 13
DistributionsWhich one to choose?
I T D E P E N D S
Saturday, June 29, 13
DistributionsWhich one to choose?
- Best feeling- Previous experience- Workplace selection- Coworker / friend knowledge
Saturday, June 29, 13
First Experience
Saturday, June 29, 13
First Experience
- Run Linux natively- Mac OSX- Virtual Machine: - VirtualBox - Parallels - VMware- VPS
Saturday, June 29, 13
First Experience#!/bin/bash
Saturday, June 29, 13
First Experience#!/bin/bash
Saturday, June 29, 13
First Experience#!/bin/bash
Saturday, June 29, 13
First Experience#!/bin/bash
Saturday, June 29, 13
First Experience#!/bin/bash
Recommended Books:- Learning the bash Shell- bash Cookbook
Saturday, June 29, 13
First Experience#!/bin/bash
Most Common Commands:
$ ls -l /path/$ cd /path$ pwd$ cat /etc/passwd$ less /etc/resolv.conf$ tail -f /var/log/*$ last$ lastb$ vi /etc/hosts
Saturday, June 29, 13
First Experience#!/bin/bash
User`s config files:
~/.bash_history~/.bash_logout~/.bash_profile PATH=$PATH:$HOME/bin export PATH
~/.bashrc alias rm='rm -i' alias cp='cp -i' alias mv='mv -i'
Saturday, June 29, 13
First Experience#!/bin/bash
Most common directories:
/etc /tmp/var/log/var/run/root/home/usr/local/opt
Saturday, June 29, 13
Installing Apache/PHP
Saturday, June 29, 13
Installing Apache/PHP# CentOS$ yum install httpd php-cli mod_php
# Ubuntu$ apt-get install apache2 libapache2-mod-php5
Mac OSX- MAMP- Native Apache + Liip PHP installation- Zend Server CE
Windows- WAMP- IIS + PHP
Saturday, June 29, 13
Installing PHPCentOS
Third Party Repositories- Remi RPMhttp://blog.famillecollet.com/- Webtatichttp://www.webtatic.com/- ServerGrove: PHP 5.3.x / 5.4.x / 5.5.x (always latest)http://repos.servergrove.com
Saturday, June 29, 13
Installing PHPCentOS
Third Party Repositories
cd /etc/yum.repos.d/wget http://repos.servergrove.com/servergrove-centos-5/servergrove-centos-5.repoyum install php53 # or php54
Saturday, June 29, 13
Installing PHPUbuntu
Third Party Repositories
- Dotdeb:http://www.dotdeb.org/- ServerGrove: (always latest)http://repos.servergrove.com
PHP 5.3.x / 5.4.x / 5.5.x
Saturday, June 29, 13
Installing PHPUbuntu
Third Party Repositories
echo “deb http://repos.servergrove.com/servergrove-ubuntu-precise precise main” >> /etc/apt/sources.list.d/servergrove.listapt-get install php53 # or php54
Saturday, June 29, 13
Installing PHPFrom Source
wget http://us2.php.net/get/php-5.4.12.tar.bz2/from/www.php.net/mirrortar jxvf php-5.4.12.tar.bz2cd php-5.4.12./configure make && make install
Saturday, June 29, 13
Installing PHPRecompiling
php -i |grep configure./configure '--with-apxs2=/usr/sbin/apxs' '--prefix=/usr/local/php53' '--with-config-file-scan-dir=/etc/php53/conf.d' '--enable-bcmath' '--enable-ctype' '--enable-exif' '--enable-mbstring' '--enable-ftp' '--enable-intl' '--enable-sockets' '--enable-sysvmsg' '--enable-pcntl' '--with-bz2' '--with-curl' '--with-gettext' '--with-gd' '--enable-gd-native-ttf' '--enable-exif' '--with-freetype-dir=/usr' '--with-jpeg-dir=/usr' '--with-t1lib=/usr' '--with-mcrypt' '--with-openssl' '--with-kerberos' '--with-iconv' '--with-xsl' '--with-xmlrpc' '--with-zlib' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd'make && make install
Saturday, June 29, 13
Installing PHPCompiling for Apache
./configure '--with-apxs2=/usr/sbin/apxs' make && make install
Saturday, June 29, 13
Installing PHPCompiling for Nginx / PHP-FPM
./configure '--enable-fpm' make && make install
Saturday, June 29, 13
Installing PHPCompiling extension
./configure --with-curl --enable-ftpmake && make install
Saturday, June 29, 13
Installing PHPCompiling extension
(dynamic)
cd ext/curlphpize./configure make && make installecho “extension=curl.so” >> php.ini
Saturday, June 29, 13
Installing PHPCompiling extension
http://pecl.php.net/
Saturday, June 29, 13
Installing PHPCompiling extension
(PECL)pecl install apc# orpecl download apctar zxvf APC-3.1.13.tgzcd APC-3.13phpize./configure make && make installecho “extension=apc.so” >> php.ini
Saturday, June 29, 13
Automate!
- Chef- Puppet- ...
Installing PHP
Saturday, June 29, 13
Installing PHPConfiguration
/etc/php/php.ini/etc/php5/cli/php.ini
/usr/local/lib/php.ini
Default location
Other common locations
/etc/php5/apache2/php.ini
Saturday, June 29, 13
Installing PHPConfiguration
php -i | grep php.iniConfiguration File (php.ini) Path => /usr/local/php5/libLoaded Configuration File => /usr/local/php5-20110426-093151/lib/php.iniScan this dir for additional .ini files => /usr/local/php5/php.dAdditional .ini files parsed => /usr/local/php5/php.d/10-extension_dir.ini,
Saturday, June 29, 13
Installing PHPConfiguration
php -i | grep mongo/usr/local/php5/php.d/50-extension-mongo.ini,mongomongo.allow_empty_keys => 0 => 0mongo.allow_persistent => 1 => 1mongo.auto_reconnect => 1 => 1mongo.chunk_size => 262144 => 262144mongo.cmd => $ => $mongo.default_host => localhost => localhostmongo.default_port => 27017 => 27017
Saturday, June 29, 13
Installing PHPConfiguration
[PHP Modules]apcbcmathbz2Corectypecurldatedomereg
php -m
Saturday, June 29, 13
Installing PHPConfiguration
php.iniextension_dir=/usr/lib/php/extensions/no-debug-non-zts-20090626
extension=apc.soextension=mongo.so
Saturday, June 29, 13
Installing PHPConfiguration
php.ini
php -i | grep extension_dirextension_dir => /usr/local/php5/lib/php/extensions/no-debug-non-zts-20090626
Saturday, June 29, 13
Installing PHPConfiguration
php.ini
date.timezone=UTCdisplay_errors = offlog_errors = onerror_log = /var/log/php.log
Saturday, June 29, 13
Installing PHPConfiguration
Securitymemory_limit = 128Mmax_execution_time = 30display_errors = offexpose_php = offmail.log = /var/log/phpmails.logdisable_functions = execallow_url_fopen = off
Saturday, June 29, 13
Installing PHPConfiguration
File uploads on .htaccessphp_value memory_limit 128Mphp_value max_file_uploads 20php_value max_input_time -1php_value post_max_size 8Mphp_value upload_max_filesize 2Mphp_value max_execution_time 0
AllowOverride=All in Apache!Saturday, June 29, 13
Installing PHPConfiguration
Include .htaccess
<Directory /path/to/document/root> Include /path/to/.htaccess</Directory>
AllowOverride=None in Apache!Saturday, June 29, 13
Installing PHPConfiguration
Apache
php_value date.timezone UTCphp_flag display_errors 1php_value memory_limit 128Mphp_value max_execution_time 0
Don’t forget to restart ApacheSaturday, June 29, 13
Installing PHPWeb Server User
- apache- nobody- www-data- ftp / ssh user (sometimes)
Possible users
Permissions issues with clear cache and uploads, Anyone?
Saturday, June 29, 13
Installing PHPWeb Server User
Fix:
Permissions issues with clear cache and uploads, Anyone?
rm -rf app/cache/*rm -rf app/logs/*
sudo chmod +a "www-data allow delete,write,append,file_inherit,directory_inherit" app/cache app/logssudo chmod +a "`whoami` allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs
http://symfony.com/doc/current/book/installation.html
Saturday, June 29, 13
Deploying PHP
Saturday, June 29, 13
Deploying PHPBefore going live
Make sure ntpd is installed and running to ensure accurate server time
yum install ntpchkconfig ntpd onntpdate pool.ntp.org
Saturday, June 29, 13
Deploying PHPBefore going live
- Disable PHP in specific directories
<Location /uploads> php_admin_flag engine off</Location>
Saturday, June 29, 13
Deploying PHPBefore going live
- Limit access by IP
<Location /admin> Order Deny,Allow Deny from all Allow from 1.2.3.4</Location>
Saturday, June 29, 13
Deploying PHPBefore going live
- Add HTTP Authentication
<Location /admin> Require valid-user AuthType Basic AuthName "SG" AuthUserFile /path/users</Location>
Saturday, June 29, 13
Deploying PHPBefore going live
- Quiet down Apache
ServerTokens Prod
Saturday, June 29, 13
Deploying PHPBefore going live
RewriteEngine On
RewriteRule ^\.htaccess$ - [F]
RewriteCond %{REQUEST_FILENAME} -fRewriteRule ^.*$ - [NC,L]
RewriteCond %{REQUEST_URI} !^/web/.*$RewriteRule ^(.*)$ /web/$1
Move document root with .htaccess
Saturday, June 29, 13
Deploying PHPBefore going live
- Case senstive filesystem
IPCheck_Form_Index_Login != IpCheck_Form_Index_Login
/path/to/file/IpCheck_Form_Index_Login.php<?php class IPCheck_Form_Index_Login {...
Saturday, June 29, 13
Deploying PHP- server-side vi- plain old FTP- SFTP- scp- rsync + ssh- git- tar / gzip- rpm / deb packages- capistrano / capifony
Saturday, June 29, 13
Deploying PHP- server-side vi- plain old FTP- SFTP- scp- rsync + ssh- git- tar / gzip- rpm / deb packages- capistrano / capifony
Saturday, June 29, 13
Deploying PHPEffing Package Management
https://github.com/jordansissel/fpm
Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.
fpm -s dir -t rpm -n "sfapp" -v 1.0 /var/www/sfapp
fpm -s dir -t deb -a all -n sfapp -v 1.0 /etc/apache2/conf.d/my.conf /var/www/sfapp
Saturday, June 29, 13
Deploying PHPCapistrano / Capifony
- multiple servers- multiple environments- setup shared folders (vendors, cache, logs, etc)- copy files- update vendors- multi versions- rollback- restart apache
Saturday, June 29, 13
Deploying PHPSetting up SSH
$ ssh-keygen -t dsa ~/.ssh/id_dsa.pub
$ ssh-copy-id user@remote-host ~/.ssh/authorized_keys ~/.ssh/authorized_keys2
$ ssh-add
Saturday, June 29, 13
Deploying PHPSetting up SSH
~/.ssh/config
Host * ForwardAgent yes Port 22123
Host gh HostName github.com Port 22 PreferredAuthentications publickey IdentityFile ~/.ssh/me_rsa
Saturday, June 29, 13
Deploying PHPSetting up SSH
/etc/ssh/sshd_config
Port 22 Port 22123 PermitRootLogin no PasswordAuthentication no
Saturday, June 29, 13
Deploying PHP Automate!
https://github.com/pgodel/m-sunshinephp/blob/master/web/deploy.php
<?php
exec('/usr/bin/env -i HOME=/var/www/vhosts/m.sunshinephp.com/m-sunshine git pull origin master');
http://m.sunshinephp.com/deploy.php
Saturday, June 29, 13
DNS Tips
Saturday, June 29, 13
DNS Tips The Power of the hosts file
Saturday, June 29, 13
/etc/hosts
10.0.1.1 www.lottery.com
DNS Tips The Power of the hosts file
Saturday, June 29, 13
/etc/hosts
10.0.1.1 example1.com example2.com
DNS Tips The Power of the hosts file
Saturday, June 29, 13
DNS TipsVirtual Document Root
UseCanonicalName Off
VirtualDocumentRoot /var/www/vhosts/%0/web
<Location /var/www/vhosts> AllowOverride All Options +FollowSymLinks</Location>
example.com => /var/www/vhosts/example.com/webexample2.com => /var/www/vhosts/example2.com/web
Saturday, June 29, 13
DNS TipsNameservers/Expiration
whois servergrove.com...Name Servers: ns1.servergrove.com ns2.servergrove.com ns3.servergrove.com
Creation date: 19 May 2005 23:34:36Expiration date: 19 May 2014 23:34:00
Saturday, June 29, 13
DNS TipsDNS records
dig -t A google.com;; ANSWER SECTION:google.com. 184 IN A 74.125.230.227google.com. 184 IN A 74.125.230.228google.com. 184 IN A 74.125.230.229google.com. 184 IN A 74.125.230.230google.com. 184 IN A 74.125.230.231google.com. 184 IN A 74.125.230.232google.com. 184 IN A 74.125.230.233google.com. 184 IN A 74.125.230.238google.com. 184 IN A 74.125.230.224google.com. 184 IN A 74.125.230.225google.com. 184 IN A 74.125.230.226
Saturday, June 29, 13
DNS TipsDNS records
dig -t A servergrove.eu @ns1.servergrove.com;; ANSWER SECTION:servergrove.eu. 3600 IN A 149.5.47.100
Saturday, June 29, 13
DNS Tips DNS Delegationhttp://www.simpledns.com/lookup-dg.aspx
Saturday, June 29, 13
DNS Tipstraceroute
traceroute google.com
traceroute to google.com (173.194.37.33), 30 hops max, 40 byte packets 1 2.69-195-222.static.servergrove.com (69.195.222.2) 0.360 ms 0.365 ms 0.432 ms 2 t0-1-0-5.br2.mia.terremark.net (66.165.161.45) 1.558 ms 1.546 ms 1.532 ms 3 core1-1-0-0.mia.net.google.com (198.32.124.133) 0.238 ms 0.224 ms 0.230 ms 4 209.85.253.74 (209.85.253.74) 0.266 ms 0.283 ms 0.312 ms 5 209.85.254.252 (209.85.254.252) 12.764 ms 12.757 ms 12.749 ms 6 64.233.175.92 (64.233.175.92) 14.177 ms 14.257 ms 14.359 ms 7 atl14s07-in-f1.1e100.net (173.194.37.33) 13.653 ms 13.606 ms 13.618 ms
Saturday, June 29, 13
DNS Tipsmtr
Saturday, June 29, 13
Monitoring
Saturday, June 29, 13
Monitoring
<Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from .your_domain.com</Location>
ExtendedStatus On
Apache Requests
Saturday, June 29, 13
MonitoringApache Requests
Saturday, June 29, 13
Monitoring
- Cacti- Ganglia- Zabbix- collectd- statsd / StatsDBundle- graphite
Saturday, June 29, 13
Monitoringstatsd / StatsDBundle / Graphite
Saturday, June 29, 13
MonitoringCPU / Memory / IO
top
Saturday, June 29, 13
MonitoringIO
iotop
Saturday, June 29, 13
MonitoringNetwork
iptraf
Saturday, June 29, 13
MonitoringHandling logs
- Centralize logs with syslog
error_log = syslog
- Monolog supports syslog- logstash, logster, loggly, logio
Saturday, June 29, 13
MonitoringHandling logs
$ grep POST /var/log/apache2/access_log
Saturday, June 29, 13
Speeding up
Saturday, June 29, 13
Speeding up
- nginx/php-fpm- APC- ZendOptimizer+ (Included in PHP 5.5)- Memcache- nginx reverse proxy cache- Varnish
Saturday, June 29, 13
Backups
Saturday, June 29, 13
Backups
- rsync- rdiff-backup- Unison- Bacula- Amanda
Saturday, June 29, 13
Backups
Don’t forget to backup your DB!
http://blog.servergrove.com/2012/01/24/backup-your-mysql-database-using-mysqldump/
Saturday, June 29, 13
Reading List
- Automating UNIX and Linux Administration- Running Linux - Learning the bash Shell: Unix Shell Programming
Saturday, June 29, 13
The End Questions?
Sysadmin skills for PHP developersSaturday, June 29, 13
http://joind.in/8696 Thank you!
Sysadmin skills for PHP developers
Pablo Godel @pgodel Saturday, June 29, 13
Recommended