Upload
barney-hanlon
View
2.133
Download
5
Embed Size (px)
DESCRIPTION
In this talk, Barney will be discussing and demonstrating how to: - Use nginx, Varnish and Apache together in a "SPDY sandwich" to support HTTP 2.0 - Setting up SSL properly to mitigate against attack vectors - Performance improvements with mod_pagespeed and nginx - Deploying Drupal sites with Docker containers Barney is a Technical Team Leader at Inviqa, a Drupal Association member and writes for Techportal on using technologies to improve website performance. He first started using PHP professionally in 2003, and has over seventeen years experience in software development. He is an advocate of Scrum methodology and has an interest in performance optimization, researching and speaking on various techniques to improve user experience through faster load times.
Citation preview
Dev Ops:the next generation
Warning!meme and star trek heavy
Who is this TREKKIE
Barney Hanlon
Technical Team Leader for Inviqa
@shrikeh
We’re hiring!
The story so far
(Yes I know that’s Star Wars)
Dev ops
DevOps
DevelopersSysadmins
“Get the code on the server”
FTP CVS
Subversion Git
What about all this OTHER stuff?
• Setting up Virtual hosts
• SSH access
• Log rotation
• Staging servers
• Build pipelines
• SSL certificates
• Minification
• User management
• File permissions
• Firewall rules
• Patching
• Concurrency
• Failover
• Alerting
Developers need to know more than how to push to
Github.
FIVE STAR DevOps
The Five Stars of DevOps
The Five Stars of DevOps
• Monitoring
The Five Stars of DevOps
• Monitoring
• Security
The Five Stars of DevOps
• Monitoring
• Security
• Performance
The Five Stars of DevOps
• Monitoring
• Security
• Performance
• Automation
The Five Stars of DevOps
• Monitoring
• Security
• Performance
• Automation
• Scaleability
Am I doing something in my application that is done better by the infrastructure or an external
service?
Probably.
Got root?
Common reasons to “just log onto the server quickly”
• server logs in /var/log require privileges to tail
• setting permissions on directories
• Processes require restarting
Monitoring
Monitoring
• Log EVERYTHING
• Drupal default visitor logging is heavy
• Should you be writing to the database to log visits?
NO.
Monitoring
• Logging is only one part of monitoring
• Send your Web logs to a remote service
• Set error_log to syslog in php.ini
Logging Services
• SplunkStorm
• Loggly
• Logentries
• Papertrailapp
HOSTING YOUR OWN LOGGING
• Splunk
• GrayLog2
• Sensu
• Munin
• Raven
Other monitoring
• Nagios
• Pingdom
• New Relic
• Piwik/Google Analytics
Profiling
• Don’t be afraid to turn XHProf on in live occasionally
• Regularly check your browser HAR
• Check APC and other caches for smells
Security
Where is the risk?
• Application security
• Infrastructure security
• End user security
Repelling Unwelcome Guests
Tools to help
• JumpCloud
• DuoSecurity
• Ubuntu ACL
Capturing Morpheus…
Not so bad.
Hardening SSL
Don’t bother hardening SSL
SSL is Dead, Long Live TLS
• No one should be using SSL any more.
• Transport Layer Security (TLS)
• Latest version 1.2
Vulnerabilities
• BEAST Attack
• CRIME Attack
• Lucky Thirteen
HTTPS without proper ciphers gives the illusion of security while providing none
ssl_protocols SSLv3 TLSv1; ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
SSLProtocol all SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5
Nginx
Apache 2Supporting SSLv3 is only required for IE6
Open to Lucky Thirteen attack
Default SSL implementations
“Today, only TLS 1.2 with GCM suites offer fully robust security. All other suites suffer from one problem or another (e.g, RC4, Lucky 13, BEAST), but most are difficult to exploit in practice…”
–Ivan Ristic, Qualys
“…Because GCM suites are not yet widely supported, most communication today is carried out using one of the slightly flawed cipher suites. It is not possible to do better if you're running a public web site.”
–Ivan Ristic, Qualys
Diffie-Hellman Key Exchange
• Diffie-Hellman (DH) and Elliptic Curve Diffie-Hellman (ECDH)
• Allows Perfect Forward Secrecy
• Slow :(
server { … ! add_header Strict-Transport-Security “max-age=31536000; includeSubDomains"; ssl_session_cache shared:SSL:10m; ssl_prefer_server_ciphers on; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS; ssl_ecdh_curve secp521r1; }
https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
https://www.ssllabs.com/ssltest/
Test your strength!
–No user ever
“I don’t mind that the site is slow, at least my data is safe.”
Performance
SPDY
• Draft HTTP 2.0
• Allows multiplexing a single connection
• Requires HTTPS
• Do you need HTTP at all?
SPDY
PageSpeed
• Library for Apache and Nginx
• Automatic minification of JavaScript, CSS and HTML
• On-the-fly optimisations based on chosen filters
The “SPDY Sandwich”
Nginx Varnish Nginx PHP-FPM
The “SPDY Sandwich”
Nginx Varnish Nginx PHP-FPM
Responsible for: • SPDY / SSL Termination • Serving static assets • Gzipping • Pagespeed is user agent-aware
The “SPDY Sandwich”
Nginx Varnish Nginx PHP-FPM
Responsible for: • Caching dynamic pages • Cookie normalisation
The “SPDY Sandwich”
Nginx Varnish Nginx PHP-FPM
Responsible for: • Serving dynamic pages • Generic Pagespeed optimisations
The “SPDY Sandwich”
Nginx Varnish Nginx PHP-FPM
Responsible for: • PHP interpreter
Cookies
Cross Site Request Forgery (CSRF)
• OWASP recommendation
• Requires a token in the form and a session token
• Breaks most reverse proxies without configuration
Am I doing something in my application that is done better by the infrastructure or an external
service?
OpenResty
• Nginx bundle
• Has modules for connecting to Redis, Drizzle, memcached and many more
• Has Lua to allow pre and post processing on requests and responses
Time for a Simplified Example
<?php !namespace Inviqa\DrupalCamp\Access; !class OpenRestyTokenGenerator implements CsrfTokenGeneratorInterface { private $token; ! public function __construct($csrfToken) { $this->token = $csrfToken; } ! public function get($value = '') { return $this->token; } }
<?php namespace Inviqa\DrupalCamp\Form; !use \Drupal\Core\Form\FormBuilder as CoreFormBuilder; !class FormBuilder extends CoreFormBuilder { public function __construct( ModuleHandlerInterface $module_handler, KeyValueExpirableFactoryInterface$key_value_expirable_factory, EventDispatcherInterface $event_dispatcher, UrlGeneratorInterface $url_generator, TranslationInterface $translation_manager, CsrfTokenGeneratorInterface $csrf_token = NULL, HttpKernel $http_kernel = NULL ) { …
OpenResty Varnish Nginx PHP-FPM
X-CSRF-Tokenize: “[[CSRF Here]]”
Redis
FIXING CSRF WITH LUA
OpenResty Varnish Nginx PHP-FPM
Header cached in Varnish
Redis
FIXING CSRF WITH LUA
OpenResty Varnish Nginx PHP-FPM
Redis
• Parses response (regex)!• Finds token placeholder!• Replaces with real token
FIXING CSRF WITH LUA
! ! header_filter_by_lua!! ! '!! ! if ngx.var.upstream_http_x_csrf_tokenize then!! ! ! -- the backend requested a CSRF token be set!! ! ! local csrf_cookie_token = nil!! ! ! if ngx.var.cookie_csrf then!! ! ! ! -- they have a cookie, just re-use it!! ! ! ! local csrf_cookie_token = ngx.var.cookie_csrf!! ! ! end!!! ! ! local resty_random = require "resty.random"!! ! ! local str = require "resty.string"!!! ! ! if not csrf_cookie_token then!! ! ! ! -- no valid csrf cookie found, let us make one!! ! ! ! !! ! ! ! local cookie_random = resty_random.bytes(16,true)!!! ! ! ! while cookie_random == nil do!! ! ! ! ! -- attempt to generate 16 bytes of!! !! ! -- cryptographically strong (enough) random data!! !! ! cookie_random = resty_random.bytes(16,true)!! ! ! ! end!!! ! ! ! ngx.var.csrf_cookie_token = str.to_hex(cookie_random)!
OpenResty Varnish Nginx PHP-FPM
Redis
• Stores HMAC as value in Redis with random key “csrf_”
• Generates cookie with Redis key as value
FIXING CSRF WITH LUA
!! ! -- we are about to mess around with the content of the page!! ! -- so we need to clear this as it will be wrong!! ! ngx.header.Content_Length = ""! ! ! !! ! -- set the Cookie for the CSRF token!! ! ngx.header.Set_Cookie = "csrf=" .. ngx.var.csrf_cookie_token!! ! ngx.header.tokenize = ngx.var.upstream_http_x_csrf_tokenize!!! ! -- now generate one for the form token!! ! while form_random == nil do!! !! form_random = resty_random.bytes(16,true)!! ! end!!! ! ngx.var.csrf_form_token = str.to_hex(form_random)!!! ! local redis = require "redis"!! ! local client = redis.connect("127.0.0.1", 6379)! !! ! client:set("csrf_" .. ngx.var.csrf_cookie_token, ngx.var.csrf_form_token)!! ! end!
OpenResty Varnish Nginx PHP-FPM
Redis
FIXING CSRF WITH LUA
Check token on way back in
location @backend {!! ! # You can't set variables in nginx dynamically, ! # so set this up as empty first!! ! set $csrf_validate "";!! ! access_by_lua !! ! '! if ngx.req.get_method() == "POST" then! !-- set up forbidden as default! !ngx.var.csrf_validate = ngx.HTTP_FORBIDDEN! !if ngx.var.cookie_csrf then!! ! ! ! local res = ngx.location.capture("/validate-csrf")!! ! ! ! if ngx.HTTP_OK == res.status then!! ! ! ! ngx.req.read_body()!! !! ! local args = ngx.req.get_post_args()!! !! ! local posted_token = tostring(args["csrf"])!! !! ! if posted_token == res.body then!! !! ! ! ngx.var.csrf_validate = ngx.HTTP_OK!! !! ! end!! !! end!! !end! !end! !';
Full gist:https://gist.github.com/shrikeh/4722427
Automation
Tooling
• Bash scripts (!)
• Chef/Puppet (retro)
• Ansible!
Ansible
• Requires no agent!
• Pure SSH
• Modules
• YAML-based configuration
• Playbooks (and playbooks of playbooks)
Ansible PLAYBOOK!- name: ensure SSH key exists digital_ocean: > state=present command=ssh name=case - name: ensure droplet exists digital_ocean: > state=present ssh_key_ids=57705 name={{ inventory_hostname }} size_id=66 region_id=4 image_id=1505699 wait_timeout=500 private_networking=yes virtio=yes wait=yes unique_name=yes wait_timeout=500 register: launched - debug: msg="IP is {{ launched.droplet.id }}" - debug: msg="IP is {{ launched.droplet.ip_address }}"
Ansible
ansible-playbook base.yml -vvv -i "hosts/production"
Configuring Your Application
<?php # /sites/default/settings.php ... $databases['default']['default'] = array( 'driver' => 'mysql', 'database' => 'drupal', 'username' => 'testuser', 'password' => '123secure', 'host' => 'localhost', 'prefix' => '', );
–The Twelve Factor App
“A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open
source at any moment, without compromising any credentials”
“The twelve-factor app stores config in environment variables (often shortened to
env vars or env). Env vars are easy to change between deploys without changing
any code” –The Twelve Factor App
env[db_name] = drupal_live env[db_host] = 192.168.0.2 env[db_user] = liveuser env[db_pass] = verysecurepass env[db_prefix] = drupalcamp_
Put your variables in PHP-FPM
/etc/php/fpm/pools/live.conf
<?php # /sites/default/settings.php ... $databases['default']['default'] = array( 'driver' => 'mysql', 'database' => getenv('db_name'), 'username' => getenv('db_user'), 'password' => getenv('db_pass'), 'host' => getenv('db_host'), 'prefix' => getenv('db_prefix'), );
Provisioning
• Idempotent deployments
• Provision every environment the same way
• Resist the urge to do something manually
• Get into a workflow of automation
Docker
Docker - AN OVERVIEW
• Lightweight Linux Container
• Portable environment
• Install all your PECL dependencies into a container
• Ship it
Problems
Problems
!
• Still heavily in development, no “right way” yet
• Hard to set up syslog inside a container
• Runs as root on the box
That’s being fixed though
We’re Done!
Questions
Thank You!
My first official talk!• Special thanks to Lorna Mitchell, Ian
Barber and Rowan Merewood for all the coaching
• All feedback welcome!