Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
SCALABLE DRUPAL SITES
USING AMAZON WEB SERVICES
Amazon Web Services commonly referred to as AWS.
INTROSHANE DAVIS
Service Delivery Manager, Lead Architect ANZ
AWS Advanced consulting partnerMember of CKAN (Open Data platform) & Drupal associationsAWS certified
INTROLINK DIGITAL CLIENTS
Australians At War Film Archive
Federal and State Govt. clientsAll Australia’s major Open Data Portals; data.gov.au, data.vic.gov.au data.nsw.gov.au, data.sa.gov.au, data.wa.gov.auSuncorp Internal BI platformNCI Supercomputing facilityUNSW Australians at War Film Archive - Scalable Drupal implementation with 11,000 hours of video.
PET MENTALITYPRECIOUS, DELICATE, IRREPLACABLE
When I started in IT, this was the state of the art PC. Everything was hand coded and time consuming to set up.People have DR plans because they’re scared of losing their one off masterpieces. Their pets.
CLOUD COMPUTING
WHAT’S CHANGED?
• Desktop PCs
• Server in a cupboard
• Virtualisation
• Virtual Dedicated Servers
• Cloud Computing
Over the last 30 years we’ve moved from desktops to the cloud in progressive steps.Server in the cupboard during the early 2000’s, VMWare virtualisation from 2005 - 2010/11Cloud Computing 2011 onwards.
CLOUD BENEFITS
• Low Cost
• Agile
• Easily Managed
Cents per hour to run
Pay for what you use
Scales up and down easily
Point and Click: • Provisioning • Monitoring • Management
40 core server on spot market as little as 08c/hour. 80c per hour for a 10 node computing cluster with 400 cores.Cloud allows environments to scale up and down within a few short minutes. The time it takes to boot a linux server.True single glass of pane management over all your computing and network services. In traditional hosting this is usually very complicated to implement, and even more costly. It comes free with Amazon Web Services. Even a novice can point and click to provision a datacenter.
PREREQUISITES FOR SCALING
CATTLE VS PETS
• Standard server images
• Independent App Layers
• Dynamic IPs and DNS
• Stand alone data and files
In order to scale, we need to get away from pet mentality. We need:Standardised server images that can be brought online quicklyIndependent App Layers so web layers can be scaled independently of DB LayersDynamic IPs and DNS so our environment is flexibleStand alone data so servers can come and go without data loss
SCALING ARCHITECTUREWHAT A SCALABLE BUILD LOOKS LIKE
ELB + AMI IN ASG = SCALING
The typical scalable build has:Dynamic DNS feeding traffic to an Elastic Load BalancerAuto scaling web instances with our web app stored in an Amazon Machine Image (AMI)An HA Database backend like Amazon Relational Database Service (RDS)
SCALING DRUPALFOUR KEY PIECES
Drupal Core
Database
/sites/all - Modules and Themes
/sites/…/files - Content
There are 4 key pieces to a Drupal site that all need separate consideration.Drupal core which needs to be updated from time to timeThe database - should be Highly Available (HA)Your custom modules and themes which may have ongoing developmentThe content and files uploaded by you and users
SCALING EACH PIECECHOOSING HORSES FOR COURSES
Drupal Core• Changes Infrequently • Usually Security Updates • Almost Static Content
Bake into the base image
AMIs provide a machine image new instances are started from with NGINX, PHP and config all baked into them. AMI based instances start within a minute or so.
SCALING EACH PIECECHOOSING HORSES FOR COURSES
Database• Should be Highly Available • Needs to be a separate layer • Use RDS Multi-AZ
RDS can be scaled vertically if necessary. For high load sites, Read Replicas may be added which is easy to do with RDS.Typically the DB sees very little load. If 4 web nodes are around 80% CPU load, the DB on a medium sized instance will be no more than 10-15% CPU load.
SCALING EACH PIECECHOOSING HORSES FOR COURSES
Themes and Modules• May Change Regularly • Version Controlled • Ideally centrally located
NFS/GlusterFS/Elastic File Store
There are many options here, all with pros and cons. The best is Elastic File Store.NFS is simple, but typically has a single point of failure and is difficult to sync across a clustered setup.Baking rapidly changing themes and modules into the AMI may mean a lot of updates to the image. Manageable but could be annoying unless scripted. (https://github.com/Datashades/updateasg) provides a solution.GlusterFS is a little complicated to set up, but robust.Elastic File Store is the best choice but limited to 3 regions at time of writing.
SCALING EACH PIECECHOOSING HORSES FOR COURSES
User Content• Changes Often • Must be consistent • Can’t be lost!
Amazon S3 or NFS
S3 is the best choice with 11 9’s of reliability. AMSA solved their MH-370 issues with S3 in less than 30 minutes. By offloading video media releases to S3, they eliminated all heavy traffic from their internal web servers.NFS is a simple solution but prone to single point of failure or replication issuesEFS is a good option when available but more expensive than S3. On a cost basis alone, S3 is the better choice.
AUTO SCALINGAUTO IS THE KEY
Scheduled Policies Load Based Policies
Scaling Health Checks
ASG allows your site to scale up and down by setting a maximum and minimum number of instances.ASG ensures healthy instances are kept online. It provides Auto recovery even for single instances inside an Auto scale group.Scheduled policies control cost by reducing the minimum instances running outside peak hours.Load based policies manage sudden demand on your site. You can monitor CPU, but site latency gives a more accurate indicator that your site needs to scale up.
AWS FOR THE WIN!COOL THINGS ARE POSSIBLE
Application Load Balancer
• Admin Traffic • User traffic • Public traffic
The great thing about AWS is they’re always delivering new services.
The Application ELB allows URLs to be directed to different groups of servers, making scaling even more flexible.A good use case for this is sending admin users to a separate group of instances with independent scaling rules. Authenticated users to another scaling group, and general users to your main pool of instances. This ensures everybody gets the best experience, and your costs are even better controlled by not having to scale out your entire build because of a single user type creating one off load.
OPSWORKSINFRACODERS ORCHESTRATION TALK
13th September Geoscience Australia
• Benefits of Orchestration • Deep dive into OpsWorks • Implementing CI
https://au.linkedin.com/in/kiwicloudninja
http://linkdigital.com.au
This talk will do a deeper technical dive into the benefits of automated provisioning and system orchestration using AWS OpsWorks to create HA scalable sites. It will explain how this allows you to implement continuous integration.Connect with me on LinkedIn: https://au.linkedin.com/in/kiwicloudninjaCompany website: http://linkdigital.com.au