View
229
Download
1
Category
Preview:
Citation preview
L A S T U P D A T E D F E B R U A R Y 6 , 2 0 1 7
DEVELOPING DEVOPS ON AWSBy Eric Johnson
INTRODUCTION 3
CONTINUOUS INTEGRATION AND CONTINUOUS DELIVERY (CI/CD) 4
Continuous Integration 4
Continuous Delivery 4
The Bottom Line 5
THE RIGHT TOOLS 5
AWS CodeCommit 5
AWS CodePipeline 6
AWS CodeBuild 7
AWS CodeDeploy 8
WHAT TO WATCH FOR 9
CONCLUSION: PUTTING IT ALL TOGETHER 10
W H I T E P A P E R
DEVELOPING DEVOPS ON AWS
2 / 11
3 / 11DEVELOPING DEVOPS ON AWS
I N T R O D U C T I O N
Put 20 people in a room and ask them to define DevOps; chances are you will
get 15–20 different responses. In its most basic form, DevOps is a merging of the
words “Development” and “Operations.” But the truth is, DevOps is more than a
word; it is a process or even a culture. In her article on automic.com, Courtney
Glyph explains DevOps this way:
DevOps defines a set of practices that, when applied correctly,
“emphasizes the collaboration and communication of both software
developers and other IT professionals while automating the process of
software delivery and infrastructure changes. It creates a culture and
environment where building, testing and releasing software can happen
rapidly, frequently and more reliably.” [read full article]
This white paper focuses on the collaboration and automation aspects of
DevOps and how they are accomplished in AWS. By the end, you will have a
conceptual idea of how collaboration and automation come together to provide
a continuous integration and continuous delivery (CI/CD) process that can
benefit your development process and, furthermore, your bottom line.
W H I T E P A P E R
DEVELOPING DEVOPS ON AWS
4 / 11DEVELOPING DEVOPS ON AWS
CONTINUOUS INTEGRATION AND CONTINUOUS DELIVERY To understand what it takes to create a proper CI/CD process on AWS, it
important to establish what that process looks like and why it’s needed. First,
examine at the process. A solid CI/CD process offers the ability to version and
integrate code, build code specific to an environment, allow for manual approval
or rejection, and deploy code with no downtime as well as the ability to rollback
when needed.
Visualizing the process (above) enables the breaking down of the CI/CD process
into separate parts and allows an explanation of their benefits individually and
then as a whole.
Continuous Integration
The first part of the equation is the process of continuous integration - a
development process that encourages developers to code in very small chunks.
As a developer adds a feature or fixes a bug, they should then commit that
code to the shared repository. When that code is committed, there is then an
automated process that rebuilds the entire code set with the new changes
included and updates the testing location with that new code. The end result is
that the code is continuously integrated and made available for testing.
The advantages of a continuous integration system are numerous. The first
advantage is the ability to test early and test often. When a QA team can test
code early in the process, they can flag issues in the code before the developer
has moved on too far. A second advantage is the ability for the business to also
catch issues in the code direction. As code is updated early and often, testers
can verify that what the developer built matches the scope that has been
laid out. This also allows the business to make early adjustments if what they
envisioned does not work in real life.
Continuous Delivery
The second part of the equation, continuous delivery, builds on top of the
process of continuous integration. When a single piece of code passes all tests,
and it gets the stamp of approval from the business, the logical next step is to
ship it. Continuous delivery is the automated process of shipping code through
multiple environments, with it eventually ending up in production. This is a
paradigm shift from the two-to-three-week sprints culminating in a manual
deploy that many businesses still use today.
5 / 11DEVELOPING DEVOPS ON AWS
The Bottom Line
A properly architected CI/CD process affects a company’s bottom line in
several different ways. Foremost, by being able to deploy with a push of a
button, a company can be incredibly responsive to the customer. For example,
if code is deployed and a company finds that it is not performing well, they
can immediately rollback. Also, with the ability to change code frequently and
quickly, a company can make good use of multi-variant (A, B, (C)) testing to
help improve the user experience. Any time the user is helped to better use a
company’s product, the bottom line is directly affected.
T H E R I G H T T O O L S The DevOps process of CI/CD is not an entirely new concept. There are quite a
few tools on the market such as TeamCity, Jenkins, Hudson, Octopus Deploy,
and others that play in this space. However, AWS has a full suite of native tools to
accomplish this task. Let’s take a look at the different tools available.
AWS CodeCommit
The first tool worth examination is AWS CodeCommit. AWS CodeCommit is a
fully managed source control service that provides private GIT repositories within
your AWS account. Now you may be asking yourself, “Why not just use GitHub?”
This is a great question. over. While there are some advantages to using GitHub,
such as the advanced tooling GitHub adds, there are two clear advantages to
using AWS CodeCommit over GitHub. First of all, and most importantly, AWS
CodeCommit enables greater security. By using AWS CodeCommit your code
never leaves your AWS environment, with the exception of when it is on the
developer’s machine. Furthermore, it is protected by the many layers of security
provided by AWS and the granular access control that Identity and Access
Management (IAM) provides. The second advantage of AWS CodeCommit is
pricing. AWS CodeCommit gives you five users free; each additional user after
that is one dollar.
Within the AWS CI/CD process, AWS CodeCommit is where developers check
their completed code. The repository uses GIT to handle merging, conflicts, and
versioning of code.
Learn more about AWS CodeCommit.
6 / 11DEVELOPING DEVOPS ON AWS
AWS CodePipeline
The second tool in AWS’s DevOps suite is AWS CodePipeline. Notice (above)
that AWS CodePipeline deals with build, manual interface, and deploy. AWS
CodePipeline is the automation tool designed to detect code changes and then
move the code through the other phases. The beauty of AWS CodePipeline is
that it is very configurable. Different jobs can be added to your pipeline and
decisions based on the outcomes of that job.
For example, you can grab the latest code from AWS CodeCommit and ship it
off to a builder like Jenkins. Jenkins can then build and test your code. If all goes
well, Jenkins will notify AWS CodePipeline to move on. AWS CodePipeline can
then send an email to your QA team. The QA team can then test the latest code.
On approval, the user has the choice to approve or reject the code by clicking
on the appropriate button in the email. If the approval is given, then AWS
CodePipeline moves on to the next step. If not, then the pipeline is exited.
Learn more about AWS CodePipeline.
7 / 11DEVELOPING DEVOPS ON AWS
AWS CodeBuild
AWS CodeBuild is the latest tool in the AWS DevOps arsenal. Announced at
AWS Re:Invent in 2016, AWS CodeBuild closes a large gap in the tooling suite by
handling the process of building and testing your code. Before the introduction
of AWS CodeBuild, organizations were forced to look outside of AWS to third-
party tools like Jenkins and Solano. Additionally, when using third-party tools,
organizations were also forced to maintain the infrastructure the tool ran on in
the form of an instance or container, or they had to pay a separate provider to
maintain it and ship the code externally to be built and tested. CodeBuild is
serverless and empowers companies to build and test code while keeping that
code inside of their preferred AWS environment and pipeline.
In looking at how AWS CodeBuild works, it is important to understand where
AWS CodeBuild runs: AWS CodeBuild runs in a pre-built container provided by
AWS. AWS offers many different container flavors like Node, Golang, Python, and
Java, however you can also create your own container for AWS CodeBuild to
launch.
If you’re saying, “Wait a minute, that’s not serverless!” please note that it is as
serverless as any other serverless technology AWS offers. AWS CodeBuild is what
is known as a “backplane” technology, meaning they manage the infrastructure
and it, therefore, is serverless to your organization.
Using AWS CodeBuild to build your project then becomes very easy. The beauty
of the tool is that anything you can do on a command line, you can do on AWS
CodeBuild. When running a project, AWS CodeBuild will look in the root of your
code for a buildspec.yml file. In the buildspec file you define five phases and
what to do in them. Let’s break those out and explain them.
Install Phase: This phase allows you to install any external libraries that may
not be included in AWS CodeBuild. For example, you might install “mocha,”
a node test library.
Pre-Build Phase: This phase allows you to run any pre-build commands
or scripts. This would be a good place to set environment parameters and
create mock data for tests.
Build Phase: This phase allows you to actually build out your code. In a node
world, this is where you would transpile and pack code using tools like Babel
and WebPack.
8 / 11DEVELOPING DEVOPS ON AWS
Post-Build Phase: This phase allows you to run commands after your code
is built. Here you might run unit and regression tests against prepared code.
Artifacts Phase: This phase allows you to define what files should be
returned from AWS CodeBuild back to AWS CodePipeline for the next job in
the pipeline.
AWS CodeBuild is a very simple and very powerful tool. It allows developers to
use the same command line tools they utilize locally, to build and test code in
an automated fashion to push to other environments.
Learn more about AWS CodeBuild.
AWS CodeDeploy
Up until this point, it’s been possible to merge and version code, build and
test that code in the cloud, and get manual approval from your QA team. The
final piece of the puzzle is the ability to deploy the code. This is where AWS
CodeDeploy becomes important. AWS CodeDeploy provides the ability to
deploy and rollback code to multiple servers at a time, plus allows you to deploy
to servers outside of AWS in addition to AWS servers. For example, if you are
running a hybrid architecture, you can deploy to your cloud servers and your
servers on-premises as well.
It’s easiest to get a full understanding of how AWS CodeDeploy works by talking
through a deployment scenario and the components. To begin, you need
to understand what code you are going to deploy and where it exists. One
requirement of AWS CodeDeploy is to have deployment code in a versioned
format. This can be either in the form of an Amazon Simple Storage Service
(Amazon S3) bucket with versioning on, or a GIT repository. When using AWS
CodePipeline and AWS CodeBuild together, you can drop your final artifacts
in the bucket or repository. By using a versioned source, AWS CodeDeploy can
rollback to an earlier version if necessary.
Much like AWS CodeBuild, AWS CodeDeploy will be looking for a file called
appspec.yml. This file is also broken down into several parts.
Files: In the files section, you can indicate your source and destination
locations for deployment. This might include configuration files in one place,
and application files in another.
9 / 11DEVELOPING DEVOPS ON AWS
Hooks: In the hooks section, you can call scripts to be run before installation,
after installation, and on application startup. When all is done, you can also
run a script to validate the service.
AWS CodeDeploy also offers the ability to define how you want your code to
deploy. At a top level, there are two types of deployment, in-place and blue/
green. An in-place deployment keeps the same servers and updates the code
on these servers. A blue/green deployment places the new code on new servers
and re-routes traffic to the new servers.
Within these deployment types there are three approaches. The first approach,
All at Once, attempts to deploy to as many instances at one time as it can. The
second approach, Half at a Time, deploys to half of the instances at a time with
fractions rounding down. The final approach to deployment is One at a Time, in
which AWS CodeDeploy deploys code to one instance at a time.
So, what happens if a deploy fails or needs rollback? If automatic rollbacks have
been configured, then rollbacks happen automatically. In the case of a failed
deploy, or if a monitoring threshold is met, AWS CodeDeploy will automatically
rollback to the last know good version. If automatic rollbacks are not configured,
a manual rollback can be initiated by creating a new deploy using a previous
revision.
Learn more about AWS CodeDeploy.
W H A T T O W A T C H F O R At the time of this writing, AWS CodeBuild does not support Microsoft Windows
build tools. However, AWS is quickly working to include a Windows tool set and
are collecting emails for notification when they release it.
Be aware that these tools are very configuration heavy. Configuring the AWS
DevOps tools can be done from the AWS dashboard, but it’s better managed
and maintained when using the command line interface, or better yet, AWS
CloudFormation. AWS CloudFormation truly lets you experience “infrastructure
as code” and all your build configurations can be included.
10 / 11DEVELOPING DEVOPS ON AWS
C O N C L U S I O N : P U T T I N G I T A L L T O G E T H E R With the combined power of AWS CodeCommit, AWS CodeBuild, and AWS
CodeDeploy working in concert with AWS CodePipeline orchestrating, you can
build out a very powerful CI/CD process.
This example (above) gives a full CI/CD process, including deployment to
multiple environments. Take a look at the scenario:
1. Multiple developers are able to merge their completed code into AWS
CodeCommit.
2. On a completed merge and commit, AWS CodePipeline then grabs the
latest code and passes it to AWS CodeBuild.
3. AWS CodeBuild then compiles, minifies (compresses), and tests the
code. If all goes well and all tests are passed, the final code is then
dropped into the proper bucket and the process is passed back to AWS
CodePipeline.
4. AWS CodePipeline then notifies AWS CodeDeploy.
5. AWS CodeDeploy then deploys the latest version of code to a beta
environment. Upon completion, AWS CodeDeploy then passes the
process back to AWS CodePipeline.
6. AWS CodePipeline then sends an email to a user asking them to
manually verify the code. If the user verifies the code and accepts the
change, AWS CodePipeline then notifies AWS CodeDeploy to continue.
7. AWS CodeDeploy then repeats the process for staging and, finally,
production.
As you can see, using the AWS DevOps tools, it’s possible
to automate versioning, compiling, testing, approval and
deployment to all environments. Several advantages
come with this automation. Primarily, it is a repeatable
process that is monitored. If anything in the process needs
changing, it is then changed every time. Secondly, this
process verifies that all automation tests are passed and
that the project has also been manually tested. While
the process cannot prove that a person has tested the
code, it can prove they acknowledged that they did,
which promotes accountability. An added bonus is that
the entire pipeline is serverless, meaning there is no
infrastructure to maintain.
Learn more about how Rackspace can help you get the
most out of AWS at https://www.rackspace.com/en-us/
managed-aws.
Eric Johnson is the AWS Evangelist at Rackspace. He has
been working with AWS technologies as a developer and
an architect for the last six years. His passion for all things
new in cloud technologies drive him to be a lifetime
student and fanatical speaker on all things cloud. Connect
with him on Twitter and LinkedIn.
© 2017 Rackspace US, Inc. Rackspace®, Fanatical Support® and other Rackspace marks are either registered service marks or service marks of Rackspace US, Inc. and are registered in the United States and other countries. All other trademarks, service marks, images, products and brands remain the sole property of their respective holders and do not imply endorsement or sponsorship.
This case study is for your informational purposes only. RACKSPACE MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS CASE STUDY. All customer examples and the information and results illustrated here are based upon the customer’s experiences with the referenced Rackspace services and are not necessarily indicative of the future performance of Rackspace services. Rackspace detailed services descriptions and legal commitments are stated in its services agreements. Rackspace services’ features and benefits depend on system configuration and may require enabled hardware, software or additional service activation. Actual cost of specific hosted environment and performance characteristics will vary depending on individual customer configurations and use case.
About AWS: For 10 years, Amazon Web Services has been
the world’s most comprehensive and broadly adopted cloud
platform. AWS offers over 70 fully featured services for compute,
storage, databases, analytics, mobile, Internet of Things (IoT) and
enterprise applications from 35 Availability Zones (AZs) across 13
geographic regions in the U.S., Australia, Brazil, China, Germany,
Ireland, Japan, Korea, Singapore, and India. AWS services are
trusted by more than a million active customers around the
world – including the fastest growing startups, largest enterprises,
and leading government agencies – to power their infrastructure,
make them more agile, and lower costs.
To learn more about AWS, visit aws.amazon.com.
Rackspace, the #1 managed cloud company, helps businesses
tap the power of cloud computing without the complexity and
cost of managing it on their own. Rackspace engineers deliver
specialized expertise, easy-to-use tools, and Fanatical Support®
for leading technologies developed by AWS, Google, Microsoft,
OpenStack, VMware and others. The company serves customers
in 120 countries, including more than half of the FORTUNE
100. Rackspace was named a leader in the 2015 Gartner Magic
Quadrant for Cloud-Enabled Managed Hosting, and has been
honored by Fortune, Forbes, and others as one of the best
companies to work for.
Learn more at www.rackspace.com or call us at 1-800-961-2888.
Recommended