View
434
Download
0
Tags:
Embed Size (px)
DESCRIPTION
With Steven Boone (Techical Sales Specialist, UrbanCode) and Michael Elder (STSM, DevOps).
Citation preview
Continuous Delivery to the Cloud
Steven Boone IBM UrbanCode Technical Sales Specialist
Michael Elder IBM Senior Technical Staff Member
Sanjeev Sharma IBM WorldWide Lead – DevOps Technical Sales
© 2013 IBM Corporation
Please noteIBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Agenda
• DevOps – a Lean approach
• DevOps and Cloud – Cloud Portability
• DevOps with IBM Cloud Platforms
• Full Application Stack support
Agenda
• DevOps – a Lean approach• DevOps and Cloud – Cloud Portability
• DevOps with IBM Cloud Platforms
• Full Application Stack support
New Modes of EngagementSystems of Record
Data & Transaction Integrity Smarter Devices & Assets
• Data & Transactions• App Infrastructure• Virtualized Resources
• Expanding Interface Modalities• Big Data and Analytics• Social Networking
Next Generation
Architectures
Focus on Speed and AgilityFocus on Operational Costs
We are at an inflection point in the industry
DevOps approach: Apply Lean principles accelerate feedback and improve time to value
Line-of-business
Customer
1
3
2
1. Get ideas into production fast2. Get people to use it3. Get feedback
Change speed must be an asset, Not an anchor
Non-Value-added waste
Value-added production work
DevOpsTransformation
7
DevOps: Using Lean Thinking to Eliminate Bottlenecks
Type of Waste Create Feature Deliver FeatureUnnecessary Overhead
Communicating ideas/knowledge Communicating between development and operations
Unnecessary Re-work
Tasks assigned back to developers from testing and usage
Tasks assigned back to developers from production rollbacks
Over-production Unnecessary functionality produced Unnecessary hardware, data center, personnel
Non-Value-added waste
Value-added production work
DevOpsTransformation
Agenda
• DevOps – a Lean approach
• DevOps and Cloud – Cloud Portability • DevOps with IBM Cloud Platforms
• Full Application Stack support
Implementing a DevOps toolchain
SCM
Build / CI Server
Unit testingTest Automation
Test Stubbing
Delivery Pipeline
Environment Configuration
Automated Monitoring
Asset Repository
Continuous Delivery Pipeline
DevOps and Cloud adoptionAutomating for faster delivery with DevOps and cloud
Networking Networking
Storage Storage
Servers Servers
Virtualization Virtualization
O/S O/S
Middleware Middleware
Mid Config Mid Config
Data Data
Applications Applications
Traditional On-Premises
Infrastructureas a Service
Manual
Customization; higher costs; slower time to value
Standardization; lower costs; faster time to value
Man/Auto
UC Deploy UC Deploy
Man/Auto
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Platformas a Service
Codename: BlueMixPureApplication
SystemSmartCloud OrchestratorSoftLayer
JazzHub(SaaS)
Progression of Cloud Adoption: Physical Servers
Physical Servers (DEV - TEST) Physical Servers (PROD)
Environments
QA ... PROD
World-Wide Banking Applicationwar ddl mq
World-Wide Banking Applicationwar ddl mq
World-Wide Banking Applicationwar ddl mq
DEV
UrbanCode Deploy
Application
Component
Tomcat MySQL JMS WebSphere Liberty DB2 WSMB
IBM UrbanCode Deploy
VM
Progression of Cloud Adoption: Cloud for DEV-TEST
Cloud Resources (DEV - TEST) Physical Servers (PROD)
Environments
QA ... PROD
World-Wide Banking Applicationwar ddl mq
World-Wide Banking Applicationwar ddl mq
World-Wide Banking Applicationwar ddl mq
DEV
UrbanCode Deploy
Cloud
Application
Component
Tomcat MySQL JMS WebSphere Liberty DB2 WSMB
IBM UrbanCode Deploy
Public Clouds
VM VM VM
Progression of Cloud Adoption: Cloud Portability with Patterns
Cloud Resources (DEV) Cloud Resources (PROD)
Environments
QA ... PROD
World-Wide Banking Applicationwar ddl mq
World-Wide Banking Applicationwar ddl mq
IBM Virtual System PatternsOpenStack HOT templates
World-Wide Banking Applicationwar ddl mq
DEV
UrbanCode Deploy
Cloud
Application
Component
Tomcat MySQL JMS WebSphere Liberty DB2 WSMB
SmartCloud Orchestrator
IBM PureApplication System
IBM UrbanCode Deploy
Private CloudsHybridPublic Clouds
VM VM VM
Agenda
• DevOps – a Lean approach
• DevOps and Cloud – Cloud Portability
• DevOps with IBM Cloud Platforms• Full Application Stack support
Application Blueprints: Resource + Application templates
What to be deployed
Where to be deployed (On-
prem or Multiple Cloud Resources)
- Patterns
How to be deployed
DevOps and Cloud adoptionAutomating for faster delivery with DevOps and cloud
Networking Networking
Storage Storage
Servers Servers
Virtualization Virtualization
O/S O/S
Middleware Middleware
Mid Config Mid Config
Data Data
Applications Applications
Traditional On-Premises
Infrastructureas a Service
Manual
Customization; higher costs; slower time to value
Standardization; lower costs; faster time to value
Man/Auto
UC Deploy UC Deploy
Man/Auto
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Platformas a Service
Codename: BlueMixPureApplication
SystemSmartCloud OrchestratorSoftLayer
JazzHub(SaaS)
Codename: BlueMixDelivering a Composable Services development environment
Run Your AppsThe developer can chose any language runtime or bring their own. Just upload your code and go.
DevOpsDevelopment, monitoring, deployment and logging tools allow the developer to run the entire application with JazzHub and Mobile Quality Assurance
APIs and ServicesA catalog of open source, IBM and third party APIs services allow a developer to stitch together an application in minutes.
Cloud IntegrationBuild hybrid environments. Connect to on-premises systems of record plus other public and private clouds. Expose your own APIs to your developers.
Extend SaaS AppsDrop in SaaS App SDKs and extend to new use cases (e.g,. Mobile, Analytics, Web)
DevOps Service (on BlueMix): Delivery pipelines as a Service
Developer
Running Application(Dev Space)
Create & edit
Running Application(test) Running Application
(Prod Space)Running Application(Test Space)
Everything is a service in the Cloud
Deploy & test
Build Publish build
Deploy
Promote
Test as a Service
Test
Monitoring as a Service
Monitor
Dev as a Service Build as a Service Deploy as a Service
DevOps and Cloud adoptionAutomating for faster delivery with DevOps and cloud
Networking Networking
Storage Storage
Servers Servers
Virtualization Virtualization
O/S O/S
Middleware Middleware
Mid Config Mid Config
Data Data
Applications Applications
Traditional On-Premises
Infrastructureas a Service
Manual
Customization; higher costs; slower time to value
Standardization; lower costs; faster time to value
Man/Auto
UC Deploy UC Deploy
Man/Auto
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Platformas a Service
Codename: BlueMixPureApplication
SystemSmartCloud OrchestratorSoftLayer
JazzHub(SaaS)
Driving consistency with IBM’s patterns of expertise
What’s required…What the business wants… What a pattern automates…
Patterns of Expertise: Proven best practices and expertise learned from decades of client and partner engagements
• Pre-defined architecture of an application or Cloud service
• Captures best practices for complex tasks• Optimized into a deployable form for private
or public cloud• Repeatable deployment with full lifecycle
management reducing operational costs
UrbanCode Deploy supports Virtual System PatternsPureSystems and SmartCloud Orchestrator
Virtual Application Patterns
• Highly automated deployments using expert patterns
• Business policy driven elasticity
• Built for the cloud environment
• Leverages elastic workload management services
Best TCOcloud applications
Virtual System Patterns
• Automated deployment of middleware topologies
• Traditional administration and management model
• Application and infrastruture driven elasticity
Improved TCOvirtualized applications
Standard TCOexisting applications
Virtual Appliances• Standard software installation
and configuration on OS
• Images created through extend/capture
• Traditional administration and management model
• Infrastructure driven elasticity
Virtual Appliance
Metadata
Softwareapplication
Operatingsystem
Virtual Appliance
Virtual Appliance
Metadata
ApplicationServer
Operatingsystem
Virtual Appliance
Metadata
ApplicationServer
Operatingsystem
Virtual Appliance
Metadata
HTTPServer
Operatingsystem
Virtual Application PatternsVirtual System PatternsVirtual Appliances
Softwareapplication
IBM UrbanCode Deploy
Artifact Library
Application Resource Template
Continuous Delivery with Cloud: IBM Cloud Platforms
SCM Build Automation
DEV QA PROD
Provision infrastructure Execute application deployment and manage settings across environments
Publish builds
Pull changes
IBM SmartCloud Orchestrator
IBM PureApplication System
Cloud Environments
Manage source code for apps under development
Execute provision with configuration settings
Application binaries (versioned)
Environment configurations (versioned)
Automate provisioning of environments as part of the end-to-end delivery process – Establish and automate deployment of Application Blueprint with resource templates imported from Cloud patterns.
Deploy early and often to ensure high quality and faster releases using repeatable, reliable, and managed automation - Seamless process flow for incremental, full stack provisioning and application deployment automation
Deployment Automation
Blueprint
Application Resource Template
Continuous Delivery to IBM Cloud – Virtual System Patterns
• Capture cloud pattern to be used for creating an Environment
• Incremental deployment of application builds to cloud environments
• Map the application to multiple cloud patterns
The freedom to provision a version of a full stack or incrementally deploy an application version into an already provisioned environment
Environments | Processes | Configurations
Create env from pattern
Deploy app QA PRODDEVDEVDEVDEV
Import pattern
Cloud Platform
Creating Environment + Application Blueprints• Components capture Process and Versions
– Automation processes know how to deploy the component, undeploy the component, or any repeatable tasks relevant to the Component
– Versions know about the artifacts which are produced by the build or third parties
• Component Resources are prototypes for real Resources
– Place these in the Resource Tree wherever a matching Component Version should be deployed
– Replace specific Component mappings
Component
VersionVersionVersionVersionVersionVersionVersionProcess
JKE DB v1.0.2
JKE Tomcat v6.0.2
JKE War v3.0.10
IBM UrbanCode Deploy and Softlayer
26
Customer Needs• DevOps Motivation / Business Drivers• Principles• Continuous Delivery Pipeline• DevOps and Cloud
Customer Use Case• Requirements• Solution Architecture• Use Cases
Disclaimer
• The Use Case is about a pilot project based on an RFP conducted by a major German customer from the automotive industry
• We have signed an NDA (Non Disclosure Agreement), so any reference to the customer is not allowed
• All Use Cases are industry-agnostic
Business Drivers
Guiding Principles for a new DevOps Operating Model
The Concept of a “Continuous Delivery Pipeline”
A “Continuous Delivery Pipeline” with Cloud
CLM / SCM
Cloud Provider
PROD Stage
Continuous Integration / Build Management
and Automation
ArtefactRepository
INT Stage`QA Stage Dev Stage`
Test Mgmt / Test Automation
DeplyomentAutomation
[triggers]
[get artifatcs]
[publish artifacts /trigger Deployment]
[provision environment / deploy into Cloud]
ServiceVirtualization
[store artifacts]
[setup]
RequirementsMgmt.
[plan work items / version source code]
Development
[specifyBusinessrequirements]
[get sources]
Application Deployments and Cloud Platform Provisioning based on a common Blueprint
INT QA PROD
Instance A Instance B Instance C
App + Config D App + Config E App + Config F
Development & Build
Cloud Pattern as Infrastructure as Code”
Cloud Instances are grouped into logical environment patterns
The High Availability Cloud Pattern is used in the Integration and Productive Environment
The Standard Cloud Pattern is used in the Development Staging Environment
HA Pattern
PatternPattern
Standard HA Cloud PatternWeb
ServicesApp
Services
WAS ClusterNode 1
WAS ClusterNode 2
– Data Services
DB ClusterNode 1
DB ClusterNode 2
Web 1
Web 2
Standard Cloud PatternWeb
ServicesApp
Services
WAS ClusterNode 1
Data Services
DB ClusterNode 1Web 1
Three types of environments
• Development environments• Test / INT environments• Production environments
A pattern is a logical composition of virtual server instances and the specific middleware components plus network elements and necessary communication configuration between the elements.
IBM UrbanCode Deploy and Softlayer
34
Customer Needs• DevOps Motivation / Business Drivers• Principles• Continuous Delivery Pipeline• DevOps and Cloud
Customer Use Case• Requirements• Solution Architecture• Use Cases
Key requirement is to implement a “DevOps workflow automation” simplifying and integrating the deployment and provisioning process
Request New Environment
Request ApplicationDeployment
Infrastructure Provisioning
ApplicationDeployment
Infrastructure Provisioning
ApplicationDeployment
Solution Architecture
VLAN
Web Services /
Internet Web
VLAN
App Server Zone
WAS Node(s)
VLAN
Data Zone
DB Node(s)Web Node(s)
VLAN Management and Security Zone
IBM SoftLayer Data Center Amsterdam
Load Balancer
Security / Jump Server
Script Server
IBM UCDServer
NagiosServer
Internet
SoftLayer AutomationSystem (IMS) and API
SoftLayer Portal
Customer
Use Cases (1 of 2)
Use Cases (2 of 2)
DMS and IBM confidential •39
•13/12/2013
Use Case Example – New Release
VLAN VLAN
WAS App Server
VLAN
DB ServerActive
Web Server
VLANManagement and Security Zone
IBM Softlayer DataCenter Amsterdam
Loadbalancer
Security / Jump Server
Script Server
uDeployServer
NagiosServer
Internet
Softlayer AutomationSystem (IMS) and API
Softlayer Portal
Customer
HA PaaS PatternPassive Line
DB ServerPassive
ExistingHA PaaS PatternActive Line
1
2
3
4
65
(1) Use IBM UrbanCode Deploy to provision environment based on HA PaaS Pattern / The Script Library is used
(2) The Script Library makes use of the Softlayer API library
(3/4) The environment is provisioned and connected to the HADR DB System
(5) UCD deploys the application followed by a functional test and a manual approval
(6) Systems are visible in Nagios Monitoring System
Some Technical Insight
• A reusable Script Library has been developed to interface with the SL API (see next slide)
• The Script Library can be used independant or integrated into IBM UrbanCode Deploy
• SL FlexImage technology has been used („portable“ between Virtual and BareMetal resources in SoftLayer) for Image templating
• Production environments share the same DB/2 cluster while test and development environments have their own
• The network infrastructure is static while the systems will be dynamically integrated. We used the SL Gateway appliance based on the Vyatta Network OS
• The only system with Internet access will be the load balancer which routes the incoming traffic to the pool of webservers. Only production environments are connected to internet. All other environments are only reachable through a Security proxy
Script Library • A mixture of SoftLayer CLI and the SL SOAP API has been used for the Script
Library implementation • All scripts have been written in Perl / no root required• Re-usable solution given you have your own SL account
w3t_create Create a new environment
w3t_destroy Destroy an Existing Environment
w3t_status Show the Current Status of Environments
w3t_changeto Change the Status of an Environment
w3t_crash Crash a Single Service Instance or a Complete Environment
w3t_switch Switch the Active Environment/Reconfigure Load Balancer
w3t_mrg_etc_hosts Merge Given Config File and /etc/hosts
Script Library example
w3t_create -[h?] [-{Dsv}] [-c] [-{dpt}] -e <envno> [-l <loc>] [-b] [-i] [-n] [-y] [-A] [-S]
[-V <version>]-b Create Bare Metal / dedicated Cloud instances-c Create clustered CCIs -d Create development environment -e <envno> Environment to create envno = unique number 1 to 99 -i Infinite - don't time out waiting for CCIs to be created-l <loc> Location were to create [default: ams01] -n No execution – dry-run: show what would have been done-p Create production environment [default] -t Create test environment -y Yes – answer questions with yes (batchmode)-A Create an All-in-a-Box system-S Use Standard images instead of Flex images-V <version> Version of the disk images to be used
config/template.cci.create.<service>[.<instance>]<service> ::= “app” | “db” | “web” | “all”
<instance> ::= “1” | “2”
IBM UrbanCode Deploy Integration Concept
Environment Definition(Dev, INT,
PROD)
Application
Components Components
CustomerApplication
WAR, EAR Database DDL
ResourceTemplate
collection of resources as a model
for setting up an environment (hierarchical
representation
Agent Prototype
Other Resources
Collection of Resources
Represent Agentnot yet installed
Application Blueprint[has]
[belong to]
[contains]
[assign to]
SoftLayerScript Library
UCD calls Script Library
to generate Environment
[instantiates]
Agenda
• DevOps – a Lean approach
• DevOps and Cloud – Cloud Portability
• DevOps with IBM Cloud Platforms
• Full Application Stack support
Introducing IBM UrbanCode Deploy with Patterns
Pattern designer– Design open, full stack application
environments in a diagram or textual editor
Design once, deploy anywhere– Deploy full stack environments to multiple
clouds
Environment lifecycle management– Manage infrastructure change and easily
apply changes to existing environments
Delivery process automation– Automated delivery process with integrated
full stack environments
Design and deploy full stack application environments for multiple clouds
IBM UrbanCode Deploy with Patterns
Application
Middleware Config
Middleware
OS Config
Hardware
Envi
ronm
ent
Blu
eprin
t
• The adoption of DevOps => increased velocity of application delivery
• Puts pressure on the infrastructure to respond more quickly
• Software Defined Environments enable you to capture infrastructure as a software artifact
Deploying infrastructure changes is the current bottleneck for delivery pipeline
Application Changes
InfrastructureChanges
What is a #fullstackapp ?
Application Changes
InfrastructureChanges
Consistent Incremental Change
… …
Long, complex lifecycle
Small iterative changes
Role Layer
… …Developers/Testers
Integrators
SpecialistsCompute, Network,
and Storage
Platforms
Apps
Developers/Testers
Integrators
SpecialistsCompute, Network,
and Storage
Platforms
Apps
Full Stack Blueprint (HOT)
Cloud Orchestration
Describe software defined resources (Compute, Network, Storage) alongside middleware and applications
Capture the full stack as a blueprint document,actionable by an automated process
Compute, Network, and
Storage
Platforms
Apps
Compute, Network, and
Storage
Platforms
Apps
Compute, Network, and
Storage
Platforms
Apps
Dev/Test Cloud Production Cloud
Full StackBlueprint
Dev QA Prod
Apply the same blueprint to multiple environments
51
• Diagram editor• Source text editor • HOT as-you-type validation• Annotated TODO & errors• Quick palette for searching• Editor functions: Cut, Copy,
Paste, Find & Replace• Zoom diagram or text• Comment out infrastructure
• Defined by the community and vendors
• Integration with UrbanCode Deploy enables rapid configuration of full application stack
• Support ongoing updates of deployed versions through UrbanCode Deploy
Example: Software
• Version the blueprint using your source control repo (native support for git)
• Apply the new versions of the template to an existing environment or provision an entire new stack
• Validate changes with a "canary" pattern to ensure correctness and detect problems earlier
• Verify entire stack before production
Full-StackBlueprint
Software Defined Delivery
54
• View current environments• Create new environments in
the cloud• Select available resources
from Nova, Neutron, Cinder• Validation feedback in place• Update environment over time• Deploy new component
versions
• Parameterize differences between cloud environments (network ids, image ids, software versions)
• Overlay resource types for different kinds of clouds in the same blueprint
• Enable portability but optimize for each specific cloud platform
Full StackBlueprint
Config/Params
Cloud ACloud B
Config/Params
Software Defined Portability
56
• Provision Network architecture to OpenStack
• Provision Network architecture to Amazon (no OpenStack required)
• Select pre-defined configuration for each cloud
57
Resources:
IBM DevOps Page: http://ibm.com/DevOps DevOps For Dummies Book: http://ibm.co/devopsfordummies Release and Deploy For Dummies Book: http://ibm.co/1bplaQV IBM DevOps YouTube Playlist: http://bit.ly/1fiDOtl
58
Thank You!
Your Feedback is Important!
Access the Innovate agenda tool to complete your session surveys from your smartphone, laptop or conference kiosk.