David Grizzanti, Bill Jones Sungard Availability Services...Working with the Event Bus Benefits ! Real time notifications (async jobs results) ! No polling required Node.js + SockJS

© 2014 Sungard AS. All rights reserved.

Building a CloudStack UI for the Enterprise David Grizzanti, Bill Jones

Sungard Availability Services

CCCNA’14 - APRIL 10TH, 2014 - DENVER, CO

Overview

!   Who are we? –  David Grizzanti (Software Engineer @ Sungard AS) –  Bill Jones (Software Engineer @ Sungard AS)

!   Cloud Engineering @ Sungard Availability Services –  Current Cloud Offering & why CloudStack is the future

!   What’s unique about our UI –  Features –  Technologies we’re using

!   Overall Architecture

!   Integrating with CloudStack

!   Demo

!   Continuous Integration

2 CCCNA’14 - April 10th, 2014 - Denver, CO

About Sungard Availability Services

Sungard Availability Services provides IT Operations support to IT departments that need to ensure business viability by keeping mission critical information and applications up and running. Sungard AS provides tailored enterprise cloud services, as well as innovative solutions including: !   Disaster Recovery !   Managed Hosting !   Software !   Consulting

As of March 31st, 2014 Sungard AS was split from Sungard and became a separate, independent company.

Cloud Engineering at Sungard AS

Current Cloud Platform

!   Enterprise Cloud Services – we operate a shared, multi-tenant infrastructure

!   We provide a fully managed “Virtual Data Center” environment for our customers

!   Our customers get cloud economics and agility without needing to re-architect their applications

!   We currently use traditional network isolation and security techniques

!   We have developed our own orchestration platform for our fully automated service provisioning

Issues with this approach

!   Runs on complicated hardware

!   Difficult to perform upgrades, general lifecycle issues

!   Need for a self-service option


Cloud Engineering at Sungard AS

Future/Choosing CloudStack

!   We are growing, so scaling is a concern

!   Customers want more flexibility and provide more customization

!   Improve reliability of provisioning automation

!   Provide more features to customers in an agile fashion

!   Need for a “Public” cloud as well as enterprise offerings

What UI do we offer our customers?


UI: Necessary Features

Self Service !   Ability to enter credit card and spin up a VM within minutes Utility Billing !   Customer shouldn’t be blocked from adding more capacity Cost Control !   Customers can see projected spend and manage budgets across their

entire company

Make it straightforward and easy to use!


UI: What sets us apart

Real Time Notifications to the UI !   Integrates with Event Bus/RabbitMQ

Elevated Permissions on top of API !   Allow more than just ROOT Admin to administer accounts

Roles !   Allow for multiple user Roles above what CloudStack provides

Customer centric UI !   Workplace concept to integrate CloudStack resources and billing


Overall Architecture Unified Cloud Portal

Web

Single Sign On

Bunny Ears

Billing Integration API

Hermes

Client/Browser CloudPlatform

SockJS

Let’s talk about the tech!

!   Ruby (Bunny Ears) –  Worker that’s reading from RabbitMQ

!   Ruby on Rails –  Main web app (running under nginx + unicorn)

! Node.js (Hermes) –  Notifies client/browser of changes CloudStack resource state

! Backbone.js/Marionette.js + SockJS(WebSockets) –  Javascript frameworks used to drive UI changes in the browser –  Websockets keep connection open to Node.js for updating client’s

browser ! Redis

–  Storing information related to each async job –  User invites

!   Bootstrap –  Responsive front-end framework


Working with the CloudStack API

Benefits !   Fully featured API ! Async Jobs Job Management !   Storing request parameters + Job ID in Redis for retrieval and

correlation Areas for improvement !   Better permissions model (this is already in progress) !   A Truly RESTful interface !   Consistency around responses for all requests (async and non-async)


Web

Example…

Working with the Event Bus

Benefits !   Real time notifications (async jobs results) !   No polling required Node.js + SockJS (Websockets) !   Worker process reads from RabbitMQ !   Queries CloudStack/Redis !   Sends events to Hermes -> Client’s browser Areas for improvement !   Multiple events types can sometimes be confusing !   Data spread across more than 1 event !   Incorrect information included in events


Hermes

Bunny Ears

Client/Browser

Overall Architecture Unified Cloud Portal

Web

Single Sign On

Bunny Ears

Billing Integration API

Hermes

Client/Browser CloudPlatform

SockJS

Let’s see a Demo!

VM Creation

VM Events

Network Configuration

Testing is Hard

Try as hard as possible to TDD Tried to use VCR in the beginning… The rise and fall of Smokestack !   Built a replica of CloudStack for integration testing (well, almost) !   Became too much work to maintain !   Went with “simulator” (quicker on dev cycle)

Test Coverage !   Code Climate !   Quality & security analysis for Ruby on Rails and Javascript


Continuous Integration with CloudStack

Typical Flows !   Run master after each push !   Job in jenkins to build new projects for each new branch we push,

auto-build on intial push or update !   Both master and “feature” branches run full unit and integration tests

Run full test suite after every code push !   Challenge of trying to run “integration” tests on every push !   Using CloudStack Simulator for feature tests

–  Pool of CloudStack instances –  Vagrant/VirtualBox + Rails app to handle resetting each instance

after use


Simulator Pool

Next Steps

Move to Docker? Docker CloudStack Simulator ! github.com/dgrizzanti/docker-cloudstack-simulator


Tools for sharing

Built a fully featured ruby client for CloudStack (StackerBee) !   Exception handling !   Idiomatic Ruby formatting for names (snake_case vs mixedCase) !   Handling of map parameters !   Configurable API version !   CloudStack REPL !   Faraday – HTTP lib (configurable middleware) !   Exposes console access URL !   Tests! github.com/promptworks/stacker_bee

Vagrant CloudStack Simulator ! github.com/promptworks/vagrant-cloudstack-simulator


Logging

Tracing requests while testing !   Need a reliable way to trace a request all phases of the application !   UI -> CloudStack !   CloudStack -> RabbitMQ -> BunnyEars -> Hermes -> Client


Credits/Acknowledgements


Questions? David Grizzanti @dgrizzanti [email protected] Bill Jones [email protected]

Documents

David Grizzanti, Bill Jones Sungard Availability Services...Working with the Event Bus Benefits ! Real time notifications (async jobs results) ! No polling required Node.js + SockJS