building_private_cloud_with_oss_for_scientific_environments-libre

Zoran Pantic & M. Ali BabarIT University of Copenhagen, Denmark

Nordic Symposium on Cloud Computing &

Internet Technologies (NordiCloud)

August 21th & 22th, 2012, Helsinki, Finland

Building Private Cloud with Open Source

Software for Scientific Environment

Zoran Pantic

Infrastructure Architect & Systems SpecialistCorporate IT @ University of Copenhagen

E-mail: [email protected] & [email protected]

Academic profile: http://itu.academia.edu/ZoranPanticBlog: http://zoranpantic.wordpress.com

LinkedIn: http://www.linkedin.com/in/zoranpantic

M. Ali Babar

Agenda

� Non-technical part:� Why Private Cloud?� Why OSS?

� Technical part:� Reflections on diverse IT-infrastructure aspects� OSS Private Cloud solutions:

� UEC/Eucalyptus� OpenNebula� OpenStack

� Conclusion� Questions? (also during the session)

Tutorial Goals� Understand the role and use of private cloud in specific

environments, e.g., scientific & academic

� Gain knowledge of the technologies for setting up a private cloud with open source software

� Learn about the process for designing & implementing a private cloud solution

� Appreciate the socio-technical & technical challenges involved and some potential strategies

Cloud Computing

“Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”

(A definition by the US National institute of standards and technology - NIST)

Main Aspects of a Cloud System

Reproduced from Figure 1 of The Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010.

Commercial Efforts & NFRs

From The Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010

Service & Deployment Models

Infrastructure as a Service (IaaS)

Amazon EC2 Eucalyptus IBM – ComputingOn Demand (DoC)

VMWarevSphere

Platform as a Service (PaaS)

Google AppEngine

MicrosoftAzure

Force.com YahooOpen Strategy

Software as a Service (SaaS)

Google Apps Zoho SalesforceCRM

Microsoft CloudServices

Public Clouds

Private Clouds

Community Clouds

Virtual Private Clouds

Hybrid Clouds

Service Models

Deploym

ent Models

Private Cloud

� Private cloud has different meanings to different people

� But basically, it’s a cloud infrastructure set up, managed, and upgraded by an organization or community for their own use

� Commercial vendors are entering in this domain quite fast and Open Source providers are also there:

Eucalyptus, Open Nebula, OpenStack

Steps for Setting Up Private Cloud

� Adopt a machine virtualization strategy

� Profile application compute,memory, and storage usage and performance requirements

� Design a virtual machine development consultancy

� Accounting and recharge policies adapted to self-service

� Architect a deployment and deploy a private cloud

Source: Five Steps to Enterprise cloud computing, a White paper of Eucalyptus Systems, Inc.

Why Private Cloud? 1/2

� Usually, the budget is low, and the project should start as soon as possible

� Growing strongly:� The need for processing large data volumes� The need to conserve power by optimizing server utilization

� Non-standard highly-adaptable solution needed

� Analyzing large amounts of data to get results

� Many different research projects in one organization

Why Private Cloud? 2/2

Private clouds:

� Have higher ROI than traditional infrastructure

� Are more customizable

� Can quickly respond to changes in demands

� Support rapid deployment

� Have increased security

� Focus on an organization’s core business

� Have effort required for running them tending downward

Why OSS?

In general:

� Lowering the costs (i.e. no licensing headaches!) – the budgets aren’t growing - but the demands are

� Interchangeability & portability (general, avoiding vendor lock-in)

� Socio-organizational reasons

� Energy efficience

Examples: UEC/Eucalyptus, OpenNebula, OpenStack, Joyent SmartOS

Private Cloud Challenges

Challenges:

� Socio-technical

� Technical

Socio-technical Challenges

Socio-technical challenges: mostly political and economic:

� Existing structures oppose implementation of private cloud

� Weak transparency of who is in charge of systems and economy,

� Researches cannot be market cost-effective,

� Administrators de facto in charge - instead of scientific groups

� Tendency of implementing things because they are interesting and “fun”, while maybe there is no need for those systems.

Technical Challenges

� Private cloud maturity,

� Problems porting of programming code,

� IT departments should be big enough, with enough expertise,

� OSS: community cannot fix all your problems.

Implementing Cloud Solutions

� Determine the needs and their nature – extensive interaction with all the major stakeholders, e.g., project leader

� Top-down steering of the process

� Design and implement a test case

� End users also thoroughly test the solution - free of charge,

� Make sure that implementation succeeds first time!

� Get a very clear picture of what services are to be offered, who will use them, what they will use them for, and how!

Private Cloud in Scientific Environment

� Based on Open Source Software (OSS)

� Focus on the logistical and technical challenges, and strategies of setting up a private cloud for scientific environment

� General scenarios:� Local DIY

� OSS Private Cloud

� Enterprise Private Cloud (with mgmt solution)

� Virtual Private Cloud

� ... or just going Public Cloud

Focus on Scientific Environment

� Difference in implementing for “infantry” and “supply troops”

� “Infantry” - to support research, scientific computing and High Performance Computing (HPC)

� “Supply” - to support daily operational systems and tasks i.e. joint administration� Bookkeeping, administration, Communications (telephony, e-

mail, messaging)

� “Infantry” – stateless instances vs. “Supply” – stateful instances

Scientific Environment: “Infantry” 1/2

� Uses non-standard & advanced research instruments

� Applicable in research, scientific computing and HPC, i.e.:� Generally if users need VMs that they administer themselves

(root access) - more appropriate to supply them with machines from private cloud, then giving access to virtual hosts behind firewall

� Organizations like ITU (Denmark): for numerous different projects

� Organizations like DCSC (Denmark): 1/3 of the jobs would be runnable on private cloud

� in HPC: Only in low end, for low memory and low core number jobs

Scientific Environment: “Infantry” 2/2

Summarized suggestions� Have social psychology in mind as important factor

� Consult the professor in charge of money for the project

� Implement an open source solution – OpenStack, OpenNebula, UEC based on Eucalyptus, Joyent SmartOS, ...

Scientific environment: “Supply”

� Needs a stable and supported solution

� Summarized suggestions� Have social psychology in mind as important factor

� Consult the system owner in charge of money for the project

� Implement a proprietary solution from reputable provider� Microsoft Hyper-V, VMware Virtual Infrastructure, …

� Sign a support agreement & negotiate a good SLA

CPU and Memory

� Processor architecture:� Intel & AMD

� Definitely 64-bit – for performance reasons

� Multiprocessor, multicore, hyper threading

� Virtual Extensions enabled hardware is a must� Intel VT-X or AMD-V virtualization extensions – virtualization

enabled hardware (check by viewing /proc/cpuinfo)

� Host’s RAM minimum 4 GB

� Enable KSM (Kernel SamePage Merging)

Storage Options

� Disk intefaces: parallel & serial

� Contemporary disk types:� SATA

� SCSI

� SAS

� SSD

� Hybrid drives

Storage Types 1/3

Local vs Remote storage:

� Local storage:� disks in the host itselv

� DAS – attached directly to the host

� Remote storage:� NAS - File Level Storage (NFS, SMB/CIFS)

� Also distributed file systems (see i.e. MooseFS and GlusterFS)

� SAN - Block Level Storage (FC/FCoE, iSCSI)� OSS/free SAN/NAS appliance example: Napp-it , based on

ZFS/Nexenta

Storage Types 2/3Storage levels:� Block – bits stored sequentially in a block of fixed size; read & write raw

data blocks; for file systems or DBMSs� File – maintains physical location of files, mapping them to blocks (i.e.

inode number / pointers)� Object – data organized in flexible sized containers, objects, consisting

of data (sequence of bytes) and metadata (extensible attributes describing the object); for static data; distributed storage spread accross multiple disk drives and servers; no „central brain“ or „master point“ –scalable, redundant, durable

Partitioning in Linux using Logical Volume Manger (LVM)� Physical Volume (PV)� Logical Volume (LV): multiple PVs make one LV � Volume Group (VG): multiple LVs make one VG

Storage Types 3/3

Disk configuration:� Independent disks (JBoD)

� RAID:� Multiple drives comprising one logical unit� Can be based on software, hardware or firmware� Some of the RAID levels:

� 0 – block-level striping without parity or mirroring� 1 – mirroring without parity or striping� 5 – block-level striping with distributed parity� 6 – block-level striping with double distributed parity� 01 (0+1) – striped sets in a mirrored set� 10 (1+0) – mirrored sets in a striped set

Virtualization 1/2

Different types of virtualization:

� Hardware

� Storage

� Network

� Memory

� Application

� Desktop

� ...

Virtualization 2/2

Hardware virtualization:

� Full virtualization: guest unmodified, unaware� HW-assisted virtualization: hw architecture supports virtualization

� Partial virtualization: partially simulates the physical hardware of a machine; i.e. each guest has independent address space

� Paravirtualization: guest is aware that it’s not „alone“; guest modification required (drivers)

� OS-level virtualization (Container-based virtualization): physical server virtualized at OS-level, enabling multiple isolated and secure virtualized servers to run on a single physical server; guest and host share the same OS

Types of Hypervisors

Types of hypervisors:� Native / bare metal – run directly on the host’s hardware� Hosted – run within OS

Major virtualization vendors & technologies used in hypervisor layer: http://www.cloudcomputeinfo.com/virtualization (source: Paul Morse)

Todays most used hypervisors:� KVM/QEMU� Xen� VirtualBox� VMware� Hyper-V� SmartOS

KVM� KVM - „Kernel-based Virtual Machine“, http://www.linux-kvm.org� Linux kernel module that allows a user space program to utilize the

hardware virtualization features of various processors (Intel and AMD processors - x86 and x86_64, PPC 440, PPC 970, S/390)

� KVM included in kernel » more recent kernel gives updated KVM features, but is less tested)

� virtualization solution that can run multiple virtual machines running unmodified Linux or Windows guests

� Supports .raw, .qcow2 and .vmdk disk image formats� Available as integrated part of every Linux distribution since 2.6.20� Components:

� loadable kernel module „kvm.ko“ that provides the core virtualization� processor specific module „kvm-intel.ko“ or „kvm-amd.ko“

� KVM is only an intefrace that is called through a special system file, and requires QEMU to be a full virtualization environment

QEMU

� QEMU – Quick Emulator - http://wiki.qemu.org

� generic open source machine emulator and virtualizer:� Emulator: runs OS’es made for one machine on different machine

� Virtualizer:

� executes guest code directly on the host CPU

� Executed under Xen hypervisor, or using the KVM kernel module

Xen

� Open Source virtualisation technology - http://www.xen.org

� Started as XenoServer project at Cambridge University

� Used as standalone hypervisor, or as hypervisor component in other cloud infrastructure frameworks

� Supports .raw and .vmdk disk image formats

VirtualBox

� Oracle VirtualBox - https://www.virtualbox.org

� Free software released under GNU GPL

� A x86 virtualization platform, created by Innotek, purchased by Sun, and now owned by Oracle

� Installed on a host OS as an application

VMware

� VMware - http://www.vmware.com

� Different hypervisors:� ESX – mainline product; commercial license

� ESXi – mainline product, free (not OSS); boot from flash cards supported

� Server – free (not OSS), installs on Linux & Windows

� Workstation/Player – virtualization on user PC’s

� Supports .vmdk disk image format

VMware

Hyper-V

� Microsoft Hyper-V http://www.microsoft.com/en-us/server-cloud/windows-server/hyper-v.aspx

� Released in 2008, new 2012 release expected in November

� Virtualization platform that is integral part of Windows Server

� Only for x86-64

� Can boot from flash card on servers motherboard

� Variants:� Stand-alone product, free, limited to command line interface

� As Hyper-V role inside Windows Server

� Supports .vhd disk image format

SmartOS

� Joyent SmartOS - http://smartos.org

� Free, gone Open Source August 2011, descent from OpenSolars - Illumos

� Hypervisor powering Joyent’s SmartDataCenter, can run private, public and hybrid cloud

� Enables HW-level and OS-level virtualization in a single OS

� Features: KVM, Zones, DTrace, ZFS

Networking Services 1/3

� Providing basic network services (DNS, GW, NAT, ...) is a good idea

� Physical & virtual networks

� Physical network:� Implementing private cloud using 2 or 3 networks: WAN,

Cloud public & Cloud private

� Firewall: OSS based pfSense - to make the whole environment independent of the network infrastructure / environment where it will be “plugged in”


Virtual networks: (i.e. Nicira, Xsigo)

� Independece from network HW

� Reproduction of the physical network

� Operating model of computing virtualization

� Different hypervizor compatibility

� Isolation between virtual and physical network, and control layer

� Scalling & performance cloud-like

� Programmatic provisioning & control


Network virtualization (example: Nicira)

Redundancy

� Automatic/manual failover/failback

� Clusters: active/active, active/passive (quorum)

Private Cloud

� Some HA features, but local to every provider

� Work in progress:Corosync + Pacemaker� „Corosync“ – Open Source cluster solution

� „Pacemaker“ – Open Source HA cluster resource manager

Private Cloud Offerings

List of OSS Private Cloud offerings: (source: Paul Morse)

http://www.cloudcomputeinfo.com/private-clouds

Covered:

� Eucalyptus (Ubuntu Enterprise Cloud, UEC)

� OpenNebula

� OpenStack

Eucalyptus� Was bundled with Ubuntu (UEC); now „only“ supported

(Ubuntu is bundling OpenStack from 11.10)� UEC/Eucalyptus is an on-premise private cloud OSS based

platform, sponsored by Eucalyptus Systems� Started as research project in 2007 @ UCSB� Linux based – RHEL, CentOS, Ubuntu� Support for VMware� For scalable private and hybrid clouds� Hybrid clouds achieved by API compatibility with Amazon’s

EC2, S3, and IAM services� New feature since UEC: Eucalyptus HA

All figures taken from http://www.eucalyptus.com

Requirements

� All components must be on physical machines (no VMs!)� Processor Intel or AMD with 2 cores of 2 GHz� Min 4 GB RAM� Storage: min 30 GB for each machine, 100-250 GB and more for

SC & NC recommended� Network: min 1 Gbps NICs, bridges configured on NCs� Linux – if Ubuntu, choose LTS (Long Time Support) version� Hypervisors: (Xen, KVM, VMware)

� RHEL & CentOS must have Xen� Ubuntu must have KVM� Vmware

� SSH connectivity between machines

Components

Designed as a distributed system with a set of 5 (6) elements:

� Cloud Controller (CLC)

� Walrus Storage Controller (WS3)

� Cluster Controller (CC)

� Storage Controller (SC)

� Node Controller (NC)

� VMware Broker (Broker or VB) - optional

Architectural Layers

Three levels:

1. Cloud level� Cloud Controller (CLC)

� Walrus Storage Controller (WS3)

2. Cluster level� Cluster Controller (CC)

� Storage Controller (SC)

� VMware Broker (Broker or VB)

3. Computing level� Node Controller (NC)

Cloud Controller (CLC)

� Entry point to Eucalyptus cloud� web interfaces for administering the infrastructure

� web services interface (EC2/S3 compliant) for end users /client tools

� Frontend for managing the entire UEC infrastructure

� Gathers info on usage and availability of the resources in the cloud

� Arbitrates the available resources, dispatching the load to the clusters

Walrus Storage Controller (WS3)

� Equivalent to Amazon’s S3

� Bucket based storage system with put/get storage model

� WS3 is storing the machine images and snapshots

� Persistent simple storage service, storing and serving files

Cluster Controller (CC)

� Entry point to a cluster

� Manages NCs and instances running on them

� Controls the virtual network available to the instances

� Collects information on NCs, reporting it to CLC

� One or several per cloud

Storage Controller (SC)� Allows creation of block storage similar to Amazon’s Elastic

Block Storage (EBS)

� Provides the persistent storage for instances on the cluster level, in form of block level storage volumes

� Supports creation of storage volumes, attaching, detaching and creation of snapshots

� Works with storage volumes that can be attached by a VM orused as a raw block device (no sharing though)

� Works with different storage systems (local, SAN, NAS, DAS)

VMware Broker (Broker or VB)

� Optional component for Eucalyptus subscribers

� Enables deploying VMs on VMware infrastructure

� Responsible for arbitrating interactions between CC and ESX/ESXi hypervisors

� located with CC

Node Controller (NC)� Compute node (“work horse”), runs and controls the instances

� Supported hypervisors:� KVM (preferred, open source version)

� Xen (open source version)

� VMware (ESX/ESXi, for subscribers)

� Communicating with both OS and the hypervisor running on the node, and Cluster Controller

� Gathers the data about physical resource availability on the node and their utilization, and data about instances running on that node, reporting it to CC

� One or several per cluster

Plan Installation

� Integration with LDAP or AD

� Support for remote storage (SAN/NAS – check supported devices)

� Choosing from “ installing NC on one server and all other on another“, to „each of components on separate server“

� Trade-off between simplicity and performance & HA

Installation� Using Ubuntu+Eucalyptus bundled installation (not available

in new versions of Ubuntu, since version 11.10 Ubuntu includes OpenStack instead)

� Manually:� Install OS� Verify network (connectivity, FW, VLAN, DNS...)� Install hypervisor� Configure bridges, NTP and MTA� Install Eucalyptus� Configure Eucalyptus (network modes, hypervisors, runtime

environment)� Eventually configure HA

Scale-out Possibilities

� 2 physical servers� Server 1:

CLC/WS3/CC/SC� Server 2: NC

� 3 physical servers:� Server 1: CLC/WS3� Server 2: CC/SC� Server 3: NC

� 4 physical servers� Server 1: CLC� Server 2: WS3� Server 3: CC/SC� Server 4: NC

� 5 physical servers� Server 1: CLC/WS3� Server 2: CC1/SC1� Server 3: NC1� Server 4: CC2/SC2� Server 5: NC2

Scaling Out

NC NC NC NC NC NC

CLUSTER 1 CLUSTER 2CLUSTER

3CLOUD

Networking

Networking modes offering different level of security and flexibility:

� Managed

� Managed No VLAN

� System

� Static

High Availability

Redundancy - Eucalyptus HA:

� By configuring HA, primary and secondary cloud and cluster components are introduced

� Hot-swappable components: CLC, Walrus, CC, SC, and VB

� Must have 3 NICs if fearing network hardware failure

� For HA SCs, supported SANs needed

� NCs are not redundant

� Externally accessible components (cloud level) must have DNS Round-Robin support

� Arbitrator service uses ICMP messages to test reachability

� If all arbitrators fails to reach some component, failover is initiated

WebGUI

GUI using HybridFox

OpenNebula

� An Open Source project aiming at implementing the industry standard for building and managing virtualized data centres and cloud infrastructure (IaaS)

� Sponsors:� EU through various programs (via DSA, RESERVOIR, 4CaaSt,

StratusLab, BonFIRE)

� National grants

� C12G Labs

� Microsoft

All figures taken from http://opennebula.org

History

Characteristics 1/3

� Doesn’t have specific infrastructure requirements, making it easy to fit in the existing environment

� Try it on your laptop! Does not require any special hardware or software configuration (single server + distro of your choice)

� Supports implementations as Private, Hybrid (with both Bursting and Federation) and Public Cloud

� Provides Storage system (storing disk images in datastores; images can be OS installations, or data blocks), Template Repository (registering VM definitions), Virtual Networking & Management (CLI & Sunstone GUI, features live and cold migration, stop, resume, cancel)

Characteristics 2/3

� Has great modularity, which eases the integration with other solutions

� Implemented on a plugin model, making it easy to customize different aspects (virtualization, storage, authentication & authorization, ...)

� Any action is performed by a bash scirpt� Doesn’t implement a „default“ hypervizor� The core of OpenNebula written in C++, making it robust

and scalable� Monitoring: Configurations of VM’s and all monitoring

information is stored in a (SQL) database

Characteristics 3/3

� Uses common open industrial standards – i.e. Amazon EC2 API and Open Cloud Computing Interface (OCCI)

� OpenNebula’s native cloud API:� available as Java, Ruby, and XML-RCP API

� gives access to all the functions

� enables integration of own procedures

� Security at high level: host communication using SSH (RSA) and SSL

� Quality: relies on Community and own QA

� Making VNC sessions to running VMs supported

Main components

Main features (v3.6)

� User Security & Multitenancy using Group Management

� Virtual Data Centers

� Control & Monitoring of Physical & Virtual Infrastructure

� Supports multiple hypervisors, data stores, network integrations, datacenter monitoring (Ganglia)

� Distributed Resource Optimization

� High Availability

� Hybrid Cloud & Bursting

� Self-service provisioning portal

Internal Architecture 1/4The three layers of the internal architecture:

Internal Architecture 2/4

Drivers communicate directly to the OS

� Transfer driver: manage the disk images on the storage system, that could be NFS or iSCSI, or copying using SSH

� Virtual Machine driver: specific to the hypervisor implemented; manage the VM’s running on the hosts

� Information driver: specific to the hypervisor implemented; showing the current status of hosts and VM’s hosts


Set of components to control and monitor VMs, VNs, storage & hosts:� Request Manager: handles client requests� Virtual Machine Manager: manages & monitorsVMs� Virtual Network Manager: manages virtual networks� Host Manager: manages & monitors physical resources� Database: persistent storage (state)


� CLI: manual manipulation of the virtual infrastructure

� Scheduler: invokes actions on VMs (using XML-RPC interface)

� Other: 3rd party tools (using XML-RPC interface or OpenNebula Cloud API)

OpenNebula – hypervisors

� Xen

� KVM/QEMU

� VMware

OpenNebula – hardware

� Processor requirement: CPU with virtualization support

� Memory:� Host: minimum 4 GB

� Guest: 256 MB for smallest instance

� Storage based on RAID: local disk for PoC, SAN for production systems

� Network: gigabit network card(s), eventually bundling several cards together (performance & redundance)

OpenNebula – system components

� Frontend

� Hosts

� Image Repository

� Physical network

OpenNebula - networking

� Service Network is recommended to be dedicated network

� VM’s network interface is connected to a bridge in the host (i.e. a host with two NICs, public and private, should have two bridges)

� Create bridges with the same name in all the hostsDrivers that may be associated with each host:� Dummy� Fw� 802.1Q� Ebtables� Ovswitch� VMware

Fw Ovsw 802 ebtbl VMw

KVM Yes Yes Yes Yes No

Xen Yes Yes Yes Yes No

VMware No No No No Yes

Installation 1/7

Installation steps:

� Planning and preparing the installation

� Installing OS

� Installing the OpenNebula software

� Configuring the OpenNebula components

Installation 2/7

Planning & preparing: OpenNebula is a simple setup consisting of front end(s) and hosts (cluster nodes).

Basic components:

� Front end

� Host

� Datastores

� Service Network

� VM networks

Installation 3/7

Storage types: shared & non-shared

Non-shared storage:

� Simple to configure

� Initial start of an VM will be slower as image is copied to the host

Shared storage:

� Any host has access to the image repository

� Any operation on a VM goes quicker because there is direct access to the images, no copying needed

� In smaller environments or PoCs, implemented on front end

� In bigger environments, implemented on NAS/SAN

Installation 4/7OS installation:

� Choose Linux distribution (i.e. Ubuntu)

� Choose installation media: .iso or network

� Use default installation steps, except evt. for partitioning

� Partitioning:� If HW raid exists, it will appear as single disk; if SW raid should be

configured, can be done after creating partitions

� Partitions for system, user and swap files

� Default user creation (oneadmin)

� The same account and group needed on both Front end & host

� All the accounts need the same UID and GID (user & group IDs)

Installation 5/7Front end:� Install OpenNebula software� Requirement:

� Needs access to storage (direct or via network)� Needs access to each host� SSH to hosts using SSH keys (without passwords, auto-add to known hosts)� Ruby (≥ v1.8.7)

Hosts:� No OpenNebula software needed� Different hypervisors on different distros inside a cluster possible� Requirements:

� Hypervisor� SSH server� Ruby (≥ v1.8.7)

� Host should be registered in OpenNebula (onehost)

Installation 6/7Configuring the OpenNebula components:� Hypervisor: KVM by default (and easiest), but other drivers can be

selected / modified� Host monitoring� Storage: shared filesystem used by default, can be changed� Networking� Users & Groups (admins, regular, public & service users; integration

with LDAP infrastructure possible)� Sunstone (Web GUI with same functionality as CLI)� Accounting & Statistics (info on usage, accounting, graphs)� Zones (oZone server, managing Zones and VDCs)� Hybrid clouds (for peak resource usages)� Public clouds (using public interfaces, EC2 query and OCCI)

Installation 7/7

Management tasks fter the installation:

� Check if deamons are running

� Check passwordless inter-host connectivity

� Check / enable KSM

� Managing hosts:� Registering (adding a host to OpenNebula)

� Deleting (deleting a host, i.e. dismissing a host)

� Enabling/disabling (no monitoring nor launc of new instances)

Hybrid cloud

AWS EC2 or compatibile

Public cloud

� Giving access to the „outside world“ using:� EC2 Query interface using Amazon EC2 Query API

� Open Cloud Computing Interface (OCCI)

� Centralized management using oZone

� Zones: several physical hosts with same or different hypervisors, controlled by one front end

� VDCs (Virtual Data Centers): several hosts from the same zone logically grouped

Redundancy

� Redundant frontends, but no automatism

� Use separate MySQL backend (though oZones currently suppors only SQLlite)

� Sunstone can be deployed on a separate machine (not necessarily on front end)

OpenStack 1/4

� IaaS platform for building cloud solutions using any of the deployment models

� Open Source, released under Apache license

� Co-founded by NASA and Rackspace in 2009 in a joint open source project, with NASA delivering cloud compute code („Nebula“), and Rackspace delivering cloud object storage („Cloud Files“)

� First release to public in November 2010 – „Austin“

� Backed by i.e. HP, Cisco, IBM, RedHat, Dell, Citrix, Canonical, ...

All figures taken from http://www.openstack.org

OpenStack 2/4

� Has considerable take-off in use ...

� ...though NASA reported moving a part of its infrastructure to Amazon, saving $1 million/yr

(http://www.wired.com/wiredenterprise/2012/06/nasa-web-services-openstack/)

� Some contributors left NASA going to the private sector (Nebula, Piston Cloud Computing, RackSpace, ...)

� Active community:� http://forums.openstack.org

� http://wiki.openstack.org

� http://docs.openstack.org

OpenStack 3/4

� Supported distros: Ubuntu, Debian, RHEL, CentOS, Fedora, SUSE, Piston Enterprise OpenStack, SwiftStack, Cloudscaling & StackOps

� Releases: Austin (2010), Bexar, Cactus, Diablo (2011), Essex (2012, current stable), Folsom (under development)

� Hypervisors: KVM, Xen, ESXi� OS-level virtualization also supported, i.e. LXC� Networking modes: Flat (bridging), VLAN (vlan-switch)� Trying it (one or multiple servers):

� on free „sandbox“ hosted environment (trystack.org), or � locally using a documented script (devstack.org)

OpenStack 4/4

� Written in Python

� Consists of:Compute, Networking, Storage, Shared Services

� Managed through a dashboard

� Implements on standard hardware, supported on ARM

Compute� Provides on-demand computing ressources by provisioning VMs

� Access through APIs and web GUIs

� Scales horizontally (scale-out)

� Some features:� Manage CPU, memory, disk, network� Distributed and asynchronous architecture� Live VM management� Floating IP� Security groups & RBAC (Role Based Access Control)� API with rate limiting and authentication� Resource utilization: allocating, tracking, limiting� VM image management & cashing

Storage

� Supports both Object Storage and Block Storage:� Object Storage – distributed, API-accessible, scale-out storage used

by applications, for backup, archiving and data retention (static data)

� Block Storage - enables block storage to be used by VMs; supports integration with enterprise storage solutions (i.e. NetApp, Nexenta, ...)

� Some features:� Vertical and horizontal scalability

� Huge & flat namespace

� Built-in replication

� RAID not required

� Snapshot & Backup API

Networking

� Managing networks and IP addresses

� Pluggable, scalable and API-driven system

� Flat networks & VLANs

� Static Ips, DHCP & Floating IP

Shared services

Dashboard:� GUI for admins and users, brandable� Pluggable / 3rd party: billing, monitoring, additional managementIdentity Service:� Central directory of users mapped to services they can access� Queryable list of all of the services deployed Image Service:� provides discovery, registration and delivery services for disk and

server images� Stores images, snapshots, templates in OpenStack Object Storage� Supports following image formats: raw, AMI, VHD, VDI, qcow2,

VMDK, OVF

Service families� Nova - Compute Service

� Swift – Object Storage Service

� Glance – Image Registry & Delivery Service

� Horizon – User Interface Service, „Dashboard“

� Keystone – Identity Service

� Quantum (in development) –Virtual Network Service

Nova� Main part – cloud computing fabric controller� One of 1st projects, descends from NASA’s Nebula� provides API to dynamically request and configure VMs� Two major components: messaging queue (RabbitMQ) and database,

enabling asynchronous orchestration of complex tasks through message passing and information sharing

� Components: Database, Web Dashboard, API, Auth Mgr, ObjectStore, Scheduler, Volume Worker, Network Worker, Compute Worker

� all of its major components can be run on multiple servers (designed as distributed application)

� Supported virtualization: KVM, Xen, Citrix Xen, ESX/ESXi, Hyper-V, QEMU, Linux User Mode & Containers

� Uses a SQL-based central database (in future, for larger deployments, aggregated multiple data stores are planned)

Swift

� Object/blob storage� One of 1st projects, descends from Rackspace’s Cloud Files� Components: Proxy Server, Ring, Object Server, Container

Server, Account Server, Replication, Updaters, Auditors� Can be clustered using Proxy nodes and Storage nodes� Filles cannot be accessed through filesystem, but via API client� Scalability and redundancy: writing multiple copies of each object

to multiple storage servers within separate zones� zone: isolated storage server groups� Isolation levels: different servers, racks, sections of a datacenter,

datacenters� Best practice: write 3 replicas across 5 zones (distributed

writes/reads)

Glance

� Discovers, registers and retrieves VM images

� Uses RESTful API for querying & retrieval

� Supports various back-end storage solutions: VM image can be stored on simple file systems and object storage systems (Swift)

� Components: Glance API server, Registry Server, Store Adapter

� Supported disk formats: raw, VHD, VMDK, qcow2, VDI, ISO, AMI, ARI, AKI

� Supported container formats: OVF, AMI, ARI, AKI

Horizon

Keystone

� Cloud identity service

� provides Identity, Token, Catalog and Policy services

� implements OpenStack Identity API

Quantum

� Virtual network service („Networking as a Service“)

� Still under development, to be released with the release of „Folsom“ 27th September 2012

� Provides API to dynamically request and configure virtual networks

� Quantum API supports extensions providing advanced networking (i.e. Monitoring, QoS, ACLs, ...)

� Plugins for Open vSwitch, Cisco, Linux Bridge, Nicira NVP, Ryu OpenFlow, NEC OpenFlow, MidoNet

Advanced setup

Redundancy

� Comming with „Folsom“

� „Corosync“ – open source cluster

� Have multiple Swift and Nova servers

� Cloud controller – single point of failure (nova-api, nova-network):� Run multiple instances on multiple hosts (state is saved in DB)

� Use (--multi host configuration in Nova)

Recommendations

� Although still at an early stage, easier to install but still hard to manage and maintain for a regular admin, and having steep learning curve (admins & users), implementation is suggested, at affordable, smaller scale

� Implement on a current/modern hardware� Keep the knowledge updated� Keep software platform and hardware updated if possible� Monitor & analyze

� costs, available features and complexity, compared to� budget, needs and internal resources available

� Asses the implementation possibilities based on the analyses

Sources of Further Material 1/5

� http://www.openstack.org/� http://opennebula.org/� http://www.ubuntu.com/� http://www.eucalyptus.com� http://www.napp-it.org/index_en.html� http://www.cloudcomputeinfo.com/virtualization� http://www.cloudcomputeinfo.com/private-clouds� http://www.linux-kvm.org� http://wiki.qemu.org� http://www.xen.org� https://www.virtualbox.org� http://www.vmware.com� http://www.microsoft.com/en-us/server-cloud/windows-server/hyper-v.aspx� http://smartos.org

Sources of Further Material 2/5

� Armbrust, M., et al., 2010, A View of Cloud Computing, ACM, 53(4), pp. 50-58.

� Zhang, Q., Cheng, L., Boutaba, R., Cloud Coomputing: state-of-the-art and research challenges, Journal of Internet Services and Applications, 2010, 1:7-18.

� The Future of Cloud Computing: Opportunities for European Cloud Computing Beyond 2010.

� Chapman et. al. 2010. Software architecture definition for on-demand cloud provisioning. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing(HPDC '10), 2010, A View of Cloud Computing, ACM, 53(4), pp. 50-58.

� Ali Babar, M.; Chauhan M.A.; , A tale of migration to cloud computing for sharing experiences and observations, SECLOUD '11, ACM.

Sources of Further Material 3/5� http://nicira.com/� http://www.xsigo.com/� http://www.reservoir-fp7.eu/� http://www.c12g.com/� http://dsa-research.org/� http://portal.ucm.es/en/web/en-ucm� http://occi-wg.org/� http://opennebula.org/documentation:rel3.6:ganglia� http://www.nasa.gov/� http://www.rackspace.com/� http://www.nebula.com/� http://www.pistoncloud.com/

Sources of Further Material 4/5� http://www.pistoncloud.com/openstack-cloud-software� http://swiftstack.com/� http://www.cloudscaling.com/� http://www.stackops.com/� http://www.rabbitmq.com/� http://www.corosync.org/� http://www.clusterlabs.org/� OpenNebula 3 Cloud Computing”, Giovanni Toraldo, Packtpub,

May 2012� Eucalyptus Guides, Eucalyptus Systems, Jun 2012� “Deploying OpenStack”, Ken Pepple, Oreilly, July 2011� OpenStack Manuals, docs.openstack.org, May 2012

Sources of Further Material 5/5� “Ubuntu Enterprise Cloud Architecture”, Technical White Paper, Simon

Wardley, Etienne Goyer & Nick Barcet – August 2009� “Building a Private Cloud with Ubuntu Server 10.04 Enterprise Cloud

(Eucalyptus)”, OSCON 2010� “Eucalyptus Beginner's Guide”, UEC Edition, 23 Dec 2010, Johnson D,

KiranMurari, Murthy Raju, Suseendran RB, Yogesh Girikumar� “Dell releases Ubuntu-powered cloud servers”, Joab Jackson, IDG News

Service, NetworkWorld� Interview at Danish Center for Scientific Computing (DCSC), 30th

March 2011� White Paper “Ubuntu - An Introduction to Cloud Computing”� Deployment Guide - Ubuntu Enterprise Cloud on Dell Servers SE� White Paper “Ubuntu Enterprise Cloud Architecture”, Wardley, Goyer,

Barcet, August 2009� “Practical Cloud Evaluation from a Nordic eScience User Perspective”,

Edlund, Koopmans, November 2011.

Questions

?

Thank you!

Thank you for your attention!

Still having questions?

[email protected]

[email protected]

Technology

building_private_cloud_with_oss_for_scientific_environments-libre