Understanding Cloud Options for ArcGIS Server … Computing Categories Infrastructure As A Service...

Understanding Cloud Options for ArcGIS Server Implementation

Marwa Mabrouk

Agenda

• Cloud Types • ArcGIS in Public Cloud • ArcGIS in Private Cloud • ArcGIS in a Federal Cloud (an example) • Examples for Choices

Cloud Computing Categories

Infrastructure As A

Service (IAAS)

Platform As A

Service (PAAS)

Software As A

Service (SAAS)

Web Services API (ex: Amazon)

Web Browser (ex: YouTube, ArcGIS online)

Framework API (ex: Azure, ArcGIS online web services)

Web Services API (ex: Amazon)

Framework API (ex: Azure, ArcGIS online web services)

Leveraging the Cloud

Private Cloud

Public Cloud

LESS MORE

Security & Control

Public CloudCloud

The Flexibility of the Cloud

Procurement

Hardware Setup

IT Setup

Software Setup

Data & Application

Loading Launch an Instance

from Vendor Image

Data & Application

Loading

Launch an Instance

from your Image

g

Infrastructure Management

Leveraging the Cloud

Private Cloud

Public Cloud

LESS MORE

Security & Control

Private CloudddddPublic CloudCloud

LESS MORE

Hybrid

What are people looking for in the cloud?

Cost Savings New agile features Optimization for higher quality

Cost Savings New agile features Optimization for higher qualitye

Dynamic Resource Allocation

Organizational Resources

Dynamic Resource Allocation

Organizational Resources

Persistence

GIS

Development & QA

Staging Production

Persistence

Development & QA

Staging Production

GIS

Copy and Run

GIS

Data Management

Data Visualization

ArcGIS in the Public Cloud

Amazon Web Services

• Pioneers IaaS

• IT consumerization - Buy resources based on an hourly rate and per GB - Monthly bill - Get started in 5 mins using your credit card

• Geographically controlled cloud

- Regions and Zones - S3 and CloudFront

Amazon Instance types

• Sizes - Standard - High CPU

High CPU Extra Large

High Memory Quadruple Extra Large

Standard Large

7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)

7 GB of memory, 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)

68.4 GB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each)

- High Memory - HPC / Clusters

Amazon Instance Costs

• Hourly rate

• Elastic Block Storage • $0.10 per GB-month of provisioned storage • $0.10 per 1 million I/O requests

• Data Transfer In/ Out

• Data Transfer

Data Transfer In

US & EU Regions

All Data Transfer Free

Data Transfer Out

US & EU Regions

First 1 GB per Month $0.00 per GB

Up to 10 TB per Month $0.12 per GB

Amazon Machine Images (AMIs)

• A virtual image that works in the Amazon Cloud

• A cookie-cutter approach for configuring servers

• OS + Software + Data

• ESRI provided AMIs 10.0

ArcGIS Server 10 AMI

Enterprise Geodatabase AMI

Windows 2008 64 bit ArcGIS Server GIS ServicesMicrosoft SQL Server Express (ArcSDE enabled) ArcGIS Desktop 100Gb of storage (optional)

Windows 2008 64 bit PostgreSQL 8.3.0 (ArcSDE enabled) 100Gb of storage (optional)

ESRI provided AMIs 10.1

ArcGIS Server 10.1 AMI

Enterprise Geodatabase AMI (Created Automatically)

Ubuntu 64 bit ArcGIS Server GIS Services ArcGIS Desktop 100Gb of storage (optional) Postgres SQL - ArcSDE enabled (Optional)

Windows 2008 R2 64 bit Microsoft SQL Server Standard (ArcSDE enabled) 100Gb of storage (optional)

ArcGIS Server 10.1 AMI Windows 2008 R2 64 bit ArcGIS Server GIS Services Microsoft SQL Server Express or Microsoft SQL Server Standard - ArcSDE enabled (Optional) ArcGIS Desktop 100Gb of storage (optional)

Enterprise Geodatabase AMI Ubuntu 64 bit PostgreSQL (ArcSDE enabled) 100Gb of storage (optional)

Custom AMIs

Cloud Elasticity

• Auto Scaling Service

• How it works? - Launch configuration – AMI - Set Metric Alarm and Scaling policy - Set min and max number of instances

Small load / Test Server

• Launch a ready to use Amazon Instance

• On instance failure, launch new

instance to replace and assign eip to the new instance.

eip

eip

ArcGIS Server Redundancy

• Launch a ready to use Amazon Instance

• Add instances to ELB

Zone A

Zone B

Zone C

Region US East

ArcGIS Server Redundancy with EGDB

• Use mirrored database

instances to failover

Mirrored

Database

ELB

Redundancy across different sites

• On Premise system for daily use and editing

• Cloud is used for publishing • Overnight replication over HTTPS or

secure ftp

Geodatabase replication

Over https

Large Data Deployments

• Imagery content is very large in size

• Striped EBS drive

Security in Amazon Web Services

• Shared model - Infrastructure - Application

• Federal certifications

• Private cloud options

ArcGIS / Amazon Integration in Public Cloud

• Esri provided AMIs - Automatically setup ArcGIS with EGDB - Reconfigure ArcGIS with new ip and hostname - Automatically adding more instances to Server Clusters

• Tool to create sites in the cloud: Cloud Builder

• Documentation

• Support for ArcGIS in Amazon

Cloud Builder Demo

ArcGIS Server in other Public Clouds

• Azure - VM Role

• IBM Smart Cloud Enterprise

• Other clouds

ArcGIS compatibility with Cloud Stack

Hardware

Virtualization Platform (VMWare vSphere, Microsoft Hyper-V)

Operating System (Windows, Linux)

ArcGIS

Management Tools

Support for ArcGIS Server on the cloud

• Esri certified is full support and documentation

• Not Esri certified - Duplicate problem using OS on physical machine - If duplicated in that setup it’s logged as a bug

ArcGIS and Public Cloud Deployment Compatibility

Public Cloud Vendors

Esri Certification

Vendor Self Certification

Esri testing User testing

Amazon Web Services

Microsoft Azure

IBM Smart Cloud Enterprise

Other clouds

ArcGIS online and ArcGIS Server Compliment each other

ArcGIS Online

ArcGIS Server

ArcGIS in the Private Cloud

Private Cloud Types

On-Premise Off-Premise Appliance Software Stack

ArcGIS & Private Cloud Vendors

Amazon Web Services Private Cloud

• Off-premise cloud option

• Virtual Private Cloud • For more information:

- http://aws.amazon.com/vpc/

• GovCloud • For more information:

- http://aws.amazon.com/govcloud-us/

ArcGIS / Amazon Integration in Private Cloud

• Esri provided AMIs - Automatically setup ArcGIS with EGDB - Reconfigure ArcGIS with new ip and hostname - Automatically adding more instances to Server Clusters

• Documentation

• Support for ArcGIS in Amazon

ArcGIS Integration with Amazon Web Services

Hardware

Virtualization Platform (Citrix Xen)

Operating System (Windows, Linux)

ArcGIS

Management Tools

VCE vBlock

• VCE: venture of Cisco, EMC, VMWare

• Cloud in an appliance deployed on-premise - Hardware & software

• Vendor self-certified, Esri tested

• For more information:

- www.vce.com/Vblock

ArcGIS compatibility with vBlock

Hardware

Virtualization Platform (VMWare vSphere 4.x/ 5.x)

Operating System (Windows, Linux)

ArcGIS

Management Tools

VMWare vCloud

• Software solution for on-premise

• Virtualization based on VMWare vSphere

• Management tools to facilitate cloud operations

• ArcGIS is certified on VMWare vSphere 4.x

• For more information: - http://www.vmware.com/products/vcloud/overview.html

ArcGIS compatibility with VMWare vCloud

Hardware

Virtualization Platform (VMWare vSphere 5.x)

Operating System (Windows, Linux)

ArcGIS

Management Tools (vCloud Director)

IBM Federal Cloud

• Off-premise cloud option

• Authorized for Federal Security Standards

• VMWare for virtualization

• For more info: - www.ibm.com/federalcloud

IBM Cloudburst

• On-premise Appliance solution - Hardware and software

• Virtualization based on KVM

• Utilizes IBM tools for VM Management

• For more info:

- www.ibm.com/systems/power/solutions/cloud/cloudburst/index.html

ArcGIS compatibility with IBM cloud

Hardware

Virtualization Platform (KVM/ VMWare vSphere)

Operating System (Windows, Linux)

ArcGIS

Management Tools

Microsoft Private Cloud

• Software solution for on-premise

• Built on Microsoft Windows Server and System Center technology.

• Uses Hyper-V for Virtualization

• For more info: - www.microsoft.com/privatecloud

ArcGIS compatibility with Microsoft Private Cloud

Hardware

Virtualization Platform (Microsoft Hyper-V)

Operating System (Windows, Linux)

ArcGIS

Management Tools

ArcGIS Server Redundancy with EGDB

• Use mirrored database

instances to failover for high availability

• Use redundancy on Servers for capacity and high availability Mirrored

EGDB

FW

Large Data Deployments

• Common Storage

• Security settings for data access

Redundancy across different clouds

• Overnight replication over HTTPS or secure ftp or VPN

Geodatabase replication

Over https and VPN

ArcGIS and Private Cloud Deployment Compatibility

Public Cloud Vendors

Esri Certification

Vendor Self Certification

Esri testing

User testing

Amazon Web Services VPC

VCE vBlock * VMWare vCloud * IBM Federal Cloud * IBM Cloudburst Microsoft Private Cloud

*

Terremark

* ArcGIS is certified on vSphere 4.x

Esri Private Cloud

• Services based cloud for Off-premise or On-premise

• Managed or implemented by Esri

• Leverage Esri expertise in ArcGIS on the Cloud

Using ArcGIS in a Federal Cloud (An Example)

Cross-Agency Cloud Utilization Potential: GeoCloud Sandbox Initiative

• Initiated as an activity between the Federal Geographic Data Committee (FGDC) and GSA in December 2009 with a call to federal agencies to nominate geospatial applications for deployment, testing and monitoring in the Cloud environment. This is the first Federal implementation of community-defined Platform-as-a-Service done through the GSA Federal Cloud Computing Initiative (FCCI) PMO.

• Up to 12 projects/year for FY 2012 and 2013 were nominated by federal agencies as existing projects with existing software suites to be deployed in commercial- or government-hosted Cloud environments

• Two deployment environments (cloud platforms) were abstracted from the nominated projects: Open source service stack on Linux64 and a Commercial service stack on Windows 2008 Server

Courtesy of Doug Neebert, FGDC

Deploying Federal Geospatial Services in the Cloud GeoCloud Goals and Activities

One-year project to incubate externally-hosted Cloud data and service solutions for the geospatial domain, and in support of the Geospatial Platform activity

Architectures • Identify requirements-driven solution architectures and

platforms for various sized deployments of geospatial data and services

Cost Models • Document and assess cost models to support scalability,

reliability, and redundancy

Certification • Expedite FISMA (security) certification and accreditation for

agency adoption of packaged solution architectures

Comparisons • Support and collect cost comparison information from

agencies for existing and externally-hosted Cloud solutions • Document lessons learned and best practices

Acquire, compose, document, and deploy reference platform cloud that support Geospatial

Platform standards

Monitor costs, loads, issues and options in support of OMB IT project document guidance

Certify Geospatial Solution Packages to facilitate re-use

Document Best Practices and guides to agencies on adoption

of geospatial Cloud infrastructure

Activities, Outcomes

GeoCloud-II Instance Architecture in Amazon

GIS Server, Web Services,

Software, 64-bit Operating System, on EC2

ArcGIS Server

SQLServer, Script envt.

Win 2008 R2.

OpenGeo Server

PostgreSQL, Script envt. CentOS 5.X

OR

Data Storage (Virtual

Hard Disk)

Mounted Project-specific Data, configuration files,

Elastic Block Store (EBS), could host files behind DB

mounts

Other System Image

Incl. OS (Optional)

Amazon Machine Image of other apps (Oracle, GeoNode, etc.)_ or clone of secure system

from agency

• Each of these boxes represents a Virtual Machine • A typical project will use two instances – one for software (non-persistent), one for data •Security policies applied to each image at the OS level

Geospatial App Server Persistent Volume

accesses

Courtesy of Doug Neebert, FGDC

GeoCloud Projects, 2012

• Web-Based Vector Feature Editing of the NHD (USGS)

• School District Demographic Data System (Dept of Education)

• Administrative Forest Boundary Data (USDA USFS)

• Farm Service Agency Imagery - NAIP (USDA FSA)

• TIGERWeb (Census) • StreamStats (USGS)

• Inland Electronic Navigation Chart (IENC) data set (USACE)

• Wildlife Data and Mapping, Joint Services (FWS/USGS)

• Civil Applications Center Global Fiducials (USGS)

• GeoMine Project (OSM) • GeoNode (NOAA) • General Land Office Proposal

(BLM)

Courtesy of Doug Neebert, FGDC

GeoCloud-II Next Steps

• Providing training for projects • Preparing the two latest suites for deployment by the

projects • Documenting and costing the projects • Beginning deployment

Courtesy of Doug Neebert, FGDC

Using ArcGIS in the cloud

Patterns and Alignment

Data Management

A Complete Integrated System

Planning & Analysis

Field Mobility

Visualization Engagement

Transform Data Into Actionable

Information

Collect, Organize, & Exchange Data

Get Information Into and Out of

the Field

Disseminate Information Where

and When it is Needed

Get Feedback and Make Informed

Decisions

Field Mobility Use Case

• Collect data from mobile devices in different locations

• Data Validation

• Data Sensitivity

• Limited time frame project

Case Analysis

• Architecture

• Geo Services - Feature Services for editing - GP Services for data validation and processing - Data Replication between solution database and master

database

• Data/ application Security requirements

• Cost Analysis

- Budget - Time lines - Cost estimates

Public Cloud Deployment with redundancy

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

On-Premise / Public Cloud User

EGDB

ArcGIS EGDB Server

editing

ArcGIS Server

ArcGIS Desktop

Public Cloud High Availability Deployment

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

editing

ArcGIS Server

ArcGIS Desktop

Database Mirroring & Geodatabase replication

ArcGIS Server

Private Cloud Deployment with redundancy

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

On-Premise User

ArcGIS EGDB Server

editing

ArcGIS Server

ArcGIS Desktop

Private Cloud High Availability Deployment

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

editing

ArcGIS Server

ArcGIS Desktop

Database Mirroring & Geodatabase replication

ArcGIS Server

Planning & Analysis Use Case

• Periodic Analysis vs Ad-hoc Analysis

• On- Premise Redundant compute resources

• Public Cloud compute resources

• Data Aggregation

• Time efficiency

Case Analysis

• Architecture/ Data locations

• Capacity plan: Time vs Cost

• Geo Services - Map Services - GP Services analysis

• Data/ application Security requirements

• Cost Analysis

Public Cloud Deployment with Redundancy

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

On-Premise User

Private Cloud Deployment with redundancy

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

ArcGIS EGDB Server

On-Premise Planning & Analysis

Hybrid Cloud for Compute Resources

ArcGIS Server 1

ArcGIS Server 2

ArcGIS Server 3

Private Cloud

On-Premise Planning & Analysis

ArcGIS Server 1

ArcGIS Server 2

ArcGIS Server 3

Public Cloud

Visualization Use Case

• User Special Access control

• Data updates

- Static - Dynamic

• End user types

Case Analysis

• Architecture

• Geo Services - Map Services - GP Services for data visualization

• Data/ application Security requirements

• Cost Analysis

- Budget - Time lines- Cost estimates

High Availability Deployment with Redundancy

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

Database Mirroring

ArcGIS Server

High Availability Deployment Staging & Production

ArcGIS Server 1

ArcGIS Server n

ArcGIS Server 2

ArcGIS Server 3

editing

ArcGIS Server

ArcGIS Desktop

2-way replication

active active

ArcGIS Server

Test/ Staging

ArcGIS Server

Visualization on a Hybrid Cloud

ArcGIS Server 1

ArcGIS Server 2

ArcGIS Server

Private Cloud

ArcGIS Server 1

ArcGIS Server 2

Public Cloud

Database Mirroring

GIS Se

GIS Ser

Replication Over HTTPS

Recommendations

• Apply case analysis to each project

• Identify cloud options and costs

• Build up to major cloud investments when possible

Summary

• What do you get in the cloud? - Better advantage of their resources - Better means to implement best practices - Deployment Isolation

• Leverage Proven Technology

- ArcGIS - Cloud Computing

• Use Case - Case Analysis - Identify options - Estimate costs

Q&A (Please Complete Session Surveys)

Sample Maps

Sample Screenshots Layout (preferred)

Sample Screenshots Layout

Railroads

Switzerland

Oil Platform

Norway

Building/Room

Panama

Utility Network

Germany

Utility Network

GGGGeGeGermrmrmanananyyyy

University

Pennsylvania

Underground Utilities

Texas

Grids for Images/Screenshots (may ask designer for assistance)

environmental conservation

disaster response

demographic analysis

Diagrams/Icons

Arrows for Connecting Items

Arrows

Arrows for Connecting Large Concepts

Shapes for Diagrams

ArcGIS

ArcGIS

ArcGIS

ArcGIS

ArcGIS

ArcGIS ArcGIS

ArcGIS

ArcGIS

ArcGIS

ArcGIS

ArcGIS

Quick Style: Subtle Effect

Quick Style: Moderate Effect

DON’T APPLY EFFECTS from the Design tab

Shapes for Diagrams (continued)

Circle behind a group of objects

Content box for each tier (see sample diagrams)

Optional: Use as a frame around showcased screenshots

Cloud optimized for use behind diagrams

Cloud for general diagram

Sample Diagrams

ArcGIS Supports Multiple Implementation Patterns Working Independently and As An Integrating System

Enabling Enterprise and Pervasive GIS abling Enterprise and Pervasive GIS

Cloud / Web GIS Desktop

E

Server

ArcGIS 10 — A Complete System

Easier More Powerful and Everywhere

Cloud

Enterprise

Local

• Discover • Create • Manage • Visualize • Analyze • Collaborate

Mobile

Desktop

E

Web

Quick Style — Moderate Effect

GDB Share Point

REST Flex

Silverlight

Open API’s SOAP

WCS

ISO

KML WFS

CAD

WWW

SQL

INSPIRE

Standards

OGC

Java WMS

Open Data Access

Esri is Publishing Our REST Interface as an Open Standard

Any Application

REST API Open REST API

Access the Entire Icon Library

Over 110 items added in 2011

• 535 total icons available for Esri use

• Browse and search from any Microsoft Office application

• New! Read-me PDF with complete instructions \\pizzabox.esri.com\space\•Diagrams\Elements\

ArcGIS Desktop Mashups

Explorer Map

Map Map

Web Map

Open Standards Web Map Web Map

Web Map

Browser

Web Blog Web Blog

ArcGIS Desktop Authors

Raster Files

Mashups

Raster Files Web Map Map Web Map Map Web Map

Map Web Map Map Web Map

Map

A Selection of Frequently Used Icons

ArcGIS Online

Web Map

Globes

Layers

CD/DVD

Files

Files

Models

Internet/Cloud

Databases

Table

Network Legend

Business Partner

Education

Professional Services

Professional Services

Designing & Planning

Situational Awareness

GIS User Mobile GIS User

Education

Data Appliance

Data Server

GIS Users

Mashups

Geodatabase Web GIS

1”

.75”

.75”

Title Safe Area — Please Read

This area is the safe area Information outside the title safe area runs the risk of being cropped

when captured to video.

Please place all text and graphics within this area.

1”

Presentation Guidelines

• Know your target audience

• Use slides to lead not read

• 1-5 words per bullet point

• Avoid too much animation—Keep it simple!

Additional ESRI presentation resources available on ArcZone http://arczone/resources/presentations.cfm

Need Help Updating Old Files?

In November, 2010, the corporate template was rebuilt from scratch and fixed problems found in previous versions.

• Download the instructions and support files from http://arczone/resources/presentations.cfm

Color Guidelines

Color Swatches

Use Esri 2011 as theme colors (see presenter’s notes for the instruction of how to add the Esri theme and theme colors)

Projector Color Guidelines

Use the sRGB video mode on the projector. Most projectors have this setting.