AIAA JPL Affordability Analysis and Cloud Costing finalgalorath.com/wp-content/uploads/2014/08/AIAA JPL Affordability...Massive Scale Resilient Computing ... BRM 2 Launch Solution

And the Sky is not Cloudy all Day: Cloud Costing with an Affordability Focus: JPL AIAA Meeting 3/13

Galorath Incorporated

Daniel D. Galorath: Founder & CEO

[email protected]

Copyright Galorath Incorporated 2013

Key Points

Viable affordability

decisions yield project

achievements

Applying affordability

process to cloud computing decisions

ensures ROI

Cloud computing usage is

becoming viable more situations

2

© 2012 Copyright Galorath Incorporated 2

Use An Affordability Process To Determine Best Value

Step 1. Procure Key Performance Parameters that

are inviolate

Step 2. Identify Affordability Goals & Figures of Merit

Step 3. Gather Requirements,

Features, Performance

Step 4. Define Baseline

Alternatives & Assumptions

Step 5. Perform Technical Design Analysis for Each

Alternative

Step 6. Perform Cost Schedule

Analysis of Each Alternative

Step 7. Assess Benefits Based on Figures of Merit

Step 8. Perform Probabilistic Risk

Analysis

Step 9. Assess Alternatives & Select Optimal

Alternative

Step 10. Document

Analysis and Lessons Learned

Basic Cloud Terminlogy

• Infrastructure as a Service (IaaS) Run whatever software you want over the internet (operating systems, applications, Etc.)

• Platform as a Service (PaaS) Your IT staff designs and develops its own custom software, but they use the APIs and tools to build them on the cloud platform (Running on Iaas)

• Software as a Service (SaaS) Software available over the internet (Running on IaaS)


The NIST Cloud Definition Framework

5

CommunityCloud

Private Cloud

Public Cloud

Hybrid Clouds

DeploymentModels

ServiceModels

EssentialCharacteristics

Common Characteristics

Software as a Service (SaaS)

Platform as a Service (PaaS)

Infrastructure as a Service (IaaS)

Resource Pooling

Broad Network Access Rapid Elasticity

Measured Service

On Demand Self-Service

Low Cost Software

Virtualization Service Orientation

Advanced Security

Homogeneity

Massive Scale Resilient Computing

Geographic Distribution

Based upon original chart created by Alex Dowbor - http://ornot.wordpress.com

Step 1 Key Performance Parameters (KPPs)• Key Performance Parameters Defined: Critical

subset of performance parameters, capabilities and characteristics so significant that failure to meet them can cause concept or system selected to be reevaluated or the project reassessed or terminated. (Adapted from Glossary of Defense Acquisition) Example Characteristics: • Essential for defining the required capabilities?

• Contributes to significant improvement in the operational capabilities of the enterprise?

• Achievable and affordable?

• Measurable and testable/verifiable?

• Can KPP attribute be analyzed throughout the life cycle?

• If not met, will the sponsor of the project be willing to cancel or significantly restructure the project?


Step 2. Identify Affordability Goals & Figures of Merit• Figure of merit: A quantity used to characterize the

performance of a device, system or method, relative to its alternatives e.g.• Cost

• Response time of a computing action

• Survivability

• Calories in a serving

• Resolution of a digital camera

• Battery life

• Coverage


Used to compare alternativesFor example more cheaper UAVs may provide

better coverage for the same $ than fewer more powerful UAVs

Used to compare alternativesFor example more cheaper UAVs may provide

better coverage for the same $ than fewer more powerful UAVs

8

Mission Design

Key Figures of Merit (Source NASA Space Systems Engineering)

19/0

6:00

19/0

0:00

18/1

8:00

18/1

2:00

18/0

6:00

18/0

0:00

17/1

8:00

17/1

2:00

17/0

6:00

17/0

0:00

16/1

8:00

20/18:00

21/00:00

21/06:00

21/12:00

21/18:00

22/00:00

22/06:00

22/12:00

22/18:00

23/00:00

23/06:00

L1 Departure Date (dd/hh:mm)

Eart

h Va

cuum

Per

igee

Arr

ival

Dat

e (d

d/hh

:mm

)

L1-Earth Co-Planar Inbound Delta V Requirement (m/s)

700-800 800-900 900-1000 1000-1100 1100-1200 1200-1300 1300-1400

• Moon: Inclination near maximum, Distance near perigee• L1 Departure Time in June 2006

900

800

9001000110012001300

800

Key Figures of Merit

Safety• # of Critical Events• Mission Complexity

• Abort Options• Crew Time

• Technology Risk• Probability of launch

success• Etc.

Effectiveness• Total Mass• Dry Mass

• Surface Time• Etc.

Extensibility• Long-Stays

• Mars• Other destinations

• Etc.

Reference Operations Concept

Earth Departure Stage Expende

d

MOONMOON

LEO 407 km

Continue Missions

Expended

CEV Reused?

L1 (~322,000 km)

EARTHEARTH

4 weeks

Low Lunar Orbit

Kick Stage Expended

Water Landing

Service Module

Expended

Earth Departure Stage Expende

d

Initial Mass in LEO

CEV CM , 9 CEV CM , 9 CEV CM , 9 CEV CM , 9 CEV CM , 9 CEV CM , 11 CEV CM , 8

CEV SM , 18 CEV SM , 15 CEV SM , 15 CEV SM , 18 CEV SM , 12CEV SM , 20 CEV SM , 27

CEV EDS # 139 CEV EDS # 1

33CEV EDS # 1

33

CEV EDS # 121 CEV EDS # 1

64CEV EDS # 1

45

CEV EDS # 142

A scent Stg20 A scent Stg

20A scent Stg

20

A scent Stg20 A scent Stg

20

Descent Stg23 Descent Stg

23Descent Stg

23

Descent Stg23 Descent Stg

23Descent Stg

13

Kick Stage27 Kick Stage

27

Kick Stage33 Kick Stage

27

Lander EDS # 194

Lander EDS # 190

Lander EDS # 193

Lander EDS # 125

Lander EDS # 164

Lander EDS # 194

Lander EDS # 142

CEV EDS # 221

A scent Stg, 10A scent Stg20

Descent Stg25

Kick Stage27

Kick Stage26

Lander EDS # 425

Lander EDS # 325

Lander EDS # 225

216 220

241

223

240

142

230

0

40

80

120

160

200

240

280

320

BRM 2 LaunchSolution

3 LaunchSolution

25t Launch Limit Initial Mating inLEO

Aerocapture &Land Landing

"Pseudo-Apollo"

Tota

l Arc

hite

ctur

e M

ass

(kg/

1000

)

Lander EDS #1Lander EDS #2Lander EDS #3Lander EDS #4Kick StageDescent StgAscent StgCEV EDS #1CEV EDS #2CEV SMCEV CM

But When We Look at Figures of Merit• Is the cloud secure enough?

• Is the cloud fast enough?

• Is cloud vender reliable enough?

• Other figures of merit for this system?


Every case is differentWe cant say cloud or on-premises is always better

Every case is differentWe cant say cloud or on-premises is always better

Some Potential Cloud Black Swan Costshttp://www.datacenterknowledge.com/archives/2012/12/05/the-cloudy-side-of-cloud-computing/

• Security & Breaches: Anticipate growing Malicious attacks and accidental data loss

• Outages: 2007- late 2012 568 hours downtime between 13 major cloud carriers. Cost the customer base about $72 million (International working group on cloud computing resiliency)

• Learning curve: Successful cloud model takes knowledge around multiple technological disciplines. Once in place, however, managing can also be issue

• Vendor lock-in: Migrating cloud environment to anther provider difficult… Not often considered

• Data portability and porting costs

• Software modification Costs (PaaS)

• Software Setup (Saas)© 2013 Copyright Galorath Incorporated 10

Dealing With the “Problem of Assumptions”• Assumptions are essential but… Incorrect

assumptions can drive an estimate to uselessness

• Use an assumption verification process


1. Identify assumptions2. Rank order

assumptions based on estimate impact

3. Identify high ranking assumptions

that are risky

4. Clarify high ranking, high risk assumptions

& quantify what happens if those

assumptions change

5. Adjust range of SEER inputs to describe the

uncertainty in assumption

Step 6. Perform Cost Schedule Analysis of Each Alternative


Bad Estimates Are A Root Cause of Project Failure• An estimate is the most knowledgeable statement you

can make at a particular point in time regarding:• Effort / Cost

• Schedule

• Staffing

• Risk

• Reliability

• Estimates more precise with progress• A WELL FORMED ESTIMATE IS A

DISTRIBUTION


We Know How To Estimate Cloud Costs and ROI• Cloud isn't so different that alternate approaches to

cost, ROI or business case are needed

• Important to identify costs that will increase as well as decrease.. E.g. bandwidth

• Risk must be factored in• E.g. data inaccessibility

• Potential issues in requirements for SaaS are the same as packages hosted in house

• Measurement, estimation and ROI processes are essential to make the most viable decisions


Remember History Shows MANY software project nevershow a positive ROI….

The cloud doesn’t solve uninformed decisions

Affordability Alternatives Generally Provide ROM Estimates (Source APMP: Just Say No)

15

Sco

pe

& A

ccu

racy

Us e

1 2 3 4Phase 0-1

ROM

ROM

ROM

ROM

AcquisitionPlanning/ POMand Plus Ups

Market Assessment/

“What If’s”

OpportunityCreation/ Customer

Decision PlansProcurement

Initiation Draft RFP RFP

DDE

Modified Budgetary Estimate

Draft RFP/Gate 36-8 people, 3

weeks(Bid Stds + History

)

Formal BidGate 4

15-20 people4 weeks

(Bid Stds+ History)

EARLY ESTIMATING3-5 people, 3 - 5 days

Top Down, parametric model based price estimating

Vs.Current state: 90 people, 6wks

Affordability Initiatives With “Should Cost” and “Will Cost”

Should Cost Performance

Cost Initiatives (Applied practices& improvements)

Will CostPerformance - =


Many View Bottoms up estimates as the requirement for Should Cost / Will Cost Analysis

But parametrics can do analysis faster as well as provide more tradeoffs

Cloud Costs Study Concusionshttp://www.forbes.com/sites/kevinjackson/2011/09/17/the-economic-benefit-of-cloud-computing/

• Startups Easier: Cloud computing makes web startups easier

• 50=67% lifecycle cost savings: 1,000 server deployment (BAH)

• Greater ROI & Shorter payback: Cloud delivered greater investment returns with a shorter payback compared to traditional on-premise (Deloitte)

• GSA IaaS Should save about 7 to 1: Transitioning IT services from agency-owned IT infrastructure to GSA IaaS platform (Assumed From BAH study)

• PaaS can increase costs: Application portability, particularly in a PaaS scenario, and associated costs can be significant. Microsoft

• Cloud 30% More Cost: One analysis of moving to cloud http://www.uptimesoftware.com/uptimeblog/cloud-virtualization/cost-of-cloud-computing-expensive/


Step 4 Define Baseline Alternatives & Assumptions


Example Showing Lower Cost Cloud Implementation


Step 7. Assess Benefits Based on Figures of Merit• Return on Investment often main criterion in IT

systems


21

Affordability Trades (Source NASA Space Systems Engineering)

ROI Analysis of A New System

Cost of capital 8.0%

Initial Investment Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7Total

Ownership

Investment $100,000 $100,000 Increase/(dec.) in revenue ($40,000) $60,000 $110,000 $100,000 $100,000 $150,000 $150,000 $630,000 Increase/(dec.) in op. exp. $90,000 $70,000 $70,000 $22,000 $24,000 $27,000 $28,000 $331,000 Cash Flow ($100,000) ($130,000) ($10,000) $40,000 $78,000 $76,000 $123,000 $122,000 $199,000

PV of Cash Flow ($100,000) ($120,370) ($8,573) $31,753 $57,332 $51,724 $77,511 $71,186 $60,563

NPV 60,563 $60,563

IRR 13.5% 13.5%

ROI 121% 121.1%

A Complete ROI analysis should analysis risk and uncertainty as well as likely

A Complete ROI analysis should analysis risk and uncertainty as well as likely

Cloud Metrics

• Cost Metrics

• Business Value Metrics

• Quality (Construction) Metrics

• Performance & Reliability (Operations) Metrics

• Usability Metrics

• Customer Satisfaction Metrics


Cloud Economics Fall Apart When Application Needs Rewrite for Cloud• Rewriting applications to make them work in the

cloud

• Dave Linthicum, who also participated in Dana's latest analyst roundtable, points out that there's a lot more to enterprise IT than simply accessing and running applications.

• "Cloud computing typically is going to be a better, more strategic, more agile architecture, but it's also typically going to be more expensive, at least on the outcome,“ Can be lots of costly infrastructure changes Dave Linthicum


Cloud Costs Antidotal Evidencehttp://www.forbes.com/sites/kevinjackson/2011/09/17/the-economic-benefit-of-cloud-computing/

• Cloud computing makes web startups easier

• 1,000 server deployment can save 50% to 67% of lifecycle cost (BAH)

• Cloud delivered greater investment returns with a shorter payback compared to traditional on-premise (Deloitte)

• Transitioning IT services from agency-owned IT infrastructure to GSA IaaS platform should deliver about 7:1 cost (Assumed From BAH study)

• Application portability, particularly in a PaaS scenario, and associated costs can be significant. Microsoft

• One analysis of moving to cloud over 30% more http://www.uptimesoftware.com/uptimeblog/cloud-virtualization/cost-of-cloud-computing-expensive/


Example: Project Cost Alone Is not The Cost of IT Failure (Source: HBR)

• Case Study: Levi Strauss• $5M ERP deployment contracted

• Risks seemed small

• Difficulty interfacing with customer’s systems

• Had to shut down production

• Unable to fill orders for 3 weeks

• $192.5M charge against earnings on a $5M IT project failure

“IT projects touch so many aspects of organization they pose a new singular risk”

http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/ar/1

Key Points

Cloud computing

usage becoming

viable more situations

27


Traditional On Premises Software Total Ownership Cost Allocation

SoftwareDevelopmentSoftwareMaintenaanceIT Infrastructure

IT Services


Development =Biggest Risk

For Cloud Some Costs reduced or eliminated.. Other newCosts occur

For Cloud Some Costs reduced or eliminated.. Other newCosts occur

IT Services & Infrastructure Are Situational but

Generally 60% of TOC

Software/ Database Services

End User Services

Infrastructure Services

Application Development

Service Desk Training Facilities Documents Purchased

Items/Other

Analysis Design Procurement Construction Testing Training Distribution Operations

Labor & Hardware Change From Iaas To PaaS To SaaS

Manager Analyst Architect Developer Administrator Technician Test/QA Tech Writer Trainer

AdministrationEnvironmentNetworkStorageServers

Current Costs of IaaS Are Readily Available


http://tco.2ndwatch.com/#compare

Security Is Often An Issue

© 20103Copyright Galorath Incorporated 31

Building Secure Systems Is Very Costly• Security cost can be massive…

• Depending on the cloud supplier’s investment in security and your organizations investment

• Cost of breaches can be worse


Key Points



ensures ROI

33


We Know How To Estimate Cloud Costs and ROI• Cloud isn't so different that alternate approaches to

cost, ROI or business case are needed

• Important to identify costs that will increase as well as decrease.. E.g. bandwidth

• Risk must be factored in• E.g. data inaccessibility

• Potential issues in requirements for SaaS are the same as packages hosted in house

• Measurement, estimation and ROI processes are essential to make the most viable decisions


Remember History Shows MANY software project nevershow a positive ROI….

The cloud doesn’t solve uninformed decisions

Galorath Force.com Using SEER: FAA Funded Study Findings• Effort reduction from 30% to 40% over JAVA

• About 80% reduction in the actual development effort for appropriate applications

• Point-and-click operations to support prototyping reduced requirements & design from 10% to 25%.

• Testing reduced due to robustness of built in capabilities reduced effort by an estimated 5% to 20%.


Much due to the development environment

Infrastructure cost savings are situational

Much due to the development environment

Infrastructure cost savings are situational

App Development Doesn’t Stop At the Smartphone


ServerService ServiceAPI API

API

Cloud Apps Cobble Existing Components Making Testing and Configuration Mightmares


NginxProxy server

Load balancing through AWS ELB and DNS resolution through AWS Route53

DjangoPython app framework

GunicornWSGI Server

Fabricmulti-instance administration

PostgreSQLUser data

MdadminSoftware

RAID

PostgreSQLUser data

AmazonS3

for photos

RedisKey store server

Apache SOLRGeo search

(CDN)

MemcachedFor caching

Gearmantask queue system

Shares &posts

SentryPython reporting

MuninResource mon. Pgbouncer

pooler

Source:’”What Powers Instagram:Hundreds of Instances, Dozens of Technologies”

Instagram Engineering Blog

replication

Requirements And Features

Analysis

Preliminary Design

Process

•Human Factors•Security

•Reliability•Availability

•Survivability

•Supportability•Testability

•Producibility•Reuse

•Transportability

Other Factors

Alternative 1Alternative 2Alternative 3Alternative 4

SEER Assessment

•Performance•Schedule

•Risk Assessment•Life Cycle Cost

1

23

4 5

SelectionProcess

OK?Optimized?

Yes

CostPerformance

ScheduleRisk

Bottoms Up Estimationas Required

NoIterate

Should Cost: Trade Study Flow


39

What is likely to happen

Feel lucky?

Firm Fixed Price?

Understand the risk before you commit!

Key Points

Viable affordability

decisions yield project

achievements



ensures ROI

Cloud computing usage is

becoming viable more situations

40
