Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
State of the Stack - 2013Game. Over. OpenStack is The Stack.
@randybias
Fix subtitle/subtext
Who
3
OpenStack FoundationBoard of Directors
Prod. OpenStack pioneer, Cloudscaling:Wins: KT, Internap, LivingSocial, Seagate (EVault), IBS Datafort, major U.S. carriers, & othersPart of OpenStack community since July 2010 (launch)
Top 10 Cloud Computing Pioneer
Need to add some more personable stuff to thisFix bulletstop 10 cloud computing pioneer InformationWeek
I run an OpenStack product company
I believe the pioneers to emulate are:
I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switches
My Bias
4
Logo-ize
5
1 What is OpenStack?
3 History & Momentum
4 Stackology - a stack taxonomy
5 Stacking it Up - a dive into the projects
6 Stack Gaps - what’s missing?
7 Stack Politics - who’s playing?
9 Summary
2 Why the Success?
8 Who’s using it and how?
OpenStack From 10km
7
Networking
OPENSTACK CLOUD OPERATING SYSTEM
Standard Hardware
Compute Storage
Your Applications
OpenStack Dashboard
OpenStack Shared Services
APIs
OpenStack Mission
8
"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to
implement and massively scalable."
Code Community
Fix the pic by changing the fonts on the Code and Community
OpenStack Foundation Mission
9
The OpenStack Foundation is an independent body providing shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.
The ubiquitous cloud computing platform
Think: Apache Software Foundation
What it is
Some say ...... it’s an Infra-as-a-Service (IaaS)... it’s a cloud operating system... it’s a tool for building private clouds
We say it’s “The Stack”... think Linux... think Java... think ubiquitous open source cloud toolkit... think Game Changer
10
The Battle is Over (open src)
11
OpenStack Launch
OpenStack CloudStack Eucalyptus OpenNebula
Source: trends.google.com
OpenStack, CloudStack, Euca, and OpenNebula (not sure which is which in the last two colors)
Battle is Nearly Over (closed src)
12
OpenStack vSphere vCloud
Source: trends.google.com
Fix slide before this one so both use same color for graphics.
also label colors (D’OH!! this one is vSphere, then OpenStack, then vCloud)
Linux 2000 vs. Linux 2009
13
Is this OpenStack’s Trajectory?
Operating system family market share
2000 2009
Unix Linux
Linux Unix
Source: Linux Magazinehttp://www.linux-mag.com/id/7749/
Linux
UNIX
BSD
Windows/Other
Mixed
Fastest Growing Global Open Source Community
14
COMPANIES
TOTAL DEVELOPERS AVERAGE MONTHLY CONTRIBUTORS
CODE CONTRIBUTIONS
929 245 3,241
189TOP 10 COUNTRIES
9,000+INDIVIDUAL MEMBERS
“OpenStack appears to be a more advanced or more modern open source project than some of its predecessors because it's a highly coordinated effort.”
– Charlie BabcockInformation Week
COUNTRIES
100United States, China, India, Great Britain, Australia, France, Russia, Canada, Ireland, Germany
15
Grizzly StatsCONTRIBUTORS
PATCHES / DEV NEW DRIVERS TEST CLOUDS DEPLOYED DAILY
~14 15 700
517 (+56%)TOP 10 CONTRIBUTING COMPANIES
7,620PATCHES SUBMITTED
“OpenStack appears to be a more advanced or more modern open source project than some of its predecessors because it's a highly coordinated effort.”
– Charlie BabcockInformation Week
NEW FEATURES
230Red Hat, Rackspace, IBM, HP, Nebula, Intel, eNovance, VMware, Cloudscaling, DreamHost
Who or What Should We Thank?Rackspace for Letting GoOpenStack Foundation & Community
Particularly, all of the companies who realized this could be big
Hype CurveThe OpenStack Infrastructure TeamOslo Project (openstack-common)
A thankless job allowing shared code & cleaner projects
The Big Enterprises for Driving InterestPTL Generational Shift
17
Infrastructure Team
Massive Effort -> Improved Quality
Gated CommitsAll Code Has to Jump Through GatesTempest Test Framework
Code Reviews & Continuous Integration
Jenkins, GerritAt scale: jenkins.openstack.org
18
1/4 pages
Jul
Inaugural Design
Summit in Austin
2010
20
OpenStack launches with 25+ partners
First ‘Austin’ code release
with 35+ partners
Oct Nov
First public Design
Summit in San Antonio
AustinOpenStack Object Storage prod
OpenStack Compute dev preview
Launch!
2010 - The Launch Year
Source: Too many to list; blame me for inaccuracies
FIX TIMELINE HERE
2011 - Growing Pains & Early Adopters
Feb
2nd Summit
21
Rackspace announces plans
to launch independent Foundation in
2012
OctApr
3rd Summit (Santa Clara)
adds Conference
Governance moves forward with project technical leads (PTL), policy board elections (PPB)
Jul
First Anniversary
BexarOpenStack Compute for mid-size prod
OpenStack Image Service added to core
CactusOpenStack Compute for larger-
scale prod
Sep
DiabloMajor stability release
First 6-mo cycle release
2011
Decision to shift from 3-mo to 6-mo dev cycle
Jan
1st Swift Public Cloud
Internap w/ Cloudscaling
Happy Birthday!
1st Nova Public Cloud
Internap w/ Cloudscaling
Created framework for Foundation as a community
Feb Apr Aug
19 companies announce
public support for Foundation
Framework & documents ratified
by community
22
Drafting committee formed – creating legal documents
OpenStack Foundation“officially” launches
Sep
EssexOpenStack Identity in core
OpenStack Dashboard in core
FolsomOpenStack Block Storage in core
OpenStack Networking in core
2012
May
HP Cloud
Launch(Beta)
Citrix Bails(how’s that going for ya?)
Jan
AT&T Joins OpenStack
Internal production (private)
Oct
Gartner Report
(teeth gnashing followed)
Inaugural OpenStack Foundation Board
meeting
VMware, Intel, & NEC accepted as Gold members
Board Elections
2012 - Rise of the Foundation & Prod Deployments
2013/2014 - Breakout Growth Years
Apr Oct
23
Q1
HavanaOpenStack Metering in integration
OpenStack Orchestration in integrationLBaaS?
“I” Release
2014
GrizzlyOpenStack Metering in incubation
OpenStack Orchestration in incubation
2013
First Summit 100% run and
funded by Foundation
First International
Summit(APAC?)
INTO THE FOUNDATION FORMATION
In every single category, the top 3 vendors support OpenStack
Incredible Industry Support
24
top 3 switch vendors top 3 storage vendors top 3 hypervisors
top 3 router vendors top 3 blade vendors top 3 linux vendors
top 3 x86 vendors
Dev Growth by Git Contributors
Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=3120
27
Accumulated Community
Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=3120
28
Growth by Domain (company - roughly)
Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=3120
29
For the CloudStack projects, influence from Citrix is quite obvious, over 45% of github.com commits come from accounts belonging to citrix.com and cloud.com.
0
750
1500
2250
3000
Austin Santa Clara San Francisco Portland
Summit Growth
30
Austin - 75San Antonio - 200Santa Clara - 500Boston - 650San Francisco - 1000San Diego - 1320Portland - 2600+
Established Marketing ReachOpenStack.org 241k/visits month:
Software: 300K downloadsMembership: 9000+, Over 90% subscribe to newsletter Relationships with Tier 1 publications and analysts
31
17,693Followers
(+50% from 8/12)
DATA REFRESH
OpenStack From 10km
33
Networking
OPENSTACK CLOUD OPERATING SYSTEM
Standard Hardware
Compute Storage
Your Applications
OpenStack Dashboard
OpenStack Shared Services
APIs
CLI toolsDashboard Other tools
ComputeNetworkingOrchestration
Hypervisor(s) QueuingDatabase
/ KVS/ Cache
External Block
Provider
Physical Network Provider
Provisioning
Log Aggregation
Health Monitoring
etc.
REST Meter Data
REST
SQL, etc. Varies Varies Varies AMQP/0MQ
Topology & Metadata
MeteringREST
REST REST
DNS
ImageManagement
IdentityManagement
REST
Block Storage
Object Storage
OpenStack (m)Architecture Slide
34
UI Layer
ElasticServices
Layer
Oth
er s
tuff,
you
pro
babl
y ne
ed/w
ant
Data &Resource
Layer
SharedServices
Layer
Fix slide nameMake it clear that top right box is all OpenStack and the rest isn’t (change background probably)
Project Name Description Layer AWS
Equivalent Codename
Dashboard Self-service, role-based web interface for users and administrators UI Console Horizon
Compute Provision and manage large pools of on-demand computing resources
Elastic Service EC2 Nova
Block Storage Volumes on commodity storage gear, and drivers for turn-key block storage solutions
Elastic Service EBS Cinder
Object Storage
Petabytes of reliable storage on standard gear
Elastic Service S3 Swift
Networking L2-focused on-demand networking with some L3 capabilities
Elastic Service VPC Quantum
Orchestration Application orchestration layer that runs on top of and manages OpenStack Compute
Elastic Service
CloudFormation, CloudWatch Heat
Metering Centralized metering data for all services for integration to external billing
Shared Service N/A Ceilometer
Identity Multi-tenant authentication system that ties to existing stores (e.g. LDAP) and Image Service
Shared Service None Keystone
Image Management
Upload, download, and manage VM images for the compute service
Shared Service
VM Import/Export Glance
35
NEEDS UPDATE -‐ “Projects” and change to “Codenames” for the names
Revisit descripBons
6 month integrated release cycle
Every 6 months, we coordinate and integrate:Thousands of patches & commitsAcross hundreds of developersAnd 9 “integrated” or “core” projects
Completely impossible without:The OpenStack infrastructure team (CI, etc.)Dedicated PTLs and individual developers
No other similar project does this
36
OpenStack is Well Organized
Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=3120
37
“[the coordinated releases are] an indicator that the OpenStack project is well organized in terms of sub-project management.”
Figure 11 shows the monthly number of commit operations for the sub-projects of OpenStack. Generally speaking, the commit frequency of the Nova sub-project is about 3 times as high as the other sub-projects. It should be noted that although the commit frequency of these sub-projects are different, but they exhibit similar time-series curves, and their highs and lows occur at the same period of time. This indicates that although these sub-projects are relatively independent, but they work around the same development plan and the same release schedule. This is an indicator that the OpenStack project is well organized in terms of sub-project management.
A Quick Note of Thanks
39
These diagrams would not have been possible without the prior work of:
Ken Pepple, Solinea (@ken_pepple)Dina Belova, Mirantis
... and the help of several Cloudscalers:Eric Windisch (@ewindisch)Joe Gordon (http://github.com/jogo)Matt Joyce (@openfly, http://www.music-piracy.com)Dan Sneddon (@dxs)Joseph Glanville (@jpgvm)
Caveat Emptor
40
The focus for these diagrams was ease of reading, not accuracy.
See Ken Pepple’s originals or the code if you need truth.
That being said, our team tried really hard for accuracy.
Blame me for any errors.
OpenStack RPC
42
{ 'oslo.version': '2', 'oslo.message': json ( { 'method': 'method_name', 'args': { 'keyword': 'value' } } )}
nova-api nova-scheduler
Remote Procedure Call(invoked via (a)synchronous message passing)
Generic what’s it for slide: Python code marshals an object, discovers where to send it, sends out a fire and forget message* (cast, call, blah), receiver gets it, unmarshals the object, takes action
OpenStack Compute (Nova)
43
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Compute Thoughts
44
Nova still runs best w/ KVMDo we need another hypervisor? What’s the biz case?
Multiple Availability Zones still not solvedCells are for making one AZ bigger
complex, tight-coupling
We need a clean sharding mechanism for AZesalso what about Cinder and Quantum?
Integ. to Cinder & Quantum needs rethinkMore information needs to be able to be passed back
Compute (Networking) Thoughts
45
nova-network still requiredQuantum has been L2 focused & L3 gap still exists
centralized nova-networking is #fail
decentralized is more #failnova-conductor security for hypervisor obviatednova-metadata-api & nova-network on every hypervisor?
security implications
reconciling Quantum and nova-network?Quantum needs more L3 capability, but ...
OpenStack Block Storage (Cinder)
46
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Block Storage Thoughts
47
Default “nova-volume” func. is too minimalWhen people think Block Storage service they assume:
Persistent, Network-based, & Performant - it isn’t
Cinder scheduler needs info from NovaAssuming you want to do anything interesting
Point of lock-in since default isn’t usefulYou have to place a bet on a block storage solutionThese are expensive, experimental or proprietary
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Networking (Quantum)
48
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Networking Thoughts
49
Default networking functionality is minimalThe APIs have been L2-centricL3 functionality is same as existed with nova-network
Same architecture, same basic layout, with all of the downsides
Needs a Quantum plugin for full func.Can’t run more than one plugin at a time per functionOnly truly baked plugin is probably Nicira?
Others in process, but it’s not clear how many production deployments there are
Good news is that this area is hotSo hopefully this is unstuck soon
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Object Storage (Swift)
50
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Object Storage Thoughts
51
Swift has continued to lag OpenStack dev3 yrs on, auditor is slow & does not prioritize replicationContainer replication is a bad hack
Ugly stepchild of OpenStackKeystone authentication woes (integration, performance)Isn’t universally loved like Nova
Sad, since it was the more mature of the two projects at launch in 2010
Good news: this area has new playersEVault, Seagate, SwiftStack
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Image Mgmt (Glance)
52
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Image Mgmt Thoughts
53
Still not clear why this is standalone projectReally a sub-function of Compute
Semi-pluggable (but not really)Uses different backends for image storage
To be really useful it needs more features:P2V, V2V, and other image conversionAbility to slipstream PV drivers into imagesConvert from popular formats: OVF, AMI, etc.
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Identity (Keystone)
54
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Identity Mgmt Thoughts
55
Mixed identity / schizophreniaVerifies identity, authorization, AND service registry
Service registry is one of manyNova, Cinder, et al have their own internal registries
Slows everything downSee LivingSocial preso from Folsom SummitSee caching tricks with memcache some projects use
OpenStack needed to reinvent wheel here?This could have just been LDAP with a schema + caching
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Dashboard (Horizon)
56
Source: Wikipedia (http://en.wikipedia.org/wiki/Model–view–controller)
SCREENSHOT??
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Dashboard Thoughts
57
It’s gotten a lot betterSame UI for end-user and admin is bad idea
CloudStack did this and it was a messThe workflows and views are too differentSecurity considerations exist
General lag: many things aren’t accessible e.g. Heat
Need better docs on extending, w/o harmCustomers, product companies, SPs all want to modifyAllow for customizations, while supporting upgrades, etc.
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Metering (Ceilometer)
58
STEAL OTHER’S DIAGRAMS?
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Metering Thoughts
59
Metering systems are hardBad or incomplete data for SPs is existentialMetering system should be *very* baked (is 1yr enough?)
No tokenized meter dataInstance hours not enoughHow do you bill for Windows, Oracle, RHEL licenses?Tokens stack: size of instance, OS and app licenses, etc.
Needs to get flow data from edge switchesNetflow and/or Sflow support for physical switches
Data from the vSwitches is not the best source in the real world
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
OpenStack Orchestration (Heat)*
60
* Source: http://www.slideshare.net/dbelova/openstack-heat-slides
What is it? What does it do? What other projects does it talk to?What’s an equivalent elsewhere? (outside of OpenStack)Basic timeline / evolution
Orchestration Thoughts
61
Huge potentialAdds additional AWS func: CloudWatch, CloudFormationProvides clean templates for stacks, which means:
OpenStack on OpenStack (OoO) for testing, etc.
First primary project that rides “on top”Clear differentiator over other projects
Initiative: Heat templates for Ref ArchVendors, customers, etc. could feed to prov systems:Crowbar, Piston, Cloudscaling, etc.
Data Point: Rackspace Block Storage is a hand-rolled iSCSI+LVM on Linux solution.
Strengths:De facto winnerIncredible communityUnstoppable velocityClear innovation curve
SWOT: OpenStack
62
Opportunities:Build an SQL92 base for cloud compute (see Threats)Public cloud compatibility as first order initiativevCloud private cloud compatibility as first order initiative
Weaknesses:No benevolent dictatorLack of IaaS experience for many developersInteroperability will be difficult
Not impossible, *difficult*
Threats:Splintering, fragmentation, and customizationForking or ivory tower thinking
General slide on OpenStack SWOT???? To wrap this section?
What’s In a Complete Cloud OS?
64
OpenStack Relationship
Who? OpenStack Score
Ecosystem Score
Vendor Target*
User Interface(s) Horizon, CLI, ... OpenStack, Vendor, Ecosystem 4 2 4
Elastic Resource Management
Nova, Swift, Quantum, Cinder, ...
OpenStack, Vendor, Ecosystem 4 1 4
Service Discovery Scattered:Nova, Keystone, ...
OpenStack, Vendor 2 0 4Authentication, Authorization, and Access Controls (AAA)
Keystone (authen/author), various projects (ACLs)
OpenStack 2 0 4HW/SW Life Cycle Management
N/A Vendor, Ecosystem 1 2 4Service Management N/A Vendor 0 1 4Health & Logging N/A Vendor, Ecosystem 0 0 4Topology & Inventory N/A Vendor 0 0 4Hardware Certifications N/A
OpenStack, Vendor, Ecosystem 1 1 2**
* We’re all _trying_ to close this gap ** It’s a hard problem no one will solve individually
Your Basic Choices
Download OpenStack and DIY
OpenStack Distributions
Turn-key Systems powered by OpenStack
65
1
2
3
Fix slide
Types of OpenStack Players
67
Type Description Example
Hardware Vendor Selling hardware that integrates or supports OpenStack Juniper, NetApp, Cisco, EMC
Component VendorPoint solution, usually software, that
provides subset of OpenStack functionality or supports it
Midokura, Nexenta
Distribution / Packager
Basic packaging, some installation/setup, etc. RedHat, SUSE, Canonical
Turn-key System Complete, integrated, OpenStack solution, with value adds Cloudscaling, Nebula, Piston
Service Companies Professional or managed services to customize or operate OpenStack
Mirantis, Metacloud, Rackspace Private
Public Clouds Public IaaS HP, Rackspace Public
PaaS / ISVs Value add on top of OpenStack deploymentsScalr, ActiveState (Stackato),
CloudFoundry
Private Clouds Users Wikimedia, AT&T, Yahoo!
components - compute
components - storage
systems
Linux distros public clouds private clouds
PaaS / layered ISVsservice companies
components - networkhardware
Who’s Playing the OpenStack Game?
68
Put “top 5” in each box - get more complete list of “top 5” - consider breaking out component suppliers by network, storage, hypervisor, etc.
components - compute
components - storage
systems
Linux distros public clouds private clouds
PaaS / layered ISVsservice companies
components - networkhardware
Player Motivations
69
Sell Hardware Sell SDN Software
Sell Storage Software
Sell HV Software &
Support
Sell Turn-key Systems &
Support
Sell Labor (T&M), Monthly
Management, etc,
Sell Software on Top of IaaS
Sell Support via “owning” the community
Sell Online Cloud Resources
Use OpenStack for Business Leverage
Put “top 5” in each box - get more complete list of “top 5” - consider breaking out component suppliers by network, storage, hypervisor, etc.
First OpenStack Survey
71
414#survey#responses#
16%
7%
8%
4%
11% 17%
37%
More#than#10,000#employees#5,001#to#10,000#employees#1,001#to#5,000#employees#501#to#1,000#employees#101#to#500#employees#217100#employees#1720#employees#
Company Size
Information Technology 60%#
Academic / Research 15%#
Telecommunication 10%#
Industries
Government / Defense 3%#
CC Icons http://vathanx.deviantart.com/
175 29 28 23 18
56 countries
Country
124$100$
77$
151$
Service Provider
Ecosystem Vendor
Cloud Consumer
Cloud Operator
Type of Involvement
Deployments at a Glance
Type
35#Hosted#Private# 15#
Hybrid#
#37#
Public#
106#On#Premise#Private#
Trunk&8%&
Grizzly&15%&
Folsom&47%&
Essex&25%&
Diablo&5%&
Version 84
92 94
Production Proof of Concept
Dev/QA
Stage
134#
94#
94#
89#
66#
46#
Dashboard
Object Storage
Snapshotting to new images
Live Migration
EC2 Compatibility API
S3 Compatibility API
Features
181#
171#
169#
153#
147#
121#
103#
20#
16#
Nova
Glance
Keystone
Horizon
Quantum
Cinder
Swift
Ceilometer
Heat
Components
197&Deployments&
Size of 98 Production Compute Systems
73
1"100$$52%$
101"500$18%$ 501"1,000$$
8%$
1,001"5,000$$8%$
5,000"10,000$$3%$
>$10,000$$6%$
Unspecified$5%$
Other$30%$
Instances
1"50$$71%$
51"100$$8%$
101"500$$9%$
501"1,000$$2%$>1,000$$4%$unspecified$
6%$
Other$29%$
Nodes
1"100$$51%$
101"500$$21%$ 501"1,000$$
4%$1,001"5,000$$
12%$
5,001"10,000$3%$
>10,000$$4%$
unspecified$5%$
Other$16%$
Cores
Usage: KVM, LVM, OVS & SQL
74
KVM$71%$
ESX$8%$Xen$8%$
Xenserver$5%$
Lxc$5%$Hyperv$3%$
Other$29%$
Hypervisors
LVM$36%$
NFS$19%$
Ceph$RBD$13%$ Netapp$
10%$GlusterFS$
8%$SAN/HP$5%$
Windows$4%$EMC$3%$
Solidfire$2%$
Other$32%$
Open$Vswitch$39%$
Linux$Bridge$31%$
Cisco$11%$ HyperUv$
5%$
Nicira$5%$
Brocade$3%$Ryu$2%$
big$switch$2%$NEC$2%$
Other$19%$
Storage Drivers
Network Drivers
SQL$55%$
LDAP$34%$ PAM$
8%$
KVS$3%$
Other$11%$
Identity Drivers
150$
62$
33$
JSON$ XML$ Both$
API Format
OpenStack by TKO?
OH: “Finish him!”We still have work to do
Your participation mattersRegardless of whether you: build, develop, or operate
Get involvedhttp://is.gd/openstack
76