Upload
marat-zhanikeev
View
264
Download
0
Tags:
Embed Size (px)
Citation preview
.
Visitation Platform + Micropayments.Problem is.........that cloud federations are rare ... because no one knows how to build them
• most federations have rigid structure (can't grow)
• heterogeneous environments are a major problem• there are no examples of resource economies in federations
.Visitation 01 ......... is when VMs can visit with any cloud environment
.Micropayments 02 .....
....is a technology for resource accounting in federations01 myself+0 "A Cloud Visitation Platform for Federated Services at Network Edge" CISSE (2014)
02 myself+0 "Coins in Cloud Drives Can Use OAuth for Micropayments and Resource Metering Alike" CFI (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 2/21...
2/21
.
The 3 Know-Hows
1. the visitation platform itself, compatible with Xen platforms 01
◦ migration, APIs, interfaces, automation
2. binary diff technology for efficient migration within a federation 03
3. cloud coin for resource metering in an open federation 02
01 myself+0 "A Cloud Visitation Platform for Federated Services at Network Edge" CISSE (2014)
03 myself+0 "「Cloudy会」DIFF法:テキスト・ファイルシステム・バイナリ差分" http://goo.gl/v8kLpQ (2015)
02 myself+0 "Coins in Cloud Drives Can Use OAuth for Micropayments and Resource Metering Alike" CFI (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 3/21...
3/21
.
Visitation Platform Basics
Storage Sensors
…
Cloud Visitation Platform
Container
Cloud Visitation Platform
App App App …
APIs
…
Multitenancy
Physical Device
… Interfaces Billing,
Fairness
Generics, Resilience
Application Specifics
• today: VMs are running inblackboxes, zeroawareness of local hardware
• visitation: the oppositeapproach -- VMs/containersare fully aware of localresources 01
• implemented as APIs/Interfaces
• not hard -- just yum installand add your software ontop of XCP
01 myself+0 "A Cloud Visitation Platform for Federated Services at Network Edge" CISSE (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 4/21...
4/21
.
Cloud Coin BasicsCRITICAL-ness
Distributed Resource APIs
• micropayment: payment ofa very small amount (sub-yen)
• problem: a fixedtransaction cost per pay
• solution: open coin,service can accumulatepayments (cost per batch)
• cloud coin: an ecoin storedin a cloud drive 02
02 myself+0 "Coins in Cloud Drives Can Use OAuth for Micropayments and Resource Metering Alike" CFI (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 5/21...
5/21
.
Cloud Coin = OAuth-shared JSON file• can be a simpleJSON file stored in a cloud drive (Dropbox, ...) 02
• client - service provider: check/deduct the amount, store/changethe certificate (md5)
• service - coin provider: payments are cleared based on certificates◦ note: there is a probability of collision, the coin is lossy, loss ratio is controlled byservice statistically
Client ID
Coin Structure (JSON)
Coin ID
Coin Amount
Latest Certificate
Other Info
02 myself+0 "Coins in Cloud Drives Can Use OAuth for Micropayments and Resource Metering Alike" CFI (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 6/21...
6/21
.
From Talk to Action
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 7/21...
7/21
.
Talk → Action : A Practical Visitation
• say, I have (1) Hadoop and (2) a BigData replay platform 05
◦ bigdata replay on multicore, data streaming sketches, etc.
• I want to let other people/clouds to schedule jobs on my platform• I can let VMs visit with my platform to:
1. run jobs locally2. process data and run more jobs3. ...
• I need cloud coin as a quota meter◦ each VM has to let me know where its coin is before it visits
05 myself+0 "Replayable BigData for Multicore Processing and Statistically Rigid Sketching" IC2014 (2014)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 8/21...
8/21
.
2 Demos: BigData Replay, Cloud Coin
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 9/21...
9/21
.
Step 1: VM Migration Automation
• the problem is to receive and run VM image as.xva in any network environment
• PV_args is the easiest way, visitationtools in VM parse network environment in /proc/cmdline1. xe... vm-import filename=?.xva2. xe ... PV_args3. xe ... vm-start
• when done, check if VM is on the network andnotify the remote client
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 10/21...
10/21
.
Step 2: .xva diffs
• idea: keep a library ofreference .xva filesand send only diffs formigration
• 90-95% reduction involume
• diff is definitely underratedand underused in cloudstoday 03
03 myself+0 "「Cloudy会」DIFF法:テキスト・ファイルシステム・バイナリ差分" http://goo.gl/v8kLpQ (2015)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 11/21...
11/21
.
Step 3: the DiffHub technology
Cloud Drive
User A
User B
New
Old
Old
Diff Diff
Old version New version Binary diff
DiffHub
• work in progress
• a DiffHub for federations
• a file synctechnology where onlydiffs are transferred
• biggest merit: masterstorage is for diffs only
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 12/21...
12/21
.
Step 3b: Docker wants diffs?
• Docker's main feature is filesystem diffs◦ .. which is a redundancy avoidance technology
• binary diffs can be more efficient than Docker filesystem layers
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 13/21...
13/21
.
Practical Visitation: Hadoop
• problem: very small publicIP space (security, rules, etc.)
• solution: run the VMs inprivate space with Internetaccess
• demerit: outside cannotpush into your VM, but yourVM can pull anything
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 14/21...
14/21
.
Practical Visitation: Hadoop (2)
• VM installs visitation tools,which know where to find theAPI for its PM
• the API is generic --provides any functionality
• storage, sensors, indexing,hadoop jobs, etc.
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 15/21...
15/21
.
Public vs Private Cloud (looks like this)
• you need at least 2 PMs, 1 public + 1 private• note: hide your PM! (non-public IP on thesame subnet)
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 16/21...
16/21
.
Practical Visitation : Cloud Coins
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 17/21...
17/21
.
Push vs Pull
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 18/21...
18/21
.
Visitation Platform: PULL Type
Cloud Visitation Platform
App App App …
Pull-Based Installation (of Apps)
End User
Device Provider 3rd Party
Service
Local services
One Cloud
• PULL: heterogeneous,large-scale cloudfederations, fog computing
• active probing network, sensorcloud, vehicular cloud, etc.
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 19/21...
19/21
.
Visitation Platform: PUSH Type
Cloud Visitation Platform
App App App …
Cloud/device Owner
3rd Party Service Federated
Clouds
Local services
Federated Cloud
Manager • PUSH: trusted cloudfederation betweenlarge providers
• DC-DC cross-providerclouds
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 20/21...
20/21
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- VM Visitation and Resource Micropayments ... towards Cloud Federations -- http://bit.do/marat150227 21/21...
21/21