Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Basics of Cloud Computing – Lecture 7
Cloud Computing – Summary and
leftoversleftovers
Satish Srirama
Outline
• Summarize what we have learnt
• Amazon SQS
• Some more PaaS
– Google App Engine– Google App Engine
– Force.com
• AJAX
• Virtualization
24.05.2010 Satish Srirama 2
What is Cloud Computing?
• Computing as a utility
– Consumers pay based on their usage
• Cloud Computing characteristics
– Illusion of infinite resources
– No up-front cost
– Fine-grained billing (e.g. hourly)
• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”
Satish Srirama24.05.2010 3/30
Cloud Computing – Services - Recap
• Software as a Service – SaaS– A way to access applications
hosted on the web through your web browser
• Platform as a Service – PaaS– A pay-as-you-go model for IT
resources accessed over the
SaaS
Facebook, Flikr, Myspace.com,
Google maps API, Gmail
Level of
Abstraction
resources accessed over the Internet
• Infrastructure as a Service –IaaS– Use of commodity computers,
distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data
– Virtualization
Satish Srirama
PaaS
Google App Engine,
Force.com, Hadoop, Azure, Amazon S3, etc
IaaSAmazon EC2, Joyent
Accelerators, Nirvanix Storage
Delivery Network, etc.
24.05.2010 4
Scientific Computing Cloud (SciCloud)
• Distributed Systems Group owned private cloud infrastructure
• Eucalyptus setup
• Goal of the project• Goal of the project
– To Establishing a private cloud at universities
– To efficiently use the already existing resources of universities
– To address computationally intensive scientific, mathematical, and academic problems
• Hope it was a pleasant experience !!!
24.05.2010 5Satish Srirama
Economics of Cloud Providers –
Failures - Recap
• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter
• Caveat: User software has to adapt to failures• Caveat: User software has to adapt to failures
• Solution: Replicate data and computation– MapReduce & Distributed File System
• MapReduce = functional programming meets distributed processing on steroids – Not a new idea… dates back to the 50’s (or even 30’s)
24.05.2010 6/30 Satish Srirama
MapReduce
• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together
• The execution framework handles everything else…• Not quite…usually, programmers also specify:• Not quite…usually, programmers also specify:
partition (k’, number of partitions) → partition for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic
24.05.2010 7Satish Srirama
MapReduce Examples
• Distributed Grep
• Count of URL Access Frequency
• Reverse Web-Link Graph
• Inverted Index • Inverted Index
• Distributed Sort
• MapReduce in Information Retrieval
24.05.2010 Satish Srirama 8
Hadoop Processing Model
• Create or allocate a cluster
• Put data onto the file system– Data is split into blocks
– Replicated and stored in the cluster
• Run your job• Run your job– Copy Map code to the allocated nodes
• Move computation to data, not data to computation
– Gather output of Map, sort and partition on key
– Run Reduce tasks
• Results are stored in the HDFS
24.05.2010 Satish Srirama 9
LEFTOVERS
24.05.2010 Satish Srirama 10
Amazon Simple Queue Service (SQS)
• A reliable, highly scalable hosted distributed queue
for storing messages as they travel between
computers.
• Amazon's web-scale messaging infrastructure as a
web service
• Platform-agnostic, allowing any computer on the
Internet to add or read messages without any
installed software or special firewall configurations
24.05.2010 11Satish Srirama
Amazon SQS Highlights
• Reliable
– Runs within Amazon's high-availability data centers.
– Messages are stored redundantly across multiple servers and data centers.
• Simple: Only Six Methods• Simple: Only Six Methods
• Scalable
• Secure: Strong access controls
• $0.10 per 1,000 messages sent ($0.0001 per message sent)
• $0.20 per GB of data transferred
24.05.2010 12Satish Srirama
Google AppEngine
24.05.2010 Satish Srirama 13
What is AppEngine?
• Google platform to build web application on
the Cloud
• Example of PaaS
• Does one thing well: running web apps• Does one thing well: running web apps
• Request → Processing → Response
1424.05.2010 Satish Srirama
No More LAMP
1524.05.2010 Satish Srirama
Why App Engine
1624.05.2010 Satish Srirama
Application Life Cycle
1724.05.2010 Satish Srirama
Development Life cycle
1824.05.2010 Satish Srirama
Physical Deployment Diagram
1924.05.2010 Satish Srirama
Developing applications with App Engine
2024.05.2010 Satish Srirama
Developing Applications
App Engine SDK
• Web Server
dev_appserver.py
• Application uploader• Application uploader
appcfg.py
• Local DataStore
implementation
• App Engine APIs
2124.05.2010 Satish Srirama
App configuration – app.yaml
application: helloworld
version: 1
runtime: python
api_version: 1 api_version: 1
handlers:
- url: /.*
script: helloworld.py
2224.05.2010 Satish Srirama
Helloworld.py
2324.05.2010 Satish Srirama
Running Application
2424.05.2010 Satish Srirama
Data Store
• A powerful non relational distributed data
storage service(object database)
• Transactional data store
• Query engine• Query engine
– GQL – SQL like language
• Use BigTable in back
2524.05.2010 Satish Srirama
Deployment and Testing
• Local testing
dev_appserver.py helloworld/
• Deploy
appcfg.py update helloworld/appcfg.py update helloworld/
• On the cloud
http://distributedseminartartu.appspot.com
2624.05.2010 Satish Srirama
APIs
2724.05.2010 Satish Srirama
End User Administration
Access Control Authentication / SSO Services
Web Services API
Workflow Engine
A B D
C
Powerful Application Service
Multi-Device Multi-LanguageMulti-
CurrencyReporting &
AnalyticsSearchEngine
28
Force.com - cloud platform for business apps
24.05.2010 Satish Srirama
Infrastructure
Services
Network
Storage
Operating System
Database
Application
Services
Authentication
Security
Sharing
Integration
Operations
Services
Availability
Monitoring
Patch Mgmt
Upgrades
IT
Force.com Lets You Focus on
Innovation
You
InnovationApp Server
Web Server
Data Center
Disaster Recovery
Customization
Web Services
API
Multi-Language
Multi-Currency
Workflow
Analytics
Multi-Device
Messaging
Search
Backup
NOC
Sales force2924.05.2010 Satish Srirama
3024.05.2010 Satish Srirama
Create any table
Add any field
Build any relationship
Database as a Service
Build any relationship
Security & sharing
Automatic tuning
Automatic backups
Automatic upgrades
3124.05.2010 Satish Srirama
Integration as a Service
3224.05.2010 Satish Srirama
History Tracking
Workflow Rules
Logic As A Service
Declarative Logic
Formula-Based LogicFormula Fields
Procedural Logic
Formula Fields
Validation Rules
Apex Triggers
Apex Classes
Apex Web Services
3324.05.2010 Satish Srirama
User Interface as a Service
UI generated
automatically,
no technical
skills required
UI generated
by developer /
technologist
Profile /
Record Type /
Page Layout
mapping
controls UI
behavior
Full
Page Layout Visualforce
Limited
control over
look and feel,
but all UIs are
consistent
Full
control of UI
behavior
Point and click to
add Custom
Buttons and
Embedded
Mashups
3424.05.2010 Satish Srirama
Next Week
• All the best with the exams
• On 27th May 2010, we will have the project
presentations
24.05.2010 Satish Srirama 35
References
• Google App Engine
http://code.google.com/appengine/
• Force.com http://www.salesforce.com/force/
• Amazon SQS http://aws.amazon.com/sqs/• Amazon SQS http://aws.amazon.com/sqs/
24.05.2010 Satish Srirama 36
Virtualization
• Virtualization technologies partition hardware
and thus provide flexible and scalable
computing platformsApp App App
• Types of Virtualization
– Hardware Virtualization
– Virtual Machine Virtualization
– Application Virtualization
24.05.2010 Satish Srirama 37
Hardware
OS
Hypervisor
OS OS
Virtualized Stack
Hardware Partitioning
• Physical system is
divided into multiple
partitions
• One or more CPUs per • One or more CPUs per
partition
• Full electrical isolation
between partitions
• Redundant hardware
components
24.05.2010 38Satish Srirama
Virtual Machine Virtualization
Virtual MachineVirtual Machine
Guest OSGuest OS
(Windows 2003)(Windows 2003)
ApplicationsApplications
Virtual MachineVirtual Machine Virtual MachineVirtual Machine
Guest OSGuest OS
(Windows NT)(Windows NT)
Guest OSGuest OS
(Windows 2000)(Windows 2000)
ApplicationsApplications ApplicationsApplications
V i
r t
u a
l
System HardwareSystem Hardware
Host Operating SystemHost Operating System
Virtual Machine ManagerVirtual Machine Manager
P h
y s
i c
a l
Virtualization platforms
• XEN– XEN 3.0 -- Open source
• Limited guest support
• Requires OS Source code changes
– XEN Enterprise – commercial– XEN Enterprise – commercial• Windows and Linux support
• Virtuozzo– Linux roots
– Virtualizes the OS rather than the hardware
• VMWare
24.05.2010 Satish Srirama 40
Application Virtualization
• Runs as an application
• Virtualizes OS settings
– Registry– Registry
– File System
Ajax
• Asynchronous JavaScript and XML: an approach for building interactive Web applications
• Ajax refers to a number of technologies:• Ajax refers to a number of technologies:
– XHTML/CSS for presentation
– XML for data exchange (or JSON)
– XMLHttpRequest object for asynchronous communication
– JavaScript to tie everything together
From “old-school” Web applications to Ajax…
24.05.2010 42Satish Srirama
“Old-School” Web Applications
server-side systemsbrowser
1
user does something
2
browser sends
request to server
3
server generates Web
page as a response to
the request
backend
database
Inte
rfa
ce
Web
server
HTTP request
HTTP response
2
4
data is returned
in response to the
request
5
browser replaces view with
data sent from server
24.05.2010 43Satish Srirama
Characteristics
• User-driven: Things only happen when the user
does something (e.g., clicks on a link or button)
• Views defined by URLs: You can bookmark
something and come back to it; use the something and come back to it; use the
forward/backward button
• Simple user interaction model: Not that many
things you can do in browser
• Synchronous Interaction: System responses are
synchronized with user-driven events
24.05.2010 44Satish Srirama
browser
Synchronous Interactions
server-side
Time
user activity user activity user activity
1
2
4
5
server-side
server processing server processing
browser server-side systemsHTTP request
HTTP response
12
45 3
3
24.05.2010 45Satish Srirama
L
A
Linux
Apache
So what do you run on the server side?
A
M
P
MySQL
PHP/Python/Perl
24.05.2010 46Satish Srirama
From “Old-School” to Ajax
browser
Ajax intermediates between the interface and the server.
server-side systems
Inte
rfa
ce request
response
backend
database
Web
server
data
management
Ajax
“engine”
interaction management
24.05.2010 47Satish Srirama
Inside the Browser
browser
Re
nd
eri
ng
En
gin
e
HTTP request
HTML / CSS
data
other data
(e.g. images)
Inte
rfa
ce
Re
nd
eri
ng
En
gin
e
HTTP response
24.05.2010 48Satish Srirama
Enter JavaScript
browser
Re
nd
eri
ng
En
gin
e
HTTP requestJavaScript Engine
HTML / CSS
data
other data
(e.g. images)
Inte
rfa
ce
Re
nd
eri
ng
En
gin
e
HTTP response
JavaScript
code
24.05.2010 49Satish Srirama
Enter Ajax
browser
Re
nd
eri
ng
En
gin
e
HTTP requestJavaScript Engine
XMLHttpRequest
HTTP request
HTML / CSS
data
other data
(e.g. images)
Inte
rfa
ce
Re
nd
eri
ng
En
gin
e
HTTP response
XML
data
JavaScript
code
24.05.2010 50Satish Srirama
AJAX Interactions
browser
user activity
client-side processing
server-side
Time
server processing server processing
24.05.2010 51Satish Srirama
Components of an Ajax Interaction
1. A client event occurs (captured by JavaScript event handlers)
2. An XMLHttpRequest object is created and configured
3. An asynchronous request is made to the server 3. An asynchronous request is made to the server via the XMLHttpRequest object
4. Server processes request and returns data, executing a callback in the XMLHttpRequest object
5. The HTML DOM is updated based on response data
24.05.2010 52Satish Srirama
DOM
• Document Object Model: platform- and
language-independent way to represent XML
– Adopts a tree-based representation
– W3C standard, supported by modern browsers– W3C standard, supported by modern browsers
• JavaScript uses DOM to manipulate content
– To process user events
– To process server responses (via XMLHttpRequest)
24.05.2010 53Satish Srirama
Ajax: Things to watch out for!
• Hype– Best thing since sliced bread?
• Application development/maintenance cost– Brower incompatibilities
– Many different approaches and tools– Many different approaches and tools
– For many things, lack of agreed-on best practices
• Behavior is not ‘Web-like’ – Standard things often don’t work correctly (e.g., browser
‘back’ button, bookmarks)
– Usability issues for users with disabilities
• Security issues– Whole new class of cross-site scripting (XSS) exploits
24.05.2010 54Satish Srirama