Cloud Computing Lec7 V1kodu.ut.ee/~srirama/cloud/L7_Leftovers.pdf · Outline • Summarize what we...

Basics of Cloud Computing – Lecture 7

Cloud Computing – Summary and

leftoversleftovers

Satish Srirama

Outline

• Summarize what we have learnt

• Amazon SQS

• Some more PaaS

– Google App Engine– Google App Engine

– Force.com

• AJAX

• Virtualization

24.05.2010 Satish Srirama 2

What is Cloud Computing?

• Computing as a utility

– Consumers pay based on their usage

• Cloud Computing characteristics

– Illusion of infinite resources

– No up-front cost

– Fine-grained billing (e.g. hourly)

• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”

Satish Srirama24.05.2010 3/30

Cloud Computing – Services - Recap

• Software as a Service – SaaS– A way to access applications

hosted on the web through your web browser

• Platform as a Service – PaaS– A pay-as-you-go model for IT

resources accessed over the

Facebook, Flikr, Myspace.com,

Google maps API, Gmail

Level of

Abstraction

resources accessed over the Internet

• Infrastructure as a Service –IaaS– Use of commodity computers,

distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data

– Virtualization

Satish Srirama

Google App Engine,

Force.com, Hadoop, Azure, Amazon S3, etc

IaaSAmazon EC2, Joyent

Accelerators, Nirvanix Storage

Delivery Network, etc.

24.05.2010 4

Scientific Computing Cloud (SciCloud)

• Distributed Systems Group owned private cloud infrastructure

• Eucalyptus setup

• Goal of the project• Goal of the project

– To Establishing a private cloud at universities

– To efficiently use the already existing resources of universities

– To address computationally intensive scientific, mathematical, and academic problems

• Hope it was a pleasant experience !!!

24.05.2010 5Satish Srirama

Economics of Cloud Providers –

Failures - Recap

• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter

• Caveat: User software has to adapt to failures• Caveat: User software has to adapt to failures

• Solution: Replicate data and computation– MapReduce & Distributed File System

• MapReduce = functional programming meets distributed processing on steroids – Not a new idea… dates back to the 50’s (or even 30’s)

24.05.2010 6/30 Satish Srirama

MapReduce

• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together

• The execution framework handles everything else…• Not quite…usually, programmers also specify:• Not quite…usually, programmers also specify:

partition (k’, number of partitions) → partition for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic

MapReduce Examples

• Distributed Grep

• Count of URL Access Frequency

• Reverse Web-Link Graph

• Inverted Index • Inverted Index

• Distributed Sort

• MapReduce in Information Retrieval

Hadoop Processing Model

• Create or allocate a cluster

• Put data onto the file system– Data is split into blocks

– Replicated and stored in the cluster

• Run your job• Run your job– Copy Map code to the allocated nodes

• Move computation to data, not data to computation

– Gather output of Map, sort and partition on key

– Run Reduce tasks

• Results are stored in the HDFS

LEFTOVERS

Amazon Simple Queue Service (SQS)

• A reliable, highly scalable hosted distributed queue

for storing messages as they travel between

computers.

• Amazon's web-scale messaging infrastructure as a

web service

• Platform-agnostic, allowing any computer on the

Internet to add or read messages without any

installed software or special firewall configurations

Amazon SQS Highlights

• Reliable

– Runs within Amazon's high-availability data centers.

– Messages are stored redundantly across multiple servers and data centers.

• Simple: Only Six Methods• Simple: Only Six Methods

• Scalable

• Secure: Strong access controls

• $0.10 per 1,000 messages sent ($0.0001 per message sent)

• $0.20 per GB of data transferred

Google AppEngine

What is AppEngine?

• Google platform to build web application on

the Cloud

• Example of PaaS

• Does one thing well: running web apps• Does one thing well: running web apps

• Request → Processing → Response

1424.05.2010 Satish Srirama

No More LAMP

Why App Engine

Application Life Cycle

Development Life cycle

Physical Deployment Diagram

Developing applications with App Engine

Developing Applications

App Engine SDK

• Web Server

dev_appserver.py

• Application uploader• Application uploader

appcfg.py

• Local DataStore

implementation

• App Engine APIs

App configuration – app.yaml

application: helloworld

version: 1

runtime: python

api_version: 1 api_version: 1

handlers:

- url: /.*

script: helloworld.py

Helloworld.py

Running Application

Data Store

• A powerful non relational distributed data

storage service(object database)

• Transactional data store

• Query engine• Query engine

– GQL – SQL like language

• Use BigTable in back

Deployment and Testing

• Local testing

dev_appserver.py helloworld/

• Deploy

appcfg.py update helloworld/appcfg.py update helloworld/

• On the cloud

http://distributedseminartartu.appspot.com

End User Administration

Access Control Authentication / SSO Services

Web Services API

Workflow Engine

Powerful Application Service

Multi-Device Multi-LanguageMulti-

CurrencyReporting &

AnalyticsSearchEngine

Force.com - cloud platform for business apps

Infrastructure

Services

Network

Storage

Operating System

Database

Application

Services

Authentication

Security

Sharing

Integration

Operations

Services

Availability

Monitoring

Patch Mgmt

Upgrades

Force.com Lets You Focus on

Innovation

InnovationApp Server

Web Server

Data Center

Disaster Recovery

Customization

Web Services

Multi-Language

Multi-Currency

Workflow

Analytics

Multi-Device

Messaging

Search

Backup

Sales force2924.05.2010 Satish Srirama

Create any table

Add any field

Build any relationship

Database as a Service

Build any relationship

Security & sharing

Automatic tuning

Automatic backups

Automatic upgrades

Integration as a Service

History Tracking

Workflow Rules

Logic As A Service

Declarative Logic

Formula-Based LogicFormula Fields

Procedural Logic

Formula Fields

Validation Rules

Apex Triggers

Apex Classes

Apex Web Services

User Interface as a Service

UI generated

automatically,

no technical

skills required

UI generated

by developer /

technologist

Profile /

Record Type /

Page Layout

mapping

controls UI

behavior

Page Layout Visualforce

Limited

control over

look and feel,

but all UIs are

consistent

control of UI

behavior

Point and click to

add Custom

Buttons and

Embedded

Mashups

Next Week

• All the best with the exams

• On 27th May 2010, we will have the project

presentations

References

• Google App Engine

http://code.google.com/appengine/

• Force.com http://www.salesforce.com/force/

• Amazon SQS http://aws.amazon.com/sqs/• Amazon SQS http://aws.amazon.com/sqs/

Virtualization

• Virtualization technologies partition hardware

and thus provide flexible and scalable

computing platformsApp App App

• Types of Virtualization

– Hardware Virtualization

– Virtual Machine Virtualization

– Application Virtualization

Hardware

Hypervisor

Virtualized Stack

Hardware Partitioning

• Physical system is

divided into multiple

partitions

• One or more CPUs per • One or more CPUs per

partition

• Full electrical isolation

between partitions

• Redundant hardware

components

Virtual Machine Virtualization

Virtual MachineVirtual Machine

Guest OSGuest OS

(Windows 2003)(Windows 2003)

ApplicationsApplications

Virtual MachineVirtual Machine Virtual MachineVirtual Machine

Guest OSGuest OS

(Windows NT)(Windows NT)

Guest OSGuest OS

(Windows 2000)(Windows 2000)

ApplicationsApplications ApplicationsApplications

System HardwareSystem Hardware

Host Operating SystemHost Operating System

Virtual Machine ManagerVirtual Machine Manager

Virtualization platforms

• XEN– XEN 3.0 -- Open source

• Limited guest support

• Requires OS Source code changes

– XEN Enterprise – commercial– XEN Enterprise – commercial• Windows and Linux support

• Virtuozzo– Linux roots

– Virtualizes the OS rather than the hardware

• VMWare

Application Virtualization

• Runs as an application

• Virtualizes OS settings

– Registry– Registry

– File System

• Asynchronous JavaScript and XML: an approach for building interactive Web applications

• Ajax refers to a number of technologies:• Ajax refers to a number of technologies:

– XHTML/CSS for presentation

– XML for data exchange (or JSON)

– XMLHttpRequest object for asynchronous communication

– JavaScript to tie everything together

From “old-school” Web applications to Ajax…

“Old-School” Web Applications

server-side systemsbrowser

user does something

browser sends

request to server

server generates Web

page as a response to

the request

backend

database

server

HTTP request

HTTP response

data is returned

in response to the

request

browser replaces view with

data sent from server

Characteristics

• User-driven: Things only happen when the user

does something (e.g., clicks on a link or button)

• Views defined by URLs: You can bookmark

something and come back to it; use the something and come back to it; use the

forward/backward button

• Simple user interaction model: Not that many

things you can do in browser

• Synchronous Interaction: System responses are

synchronized with user-driven events

browser

Synchronous Interactions

server-side

user activity user activity user activity

server-side

server processing server processing

browser server-side systemsHTTP request

HTTP response

Apache

So what do you run on the server side?

PHP/Python/Perl

From “Old-School” to Ajax

browser

Ajax intermediates between the interface and the server.

server-side systems

ce request

response

backend

database

server

management

“engine”

interaction management

Inside the Browser

browser

HTTP request

HTML / CSS

other data

(e.g. images)

HTTP response

Enter JavaScript

browser

HTTP requestJavaScript Engine

HTML / CSS

other data

(e.g. images)

HTTP response

JavaScript

Enter Ajax

browser

HTTP requestJavaScript Engine

XMLHttpRequest

HTTP request

HTML / CSS

other data

(e.g. images)

HTTP response

JavaScript

AJAX Interactions

browser

user activity

client-side processing

server-side

server processing server processing

Components of an Ajax Interaction

1. A client event occurs (captured by JavaScript event handlers)

2. An XMLHttpRequest object is created and configured

3. An asynchronous request is made to the server 3. An asynchronous request is made to the server via the XMLHttpRequest object

4. Server processes request and returns data, executing a callback in the XMLHttpRequest object

5. The HTML DOM is updated based on response data

• Document Object Model: platform- and

language-independent way to represent XML

– Adopts a tree-based representation

– W3C standard, supported by modern browsers– W3C standard, supported by modern browsers

• JavaScript uses DOM to manipulate content

– To process user events

– To process server responses (via XMLHttpRequest)

Ajax: Things to watch out for!

• Hype– Best thing since sliced bread?

• Application development/maintenance cost– Brower incompatibilities

– Many different approaches and tools– Many different approaches and tools

– For many things, lack of agreed-on best practices

• Behavior is not ‘Web-like’ – Standard things often don’t work correctly (e.g., browser

‘back’ button, bookmarks)

– Usability issues for users with disabilities

• Security issues– Whole new class of cross-site scripting (XSS) exploits

Cloud Computing Lec7 V1kodu.ut.ee/~srirama/cloud/L7_Leftovers.pdf · Outline • Summarize what we...

Documents

Mobile & Cloud Computing: Research Challengeskodu.ut.ee/~srirama/talks/MobileandCloudComputing-ResearchChall… · REMICS • Reuse and migration of legacy applications to the cloud

lec7 - classes.engineering.wustl.edu

Cloud Computing Lec7 - Kursused · PDF fileBasics of Cloud Computing –Lecture 7 More AWS and Cloud-based ... Oracle or Microsoft SQL Server • NoSQLdatabases ... • Cisco believes

Lec7 cellular network

Es Lec7 Amba

Mobile and Cloud Computing Laboratorylepo.it.da.ut.ee/~srirama/talks/MobileandCloudComputingLaboratory... · Mobile and Cloud Computing ... Services (IaaS, SaaS, PaaS) and Providers

Mobile and Cloud Computing Seminar - … and Cloud Computing Seminar MTAT.03.280 Spring 2017 Satish Srirama ... – Programmatically via Java API – Declaratively, via serialized

MAE 241 - Lec7

Mobile Web and Cloud Services Enabling Internet of Thingslepo.it.da.ut.ee/~srirama/talks/Keynote_ICAC_2016_V1_Srirama.pdf · • Mobile Cloud based system should take advantage of

Cloud Computing - Summary · 2018-04-03 · Cloud Computing Summary Satish Srirama. Outline ... Challenges and technical problems • Inaccurate code profiling – Code has non-deterministic

Mobile and Cloud Centric Internet of Thingskodu.ut.ee/~srirama/talks/Keynote_RWCC_2016.pdf · • Mobile Cloud based system should take advantage of some of the key intrinsic characteristics

Mobile Cloud Computinglepo.it.da.ut.ee/~srirama/talks/MobileCloud_2016.pdf · Mobile Cloud –Interpretation • We should not see Mobile Cloud to be just a scenario where mobile

Cs419 lec7 cfg

Ch6 lec7&8

Lec7 Clean

Satish Srirama - kodu.ut.ee

Cloud Computing Lec7 - Kursused€“ Building a fault-tolerant BSP framework (NEWT) [Kromonov et al, HPCS 2014] 3/22/2016 Satish Srirama 22/41 ... cloud-aware – Increased data storage

A Srirama Desikan reader

Cloud Computing Lec7

Lec7(Linked Lists)