51
AIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI Class : III B.Sc Computer Science –VI Semester Subject Title : Cloud Computing Subject Code : 16SMBECS2:2 Staff Name : N. Shameem Banu Designation : Asst. Professor CHAPTER 1 INTRODUCTION TO CLOUD COMPUTING CLOUD COMPUTING BASICS In the real world, computing developed progressively and can be viewed in two stages. One is more centralized and the other is the desktop. The limitations of centralized computing did not give users enough control and was inflexible. Distributed computing made every user a system administrator but it was still inefficient. In the last few years, as the cost of computing power continued to decrease, the cost of manpower to implement and manage computer systems have increased. Centralized computing facility needs more version upgradation to reach the cloud computing stage. Some computer scientists have suggested a vast grid of computers attached via the Internet, whose power is combined for large-scale tasks, when needed. In certain cases, large numbers of computing systems are used for particular tasks. Other scientists have recommended a computing utility which would offer just as much computing power as a society needs in an on-demand basis, as in the case of electricity. Therefore, Google and Amazon web users have built enormous data centres for their own use. They have realized that they can allow others to access these data centres at reasonable prices. Thus began the era of cloud computing. HISTORY OF CLOUD COMPUTING

oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

  • Upload
    others

  • View
    44

  • Download
    4

Embed Size (px)

Citation preview

Page 1: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

AIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI

Class : III B.Sc Computer Science –VI SemesterSubject Title : Cloud ComputingSubject Code : 16SMBECS2:2Staff Name : N. Shameem BanuDesignation : Asst. Professor

CHAPTER 1INTRODUCTION TO CLOUD COMPUTINGCLOUD COMPUTING BASICSIn the real world, computing developed progressively and can be viewed in two stages. One is more centralized and the other is the desktop. The limitations of centralized computing did not give users enough control and was inflexible. Distributed computing made every user a system administrator but it was still inefficient. In the last few years, as the cost of computing power continued to decrease, the cost of manpower to implement and manage computer systems have increased. Centralized computing facility needs more version upgradation to reach the cloud computing stage. Some computer scientists have suggested a vast grid of computers attached via the Internet, whose power is combined for large-scale tasks, when needed. In certain cases, large numbers of computing systems are used for particular tasks. Other scientists have recommended a computing utility which would offer just as much computing power as a society needs in an on-demand basis, as in the case of electricity.

Therefore, Google and Amazon web users have built enormous data centres for their own use. They have realized that they can allow others to access these data centres at reasonable prices. Thus began the era of cloud computing.

HISTORY OF CLOUD COMPUTINGIn 1960, the famous scientist John McCarthy predicts that the computation being available as pubic service. In 1960, the cloud computing evolved along with numbers of lines, Web 2.0 being the most recent development. Salesforce.com was the first cloud computing to have arrived in the 1999, which pioneered the idea of delivering enterprise applications through a simple website.

It was in the late 1990s that companies such as Sun Microsystems decided to introduce the concept called ‘the network is the computer’. The idea that Oracle founder Larry Ellison stated was as follows ‘all these ideas were indeed profound but they never failed with consumers, who were looking for more complete personal computer solutions specializing with, some storage capacity availabilities’. As of now, the personal computer is not a dummy terminal. In reality, the rise of the Internet, in the mid 1990s, changed the usage of computers

Page 2: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

and information distribution. With the idea of utility computing, Amazon began to establish and control server farms, to offer apps to their buyers.

IT professionals recognized that there are eight basic components that are very important in enabling the cloud computing concept (Figure 1.1) for the cloud to work in the public or private sector, they are as follows:

1. World wide connectivity: users should have near-ubiquitous access to the Internet.2. Open access: Users should have fair, open-minded access to the Internet.3. Reliability: The cloud’s performance should equal to or better than recent

standalone systems.4. Interoperability and user choice: Users must be able to progress among different

clouds.5. Security: It should ensure that data of users are safe. 

Figure 1.1 Basic Components of Cloud Computing6. Privacy: Users’ rights must be clearly defined and allow access based on rights.7. Economic value: The cloud must provide substantial savings and benefits.8. Sustainability: The cloud must increase power effectiveness and reduce

environmental impact.

CHARACTERISTICS OF CLOUD COMPUTING

1.Dynamic Computing Infrastructure

2.IT Service-centric Approach

3.Self-service Based Usage Model

4.Minimally or Self-managed Platform

5.Consumption-based Billing

Page 3: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

CHAPTER 2MOVE TO CLOUD COMPUTING

PROS AND CONS OF CLOUD COMPUTING

Advantages of Cloud Computing in IT Field

Advantages of Cloud Computing1.Cost reduction:2.Scalability3.Easier collaboration4.Affordable5.Scalable and flexible6.Efficiency

Page 4: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Disadvantages of Cloud Computing1.Security concerns2.Risk of losing internet connection3.Limited resources for customizations4.Availability5.Data mobility and ownership:6.Privacy

Service-oriented Architecture (SOA)

Cloud Computing Migration Issues1.Security2.Vendor Management3.Technical Integration4.Process and Culture5.The Business View

The Process: Key Stages in Migrating to the Cloud

Cloud migration process can be divided into three areas:

1. Plan1. Determine key business drivers2. Define business objectives3. Get executive sponsorship4. Set project guiding principles5. Form project team made up of IT and business representatives

Develop a project plan by including the following:6. Define business requirements7. Set key success metrics8. Set timeline9. Identify decision-making authorities

2. Execute1. Execute the plan2. Stay away from ‘scope creep’—stay focused on original project scope;

this becomes a challenge particularly in cases, where a major legacy application with large users set is being replaced

3. Remember to follow the guiding principles at all times4. Communicate to all stakeholders regularly (no surprises!)5. Train users

3. Monitor

Page 5: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

1. Monitor adoption2. Track success metrics3. Stay away from scope creep (this one may well decide the success or

failure of the project)4. Follow guiding principles5. Only implement changes based on quantifiable business needs

SEVEN-STEP MODEL1.Know that there are many different variances of cloud services:2.Move towards the cloud as a tool or an additional option to supply IT functionality3.Recognize which constituent of your environment may be ‘cloud compatible’:4.To better compute the advantage of cloud services lies on understanding about current costs5.Preparation of organization to ‘manage’ rather than ‘operate6.To simplify and de-risk your migration7.Question to gain more knowledge:

CHAPTER 3TYPES OF CLOUD

Types of Cloud Computing

1. Public cloud: This computing infrastructure is hosted at the vendor’s workplace. The end user cannot view the infrastructure. The computing infrastructure is shared between companies.

2. Private cloud: Here the computing infrastructure is dedicated to the customer and is not shared with any other companies. They are costly and highly secure than public clouds. Private clouds may be hosted externally as well as in their own premise hosted clouds.

3. Hybrid cloud: Organizations can submit less valued applications in public cloud and high valued applications in the private cloud. The combination is known as hybrid cloud. Cloud bursting is used to define a system where the organization uses its own infrastructure for normal usage and cloud is used for peak times.

4. Community cloud: The cloud infrastructure is shared between the companies of the same community. For example, all the government organizations in a city can share the same cloud but not the non-governmental organizations.

Public Cloud

A public cloud is based on the standard cloud computing model, where the service provider makes the resources such as storage and applications available to the public over the WWW. These services may be free or on a pay-per-usage model. The advantages of using a public cloud computing service are as follows:

Inexpensive and easy to setup

Page 6: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Scalability No resources are wasted

Private Cloud (Internal Cloud or Corporate Cloud)

Private cloud is a proprietary computing model that provides services to users who are behind a firewall.

Public versus Private CloudThe first difference will be the deployment location. Cloud can be offered as a service over the WWW is a public cloud, whereas a private cloud sits inside the firewall.Public cloud is charged on a monthly basisPrivate cloud is deployed in the customer location, providing all necessary hardwares and operating systems.Elementary expenseVolume of dataPerformance expectations

Duration of data storage Access patterns and locations Security and data isolation Confidentiality and destruction of data SLAs (Service Level Agreements) In-house technical crew

Cloud Computing Infrastructure

CLOUD APPLICATION ARCHITECTURE

Scale up and down rapidly Increased fault tolerance Bring up development or test instances Speedier versions to the customer base Load and test an application

Page 7: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

CHAPTER 4WORKING OF CLOUD COMPUTING

Technology Trends1.Virtualization2.Data Growth3.Energy and Green IT4.Complex Resource Tracking5.Consumerization and Social Software

CLOUD SERVICE MODELS

SaaS

Provider of SaaS has full administrative rights for its application and responsible for activities such as deployment, maintenance and update.

PaaS

PaaS is service, where application/software can be build, tested and deployed as a single unit. PaaS is useful for application builders, developers, deployers and testers.

IaaS

When the customer requires an end-to-end infrastructure such as computer resources, storages and network, he/she can opt for IaaS

Cloud Service Models

CLOUD DEPLOYMENT MODELS

Page 8: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

PROS AND CONS OF CLOUD COMPUTING1.Risks in the Cloud2.Cloud Storage as ServiceCLOUD COMPUTING AND SERVICES: PROS AND CONS1.Centralized Data Storage in Cloud Computing 2.Cloud Servers Maintenance and Security3.Data Access and Network Connectivity4.Cost Factor5.Cloud Servers Data Backup and Availability

CHAPTER 5CLOUD COMPUTING TECHNOLOGY

CLOUD LIFECYCLE MODEL

Cloud lifecycle management provides:

Ease in administrating cloud and service portal Manageable service Established multi-tenancy Include performance and capacity management Support heterogeneity

Phases of CDLC

Page 9: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

REFERENCE MODEL FOR CLOUD COMPUTING

CHAPTER 6CLOUD ARCHITECTURE

Definition

‘cloud computing is a type of computing environment, where IT businesses outsource their computing needs, which include software application services to outside vendors when they are in need of computing power or other resources like storage, database, e-mails, etc., which are accessed via WWW.’

Public cloud: This is off-premises or external. Public cloud is a widely used model where infrastructure comprising of hardware systems, network, storage and applications areprovided and owned by the provider. When the organization needs some kind of security for its applications and data, they resort to private cloud, which is behind the firewall and accessed only by customers belonging to that organization.

Private cloud: It is an on-premises or internal cloud set-up. Hybrid cloud: It is a combination of both public and private clouds.

Cloud Computing Reference Model

1. Cloud enablement model: This model describes the various layers of cloud and its advantages for business operations. This model comprises of various cloud computing technologies and solutions for the consumers.

2. Cloud deployment model: The cloud deployment model (CDM) describes the various cloud models such as private, public, hybrid and community clouds.

3. Cloud governance and operations model: The cloud governance and operations model defines the requirements for cloud computing such as governance, privacy, security operations, management, support and monitoring.

4. Cloud ecosystem model: The cloud ecosystem takes cares of development and sustenance. It consists of cloud providers, consumers, intermediaries and networks.

CLOUD DEPLOYMENT MODEL

Private Cloud

Page 10: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Hybrid Cloud

Community Cloud

CHAPTER 7

CLOUD MODELLING AND DESIGN

Key Principles of Cloud Computing1.Abstraction2.Automation3.ElasticityCloud Federation

Cloud federation is interconnecting the cloud computing environments with two or more service providers for balancing the traffic load and to surge spikes while there is demand.

Page 11: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Cloud Ecosystem

Cloud ecosystem is a term, which defines the complexity of the systems in terms of its interdependent components that work together to enable cloud services.

Cloud Unified Process (CUP)

The reason why usage of the cloud computing is still low in the SMBs, is due to the following reasons:

Cloud availability Cloud security Erosion of data integrity Data replication and consistency issues Potential loss of privacy Lack of auditing and log ging visibility Potential for regulatory violations Application sprawl and dependencies Inappropriate usage of services

CHAPTER 8

FOUNDATIONSDEFINITION OF VIRTUALIZATION

‘Virtualization is a methodology for dividing the computer resources to more than one execution environment by applying more concepts like partitioning, time-sharing, machine simulation and emulation.’

Page 12: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Reasons for using virtualization:

Virtual machines (VM) consolidate the workloads of under-utilized servers. Because of this one can save on hardware, environmental costs and management.

To run legacy applications, VM is used. VM provides a secured and sandbox for running an untrusted application. VM helps in building secured computing platform. VM provides an illusion of hardware. VM simulates networks of independent computers. VM supports to run distinct OS with different versions. VMs are uses for performance monitoring. Operating systems can be checked

without disturbing the productivity. VM provides fault and error containment. VM tools are good for research and academic experiments. VM can encapsulate the entire state of a system by saving, examining, modifying

and reloading. VM enables to share memory in multiprocessor architecture. VM makes the job easier for the administrative staff in migration, backup and

recovery.

TYPES OF VIRTUALIZATION

Server virtualization is a kind of virtualization, used for masking of server resources, which includes number of physical servers, processors and operating systems. The intention of using this method is to spare the complicated server resources and hence increasing the sharing, utilization and maintaining the capacity of servers.

Network Virtualization is a method where network resources are combine based on available bandwidth. Each channel is assigned to a particular server. By adopting this method of virtualization, a true complexity of network is hidden and managed

Page 13: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

like partitioning the hard drive. Because of network virtualization, lower TCO, higher return of investment, security and dynamic computing are obtained.

Storage virtualization is a type of virtualization, where a pool of physical storage from different network of storage devices appears as a single storage device. Usually this kind of virtualization is adopted in SAN (storage area networks). Storage virtualization is advantageous in disaster recovery, business continuity, lower TCO, higher return of investment, dynamic computing, security, testing and development. 

Examples1.Wine2.FreeBSD3.Hive4.Microsoft Virtual Server5.Nemesis6.SimOS

Server Virtualization

1. a physical server can be partitioned into multiple virtual servers. Each virtual server has its own operating system and applications and acts as an individual server.

2. Server virtualization masks the server resource which includes the servers, processors and operating systems from the server users

3. The virtual environments are called as guests, instances, containers or emulations.

4. The need for server virtualization is consolidation, redundancy, legacy systems and migration

5. There are three ways to create virtual servers—full virtualization, paravirtualization and OS-level virtualization

6. Complete virtualization utilizes software called hypervisor.

OS Virtualization

1. the host runs a single OS kernel and exports different operating system functionalities to every visitors. In this model, common binaries and libraries are shared on the same physical machine, which permits an OS level virtual server to serve huge number of visitors.

Page 14: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

The two types of virtual disks in most implementations are private virtual disk and shared/common virtual disk.

1. Private virtual disk: A private virtual disk acts as a key for a client, he can use it to store information, based on the rights assigned to him. Therefore, when the client’s disk is restarted, the settings are retained, just like working with a physical local hard disk.

2. Shared/common virtual disk: Multiple clients use a shared virtual disk simultaneously. During access, changes are stored in to a special cache. But the cache content will be cleared when the client is shut down or restarted.

OS Virtualization: Pros

Flexible provisioning: It is very simple and easy to connect different virtual disks to a system through OS virtualization. Starting another operating system or any other application can be done easily by the client.

Rapid software deployment: Adding a new server or workstation happens within few seconds. Using deployment tools to install and configure the system or doing it manually takes at least a few hours, with a few steps the client is allocated to a virtual disk and can be used in production.

Easy and efficient implanting updates and hot fixes of the operating system and applications: In OS virtualization, it is enough to add an update to the virtual disk image and not to all servers.

Easy rollback scenarios: Rollback to previous state is easy in OS virtualization.

OS Virtualization: Cons

No work off-line capability: OS virtualization products must be connected to the virtualization server for using the operating system virtually.

High-speed LAN recommended: A high-speed LAN is needed as the virtual disk is connected to the OS virtualization server through the network.

Page 15: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Limited numbers of operating systems are supported: Limited number of OS supports virtualization. Some Linux distributions do not support the OS virtualization technique.

Imaging disadvantages apply to this technique: Virtual disk is created using image-based techniques. All disadvantages of imaging techniques are also applicable for the OS virtualization component.

Storage VirtualizationStorage virtualization (SV) is a new concept under virtualization. Storage systems use virtualization concepts for better functionality and have more features within the storage system

There are two primary types of storage virtualizations:

1. Block virtualization separates the logical and physical storage. This gives greater flexibility for the administrators in managing storage for consumers.

2. To eradicate the dependencies between the facts and numbers accessed at the document level and the position where the documents are retained, file virtualizationmethod is utilized. This method optimizes usage of storage and server consolidation.

Mainly storage virtualization is structured in three ways: host-based, storage device-based and network-based.

1. Host-based: Traditional device handles physical drives.2. Storage device-based: Pooling and managing metadata.3. Network-based: Device which uses fibre channel.

Network VirtualizationThe procedure of blending the accessible assets in a mesh by dividing up the accessible bandwidth into channels is called network virtualization

Following is a common network virtualization scenarios and examples:

External network virtualization: In this situation, to achieve the objective of improving the efficiency of a large corporate network or data centre by combining one or more local networks or subdivided into virtual networks.

Internal network virtualization: A single system is configured with containers using Xen/KVM domain and combined with hypervisor control programs, for example, VNIC (Virtual Network Interface Card) to create a ‘network in a box’. Examples of internal network virtualization are NetworkStack project, OpenSolaris network and Microsoft virtual server.

VIRTUAL CLUSTERING‘a type of parallel or distributed system that consists of a collection of interconnected computers and is used as a single, unified computing resource’

Benefits of Clustering

Page 16: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Scientific applications: Applications running on supercomputers can be migrated to Linux cluster (which is more cost effective).

Large ISPs and E-commerce enterprise with large database: Internet service providers or e-commerce web site that requires high availability and load balancing and scalability.

Graphics rendering and animation: Film industry is benefited because of clustering, in particular for rendering quality graphics and animation scenes. Examples include Titanic, True Lies and Interview with the Vampire.

Fail-over clusters: Using clusters, network services are increased in terms of availability and serviceability and when a server fails, its services are migrated to another system. For example, failover servers are database servers, mail servers and file servers.

High availability load balancing clusters: Application can run on all computers and can host multiple applications. Individual computers are hidden to the outside world as they interact only with the clusters. This kind of clustering produces best results with stateless application and when executed concurrently.

CHAPTER 9GRID, CLOUD AND VIRTUALIZATION

Grid Computing

three points to describe a grid more in detail:

1. Nontrivial qualities of service delivery2. Habit of using standards in all places3. Resources coordinationVirtualization in Grid ComputingVirtualization integration and grid computing can be encouraged at various levels

1.Grid Computing Level

2.LRMS Level

3.Virtual MachinesSimilarities Between Cloud Computing and Virtualization

Cloud Computing Virtualization

Location of virtual machine On any host On a specifi c host

VM/instance storage Shortly lived Persistent

VM resource (CPU, RAM, etc.)

Standard Customizable

Resource changes Spin p new instance Resize VM itself

Page 17: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Cloud Computing Virtualization

Recovery from failures Discard instance spin up new one

Attempt to recover failed VM

CHAPTER 10

VIRTUALIZATION AND CLOUD COMPUTING

Cloud Stack

managed cloud stack consisting of two parts: first part consists of cache, dynamic code and attached storage and logic for the dynamic code; second part consists of stable and efficient OS, security features and business logic written using some programming language

CHAPTER 11

DATA STORAGE

ENTERPRISE DATA STORAGE

The various types of storage subsystems are:

Direct Attached Storage (DAS) Storage Area Network (SAN) Network Attached Storage (NAS)

Data Storage Management Tools

Storage Resource Management (SRM) tools include configuration tools, provisioning tools and measurement tools.

Configuration tools handle the set-up of storage resources. These tools help to organize and manage RAID devices by assigning groups, defining levels or assigning spare drives.

Page 18: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Provisioning tools define and control access to storage resources for preventing a network user from being able to use any other user’s storage.

Measurement tools analyse performance based on behavioural information about a storage device. An administrator can use that information for future capacity and upgrade planning.

Data Storage Challenges

1.Massive Data Demand

2.Performance Barrier

3.Power Consumption and Cost

Unified Storage

A new innovative solution ‘Unified Storage’ is developed and addresses the issues discussed earlier. Basically this type of storage solution is a combination of NAS and SAN and termed as NUS (network unified storage).

Cloud File System

In cloud file systems, the considerations are:

It must sustain basic file system functionality. It should be an open source. It should be grown-up enough that users will at least think about trusting their data

to it. It should be shared, i.e., available over a network. It should be paralleling scalable. It should provide honest data protection, still on commodity hardware with only

internal storage.

Ghost File System

Ghost cloud file system is used in Amazon Web Services (AWS). It gives high redundant elastic mountable, cost-effective and standards-based file system. A fully featured scalable and stable cloud file systems is provided by ghost cloud file system.

Benefits of Ghost CFS

Elastic and cost efficient: Pay for what you use from 1 GB to hundreds of terabytes. Multi-region redundancy: Aiming to take advantage of AWS’s 99.99% availability Highly secure: Uses your own AWS account (ghost cannot access your data). No administration: Scales elastically with built in redundancy—no provisioning or

backup.

Page 19: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Anywhere: Mount on a server or client or access files via a web page or from a mobile phone

Gluster File System

GlusterFS is an open source, distributed file system capable of handling multiple clients and large data. GlusterFS clusters storage devices over network, aggregating disk and memory resources and managing data as a single unit.

Hadoop File System

A distributed file system designed to run on commodity hardware is known as Hadoop Distributed File System (HDFS).

XtreemFS: A Distributed and Replicated File System

XtreemFS is a distributed, replicated and open source. XtreemFS allows users to mount and access files via WWW. Engaging XtreemFS a user can replicate the files across data centres to reduce network congestion, latency and increase data availability

Kosmos File System

Kosmos Distributed File System (KFS) gives high performance with availability and reliability. For example, search engines, data mining, grid computing, etc.

CloudFS

CloudFS is a distributed file system to solve problems when file system is itself provided as a service. CloudFS is based on GlusterFS, a basic distributed file system, and supported by Red Hat and hosted by Fedora.

CLOUD DATA STORES

A data store is a data repository where data are stored as objects. Data store includes data repositories, flat files that can store data. Data stores can be of different types:

Relational databases (Examples: MySQL, PostgreSQL, Microsoft SQL Server, Oracle Database)

Object-oriented databases Operational data stores Schema-less data stores, e.g. Apache Cassandra or Dynamo Paper files Data files (spread sheets, flat files, etc)

Types of Data Stores

BigTable

Dynamo: A Distributed Storage SystemCLOUD STORAGE

Page 20: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Cloud storage is data storage hosted remotely using data storage devices in WWW and maintained by the third party (service provider). Cloud storage is a part of cloud computing. It is deployed using WAN infrastructure which includes hardware components such as switches and routers.

Cloud storage can be deployed in many ways. For example:

Local data (desktop/laptop) can be backed up to cloud storage. A virtual disk can be ‘sync’ to the cloud and distributed. The cloud can be used as a reservoir for storing data.

Following are some additional cloud storage attributes:

Resource pooling and multi-tenancy: Multiple consumers can use shared single storage device. Storage resources are pooled and consumers can be assigned and unassigned resources according to their needs.

Scalable and elastic: Virtualized storage can be easily expanded on need basis. Accessible standard protocols including HTTP, FTP, XML, SOAP and REST. Service-based: Consumers no need to invest, that is, no CAPEX (Capital

Expenditure) and only pay for usage, that is, OPEX (Operational Expenditure). Pricing based on usage Shared and collaborative

On-demand self-service

Cloud Data Management Interface (CDMI)

To create, retrieve, update and delete objects in a cloud the cloud data management interface (CDMI) is used. The functions in CDMI are:

Cloud storage offerings are discovered by clients Management of containers and the data Sync metadata with containers an objects

CDMI is also used to manage containers, domains, security access and billing information. CDMI standard is also used as protocols for accessing storage.

Cloud Storage Requirement

1.Multi-tenancy

2..Security

3.Secure Transmission Channel

4.Performance

5.Quality of Service (QoS)

Page 21: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

6.Data Protection and Availability

7.Metering and Billing

CHAPTER 13CLOUD STORAGE FROM LANS TO WANS

DISTRIBUTED DATA STORAGE

Distributed storage means are evolving from the existing practices of data storage for the new generation of WWW applications through organizations like Google, Amazon and Yahoo

Amazon Dynamo - key-value store, primary-key only interface, Dynamo is said to be highly accessible for composing as opposed to reading

CouchDB - document-oriented database server, accessible by REST APIs, Cluster Of Unreliable Commodity Hardware Save: ACID compliant, save efficiently See: Easy retrieval, straightforward describing procedures, fulltext search Secure: Strong compartmentalization, ACL, connections over SSL Share: Distributed means

ThruDB - administration of the up-to-date WWW data level (indexing, caching, replication, backup)

APPLICATIONS UTILIZING CLOUD STORAGE Online File Storage - accessing documents from any location and from any

computer DropBox - permits us to upload any kind of document, as long as it is lesser than

350MB Box.net - preferred locations to shop articles online Live Mesh - upload any kind of document, will not edit Oosah - it mechanically converts them into JPGs JungleDisk - client can just pull and push documents back and forth between

online storage and the localized desktop

Cloud Storage Companies

Box cloud storage: Box makes it straightforward to set-up a cloud storage account. Surprisingly, clients can start the cloud and run it within few minutes

Amazon cloud: The Amazon Cloud Drive was one of the pioneering technologies in the cloud industry. This may be the most -preferred business for a client looking to backup his files on a daily basis.

SugarSync online backup: SugarSync permits not only backup for the documents but also links up the multiple devices

Page 22: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Hubic online storage: Hubic gained popularity because it is expressly conceived as an submission for both the iPhone and the Android platforms.

Google cloud drive: Google Cloud Drive may be outdated but they are proposing a good 5 GB for free

CLOUD SERVICESAdvantages of Cloud Computing

Resilience: Cloud computing eliminates single points of collapse. The malfunction at one node of the system has no influence on data accessibility and does not result in perceivable downtime.

Scalability: Cloud computing endows organizations to rapidly scale up their operations.

Flexibility and effectiveness: Cloud computing permits organizations to elaborate or agree computing power as required and permits ‘bursts’ of computing power to be utilized on an on-demand basis.

Outsourcing: Cloud computing makes outsourcing of undertakings straightforward and controllable to some extent. For numerous rudimentary organizational applications, outsourcing becomes a straightforward method, with fee only being paid for the computing power or data storage that is utilized, with no hidden cost or administration fees.

SOFTWARE AS A SERVICE (SAAS)

Best SaaS Examples

SalesForce CRM Google Apps DeskAway

Page 23: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Impel CRM Wipro w-SaaS

Two Main Categories of SaaS

Line of enterprise services

Customer-Oriented Services

Characteristics of SaaS

The application is hosted centrally, so new add-ons can be put in place without installing a new software.

The application has a single configuration, developing and checking faster. The application vendor has access to all clientele data, expediting conceive and

regression testing. The solution provider has access to client demeanour inside the application (usually

by WWW analytics), making it simpler to recognize areas where there is scope for improvement.

PLATFORM AS A SERVICE (PAAS)PaaS has some advantages for programs developers

Off location programs development Working on heterogeneous stages without bothering upgradation Initial and ongoing charges are minimized as the infrastructures are not bought Combination of development programmings minimize the overall costs Platform as a service (PaaS) is a combination of a computing phases and gurantess

deployment of applications without the cost and complexity of buying and managing inherent hardware and software. Some examples of PaaS are as follows:

Google’s AppEngine, which is based on Python Force.com  from SalesForce, based on the SalesForce SaaS infrastructure and Apex

language Bungee Connect: visual development studio based on Java LongJump: based on Java/Eclipse Winemaker: A studio based on Java and hosted at EC2

Definition of PaaS

Where businesses lease hardware and programs from a third party is said to be Platform as a Service (PaaS).It is one of the three major types of cloud computing. The stage is accessed over a personal network or the Internet and utilized to construct applications other than owning, running and evolving on an interior IT infrastructure.

Types of PaaS

Social application platforms Computation platforms Web application platforms

Page 24: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Business application platforms

Characteristics of PaaS

Multi-tenant Architecture

Support for Development of Group Collaboration

Characteristics of PaaS

Every phase is acted as a service. Provides services needed to support the entire lifecycle of construction and

consigning WWW applications. Provides services to establish, check, own and sustain application in the identical

IDE. Service provisioning encompasses concurrently, i.e., multiple users use the

identical IDE. Being a cloud, it pursues cloud service form, i.e., pay per use/pay per proceed

model. PaaS decreases TCO (total cost of ownership), as there is no need to purchase all

the frameworks, programs, devices and kits required to construct, run and deploy the application.

In-built scalability and elasticity to supply similar effectiveness. PaaS is flawless for agile programs development methodologies. PaaS assists in the fast building of applications in the cloud by supplying the

essentials.Live Examples of PaaS and Its Advantage1.Business Drivers for PaaS2.Portability of Assets and PaaS3.Cost Savings with PaaS

Page 25: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

INFRASTRUCTURE AS A SERVICE (IAAS)

Definition

Infrastructure as a Service (IaaS) is a form of service, consigning and outsourcing all types of computing infrastructures. It presents hardware, servers, storage and softwares services

IaaS sub-services are

DBaaS: Database get access to and use database administration system as a service STaaS: Data storage as a service, comprising of database-like services CCaaS: Compute capacity (CPUs) as virtual services are founded on on-demand

usage

IaaS Characteristics

Resources are supplied as a service Allows for dynamic climbing and elasticity Has a variable cost, usage-based cost form (pay per proceed and yield per use) Has multi-tenet architecture, encompasses multiple users on a single hardware IaaS normally has enterprise level infrastructure

IaaS Classifications

Private cloud: IaaS on a private network dedicated for a business. Private cloud is advised as the most protected IaaS offering.

Public cloud: IaaS service is established over the Internet and the client has to pay and use the service. It is similar to utility services such as electricity, gas and water.

Hybrid cloud: A blend of both private cloud and public cloud in an organization.

Benefits of IaaS

Allows IT to shift focus Hassle-free service Dynamic scaling Multiple tenants Investment cap Reduces TCO (total cost of ownership) Metered service Flexible offering Enables green IT

CLOUD COMPUTING AT WORK

Page 26: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Integrated Development Environments (IDEs) comprise of source codes, automation builders and a debugger.

CloudBees is a type of IaaS solution, which presents all services required to be established in the cloud free of charge.

Cloud-first is a cloud development platform, where first a cloud platform is constructed and then development application is constructed.

Tool-first is a cloud development platform, where development platform is constructed first that is host-friendly (e.g., studio sprints in a browser), then ‘push’ that platform into the cloud.

Event management engages revising the intricacies, recognizing the target audience, developing the happening notion, designing the logistics and coordinating the mechanical facets before really commencing the event.

A workflow administration system is a system that organizes and characterizes a sequence of jobs within an organization to make a final choice or outcomes.

Workflow administration systems permit the client to characterize distinct workflows for distinct occupations or processes.

A contact manager is a software program that enables users to effortlessly share, find and communicate data such as names, addresses and telephone numbers.

CRM (Customer Relations Management) is a methodology that assists an enterprise to organize clientele connections in a coordinated fashion.

Project management is the art of organizing all the facets of a task from inception to closure utilizing a technical methodology.

Mindjet Connect is a free collaboration platform that enables work groups to visually capture concepts and data, share documents and collaborate with others in real time or anytime.

RISKS IN CLOUD COMPUTING

Cloud Computing RisksRisk #1—The solution may not meet its economic objectives:Risk #2—The solution may not work in the context of the client enterprise’s association and culture:Risk #3—The solution may be tough to evolve due to the adversity of incorporating the cloud services involved:Risk #4—A catastrophe may occur from which the solution will not recover:Risk #5—System value may be insufficient, in order that it does not meet the users’ needs:Risk #6—There may be an existing need for service orientation

RISK MANAGEMENT

Risk management is a method for recognizing, considering and prioritizing risks of distinct kinds. Once the risks are recognized, the risk supervisor will conceive a design to minimize or eradicate the influence of contradictory events. There are several risk administration

Page 27: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

measures, encompassing those evolved by the Project Management Institute, the International Organization for Standardization (ISO), the National Institute of Science and Technology and societies.

The Risk Management Process

TYPES OF RISKS IN CLOUD COMPUTINGThreat #1—Misuse and illicit use of cloud computing:Threat #2—Insecure interfaces and APIsThreat #3—Vicious insidersThreat #4—Issues-related technology sharingThreat #5—Data loss or leakage:Threat #6—Hijacking (Account/Service)Threat #7—Unknown Risk Profile

DATA SECURITY IN CLOUDSecurity Advantages in Cloud Environments

Data centralization: In a cloud atmosphere, the service provider takes responsibility of storage and small organizations need not spend more money for personal storage devices. Also, cloud-based storage provides a method to centralize the data much faster and probably with low cost.

Incident response: IaaS providers contribute dedicated legal server which can be used on demand. Whenever there is a violation of the security policy, the server can be intimated through online.

When there is an inquest, a backup of the environment can be effortlessly made and put up on the cloud without affecting the usual course of business.

Forensic image verification time: Some cloud storage implementations reveal a cryptographic ascertain addition or hash. For example, MD5 hash function is developed automatically by Amazon S3 during object storage. Therefore in

Page 28: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

principle, the time to develop MD5 checkups utilizing external devices is eliminated.

Logging: In a usual computing paradigm by and large, logging is regular feature. In general, insufficient computer disk space is assigned that makes logging either non-existent or minimal. However, in a cloud, storage requirement for benchmark logs is mechanically solved.

Security Disadvantages in Cloud Environments

Investigation: Investigating an illegal undertaking may be unrealistic in cloud environments. Cloud services are particularly hard to enquire, because data for multiple clients may be co-located and may also be dispersed over multiple datacentres. Users have little information about the mesh topology of the inherent environment. Service provider may also enforce limits on the network security of the users.

Data segregation: Data in the cloud is normally in a distributed simultaneously with data from other customers. Encryption will not be presumed as the single solution for data segregation issues. Some clients may not desire to encrypt data because there may be a case when encryption misleads can decimate the data.

Long-term viability: Service providers should double-check the data security in altering enterprise positions, such as mergers and acquisitions. Customers should double-check data accessibility in these situations. Service provider should furthermore confirm data security in contradictory situations such as extended outage, etc.

Compromised servers: In a cloud computing environment, users do not even have an alternative of utilizing personal acquisition toolkit. In a situation where a server is compromised, they require to shut their servers down until they get a backup of the data. This will further create source accessibility concerns.

Regulatory compliance: Traditional service providers are exempted from outside audits and security certifications. If a cloud service provider does not adhere to these security audits, then it directs to a conspicuous decline in clientele trust.

Recovery: Cloud service providers should double-check the data security in natural and man-made disasters. Generally, data is duplicated over multiple sites. However, in the case of any such redundant happenings, provider should do an absolute and fast restoration.

Password validation: Username and password are validated contrary to the delegated administration rather than of the interior Salesforce password store.

Token validation: Users should first authenticate to their enterprise and the enterprise in turn should conceive a Salesforce by dispatching (via HTTP POST) the username and a token to Salesforce for validation by the delegated authority.

Page 29: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Hybrid model: While accessing the Salesforce website, users are required to use token validation, but they are permitted to validate using password validation on a consumer application.

CLOUD SECURITY SERVICES

CIA is a broadly utilized standard for evaluation of data systems security, focusing on the three centre goals of confidentiality, integrity and availability of information.

Data ConfidentialityAccess controlPasswordsBiometricsEncryptionPrivacyEthicsData IntegrityData integrity is characterized as safeguarding the correctness and completeness of data and processing procedures from intentional, unauthorized or unintentional changes. Maintaining data integrity is absolutely crucial to the privacy, security and reliability of enterprise data. Integrity of data can be compromised by malicious users, hackers, programs mistakes, computer virus, hardware constituent flops and by human mistake while moving data.Data Availability

Availability is double-checking that the authorized users have access to data and affiliated assets when required. This can be carried out by utilizing data backup, catastrophe recovery and enterprise continuity/recovery plans. Employees should have knowledge about their responsibilities as it concerns data backups, catastrophe recovery and enterprise continuity.

DRP is a design that is utilized to retrieve rapidly after a catastrophe with a smallest of liability to the organization.

Authorization entails for double-checking that only authorized persons are adept to get access to resources within a system.

Cloud tools are a set of tools for establishing, organizing and testing Java EE applications on elastic computing cloud owned by Amazon.

PushToTest TestMaker is a distributed testing environment that can run tests on test gear or in a cloud computing environment.

TOOLS AND TECHNOLOGIES FOR CLOUD

High performance computing needs the use of Massively Parallel Processing (MPP) systems encompassing thousands of mighty CPUs.

The most famous two eras of computing are the (i) sequential and (ii) parallel eras. Cloud computing refers to both the applications consigned as services over the

Internet and the hardware and systems programs in the data hubs that supply those services.

Cloud technologies for HPC are Hadoop, Dryad and CGL-MapReduce.

Page 30: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Cloud technologies like Google MapReduce, Google File System, Hadoop and Hadoop Distributed File System, Microsoft Dryad and CGL-MapReduce take a more data-centred set regarding two parallel runtimes.

Services in the cloud can be grouped into three categories: (i) Software as a Service (SaaS), (ii) attached services and (iii) cloud platforms.

Development tools are another significant part in platforms. Modern tools assist developers in constructing applications utilizing the components of an application platform.

On-premises platform is split into two very broad categories: (i) packaged applications and (ii) custom applications.

Cloud computing platforms are (i) Abicloud Cloud Computing Platform, (ii) Eucalyptus Cloud Platform, (iii) Nimbus Cloud Computing Platform and (iv) OpenNebula Cloud Computing Platform.

Distributed computing is a by-product of Internet. Distributed development is global development, which adds its own trials with collaboration and code management.

Git and subversion are two tools broadly utilized in distributed environments. There are eight key components to address when constructing an internal or

external compute cloud: (i) shared infrastructure, (ii) self-service automated portal, (iii) scalable, (iv) rich application container, (v) programmatic control, (vi) 100% virtual hardware abstraction, (vii) strong multi-tenancy and (viii) chargeback.

KEY TERMS

In parallel processing, simultaneous use of more than one CPU is used to execute a program.

High performance computing is a fork of computer science that concentrates on devising supercomputers and software to run on it.

GoGrid is a leader in providing hybrid and cloud infrastructure hosting. GoGrid endows sysadmins, developers and IT professionals to conceive, establish

and command dynamic load-balanced cloud infrastructures. Dryad is an ongoing study task at Microsoft Research for runtime execution of

facts and number-aligned applications.CLOUD MASHUPS

MASHUPS

Mashups stands on the basic concept of data and services integration. To function in this way, combination, aggregation and visualization are three main primitives:

1. Combination collects data from heterogeneous sources, uses it among the same application.

2. Aggregation operates on collected data having a measure and builds new information using the obtained data.

3. Visualization is used to integrate data in a diagram way using maps or by using other multimedia objects.

CLOUD MASHUPS

Page 31: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Mashups use API software (application programming interface) to combine one or more website elements. A cloud mashup is simply an instance of a web-based mashup, but the application content resides in the cloud. The reasons are as follows:

If cloud mashup is hosted in the cloud, then it will be placed next to some useful software building tools, if the user subscribes to a cloud mashup centre service. A good enterprise mashup platform features reusable application blocks that can be used to build new applications.

APACHE HADOOPHADOOP

Hadoop is a sub-project of Lucene, under the Apache Software Foundation. Hadoop parallelizes data processing over numerous nodes (computers) in a compute cluster, racing up large computations and concealing I/O latency through improved concurrency. Hadoop is particularly well-suited to large data processing jobs (like seeking and indexing). It can also leverage its distributed file system at a reduced cost and reliably duplicate chunks of data to nodes in the cluster, producing data accessible in the local area on the appliance that is processing it.

Map and Reduce

The MapReduce paradigm takes idea from the map and it reduces programming constructs widespread in abundant programming languages.

Hadoop Nodes

NameNode: The NameNode is the centred position for data about the file system established in a Hadoop environment.

DataNode: DataNodes make up the most of the servers comprised in a Hadoop environment. The DataNode assists two functions: It comprises a piece of the data in the Hadoop Distributed File System (HDFS) and it actions as a compute stage for running occupations, some of which will utilize the localized data inside the HDFS.

EdgeNode: The EdgeNode is the access point for external applications, devices and users that require utilizing the Hadoop environment. The EdgeNode is seated between the Hadoop cluster and the business mesh to supply access to command, principle enforcement, logging and entrance services to the Hadoop environment.

HDFS: Goals and AssumptionsHardware failureStreaming data accessLarge data setsSimple coherency modelMoving computation is cheaper than moving dataPortability over heterogeneous hardware and software platforms

Page 32: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

MAPREDUCE

MapReduce is a programming model and an affiliated implementation for processing and developing large data sets. A Map function is created by users, which contains key/value pair and they build an intermediate set of same pairs. A function called reduce merges all intermediate values with the same pairs. Many genuine world jobs are expressible in this model.

Map and Reduce

MAP function: Map Function, written by the client, takes an input pair and makes a set of Intermediate Key/Value Pairs. The MapReduce library assembles all intermediate values affiliated with the identical intermediate key ‘I’ simultaneously and passes them to the Reduce function. 

Reduce function: Reduce function, written by the client, acknowledges an intermediate key ‘I’ and a set of values for that key. It merges these values to pattern a lesser set of values. Typically, just none or one yield value is made per Reduce invocation. The intermediate values are provided to the user’s reduce function by an iterator.

HADOOP VERSUS DISTRIBUTED DATABASESScalabilityFault tolerancePerformance:Data loading:

CLOUD TOOLS

VMWARE

VMware supports:

Desktop software consisting of: VMware workstation VMware fusion VMware player

In the server software VMware markets two virtualization products for servers: VMware ESX

and VMware ESXi. The VMware server is furthermore supplied as freeware for non-

commercial use, like VMware player, and it is likely to conceive virtual machines with it. It is a ‘hosted’ application, which sprints inside an existing Linux or Windows OS.

The cloud management software consists of:

Page 33: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

VMware vCloud VMware Go

EUCALYPTUS

For Linking Your Programs To Useful Systems can use Elastic Utility Computing Architecture is the GPL-licensed software which presents tooling to create and organize a private cloud that can even be accessed as a public cloud. It is a compatible platform for Amazon EC2 and S3 storage. It makes its services accessible through EC2/S3 compatible APIs. Features in it are:

Interface compatibility with EC2 Simple setting up and deployment utilizing rocks Simple set of extensible cloud share policies Overlay functionality needing no modification into the Linux environment Basic administrative tools for system administration and client accounting Configuring multiple clusters with private mesh locations into a single cloud Portability

Components of Eucalyptus

1. Cloud controller (CLC): Comprises the front-end services and the Walrus storage system.

2. Cluster controller (CC): Gives support for the virtual mesh overlay.3. Node controller (NC): Interacts with VT to classify individual VMs.

The two constituents are utilized for storage administration:

1. Storage controller (SC): Presents continual impede storage for the instances.2. Walrus storage controller (WSC): Presents continual and straightforward storage

service.

CLOUDSIM

CloudSim functionalities are as follows:

Support for modelling and replication of large-scale cloud computing data centres. Support for modelling and replication of virtualized server hosts, with customizable

principles for provisioning host assets to virtual machines. Support for modelling and replication of energy-aware computational resources. Support for modelling and replication of data centre mesh topologies and message-

passing applications. Support for modelling and replication of federated clouds. Support for dynamic insertion of replication components, halt and restart of

simulation.

Page 34: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Support for user-defined principles for share of hosts to virtual appliances and principles for share of owner assets to virtual machines.

OPENNEBULA

OpenNebula does the following:

Management of the network, computing and storage capacity Management of VM life-cycle Management of workload placement Management of virtual networks Management of VM images Management of information and accounting Management of security Management of remote cloud capacity Management of public cloud servers

NIMBUS

Enables asset providers to construct personal and community IaaS cloud. Enables users to use IaaS clouds. Enables developers to continue, trial and customize IaaS.

Major features are as follows: Open source IaaS: Nimbus presents a 100% freely accessible and open source

infrastructure as a service (IaaS) system. Every characteristic that a community develops is freely accessible and there are no add-on or improvement costs.

Storage cloud service: Cumulus is a storage cloud service that is matching with the S3 REST API. It can be utilized contrary to numerous existing purchasers (boto, s3cmd, jets3t, etc.) to supply data storage and transfer services.

EC2 based clients are capable of utilizing Nimbus installations. Both SOAP API and the REST API have been applied in Nimbus. S3 REST API clients can also be utilized for organizing VM storage with the Nimbus system.

Easy to use cloud client: The workspace cloud client permits authorized clients to get access to numerous workspace service characteristics in a client amicable way. It is conceived to get users up and running in a time-span of minutes, even from laptops, NATs, etc. The workspace cloud client supports for storing data in cloud also acts as IaaS. Even the uninitiated finds this completely incorporated device so straightforward to use.

Per-user storage quota: Cumulus (the VM likeness repository supervisor for Nimbus) can be configured to enforce per client storage usage limits.

Easy client management: New in Nimbus 2.5 are a set of client administration tools that make administering a Nimbus cloud considerably easier. The tools are both straightforward to use and scriptable.

Page 35: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

MOVING APPLICATIONS TO THE CLOUD

CLOUD OPPORTUNITIES1.Cloud for cost reduction2.Cloud for enterprise growth3.Cloud for fast innovation4.Cloud for enterprise agility

The S, P and I

The cloud has three elements: Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS). There is, of course, a fourth element: IT services on the cloud.

APPLICATIONS IN THE CLOUD

1. Clients would be adept to get access to their applications and data from any location at any time. They could get access to the cloud computing system utilizing any computer connected to the Internet.

2. It could convey hardware charges down. The cloud computing system would decrease the requirement for sophisticated hardware on the purchaser side.

Applications Shifted to the Cloud

1.E-mail 2. Conferencing software 3. CRM 4. Web hosting 5. Batch processing applications

CLOUD DESKTOPAccess anywhere, everywhere, anytimePersonalized and persistentInexpensive and hassle-freeSecure and reliableEasy to manage

Page 36: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

MICROSOFT CLOUD SERVICES

WINDOWS AZURE PLATFORM

1.Microsoft Cloud Services

The platform comprises diverse on-demand services hosted in Microsoft data hubs and consigned through three merchandise brands:

1. Windows Azure (a functioning system supplying scalable compute and storage facilities).

2. SQL Azure (a cloud-based, scale-out type of SQL server).3. Windows Azure AppFabric (an assemblage of services carrying applications both in

the cloud and on premise).

Here are some example applications that might be constructed on Windows Azure:

A self-determining software vendor (ISV) could visualize an application that aims enterprise users; an approach that is often brought up is Software as a Service (SaaS). ISVs can use Windows Azure as a base for business-oriented SaaS applications.

An ISV might conceive a SaaS application that targets consumers. Windows Azure is conceived to support very scalable programs and so a firm that designs to target a large buyer market will select it as a base for a new application.

Employees use Windows Azure to construct and run applications within Enterprises. While this position will not likely need the tremendous scale of a consumer-facing application, the reliability and manageability that Windows Azure boasts could still make it an appealing choice.

Windows Azure has three centre components:

1. Compute which presents a computation environment with Web Role, Worker Role and VM Role.

2. Storage which focuses on supplying scalable storage (Blobs, non-relational Tables and Queues) for large-scale needs.

3. Fabric which values high-speed attachments and swaps to interconnect nodes comprising some servers. Fabric resources, applications and services running are organized by the Windows Azure Fabric Controller service.

4. Cloud promises to make IT not just lower, but furthermore much quicker, simpler, more flexible and more effective.

Page 37: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

5. Microsoft boasts the advantages of cloud with the familiarity of Microsoft applications that users, developers, IT professionals and leaders currently understand and trust.

6. Cloud-based services that add worth to these existing installations are very well aimed at niche opportunities.

7. The Microsoft cloud computing platform utilizes Windows Azure to construct and extent world wide web applications using its data centres.

8. Windows Azure provides a Microsoft Windows Server–based computing environment for applications and continual storage for both organized and unstructured data, as well as asynchronous messaging.

9. The Windows Azure AppFabric provides a variety of services that assist consumers to attach users and on-premise applications to cloud-hosted applications, authentication and data administration.

10. SQL Azure is vitally an SQL server supplied as a service in the cloud.11. Windows Azure is the application service in cloud which allows Microsoft data

centres to host and run applications.12. Windows Azure has three centre components (i) compute, (ii) storage and (iii)

fabric.13. The Windows Azure Compute Service can run numerous distinct types of

applications.GOOGLE CLOUD APPLICATIONS

GOOGLE APPLICATIONS UTILIZING CLOUD

Gmail

E-mail, IM, voice and video chat: Each client gets 25 GB of e-mail and IM storage. Anytime, any location, get access to your e-mail: Gmail is securely powered by the

World Wide Web, so you can be creative from your table, on the street, at home and on your wireless telephone, even when you are offline.

Sync with Android, iPhone and BlackBerry: Get the advantages of Apps on premier wireless platforms.

Search and find e-mails instantly: Spend short time in managing e-mail and locate e-mails quickly with Google-powered search to your inbox.

Get less spam: Gmail spam is powerful filtering which assists you to concentrate on important ones.

Google Calendar

Easily schedule appointments: Overlay multiple calendars to glimpse when people are available. Google Calendar drives requests and organizes them.

Integrate with e-mail system: Google Calendar is incorporated into Gmail and interoperable with well-liked calendar applications.

Page 38: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Share task calendars: Calendars can be distributed company-wide or with chosen coworkers. A variety of distributing consent controls assist to sustain security and privacy.

Access with your wireless device: View and edit happening minutia, add new happenings and ask for visitors on wireless devices like the BlackBerry and iPhone. Even obtain calendar notifications by SMS.

Publish calendars: Publicize external business happenings by announcing a calendar to make it searchable in the Google Calendar gallery. Easily embed calendars into web pages.

Google Docs

Works over other operating systems: Google Docs works in the browser on PC, Mac and Linux computers and carries well-liked formats, for example,.doc,.xls,.ppt and.pdf.

Easily upload and share files: Files retained on Google Docs are habitually accessible and backed-up online.

Secure access to controls: Administrators can organize document distributing permissions system-wide and article proprietors can share and revoke document access at any time.

Google Sites

Organize knowledge in a central place: Use Google Sites to centralize written material, spreadsheets, demonstrations, videos, slideshows and more to aid to retain the teams organized.

Anytime, everywhere access: Google Sites are securely propelled by the web, so you can gather pages from your office desk, on the move, at home and on your portable phone.

Works through various operating systems: Google Sites work on the browser installed on PC with Mac and Linux computers.

System and site-level security controls: Administrators can supervise site sharing permissions through the enterprise and authors can share and revoke file access at any time.

GOOGLE APP ENGINE

Dynamic world wide web assisting with full support of widespread web technologies.

Persistent storage with queries, sorting and transactions that are powered by Bigtable and GFS.

Scalability and load balancing are done automatically. Google APIs for authenticating users and dispatching e-mail. Fully boasted localized development environment.

Page 39: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Google Apps for Business

With Google Apps for Business, client gets:

Customized e-mail addresses Mobile e-mail, calendar and IM access No added hardware or programs Industry-leading spam filtering 24/7 internet note and telephone support 99.9% uptime assurance Dramatic cost savings

Google Apps for Education

Google Apps advantages the campus are listed herewith:

Students will love you for it Free up your IT Easy to deploy Save money Google defends your privacy Security as powerful as Google Innovation in real-time Collaborate globally

Google Apps for GovernmentGoogle Apps for Government presents all of the identical advantages that

Google Apps for Business does, but with a supplemented level of security that stands up to even the largest levels of government standards. With Google Apps, the government department benefitted from the scale and redundancy of distributed data centres around the globe.

AMAZON CLOUD SERVICES

Amazon Web Services

Reliability: The services run in Amazon’s battle-tested, highly obtainable data centres that run Amazon’s own business.

Security: Basic security and authentication methods are obtainable out of the packing box and customers can enhance them as wanted by layering his/her application-specific security on apex of the services.

Cost benefits: No fastened charges or support costs. Ease of development: Simple APIs allow us to harness the full power of this virtual

infrastructure and libraries, obtainable in most extensively employed programming languages.

Elasticity: Scale the computing supplies based on demand.

Page 40: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Cohesiveness: The four quintessence building-blocks using which services (storage, computing, messaging and datasets) are created from scratch currently work well and give a whole result through a large type of request for paid job domains.

Community: Tap into the vibrant and dynamic customer community that is propelling the extensive adoption of these web services and is bringing ahead sole requests for paid jobs assembled on this infrastructure.

Amazon S3 (Storage)

Amazon Simple Storage Service (S3) presents a web service interface for the storage and retrieval of data. The data can be of any kind and can be retained and accessed from any location over the internet. Users can shop an unlimited number of things in S3, and the dimensions of each retained object can vary from 1 byte to 5 GB.

Amazon EC2 (Elastic Computing)

Amazon EC2 is a web service that permits us to use virtual machines within minutes and effortlessly scale the capability up or down founded on demand. These instances are based on Linux and can run any submission or software. The EC2 natural environment itself is constructed on the peak of the open source Xen hypervisor.

Amazon SQS (Simple Queue Service)

Amazon Simple Queue Service (SQS) presents get access to the dependable messaging infrastructure utilized by Amazon. Users can send and receive messages from any location utilizing straightforward REST-based HTTP requests.

ELASTIC COMPUTE CLOUD (EC2)

Amazon’s features are:

A service grade affirmation for EC2 Microsoft Windows in beta pattern on EC2 Microsoft SQL Server in beta pattern on EC2 Designs for an AWS (Amazon Web Service) administration console Designs for load balancing, auto-scaling and cloud supervising services

Amazon S3

Amazon S3 is storage for the internet. It presents a straightforward World Wide Web service interface that endows us to shop and get any amount of data from any location on the web. Amazon EBS presents the instances with continual, block-level storage. Amazon EBS volumes are vitally hard computer disks that can adhere to a running instance.

Page 41: oms.bdu.ac.inoms.bdu.ac.in/...16SMBECS2:2-16SMBECA2:1-16SMBEIT2…  · Web viewAIMAN COLLEGE OF ARTS & SCIENCE FOR WOMEN,TIRUCHIRAPPALLI. Class : I. II . B.Sc. Computer Science –V

Databases

1. Use Amazon Relational Database Service (Amazon RDS), which endows us to effortlessly get an organized relational database in the cloud.

2. Launch an instance of a database AMI and use that EC2 instance as the database.

Amazon CloudWatchAmazon CloudWatch is a web service that presents real-time supervising to Amazon’s EC2 clients on their asset utilization, such as CPU, computer disk and network. CloudWatch does not supply any recollection, computer disk space or load average metrics.