h8281 Atmos Ve Vnx Deployment

8/12/2019 h8281 Atmos Ve Vnx Deployment

1/79

EMCAtmosVirtualEdition with EMCVNXSeriesDeployment Guideh8281.2


2/79

EMC Atmos Virtual Edition with EMC VNX Series

Deployment Guide

Copyright , 2011 EMC Corporation. All rights reserved.

Published October, 2011

EMC believes the information in this publication is accurate as of its publication date.The information is subject to change without notice.

The information in this publication is provided as is. EMC Corporation makes norepresentations or warranties of any kind with respect to the information in thispublication, and specifically disclaims implied warranties of merchantability orfitness for a particular purpose. Use, copying, and distribution of any EMC softwaredescribed in this publication requires an applicable software license.

EMC2, EMC, Data Domain, RSA, EMC Centera, EMC ControlCenter, EMC LifeLine, EMCOnCourse, EMC Proven, EMC Snap, EMC Source-One, EMC Storage Administrator,Acartus, Access Logix, AdvantEdge, AlphaStor, ApplicationXtender, ArchiveXtender,Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart,AutoSwap, AVALONidm, Avamar, Captiva, C-Clip, Celerra, Celerra Replicator, Centera,CenterStage, CentraStar, ClaimPack, CLARiiON, ClientPak, Codebook CorrelationTechnology, Common Information Model, Configuration Intelligence, Configuresoft,

Connectrix, CopyCross, CopyPoint, CX, Dantz, DatabaseXtender, Data Domain, DirectMatrix Architecture, DiskXtender, DiskXtender 2000, Document Sciences,Documentum, elnput, E-Lab, EmailXaminer, EmailXtender, Enginuity, eRoom, EventExplorer, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global FileVirtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, InfoMover,Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, ISIS, Max Retriever,MediaStor, MirrorView, Navisphere, NetWorker, nLayers, OnAlert, OpenScale,PixTools, PowerPath, PowerSnap, QuickScan, Rainfinity, RepliCare, RepliStor,ResourcePak, Retrospect, RSA, SafeLine, SAN Advisor, SAN Copy, SAN Manager,Smarts, SnapImage, SnapSure, SnapView, SRDF, StorageScope, SupportMate,SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder,UltraFlex, UltraPoint, UltraScale, Unisphere, Vblock, VMAX, VPLEX, Viewlets, VirtualMatrix, Virtual Matrix Architecture, Virtual Provisioning, VisualSAN, VisualSRM,Voyence, WebXtender, xPression, xPresso, YottaYotta, the EMC logo, the RSA logo,and where information lives are registered trademarks or trademarks of EMCCorporation in the United States and other countries. All other trademarks usedherein are the property of their respective owners.

For the most up-to-date regulatory document for your product line, go to the technicaldocumentation and advisories section on EMC Online Support.

EMC Atmos Virtual Edition with EMC VNX SeriesDeployment GuidePart Number h8281.2


3/79


Deployment Guide

Contents

Preface ..................................................................................... 11Chapter 1 Introduction ........................................................................ 13Introduction to EMC VNX Series .................................................................... 14

Software suites available ..................................................................................... 14Software packs available ..................................................................................... 15

Introduction to EMC Atmos ............................................................................ 15Introduction to Atmos on VNX solution ......................................................... 20Atmos on VNX solution topology ................................................................... 20Chapter 2 Setup and Configuration ..................................................... 23Introduction .................................................................................................. 24Atmos VE on VNX test environment ............................................................... 24

Hardware resources ............................................................................................. 26Software resources .............................................................................................. 26

Configure VNX storage .................................................................................. 27Configure file-based storage (NFS) for Atmos VE ................................................... 28Storage layoutfile-based storage (NFS).............................................................. 28Configure storageboot disks of Atmos VE nodes................................................ 28Configure storagedata and metadata disks of Atmos VE .................................... 29Configure storage for application servers ............................................................. 29Configure block-based storage (FC) for Atmos VE ................................................. 30Storage layoutblock-based storage (FC) ............................................................ 30Configure storageboot disk of Atmos VE ............................................................. 30Configure storagedata and metadata disks of Atmos VE .................................... 31Configure storage for application servers ............................................................. 31

Configure network ......................................................................................... 32Configure private and public networks ................................................................. 32

Configure storage network ............................................................................ 33Configure network for file-based storage (NFS) ..................................................... 33Configure network for block-based storage (FC) ................................................... 34


4/79

Contents


Deployment Guide

Configure vSphere......................................................................................... 35Configure virtual machines for Atmos VE nodes.................................................... 36Post-configuration steps for the Atmos VE nodes ................................................. 37

Configure Atmos Virtual Edition .................................................................... 39Configure multisite object configuration ............................................................... 39

Chapter 3 Monitor and Management ................................................... 43Introduction .................................................................................................. 44EMC Unisphere ............................................................................................. 44Atmos administration GUI ............................................................................. 44Ganglia Atmos grid report ............................................................................. 46EMC VSI for VMware vSphere Unified Storage Management .......................... 46Migrate Atmos VE nodes ............................................................................... 47

Migrate Atmos nodes with vMotion and Storage vMotion ..................................... 48Observations ....................................................................................................... 48Conclusions ......................................................................................................... 51

Chapter 4 Atmos on VNX Performance ................................................. 53Introduction .................................................................................................. 54Atmos on VNX performancefile and block .................................................. 54

Test method ......................................................................................................... 54Result analysis ..................................................................................................... 54Conclusion ........................................................................................................... 56

Atmos on VNX performanceobject and non-object ..................................... 56Test method ......................................................................................................... 56Result analysis ..................................................................................................... 57Conclusion ........................................................................................................... 60

Chapter 5 Storage Efficiency ............................................................... 61Introduction .................................................................................................. 62Storage efficiency with thin provisioning ...................................................... 62

Configure thin provisioning on file-based object store .......................................... 62Configure thin provisioning on block-based object store ...................................... 62Test method ......................................................................................................... 63Result analysis ..................................................................................................... 63Conclusion ........................................................................................................... 68

Storage efficiency with compression ............................................................. 68Configure compression on file-based object store ................................................ 68Configure compression on block-based object store ............................................ 70Test method ......................................................................................................... 72


5/79

Contents

EMC Atmos Virtual Edition with EMC VNX SeriesDeployment Guide

Result analysis ..................................................................................................... 73Conclusions ......................................................................................................... 75

Appendix A Using Grinder to Generate REST Workload on Atmos ............ 77Grinder tool ................................................................................................... 78

Grinder test driver system configuration ............................................................... 78Grinder script configuration .................................................................................. 78Run a Grinder test and generate report ................................................................. 78


6/79

Contents


Deployment Guide


7/79


Deployment Guide

Figures

Figure 1. Optimize applications across the business with Atmos ...................... 16Figure 2. Atmos physical hardware ................................................................... 18Figure 3. Atmos Virtual Edition .......................................................................... 19Figure 4. Atmos administration GUI ................................................................... 20Figure 5. Atmos on VNX solution topology ......................................................... 21Figure 6. Atmos on VNX high-level typical configuration .................................... 25Figure 7. File-based storage layout for Atmos VE ............................................... 28Figure 8. Block-based storage layout for Atmos VE ............................................ 30Figure 9. vSphere Client .................................................................................... 32Figure 10. Private network configured ................................................................. 33Figure 11. Storage network diagramNFS ........................................................... 34Figure 12. Network diagramFCoE ...................................................................... 35Figure 13. Atmos node properties ....................................................................... 37Figure 14. Atmos VE nodes with DRS resources pools ......................................... 38Figure 15. Automatic startup for Atmos VE nodes ................................................ 38Figure 16. RMG list .............................................................................................. 39Figure 17. Tenant list .......................................................................................... 39Figure 18. Tenant Basic Information window ....................................................... 40Figure 19. Policy Specification window ............................................................... 41Figure 20. Configure storage pool ....................................................................... 44Figure 21. Add RMG ............................................................................................ 45Figure 22. System Summary window ................................................................... 45Figure 23. Ganglia Atmos grid report ................................................................... 46Figure 24. VSI icon .............................................................................................. 46Figure 25. Provision storage ................................................................................ 47Figure 26. REST TPS without vMotion .................................................................. 49Figure 27. REST TPS with vMotion ....................................................................... 49Figure 28. Performance comparison for large objects .......................................... 50Figure 29. Performance comparison for small objects ......................................... 51Figure 30. Performance of object store for NFS and FCoE configurations .............. 55Figure 31. Performance of small objects for NFS and FCoE configurations ........... 55Figure 32. Performance of large objects for NFS and FCoE configurations ............ 56Figure 33. Performance of object store for NFS and FCoE configurations .............. 58Figure 34. Small objects in NFS and FC datastoresREST and OLTP workload ..... 59Figure 35. Large objects in NFS and FC datastoresREST and OLTP workload ..... 60Figure 36. Thin provisioned FC ............................................................................ 62Figure 37. Performance of object storethin provisioned NFS ............................. 64Figure 38. Performance of object storethin provisioned FC ................................ 64Figure 39. Large objectsbaseline compared with thin provisioned FCoE ............ 65


8/79

Figures


Deployment Guide

Figure 40. Large objectsbaseline compared with thin provisioned NFS ............. 66Figure 41. Small objectsbaseline compared with thin provisioned FCoE ........... 66Figure 42. Small Objectsbaseline compared with thin provision NFS ................ 67Figure 43. Small objectsthin provisioned NFS compared with FCoE ................... 67Figure 44. Large objectsthin provisioned NFS compared with FCoE ................... 68Figure 45. Compression file system .................................................................... 69Figure 46. Compression ratio .............................................................................. 70Figure 47. Create LUN ......................................................................................... 71Figure 48. LUN Propertiescompression ............................................................ 72Figure 49. Performance of large objectsbaseline and LUN compression ........... 74Figure 50. Performance of small objectsbaseline and LUN compression .......... 75


9/79


Deployment Guide

Tables

Table 1. Hardware resources ............................................................................ 26Table 2. Software resources ............................................................................. 26Table 3. Virtual machine configuration ............................................................. 36Table 4. Application server configuration specifications .................................. 57Table 5. Thin provisionedNFS ........................................................................ 63Table 6. Thin provisionedFC .......................................................................... 63Table 7. LUN CompressionFC ........................................................................ 73


10/79

Tables


Deployment Guide

0


11/79


Deployment Guide

Preface

As part of an effort to improve and enhance the performance and capabilities of itsproduct line, EMC from time to time releases revisions of its hardware and software.Therefore, some functions described in this guide may not be supported by allrevisions of the software or hardware currently in use. For the most up-to-dateinformation on product features, refer to your product release notes.

If a product does not function properly or does not function as described in this

document, please contact your EMC representative.

Note This document was accurate as of the time of publication. However,as information is added, new versions of this document may be released tothe EMC Online Support website. Check the EMC Online Support website toensure that you are using the latest version of this document.

PurposeThis document describes how to configure a REST-based and SOAP-based objectstore using EMCAtmosVirtual Edition (Atmos VE) and EMCVNX series. Thissolution is called Atmos on VNX solution in this document.In addition, several aspects of this solution such as management, performance, andstorage efficiency are reviewed and the details are provided in this document.

ScopeThis document focuses primarily on how to deploy an object store based on thissolution. The document focuses on key configurations that were found suitable andincludes high-level description of the end-userspublic network and the requirednetwork load balancer.

To learn more about this solution, contact an EMC representative

AudienceThis document is intended for internal EMC personnel, partners, and customers who

are looking to build an object store using Atmos VE on the available VNX seriesstorage and servers.


12/79

Preface


Deployment Guide

2

Related documentsThe following documents, located on the EMC Online Support website, provideadditional, relevant information. Access to these documents is based on the logincredentials. If you do not have access to the following documents, contact your EMCrepresentative:

EMC Atmos Virtual EditionBest Practice Guide

EMC AtmosAdministrators Guide

Using EMC VNX Storage with VMware vSphereTechBook

EMC VSI for VMware vSphere: Unified Storage ManagementProduct Guide

VMware documentsThe following documents are available for download from the VMware website:

ESXi Configuration Guide

Resource Management guide

Note There are multiple versions of these guides on the VMware website.Refer to the guide that corresponds to the appropriate vSphere release.


13/79


Deployment Guide

Chapter 1 Introduction

This chapter presents the following topics:

Introduction to EMC VNX Series .................................................................. 14Introduction to EMC Atmos ......................................................................... 15Introduction to Atmos on VNX solution ....................................................... 20Atmos on VNX solution topology ................................................................ 20


14/79

Introduction


Deployment Guide

4

Introduction to EMC VNX SeriesThe EMCVNXseries delivers uncompromising scalability and flexibility for themidtier while providing market-leading simplicity and efficiency to minimize total costof ownership. The VNX series is powered by Intel(r) Xeon(r) Processors, for intelligentstorage that automatically and efficiently scales in performance, while ensuring data

integrity and security.Customers can benefit from the new VNX features such as:Next-generation unified storage, optimized for virtualized applications

Extended cache using Flash drives with FAST Cache and Fully AutomatedStorage Tiering for Virtual Pools (FAST VP), which can be optimized for thehighest system performance and lowest storage cost simultaneously on bothblock and file.

Multiprotocol support for file, block, and object with object access throughAtmos Virtual Edition (Atmos VE).

Simplified management with EMC Unisphere for a single managementinterface for all NAS, SAN, and replication needs.

Up to three times improvement in performance with the latest Intel multicoreCPUs, optimized for Flash.

6 Gb/s SAS back end with the latest drive technologies supported:

o 3.5 100 GB and 200 GB Flash, 3.5 300 GB, and 600 GB 15k or 10krpm SAS, and 3.5 2 TB 7.2k rpm NL-SAS

o 2.5 300 GB and 600 GB 10k rpm SASExpanded EMC UltraFlex I/O connectivityFibre Channel (FC), InternetSmall Computer System Interface (iSCSI), Common Internet File System (CIFS),Network File System (NFS) including parallel NFS (pNFS), Multi-Path File

System (MPFS), and Fibre Channel over Ethernet (FCoE) connectivity forconverged networking over Ethernet.

The VNX series includes five new software suites and three new software packs,making it easier and simpler to attain the maximum overall benefits.

Software suites available VNX FAST SuiteAutomatically optimizes for the highest system performanceand the lowest storage cost simultaneously (FAST VP is not part of the FASTSuite for the VNX5100).

VNX Local Protection SuitePractices safe data protection and repurposing.VNX Remote Protection SuiteProtects data against localized failures,outages, and disasters.VNX Application Protection SuiteAutomates application copies and provescompliance.

VNX Security and Compliance SuiteKeeps data safe from changes,deletions, and malicious activity.


15/79

Introduction


Software packs available Total Efficiency PackIncludes all five software suites (not available for theVNX5100 and VNXe series).

Total Protection PackIncludes local, remote, and application protectionsuites (not available for the VNXe3100).Total Value PackIncludes all three protection software suites and theSecurity and Compliance Suite (the VNX5100 and VNXe3100 exclusivelysupport this package).

Introduction to EMC AtmosEMCAtmosis a multi-petabyte platform for information storage and distribution. Itcombines massive scalability with automated data placement to efficiently delivercontent worldwide.

The Atmos platform provides the following features:

Information lifecycle managementAtmos includes robust policy-basedinformation management functions that automate data placement andprotection. The policy engine supports advanced information services, suchas, erasure codes, compression, replication, deduplication, and disk drivespin-down. You can define different policies based on the data value to thebusiness over time, or based on the needs of different customers ordepartments within the organization.

MultitenancyEnables to segregate storage into logical units called tenants.This multi-tenant architecture enables you to deploy multiple applicationswithin the same infrastructure with each application securely partitioned sothat data is only accessible by the tenant who owns it.

A browser-based administration toolEnables to efficiently manage aglobally deployed system.

Representational State Transfer (REST) and Simple Object Access Protocol(SOAP) web service APIsProvides a standards-based web services API forcreating and managing content. It also supports file-based access forconvenient integration to virtually any application.

A universal namespaceA distributed, hierarchical structure that presentsfile system addressing services to the content stored in Atmos. It aggregatesmultiple storage segments, and sites into a single addressable storage entityseparated through the use of secure multitenancy.

Advanced auto-managing and auto-healing capabilitiesThe Atmos platformconsists of a set of redundant, distributed services that handle the underlyingdata discovery, data management, and data storage tasks. These services areable to restart themselves and rebuild objects when a storage disk iscorrupted.

Atmos is global scale storage on physical or virtual appliances. These are groupsof appliances that make a storage cloud that can be managed as a single system.Atmos is based on a unique set of data services with no limits on namespace or


16/79

Introduction


Deployment Guide

6

location, intelligent protection and efficiency services, web-based services, andmulti-tenancy.Users can access an Atmos object store globally with REST overHTTP or HTTP/S or locally over file services. Atmos canaddress challenges withstoring and managing vast amounts of unstructured content for custom orpackaged applications.

Figure 1 illustrates Atmos can optimize applications across the business.

Figure 1. Optimize applications across the business with AtmosCustom applications can leverage the Atmos Software Development Kit (SDK),which provides ways to integrate applications of nearly any language orframework, including:

CAS applications

Mobility capabilities, such as Windows or Linux desktop access to Atmosthrough the new Atmos GeoDrive, or iOS access through EMCspartnerOxygen Cloud

Cross-site content storage for content management applications such as EMCDocumentumor Microsoft SharePoint

Movement of inactive content, typically 80 percent of content, from Tier 1 NASto the cloud

Multisite storage of medical images

Atmos is managed as a single system over many sites andcan be delivered as a self-service experience to consumers.The EMC Atmos platform has two storage optionsthe Atmos physical hardware andAtmos VE.

The Atmos physical hardware includes Atmos nodes that run the Atmossoftware, and Disk Array Enclosures (DAEs) that provide storage. EMC has


17/79

Introduction


strengthened the Atmos scale-out storage platform by leveraging thecapabilities of the latest Intel(r) Xeon(r) processor 5600 for a more than 50%

performance improvement, at reduced power/TB for greater efficiency.Figure

2 on page18illustrates this storage option.

Atmos VE includes a virtualized environment that is based on VMware

vSphere. With Atmos VE, the Atmos nodes that run the Atmos software aremapped to virtual machines that are provisioned on a VMware vSphere

supported storage array such as EMC VNX series.Figure 3on page19illustrates this storage option.

This paper focuses on the configuration that uses Atmos VE on top of

vSphere-managed storage similar to site #4 inFigure 1on page16.

Atmos software is a tiered capacity-based software license. For Atmos VE, thecapacity is calculated based on the usable capacity provisioned to virtualizedenvironments. For example, if 30 TB of usable capacity is provisioned to ESXi, then 30TB of Atmos capacity license is required.

An Atmos capacity license includes all features offered by Atmos. There are no add-ons or modular software for Atmos.Figure 2on page18shows the physical hardwareof Atmos.


18/79

Introduction


Deployment Guide

8

Figure 2. Atmos physical hardware


19/79

Introduction


Figure 3. Atmos Virtual EditionDespite the physical differences between the Atmos physical hardware and theAtmos VE storage options, it is important to note that in both options the same Atmossoftware is used. As a consequence, object stores implemented using both storageoptions appear the same to end-users. Furthermore, even to Atmos systemadministrators, such objects stores, once installed and configured, can be similarly

managed using the web-based Atmos administration GUI as shown inFigure 4on

page20.


20/79

Introduction


Deployment Guide

0

Figure 4. Atmos administration GUIIntroduction to Atmos on VNX solution

The Atmos on VNX solution includes the framework and components for deploying anobject store on VNX platforms using the Atmos object technology and the VMwarevSphere virtualization software suite. The Atmos on VNX solution builds on thestrengths of its three core technologies: Atmos VE, VMware vSphere, and VNX.

Atmos on VNX solution topologyThe topology of the Atmos on VNX solution includes four layersstorage, vSpheremanaged servers, Atmos nodes, and the Atmos access/integration layer.


21/79

Introduction


Figure 5illustrates the topology of the Atmos on VNX solution.

Figure 5. Atmos on VNX solution topologyFollowing is a review of each layer in this topology:

VNX storageThis layer includes the VNX platform that is used to store theobject store. This solution is viable for both VNX platform for File and VNXplatform for Block. Because storage in this solution is presented to VMwarevSphere, any storage protocol that is supported by vSphere can be used. This

includes NFS, iSCSI, FC, and FCoE.

vSphere managed serversThrough a storage network, VNX storage objects(file systems for file-based storage and LUNs for block-based storage) arepresented to servers that are running the vSphere bare-metal hypervisorESXi. On ESXi, datastores (NFS for file based storage, and VMFS for block-based storage) are created on the storage objects that are presented fromVNX. These datastores are used to provision virtual machines that runs theAtmos software.


22/79

Introduction


Deployment Guide

2

Atmos nodesEach virtual machine that runs the Atmos software function asan Atmos node that holds a portion of the Atmos object store. Each Atmosnode is configured with virtual disks created on the datastores that reside onthe VNX storage objects. These virtual disks consists of the following two dataelements of an Atmos object store:

o DataData that is stored in the object store by the end-userso MetadataInformation that describes the data that is stored by the

end-users

Atmos access and integration layerEnd-users access the object storethrough this layer using REST-based and SOAP-based applications. Eventhough the object store is contained and managed by multiple Atmos nodes,end-users refer to the object store as a single entity. To access the integrationlayer, end-users rely on a public network.

This solution topology does not require a dedicated VNX server or vSphere resource.The VNX platform can be shared with other file-based or block-based applicationsand the servers can be used to run other non-Atmos virtual machines. In essence,

available resources in the private cloud can be used to build an object store usingthis solution.


23/79


Deployment Guide

Chapter 2 Setup andConfiguration


Introduction ..................................................................................... 24Atmos VE on VNX test environment ............................................................ 24Configure VNX storage ............................................................................... 27Configure network ..................................................................................... 32Configure storage network ......................................................................... 33Configure vSphere ..................................................................................... 35Configure Atmos Virtual Edition .................................................................. 39


24/79

Setup and Configuration


Deployment Guide

4

IntroductionThe configuration of the Atmos on VNX solution consists of three primary partsstorage configuration on the VNX layers, vSphere configuration on the virtualizationlayer, and Atmos VE configuration on the Atmos layer.

The architecture of this solution relies heavily on network resources to, not only

interconnect VNX, vSphere, and Atmos VE, but also to enable end-users to access theAtmos-based object store. Therefore, a key element in the configuration of thissolution is the network configuration, both the storage network and the end-userspublic network.

This chapter covers the setup and configuration of the following layers:

Configure VNX storage

Configure network (storage, private, and public)

Configure vSphere

Configure Atmos

Atmos VE on VNX test environmentThis section includes information on the test environment that was used to developthis solution. Use this configuration as a guideline to design the actual environmentfor this solution.

Different VNX platforms, servers, and network switches can be used as long as theyare supported by VMware vSphere and Atmos VE (where applicable). The AtmosVirtual Edition Best Practices Guide provides more information aboutAtmos VE.

The selected VNX platforms, servers, and network switches must also meet therequired object store capacity, and the network connectivity requirements that are

specified in this chapter.Figure 6on page25shows a high-level typical configurationof the Atmos on VNX solution that includes two sites.


25/79



Figure 6. Atmos on VNX high-level typical configurationThe following two sections provide details of the hardware and software resourcesthat were used to deploy this solution.


26/79



Deployment Guide

6

Hardware resourcesTable 1lists the hardware resources used to deploy this solution.

Table 1 Hardware resourcesDescription Quantity Use/setupEMC VNX5300:

2 Data Moversactive/passive

2 storage processorsactive/active

7 disk-array enclosures (DAEs) fullypopulated with 15 SAS 300 GB/15K spindles

Two VNX for File, VNX forBlock, and object

One for each location

Dell PowerEdge R810:

4 quad-core CPU

128 GB memory

1 GbE network interface card (NIC)virtualmachine network

2 dual-port 10 GbE FCoE CAN adaptersstorage network

Four ESXi hosts (NAS, iSCSI,FCoE)

A pair for each location

Cisco Nexus 5010:

Twenty six 10 GbE FCoE ports

Four 10 GbE FCoE convergednetwork switches

A pair for each location

Software resourcesTable 2lists the software components used to deploy this solution.

Table 2 Software resourcesDescription Minimum versionVNX Operating Environment (OE)

VNX OE for File

VNX OE for Block

7.0.13.0

05.31.000.5.006

VMware vSphere:

ESXi Installable

vCenter Server

4.1 Update 1

4.1 Update 1

Operating system Windows Server 2008 SP2

EMC VSI for VMware vSphere UnifiedStorage Management

4.1

EMC Atmos Virtual Edition (VE) 1.4.1 Maintenance Release (1.4.1.59250)


27/79



Configure VNX storageThis section explains the NFS and FC storage configuration used by the Atmos VEnodes.

Note This solution supports iSCSI storage. However, the iSCSI storageconfiguration is not included in this document.

To ensure optimal performance and improve troubleshooting, the following principleswere followed in the storage configuration for the Atmos on VNX solution:

Separate object and non-object data

Separate between boot and data. With an Atmos object store, data includesboth user data and metadata of the Atmos nodes

Leverage VNX to self-optimize the virtual storage pools

Standardize on RAID 5 data protection to simplify configuration, and balancebetween optimal performance and high-capacity needs

Use the EMC VSI for VMware vSphere: Unified Storage Management (USM)feature to easily provision the storage and present it to the ESXi servers

Note To simplify the configuration procedure, place all ESXi servers fromeach site in a single folder or cluster in the vCenter Server inventory. Thisenables a much smoother operation of the USM feature to provision the VNXstorage and to present it to the servers in a single operation.

In this section, two examples are provided to illustrate the typical storageconfiguration of this solution (with block-based and file-based storage). In both theexamples, storage was configured to contain four Atmos nodes. Each Atmos nodeconsists of 4 TB of data and 1 TB of metadata (total object store size is 20 TB). Inthese examples, the ratio of data to metadata was set to 4:1

Determine this ratio based on the applications the end-users use with the objectstore and the amount of metadata these applications write for each object. Atmossupports a range of ratios. The Atmos Virtual Edition Best Practices Guide availableon the EMC Online Support website provides more information on the data andmetadata ratio.

In addition, allocate storage to a separate storage pool to accommodate non-objectapplication servers.

Figure 7on page28shows the storage configuration with file-based storage.

Figure 8on page30shows the storage configuration with block-based storage.

For each device type in VNX, allocate an adequate number of hot spare drives toreplace any failed drives. In this example, for the 600 GB SAS drives, three hot sparedrives were created in the storage layout.

This section provides more details on each storage configuration.


28/79



Deployment Guide

8

Configure file-based storage (NFS) for Atmos VEThis section describes the procedures required to configure the file-based storage foran Atmos VE based storage on the test environment.

To configure storage configuration for Atmos VE, complete the following steps:

1. Configure storage for the boot disk of the Atmos VE nodes.2. Configure storage for data and metadata disks for the Atmos VE nodes.3. Configure storage for non-object data.

The following four sections describe each step of the storage configuration on thelocal VNX for the local RMG Atmos VE nodes. Use similar configuration on the remoteVNX for the remote Resource Management Group (RMG) Atmos VE nodes.

Storage layoutfile-based storage (NFS)Figure 7shows the file-based storage layout for the Atmos on VNX solution on thetested storage configuration.

Figure 7. File-based storage layout for Atmos VEThe Unisphere Online Help available on Unisphere provides the procedure to createstorage pools for files and hot spares for VNX.

Configure storageboot disks of Atmos VE nodesCreate a storage pool with RAID 5 data protection to contain the boot disks of theAtmos VE nodes.


29/79



Create two file systems on this storage pool with the USM feature to contain the boot

virtual disks of the local RMG Atmos VE nodes (the sectionConfigure Atmos Virtual

Editionon page39provides more information).

The boot disks of the Atmos VE nodes that are deployed on the VNX storage must bedistributed evenly between these two file systems. This is based on the best practicefor VMware multipathing with VNX file systems. The Using VNX storage with VMware

vSphere

TechBook available on the EMC Online Support website provides moreinformation.

Figure 7on page28shows a pair of 200 GB file systems was created in the storagepool 2. Storage pool 2 was created to contain the boot disks of the Atmos VE nodes.

Configure storagedata and metadata disks of Atmos VECreate a storage pool with RAID 5 data protection. This storage pool contains the dataand metadata disks on the Atmos VE nodes.

Create two file systems on this storage pool with the USM feature to contain the dataand metadata virtual disks of the local RMG Atmos VE nodes.

The data and metadata disks of the Atmos VE nodes on the VNX storage must beevenly distributed among these two filesystems. This is based on the best practice forVMware multipathing with VNX. The Using VNX storage with VMware vSphereTechBook provides more information.

Figure 7on page28shows the ratio of data disks to metadata disks in the 20 TBobject store was 4:1. Therefore, two 10 TB file systems were created in storage pool 1for data and metadata disks on four Atmos VE nodes. Each file system contained four2 TB data disks and two 1 TB metadata disks.

Note Based on the required object store size, create additional file systemsfor the data and metadata disks when the object store exceeds the maximum

VNX file system size. In this case, it is important to create an even number offile system to optimize load distribution.

Configure storage for application serversTypically, the VNX storage must not be used exclusively for the Atmos object store.Allocate a part of the storage for non-object data. For this data, create one (or more)separate storage pools.

Figure 7on page28shows storage pool 3 was created with RAID 5 data protectionfor the non-object data. This storage pool was created with 20 disks and used for theapplication servers.

The following file systems were created for application servers on the storage pools:

Two file systems for application server boot disks

Two file systems for application server data disks

To complete the configuration, three hot spares were created in the storage layout.


30/79



Deployment Guide

0

Configure block-based storage (FC) for Atmos VEThis section describes the procedures required to configure the block-based storagefor Atmos VE on the test environment.

To configure storage configuration for Atmos VE, complete the following steps:

1. Configure storage for boot disk of the Atmos VE nodes2. Configure storage for data and metadata disks for the Atmos VE nodes3. Configure storage for non-object data

The following four sections describe each step of the storage configuration for thelocal RMG Atmos VE nodes on the local VNX. Use a similar configuration for theremote RMG Atmos VE nodes on the remote VNX.

Storage layoutblock-based storage (FC)Figure 8 shows the block-based storage layout for the Atmos on VNX solution on a

tested storage configuration.

Figure 8. Block-based storage layout for Atmos VEConfigure storageboot disk of Atmos VE

Create a storage pool with RAID 5 data protection. This storage pool contains the bootdisks of the Atmos VE nodes.

Use the USM feature to create one VMFS data store on this storage pool. This datastore contains the boot virtual disks of the local RMG Atmos VE nodes (the section

Configure Atmos Virtual Editionon page39provides more information).


31/79



Figure 8shows one 200 GB VMFS data store was created in storage pool 2. Storagepool 2 was created to contain the boot disks of the local RMG Atmos VE nodes.

Configure storagedata and metadata disks of Atmos VECreate a storage pool with RAID 5 data protection. This storage pool contains the datadisks and metadata disks of the Atmos VE nodes.

Typically, Atmos data and metadata disks require VMFS datastores that are muchlarger than 2 TB. In VMware vSphere 4, create the data store across multiple LUNs onthe VNX platform that are each up to 2 TB minus 512 bytes in size.

To simplify and streamline the configuration, use a unified LUN size on the data andmetadata storage pool. Use the Create LUNwizard in Unisphere to create LUNs byaccording to the overall capacity required for the data and metadata disks. Presentthe LUNs to the two ESXi servers.

Use the vSphere Client to create two VMFS datastores on the LUNs to hold the dataand metadata virtual disks of the local RMG Atmos VE nodes. Create each data storeon half of the LUNs that are created.

The data and metadata disks of these nodes must be evenly distributed between thetwo VMFS datastores. Tests have shown that creating two VMFS datastores for dataand metadata virtual disks produces better disk utilization and performance than aconfiguration with just a single data store for data and metadata virtual disks.

Figure 8on page30shows the ratio of data disks to metadata disks in the 20 TBobject store was 4:1. Each of the four nodes of the local RMG was configured with 4TB of data and 1 TB of metadata. Therefore, two 10 TB VMFS datastores were createdin storage pool 1 for data and metadata disks of the local RMG Atmos VE nodes.

In this example, each data store was created across five 2 TB LUNs on a VNX (5 VMFSextents) platform. Each data store contains four 2 TB data disks and two 1 TB

metadata disks.

Configure storage for application serversTypically, the VNX storage is not used exclusively for the Atmos object store.Therefore, allocate a part of the storage for non-object data. For this data, create oneor more separate storage pools.

Figure 8on page30 showsstorage pool 3 was created with RAID 5 data protectionfor the non-object data. This storage pool was created with 20 disks and was used forapplication servers.

The following two datastores were created for application servers on this storage

pool:One datastore for boot disks of application servers

One datastore for data disks of application servers


32/79



Deployment Guide

2

Configure networkThis section explains how to configure the network for the Atmos VE testenvironment.

For the Atmos on VNX solution, the network configuration consists of the followingtwo parts:

Private and public networksConnects the Atmos VE nodes and theend-users to the Atmos object store.

Storage networkConnects the VNX storage to the ESXi servers. The Atmos VEnodes use this network to access their configured virtual disks.

This section provides more information on the configuration of each of these parts.

Configure private and public networksFor the Atmos on VNX solution, configure both private and public networks

The Atmos VE nodes are used to connect to the private network, which is a separate

network. Even though it is technically feasible to configure the private network as aninternal virtual network (using an internal vSwitch), allocate physical networkresources for the private network. This allows seamlessly migration of an Atmos VE

node from the servers using VMware vMotion (the sectionMigrate Atmos nodes with

vMotion and Storage vMotionon page48provides more information).

To configure the private network for the Atmos on VNX solution, complete thefollowing steps:

1. Log in to the vSphere Client, and then select the server from the Inventorypanel.

2. Click the Configurationtab, and then click Networkingfrom the left panel.

Figure 9. vSphere Client3. Click the Add Networking wizard to configure the virtual machine network for

the private network.

4. Complete all the steps in the Add Networking wizard to configure the privatenetwork virtual machine network.

5. Repeat the steps from 1 to 4 to configure the private network on the secondlocal ESXi host.


33/79



Configure the Atmos nodes with their first virtual NIC connected to the private

network as shown inFigure 10.

Figure 10. Private network configuredThe public network is an external network. End-users use this network to access theobject store. This network must provide high-availability and redundancy. A networkload balancer (hardware or software) is required in the Atmos public networkconfiguration. This load balancer redirects end-users are from Atmos VE nodes thatare inactive to the alternative Atmos VE nodes (local and remote) that are active andhave access to copies of objects that are inaccessible through the inactive node.

For more information on the configuration of the public network with Atmos, contactan EMC representative for Atmos.

Similar to the private network, use the Add Networkingwizard to configure the publicnetwork in the ESXi hosts. Configure the Atmos nodes with their second virtual NICconnected to the public network.

The EMC Atmos Virtual EditionBest Practices Guide available on the EMC OnlineSupport website provides more information on how to set up the private and publicnetworks of the Atmos VE nodes.

Configure storage networkFor the Atmos on VNX solution, configure a storage network. The Atmos VE nodes usethis storage network to access the configured virtual disks on the VNX storage.Hence, it is important for this network to be highly available with more than one I/Opaths and with no single point of failure.

This section provides details on the configuration of this storage network for thefile-based storage on NFS and for block-based storage on FC.

Configure network for file-based storage (NFS)For the storage network, a GbE network is required. Because this solution is typically

deployed with non-object workloads, a 10 GbE FCoE converged network isrecommended.

Based on the best practice for VMware multipathing with VNX file storage, the storagenetwork must include two I/O paths on two network switches between the ESXi hostsand the VNX Data Movers. Use a combination of NIC Teaming on ESXi, LinkAggregation on VNX, and multichasis Link Aggregation on the switches for networkmultipathing between the two I/O paths. Use two interfaces to create an LACP bond


34/79



Deployment Guide

4

on the Data Mover. Also, configure NIC Teaming on the ESXi side. Enable trunking onall the storage network connections.

The Using VNX storage with VMware vSphereTechBook available on the EMC OnlineSupport website provides more information on the recommended configuration forVMware vSphere network multipathing with VNX file-based storage.

Figure 11shows a storage network configuration for NFS storage using a 10 GbE FCoEconverged network. In this example, four 10 GbE FCoE network connections wereused to connect the two servers to the two Cisco Nexus network switches. Similarly,four other connections were used to connect the Data Movers to the two Cisco Nexusnetwork switches.

Figure 11. Storage network diagramNFSConfigure network for block-based storage (FC)

For the storage network, FC network is required. Because this solution is typicallydeployed with non-object workloads, a 10 GbE FCoE converged network isrecommended.

Based on the best practice for VMware multipathing with VNX platform for Block, thestorage network must include two FC I/O paths on two FC switches between the ESXihosts and the VNX Data Movers. Use the VMware Native Multipathing (NMP) to

multipath between the two I/O paths. When FCoE is used for the storage network, donot enable trunking on all the storage network connections.

The Using VNX storage with VMware vSphereTechBook available on the EMC OnlineSupport website provides more information on the recommended configuration forVMware vSphere network multipathing with VNX platform for Block.


35/79



Figure 12shows the storage network configuration for FC storage with a 10 GbE FCoEconverged network. In this example, the following FCoE network connections wereused:

Each ESXi host had two FCoE connections, one to each of the Cisco Nexus5010 network switches.

Each VNX Storage Processor had two FCoE connections, one to each of theCisco Nexus 5010 network switch.

Figure 12. Network diagramFCoEConfigure vSphere

This section explains the configuration changes on the default configuration ofVMware vSphere 4.1 for the Atmos VE software. Use the USM feature to provision the

required datastores as explained in the sectionConfigure VNX storageon page27.Use Unisphere and vSphere Client to provision the data and metadata datastores onVNX platform for Block.


36/79



Deployment Guide

6

Configure virtual machines for Atmos VE nodesVirtual machines are used to create the Atmos VE nodes. These virtual machines wereconfigured to run the Atmos software. For each Atmos VE node, virtual disks werecreated on the following two different datastores:

Boot datastoreBoot disk of 20 GB

Data and metadata datastoreOne metadata disk and one (or more) datadisks

Table 3lists the configuration of an Atmos VE node virtual machine used to deploythe Atmos on VNX solution.

Table 3 Virtual machine configurationVirtual machine component SettingOS Atmos + Linux 2.6x Kernel (64-bit)

CPU 2

Memory 8 GB or 12 GB

Network adapter 1 Atmos Private Network

Network adapter 2 Virtual machine network

Disk1 20 GBboot

Disk2 1 TBmetadata operations and logging details

Disk3 2 TBdata

Disk4 2 TBdata

Deploy two Atmos nodes on each ESXi server considering the high-availability factor.Deploy all the virtual machines on datastores that are provisioned on VNX with theUSM feature.

Figure 13on page37 shows the virtual machine properties of the Atmos node.


37/79



Figure 13. Atmos node propertiesPost-configuration steps for the Atmos VE nodes

Perform the post-configuration steps after the virtual machines are configured andbefore the Atmos software is installed on them:

For each site, distribute the Atmos VE nodes evenly between the servers (twonodes on each server). It is possible to use DRS to distribute the Atmos VEnodes. However, it is required to define DRS rules to set the server for eachnode.

Optionally, when using DRS, add the nodes to a DRS resource pool in the DRScluster while adding other virtual machines to other resource pools in the

cluster as shown inFigure 14on page38.This enables to control the virtualresources that are allocated to object and non-object workloads in thecluster.


38/79



Deployment Guide

8

Figure 14. Atmos VE nodes with DRS resources poolsConfigure startup shutdown options for the nodes as shown inFigure 15.Thisenables the nodes to start automatically soon after the hosts are powered ONavoiding the need to manually power ON the Atmos VE nodes.

Figure 15. Automatic startup for Atmos VE nodesThe ESXi Configuration Guideand the Resource Management guidein the VMwarevSphere documentation website provides more details on these steps.


39/79



Configure Atmos Virtual EditionThis section includes all the configuration changes that were made to the defaultconfiguration of Atmos VE 1.4.1.

The EMC Atmos Virtual EditionBest Practice Guide available on the EMC OnlineSupport website provides information to install and configure Atmos VE nodes.

TheEMC Atmos

Administrators Guideavailable on the EMC Online Support websiteprovides information to configure the Atmos object store.

Note With Atmos VE, there is no need to manually install the VMware toolspackage on nodes after the installation of the Atmos VE software. This isbecause the VMware tools package is included in the Atmos software that isinstalled.

An Atmos VE configuration typically consists of two RMGs: RMG1 represents a localobject store and RMG2 represents a remote object store. Each RMG must contain fourAtmos VE nodes (two on each ESXi server) as shown inFigure 16.

Figure 16. RMG listOne Tenant (t1) was configured as shown inFigure 17.

Figure 17. Tenant listConfigure multisite object configuration

To configure the multisite object, define the object-replication policy. The replicationis done natively in Atmos.

Note Use VNX-level replication such as Replicator, MirrorView, orRecoverPoint only for non-object data.

With the Atmos on VNX solution, create two copies for every new objectone localcopy (synchronously) and one remote copy (asynchronously).


40/79



Deployment Guide

0

To create an object-replication policy, complete the following steps for each tenant inthe object store:

1. Log in to the Atmos node as tenant admin. The Tenant Basic Informationwindow appears.

Figure 18. Tenant Basic Information window2. In the Policy Specificationarea, click default.3. In the Policy specificationarea, complete the following steps:

a. In theReplica Typelist box, select sync and in the Locationlist box,retain same and clientCreateLocto create the first replica (Replica 1)within the local RMG.

b. In the Replica Typelist box, select asyncand in the Locationlist box,select the location to other thanand clientCreateLocto create thesecond replica (Replica 2) within the remote RMG.c. If required, select Enable Retentionand Enable Deletion to set retentionand expiration for policy.


41/79



Figure 19. Policy Specification windowNote Because the object store is deployed on VNX platform, whichis RAID protected, it is not necessary to enable Erasure Codes as partof the object-replication policy. It is not recommended to use ErasureCodes with Atmos VE.


42/79



Deployment Guide

2


43/79


Deployment Guide

Chapter 3 Monitor andManagement


Introduction ..................................................................................... 44EMC Unisphere ..................................................................................... 44Atmos administration GUI .......................................................................... 44Ganglia Atmos grid report .......................................................................... 46EMC VSI for VMware vSphere Unified Storage Management ........................ 46Migrate Atmos VE nodes ............................................................................ 47


44/79

Monitor and Management


Deployment Guide

4

IntroductionThis chapter describes the applications used in this solution. It also explains theprocedures used to monitor and manage the Atmos on VNX solution.

The applications used are:

EMC UnisphereAtmos administration GUI and Ganglia

USM feature

EMC UnisphereUnisphere is a GUI that is used to manage and monitor the VNX platform used for theAtmos object store. Unisphere is used to configure storage pools on the VNXplatform. In these storage pools, file systems and LUNs can be created for the Atmoson VNX solution.

To configure the storage pool for file, select Storage >Storage configuration>StoragePool for File.To configure the storage pool for block, select Storage >Storage Configuration>Storage Pools.

Figure 20. Configure storage poolAtmos administration GUI

The URL that is used to access the Atmos administration GUI is:

https://hostname/mgmt_login

Type the IP address of the Atmos node where hostname appears in the sample.


45/79



Use the Atmos administration GUI to perform various object store configuration taskssuch as:

1. Configure the Atmos nodes2. Add RMGs3. Create tenants4. Define policies for tenants5. Add nodes to tenants6. Monitor the Atmos nodes

To add an RMG, click Add RMGin the System Dashboardwindow.

Figure 21. Add RMGThe Atmos administration GUI provides various options to monitor the performance ofthe activities within the Atmos VE nodes.

To provide a detailed status of activities within Atmos, select System Management >System Dashboard.The System Summarywindow appears.

Figure 22. System Summary window


46/79



Deployment Guide

6

Ganglia Atmos grid reportThe URL that is used to access the Ganglia Atmos Grid Report ishttps://hostname/ganglia.

This report helps to monitor the status of RMGs. To monitor the ongoing read andwrite activity on the RMG nodes, click the TPS activity. The specific TPS graph

appears.

Figure 23. Ganglia Atmos grid reportEMC VSI for VMware vSphere Unified Storage Management

EMC VSI for VMware vSphere is a vSphere Client plug-in that is used to manage andmonitor EMC storage connected to the VMware vSphere environments. Unified

Storage Management (USM) is a VSI feature available to manage VNX and VNXeplatforms. Administrators can use USM to perform the following management taskson VNX storage in the Atmos on VNX solution: :

Provision VNX platform for File and VNX platform for Block for the Atmosobject store from the vSphere Client

Compress and decompress Atmos nodes that are provisioned on VNXplatform for File.

EMC VSI for VMware vSphere is available for download from the EMC Online Supportwebsite. After this plug-in is installed, the VSI icon appears on the vSphere Client

Home page as shown inFigure 24.

Figure 24. VSI icon


47/79



To provision storage using VSI, right-click the ESXi server (or the cluster that resideson the hosts), and then select EMC >Unified Storage> Provision Storage.

Figure 25. Provision storageNote To compress the Atmos nodes, select EMC >Unified Storage>Compress.

The amount of time to provision storage depends on the size of the Atmos node datastore. The amount of time to compress the Atmos node depends on the amount ofdata available on the virtual disk associated with the Atmos node.

The EMC VSI for VMware vSphere: Unified Storage ManagementProduct Guideavailable on the EMC Online Support website provides information on how to install

and configure the USM feature.

Migrate Atmos VE nodesThis section provides details about the Atmos node migration using VMwarevMotionand VMware Storage vMotion.

Migration is required to perform maintenance activities on the following items:


48/79



Deployment Guide

8

Server The server on which the Atmos node is running.

Note In case of code upgrade, the Atmos node virtual machine ismigrated to another server with vMotion.

VNX storage objectThe VNX storage object on which the node is provisioned(file system or LUN).

Note In this case, the Atmos node virtual machine is migrated toanother storage location with Storage vMotion.

Migrate Atmos nodes with vMotion and Storage vMotionTo migrate Atmos nodes with vMotion or Storage vMotion, complete the followingsteps:

1. Configure the Atmos private network on an external Standard vSwitch that isconnected to both ESXi servers.

2. Remove any existing DRS VM Affinity rules for the Atmos node virtualmachine.

3. Right-click the Atmos node virtual machine, then select Migrateto launch theMigrate Virtual Machine Wizard.

4. Select Change Hostto use vMotion to migrate the Atmos VE node from thesource ESXi host to the destination host.

5. Select Change Datastoreto use Storage vMotion to migrate the Atmos VEnode from its existing datastores to destination datastores.

ObservationsTesting showed that there is no operational impact on Atmos nodes because ofvMotion. For large objects, the create latency showed an improvement whencompared to the baseline testing. However, for small objects there was a smalldegradation in TPS for the create operations. vMotion takes only 28 seconds to finish

this operation.

Migration with Storage vMotion, the operation took much more time to complete dueto the large size of a typical Atmos object store.

Figure 26 on page49 shows the vMotion performance of local object store betweenbaseline operations without vMotion.


49/79



Figure 26. REST TPS without vMotionFigure 27shows the vMotion performance of local object store with vMotion.

Figure 27. REST TPS with vMotion


50/79



Deployment Guide

0

Figure 28shows the performance of Atmos nodes between baseline operations andvMotion for large objects.

Figure 28. Performance comparison for large objects


51/79



Figure 29shows the performance of Atmos nodes between baseline operations andvMotion for small objects.

Figure 29. Performance comparison for small objectsConclusions

The performance of Atmos nodes was not affected due to vMotion operations.

For both vMotion and Storage vMotion based migrations, the REST workload

continued execution without error while the migration was in progress.

For Storage vMotion based migration, given the typical large size of an Atmos objectstore, the migration took much more time. Therefore, consider an offline method toshorten the time required to migrate the Atmos nodes.


52/79



Deployment Guide

2


53/79


Deployment Guide

Chapter 4 Atmos on VNXPerformance


Introduction ..................................................................................... 54Atmos on VNX performancefile and block ................................................ 54Atmos on VNX performanceobject and non-object.................................... 56


54/79

Atmos on VNX Performance


Deployment Guide

4

IntroductionThis chapter reviews some of the performance aspects of the Atmos on VNX solution.This chapter covers the following topics:

1. Comparison of performance between file- based and block-baseddeployments of this solution.

2. Performance study of mixed environment that includes both object andnon-object workloads on the same VNX platform.

Atmos on VNX performancefile and blockThis section gives a detailed overview of the performance of Atmos VE on NFS and FCconfigurations. The performance of Atmos VE when deployed on FC LUNs and NFS filesystems was compared.

Test methodThe storage configurations for both the test cases were similar. ThesectionStorage

layout

file-based storage (NFS)on page28and sectionStorage layout

block-basedstorage (FC)on page30provide more information on storage provisioning for NFS

and FC,respectively.

For the NFS configuration, NFS file systems were created on the storage pools andexported to the hosts with the USM feature. The Atmos VE nodes were deployed onthese NFS datastores.

For the FC configuration, FC LUNs were provisioned on the storage pools andpresented to the hosts as VMFS volumes with the USM feature. The Atmos VE nodeswere installed on these VMFS datastores.

The performance of the Atmos VE nodes was measured by running REST-basedworkloads that was simulated by the Grinder tool. Grinder was configured to generatecreate and read REST operations for large objects (16 MB) and small objects (8 KB).

Appendix A Using Grinder to Generate REST Workload on Atmosprovides moreinformation on theGrinder tool and how it is used to test the REST workload withAtmos.

Result analysisTesting showed that the performance of the Atmos VE nodes was better whendeployed on VMFS datastores than on NFS datastores.


55/79



Figure 30shows the comparison of performance of local object store (in TPS) for NFSand FCoE configurations while the REST workload was in progress.

Figure 30. Performance of object store for NFS and FCoE configurationsFigure 31shows the comparison of performance of small objects (REST workload) forNFS and FCoE configurations as reported by the Grinder tool.

Figure 31. Performance of small objects for NFS and FCoE configurationsFor small objects, the performance was better on VMFS datastores than on the NFSdatastores. Also, the latency for FCoE was much smaller than the latency of NFS.


56/79



Deployment Guide

6

Figure 32shows the comparison of performance of large objects that use RESTworkload for NFS and FCoE configurations.

Figure 32. Performance of large objects for NFS and FCoE configurationsThe performance of the large objects was the same for NFS datastores and VMFSdatastores. The response time was also the same for both the configurations.

ConclusionThe performance of Atmos nodes was better on FC LUNs when the small-objectworkload was executed.

With large object workload, the performance of the Atmos nodes was similar on bothNFS and VMFS datastores.

Atmos on VNX performanceobject and non-objectThis section provides a detailed overview of the performance of Atmos VE with mixedworkload that includes both object and non-object workload.

The performance of the Atmos VE nodes is measured when application servers thatrun the non-object workload are running with the Atmos VE node. In thisconfiguration, the servers and the VNX platform are subject to both object andnon-object workload.

Test methodThe performance of the Atmos VE nodes was measured when two and four non-objectapplication servers were deployed.

In this case also, the performance of the Atmos VE nodes was measured by runningREST-based workloads that was simulated by the Grinder tool. Grinder wasconfigured to generate create and read REST operations for large objects (16 MB) andsmall objects (8 KB). This test was performed for both NFS and FC storage.

Appendix A Using Grinder to Generate REST Workload on Atmos providesdetails ofthe Grinder tool and how it is used to test the REST workload with Atmos.


57/79



The sectionConfigure VNX storage on page27provides details on how to provisionstorage and deploy Atmos nodes on NFS and FC.

Table 4explains the configuration of the non-object application servers.

Table 4 Application server configuration specificationsVirtual machine component SettingOS Windows Server 2008 SP2

CPU 1

Memory 1 GB

Disk1 10 GB OS

Disk2 100 GB data

Other specifications Antivirus (AV) updated50 GB data on data disk

The application servers were installed on a storage pool that is different from thestorage pools used by the Atmos nodes. Based on the best practice, the boot andvirtual disks were placed on different datastores. The IOmeter workload (mix-8k-50%read-random), OLTP, was generated on these application servers.

Result analysisAfter several iterations, showed the performance of the Atmos VE nodes with non-object workload on the VNX was better when deployed on VMFS datastores.

The performance of the object stores was almost the same for both NFS and FCiterations.


58/79



Deployment Guide

8

Figure 33shows the comparison of performance (in TPS) of local object store for NFSand FCoE configurations.

Figure 33. Performance of object store for NFS and FCoE configurationsThe performance of the read operations was better on VMFS datastores than on theNFS datastores for small objects. Even the latency for FC was much smaller than thelatency for NFS. However, for create operations, the performance degraded for VMFSdatastores as the number of application servers increased. The performance of the

object stores for both local and remote was the same.


59/79



Figure 34shows the comparison of performance of small objects for NFS and FCoE.

Figure 34. Small objects in NFS and FC datastoresREST and OLTP workload


60/79



Deployment Guide

0

Figure 35shows the comparison of performance of large objects for NFS and FCoE.

Figure 35. Large objects in NFS and FC datastoresREST and OLTP workloadThe performance of the large objects was the same for NFS datastores and VMFSdatastores. The response time was also same for both the configurations. The

response time of the object stores for both local and remote was also same.ConclusionThe performance of Atmos nodes was better on FC LUNs, when read operation for thesmall-object workload was run. However, the performance degraded for createoperations for small-object workload as the number of application servers increased.

With large-object workload, the performance of the Atmos nodes was similar on bothNFS and VMFS datastores.


61/79


Deployment Guide

Chapter 5 Storage Efficiency


Introduction ..................................................................................... 62Storage efficiency with thin provisioning .................................................... 62Storage efficiency with compression .......................................................... 68


62/79

Storage Efficiency


Deployment Guide

2

IntroductionThis chapter describes the storage efficiency considerations when the Atmos on VNXsolution is deployed. This chapter explains how to leverage VNX storage efficiencytechnologies to improve the overall storage efficiency and reduce the storage costsfor this solution. The focus is on thin provisioning and compression technologies ofVNX platform for File and VNX platform for Block.

Storage efficiency with thin provisioningThis section explains how to use the VNX thin provisioning technology to enhance thestorage efficiency of the Atmos on VNX solution. Thin provisioning is a method tooptimize the utilization of available storage. It allocates storage based on thedemand instead of allocating the entire storage at the beginning. Even in Atmosunified system, storage efficiency or storage savings was achieved by deploying thethin provisioned LUNs and file systems.

Performance of Atmos unified system was tested on both NFS and FC configurations.However, the deployment process for both the configurations was different.

Configure thin provisioning on file-based object storeWhen file systems are created with USM feature, thin provisioning is enabled bydefault. The EMC VSI for VMware vSphere: Unified Storage Management Productguide available on the EMC Online Support website provides information on how tomodify the default configuration for thin provisioning with the USM feature and howto disable the thin provisioning feature.

After the thin file systems and corresponding NFS datastores were created by theUSM feature,-the Atmos nodes were deployed on the thin file systems.

Note In this solution, only Atmos data and metadata disks were deployedon the thin file systems. Atmos boot disks were deployed on thickprovisioned file systems.

Configure thin provisioning on block-based object storeTo create thin LUNs, install Compression Enabler and Thin Provision Enabler on VNX.

In the Unisphere Create LUNswizard, select Thinto create thin LUNs as shown inFigure 36.

Figure 36. Thin provisioned FC


63/79

Storage Efficiency


Test methodAfter Atmos nodes were installed on the thin LUNs and the Grinder script wasexecuted, tests were conducted to measure the performance of the object storebecause it was deployed on thin provisioned storage. Also, the size of the LUNs wascompared to find the difference. Similar iterations were performed for file-basedobject stores also.

Result analysisFor both NFS and FC storage, 93-95 percent initial storage savings were achieved.

Table 5shows the details of space allocations for NFS configurations.

Table 5 Thin provisionedNFSAtmos data file system Atmos metadata filesystem

Max size 8 TB 2 TB

Initial size 20 GB 10 GB

Size after Atmos nodeinstallation 370 GB 10 GB

Size after 5thiteration 550 GB 59 GB


Table 6shows the details of space allocations for FC configurations.

Table 6 Thin provisionedFCAtmos data LUN Atmos metadata LUN

Max size 16 TB 4 TB

Size after Atmos nodeinstallation

1040 GB 101 GB


Size after 3rditeration 1560 GB 101 GB

The initial size of the datastores must be 5 percent of the storage to avoid initial

storage expansion as shown inTable 5andTable 6.The initial capacity of metadatadepends on the type of application that is running and the amount of metadata that

is expected to write.

It was also observed that the data file systems and LUNs expanded after each Grindertest run. No performance change or fluctuations were noted at the object stores for allthe test iterations.


64/79

Storage Efficiency


Deployment Guide

4

Figure 37shows the performance of the object stores for thick provisioned (baseline)and thin provisioned scenarios.

Figure 37. Performance of object storethin provisioned NFSThere was no significant difference in performance of the object stores.

Figure 38shows the performance of the object stores for thin provisioned scenariosof FC and NFS.

Figure 38. Performance of object storethin provisioned FCThere was not a significant difference in performance of the object stores for NFS andFC configurations.


65/79

Storage Efficiency


Figure 39 shows the comparison of performance of large objects on FCoE.

Figure 39. Large objectsbaseline compared with thin provisioned FCoE


66/79

Storage Efficiency


Deployment Guide

6

Figure 40shows the comparison of performance of large objects on NFS.

Figure 40. Large objectsbaseline compared with thin provisioned NFSWhen the performance of thin provisioned Atmos nodes was compared with theperformance of thick provisioned (baseline) Atmos nodes, no difference wasobserved. Hence, it is better to provision for thin storage to achieve storage savingswithout affecting the performance.

Figure 41shows the performance comparison of small objects for FCoE.

Figure 41. Small objectsbaseline compared with thin provisioned FCoE


67/79

Storage Efficiency


Figure 42shows the performance comparison of small objects for NFS.

Figure 42. Small Objectsbaseline compared with thin provision NFSFigure 43shows the performance of thin provisioned NFS and FCoE for small objects.

Figure 43. Small objectsthin provisioned NFS compared with FCoE


68/79

Storage Efficiency


Deployment Guide

8

Figure 44shows the performance of thin provisioned NFS and FCoE for large objects.

Figure 44. Large objectsthin provisioned NFS compared with FCoEThe performance is the same for thin provisioned FC and NFS Atmos nodes for largeobjects. However, for small objects, the behavior is different. The thin provisioned FCAtmos nodes performed better than the thin provisioned NFS Atmos nodes for createoperations. The thin provisioned NFS Atmos nodes performed better than thinprovisioned FC Atmos nodes for read operations.

ConclusionThe storage savings of 93-95 percent was achieved for thin provisioned NFS and FCconfigured Atmos unified systems.

Storage efficiency with compressionThis section explains the effect of compression on VNX platform for File and VNXplatform for Block.

Configure compression on file-based object storeTo enable compression on file systems, complete the following steps:

1. Compress the virtual machines from the VMware vSphere Client.2. Install the USM feature.3. Right-click the selected virtual machine, and select EMC >Unified Storage>Compress. The compression status appears in the vSphere Client.


69/79

Storage Efficiency


Figure 45. Compression file systemNote If the virtual machine is a multidisk virtual machine and thedisks are on different file systems, the feature first enablescompression and then compresses the disks on each file systems.

4. To view the compression ratio after compression is complete, right-click thecompressed virtual machine, then select EMC >Unified Storage> Properties.The Propertiesdialog box appears.


70/79

Storage Efficiency


Deployment Guide

0

Figure 46. Compression ratioConfigure compression on block-based object storeTo enable compression, install Compression Enabler and Thin Provisioning Enabler

on the VNX platform.

To configure compression on block-based object store, complete the following stepsin Unisphere:

1. Right-click the storage pool, and then click Create LUN. The Create LUNdialogbox appears.


71/79

Storage Efficiency


Figure 47. Create LUN2. Select all the fields, and then click Apply. The LUN is created.3. Right-click the created LUN, and then click Properties. The LUN Properties

dialog box appears.

4. Click the Compressiontab.5. Retain the default compression options.


72/79

Storage Efficiency


Deployment Guide

2

Figure 48. LUN Propertiescompression6. Click OKto submit a compression operation for this LUN.7. Click the compressed LUNs summary wizard to monitor all active LUN

compression operations.

Documents

h8281 Atmos Ve Vnx Deployment