Datera VMware vSphere Deployment Guide...4.3 vSphere Networking Best Practices for Datera Deployment • Have a dedicated VLAN for storage traffic representing the Datera storage access

DEPLOYMENT GUIDEDATERA VMWARE VSPHERE

1GET A FREE CONSULTATION. Contact Us | Visit datera.io | Email [email protected] ©2020 Datera, Inc. All Rights Reserved. Datera is a trademark of Datera, Inc. All other trademarks belong to their respective owners. Date: July 2020

Datera VMware vSphere Deployment Guide

1. Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Virtual Infrastructure Storage Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

3. Datera Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.1 Dynamically Adaptable Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.2 vCenter Plugin for Ease of Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.3 Fully Programmable Software Defined Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

4. Network Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34.1 Datera Access Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44.2 Summary of Networking Steps for Setting up iSCSI Interfaces in vSphere . . . . . . . . . . . . . . . . . . . . . . . . 54.3 vSphere Networking Best Practices for Datera Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

5. Setting up the Software iSCSI Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65.1 Network Configuration for iSCSI using standard switch (vSwitch switch). . . . . . . . . . . . . . . . . . . . . . . . . 65.2 Network Configuration for iSCSI using distributed switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85.3 iSCSI Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.4 VMware Storage Best Practice Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.5 Leverage VAAI to Accelerate Storage Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125.6 Types of vSphere Disks to Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

6. Datera Storage Provisioning for vSphere. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.1 Types of Provisioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.2 Traditional Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.3 Template-Based Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156.4 Creating a Storage Template for SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

7. Datera Plugin for vCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

8. Datera Specific Advanced Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248.1 VMware Advanced Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

JULY 2020

https://datera.io/contact-us/http://www.datera.iomailto:sales%40datera.io?subject=My%20Free%20Consultation



2. Virtual Infrastructure Storage ChallengesThe massive growth in data in the new economy mandates the need for a flexible and scalable storage infrastructure. Virtualization has become mainstream over the past decade with a majority of business-critical applications being virtualized. Storage has always been one of the difficult components to provision and manage in a VMware Virtual infrastructure. Deploying high performance storage in a shared virtual infrastructure is very cumbersome and time consuming. Database and other IO intensive applications require dedicated LUNS and segregation from other applications.

Many of the physical design concepts for databases need to be implemented in the virtual infrastructure as well. Special storage requirements of these high IO applications can make storage management complex and time consuming.

3. Datera OverviewDatera® is a 100% software-driven data services platform that powers today’s high-performance application environments at global scale with game-changing data orchestration and automation, in partnership with the leaders in server-based infrastructure. The platform is designed for traditional enterprises and leading-edge digital businesses, from online applications to cloud service providers, to establish a new and dynamic enterprise storage infrastructure.

Designed for containers, virtual machines and bare metal deployments, Datera supports data storage at scale for any application within traditional datacenter and software defined data center environments. The Datera platform provides continuously available and tunable quality of service for traditional, virtualized and cloud-native applications.

Datera is deployed in a scale-out configuration of industry standard x86 servers containing a heterogeneous mix of standard servers utilizing a variety of storage media types including persistent memory, NVMe flash, SATA flash and/or conventional HDDs.

The Datera VCP Plugin is a user-friendly browser based tool that integrates with the VMware™ vSphere™ Web Client, providing an alternative interface that allows you to monitor and manage the Datera Storage System.

3.1 Dynamically Adaptable PerformanceDatera is built to provide adaptable performance based on application needs. If the requirements or the performance profile of the application were to change due to usage or peak times, the storage profile can be dynamically adjusted. Datera’s Application Intent-based storage can dynamically be adapted for performance and capacity needs




3.2 vCenter Plugin for Ease of ProvisioningTraditionally all storage provisioning and management operations happen at the storage system with no visibility for the VI admin. The Datera plugin for vCenter provides the ability to manage its storage directly from vCenter. The VI admin can monitor all the recent activity in the Datera system with timestamps and types of activity performed. Adding storage usually involves creating LUNS in the storage system, provisioning them to the VMware cluster and then discovering and formatting them. New volumes can be created end-to-end using the plugin for use in a cluster with a simple wizard. As part of the process, a pair of dedicated storage targets are setup automatically and the LUNS are provisioned on the storage system, followed by the creation of VMFS data stores seen by all nodes in the Datera system.

3.3 Fully Programmable Software Defined Storage VMware has charted out its vision for the software defined data center (SDDC) that allows for compute, network and storage for virtual infrastructure to be defined through software. The storage platform should be amenable to SDDC and provide the software components and REST APIs to work in this infrastructure. Datera provides REST APIs and automation capabilities that can help make software defined storage a reality. VAAI compatible storage providing storage efficiencies in cloning, copy and other operations.

4. Network PreparationiSCSI uses the same network stack for storage as does the compute layer. Care should be taken to setup the networking in accordance with VMware best practices for ISCSI. The following topology shows a typical Datera network configuration.

Figure 1: Datera Logical Network Topology




Multiple private networks help with Datera inter-node communications and management. The Access network is the storage network that the ESX hosts use to communicate with the storage. This access network is usually dedicated for storage; public access networks are segregated into other VLANs. In planning for Datera networking all these different network segments should be architected and deployed in their own VLANs for optimal storage performance.

Figure 2: vSphere Network Design with Distributed Switch

Modern vSphere network designs leverage VLAN tagging and distributed switches to have logical separation of storage traffic through VLANS and load balancing across multiple high speed links as shown above. Each of the port groups particularly relating to storage and vMotion have their own dedicated VLAN. The iSCSI port group will be on the same VLAN as the access network VLAN used by the Datera system.

4.1 Datera Access NetworkThe vSphere servers access the Datera storage through the Access Network iSCSI ports. Each Datera node connects to the Access Network through one or more 10-100 Gigabit Ethernet (GbE) interfaces. The Access Network must be a traditional IP-based network. The target IP addresses float during node rebalancing activities that occur as part of the load redistribution activities.

The Access Network uses floating IP addresses to present iSCSI LUNs to hosts. The IP address range you provide must at least be equal to the number of nodes in the Datera system. However, the more IP addresses available, the better the load distribution for the system. Datera recommends at least 32 addresses for small systems with up to 10 nodes, 64 addresses for medium systems up to 20 nodes, and 128-250 addresses for large systems greater than 20 nodes.




The target IP addresses presented by the target iSCSI ports are drawn from the Access Network IP pools. The IP address in the pools are defined by the Access Network IP blocks. You must create at least one IP pool and at least one block of IP addresses to be assigned to Storage Instances.

4.2 Summary of Networking Steps for Setting up iSCSI Interfaces in vSphereThe iSCSI adapter configuration workflow includes the following steps:

• Activate the Software iSCSI Adapter: You must activate your software iSCSI adapter so that your host canuse it to access iSCSI storage.

• Modify General Properties for iSCSI Adapters: You can change the default iSCSI name and alias assignedto your iSCSI adapters. For the independent hardware iSCSI adapters, you can also change the default IPsettings.

• Create Network Connections for iSCSI.• Configure connections for the traffic between the software or dependent hardware iSCSI adapters and the

physical network adapters.• If you do not use VCP to provision vSphere Storage, you can set up Dynamic or Static Discovery for iSCSI.

Datera supports iSCSI redirect feature, but you can only use Access VIP 1 and Access VIP 2 addresses.Obtain the addresses from the Datera Systems Settings dashboard in the GUI. Use the addresses as iSCSItarget IPs in “Dynamic Discovery” instead of inputting every target IP. This avoids administrator effort toadd each later provisioned target, and you can just re-scan and the new added targets will be automaticallyadded to vSphere.

The vSphere Storage Configuration Guide provides detailed configuration information for iSCSI in vSphere environments.

4.3 vSphere Networking Best Practices for Datera Deployment• Have a dedicated VLAN for storage traffic representing the Datera storage access network in the vSphere

infrastructure.• Dedicate one or more physical adapters for storage traffic.• Leverage vSphere network and storage multi-pathing capabilities.• Leverage vSphere features like Network IO Control (NIOC) to give preference to ISCSI stor-age traffic over

vMotion and other traffic in the shared physical network infrastructure.

Please be sure to follow the vSphere Networking Guide to configure networking per best practices.

https://datera.io/contact-us/http://www.datera.iomailto:sales%40datera.io?subject=My%20Free%20Consultationhttps://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-storage-guide.pdfhttps://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-networking-guide.pdf



5. Setting up the Software iSCSI AdapterMulti-pathing can be leveraged with vSphere Software iSCSI that can help load-balance across all paths and handle any path failures between vSphere and the shared storage. Multiple software ISCSI adapters need to be setup to leverage multi-pathing capabilities between the vSphere servers and the storage system.

5.1 Network Configuration for iSCSI using standard switch (vSwitch switch)For software iSCSI multipathing with standard switch configure the following:

Create two dedicated vSwitches for iSCSI and two dedicated VMkernel port groups on both of them (each VMkernel port group on each vSwitch for ISCSI). These are typically setup in two dedicated storage VLANs for traffic isolation. Make sure that there is only one uplink that is bound to the dedicated vSwitch with the dedicated port group on each ESXi host per ISCSI requirements.

Figure 3: ISCSI VMkernel Configuration on two vSwitches




Bind each VMkernel network adapter to the software iSCSI adapter.

Figure 4: iSCSI Software Adapter with Multipathing

Then run a rediscovery of iSCSI targets to detect multiple paths to them. Refer to the section 5.3 to learn the network configuration and port binding procedure required to enable vmknic-based multi-pathing for software iSCSI.

Figure 5: Datera iSCSI Storage Devices




5.2 Network Configuration for iSCSI using distributed switchFor software iSCSI multipathing with distributed switch configure the following:

Create two dedicated distributed switches for iSCSI and two dedicated VMkernel port groups on both of them (each VMkernel port group on each distributed switch for ISCSI). These are typically setup in two dedicated storage VLANs for traffic isolation. Make sure that there is only one uplink that is bound to the dedicated distributed switch with the dedicated port group on each ESXi host per ISCSI requirements.

Figure 6: Distributed Switches and VMKernel Port Groups Configuration

Figure 7: iSCSI VMKernel on Distributed Switches Configuration for Each ESXi Host




Figure 8: Uplink on Distributed Switches Configuration for Software iSCSI Adapter on Each ESXi Host

Bind each VMkernel network adapter to the software iSCSI adapter.

Figure 9: iSCSI Software Adapter with Multipathing (Distributed Switch)




Then run a rediscovery of iSCSI targets to detect multiple paths to them. Refer to the following section to learn the network configuration and port binding procedure required to enable vmknic-based multi-pathing for software iSCSI.

Figure 10: Datera iSCSI Storage Devices with Distributed Switch




5.3 iSCSI ConfigurationThe default iSCSI configuration in vSphere is not set up for multi-pathing. To enable software iSCSI multi-pathing, the following steps are needed:

1. Two VMkernel port groups in either two dedicated vSwitches or two dedicated distributed switches withthe dedicated uplink, so the two independent network paths for iSCSI traffic are built.

2. The VMkernel network adapter should be bound to the software iSCSI adapter, so vSphere VMKernel canapply the defined multipathing policy on top of the above two network paths for software iSCSI traffic. TheiSCSI targets should be rescanned to detect multiple paths to them.

iSCSI Load Balanced Configuration provides additional details on configuration.

5.4 VMware Storage Best Practice Considerations1. Multi-pathing can be leveraged for improved redundancy and performance in iSCSI environments. iSCSI

multi-pathing provides multiple paths to the Datera storage. Multi-Pathing leverages alternate VMkernelnetworks outside of the vSphere host management network. Even if the management network has anoutage, there is still iSCSI connectivity through the VMkernel ports that are part of the iSCSI bindings. Makesure that the multi-pathing policy setting for Datera LUNS is set to Round Robin.

2. Storage I/O Control (SIOC) prevents the noisy-neighbor problem by preventing a virtual machine fromconsuming more than its share of bandwidth on a datastore that it shares with other virtual machines. SIOCcontrols the I/O queues on various vSphere hosts to ensure that virtual machines with a higher priority getmore priority over other virtual machines with lower priority. SIOC is a congestion-driven feature that istriggered only when the latency value on the datastore goes above a predefined threshold.

3. Network I/O Control (NIOC) ensures that when network cards are shared between multiple traffic types,iSCSI traffic gets the appropriate priority. The feature works through the use of priority and bandwidththrottling leveraging priority tags in TCP/IP. With the now more prevalent 10GbE networks this featurecan be very useful because there is one pipe that is shared with multiple other traffic types. NIOC is alsocongestion-driven and takes effect only when there is traffic contention. NIOC can be leveraged to deal withthe noisy-neighbor problem from a networking perspective.

4. Jumbo Frames can be leveraged for improved network performance. Please follow guidelines in VMware KBArticle to setup jumbo frames for iSCSI.

5. Number of LUNS and VM placement: The number of LUNS and the way virtual machines are distributed canaffect performance. More LUNs with fewer virtual machines provides capability for the vSphere servers tohave more parallel I/O requests with the storage system. Having too many LUNS can potentially saturate theI/O queues of the storage system. Care is required in the LUN design to ensure all resources are optimized.ESXTOP is a good tool to monitor storage performance and to identify bottlenecks and congestion areas. Agood practice is to use separate datastores for images, templates and ISOs and to not use datastores usedby virtual machines.

https://datera.io/contact-us/http://www.datera.iomailto:sales%40datera.io?subject=My%20Free%20Consultationhttp://www.vmware.com/files/pdf/techpaper/vmware-multipathing-configuration-software-iSCSI-port-binding.pdf



6. vSphere Storage vMotion can be leveraged to relocate storage in Datera environments for migrationand workload separation. Storage vMotion leverages the vMotion VMKernel interface and appropriatebandwidth should be available to ensure successful and timely storage vMotion operations.

7. vSphere Storage DRS leverages SIOC to monitor I/O round trip times and latency. If the latency value fora particular datastore is more than a threshold value over a prolonged period of time over an observationperiod (default 16 hours), vSphere Storage DRS works to rebalance the virtual machines across thedatastores in the datastore cluster to bring the latency under control. Storage DRS leverages vSphereStorage vMotion to move VMs between datastores. Storage DRS can be set to either automatic or manualmode based on the administrator’s comfort level with the technology.

8. vSphere Storage APIs Integration: The Datera platform supports VAAI as discussed earlier. VAAI offloadsmany performance-intensive operations to the storage system and should be leveraged for optimalperformance.

9. VMware vSphere Replication is fully supported for the replication of virtual machines that reside on iSCSIdatastores. This can be leveraged for backup solutions. vSphere Replication can also work with VMware SiteRecovery Manager (SRM) for orchestrated disaster recovery for virtualized environments.

10. RDM vs VMFS: vSphere supports raw device mapping that allows management and access or raw SCSIdisks or LUNs as VMFS. VMware recommends VMFS over RDM for ease of management and consistencyas there is no performance difference between the two. RDM is used mainly for corner cases such as inguest clustering, where there is need for a shared disk with SCSI-3 reservation needs and for tooling at thestorage system level with backups.

5.5 Leverage VAAI to Accelerate Storage OperationsDatera is a VAAI-capable storage system. The performance gains from VAAI can be especially noticeable in VDI environments (where VAAI can improve boot-storm and desktop workload performance), large data centers (where VAAI can improve the performance of mass virtual machine provisioning and of thin-provisioned virtual disks), and in other large-scale deployments. Since Datera storage supports VAAI, vSphere will automatically recognize and use these capabilities.

The VAAI-compliant Datera platform offers the following features:

• Hardware-accelerated cloning.• Speeds up creation of eager-zeroed thick disks and can improve first-time write performance on lazy-

zeroed thick disks and on thin disks.• Scalable lock management (sometimes called “atomic test and set,” or ATS) can reduce locking-related

overheads, speeding up thin disk expansion as well as many other administrative and file system intensivetasks. This helps improve the scalability of very large deployments by speeding up provisioning operationslike boot storms, expansion of thin disks, snapshots, and other tasks.

• Using thin provision UNMAP, ESXi can allow the storage system to reuse no-longer needed blocks.




5.6 Types of vSphere Disks to UseThere has always been debate on whether thin or thick disks for high performance and database workloads. Traditionally Eager Zeroed Thick disks provided the best performance. But they occupied the entire space including unused space. Thin disks used only the space actually used by the overlaid operating system or application. There are concerns about fragmentation and first write penalty for thin disks. Modern VAAI storage has revolutionized and allayed concerns about differences between the disk formats:

• VAAI supported storage systems support the block zero primitive, so zeroing blocks takes no time at all.Provisioning eager zeroed thick format at the storage system level takes no more time than thin or lazyzeroed disks because of VAAI.

• For Lazy Zeroed thick or thin LUN provisioning, the first write penalty is absent at the storage system level.The vSphere layer issues one or more IO based on the type of disk used but the impact on the storagesystem is negligible.

• Fragmentation is not an issue for thin provisioned disks because the data is spread over multiple Dateranodes.




6. Datera Storage Provisioning for vSphereThe Datera storage platform can be managed entirely using a web interface. The dashboard that shows the health and status of the system is shown below.

Figure 11: Datera Storage Dashboard

6.1 Types of ProvisioningStorage can be provisioned using a traditional or a template-based methodology.

6.2 Traditional ProvisioningDatera storage can be provisioned like any other iSCSI storage for use in vSphere environments. The unique aspect of Datera is it provides dedicated target Virtual IP (VIP) addresses for every instance of shared disks.

1. After the storage is provisioned, the network VIPs allocated for the storage can be seen in the Datera Admininterface. These VIPs will be part of the vSphere Storage VLAN dedicated for storage traffic.

2. The VIPs should be added to the iSCSI target list for the vSphere nodes that will need access to the sharedstorage.

3. Rescan the iSCSI adapter after the new target IP addresses are added and the new storage devices will bediscovered.

4. One can choose to format the new storage with VMFS or use them as raw devices in the virtual environment.




6.3 Template-Based Provisioning Template-based provisioning simplifies and automates many of the tasks for complex storage needs of applications and environments. Virtualized Business critical databases have storage requirements that mirror those of physical databases. Database data and log disks should be separated into dedicated LUNs for optimal performance. The task of creating these LUNs is time consuming for the storage and the virtualization teams. The process with which traditional virtual machines are deployed is disrupted and delayed due to the unique requirements of the dedicated LUNS for databases. The Datera platform and its application awareness can be leveraged in the form of storage templates to automate provisioning of storage for virtualized databases.

6.4 Creating a Storage Template for SQL ServerStorage provisioning for SQL databases for virtual environments is similar to physical environments. The disks have to be separated based on the different I/O file types. For transactional SQL Servers there is need to logically separate TempDB, Data and Logs into multiple volumes at the Datera system level to maximize performance. Multiple PVSCSI adapters will be leveraged with the disks load balanced across them to optimize throughput. The rationale behind these recommendations is the need to parallelize and reduce latencies across different storage types leveraged by the SQL database. For further information please refer to Architecting Microsoft SQL Server on vSphere.

Application specific storage templates can be created and instantiated easily for the database servers consistently. The picture below shows the process of template creation for SQL server.

Figure 12: Specifications for SQL Server Storage Template

https://datera.io/contact-us/http://www.datera.iomailto:sales%40datera.io?subject=My%20Free%20Consultationhttp://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/sql-server-on-vmware-best-practices-guide.pdfhttp://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/sql-server-on-vmware-best-practices-guide.pdf



7. Datera Plugin for vCenterDatera plugin for vCenter helps virtual and storage administrators. The Datera plugin provides the capability to discover the attached Datera system and provision storage from the system to a cluster of vCenter hosts. The details are displayed in the Datera plugin dashboard through vCenter.

The dashboard shows the status of the Datera Storage cluster such as the system metrics, the health and the number of nodes. The capacity of the cluster and its utilization are summarized. The dashboard provides a good view into the Datera system from the vCenter itself.

Figure 13: Specifications for SQL Server Storage Template




System level activity provides insight into the latest activity at the volume and storage instances. Details about all system activity in the Datera system can be seen through the plugin as shown in the following figure.

Figure 14: Datera System Activity View in vCenter




User level activity in Datera can also be seen through the vCenter Datera plugin. New storage can be provisioned directly from vCenter itself on Datera through the Plugin. The steps below show how seamless it is to provision storage for use in a vCenter cluster from Datera.

Figure 15: Datera User Activity View in vCenter

1. From the vSphere web client, the Datera plugin action of creating a Datastore can be initi-ated.

Figure 16: Creating Datastore from vCenter Plugin




2. Details about the Datastore, its size and performance requirements can be input prior to its creation.

3. After the process completes one can look at the Datera storage instances in its console and see that a newinstance referencing the requested Datastore has been created.

Figure 17: Datastore specifications

Figure 18: Instance for data store in Datera System




Details of the instance created through the plugin is shown in the following figure. Details about the initiator and custom targets are shown. Each instance gets its own dedicated target, which is automatically added to the vSphere nodes as part of this workflow.

Figure 19: Details of Datastore with Target Information




4. All the ESX hosts in the cluster have a new entry for the target representing the Datastore addedautomatically created for the Software iSCSI adapter as shown.

Figure 20: Target is Automatically Created in vCenter for Cluster Hosts




5. After a few minutes, the disk is automatically added to the cluster as seen in the following figure.

Figure 21: Disk is Seen After Rescan of iSCSI Adapter which is done Automatically




As a final step a Datastore is created and available for all the nodes in the cluster with the provisioned space.

Figure 22: Datastore Created Using the Plugin

The entire workflow of storage provisioning at the storage system level, provisioning it to the vSphere infrastructure followed by scanning for LUNS at the vSphere level, adding Datastores to each of the nodes in the cluster has been greatly simplified by the Datera platform.




8. Datera Specific Advanced Configurations8.1 VMware Advanced ParametersThe Datera Plugin for vCenter is used to configure advanced storage related options for VMware environments partially. The advanced parameters shown are automatically configured based on best practices for Datera partially. These parameters affect VAAI, LUN Queue Depth and Multi-pathing.

1. [Default configuration] /DataMover/HardwareAcceleratedMove -> Enabled

2. [Default configuration] /DataMover/HardwareAcceleratedInit -> Enabled

3. [Default configuration] /VMFS3/HardwareAcceleratedLocking -> Enabled

4. VMFS3.UseATSForHBonVMFS5 (if system is running Datera version > 2.1) -> Set to false

5. iscsivmk_LunQdepth: —> Set to 16 (“16 is Maximum outstanding commands for each LUN accessedthrough the software iSCSI adapter”).

6. Multipath Path selection: -> Round Robin

7. Round Robin IOPS -> 1

8. TCP delayed ack for iSCSI adapter -> disabled

VAAI features XCOPY, WRITE SAME, ATS are enabled by default for ESXi 6.x and ESXi 5.5. It is good to double-check them on ESXi host one by one. 1 means enabled, 0 means disabled.

[root@esxi:~] esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove

Value of HardwareAcceleratedMove is 1

[root@esxi:~] esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit

Value of HardwareAcceleratedInit is 1

[root@esxi:~] esxcfg-advcfg -g /VMFS3/HardwareAcceleratedLocking

Value of HardwareAcceleratedLocking is 1




ATS heartbeat should be disabled due to the issue discussed in https://kb.vmware.com/selfservice/microsites/search.do?language=enUS&cmd=displayKC&externalId=2113956.

Use the set command to disable it and list command to check if “Int Value: 0” (0 means disabled).

[root@tlx169:~] esxcli system settings advanced set -i 0 -o /VMFS3/UseATSForHBOnVMFS5

[root@tlx169:~] esxcli system settings advanced list -o /VMFS3/UseATSForHBonVMFS5

Path: /VMFS3/UseATSForHBOnVMFS5

Type: integer

Int Value: 0

Default Int Value: 1

Min Value: 0

Max Value: 1

String Value:

Default String Value:

Valid Characters:

Description: Use ATS for HB on ATS supported VMFS5 volumes

[root@tlx169:~]

iscsivmk_LunQdepth is usually 128 on ESXi 6.x or ESXi 5.5 by default, we suggest to use set command to change it to 16 and list command to show the change.

[root@esxi:~] esxcli system module parameters set -m iscsi_vmk -p iscsivmk _ LuQDepth=16

[root@esxi:~] esxcli system module parameters list -m iscsi_vmk | grep iscsivmk_LunQDepth

iscsivmk _ LunQDepth int 16 Maximum Outstanding Commands Per LUN

Datera recommends you enable ESXi host Adaptive Queue Depth (AQD) support to handle end-to-end throttling under heavy I/O load . The adaptive queue depth algorithm temporarily reduces the ESX host queue depth when a Datera system responds with QUEUE FULL SCSI status, and gradually increases back-up to the original queue depth over time. The ESXi host adaptive queue depth algorithm is disabled by default.

Set the --queue-full-sample-size to 32 and the --queue-full-threshold to 4.

esxcli storage core device set --device device_uid --queue-full-sample-size 32 --queue-full-threshold 4

https://datera.io/contact-us/http://www.datera.iomailto:sales%40datera.io?subject=My%20Free%20Consultationhttps://kb.vmware.com/selfservice/microsites/search.do?language=enUS&cmd=displayKC&externalId=2113956



Multipath parameters path selection is VMW_PSP_MRU for VMW_SATP_ALUA by default on ESXi hosts, Datera suggests creating a DATERA NMP SATP rule on each ESXi host. Doing this keeps VMWare VMW_SATP_ALUA as the default policy, at the same time DATERA iSCSI storage uses the Datera customized NMP SATP rule with VMW_PSP_RR and IOPS=1 as default.

[root@esxi:~] esxcli storage nmp satp rule add -s VMW_SATP_ALUA -P VMW_PSP_RR -O iops=1 -V DATERA -e

“DATERA custom SATA rule”

[root@esxi:~] esxcli storage nmp satp rule list | grep DATERA

VMW _ SATP _ ALUA DATERA

user V MW _ PSP _ RR iops=1 DATERA custom SATArule

Configuration changes only take effect after the ESXi host reboots for some ESXi versions. Please consult your ESXi documentation for directions on safely rebooting your ESXi host without risk.

Once the customized DATERA SATA rule takes effect, all Datera disks to be provisioned for ESXi will use this rule. The previously provisioned Datera disks will keep using the old path selection policy on ESXi when they were created.

[root@esxi:~] esxcli storage nmp device list

naa.6001405197f83328d1641f5be875317a

Device Display Name: DATERA iSCSI Disk (naa.6001405197f83328d1641f5be875317a)

Storage Array Type: VMW_SATP_ALUA

Storage Array Type Device Config: {implicit_support=on; explicit_support=on; explicit allow=on;

alua_followover=on; action_OnRetryErrors=off;

{TPG _ id=0,TPG_state=AO}}

Path Selection Policy: VMW_PSP_RR

Path Selection Policy Device Config: {policy=rr,iops=1,bytes=10485760,useANO=0; lastPathIndex=0:

NumIOsPending=0,numBytesPending=0}

Path Selection Policy Device Custom Config:

Working Paths: vmhba64:C1:T8:L0, vmhba64:C0:T8:L0

Is USB: false




Simply speaking, iSCSI is SCSI on top of TCP protocol. The original TCP delayed ack is not good for iSCSI traffic. VMware suggests you disable it for iSCSI adapter.

[root@esxi:~] export iscsi_if=`esxcli iscsi adapter list | grep iscsi_vmk |awk '{ print $1 }'`

[root@esxi:~] vmkiscsi-tool -W -a delayed_ack=0 -j $iscsi_if

Setting iSCSI Parameter ...

[root@esxi:~] vmkiscsi-tool -W $iscsi_if | grep delayed_ack

delayed_ack OFF N/A OFF Yes

9. ConclusionDatera is a highly scalable solution for virtual environments and provides economic elasticity with the ability to easily add capacity horizontally as needed. Datera allows for application aware storage provisioning that decouples it from the physical infrastructure management. The use of the Datera Plugin for vSphere shows how easy it is to provide additional capacity to a VMware virtual environment. Datera templates provide an easy way to add capacity to an environment attached to an instance with minimal effort and simplicity. Datera is truly a next-generation platform for storage in VMware virtualized environments.




Copyright

Copyright© 2020 Datera, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written consent of Datera, Inc.

Trademarks

Datera®, LIO™ and the Datera logo are trademarks of Datera, Inc., which are registered in some jurisdictions. Linux is a registered trademark of Linus Torvalds. Kubernetes is a registered trademark of the Linux foundation. OpenStack is a registered trademark of the OpenStack Foundation. Docker is a registered trademark of Docker, Inc. Mesos is a registered trademark of the Apache Software Foundation. AWS is a registered trademark of Amazon.com, Inc. VMware and vSphere are trademarks of VMware, Inc. Microsoft and Windows are trademarks of Microsoft Corp. All other trademarks are the property of their respective owners.

Changes

The material in this document is for information only and is subject to change without notice. While reasonable efforts have been made in the preparation of this document to assure its accuracy, Datera, Inc. assumes no liability resulting from errors or omissions in this document, or from the use of the information contained herein. Datera, Inc. reserves the right to make changes in the product design without reservation and without notification to its users.

Disclaimer

If this product directs you to copy materials, you must have permissions from the copyright owner of the materials to avoid violating the law, which could results in serious damages and/or remedies.

Customer Support

• Visit our website at www.datera.io• Email: [email protected]• U.S. Toll Free: +1-844-432-8372• International +1-408-404-8736


2. Virtual Infrastructure Storage Challenges3. Datera Overview3.1 Dynamically Adaptable Performance3.2 vCenter Plugin for Ease of Provisioning3.3 Fully Programmable Software Defined Storage

4. Network Preparation4.1 Datera Access Network4.2 Summary of Networking Steps for Setting up iSCSI Interfaces in vSphere4.3 vSphere Networking Best Practices for Datera Deployment

5. Setting up the Software iSCSI Adapter 5.1 Network Configuration for iSCSI using standard switch (vSwitch switch)5.2 Network Configuration for iSCSI using distributed switch5.3 iSCSI Configuration5.4 VMware Storage Best Practice Considerations5.5 Leverage VAAI to Accelerate Storage Operations5.6 Types of vSphere Disks to Use

6. Datera Storage Provisioning for vSphere6.1 Types of Provisioning6.2 Traditional Provisioning6.3 Template-Based Provisioning 6.4 Creating a Storage Template for SQL Server

7. Datera Plugin for vCenter8. Datera Specific Advanced Configurations8.1 VMware Advanced Parameters

9. Conclusion

Documents

Datera VMware vSphere Deployment Guide...4.3 vSphere Networking Best Practices for Datera Deployment • Have a dedicated VLAN for storage traffic representing the Datera storage access