Click here to load reader

FlexPod Datacenter with Microsoft Private Cloud | NVA-0010-DESIGN

  • View
    217

  • Download
    2

Embed Size (px)

Text of FlexPod Datacenter with Microsoft Private Cloud | NVA-0010-DESIGN

  • NetApp Verified Architecture

    FlexPod Datacenter with Microsoft Private Cloud

    Microsoft Private Cloud Fast Track v4 with Windows Server 2012 R2, System Center 2012 R2, and NetApp Clustered Data ONTAP

    Glenn Sizemore, NetApp

    May 2014 | NVA-0010 | Version 1.0

    Status: Final

  • 2 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    TABLE OF CONTENTS

    1 NetApp Verified Architecture .............................................................................................................. 4

    2 Solution Overview ................................................................................................................................ 4

    2.1 Problem Statement .........................................................................................................................................4

    2.2 Target Audience ..............................................................................................................................................4

    2.3 Technology Solution .......................................................................................................................................4

    2.4 Use Case Summary ........................................................................................................................................6

    3 Primary Use Case for FlexPod Datacenter with Microsoft Private Cloud Solution ....................... 6

    3.1 Resource Pooling ............................................................................................................................................7

    3.2 Elasticity and the Perception of Infinite Capacity ............................................................................................7

    3.3 Perception of Continuous Availability ..............................................................................................................7

    3.4 Predictability ....................................................................................................................................................7

    3.5 Metering and Chargeback ...............................................................................................................................7

    3.6 Multi-Tenancy .................................................................................................................................................8

    3.7 Security and Identity .......................................................................................................................................8

    4 Capacity Concepts ............................................................................................................................... 8

    4.1 Characterized Workloads ................................................................................................................................9

    4.2 Uncharacterized Workloads ............................................................................................................................9

    4.3 Resource Pools ...............................................................................................................................................9

    4.4 Capacity-Planning Methodology .....................................................................................................................9

    4.5 Defining the Resource Budget ........................................................................................................................9

    4.6 Defining Buckets for Uncharacterized Workloads ......................................................................................... 10

    4.7 Hardware Requirements ............................................................................................................................... 11

    4.8 Software Requirements ................................................................................................................................ 13

    4.9 Networking .................................................................................................................................................... 14

    4.10 Storage ......................................................................................................................................................... 16

    4.11 Storage Options for Windows Server 2012 R2 ............................................................................................. 18

    4.12 Virtual Infrastructure ...................................................................................................................................... 23

    4.13 Management ................................................................................................................................................. 29

    4.14 Data Protection ............................................................................................................................................. 32

    5 Alternative Use Cases ........................................................................................................................ 39

    6 Design Validation ................................................................................................................................ 39

    6.1 Success Stories ............................................................................................................................................ 39

    7 Conclusion .......................................................................................................................................... 40

  • 3 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    References ................................................................................................................................................. 40

    Supporting Documents ......................................................................................................................................... 40

    Recommended Documents .................................................................................................................................. 40

    Version History ......................................................................................................................................... 40

    Acknowledgements .................................................................................................................................. 41

    LIST OF TABLES

    Table 1) Example VM classes. ................................................................................................................... 10

    Table 2) Distinctions in levels of service. .................................................................................................... 10

    Table 3) CSV parameters. .......................................................................................................................... 19

    Table 4) Feature comparison of Microsoft Hyper-V capabilities. ................................................................ 24

    Table 5) Host cluster networks. .................................................................................................................. 26

    LIST OF FIGURES

    Figure 1) FlexPod component families. ........................................................................................................ 5

    Figure 2) FlexPod discrete uplink design with NetApp clustered Data ONTAP. ........................................ 12

    Figure 3) NetApp integration with Microsoft................................................................................................ 23

    Figure 4) NetApp clustered Data ONTAP controller as a monitored object on Microsoft System Manager 2012 R2 Operations Manager console. ...................................................................................................... 30

    Figure 5) NetApp OnCommand System Manager. ..................................................................................... 31

    Figure 6) NetApp SMHV 2.0 distributed application-consistent backup architecture. ................................ 34

    Figure 7) NetApp SMHV 2.0 architecture. .................................................................................................. 35

    Figure 8) Backup types. .............................................................................................................................. 37

    Figure 9) NetApp SnapVault integration into SMHV. .................................................................................. 38

    Figure 10) NetApp SMHV SnapVault options and Snapshot labels. .......................................................... 38

  • 4 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    1 NetApp Verified Architecture

    The NetApp Verified Architecture (NVA) program offers customers a validated architecture for NetApp

    solutions. The NVA provides customers with a NetApp solution architecture that:

    Is thoroughly tested

    Is prescriptive

    Minimizes customers deployment risk

    Accelerates their time to results

    2 Solution Overview

    This document describes FlexPod Datacenter with Microsoft

    Private Cloud, a solution for deploying

    Cisco and NetApp technologies as a shared cloud infrastructure that has been validated under the

    Microsoft Private Cloud Fast Track v4 program.

    Microsoft Private Cloud Fast Track is a joint effort between Microsoft and its hardware partners to deliver

    preconfigured virtualization and private cloud solutions. The program focuses on new technologies and

    services in Microsoft Windows Server in addition to investments in Microsoft System Center.

    The validated designs in the Microsoft Private Cloud Fast Track program deliver best-in-class solutions

    from Microsofts hardware partners that guide Microsoft technologies, investments, and best practices.

    The private cloud model provides much of the efficiency and agility of cloud computing, along with the

    increased control and customization that are achieved through dedicated private resources. Through the

    Microsoft Private Cloud Fast Track v4validated FlexPod Datacenter with Microsoft Private Cloud

    solution, Cisco, NetApp, and Microsoft can offer organizations both the control and the flexibility that are

    required to reap the potential benefits of the private cloud.

    Microsoft Private Cloud Fast Track uses the core capabilities of Windows Server, Hyper-V, and System

    Center to deliver a private cloud infrastructure as a service offering. FlexPod Datacenter with Microsoft

    Private Cloud builds on the Microsoft Fast Track program to deliver industry-leading integration and

    implementation guidance.

    2.1 Problem Statement

    Cloud-style architecture offers significant reductions in cost and increases business agility. However,

    these systems are complex and difficult to install and configure. This NVA document is designed to

    reduce deployment and design time for FlexPod customers and partners by providing specific guidance

    for creating a FlexPod Datacenter with Microsoft Private Cloud solution.

    2.2 Target Audience

    The FlexPod Datacenter with Microsoft Private Cloud NVA is recommended for the following audiences:

    Customer or partner architects

    Customer IT business leaders

    Private-cloud architects

    2.3 Technology Solution

    Industry trends indicate a vast data center transformation toward shared infrastructure and cloud

    computing, sometimes referred to as software-defined computing. Enterprise customers are moving away

    from isolated centers of IT operation toward more cost-effective virtualized environments. The objective of

    the move toward virtualization, and eventually to software-defined cloud computing, is to increase agility

    and reduce cost.

  • 5 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Especially because companies must address resistance to change in both their organizational and their

    technical IT models, achieving this transformation can seem daunting and complex. To accelerate the

    process and simplify the evolution to a shared-cloud, software-defined infrastructure, Cisco and NetApp

    have developed a solution called FlexPod Datacenter with Microsoft Private Cloud that is validated by

    Microsoft Private Cloud Fast Track v4.

    FlexPod is a predesigned best-practice data center architecture that is built on Cisco United Computing

    System

    (Cisco UCS), the Cisco Nexus

    family of switches, and NetApp fabric-attached storage (FAS)

    systems, as shown in Figure 1. FlexPod is a suitable platform for running a variety of virtualization

    hypervisors as well as bare-metal operating systems (OSs) and enterprise workloads. FlexPod delivers

    not only a baseline configuration, but also the flexibility to be sized and optimized to accommodate many

    different use cases and requirements.

    Figure 1) FlexPod component families.

    This document describes the FlexPod Datacenter with Microsoft Private Cloud solution from Cisco and

    NetApp, validated by Microsoft Private Cloud Fast Track v4. It discusses design choices and deployment

    best practices for using this shared infrastructure platform.

  • 6 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    2.4 Use Case Summary

    FlexPod Datacenter with Microsoft Private Cloud is a multipurpose platform that delivers a wide variety of

    workloads in an enterprise setting and offers the following key features:

    Nondisruptive operations

    Server Message Block (SMB) 3.0 protocol

    Offloaded data transfer (ODX)

    Simplified storage management

    Backup and recovery

    Nondisruptive operations are achieved by combining the flexibility and power of Windows Server 2012 R2

    Hyper-V with the performance, availability, and efficiency of NetApp clustered Data ONTAP. This

    combination empowers the infrastructure or fabric management team to fully manage all aspects of the

    cloud without affecting nested customer instances.

    Windows Server 2012 introduced an evolution in simplified virtual machine (VM) storage management

    with SMB 3.0 and continuously available file shares. Clustered Data ONTAP 8.2 added support for this

    robust protocol, allowing the Microsoft Private Cloud solution to achieve the benefits of consolidated VM

    storage and a shared-nothing architecture. Because of the NetApp Virtual Storage Tier (VST) and the

    flash storage capabilities of the Data ONTAP architecture, this consolidation offers both greater efficiency

    and improved performance.

    ODX is an offload technology in Windows Server that allows the OS to hand off any copy operation to the

    storage controller. This offload is transparent and requires no customer plug-ins or software. The result is

    that Windows hosts can be loaded with greater density because the host OS is not consumed by file-

    transfer operations. In addition, its token-exchange architecture makes ODX cross-protocol capable. New

    in System Center 2012 R2 Virtual Machine Manager (SCVMM) is the capability to implement a Fast File

    Copy VM deployment through ODX.

    Building on the Storage Management Initiative Specification (SMI-S) published by the Storage Networking

    Industry Association (SNIA), an open standard for enterprise storage management, the solution can

    achieve fully integrated storage provisioning and management, either from Windows Server itself or

    through SCVMM.

    However, standards-based management does not cover all possible contingencies because it is a subset

    of the capabilities of all vendors. Therefore, to facilitate more advanced deployments, Microsoft System

    Center Orchestrator and the NetApp Data ONTAP PowerShell toolkit enable complete end-to-end

    orchestration and automation workflows. When these solutions are combined with System Center Service

    Manager, the workflows can be extended as integrated service offerings without the need for complex

    customer-built portals. They can also be extended further through integration with Microsoft Service

    Management Automation and the Windows Azure

    Management Pack.

    NetApp SnapManager for Microsoft Hyper-V provides a complete backup and recovery infrastructure for

    private cloud. New in the latest release is the integration with distributed backup operations for Microsoft

    Cluster Shared Volumes (CSVs) and the ability to back up VMs located on an SMB share by using the

    Microsoft remote volume shadow copy service (remote VSS). These advances allow the native

    integration of NetApp Snapshot

    and FlexClone technologies to perform fast backup and restore

    operations, regardless of the size of the VMs.

    3 Primary Use Case for FlexPod Datacenter with Microsoft Private

    Cloud Solution

    The architecture principles of Microsoft Private Cloud conform to the cloud attributes outlined by the

    National Institute of Standards and Technology (NIST) definition of cloud computing: on-demand, self-

  • 7 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    service, broad network access, resource pooling, rapid elasticity, and measured service. Similarly, the

    Microsoft Hyper-V cloud architecture is based on these seven principles:

    Resource pooling

    Elasticity and the perception of infinite capacity

    Perception of continuous availability

    Predictability

    Metering and chargeback

    Multi-tenancy

    Security and identity

    3.1 Resource Pooling

    Resource optimization, which promotes efficiency and cost reduction, is primarily achieved through

    resource pooling. Abstracting the solution platform from the physical infrastructure allows resources to be

    optimized through shared use. Allowing multiple consumers to share resources results in higher resource

    use and more efficient use of the infrastructure. Behind many of the Hyper-V cloud principles is the

    element of optimization through abstraction, which helps improve agility and reduce cost.

    3.2 Elasticity and the Perception of Infinite Capacity

    From a consumers perspective, cloud services appear to have infinite capacity. Like electric utilities, they

    are available for as much or as little use as needed. This utility approach to computing requires proactive

    capacity planning so that requests can be satisfied on demand. Applying the principle of elasticity

    reactively and in isolation often leads to inefficient use of resources and unnecessary costs. But when an

    organization encourages desired consumer behavior, it can use this principle to balance the demand for

    agility with the cost of unused capacity.

    3.3 Perception of Continuous Availability

    From the consumers perspective, cloud services always appear to be available when needed. The

    consumer should never experience an interruption of service, even if failures occur in the Hyper-V cloud

    environment. To achieve this perception, organizations must take a mature service management

    approach that combines inherent application resiliency with infrastructure redundancies in a highly

    automated environment. As with the perception of infinite capacity, this principle can be achieved only in

    conjunction with the other Hyper-V cloud principles.

    3.4 Predictability

    Predictability is a fundamental cloud principle for both consumers and providers. From the consumers

    perspective, cloud services should be consistent; that is, they should have the same quality and

    capabilities each time they are used.

    For the provider, delivering this predictability requires homogenizing the underlying physical servers,

    network devices, and storage systems to create an underlying infrastructure that can offer a consistent

    experience to the hosted workloads. In relation to service management, the provider delivers predictability

    by standardizing service offerings and processes. Following the principle of predictability is fundamental

    to achieving quality of service (QoS).

    3.5 Metering and Chargeback

    When IT professionals are asked to deliver a service to the business, they typically purchase the

    necessary components and then build an infrastructure that is specific to the service requirements. Often,

    this approach results in a longer time to market and increased cost caused by duplicate infrastructure. In

  • 8 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    addition, the service often fails to meet business expectations for agility and cost control. The problem is

    often compounded when an existing service must be expanded or upgraded.

    This common approach to infrastructure deployment has forced most businesses to use complex

    forecasting models and guesswork to predict future needs for each business unit.

    Taking a service providers perspective toward delivering infrastructure transforms the IT approach. If

    infrastructure is provided as a service, IT can use a shared resource model that enables economies of

    scale. Because the resource pool is combined, variations in need among business units can be absorbed,

    and the forecasting model becomes simplified, while accuracy is increased. This principle of providing IT

    as a service, combined with the other principles, helps the organization achieve greater agility at a lower

    cost.

    3.6 Multi-Tenancy

    Multi-tenancy refers to the capability of the infrastructure to be logically subdivided and provisioned to

    different organizations or organizational units. The traditional example is a hosting company that provides

    servers to multiple customer departments. Increasingly, this model is also being used by centralized IT

    departments that provide services to multiple business units within a single organization, treating each as

    a customer or tenant.

    3.7 Security and Identity

    Security for the Hyper-V cloud is founded on three principles:

    Protected infrastructure

    Application access

    Network access

    Protected infrastructure takes advantage of security and identity technologies so that hosts, information,

    and applications are secured across all scenarios in the data center, including the physical (on-premises)

    and virtual (on-premises and cloud) environments.

    Application access helps IT managers extend vital applications to internal users as well as to important

    business partners and cloud users.

    Network access uses an identity-centric approach so that userswhether based in the central office or in

    remote locationshave more secure access no matter what device they use. This security helps users

    stay productive and work effectively.

    Most important from a security standpoint, the secure data center makes use of a common integrated

    technology to help users gain basic access by using a common identity. Management is integrated across

    physical, virtual, and cloud environments so that businesses can take advantage of all capabilities without

    the need for significant additional financial investments.

    4 Capacity Concepts

    Capacity planning relies on workloads, resource pools, and a capacity budget. Workloads in IT

    environments can generally be divided into two categories:

    Characterized workloads

    Uncharacterized workloads

    Effective capacity planning requires a basic understanding of these two distinct types of workloads.

  • 9 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    4.1 Characterized Workloads

    Characterized workloads are well studied, well understood, and well defined. They are generally

    associated with major applications that have industry-wide adoption rates. An example of a characterized

    workload is Microsoft Exchange. Generally, because the requirements of characterized workloads are

    well known and well understood, very precise sizing tools already exist for them.

    4.2 Uncharacterized Workloads

    As their name implies, uncharacterized workloads vary widely and are neither well defined nor well

    understood. There are generally no capacity-planning or sizing tools available for uncharacterized

    workloads because of their nonstandard nature. Most public and private cloud general-purpose virtualized

    client and server workloads fall into the uncharacterized category.

    4.3 Resource Pools

    Within shared infrastructure, resources are managed by grouping like resources together into resource

    pools. A public or private cloud resource can be computing (CPU or memory), storage (performance or

    capacity), or bandwidth (both external for client connectivity or internal connectivity between VMs or

    between VMs and other resources such as storage). The combination of these resource pools becomes

    the basis for a resource budget.

    4.4 Capacity-Planning Methodology

    Capacity planning relies on the definition of a set of resource pools that, when combined, become the

    resource budget. Characterized or well-defined workloads within the environment are sized using normal

    processes and tools, and the result is subtracted from the resource budget. What remains can be applied

    to uncharacterized workloads. The methodology presented categorizes uncharacterized workloads into

    averaged buckets and then subtracts the buckets from the resource budget. The first step in this

    process is to define the budget.

    4.5 Defining the Resource Budget

    A public or private cloud environment has many types of resources. For the purposes of capacity

    planning, the network is assumed, and the focus is on the computing and storage resource types.

    Computing Resources

    Computing resources fall into two broad categories: processing (CPU) and random access memory

    (RAM). When creating VMs in Hyper-V, one early decision to make is determining how much CPU and

    RAM to allocate to the VM.

    Hyper-V uses the concept of logical processors in defining CPU resources. A logical processor can be

    either a physical server or a hyperthreaded core. A physical server with two CPU sockets, each with a 6-

    core hyperthreaded processor, is said to contain 24 logical processors. Windows Server 2012 R2 has a

    hard limit of 2,048 virtual CPUs per host. This limit, however, does not mean that every host can use that

    many virtual processors without any penalty. Therefore, NetApp recommends using the number of logical

    processors to calculate the total number of virtual processors that can be supported, which is determined

    by an acceptable fan-out ratio. For server workloads, NetApp defines the fan-out ratio as up to 8:1. For

    virtual desktop infrastructure (VDI) workloads, fan-out ratios of up to 12:1 are supported. For capacity

    planning, NetApp does not automatically assume the maximum fan-out ratios. Customers should perform

    their own analyses to determine the appropriate fan-out ratio for their requirements.

    Each server in the environment contains a set amount of RAM. In Windows Server 2012 R2, memory can

    be added dynamically to a Hyper-V VM. This capability makes it possible to set minimum and maximum

    amounts of RAM, along with priorities, in addition to the fixed-memory provisioning. The method used for

  • 10 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    applying RAM determines the amount that should be subtracted from the resource pool for each VM. In

    either case, the RAM resource pool is the sum of the RAM available on the servers in the deployment.

    Storage Resources

    Storage resources can be broadly categorized in terms of performance and capacity. When defining

    storage resources, it is often desirable to have more than one storage pool, complete with its unique

    capacity and performance characteristics. The use of multiple pools is especially desirable in planning for

    different levels of service. The gold level of service would likely accommodate more I/O operations per

    second (IOPS) than the bronze level, for example. Independent of the number of pools, each storage pool

    can be defined in terms of IOPS and capacity.

    4.6 Defining Buckets for Uncharacterized Workloads

    Not all VMs are created equal. In fact, a main reason that VM workloads are uncharacterized is that they

    vary so widely. The resource consumption of a VDI desktop differs radically from the resource

    consumption of a departmental Microsoft SharePoint server. One way to bring order to the disparity

    among VM workloads is to divide these widely varying workloads into a relatively small number of

    buckets. NetApp uses the terms small, medium, and large to define VM classes. These classes are

    not static and should be customized to reflect the realities of the organization. Table 1 shows an example

    of this type of categorization.

    Table 1) Example VM classes.

    VM Class Storage IOPS Disk Capacity RAM CPUs

    Small 25 40GB 2GB 1

    Medium 125 100GB 4GB 2

    Large 460 500GB 12GB 4

    Table 2 shows distinctions in levels of service.

    Table 2) Distinctions in levels of service.

    Service Level Backup Retention Mirror VMs per LUN

    Bronze Weekly Unlimited

    Silver Weekly 2 weeks, 0 days, 0 hours Weekly 60

    Gold Daily 2 weeks, 7 days, 0 hours Daily 30

    Platinum Hourly 2 weeks, 7 days, 12 hours Hourly 15

    As illustrated in Table 2, the levels of service focus primarily on integration of backup recovery and

    disaster recovery as a service. However, these services also affect the number of VMs per CSV when

    block storage is used. For instance, the platinum service level calls for fewer VMs per CSV than the

    bronze level because the capability to restore a failed VM (caused by hardware or other failure) depends

    to some degree on how many VMs must be restored. These restrictions do not apply to VM storage on

    SMB 3.0 file shares.

    After the VM classes have been established and overlaid with the services levels, the environment can be

    sized predictably by dividing a given resource consumption from the buckets established earlier. For more

    information about sizing private cloud infrastructure, refer to NetApp TR-4014: Microsoft Private Cloud

    Built-on FlexPod Capacity Planning Guide.

    https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=69235&contentID=73807https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=69235&contentID=73807

  • 11 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    4.7 Hardware Requirements

    This section describes the hardware used in the solution.

    System Overview

    FlexPod is a best-practice data center architecture that is built with three components:

    Cisco UCS

    Cisco Nexus switches

    NetApp FAS systems

    These components are connected and configured according to the best practices of both Cisco and

    NetApp to provide the optimal platform for running a variety of enterprise workloads with confidence.

    FlexPod can scale up for greater performance and capacity (adding computing, network, or storage

    resources individually as needed), or it can scale out for environments that need multiple consistent

    deployments (rolling out additional FlexPod stacks). FlexPod delivers not only a baseline configuration

    but also the flexibility to be sized and optimized to accommodate many different use cases.

    Typically, the more scalable and flexible a solution is, the more difficult it becomes to maintain a single

    unified architecture capable of offering the same features and functions across each implementation.

    Overcoming this challenge is one of the key benefits of FlexPod. Each of the component families shown

    in Figure 1 offers platform and resource options to scale the infrastructure up or down while supporting

    the same features and functions that are required under the configuration and the connectivity best

    practices of FlexPod.

    Design Principles

    FlexPod addresses four primary design principles: availability, scalability, elasticity, and manageability.

    The related architecture goals are as follows:

    Application availability. Deliver accessible and ready-to-use services.

    Scalability. Address increasing demand with appropriate resources.

    Flexibility. Provide new services or recovered resources without requiring infrastructure modification.

    Manageability. Facilitate efficient infrastructure operations through open standards and APIs.

    Note: Performance and security are crucial design criteria that were not directly addressed in this project but are addressed in other collateral and benchmarking and solution-testing efforts. Capabilities and basic security elements were validated.

    FlexPod Discrete Uplink Design

    Figure 2 shows the FlexPod discrete uplink design with clustered Data ONTAP. As the illustration shows,

    the design is fully redundant in the computing, network, and storage layers. There is no single point of

    failure from a device or traffic-path perspective.

  • 12 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Figure 2) FlexPod discrete uplink design with NetApp clustered Data ONTAP.

    The FlexPod discrete uplink design is an end-to-end Ethernet transport solution supporting multiple local

    area network (LAN) and storage area network (SAN) protocols, most notably Fibre Channel over Ethernet

    (FCoE). The solution provides a unified 10-Gigabit Ethernet (10GbE)enabled fabric defined by dedicated

    FCoE uplinks and dedicated Ethernet uplinks between the Cisco UCS fabric interconnects and the Cisco

    Nexus switches, as well as converged connectivity between the NetApp storage devices and the same

    multipurpose Cisco Nexus switches.

    The FlexPod discrete uplink design does not employ a dedicated SAN switching environment and

    requires no dedicated Fibre Channel (FC) connectivity. The Cisco Nexus 5500 platform switches are

    configured in N_port ID virtualization (NPIV) mode, providing storage services for the FCoE-based traffic

    traversing the fabric.

    As Figure 2 shows, link-aggregation technology plays an important role, providing improved aggregate

    bandwidth and link resiliency across the solution stack. The NetApp storage controllers, Cisco UCS, and

    Cisco Nexus 5500 platform all support active port channels using IEEE 802.3ad standard Link

    Aggregation Control Protocol (LACP). Port channel technology is a link-aggregation technique that offers

    link fault tolerance and traffic distribution (load balancing) for improved aggregate bandwidth across

  • 13 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    member ports. In addition, the Cisco Nexus 5000 Series offers virtual PortChannel (vPC) capabilities.

    vPCs allow links that are physically connected to two different Cisco Nexus 5500 platform devices to

    appear as a single logical port channel to a third device, essentially offering device fault tolerance. vPCs

    address aggregate bandwidth and link and device resiliency. The Cisco UCS fabric interconnects and

    NetApp FAS controllers benefit from the Cisco Nexus vPC abstraction, gaining link and device resiliency

    as well as full use of a nonblocking Ethernet fabric.

    Note: The Spanning Tree Protocol does not actively block redundant physical links in a properly configured vPC-enabled environment, so all ports should forward on vPC member ports.

    This dedicated uplink design uses FCoE-capable NetApp FAS controllers. From a storage traffic

    perspective, both standard LACP and Cisco vPC link-aggregation technologies play important roles in the

    FlexPod discrete uplink design. Figure 2 shows the use of dedicated FCoE uplinks between the Cisco

    UCS fabric interconnects and Cisco Nexus 5500 platform unified switches. The Cisco UCS fabric

    interconnects operate in N-Port Virtualization (NPV) mode, so the servers FC traffic is either manually or

    automatically pinned to a specific FCoE uplink; in this case, either of the two FCoE port channels is

    pinned. Using discrete FCoE port channels with distinct VSANs allows an organization to maintain

    traditional SAN A and SAN B fabric separation best practices, including separate zone databases. The

    vPC links between the Cisco Nexus 5500 platform switches and the NetApp storage controllers unified

    target adapters 2 (UTA2) are converged, supporting both FCoE and traditional 10GbE traffic, providing a

    robust last-mile connection between the initiator and the target.

    The initial storage configuration of this solution is a 2-node high-availability (HA) pair with NetApp

    clustered Data ONTAP. An HA pair consists of like storage nodes, such as the NetApp FAS3200,

    FAS6200, or FAS8200 Series. Scalability is achieved by adding storage capacity (disk or shelves) to an

    existing HA pair or by adding HA pairs to the cluster or storage domain.

    For SAN environments, the NetApp clustered Data ONTAP offering allows up to three HA pairs that

    include six clustered nodes to form a single logical entity and a large resource pool of storage that can be

    easily managed, logically carved, and efficiently consumed. For network-attached storage (NAS)

    environments, up to 24 nodes can be configured.

    In both scenarios, the HA interconnect allows each HA node pair to assume control of its partners

    storage (disk or shelves) directly. The local physical HA storage failover capability does not extend

    beyond the HA pair. Furthermore, a cluster of nodes does not have to include similar hardware. Rather,

    individual nodes in an HA pair are configured alike, allowing customers to scale as needed as they bring

    additional HA pairs into the larger cluster.

    Network failover is independent of the HA interconnect. Network failover of each node in the cluster is

    supported by both the interconnect and the switching fabric, permitting cluster and data and management

    network interfaces to fail over to different nodes in the cluster, which extends failover beyond the HA pair.

    Note: Beginning with clustered Data ONTAP 8.2, NetApp storage systems can be configured to operate without cluster interconnect switches when a 2-node storage system is deployed.

    4.8 Software Requirements

    The solution uses the following NetApp software:

    Clustered Data ONTAP 8.2.1

    Data ONTAP SMI-S agent 5.1

    SnapDrive for Windows 7.0.2

    SnapManager for Hyper-V (SMHV) 2.0.2

    OnCommand Plug-In for Microsoft (OCPM) 4.0.1

    Data ONTAP PowerShell toolkit 3.1

    The solution uses the following Microsoft software:

  • 14 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Windows Server 2012 R2

    SQL Server 2012

    System Center 2012 R2 Virtual Machine Manager

    System Center 2012 R2 Operations Manager

    System Center 2012 R2 Orchestrator

    System Center 2012 R2 Service Manager

    System Center 2012 R2 App Controller

    Windows Azure Integration Pack

    The solution uses the following Cisco software:

    Cisco UCS Manager 2.2(1c)

    Cisco Nexus 1000V 4.2(1)SV2(2.1a)

    Cisco UCS Manager Management Pack for Microsoft System Center Operations Manager 2.6.2

    Cisco UCS IP 1.0 release for SCO 1.0

    Cisco UCS Microsoft System Center Virtual Machine Manager 1.0.2

    4.9 Networking

    Cisco Nexus 5500 Platform Switch

    The Cisco Nexus 5000 Series is designed for data center environments, with cut-through technology that

    enables consistent low-latency Ethernet solutions, front-to-back or back-to-front cooling, and data ports in

    the rear, bringing switching into close proximity with servers and making cable runs short and simple. The

    switch series is highly serviceable, with redundant, hot-pluggable power supplies and fan modules. It

    uses data centerclass Cisco NX-OS software for high reliability and ease of management.

    The Cisco Nexus 5500 platform extends the industry-leading versatility of the Cisco Nexus 5000 Series

    10GbE data centerclass switches and provides innovative advances toward higher density, lower

    latency, and multilayer services. The Cisco Nexus 5500 platform is well suited for enterprise-class data

    center server access-layer deployments across a diverse set of physical, virtual, storage-access, and

    high-performance computing (HPC) data center environments.

    The switch used in this FlexPod architecture is the Cisco Nexus 5548UP. It has the following

    specifications:

    A one rack-unit (1RU) 1GbE or 10GbE switch

    32 fixed unified ports on the base chassis and one expansion slot, for a total of 48 ports

    Expansion slot support for any of three module types:

    Unified ports

    1, 2, 4, or 8 Gb/sec native FC

    Ethernet or FCoE

    Throughput of up to 960Gb/sec

    Note: For more information, refer to Cisco Nexus 5000 Series Switches.

    Cisco Nexus 2232PP 10GbE Fabric Extender

    The Cisco Nexus 2232PP 10GbE fabric extender provides thirty-two 10GbE and FCoE Enhanced Small

    Form-Factor Pluggable (SFP+) server ports and eight 10GbE and FCoE SFP+ uplink ports in a compact

    1RU form factor.

    http://www.cisco.com/en/US/products/ps9670/index.html

  • 15 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    The built-in standalone software, Cisco Integrated Management Controller (IMC), manages Cisco UCS C-

    Series rack servers. When a Cisco UCS C-Series rack server is integrated with Cisco UCS Manager

    using the Cisco Nexus 2232PP, the management controller no longer manages the server. Instead, the

    server is managed by the Cisco UCS Manager software, through the Cisco UCS Manager GUI or the

    command-line interface (CLI). The Cisco Nexus 2232PP provides data and control traffic support for the

    integrated Cisco UCS C-Series server.

    Cisco Nexus 1000V Switch for Microsoft Hyper-V

    Cisco Nexus 1000V Switch provides a comprehensive and extensible architectural platform for VM and

    cloud networking. This switch is designed to accelerate server virtualization and multi-tenant cloud

    deployments in a secure and operationally transparent manner. Integrated into the Microsoft Windows

    Server 2012 R2 Hyper-V hypervisor and SCVMM, the Cisco Nexus 1000V provides these advantages:

    Advanced VM networking, based on the Cisco NX-OS operating system and IEEE 802.1Q switching technology

    Policy-based VM connectivity

    Mobile VM security and network policy

    Nondisruptive operating model for server virtualization and networking teams

    Virtualized network services, with Cisco vPath providing a single architecture for layer 4 through layer 7 network services, such as load balancing, firewall, and WAN acceleration

    These capabilities help make the VM a basic building block of the data center, with full switching

    capabilities and a variety of layer 4 through layer 7 services in both dedicated and multi-tenant cloud

    environments. With the introduction of Virtual Extensible LAN (VXLAN) on the Cisco Nexus 1000V,

    network isolation among VMs can scale beyond the limits of traditional VLANs for cloud-scale networking.

    Note: For more information about the Cisco Nexus 1000V Switch and the Cisco Nexus 1010 Virtual Services Appliance, refer to Cisco Nexus 1000V Switch for Microsoft Hyper-V and Cisco Nexus 1010 Virtual Services Appliance.

    Cisco Data Center Virtual Machine Fabric Extender

    Cisco Data Center VM Fabric Extender (VM-FEX) technology collapses virtual and physical switching

    infrastructures into a single, easy-to-manage environment that provides the following benefits:

    Simplified operations, eliminating the need for a separate virtual networking infrastructure

    Improved network security to limit VLAN proliferation

    Optimized network utilization to reduce broadcast domains

    Enhanced application performance, which offloads VM switching from the host CPU to application-specific integrated circuits (ASICs) on the parent switch

    VM-FEX is supported on Windows Server 2012 R2 Hyper-V hypervisors, and it fully supports workload

    mobility through Hyper-V quick migration and live migration.

    VM-FEX eliminates the virtual switch in the hypervisor by providing individual VMs with virtual ports on the

    physical network switch. VM I/O is sent directly to the upstream physical network switch, which takes full

    responsibility for VM switching and policy enforcement. This approach leads to consistent treatment for all

    network traffic, virtual or physical. VM-FEX collapses virtual and physical switching layers into one and

    reduces the number of network management points by an order of magnitude.

    Although software-based devices work extremely efficiently, they have unavoidable overhead on the I/O

    path. Software-based devices introduce latency, increase overall path length, and consume computing

    cycles. With the single-root I/O virtualization (SR-IOV) capability, part of the network adapter hardware is

    exposed inside the VM, and it provides a direct I/O path to the network hardware. For this reason, a

    vendor-specific driver must be loaded onto the VM to use the virtual function network adapter.

    http://www.cisco.com/c/en/us/products/switches/nexus-1000v-switch-microsoft-hyper-v/index.htmlhttp://www.cisco.com/en/US/products/ps10785/index.htmlhttp://www.cisco.com/en/US/products/ps10785/index.html

  • 16 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    4.10 Storage

    This section describes the solutions unified storage architecture.

    NetApp FAS and Clustered Data ONTAP

    NetApp offers a unified storage architecture. The term unified refers to a family of storage systems that

    simultaneously supports SAN storage (through FCoE, FC, and iSCSI) and NAS (through CIFS and NFS)

    across many operating environments, such as VMware, Windows, and UNIX

    environments. This single

    architecture provides access to data through industry-standard protocols, including NFS, CIFS, iSCSI,

    FCP, SCSI, and NDMP. Connectivity options include standard Ethernet (10/100/1000Mb or 10GbE) and

    FC (1, 2, 4, or 8 Gb/sec). In addition, all systems can be configured with high-performance solid-state

    drives (SSDs) or serial attached SCSI (SAS) disks for primary storage applications, low-cost serial ATA

    (SATA) disks for secondary applications (backup, archiving, and so on), or a mix of the different disk

    types.

    A storage system running Data ONTAP, also known as the storage controller, is the hardware device that

    sends and receives data to and from the host. This unit detects and gathers information about its own

    hardware configuration, storage system components, operating status, hardware failures, and other error

    conditions.

    The storage controller is highly redundantly connected to storage through disk shelves, which are the

    containers or device carriers that hold disks and associated hardware, such as power supplies,

    connectivity interfaces, and cabling.

    If storage requirements change over time, NetApp storage offers the flexibility to change quickly as

    needed without expensive and disruptive major equipment upgrades. This flexibility applies to a variety of

    types of changes:

    Physical changes, such as expansion of a controller to accept more disk shelves and subsequently more hard disk drives (HDDs) without an outage

    Logical or configuration changes, such as expansion of a RAID group to incorporate these new drives without requiring an outage

    Access-protocol changes, such as modification of a virtual representation of a hard drive to a host by changing a logical unit number (LUN) from FC access to iSCSI access, with no data movement required, but only a simple dismount of the FC LUN and a mount of the same LUN, using iSCSI

    In addition, a single copy of data can be shared between Windows and UNIX systems, with each

    environment allowed to access the data through native protocols and applications. In a system that was

    originally purchased with all-SATA disks for backup applications, high-performance SAS disks could be

    added to support primary storage applications, such as Oracle

    applications, Microsoft Exchange, or

    Microsoft SQL Server.

    NetApp clustered Data ONTAP expands this traditional flexibility by allowing the dynamic relocation of

    either the logical storage container or the volume through the volume move feature, as well as the

    reassignment of entire parity groups or aggregates through aggregate relocation. These features allow a

    truly nondisruptive architecture in which any component of the storage system can be upgraded, resized,

    or redesigned without disruption of the private cloud infrastructure.

    NetApp storage solutions provide redundancy and fault tolerance through clustered storage controllers

    and hot-swappable redundant components, such as cooling fans, power supplies, disk drives, and

    shelves. This highly available and flexible architecture enables customers to manage all data under one

    common infrastructure while meeting mission-critical uptime requirements.

    The storage efficiency built into Data ONTAP offers substantial space savings, allowing more data to be

    stored at lower cost. Data protection includes replication services, so that valuable data is backed up and

    recoverable from an alternative location. The following features provide storage efficiency and data

    protection:

  • 17 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Thin-provisioned volumes are created using virtual sizing. They appear to be provisioned at their full capacity but are actually created much smaller and use additional space only when it is needed. Extra unused storage is shared across all volumes, and the volumes can grow and shrink on demand.

    NetApp Snapshot copies are automatically scheduled point-in-time copies that write only changed blocks, with no performance penalty. Snapshot copies consume little storage space because only changes to the active file system are written. Individual files and directories can easily be recovered from any Snapshot copy, and the entire volume can be restored to any Snapshot state in seconds.

    NetApp FlexClone volumes are instant virtual copies of datasets that use almost no space. The clones are writable, but only changes to the original are stored, so they provide rapid, space-efficient creation of additional data copies well suited for test and development environments.

    Deduplication removes redundant data blocks in primary and secondary storage with flexible policies to determine when the deduplication process is run.

    Compression compresses data blocks. Compression can be run whether or not deduplication is enabled and can provide additional space savings, whether run alone or together with deduplication.

    NetApp SnapMirror volumes can be asynchronously replicated either within the cluster or to

    another cluster.

    All of these capabilities are exposed through the logical management construct of a storage virtual

    machine (SVM), formerly known as Vserver.

    Storage Virtual Machines

    The secure logical storage partition through which data is accessed in clustered Data ONTAP is known

    as an SVM. A cluster serves data through at least one and possibly multiple SVMs. An SVM is a logical

    abstraction that represents a set of physical resources of the cluster. Data volumes and logical interfaces

    (LIFs) are created and assigned to an SVM and can reside on any node in the cluster to which the SVM

    has been given access. An SVM can own resources on multiple nodes concurrently, and those resources

    can be moved nondisruptively from one node to another. For example, a flexible volume can be

    nondisruptively moved to a new node, and an aggregate, or a data LIF, can be transparently reassigned

    to a different physical network port. The SVM abstracts the cluster hardware and is not tied to specific

    physical hardware.

    An SVM is capable of supporting multiple data protocols concurrently. Volumes within the SVM can be

    joined together to form a single NAS namespace, which makes all of an SVMs data available to NFS and

    CIFS clients through a single share or mount point. For example, a 24-node cluster licensed for UNIX and

    Microsoft Windows File Services that has a single SVM configured with thousands of volumes can be

    accessed from a single network interface on one of the nodes. SVMs also support block-based protocols,

    and LUNs can be created and exported by using iSCSI, FC, or FCoE. Any or all of these data protocols

    can be configured for use within a given SVM.

    An SVM is a secure entity; therefore, it is aware of only the resources that have been assigned to it and

    has no knowledge of other SVMs and their respective resources. Each SVM operates as a separate and

    distinct entity with its own security domain. Tenants can manage the resources allocated to them through

    a delegated SVM administration account. Each SVM can connect to unique authentication zones, such as

    Active Directory (AD), Lightweight Directory Access Protocol (LDAP), or network interface service (NIS).

    An SVM is effectively isolated from other SVMs that share the same physical hardware.

    From a performance perspective, maximum IOPS and throughput levels can be set for each SVM by

    using QoS policy groups, which allow the cluster administrator to quantify the performance capabilities

    allocated to each SVM.

    Clustered Data ONTAP is highly scalable, and additional storage controllers and disks can easily be

    added to existing clusters to scale capacity and performance to meet increasing demands. Because the

    cluster contains virtual storage servers, SVMs are also highly scalable. As new nodes or aggregates are

  • 18 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    added to the cluster, the SVM can be nondisruptively configured to use them. New disk, cache, and

    network resources can be made available to the SVM to create new data volumes or to migrate existing

    workloads to these new resources to balance performance.

    This scalability also makes the SVM highly resilient. SVMs are no longer tied to the lifecycle of a given

    storage controller. As new replacement hardware is introduced, SVM resources can be moved

    nondisruptively from the old controllers to the new ones, and the old controllers can be retired from

    service while the SVM is still online and available to serve data.

    SVMs have three main components:

    Logical interfaces. All SVM networking is performed through LIFs created within the SVM. As logical constructs, LIFs are abstracted from the physical networking ports on which they reside.

    Flexible volumes. A flexible volume is the basic unit of storage for an SVM. An SVM has a root volume and can have one or more data volumes. Data volumes can be created in any aggregate that has been delegated by the cluster administrator for use by the SVM. Depending on the data protocols used by the SVM, volumes can contain LUNs for use with block protocols or files for use with NAS protocols, or both concurrently. For access by using NAS protocols, the volume must be added to the SVM namespace through the creation of a client-visible directory called a junction.

    Namespaces. Each SVM has a distinct namespace through which all of the NAS data shared from that SVM can be accessed. This namespace is essentially a map to all of the junctioned volumes for the SVM, regardless of the node or the aggregate on which they physically reside. Volumes can be joined at the root of the namespace or beneath other volumes that are part of the namespace hierarchy. For more information about namespaces, refer to NetApp TR-4129: Namespaces in Clustered Data ONTAP.

    For more information, refer to NetApp Data ONTAP 8 Operating System.

    4.11 Storage Options for Windows Server 2012 R2

    This section describes the storage options available for Windows Server 2012 R2, including the use of

    CSVs, SMB 3.0 continuously available file shares, and storage automation.

    Cluster Shared Volumes

    Windows Server 2008 R2 included the first version of Windows failover clustering to offer a distributed file

    access solution, allowing a single New Technology File System (NTFS) volume to be accessed

    simultaneously by multiple nodes in a cluster. Windows Server 2012 expanded on this base capability,

    introducing many new capabilities. Windows Server 2012 R2 has further expanded those base

    capabilities by adding the following features:

    Optimized CSV placement policies. Previous versions of Windows included a coordinator node that owned the physical disk resource and communicated with all other nodes for all I/O operations. In Windows Server 2012 R2, CSV ownership is automatically rebalanced whenever anything occurs that might affect CSV placement, such as a CSV failing over, a node joining the cluster, or a node being restarted. This mechanism keeps the cluster well balanced and maximizes the available I/O for all cluster resources.

    Increased CSV resiliency. Windows Server 2012 R2 adds a dedicated service to monitor the health of the CSV. With this service, if the node becomes unhealthy for any reason, the cluster automatically relocates the coordination services to a healthy node. In addition, the CSV services have been subdivided, with one service dedicated to regular file traffic, such as clients that access an NTFS share, and another service dedicated to handling internode traffic over the CSV network. These changes increase CVS resiliency and improve the scalability of SMB traffic between nodes.

    CSV cache. Windows Server 2012 added the capability to assign up to 20% of the total physical RAM to a read cache; however, the read cache was disabled by default. In Windows Server 2012 R2, the read cache is enabled by default, and it can now be configured to use up to 80% of the total RAM allocation.

    https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=89242&contentID=117703https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=89242&contentID=117703http://www.netapp.com/us/products/platform-os/data-ontap-8/index.aspx

  • 19 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Improved CSV diagnostics. Windows Server 2012 R2 now allows the state of each node to be viewed, enabling the administrator to see whether a node is in redirected I/O mode on a per-node basis.

    These enhancements, as well as others, help create an enterprise-ready hosting platform for

    organizations seeking to deploy traditional block storage. For a complete list of the enhancements made

    in Windows Server 2012 R2, refer to What's New in Failover Clustering in Windows Server 2012 R2.

    CSV Characteristics

    Table 3 shows the characteristics that are defined by NTFS and inherited by CSV.

    Table 3) CSV parameters.

    CSV Parameter Characteristic

    Maximum volume size 256TB

    Maximum number of partitions 128

    Directory structure Unrestricted

    Maximum number of files per CSV More than 4 billion

    Maximum number of VMs per CSV Unlimited

    CSV Sizing

    Because all cluster nodes can access all CSVs simultaneously, IT managers can now use standard LUN

    allocation methodologies, based on the performance and capacity requirements of the expected

    workloads. In general, isolating the VM OS I/O from the application data I/O is a good start. In addition, it

    is helpful to implement application-specific considerations, such as segregating the database I/O from the

    logging I/O and creating SAN volumes and storage pools that factor in the I/O profile (that is, random

    read-write operations rather than sequential write operations).

    The architecture of CSV differs from that of traditional clustered file systems, which frees it from common

    scalability limitations. Therefore, no special guidance is needed for scaling the number of Hyper-V nodes

    or VMs on a CSV volume. The important point to remember is that all VM virtual disks running on a

    particular CSV contend for storage I/O. For this reason, it is extremely important to give the CSV network

    appropriate priority. For more information, refer to Designating a Preferred Network for Cluster Shared

    Volumes Communication in the Microsoft TechNet Library.

    Performance

    Storage performance is a complex mix of drive, interface, controller, cache, protocol, SAN, host bus

    adapter (HBA), driver, and OS considerations. The overall performance of the storage architecture

    typically is measured in terms of maximum throughput (MB/sec) and/or maximum IOPS for a given

    latency or response time (in milliseconds [ms]). Although each of these performance measurements is

    important, IOPS for a given latency is the most relevant to server virtualization.

    Using NetApp VST uses NetApp Flash Cache

    technology. This deduplication-aware technology uses

    the flash-memory cache to intelligently store large numbers of recently accessed blocks. The NetApp VST

    model can significantly increase the performance of an array in servicing the I/O load (or challenge) of a

    boot storm or a steady-state event.

    NetApp FAS controllers use two techniques to optimize both write and read performance. Write

    performance is optimized by the NetApp WAFL (Write Anywhere File Layout) file system, which delivers

    writes to the RAID groups as a sequential stream fill stripe write operation, which is the most efficient

    http://technet.microsoft.com/en-us/library/dn265972.aspxhttp://technet.microsoft.com/en-us/library/ff182335(WS.10).aspxhttp://technet.microsoft.com/en-us/library/ff182335(WS.10).aspx

  • 20 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    method for destaging the write cache. This technique provides optimal disk use by reducing write latency.

    NetApp FAS controllers also use NetApp Flash Cache to optimize read operations.

    Multipathing

    Multipathing should be used in all cases. Generally, NetApp provides a device-specific module (DSM) on

    top of Windows Server 2012 R2 multipath I/O (MPIO) software that supports the NetApp storage platform.

    The NetApp DSM offers advanced active-active policies while providing precise failover and path

    recovery, as well as load balancing, for NetApp LUNs.

    Fibre Channel SAN

    FC is an option because it is a supported storage connection protocol. FC is a robust, mature storage

    protocol that supports multipathing through Microsoft MPIO and the NetApp DSM.

    iSCSI SAN

    As with an FC-connected SAN, which is naturally on its own isolated network, the iSCSI SAN must be on

    an isolated network, for both security and performance. Any networking standard practice for achieving

    this goal is acceptable, including a physically separate, dedicated storage network and a physically

    shared network with the iSCSI SAN running on a private VLAN. The switch hardware must provide class-

    of-service (CoS) or QoS guarantees for the private VLAN. In addition, iSCSI security and frame size

    settings can be applied through two methods:

    Encryption and authentication. If multiple clusters or systems are used on the same SAN, proper segregation or device isolation must be provided. In other words, the storage used by cluster A must be visible only to cluster A and not to any other cluster, and not to a node from a different cluster. NetApp recommends using a session-authentication protocol, such as Challenge Handshake Authentication Protocol (CHAP), to provide a degree of security as well as segregation. Mutual CHAP or IP Security (IPsec) can also be used.

    Jumbo frames. If they are supported at all points in the entire path of the iSCSI network, jumbo frames can increase throughput by up to 20%. Jumbo frames are supported in Hyper-V at the host and guest levels. If jumbo frames are not supported at any point in the network and this feature is enabled, the network device fragments the data packets and causes a decrease in performance.

    SMB 3.0 Continuously Available File Shares

    A major new component of clustered Data ONTAP 8.2 is support for the SMB 3.0 NAS protocol, which

    enables NetApp customers to use the SMB 3.0 features introduced with Windows Server 2012. With

    these new features, clustered Data ONTAP can be used to host a VMs virtual disks and configuration

    settings on a CIFS file share.

    The SMB 3.0 features implemented in clustered Data ONTAP 8.2 to support continuously available file

    shares and Hyper-V storage include the following:

    Persistent handles (continuously available file shares)

    Witness protocol

    Cluster client failover (CCF)

    Scale-out awareness

    Offloaded data transfer (ODX)

    Remote VSS

    Persistent Handles (Continuously Available File Shares)

    To enable continuous availability on a file share, the SMB client opens a file on behalf of the application,

    such as a VM running on a Hyper-V host, and requests persistent handles for the virtual hard disk format

  • 21 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    (VHDX) file. When the SMB server receives a request to open a file with a persistent handle, the SMB

    server retains sufficient information about the file handle, along with a unique resume key supplied by the

    SMB client. Persistent handle information is shared among the nodes in a cluster.

    In the case of a planned move of file share resources from one node to another, or in the case of node

    failure, the SMB client reconnects to an active and available node and reopens the file by using persistent

    handles. The application or the VM running on the SMB client computer does not experience any failures

    or errors during this operation. From a VM perspective, it appears that the I/O operations to the virtual

    disk were delayed for a short time, similar to a brief loss of connectivity to the disk; however, no disruption

    is noticed.

    Witness Protocol

    When an SMB server node fails, the SMB client usually relies on the Transmission Control Protocol (TCP)

    timeout to detect a failure of the file share resource, such as open file. SMB 3.0 allows variable values for

    TCP timeouts, and because the virtual disk is a critical resource, the VM running on a Hyper-V server

    needs fast detection of network resources failover. The Witness protocol significantly improves the SMB

    client reconnect time.

    During connection to a shared resource (TREE_CONNECT), the SMB server provides information about

    features enabled on the share: for instance, whether the resource is clustered, scaled out, and

    continuously available. The SMB client then requests this same data from other nodes. Upon receiving

    the information, the SMB client registers itself with the other node.

    In the event of a cluster node failure, the SMB client is already connected to another node that can detect

    the failure and then notify the SMB client. This feature saves the SMB client from having to wait until the

    TCP timeout ends and instead immediately initiates reconnection to the running node, reducing the

    amount of time that the client is disconnected from the resource. For VMs with virtual disks stored on

    such SMB shares, disk disconnection time is reduced to the point that the VM does not detect such

    disconnects as hardware failures.

    This feature is enabled on clustered Data ONTAP by default only if all best practices are followed and if a

    LIF is present on each node in the cluster in every SVM. Note also that the Witness protocol is used only

    with continuously available shares.

    Cluster Client Failover

    To increase redundancy in a VM environment, Hyper-V servers should be placed in a Microsoft failover

    cluster. When the Hyper-V server node running a VM fails, the VM is live migrated or moved to another

    node. Before CCF with SMB 3.0, a VM that moved to another cluster node was considered a new

    application instance. Connection of new application instances to files already open on file shares must

    wait until the TCP timeout ends and the file handle is closed. CCF enables the VM to open a virtual disk

    file on a file share and provide a unique application identifier. When a Hyper-V server cluster node fails,

    the VM starts on another Hyper-V server node and supplies the same application identifier, letting the

    SMB server close existing file handles. The SMB client can then reconnect to the previously open file.

    Scale-Out Awareness

    Clustered Data ONTAP is a scale-out architecture by design and provides the capability to serve data

    from multiple nodes. It brings additional data redundancy to the network and spreads the load of multiple

    SMB clients among multiple nodes in a cluster. Scale-out awareness allows SMB clients to connect to all

    nodes in the cluster and access the same data.

    Offloaded Data Transfer

    Although the ODX copy offload feature is not required to run a Hyper-V workload over SMB 3.0, this

    feature can drastically improve VM deployment time for typical deployments in which the customer needs

  • 22 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    to provision multiple VMs. The main advantages of this feature are that it is transparent to client machines

    and no data is sent over the network during file copy operations. Clustered Data ONTAP provides

    different mechanisms on the back end to copy data blocks. In the case of a single volume that serves a

    file share, NetApp uses its single-instance storage (SIS) clone feature, which eliminates the data copy

    process by creating only pointers. This feature accelerates back-end operations and significantly

    improves copy performance when ODX is used on the NetApp platform, compared to ODX

    implementations on other storage arrays. When data is copied outside the volume, the process remains

    offloaded, and no traffic travels through the client or the network.

    Remote Volume Shadow Copy Service

    VSS is a framework that coordinates application I/O and physical storage on the same server and allows

    creation of application-consistent Snapshot copies of the storage. Microsoft Windows Server 2012 R2

    extends the functions of VSS to multiple servers. For instance, an application running on one server has

    storage on another servers file share. Remote VSS coordinates I/O activities during a backup process

    between both servers and provides application-consistent backup Snapshot copies of the storage for

    applications running remotely on the storage server. Clustered Data ONTAP 8.2 extends the functions of

    remote VSS by plugging into the VSS framework; a VSS service runs on a NetApp controller, and a VSS

    provider runs on a Windows Server 2012 R2 device. From the perspective of a VSS, the NetApp array

    performs in the same way as a Windows file server.

    Storage Automation

    One objective of the Microsoft Private Cloud solution is to enable rapid provisioning and deprovisioning of

    VMs. Doing so on a large scale requires tight integration with the storage architecture as well as robust

    automation. Provisioning a new VM on a preexisting LUN is a simple operation. However, provisioning a

    new CSV LUN and adding it to a host cluster are relatively complicated tasks that should be automated.

    Historically, many storage vendors have designed and implemented their own storage management

    systems, APIs, and command-line utilities. This has made it a challenge to use a common set of tools and

    scripts across heterogeneous storage solutions.

    To address this challenge, NetApp supports the Microsoft management tools and APIs shown in Figure 3.

    Specifically, NetApp provides the Data ONTAP PowerShell toolkit, which allows the management of

    NetApp controllers from Microsoft Windows PowerShell in addition to the standards-based management

    offered in SMI-S.

  • 23 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Figure 3) NetApp integration with Microsoft.

    4.12 Virtual Infrastructure

    The solutions virtual infrastructure includes Microsoft Windows Server 2012 R2 Hyper-V and Microsoft

    System Center 2012 R2.

    Microsoft Windows Server 2012 R2 Hyper-V

    Microsoft Windows Server 2012 R2 Hyper-V provides significant scalability and expands support for host

    processors and memory. It includes the following new features:

    Support for up to 64 processors and 1TB of memory for Hyper-V VMs, including in many cases supporting 4 to 16 times the density of processors, memory, cluster nodes, and running VMs

    Support for innovative server features, including the ability to project a virtual nonuniform memory access (NUMA) topology onto a VM to provide optimal performance and workload scalability in large VM configurations

    Improvements to dynamic memory, including minimum memory and Hyper-V smart paging

    Note: Minimum memory allows Hyper-V to reclaim the unused memory from VMs to allow higher VM consolidation numbers. Smart paging is used to bridge the memory gap between minimum and startup memory by allowing VMs to start reliably when the minimum memory setting has indirectly led to an insufficient amount of available physical memory during restart.

    Runtime configuration of memory settings, including increasing the maximum memory and decreasing the minimum memory of running VMs

    The following updated features help the virtualization infrastructure support the configuration of large

    high-performance VMs to maintain demanding workloads:

  • 24 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    VHDX offers greater capacity (up to 64TB of storage), helps provide additional protection from corruption from power failures, and prevents performance degradation on large-sector physical disks by optimizing structure alignment.

    Virtual Fibre Channel (VFC) support offers VMs unmediated access to SAN LUNs. VFC enables scenarios such as running the Windows Failover Cluster Management feature inside the guest OS of a VM connected to shared FC storage. VFC supports MPIO, NPIV for one-to-many mappings, and up to four VFC adapters per VM.

    Microsoft Windows Server 2012 R2 includes the following networking enhancements:

    Support for SR-IOV

    Third-party extensions to the Hyper-V extensible switch

    QoS minimum bandwidth

    Network virtualization

    IEEE Data Center Bridging (DCB)

    The virtualization layer is one of the primary enablers in environments with greater IT maturity. The

    decoupling of hardware, OSs, data, applications, and user state opens a wide range of options for easier

    management and distribution of workloads across the physical infrastructure. The capability of the

    virtualization layer to migrate running VMs from one server to another without downtime, along with many

    other features provided by hypervisor-based virtualization technologies, enables a comprehensive set of

    solution capabilities. These capabilities can be used by the automation, management, and orchestration

    layers to maintain desired states and to proactively address decaying hardware or other issues that would

    otherwise cause faults or service disruptions.

    Like the hardware layer, the automation, management, and orchestration layers must be able to manage

    the virtualization layer. Virtualization provides an abstraction of software from hardware that moves most

    management and automation operations to software instead of requiring users to perform manual

    operations on physical hardware.

    With this release, Windows Server 2012 Hyper-V introduces a number of improvements in both

    virtualization features and scalability. Table 4 compares scalability improvements and feature

    enhancements.

    Table 4) Feature comparison of Microsoft Hyper-V capabilities.

    Feature Windows Server 2008 Windows Server 2008 R2

    Windows Server 2012 R2

    Scale

    Hardware logical processor support

    16 logical processors 64 logical processors 320 logical processors

    Physical memory support 1TB 1TB 4TB

    Cluster scale 16 nodes and up to 1,000 VMs

    16 nodes and up to 1,000 VMs

    64 nodes and up to 4,000 VMs

    VM processor support Up to 4 virtual processors Up to 4 virtual processors Up to 64 virtual processors

    VM memory Up to 64GB Up to 64GB Up to 1TB

    Live migration Yes, one at a time Yes, one at a time Yes, with no limits up to as many as the hardware allows

  • 25 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Feature Windows Server 2008 Windows Server 2008 R2

    Windows Server 2012 R2

    Servers in a cluster 16 16 64

    Virtual processor to logical processor ratio

    8:1 8:1 for server No limits, up to as many as the hardware allows

    12:1 for client (VDI)

    Storage

    Live storage migration No; quick storage migration through Microsoft SCVMM

    No; quick storage migration through Microsoft SCVMM

    Yes, with no limits, up to as many as the hardware allows

    VMs on file storage No No Yes, SMB 3

    Guest FC No No Yes

    Virtual disk format Virtual hard disk (VHD) up to 2TB

    VHD up to 2TB VHD up to 2TB

    VHDX up to 64TB

    VM guest clustering Yes, by using iSCSI Yes, by using iSCSI Yes, by using iSCSI, FC, or SMB

    Native 4,000-disk support No No Yes

    Live VHD merge No, offline No, offline Yes

    Live new parent No No Yes

    Secure ODX No No Yes

    Networking

    Network interface card (NIC) teaming

    Yes, by way of partners Yes, by way of partners Windows NIC teaming in box

    VLAN tagging Yes Yes Yes

    MAC address spoofing protection

    No Yes, with R2 SP1 Yes

    Address Resolution Protocol (ARP) spoofing protection

    No Yes, with R2 SP1 Yes

    SR-IOV networking No No Yes

    Network QoS No No Yes

    Network metering No No Yes

    Network monitor modes No No Yes

    IPsec task offload No No Yes

    VM trunk mode No No Yes

    Manageability

    Hyper-V PowerShell No No Yes

  • 26 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Feature Windows Server 2008 Windows Server 2008 R2

    Windows Server 2012 R2

    Network PowerShell No No Yes

    Storage PowerShell No No Yes

    SCONFIG No Yes Yes

    Enable and disable shell No (server core at OS setup)

    No (server core at OS setup)

    Yes; additional minimal shell environment (MINSHELL)

    Microsoft Windows VMConnect support for Microsoft RemoteFX

    No Yes

    The Hyper-V host cluster requires different types of network access, as described in Table 5.

    Table 5) Host cluster networks.

    Network Access Type Purpose of Network Access Type Network Traffic Requirements

    Recommended Network Access

    VM access Workloads running on VMs usually require external network connectivity to service client requests.

    Varies Public access that can be teamed for link aggregation or to fail over the cluster

    Clusters and CSVs This is the preferred network used by the cluster for communications to maintain cluster health. This network is also used by CSV to send data between owner and nonowner nodes. If storage access is interrupted, this network is used to access CSV or to maintain and back up CSV.

    The cluster should have access to more than one network for communication to make it highly available.

    Usually low bandwidth and low latency; occasionally, high bandwidth

    Private access

    SMB 3.0 Access storage through SMB 3.0. High bandwidth and low latency

    Usually, dedicated and private access

    Live migration Transfer VM memory and state. High bandwidth and low latency during migrations

    Private access

    Storage Access storage through iSCSI. High bandwidth and low latency

    Usually, dedicated and private access

  • 27 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Network Access Type Purpose of Network Access Type Network Traffic Requirements

    Recommended Network Access

    Management Manage the Hyper-V management OS; this network is used by Hyper-V Manager.

    Low bandwidth Public access that can be teamed to fail over the cluster

    Highly available host servers are one critical component of a dynamic virtual infrastructure. A Hyper-V

    host failover cluster is a group of independent servers that work together to increase the availability of

    applications and services. The clustered servers (nodes) are connected physically. If one cluster node

    fails, another node begins to provide service. In the case of a planned live migration, users experience no

    perceptible service interruption.

    NetApp Microsoft Hyper-V Storage Options

    Storage for virtual disks (VHDX files) can be provided from the NetApp storage system in the following

    ways:

    Block-level LUN over FCP or iSCSI attached directly to Hyper-V servers, presented as volumes on a standalone Hyper-V server or as CSVs in a Microsoft failover cluster

    File-level storage on a NetApp SMB 3.0 continuously available file share

    NetApp Integration with Microsoft Windows Server 2012 R2

    NetApp provides tight integration between the storage (block or file level) resources and the Windows

    Server 2012 R2 host through the following technologies and products:

    The ODX feature works transparently on a Windows Server 2012 host and provides much faster provisioning of VMs from the master image: migrating, moving, importing, and exporting either the whole VM or only the VM storage. On NetApp arrays, ODX works across protocols and between file and block storage, which allows a mix of storage options for Windows 2012 Hyper-V VM storage.

    The NetApp SMI-S agent provides a unified storage management interface that can be used to discover, monitor, and manage NetApp storage systems. It provides transparent integration of NetApp storage into Windows Server 2012 R2 and Microsoft SCVMM.

    SnapManager 2.02 for Hyper-V with remote VSS capabilities protects VM resources running on block-level attached LUNs and CSVs and on remote SMB 3.0 file shares. It uses NetApp Snapshot technology to offload the backup process from the Hyper-V host to the NetApp storage system. It can use NetApp SnapMirror technologies for off-site backup operations at remote locations. This tool has an easy-to-use management interface, along with a set of Windows PowerShell cmdlets for robust automation.

    Thin provisioning is a part of the storage efficiency technologies that, along with deduplication, reduce both the allocated disk space and the overall cost of storage.

    Microsoft System Center 2012 R2

    Microsoft System Center 2012 R2 helps organizations deliver flexible and cost-effective private cloud

    infrastructure in a self-service model, using existing data center hardware and software. It provides a

    common management experience across data centers and private-hosted or partner-hosted clouds. To

    deliver the best experience for modern applications, Microsoft System Center 2012 R2 offers deep insight

    into applications, down to the level of client script performance. System Center 2012 R2 delivers the tools

    and capabilities that organizations need to scale their capacity and, where necessary, to use cloud

    resources as well.

  • 28 FlexPod Datacenter with Microsoft Private Cloud 2014 NetApp, Inc. All Rights Reserved.

    Microsoft System Center 2012 R2 offers unique application management capabilities that can deliver

    agile, predictable application services. Using the App Controller, Operations Manager, and SCVMM

    components of System Center 2012 R2, applications can be delivered as a service, with the service as a

    deployed instance of a cloud-style application, along with its associated configuration and virtual

    infrastructure.

    Microsoft System Center 2012 R2 includes the application management capabilities described in the

    following sections.

    Standardized Application Provisioning

    Microsoft SCVMM offers service templates to help define standardized application blueprints. A service

    template typically includes specifications for the hardware, the OS, and the application packages that

    compose the service.

    SCVMM supports multiple package types for Microsoft .NET applications, including Microsoft Deploy

    (msdeploy) for the web tier (Microsoft IIS), Microsoft Server Application Virtualization (Server App-V) for

    the application tier, and SQL Server dedicated administrator connection (DAC) for the data tier. It also

    specifies application configuration requirements, such as topology, elasticity, scale-out rules, health

    thresholds, and upgrade rules.

    Server App-V, a unique technology in SCVMM, optimizes applications for private cloud deployments by

    abstracting the application from the underlying OS and virtual infrastructure. By enabling image-based

    management, Server App-V simplifies application upgrades and maintenance.

    Comprehensive Hybrid Application Management

    Microsoft App Controller offers application owners a single view to manage application services and VMs,

    whether they are on premises, at the location of service providers, or using Microsoft Windows Azure.