HP ConvergedSystem 300 for Microsoft Analytics Platform ... · • HP ConvergedSystem 300 for Microsoft Analytics Platform offers flexibility to scale to large amounts of data, up

Technical white paper

HP ConvergedSystem 300 for Microsoft Analytics Platform solution architecture Extend the power of information to everyone to speed business advantage

Table of contents Executive summary ...................................................................................................................................................................... 2

Solution overview .......................................................................................................................................................................... 3

Design objectives ...................................................................................................................................................................... 4

Components ............................................................................................................................................................................... 5

Solution hardware ......................................................................................................................................................................... 6

A “Quarter Rack” minimum configuration ........................................................................................................................... 7

A “Full Rack” configuration ...................................................................................................................................................... 8

HP Intelligent Infrastructure ................................................................................................................................................. 10

Management/Failover servers ............................................................................................................................................. 10

Data scale units ....................................................................................................................................................................... 10

Networking components ....................................................................................................................................................... 12

Management software ............................................................................................................................................................... 14

Infrastructure overview – RTM through Appliance Update 2 (AU2) ............................................................................. 14

Infrastructure overview – Appliance Update 3 (AU3) ...................................................................................................... 17

Configuration management ................................................................................................................................................. 18

Solution management ........................................................................................................................................................... 19

HP Support Pack for the HP ConvergedSystem 300 for Microsoft Analytics Platform ........................................... 22

Services.......................................................................................................................................................................................... 23

Factory Express ....................................................................................................................................................................... 23

Support services ...................................................................................................................................................................... 23

Licensing considerations ........................................................................................................................................................... 24

Appendix A – Solution hardware configuration .................................................................................................................... 24

Appendix B – Power diagram .................................................................................................................................................... 25

For more information ................................................................................................................................................................. 27

Technical white paper | HP ConvergedSystem 300 for Microsoft Analytics Platform solution architecture

2

Executive summary

HP ConvergedSystem 300 for Microsoft Analytics Platform addresses the key needs of business insights through greater understanding of business data. It is pre-configured and the only purpose-built solution customized for Data Warehouse (DW) and Hadoop, with the HP Support Pack to meet a range of business needs. The HP ConvergedSystem 300 for Microsoft Analytics Platform can be easily and rapidly deployed to support both large relational database solutions with Microsoft® SQL Server Parallel Data Warehouse, and HDInsight, the enterprise-ready Hadoop-based distribution from Microsoft that brings a 100% Apache Hadoop solution to the data warehouse. All required components, including servers, storage, and networking, are integrated into a single ConvergedSystem that is expandable to meet virtually any big data challenge.

HP ConvergedSystem 300 for Microsoft Analytics Platform combines the best in breed HP hardware solution with the software from Microsoft, along with world class support provided by both companies. The HP ConvergedSystem 300 for Microsoft Analytics Platform allows you to start small and grow your ConvergedSystem later as business needs change. The solution provides great flexibility by allowing a customer to have a SQL Server Parallel Data Warehouse solution running in the same ConvergedSystem as the HDInsight Hadoop data cluster. This gives customers the option of analyzing both structured and unstructured data sets within the same ConvergedSystem platform.

Figure 1. The solution scales from a “quarter rack” up to 7 racks

HP ConvergedSystem 300 for Microsoft Analytics Platform is available as a system from as small as a “quarter rack” solution all the way up to 56 Active Scale Unit nodes or 64 total nodes per workload region for either Microsoft SQL Server Parallel Data Warehouse (PDW) workloads or HDInsight Hadoop data technologies (HDI) within the solution, for a total of up to 7 racks of hardware for a single HP ConvergedSystem 300 for Microsoft Analytics Platform.

The solution offers up to 6 Petabytes (PB) of data storage for the PDW “region”, and up to 1.2PB of storage for Hadoop data. Storage drive choices of 1TB, 2TB, or 3TB are available to help meet varying business requirements for data capacity.

Delivers scalability and ground breaking performance • HP ConvergedSystem 300 for Microsoft Analytics Platform offers flexibility to scale to large amounts of data, up to 6PB

(PDW region)/1.2PB (HDI region).

• Designed with HP servers and storage to deliver superb performance with all components balanced to reduce bottlenecks.

• The HP ConvergedSystem 300 for Microsoft Analytics Platform has been designed with a dense server and storage configuration. Based on our market leading Converged Infrastructure (CI), the solution has been designed from the ground up.

Achieves time-to-application value with lowered risk HP and Microsoft have jointly developed, configured, tested, tuned, and validated the HP ConvergedSystem 300 for Microsoft Analytics Platform, resulting in reduced implementation risk and cost.

• Each HP ConvergedSystem 300 for Microsoft Analytics Platform is delivered along with a unique collaborative support agreement between HP and Microsoft. With a unified solution support experience, disruptions are minimized and unnecessary downtime avoided.

• Included Deployment and Accelerator Service for the HP ConvergedSystem 300 for Microsoft Analytics Platform facilitates a proper customer site assessment and on-site deployment, which is performed to allow smooth installation and allow you to focus on business issues rather than infrastructure.


3

• Automated installation and validation tools, utilized at your site by a team of HP and Microsoft engineers, provide consistent and error free deployment. All software is preinstalled and the hardware components are preconfigured in the HP factory.

Incorporates structured and unstructured (Hadoop) data

Integrates HDInsight technologies from Microsoft for an on-premise Hadoop implementation:

• Microsoft SQL Server Parallel Data Warehouse (PDW) contains PolyBase a fundamental breakthrough in the data processing engine. PolyBase provides the easiest and broadest way to access Hadoop with the standard SQL query language. A single T-SQL query model for PDW and Hadoop with the rich features of T-SQL, including joins without extract, transforms, and load (ETL) operations.

• Integration of the parallel processing of both the Massively Parallel Processing (MPP) database engine and Hadoop enhance query execution performance that joins relational data with Hadoop data. PolyBase also seamlessly moves data in parallel to and from Hadoop and PDW – allowing end users to perform their analysis without the help of IT.

Target audience: The target audience for this document is chief information officers (CIOs), chief technology officers

(CTOs), IT directors, IT architects, database administrators, business intelligence (BI) administrators, and others wishing to learn more about this solution.

This document assumes the reader has a basic understanding of several key data center technologies including, but not limited to: servers, storage, networking, power, solution management, virtualization, and hypervisors.

Solution overview

Understanding business data continues to be a challenge at most large enterprise companies. The challenges of attaining insight from their existing data, as well as integrating newer “big data” sources of information, continue to challenge IT departments. The volumes of data continue to expand, and the answers needed to achieve new business insight require more and more computing power. Data is available in multiple source systems, including non-traditional data sources such as Hadoop data clusters, as well as traditional relational data warehouse systems. Data must be accessed quickly, securely, and reliably to address the needed business insights, ultimately contributing to better insights to the enterprise.

Fortunately, the HP ConvergedSystem 300 for Microsoft Analytics Platform addresses these needs. The HP ConvergedSystem 300 for Microsoft Analytics Platform solution provides a combination of HP hardware, HP Software, and software from Microsoft to provide a massively parallel processing (MPP) version of the Microsoft SQL Server platform, known as Microsoft SQL Server Parallel Data Warehouse. This solution provides customers the ability to handle very large volumes of data (up to 6 Petabytes) in a single solution, providing lightning fast query responses to even the most difficult questions. Fast Data loading, secure data storage, and fast data loading solutions combine to provide an enterprise ready data warehousing platform. Additionally, the Microsoft Business Intelligence solutions integrate quickly and easily into the HP ConvergedSystem 300 for Microsoft Analytics Platform solution, providing an enterprise data warehousing framework for reporting, data analysis, and self-service BI, even allowing Microsoft Excel query integration directly into the HP ConvergedSystem 300 for Microsoft Analytics Platform.

Part of the infrastructure of many big data solutions has become the rapidly expanding world of Hadoop. SQL Server Parallel Data Warehouse introduced the PolyBase query engine to combine Hadoop data into the existing SQL Server PDW relational data store. Now with the HP ConvergedSystem 300 for Microsoft Analytics Platform System, you can optionally integrate a Hadoop data solution (HDInsight, or HDI) directly. The solution can have both a PDW MPP region and an HDInsight region, to provide the best of both worlds of data storage and querying. Data can be queried from the PDW side and join HDI data directly to provide insights not previously available with unstructured and semi-structured data storage.

The HP ConvergedSystem 300 for Microsoft Analytics Platform solution offering includes the following:

• HP Infrastructure

– Two InfiniBand switches for fast data transfers and redundancy

– Two Ethernet switches for management and querying capabilities to the solution

– HP ProLiant DL360p Gen8 servers hosting both SQL Server and HDI services

– HP D6000 Disk Enclosures to service the very large data volumes

– Intelligent power modules for redundant power to components


4

• Microsoft SQL Server Parallel Data Warehouse Edition

– Software to provide the SQL Server relational database services in the HP ConvergedSystem 300 for Microsoft Analytics Platform

– Data Management software to control parallel operations

– Management software, including Active Directory and System Center for internal management of the solution

– Client tools for secure, reliable data access

• Microsoft HDInsight Software

– Hadoop core infrastructure to provide data storage and retrieval for non-traditional data

• Including HDFS and MapReduce

– Hive, Hcatalog, Pig, Oozie, and Sqoop for enhanced Hadoop functionality

• HP Management Software

– The Support Pack for the HP ConvergedSystem 300 for Microsoft Analytics Platform solution to provide:

• Tools for ConvergedSystem validation and diagnostics

• Reporting tools for health and proactive care

• Validated Firmware/Driver packages for version control and updates

– HP Insight Remote Support for proactive customer care

• HP Services

– A collaborative support model with Microsoft for a single phone call to support for all aspects of the solution

– Factory integration and build of all components, including software deployment

– Delivered fully built and integrated

– On-site installation and configuration included with the solution

– Optional consulting services

Design objectives

This section outlines key design objectives for the HP ConvergedSystem 300 for Microsoft Analytics Platform.

Leadership performance Powered by HP DL360p Gen8 (2 x Intel® Xeon® E5-2690) servers, the solution provides excellent performance and scalability. The solution can be expanded with up to 56 Active Scale Unit nodes or 64 total nodes per workload region. All these servers work together to provide massively parallel performance through a single, integrated interface.

Realistic sizing

The HP ConvergedSystem 300 for Microsoft Analytics Platform must start with PDW Region Base Scale Unit (1/4 rack) with 76TB of storage (based on 1TB disks, 5:1 compression), scaling up to 6PB (based on 3TB disks, 5:1 compression) of storage on a fully populated solution.

Customers can extend to add an HDI Region Base Scale Unit which can scale up to 1.2PB of Hadoop data. After the initial PDW Region Base Scale unit, you can scale with your choice of data scale units (either PDW or HDInsight). This allows a customer to start small and grow their HP ConvergedSystem 300 for Microsoft Analytics Platform as their data sizing needs grow.


5

Figure 2. Expansion based on need

High availability

The HP ConvergedSystem 300 for Microsoft Analytics Platform solution provides for high availability at all levels. The entire solution is built around the assumption of providing highly available data warehousing with maximum uptime. The solution includes:

• Failover clustering: all physical servers in the PDW region are part of a Windows® failover cluster, which includes multiple redundant failover servers as the solution scales up

• Failover of all PDW infrastructure, which is all contained within virtual machines within the HP ConvergedSystem 300 for Microsoft Analytics Platform solution

• Failover clustering of the virtualized management infrastructure for the optional HDInsight region

• Mirrored disks for all management servers, host operating systems, and PDW data storage

• Multiple copies of the data provide redundancy for HDI data nodes

• Redundant Ethernet switches

• Redundant InfiniBand switches

• Redundant power design with redundant HP Intelligent Power Distribution Units and power supplies

Components

The HP ConvergedSystem 300 for Microsoft Analytics Platform delivers the following key components:

Hardware • HP Intelligent Infrastructure for intelligent rack and power infrastructure management

– Up to 7 racks used for a fully populated PDW and HDI solution

• HP DL360p Gen8 servers with 2 x E5-2690 processors, with 256GB memory per server

– Two HP 600GB 6G SAS 10K 2.5 inch HDD per server

– Used for hosting management components and failover capabilities

– Two servers minimum for PDW region

– Two servers total for HDI region

• Up to 28 “Scale Units”, consisting of:

– Two HP DL360p Gen8 servers with 2 x E5-2690 processors, with 256GB memory per server

• Two HP 600GB 6G SAS 10K 2.5 inch HDD per server

– D6000 Disk Enclosure with seventy (70) disks per enclosure

• Disk are either 1TB, 2TB, or 3TB 6G SAS 7.2K 3.5 inch DP MDL HDD


6

• HP 5120-24G El Ethernet switches (2)

• Mellanox IB FDR 36P Managed Switch (2)

• HP Intelligent Power Distribution Units (iPDUs)

Management software

• Windows Server 2012 R2 Standard Edition on all physical servers and virtual machines

• Active Directory for administrative control and security

• System Center 2012 Virtual Machine Manager

• Microsoft SQL Server Parallel Data Warehouse Edition

• Microsoft HDInsight software (optional)

• HP Integrated Lights-Out (iLO) Advanced

• HP Support Pack for the HP ConvergedSystem 300 for Microsoft Analytics Platform

Services

• HP Factory Express Services

• HP Deployment Accelerator Service

• HP Proactive Care (included) or HP Critical Service support offering

Solution hardware

HP ConvergedSystem 300 for Microsoft Analytics Platform has both fixed components and expandable scale units to scale the solution from as small as a single scale unit (sometimes referred to as a quarter rack) up to a seven (7) rack solution, depending upon your scalability needs.

This section will show several typical solution packages, with an explanation of the footprint, components, and scalability of each solution.

Figure 3. Components of a “Base” or “quarter rack” system


7

A “Quarter Rack” minimum configuration

A “quarter rack” configuration (Figure 4) is the minimum HP ConvergedSystem 300 for Microsoft Analytics Platform solution that can be purchased. The solution includes the InfiniBand and Ethernet switches, a management server and a failover server (HST01 and HST02, respectively), and a single scale unit. This configuration is also referred to as a 2 + 2 configuration, because there are two scale unit servers + two failover servers (including the PDW Orchestration node (HST01)).

Figure 4. Front and rear views of a “Quarter Rack” HP ConvergedSystem 300 for Microsoft Analytics Platform

HST01HST02

HSA02HSA01

R1S1

R1ETHSW2R1ETHSW1

R1IBSW2R1IBSW1

For a listing of the components included in the solution, see Table 1 in Appendix A – Solution hardware configuration.


8

Figure 5. Front view of a “Full Rack” configuration

A “Full Rack” configuration

In this “full rack” configuration (Figure 5), all possible servers for an HP ConvergedSystem 300 for Microsoft Analytics Platform solution that can be purchased for Rack #1 are included. The solution includes the InfiniBand and Ethernet switches, a management server (HST01), two failover servers (HST02 and HST05), and four scale units.

This configuration also has the HDI Region. We know this because the HDI management server (HST03) and the HDI Failover node (HST04) are present. Therefore, at least one scale unit is for the HDInsight region. As mentioned earlier, it is required that PDW be present, therefore we also know at least one scale unit is dedicated to the PDW region. Otherwise, the hardware is identical between these regions and you must examine the software running on the servers to determine their role.

This configuration is also referred to as an 8 + 5 configuration, because there are eight scale unit servers + five failover servers (including the PDW and HDI Orchestration nodes and their failover servers).


9

Note that if this were a secondary rack, the scale units would be identical, along with the InfiniBand and Ethernet switches at the top. However, there would be at most a single failover server at the top of the rack, in the position of HST01 in this diagram. In the secondary racks, however, the server name would simply increment. In the full rack example below, when a second rack of hardware is added to this solution, the failover server at the top of the rack would be HST06 (since that’s the next available number in the sequence).

The HSAxx servers would continue to be counted up from the bottom of the rack, HSA09-HSA16 in rack #2. This pattern continues through the remaining racks of a single HP ConvergedSystem 300 for Microsoft Analytics Platform solution. Finally, if the added rack only contains HDI server scale units, there is no additional passive server mounted at the top of the rack. The additional failover server is only needed for the PDW software components.

Figure 6. Front and rear views of a Full Rack #1 in an HP ConvergedSystem 300 for Microsoft Analytics Platform

HST01HST02

HSA02HSA01

R1S1

R1ETHSW2R1ETHSW1

R1IBSW2R1IBSW1

HSA04HSA03

R1S2

HSA06HSA05

R1S3

HSA08HSA07

R1S4

HST03HST04HST05


10

For a listing of the components included in the solution, see Table 1 in Appendix A – Solution hardware configuration.

HP Intelligent Infrastructure

Workload and management servers for HP ConvergedSystem 300 for Microsoft Analytics Platform are housed within HP Intelligent Infrastructure, which provides intelligent rack and power infrastructure management. For example, the HP Intelligent Series Rack family offers next-generation, enterprise-class racks that are designed to meet both the current and future requirements of the demanding data center.

These racks offer innovative features such as the following:

• Location discovery intelligence for advanced asset management

• Integration with HP Integrated Lights-Out (iLO) and HP Intelligent Power Distribution Units (iPDUs) helps you identify the U-space, row, and rack location of each server

• Unparalleled structural integrity, providing optimal support for the HP ConvergedSystem 300 for Microsoft Analytics Platform solution – in shipment and at your site

• Cooling and cable management advances

• Wide choice of power and switching options

Innovative iPDUs bring state-of-the-art management and control to rack mount solutions, helping eliminate the over-provisioning of power in the data center. These iPDUs deliver the following features:

• Accurate measurement of power, data collection, alerts, and links to HP Insight Control management

• Remote monitoring and control using a standard web browser

• Intelligent discovery and power monitoring via communications with HP Common Slot Platinum Power Supplies

• Identification and auto-mapping of redundant power supplies to reduce the risk of human error

For full redundancy, power distribution in the HP Intelligent Series Racks used in HP ConvergedSystem 300 for Microsoft Analytics Platform is split into two separate power buses. You can specify 24A or 32A single-phase iPDUs, or 32A or 40A three-phase iPDUs.

More information on power and cabling can be found in Appendix B – Power diagram.

Management/Failover servers

A ConvergedSystem 300 for Microsoft Analytics Platform uses redundant DL360p Gen8 servers as management servers, delivering exemplary performance and efficiency in a compact 1U size. This server is designed for high-performance workloads, with the adaptability you need to satisfy future requirements. In HP ConvergedSystem 300 for Microsoft Analytics Platform, each DL360p Gen8 server has two Intel Xeon Processor E5-2690 (8-core 2.90GHz processors with 20MB cache) and 256GB RAM.

Each server has redundant connections to both the Ethernet and InfiniBand networks, as well as redundant power supplies. The local storage for the operating system consists of two HP 600GB SAS 10K RPM 2.5 inch HDD in a mirrored (RAID 1) configuration for additional protection.

Data scale units

A data scale unit, as noted above, consists of two HP DL360p Gen8 servers and an HP D6000 Disk Enclosure, and is the basic unit of storage and processing scalability for PDW and HDInisght regions.

Data scale unit servers

A ConvergedSystem 300 for Microsoft Analytics Platform uses two DL360p Gen8 servers as scale unit servers, delivering exemplary performance and efficiency in a compact 1U size. This server is designed for high-performance workloads, with the adaptability you need to satisfy future requirements. In HP ConvergedSystem 300 for Microsoft Analytics Platform, each DL360p Gen8 server has two Intel Xeon Processor E5-2690 and 256GB RAM.

These servers each run a separate, virtualized workload, and differ from the management servers as they include an HP H221 Host Bus Adapter for connectivity to the HP D6000 Disk Enclosure.

http://www8.hp.com/us/en/products/servers/proliant-servers.html?compURI=1565918

http://h18004.www1.hp.com/products/servers/rackandpower/powersupplies/ipdu/index.html

http://h18004.www1.hp.com/products/servers/rackandpower/powersupplies/ipdu/index.html


11

Data scale unit storage

The HP D6000 Disk Enclosure provides data storage within each scale unit. The D6000 is populated with seventy (70) HP 6G SAS 7.2K RPM DP MDL HDD, in 1TB, 2TB, or 3TB sizes. This provides each HP D6000 Disk Enclosure with 76TB, 151TB, or, 227TB of user data capacity, respectively.

Note User data capacity is calculated based on disks allocated to database space, factored in the formatted capacity and RAID1, as well as 5x compression ratio.

Note SQL Server Parallel Data Warehouse uses the newest version of in-memory columnstore, which is both updateable and clustered. By reorienting data in a column rather than a traditional row store, organizations can see next-generation performance (up to 50 times) by reducing the query time from hours to seconds and high compression on their data (up to 15 times). This is due to the grouping of data in a columnar format (containing values from multiple rows) that allows for higher efficiency in both storage and the returned dataset. The ability to permanently convert traditional row store tables with the updateable columnstore of xVelocity is another new feature, which dramatically reduces storage (by up to 70 percent) while still providing real-time data warehouse query performance.

With the 6Gb/s SAS HP D6000 Disk Enclosure, HP extends and redefines direct-attached storage, combining the simplicity and cost effectiveness of direct-attached storage without sacrificing flexibility or performance. The HP D6000 is used as part of a straight-forward in-rack 6Gb/s SAS implementation that delivers high-density, low-cost external zoned direct attach storage for HP DL360p Gen8 servers.

The HP ConvergedSystem 300 for Microsoft Analytics Platform solution uses Windows Server Storage Spaces to provide management of the drives in the HP D6000 Disk Enclosure.

http://technet.microsoft.com/en-us/library/hh831739.aspx


12

Networking components The HP ConvergedSystem 300 for Microsoft Analytics Platform includes three logical networks; dual redundant InfiniBand networks for data movement, and a unified management network for management traffic and query requests. The networks are integrated into the HP DL360p Gen8 servers through the following components.

Figure 7. Logical InfiniBand and Ethernet Networks

InfiniBand network

The HP ConvergedSystem 300 for Microsoft Analytics Platform includes two Mellanox IB FDR 36P Managed Switch for each rack. These switches provide high bandwidth data transfers within the solution as necessary for query resolution in both the PDW and HDI regions. Additionally, customers can connect their own servers to the InfiniBand switches to provide high speed data loading and high speed backup performance to the solution.

All DL360p Gen8 servers in each rack are connected to the InfiniBand switches with the HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544QSFP Adapter. The adapter is based on next-generation Mellanox ConnectX-3 technology and PCI Express 3.0 for unprecedented scalability and fabric flexibility. They function as a dual ported InfiniBand card for the HP ConvergedSystem 300 for Microsoft Analytics Platform. Together with the next generation of HP InfiniBand FDR edge switches, they deliver low-latency InfiniBand bandwidth, up to 56Gb/s Fourteen Data Rate (FDR) or up to 40Gb/s Quad Data Rate (QDR), for performance driven server and storage clustering applications in the HP ConvergedSystem 300 for Microsoft Analytics Platform.

Every server uses one connection to each InfiniBand switch, using different Ethernet subnets. This provides full redundancy between the two networks in the event of a hardware failure in any of the InfiniBand networking components.

http://h30094.www3.hp.com/product.aspx?cache=1973508493&culture=en-US&sku=10385907

http://h30094.www3.hp.com/product/sku/10385869/mfg_partno/649281-B21

http://h30094.www3.hp.com/product/sku/10385869/mfg_partno/649281-B21


13

Ethernet network

The HP ConvergedSystem 300 for Microsoft Analytics Platform includes two HP 5120-24G El switches with 2 Interface Slots for each rack. These switches provide Ethernet uplinks to the customer network, as well as supporting management traffic, failover clustering support, and general connectivity with each rack and uplinks between multiple racks in the solution. Additionally, customers can connect their own servers to the InfiniBand switches to provide high speed data loading and high speed backup performance to the solution.

The HP 5120 EI Switch Series is comprised of Gigabit Ethernet switches that support static Layer 3 routing, diversified services, and IPv6 forwarding, as well as provide up to four 10-Gigabit Ethernet (10GbE) extended interfaces. Unique Intelligent Resilient Framework (IRF) technology creates a virtual fabric by managing several switches as one logical device, which increases network resilience, performance, and availability, while reducing operational complexity. These switches provide Gigabit Ethernet access and can be used at the edge of a network or to connect server clusters in data centers. High scalability provides investment protection with two expansion slots, each of which can support two-port 10GbE expansion modules. High availability, simplified management, and comprehensive security control policies are among the key features that distinguish this switch.

All DL360p Gen8 servers in each rack are connected to the Ethernet switches with the HP Ethernet 1Gb 4-port 331FLR Adapter. Each server has three Ethernet connections, including one iLO Ethernet connection, and two redundant Ethernet connections (one to each switch) for general connectivity.

The HP Ethernet 1Gb 4-port 331FLR Adapter features the next generation of Ethernet integration, offering a single-chip solution in a FlexibleLOM form factor, which offers the option for future upgrades to 10GbE networking speeds. It delivers full line-rate performance across all ports with low power consumption.

The HP 331FLR is ideal for virtualization, security, server consolidation, network segmentation, and other applications requiring maximum throughput and port density. The total aggregate throughput of 4Gb meets the needs of customers desiring high bandwidth.

The HP 331FLR supports advanced features such as Large Send and Receive offload capability, TCP checksum and segmentation, VLAN tagging, MSI-X, jumbo frames, IEEE 1588, HP Sea of Sensors 3D and virtualization features such as Microsoft VMQ.

Support for HP Sea of Sensors 3D Technology enhances server performance while reducing energy use and expense.

Ethernet network details

The Ethernet network serves multiple purposes. From a management and interconnect perspective, ports 1-22 of the 1Gb Ethernet connections on the HP 5120-24G El switches are used for connectivity to the various servers. Port 23 is available for customer use, including customer network connectivity if the customer desires a 1Gb Ethernet connection to their network.

By default, the HP ConvergedSystem 300 for Microsoft Analytics Platform solution ships with IRF enabled between the two HP 5120-24G El switches, so port 24 of the 1Gb ports is used for MAD APR detection between the two switches. The 10Gb connection in Module 1 Port 1 on each switch is used for switch interconnect. Module 1 Port 2 is used for connectivity to the InfiniBand switches. The remaining 10Gb ports are available for customer usage.

http://www8.hp.com/uk/en/products/networking-switches/product-detail.html?oid=4174785#!tab=features

http://h18004.www1.hp.com/products/servers/networking/331FLR/

http://h18004.www1.hp.com/products/servers/networking/331FLR/


14

Figure 8. The Ethernet switches cable wiring in an HP ConvergedSystem 300 for Microsoft Analytics Platform (Rack #1)

If the solution includes more than one rack, then the recommended infrastructure will look like Figure 9 below. The customer connections (if the customer is using 10Gb uplinks to their network) will move to the second rack’s Ethernet switches. The other 10Gb ports will be used as shown in the diagram.

Figure 9. Connecting multiple racks of Ethernet switches for the HP ConvergedSystem 300 for Microsoft Analytics Platform Solution

Management software

HP ConvergedSystem 300 for Microsoft Analytics Platform provides holistic management for both virtual and physical environments.

Infrastructure overview – RTM through Appliance Update 2 (AU2)

To understand the management infrastructure, a further understanding of the software infrastructure is necessary. From a software perspective, the HP ConvergedSystem 300 for Microsoft Analytics Platform is broken up into two “regions.” These regions are the PDW region, running the SQL Server Parallel Data Warehouse software, and the Hadoop Region, running Microsoft HDInsight. The HP ConvergedSystem 300 for Microsoft Analytics Platform always has two or three Active Directory domains. The first domain is the Fabric Domain, the second is the PDW (or sometimes referred to as the workload) domain. If HDInsight is part of the solution, there will be a third HDI domain as well. Each domain is described here, along with the members of the domain and their usage.

Color key IB cable lengths Ethernet cable lengths

.5M 2M 1.2M

1M 3M 4ft 7ft

Ethernet Switch #1 (R1ETHSW1) - U40

Ethernet ports - Front

2 4 6 8 10 12 14 16 18 20 22 24

HST01 HSA02 HSA04 HSA06 HSA08 HST03 HST01 iLO HSA02 iLOHSA06

iLOR1IBSW1 R1ETHSW2

10 GbE Uplink ports - Rear

U38 U7 U14 U21 U28 U36 U38 U7 U21 U42 U41

1 3 5 7 9 11 13 15 17 19 21 23 Port 1 Port 2 Port 1 Port 2

HST05 HST02 HSA01 HSA03 HSA05 HSA07 HST04HST05

iLO

HSA04

iLO

HSA08

iLO

HST03

iLO

R1ETHSW

2 2/1/1

R1IBSW1

Port1

U34 U37 U6 U13 U20 U27 U35 U34 U14 U28 U36 U39 U42

Ethernet Switch #2 (R1ETHSW2) - U39

Ethernet ports - Front

2 4 6 8 10 12 14 16 18 20 22 24

HST01 HSA02 HSA04 HSA06 HSA08 HST03 HST02 iLO HSA01 iLOHSA05

iLOR1IBSW2 R1ETHSW1

10 GbE Uplink ports - Rear

U38 U7 U14 U21 U28 U36 U37 U6 U20 U41 U42

1 3 5 7 9 11 13 15 17 19 21 23 Port 1 Port 2 Port 1 Port 2

HST05 HST02 HSA01 HSA03 HSA05 HSA07 HST04HST04

iLO

HSA03

iLO

HSA07

iLO

R1ETHSW

1 1/1/1

R1IBSW2

Port1

U34 U37 U6 U13 U20 U27 U35 U35 U13 U27 U40 U41

User

10GbE

User

10GbE

R1PDU2

Module 1 Module 2

User

10GbE

User

10GbE

R1PDU1

Module 1 Module 2

10 GbE Uplink

Unused

User available

Reserved


15

The fabric domain

The fabric domain is used as the master control infrastructure for the HP ConvergedSystem 300 for Microsoft Analytics Platform. All Windows servers running on the physical hardware are members of the fabric domain. The fabric domain also hosts the primary Windows Failover Cluster. Additionally, there is a Virtual Machine, named AD, which is the Active Directory Domain controller for the fabric domain. This domain will always be present in the solution.

The PDW (workload) domain

The PDW domain is used for ownership of the virtual machine infrastructure of the components that run Microsoft SQL Server Parallel Data Warehouse software. The domain controller for the PDW domain is a virtual machine named MAD01. This domain will always be present in the solution.

The HDI domain The HDI domain is used for ownership of the virtual machine infrastructure of the components that run Microsoft HDInsight software. The domain controller for the HDI domain is a virtual machine named HMN01. This domain will only be present if the customer has chosen to have HDI added to their HP ConvergedSystem 300 for Microsoft Analytics Platform solution.

The PDW region

The PDW region is the set of virtual machines that entail the Microsoft SQL Server Parallel Data Warehouse solution. The PDW region is the massively parallel software that provides a relational data warehouse for the customer. The Orchestration Server, or control or management node is the primary access/query point for customers. The Orchestration Server coordinates querying and responses from the compute nodes (CMPxx virtual machines) running on the HP ConvergedSystem 300 for Microsoft Analytics Platform scale unit servers (Figure 10), running one virtual compute node per physical server in the scale unit.

Figure 10. Inside a scale unit

Support services are also run with the System Center 2012 Virtual Machine Manager (VMM virtual machine), providing multiple layers of management support to the virtual and physical infrastructure. As mentioned earlier, MAD01 also provides Active Directory support for the PDW region.


16

The PDW Orchestration Server (HST01) is the primary server responsible for running PDW region management components (shown below as virtual machines Control Node, MAD, and VMM), as well as the fabric AD virtual machine.

Figure 11. Virtual Machines and Services – RTM thru AU2

The PDW Failover Server (HST02) can also run these components in a failover scenario, as well as running a compute node (CMPxx) virtual machine in the event of a failure of an HSA component. These, as noted earlier, will always be the first two servers near the top of the rack on Rack #1.

An optional third failover server can be added to the configuration. Both HST02 and the third failover server (which may be HST03 or HST05, depending upon whether an HDI region is present) will normally be strictly in standby mode and not running any virtual machines.

The Hadoop (HDI) region The HDI region is the set of virtual machines that provide Hadoop Data services. This is an optional component that customers can purchase in addition to a PDW region. The Hadoop region management nodes include active directory (virtual machine HMN01), a secure web gateway (virtual machine HSN01), and a Hadoop Head node (HHN01 virtual machine) which provide various Hadoop-related services to the solution.

The data nodes of the Hadoop region are split among the scale unit servers, similar to the PDW compute nodes. However, there are two Hadoop data node virtual machines on each scale unit server. These virtual machines are named HDNxxx (i.e. HDN001, HDN002, etc.).

The HDI Orchestration Server, referred to as HST03, is the primary server responsible for running the HDI management virtual machines (HHN01, HMN01, and HSN01). There is also a failover server for these components, known as HST04, which is much like the HST02 server described above. Note that the HDI and PDW regions do not mix, so no PDW components will fail over to HST04, and no HDI components will fail over to HST02. There are no additional optional failover servers in the HDI configuration.


17

Infrastructure overview – Appliance Update 3 (AU3)

With the appliance update 3 (AU3) release, the HP ConvergedSystem 300 for Microsoft Analytics Platform now has a single domain. The Active Directory domain controllers are now replicated, and run on the orchestration server and the failover server. This highly available design change does not require a failover event for the domain controller, as it is already running on the other host.

This design also allows for greater management of all virtual machines, and removes the requirement for all virtual machines on the domain controller host, to also have to failover. In addition, the corralling service is no longer needed, any node can fail to any available passive host. Windows Deployment Service (WDS) is also run in its own virtual machine and handles Windows deployment and DHCP services. (See Figure 12 below.)

Figure 12. Virtual Machines and Services – AU3

The fabric domain The fabric domain is used as the master control infrastructure for the HP ConvergedSystem 300 for Microsoft Analytics Platform. All Windows servers running on the physical hardware are members of the fabric domain. The fabric domain also hosts the primary Windows Failover Cluster. Beginning with AU3, there is now one domain, with replicated Active Directory domain controllers that run on the orchestration server, and the failover server.

The PDW region The PDW region is the set of virtual machines that entail the Microsoft SQL Server Parallel Data Warehouse solution. The PDW region is the massively parallel software that provides a relational data warehouse for the customer. The Orchestration Server, or control or management node is the primary access/query point for customers. The Orchestration Server coordinates querying and responses from the compute nodes (CMPxx virtual machines) running on the HP ConvergedSystem 300 for Microsoft Analytics Platform scale unit servers (Figure 10), running one virtual compute node per physical server in the scale unit.


18

Configuration management

The configuration of the solution is controlled through the Microsoft Analytics Platform System Configuration Manager, known as dwconfig.exe. This program resides on the MAD01 virtual machine. (With the AU3 release, Configuration Manager is launched from the control node - CTL01.) This interface is used for significant configuration changes, such as:

• Viewing the topology

• Changing administrative passwords for the components

• Installing certificates for validation of SSL connections to the web interfaces

• Configuring the Windows firewall rules

• Stopping and starting regions

• Restoring the master database

• User Management for HDI users

This tool will only be used by ConvergedSystem 300 for Microsoft Analytics Platform administrators. This utility is also used by HP during the initial installation process for basic configuration steps.

Figure 13. The Configuration Manager for the ConvergedSystem 300 for Microsoft Analytics Platform


19

Solution management

Overall management is through the Microsoft provided web administration tool within the solution. This web tool is available at the IP address of the cluster of the CTL01 (control node). Within this management interface, overall system health, performance monitoring, and region-specific management capabilities are exposed.

In the figure below, the home page of the management web page is shown. A quick overview of the health of the solution is shown, along with the option to perform performance monitoring of the system.

Figure 14. The Web Management interface for the HP ConvergedSystem 300 for Microsoft Analytics Platform


20

In Figure 15 below, the home screen of the PDW region management interface is shown. As seen in the console, the administrator can monitor the health of the PDW region, monitor queries (including examining showplans for queries), manage data loads, backups, and analyze storage usage. Finally, an additional performance monitoring interface is available.

Figure 15. The Web Management interface for the PDW region of the HP ConvergedSystem 300 for Microsoft Analytics Platform


21

Similar options, such as monitoring the HDFS system, MapReduce, Health, and storage of the HDI region (if present), are available in the same interface, as shown in Figure 16.

Figure 16. The Web Management interface for the HDI region of the HP ConvergedSystem 300 for Microsoft Analytics Platform


22

Additionally, management of the Microsoft SQL Server Parallel Data Warehouse environment can be accomplished through a query interface. Command line interfaces, such as sqlcmd.exe are available. The primary graphical query environment is accomplished through the Microsoft SQL Server Data Tools (SSDT), an add-on to Microsoft Visual Studio 2012, and integrated into Microsoft Visual Studio 2013. Figure 17 shows the SQL Server Data Tools connected to a ConvergedSystem 300 for Microsoft Analytics Platform PDW region with Microsoft Visual Studio 2012.

Figure 17. The Microsoft SQL Server Data Tools interface, using Microsoft Visual Studio 2012

HP Support Pack for the HP ConvergedSystem 300 for Microsoft Analytics Platform

In addition to the standard Microsoft management utilities, HP also supplies the HP Support Pack for the HP ConvergedSystem 300 for Microsoft Analytics Platform. The support pack provides useful maintenance utilities, assists with support events, and generates needed reports for the Proactive Care Team.

Key components of the HP ConvergedSystem 300 for Microsoft Analytics Platform Support Pack include:

• Tested and validated Firmware/Driver package

• A validation utility to confirm the system is in a supported configuration

• Diagnostics tools for deeply integrated health checks

• Reporter utility for deep device discovery and reporting

• Proactive Care Report Utility for a dashboard of the system configuration

These utilities are all value-add capabilities only available with the HP ConvergedSystem 300 for Analysis Platform System solution.


23

Services

Included with the HP ConvergedSystem 300 for Microsoft Analytics Platform, is factory integration, installation and support. This section outlines services that are available for HP ConvergedSystem 300 for Microsoft Analytics Platform.

Factory Express

Factory Express services are utilized by the HP ConvergedSystem 300 for Microsoft Analytics Platform solution. Benefits include:

• Business Agility

– Broad portfolio

– Faster technology transactions

– Smoother solution deployment

– Faster return on IT investment

• Managing Costs

– Save time, money and resources

– Collaborate with HP experts

– One-touch efficiency

• Mitigating Risk

– Predictable hassle-free experience

– Efficiency and quality

• Increase performance and quality

– Innovation based on standards

– Our factory is your factory

Support services

Support offerings for HP ConvergedSystem 300 for Microsoft Analytics Platform include the following:

HP Proactive Care

HP Proactive Care is designed to keep your systems stable and optimized in order to achieve a high level of availability. This offering provides a superior call experience, with end-to-end case management, advanced technical expertise, and proactive service management. You can customize reaction time by selecting one of the following:

– Six-hour call-to-repair (CTR)

– 24 x 7, with four-hour onsite response

Note Proactive Care leverages industry-leading remote support technology for real-time monitoring, alerts to diagnose and fix issues early, call logging for reactive support, and to help avoid downtime. This also facilitates analysis, which HP utilizes to deliver proactive reports, analysis, and recommendations, as well as proactive health-check reporting on your infrastructure.

HP Proactive Care with Personalized Support option To enhance your Proactive Care experience, you can add the Personalized Support option, whereby an assigned account services manager (ASM) provides personalized support along with operational and technical advice intended to help you meet your IT objectives. Work is guided through an agreed support plan and reviewed twice a year to enable alignment and proper focus on current and emerging issues.

HP Critical Service (recommended by HP) HP Critical Service uses ITIL best practices to minimize downtime and maximize availability for your mission-critical applications.

Mission-critical services keep your solution available 24 x 7, while CTR reactive support with access to the Global Mission Critical Solution Center enables quick resolution whenever unforeseeable problems crop up.

http://www.hp.com/go/factoryexpress


24

For more information on these support offerings, refer to your Account Manager or Reseller.

Licensing considerations

When you purchase a ConvergedSystem 300 for Microsoft Analytics Platform, the necessary Windows Server 2012 R2 licenses are included with each DL360p Gen8 server purchased.

The Microsoft SQL Server Parallel Data Warehouse Edition, which includes the Virtual Machine Manager component of Microsoft System Center 2012 Standard SP1, is subject to the terms and conditions of the Volume License program agreement under which you have acquired licenses to these Microsoft products.

Appendix A – Solution hardware configuration

Table 1 outlines the hardware configuration/components deployed in the HP ConvergedSystem 300 for Microsoft Analytics Platform. Note that the number of scale units will be dependent upon the customer order, and an additional rack is required for each set of four scale units.

Table 1. Hardware and software configurations

Component Quantity Description

Orchestration and Failover Servers

Minimum 2 per region HP ProLiant DL360p Gen8 server, each with:

• 2 x Intel Xeon processor E5-2690

• 256GB RAM (16 HP 16GB PC3-12800R (DDR3-1600) memory

• 2 x SAS hard drives, 600GB/10,000 rpm 2.5 inch

• HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544QSFP Adapter

• HP Ethernet 1Gb 4-port 331FLR Adapter

Data Scale Unit (Server) In groups of two HP ProLiant DL360p Gen8 server, each with:

• 2 x Intel Xeon processor E5-2690

• 256GB RAM (16 HP 16GB PC3-12800R (DDR3-1600) memory

• 2 x SAS hard drives, 600GB/10,000 rpm 2.5 inch

• H221 Host Bus Adapter for D6000 connectivity

• HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544QSFP Adapter

• HP Ethernet 1Gb 4-port 331FLR Adapter

Data Scale Unit (Storage) – HP D6000 Disk Enclosure

One per scale unit HP D6000 Disk Enclosure, each with:

• Dual I/O Module option kit

• 70 of either 1TB, 2TB, or 3TB HP SAS 7.2K 3.5 inch HDD

Network Switches 2 HP 5120-24G El switches with 2 Interface Slots

InfiniBand Switches 2 Mellanox IB FDR 36P Managed Switch

Infrastructure 1 per rack HP 642 1075mm Shock Intelligent Rack, each with:

• Side Panel Stabilizer Kit

• Rack Grounding Kit

• Intelligent PDUs

• HP 42U Intelligent Discovery Kit

• Single-phase (4 PDUs per rack) or Three-phase (2 PDUs per rack) power solution

• Cabling and wiring to connect all components


25

Appendix B – Power diagram

Figure 18 shows an example of the power cabling that happens within a standard ConvergedSystem 300 for Microsoft Analytics Platform rack, showing the redundancy and reliability of the design.

Figure 18. Power cable wiring in an HP ConvergedSystem 300 for Microsoft Analytics Platform – Single Phase power

Facing the rear of the rack, PDU 2 and PDU 4 are on the left, PDU 1 and PDU 3 are on the right.

Single Phase Extension Bar Cabling - REAR VIEW

PDU PDU

Extension Bar 1 Extension Bar 1

Port Device RU-Loc Port Device RU-Loc

1 R1IBSW1 Power Supply (Left) U-42 1 R1IBSW1 Power Supply (Right) U-42


3 R1ETHSW1 (Ethernet Switch) U-40 3 R1ETHSW2 (Ethernet Switch) U-39

4 HST01 Power Supply 2 (Left) U-38 4 HST01 Power Supply 1 (Right) U-38







4 Not Used 4 Not Used




1 HSA08 Power Supply 2 (Left) U-28 1 HSA08 Power Supply 1 (Right) U-28


3 R1S4 Drawer2 Primary I/O (Top Left) U-22 3 R1S4 Drawer1 Primay I/O (Top Right) U-22

4 R1S4 Drawer2 Secondary I/O (Btm Left) U-22 4 R1S4 Drawer1 Secondary I/O (Btm Right) U-22


4 4

5 5

6 6

PDU PDU

1 1

2 2

3 3






















Load

Segmen

t

1

2

3

Not Used

Not Used

Load

Segmen

3

Not Used

Not Used

Not Used

4

5

6

Load

Segmen

4

4

5

6

Load

Segmen

t

2

1

2

3

Not Used

Not Used

Not Used

1

Not Used

Not Used

Not Used

Not Used


26

Figure 19. Power cable wiring in an HP ConvergedSystem 300 for Microsoft Analytics Platform – Three Phase power

Three Phase Extension Bar Cabling - REAR VIEW

PDU PDU





3 R1ETHSW1 (Ethernet Switch) U-40 3 R1ETHSW2 (Ethernet Switch) U-39






































Load

Segmen

t

2

1

2

3

4

5

6

Load

Segmen

t

1

1

2

3

4

5

6


For more information

Contact HP hp.com/country/us/en/contact_us.html

HP Services hp.com/go/services

HP ConvergedSystem 300 for Microsoft Analytics Platform hp.com/go/convergedsystem/cs300APS

Microsoft Analytics Platform System microsoft.com/en-us/server-cloud/products/analytics-platform-system

http://blogs.technet.com/b/dataplatforminsider/archive/2014/05/20/architecture-of-the-microsoft-analytics-platform-system.aspx

HP Factory Express hp.com/go/factoryexpress

HP Networking hp.com/go/networking

HP Servers hp.com/go/servers

To help us improve our documents, please provide feedback at hp.com/solutions/feedback.

Sign up for updates

hp.com/go/getupdated

© Copyright 2014-2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only

warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should

be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

Microsoft and Windows are trademarks of the Microsoft Group of companies. Intel and Xeon are trademarks of Intel Corporation in the U.S. and other

countries.

4AA5-2621ENW, January 2015, Rev. 2

http://www.hp.com/country/us/en/contact_us.html

http://hp.com/go/services

http://www.hp.com/go/convergedsystem/cs300APS

http://www.microsoft.com/en-us/server-cloud/products/analytics-platform-system






http://www.hp.com/go/networking

http://www.hp.com/go/servers

http://www.hp.com/solutions/feedback

http://www.hp.com/go/getupdated

Documents

HP ConvergedSystem 300 for Microsoft Analytics Platform ... · • HP ConvergedSystem 300 for Microsoft Analytics Platform offers flexibility to scale to large amounts of data, up