How to break the 30% virtualization barrier A Pepperweed white paper, sponsored by HP Software

Business white paper

Redefining operations management to support virtualization is the key

Virtualization was initially embraced by information technology professionals for infrastructure consolidation, drawing on the technology’s ability to run multiple virtual workloads, especially servers, on one hardware platform. More recently, the concept of private cloud computing, with its reliance on virtual servers, storage, and networks for such forward-looking features as automatic apportionment of resources, has raised virtualization’s profile even more. However, IT professionals are increasingly aware that virtualization brings with it some often unforeseen challenges.

This white paper outlines factors that constrain the virtualized environment and offers a proven prescription for getting more utility from your investments in computer hardware and the virtualization technology that runs on it. You’ll learn about the HP business software solutions that complement your virtual environment and increase your ability to manage and operate your physical and virtual infrastructure. While this white paper is not about private cloud computing, it does illustrate techniques that are critical to successfully managing a private or public cloud deployment.

3

What is the “30% barrier” and why am I hitting it?With the ability to transform a single x86 server into 10 virtual ones, or create a “storage democracy” by melding all a company’s physically scattered storage assets into a single pool accessible to all, virtualization solutions from the likes of VMware, Microsoft®, and Citrix represent the biggest advance in computing since silicon circuitry replaced the punch card. CIOs and system administrators have embraced virtualization with gusto. Indeed, who wouldn’t respond to a technology that promises to wring five to 10 times the useful work from existing servers and storage? Plus, virtualization is a key enabling technology for that nirvana of enterprise computing, the private cloud, with its pools of resources (processor cores, random-access memory, disk space, network elements, and more) that can be easily and dynamically reconfigured.

But the reality in too many organizations is that virtualization has failed to fulfill the lofty promises made for it. In fact, companies and enterprises often come up against a “30 percent deployment barrier” with virtualization. Although they indeed achieve greater efficiency from their virtualized servers and storage, a closer company-wide examination reveals that less than a third of their workloads are running on virtualized systems.

The reasons are many, but they boil down to risk and management. Companies initially adopted virtualization for transitory activities like test and development, where provisioning a one-off virtual server could take place on existing hardware resources without impact to the business. With production applications, however, the need to monitor and manage to business service levels made deployment using virtualization technology more complex and created tension with business users who were reluctant to put mission-critical applications at risk. As a result, the increased effort of managing the virtual environment combined with its perceived risk has slowed adoption to about the 30 percent level.

Table 1 illustrates the benefits of virtualization, along with some of the possible negative effects of those benefits. It shows how the virtualized infrastructure’s flexibility and better use of physical assets can be counterbalanced by the increased management and security burdens of virtual and physical sprawl, innovation-choking complexity, poor visibility, and splintered control. A lack of planning and robust management tools—as well as the foibles of human nature—can combine to torpedo the utopian vessel of virtualization before it leaves the harbor.

1 Thomas Bittman, Q&A: Six Misconceptions About Server Virtualization, Gartner, July 29, 2010.

“Gartner has talked with several organizations that are roughly 30% virtualized, but that have halted their virtualization deployments because of cost overruns and process issues.”1

Table 1: Why the virtual utopia is elusive: every virtual benefit has a downside

This virtualization feature: Offers these benefits: But results in:

Multiple VMs per physical server • Better utilization of physical assets• Reduced capital costs

• Greater complexity in managing physical and virtual IT infrastructure • Duplication of effort by virtualization specialists and operations teams, resulting in increased

operating costs• Increased downtime due to complexity of troubleshooting

Multiple applications on each server; shared storage

• More efficient hardware utilization

• Decoupling SW from physical infrastructure

• Applications that are more complex to deploy and control; more management required• Greater risk from single point of hardware failure

Multiple vendors (VMware, Microsoft, Citrix, etc.)

• Freedom of choice, no vendor lock-in

• No management standards or common tools

Easily created pools of virtual server, storage assets

• More assets in more places • Virtual sprawl• System performance affected• Difficulty achieving SLAs for VMs

Easily added, moved applications, servers, storage

• More flexibility, easier scaling• Dynamic changes

• Complex, interrelated dependencies that change often, nearly impossible to track manually• Need for higher levels of collaboration across IT teams due to the dynamic nature of

virtualization

Automated provisioning and management

• Faster response to business changes

• Elimination of manual management of physical servers and applications

• More ad hoc changes that are easy to make, difficult to control• More issues when a change has an adverse effect on applications, servers, network, etc. • Still takes too long to provision applications correctly

High mobility (e.g., vMotion or Live Migration)

• Instant, seamless failover • More complex discovery; manual tracking of infrastructure, applications, and relationships no longer feasible

• Security issues• Difficulty identifying root cause of problems

Vendor-supplied monitoring tools • Built-in, VM-specific or application-specific monitoring and control for free

• Tools aren’t standard, widely varying information supplied• Multiple IT resources involved in troubleshooting at business service level • Lack of consistency, introducing further complexity

Software license consolidation • Decoupling SW from physical infrastructure

• SW licenses unlimited by server capacity• License compliance difficult to track• No workable metering technology yet

4

What can I do to improve my virtualized environment?The route to better use of the virtualized environment is based on better control via comprehensive IT infrastructure operations and management software. In fact, the keys to breaking the 30 percent adoption barrier lie in making improvements in four interlocking areas:1. Discovery—Using automatic discovery tools to continually catalog your entire IT infrastructure, including both physical and virtual assets, into a Configuration Management System (CMS).2. Operation—Consolidating the operational view and control of your physical and virtual infrastructure, then focusing on key “root-cause” events. 3. Deployment—Leveraging one solution for the deployment of operating systems, applications, and patches in both your physical and virtual infrastructure. 4. Change—Making sure you correctly track and manage the numerous and rapid changes made possible by virtualization.

As it happens, components of HP Business Technology Optimization (BTO) software are the ideal answer to taming the virtualized jungle. While the complete BTO suite offers a number of benefits, some applications are particularly suited to managing these four areas affected by virtualization. What follows is a “how-to” blueprint using tested, off-the-shelf HP Software solutions that have proven their worth in the real world.

1. DiscoveryDiscovery is the foundation for operational visibility and event resolution, asset management, ITIL-based service management, and more. In the physical realm, discovery is always important, but in the virtual environment it’s vital. The relationships among physical and virtual assets, the applications that run on them, and events that occur are tremendously dynamic and complex, and it’s essential to have a clear, up-to-date picture of how physical and virtual assets are related, and which assets are handling which tasks at what time.

5

DDMa

uCMDBDDMi

Because of the speed at which changes can occur, manually tracking the infrastructure components, applications, and relationships is no longer feasible, but automated discovery can provide a current view of the physical and virtual infrastructure and how applications and data are deployed. Such discovery enables rapid root-cause diagnosis of events, simplifies asset management, and eliminates duplication of effort in resolving problems. Without automation, keeping discovery up to date in near-real time is particularly challenging for virtual machines (VMs) that can be moved dynamically.

HP discovery solutionsHP has a pair of solutions, HP DDMi and DDMa, that together provide the capabilities needed by an IT administrator to automatically discover both the virtual and physical infrastructure, and to manage the complete infrastructure effectively.

Discovery for configuration management with HP DDMa: HP Discovery and Dependency Mapping, Advanced (DDMa) is an important part of the discovery toolset. This solution focuses on discovery for configuration management; it understands device relationships, allowing you to discover your IT resources and to collect valuable information about their attributes and interdependencies, without the use of agents. The solution discovers not only traditional servers, but also virtual servers, databases, applications, network devices, and more. Once the resource is discovered, information about it is stored in an operational database called the HP Universal Configuration Management Database (Universal CMDB).

Discovery for inventory and software utilization with HP DDMi: Another part of the HP discovery toolset, HP Discovery and Dependency Mapping Inventory (DDMi) is aimed at discovery of IT resources for the purpose of inventory and software utilization. HP DDMi is primarily used for asset management and it focuses on device lifecycle. For example, you can use IT resource discovery in DDMi for tracking desktop inventory to assist with financial depreciation calculations, or for tracking software utilization to control software license costs.

HP Universal CMDBThe HP Universal Configuration Management Database is a run-time service model that provides a central repository for all discovered devices and services in the data center. The database accepts data feeds from both HP and third-party products; this data could be configuration items imported from another solution such as HP Network Node Manager, or performance/event data from an event management solution such as HP Operations Manager.

In addition to being a central storehouse for resource data, the real benefit of the Universal CMDB is the auto-discovered relationships between the configuration items. Suppose a network device fails in the data center, and you know from the Universal CMDB topology that the network device is connected to, say, Server A, which is a VMware ESX server running an Oracle virtual machine. You can then infer that any events received about the failed Oracle server can be treated as “symptom” events and focus your troubleshooting efforts on the “root-cause” network failure event.

Figure 1. In the Universal CMDB, HP discovery solutions compile a clear picture of the relationships between physical and virtual infrastructure, applications, and business services.

6

The Universal CMDB also plays a critical role in the change management and service-level management processes. Any change request entered into a system such as HP Service Manager leverages the knowledge of the Universal CMDB to determine the relationship of other configuration items to the one being changed; this ensures the change will not interrupt a critical business service. Moreover, if a device fails, you can reference the Universal CMDB to determine what business services may be affected, or what recent changes to that device (configuration item) may have caused the failure.

2. OperationThe modern data center is almost always made up of hardware and software from many different vendors. And that’s often true of a virtualized environment as well. Chances are, you need to be able to manage not only VMware, but also Microsoft Hyper-V, Citrix XenServer, and perhaps even UNIX® and Linux virtualization variants, too—not to mention the physical hardware and a host of other business applications. So before you can effectively focus on operation, you need to make sure up-to-date discovery is in place so you know what you’re dealing with.

After discovery, effective operation and control relies on an operations bridge (also known as a common events bridge) that combines events and traps from all different applications, devices, and systems in the infrastructure. This tool must be vendor-agnostic, and must tap into the various monitors and the results of different element managers, such as VMware vCenter Server and HP Insight Control Software, and pull them together into a single view.

Besides cataloging all events, the operations bridge must also allow the support staff to quickly identify an event’s root cause. A typical network operations center (NOC) at a midsize company or a small enterprise might see a few thousand events per day, which are far too many to analyze or troubleshoot effectively. With the discovery techniques and topology-based event correlation made possible by HP tools in the operations bridge, the flashing red lights in the NOC can be reduced to perhaps a hundred or fewer root-cause events—a significant benefit to frazzled operators, and a tremendous aid in fast, effective troubleshooting.

HP tools for the operations bridgeYou can support an operations bridge with HP tools, including:• HP Operations Manager (HP OM): HP OM is an

agent-based solution that consolidates events from servers and applications and feeds them into a central event console. From this console, operations bridge engineers can send out notifications to troubleshooting personnel, create incident tickets in a service desk application, or initiate automations preconfigured to resolve an issue.

• HP Operations Manager i (HP OMi): A tool that sits on top of HP Operations Manager and performs topology-based event correlation (TBEC) to match the discovered elements with events. By correlating events with topology, unnecessary data is suppressed and operators can identify root-cause events—which means operators can go right to the underlying problem rather than chasing events that are merely symptoms.

• HP Operations Smart Plug-Ins (SPIs): Extending the native capability of the HP Operations Agent, the Smart Plug-Ins provide application-specific monitoring and data collection for some of the industry’s most common applications—such as Oracle, SAP, Exchange, SQL Server, Active Directory, WebSphere, BlackBerry, and many others. These Smart Plug-Ins provide custom HP Operations policies, scripts, performance metrics, and troubleshooting knowledge, helping to ensure complete coverage of your data center.

• HP Performance Manager: Web-based analysis and visualization tool that analyzes performance trends of applications, systems, and services based on data collected by the HP Operations Agents.

In a virtualized data center, HP Operations Manager can be used to centrally review and act upon events from multiple virtualization management solutions such as VMware and MS Hyper-V, and can also perform sophisticated event correlation. Providing a central platform for monitoring your virtualized data center, HP Operations Manager can then launch data center automation solutions such as HP Operations Orchestration to manage the virtual infrastructure in response to these informative or critical events.

7

Figure 2. HP BTO software provides the ability to reduce multiple events to their root causes and even provide real-time alerts.

A) At the operations bridge, HP Operations Manager takes inputs from a variety of heterogeneous monitors and places them into a single, all-encompassing view.

Applicationservers

Application Specific SPIsBES SPISQL SPIOS SPI

Agent

HP Operations Manager

• HP• BMC• Cisco• VMware• etc.

Infrastructure Monitoring

100s of SPIs available

Events from Operations Manager Understanding of the topology and relationships

HP OMi topology-based event correlation

HP Operations Manager

Consolidation Correlation

Real-time alert management

HP OMi

xMatters (AlarmPoint)

B) HP Operations Manager i correlates events based on the topology that was revealed by discovery.

C) A third-party add-on automatically handles alert management for those significant root-cause events.

8

Third-party alert managementOnce they’ve identified a root-cause event, operators need to be able to get that information to support staff who can repair the issue. One preferred solution is AlarmPoint, a relevance engine provided by xMatters that has hooks into many HP solutions, including HP Operations Manager, HP Network Node Manager, and HP Service Manager. When a root-cause event is detected, AlarmPoint notifies the appropriate engineer using email, text, Blackberry, or other medium of choice. The engineer can then use the same medium to interact with the AlarmPoint system and perform actions such as rebooting a server, deploying a virtual machine, or restarting a process.

3. DeploymentDeployment in the virtual environment is similar to that in a physical environment—except that it can occur with much more speed and flexibility. The ultimate goal of virtual deployment is the private cloud; here’s how it might work:1. A user needs a server, and possibly preconfigured applications.2. With a Web browser, the user visits a service catalog containing a list of server builds available for checkout. 3. The user checks out a Windows® 2008 server with two x86 cores, 4 GB of memory, and 80 GB of disk space. All of these resources come from virtual pools. In the physical world, that server might require two weeks to deploy; but in the virtualized environment, it could take just a couple of minutes. 4. When the user is done with the machine, it is checked back in and the resources returned to the virtual pools, ready for the next end user.

The deployment advantage is the virtual pools of resources—processor cores, memory, disk space, network resources, and more—and the automatic provisioning of them. The tasks of deploying applications and even applying OS patches are handled the same way. Thanks to virtualization, you can not only create virtual machines, but you can actually automate their build-out as well, saving hours of process time and manual work. You can even extend automation to the task of ensuring your systems are kept in compliance with the latest patches, updates, and security configurations. In fact, achieving this virtual promised land is possible today, thanks to HP BTO software like HP Server Automation and HP Storage Essentials, which are part of the HP Converged Infrastructure solution.

HP Server Automation softwareAs any VMware user knows, it’s easy to quickly configure a virtual version of a basic server. However, without an automation product such as HP Server Automation software, the task of fully provisioning the application, patches, or business services for that server can be a long manual process subject to error and inconsistency. With HP Server Automation software, you can provision the application layer so it’s consistent and compliant—and do it with greater speed and accuracy.

True, virtualization vendors often provide management tools such as VMware vCenter. But these tools often lack the ability to manage the ongoing compliance, performance, and change issues faced by IT operations personnel. HP solutions can sit on top of native tools and fill in the missing gaps. For instance, HP Server Automation is designed to lower costs and automate common operational tasks such as patching, provisioning, and configuration management. It provides proactive compliance management across applications, servers, and storage, giving a single view of application dependencies, network details, server configuration, and local storage information. The best part is that it manages not only your virtual infrastructure, but also the physical one.

With virtual resources, HP Server Automation offers improved visibility and control of virtual sprawl. You can establish a baseline for the VM-hypervisor, then create and configure virtual machines, deploy and configure applications on the VMs, and even configure auto-updates. This software also has facilities for auditing the hypervisor, along with VMs, servers, and applications, to ensure they adhere to a “gold standard.”

HP Storage Essentials software What Server Automation software does for virtual servers and applications, HP Storage Essentials does for the pools of virtual (and physical) disk resources. This software provides comprehensive storage resource management and storage automation for physical and virtual infrastructures, allowing you to establish baseline values for storage devices. You can manage how a VM uses storage, map virtual machines to storage, or provision and configure storage for a virtual machine.

9

Storage Essentials is also the pathway to automating discovery, visibility, and control of your physical and virtual storage environment, from application to array spindle. It also can provide comprehensive storage management, with control of provisioning, capacity, performance, chargeback, and reporting. Plus, Storage Essentials delivers data center automation and optimization via out-of-the-box integration with other HP software.

With both HP Server Automation and HP Storage Essentials in your data center, it’s easy for your service catalog to leverage these automation tools, which can dynamically build the resource pools required by virtual management platforms. If, as is often the case, your IT environment has more than one virtualization solution, HP also offers products such as HP Operations Orchestration, which hook into VMware or Microsoft Hyper-V to provision your virtual machines using the dynamic pools of resources managed by HP Server Automation and Storage Essentials.

4. Change management“Through 2015, 80% of outages impacting mission-critical services will be caused by people and process issues, and more than 50% of those outages will be caused by change/configuration/release integration and hand-off issues.”2 And change management is even more important in a virtual environment. Virtualization makes it all too easy to throw “wildcard” VMs into the data center willy-nilly, often without proper IT authorization or a change control process. What’s more, the complexity of a virtualized environment makes it impossible to understand the labyrinthine dependencies among network elements without discovery and change management tools that can automatically check to ensure changes do not have an unwanted effect. Virtualization without change management is a short-term strategy that leads to virtual sprawl and often contributes to that 30 percent barrier of virtualization adoption.

2 Ronni J. Colville and George Spafford, Top Seven Considerations for Configuration Management for Virtual and Cloud Infrastructures, Gartner, October 27, 2010.

To help manage change and forge a pathway to automated change management, HP offers the Change Configuration and Release Management (CCRM) solution, which comprises three products: HP Universal CMDB, HP Service Manager, and HP Release Control. In addition, HP Operations Orchestration includes automation software that not only orchestrates change management, but also coordinates everything else in the data center, from incident resolution and self-service provisioning all the way down to the simplest of routine maintenance tasks.

We’ve already looked at the Universal CMDB. Let’s see how the other components of the CCRM solutions can leverage the discovered topology data to ensure successful change management.

HP Service ManagerHP Service Manager provides a core Change Management module that interfaces with the HP Universal CMDB (kept up to date with discovery tools), enabling you to share information about the actual state of configuration items and open up related incident tickets or change requests. Using a built-in workflow, you can standardize how you roll out changes and ensure that the framework is in place to allow HP Release Control to perform impact analysis on your potential changes.

HP Release Control HP Release Control (formerly known as HP Change Control Management) is a decision-support solution that ties the change planning and analysis process to automated change execution and validation, helping reduce the risk of service downtime. This software increases visibility with a consolidated view into all change activity, whether ITIL-based or ad hoc. It automates impact analysis, risk analysis, and collision detection, and even updates the aggregated change calendar. Note that putting control points in place doesn’t slow down IT’s ability to execute, but actually enables faster and more accurate change management with significantly less risk.

HP Operations Orchestration softwareOrchestration is the automation that binds together all the other components, virtual and physical. HP’s Operations Orchestration software is a workflow and runbook tool that automates routine operational and maintenance tasks in your data center. Operations Orchestration automates the key steps in executing approved change requests—and does it consistently and without errors.

10

How can I get the best results from my investment in HP BTO software?To maximize their investment in HP Business Technology Optimization software, and accelerate time to virtualization value for these tools, many customers choose to engage a third-party consultant. The right consultant can help break through the 30 percent deployment barrier more quickly, and can help you lower the risks of virtualization, get your facilities and your staff up and running in less time, and actually reduce your overall costs. A consultant is a valuable shortcut to gaining the benefits of virtualization.

One such vendor is Pepperweed Consulting, an HP Software Platinum Business Partner that has been orchestrating, managing, and monitoring virtual deployments since the days of the first commercial virtualization software. Pepperweed sells, delivers, and supports solutions based on HP products, and today is the largest provider of HP Software support, with 14 years of experience and more than 2,500 successful HP Software implementations. Pepperweed’s staff includes ITIL Foundation–certified HP technical consultants who have earned some 120 HP BTO software implementation and support certifications.

Pepperweed consultants can draw on a wealth of knowledge about virtualization and HP BTO software. Enriching these consultants’ own capabilities, Pepperweed has developed a vast repository, collectively termed Evolution IP, of software integrations to HP and non-HP products, as well as knowledge bases, best practices, delivery methodologies, custom scripts, and other customer software.

What’s more, Pepperweed has coupled Evolution IP with specific HP BTO product sets and select third-party products to create specific Evolution solutions. Four such solutions exist for accelerating the deployment and management of virtualization. They are: Discovery/Evolution, Operate/Evolution, Change/Evolution, and Deploy/Evolution. Customers can implement and integrate these solutions into their existing environment at a lower risk and cost and within very reasonable time periods. In fact, Pepperweed offers the shortest, most straightforward path to breaking through the 30 percent deployment barrier and reaching the nirvana of the private cloud.

“The big Pepperweed advantage is in accelerating the rate of adoption for customers by augmenting their staff with experienced consultants and a proven methodology for building a strong management framework to support virtualization. This allows the customer to see the benefits sooner with less risk to their business services.” Dennis Pereira, Partner Programs Manager, HP Software & Solutions

11

“Cloud computing depends on three things: Internet technology, IT process standardization, and virtualization. And virtualization is the key—without it, the other two don’t matter.” Alex Ryals, Chief Technology Officer, Pepperweed Consulting, LLC

What are my next steps?In this white paper you’ve learned about the basic improvements necessary to achieve higher levels of utilization from your virtual resources. And with these improvements, you also have the main building blocks in place to move to the next level—the promised land of the private cloud.

There’s no time like the present to get started knocking down that 30 percent barrier. Take the time to learn more.Learn about HP BTO software at: www.hp.com/software

Learn about Pepperweed at: www.pepperweed.com

See the Pepperweed webinar series at: http://pepperweed.com/managing-it-operations-in-a-virtualized-world-webinar-series.html

Get connectedwww.hp.com/go/getconnected

Get the insider view on tech trends, alerts, and HP solutions for better business outcomes

© Copyright 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. Oracle is a registered trademark of Oracle and/or its affiliates. UNIX is a registered trademark of The Open Group.

4AA2-4119ENA, Created November 2010

Share with colleagues