Network Instruments EMA ANPM Q1 2013 RADAR Summary

EMA Radar™ for Application-Aware Network Performance Management 2013: Illuminating the PipesReport Summary and Network Instruments ProfileBy Jim Frey, Vice President of Research, Network Management and Tracy Corbo, Principal Research Analyst ENTERPRISE MANAGEMENT ASSOCIATES® (EMA™) Radar Report

May 2013

Table of Contents

©2013 Enterprise Management Associates, Inc. All Rights Reserved.

2

EMA Radar™ for ANPM 2013: Illuminating the PipesReport Summary and Network Instruments Profile

Executive Summary .......................................................................................................................... 1Introduction ..................................................................................................................................... 1Application Awareness in Network Performance Management ......................................................... 3Criteria for Inclusion ........................................................................................................................ 4Scope of this Research ....................................................................................................................... 4

Architecture & Integration Criteria for ANPM Solutions ........................................................... 5Functionality Criteria for ANPM ................................................................................................ 6Resource Efficiency for ANPM ................................................................................................... 7Vendor Strength for ANPM ........................................................................................................ 9

ANPM Study Results ...................................................................................................................... 10EMA Radar Map for ANPM Large/SP Deployments .................................................................... 12Distribution of Results – Large/SP Deployments ........................................................................... 12

Value Leader: Network Instruments .......................................................................................... 13EMA Analysis ................................................................................................................................. 13Network Instruments ...................................................................................................................... 15



PAGE 1

Executive SummaryThe days when network managers could simply be concerned with interoperability and uptime are long past, and the focus has turned towards recognizing and eliminating performance degradations. This demand has also elevated and coalesced specifically around gaining a direct understanding of how applications and services – the life’s blood of IT-enabled organizations – are performing from the network perspective. Management products that address these objectives are known as Application-aware Network Performance Management (ANPM) solutions, and are increasingly becoming essential tooling for enterprise network management and operations teams.

For this EMA Radar™ Report, focus was placed specifically on core capabilities and features associated with the needs that network engineers, managers, and operators have to recognize, characterize, troubleshoot, and communicate details of how applications and services perform as they transit the network. Three common use cases – capacity planning, sustained monitoring, and troubleshooting – were examined in detail for solutions that are based (at least in part) on inspection of network packets or collection and analysis of flow records such as NetFlow, IPFIX, and sFlow. A total of 24 ANPM solutions were reviewed in detail and ranked according to solution impact, resource efficiency, and vendor strength.

IntroductionNetwork management as a whole has long been a mature discipline, particularly when it comes to understanding health and availability of network elements and devices. Network Performance Management (NPM) is, on the other hand, a relatively newer phenomenon, though it too has been around for a considerable period of time. The newest emerging aspect of NPM best practices is application awareness, built into and on top of infrastructure-centric network performance management. As IT operations teams make the transition from tactical firefighters to strategic, proactive assurance, one of the most important unifying themes has been widespread awareness of IT user experience. And what IT users experience is largely not the network – it is the applications and services that the network delivers. Building awareness of applications and services has arguably become one of the most important evolutionary expansions of network engineering and operations disciplines today.

Starting with the advent of RMON solutions 20+ years ago, network management technologists recognized that valuable application performance insights were available from the network perspective, if you knew where and how to look for them. And while RMON itself has not directly survived as a commonplace, independent technology (it relied heavily on network elements gathering and forwarding large volumes of performance metrics), the idea of putting the health and operation of the network in context with the payloads being delivered has become an indispensable aspect of responsible management practices. To answer this call, Application-aware Network Performance Management (ANPM) solutions have arisen and evolved, and now represent a thriving community of tools and technologies designed to serve this precise purpose.

This EMA Radar Report focuses on assessing and analyzing the landscape of ANPM solutions available in the management tools marketplace today. It represents an update to EMA’s seminal publication on this topic, EMA Radar™ for Application-Aware Network Performance Management Q3 2010, published in July 2010. While this set of solution reviews and analysis should not be considered truly exhaustive,



PAGE 2

EMA firmly believes that it is thorough and that the solutions reviewed represent the vast majority of options that are commercially available today. The specific ANPM solutions reviewed and analyzed for this report include those offered by the following vendors:

1. AccelOps

2. AppNeta

3. CA Technologies

4. Compuware

5. Dell (Quest)

6. EMC

7. ExtraHop

8. Fluke Networks

9. HP

10. InfoVista

11. Help/Systems

12. Ipswitch

13. iTrinegy

14. Lancope

15. Net Optics

16. NetScout Systems

17. Network Instruments

18. Plixer

19. Riverbed

20. SevOne

21. Solana Networks

22. SolarWinds

23. WildPackets

24. Zyrion



PAGE 3

Application Awareness in Network Performance ManagementToday’s ANPM solutions are many and varied. Independent software vendors deliver some, network equipment vendors deliver others, and yet more are elements of large multipart integrated suites offered by the largest management solutions providers. The data source techniques being used to deliver application awareness are similarly varied, though essentially clustered around four key technical mechanisms:

1. Packet inspection: This technique is perhaps the most comprehensive, delivering visibility by looking into packet headers as well as deeper into packet contents, even including payloads, in order to recognize and track applications and services.

2. Flow record analysis: These are session/flow summary records issued by network infrastructure elements (or passive monitoring devices) that provide information regarding who is using the network, what application is being used, and how well that application or service has been delivered. The most well known of these are NetFlow and sFlow, though many variants exist (i.e., J-Flow, cflow, NetStream) as well as an emerging standard, IPFIX.

3. Synthetic agents: These are software elements that generate test traffic in a variety of patterns to assess both availability and performance of specific applications or services, either from the end user’s viewpoint or from key test points around the delivery infrastructure, and measure detailed aspects of a simulated user’s experience.

4. Log file analysis: Valuable application-oriented activity insights can be found in syslog and similar data files, which capture activities and events from various systems and devices within the network infrastructure.

As with any performance management solution, vast quantities of metrics and measurements are gathered from one or more of these sources, and assembled for use by operations, engineering, and support professionals. The information gathered must be processed and presented in a way that facilitates rapid, intuitive analysis, and cross-team collaboration. A solid solution not only puts network operations and engineering professionals in a position to effectively troubleshoot application performance issues, it can also transform their entire approach to designing, operating, and managing the network infrastructure.

EMA’s focus in assessing ANPM solutions has been placed on three core use cases, which generally account for the vast majority of the purchase and deployment of such products:

1. Troubleshooting: This is perhaps the heart and soul of ANPM solutions. Product users cite the capability available within such products to reveal precisely who is using network resources and for what purposes consistently and constantly as a core value received. Primarily a reactive function, ANPM solutions can and regularly do save network teams significant spans of precious time in getting to the bottom of outages and degradations quickly and efficiently.

2. Sustained Monitoring: Most ANPM solutions have the capability to be deployed in configurations that collect performance measurements and metrics on a regular, ongoing basis, assembling a historical database and record of performance over time. A more advanced aspect of monitoring is turning on the ability of ANPM solutions to generate performance alerts and alarms, and often to add those event feeds into broader operations views and dashboards.



PAGE 4

3. Capacity Planning: The usage and activity details available with ANPM solutions are directly relevant to the research and justification processes around network planning – particularly for critical, constrained, high-cost connections such as WAN links.

Other use cases certainly exist for ANPM solutions, but are not a part of this particular research effort. Supplemental use cases that EMA encountered during this study (and during other research activities in general) are security monitoring and usage accounting/charge-back.

Criteria for InclusionIn order to be included in this study, solutions were required to support either packet inspection, flow record analysis, or both. Those solutions centered on log file analysis or synthetic agents were only included if those data sources were complemented by packet inspection and/or flow record analysis within an integrated solution. There was no restriction on size or scalability of solutions – all solutions that address ANPM requirements via the requisite techniques were eligible to be included. With that stated, EMA did not attempt to include or cover shareware, freeware, or open source solutions, and did not attempt to find and include all possible SMB-focused, entry-level products.

Scope of this ResearchIn all EMA Radar Reports, EMA evaluates solutions based on five key axes: Deployment and Administration, Cost Advantage, Functionality, Architecture and Integration, and Vendor Strength. The last category, perhaps the only one that’s not self-explanatory, is focused on the market and industry presence, vision, and financial stability of the vendor. For the 2013 EMA Radar for ANPM particular emphasis has been placed on deep and thorough analysis of Functionality, Architecture and Integration, and Cost Advantage. Deployment and Administration and Vendor Strength were also fully addressed, but on a less rigorous basis.

Functionality

100

Deployment &

Administration

100

Cost Advantage

100

Vendor Strength

100

Architecture

& Integration

100

0

10

20

30

40

50

60

70

80

90

100

Ideal 500

000

1100

220

330

4400

5555000

66600

7777777000

880

999009

Figure 1: Radar Reports will generate profiles for individual vendors based on numerical scores along five axes. An “ideal” score would look like a pentagon, as shown above.



PAGE 5

This EMA Radar Report is intended to assemble a clear picture of the many management technology vendors offering ANPM solutions, how they differ in terms of product approach, and what their core strengths and weaknesses are, so that IT practitioners engaged in research can identify the best solution for their needs. This required breaking down the major categories listed above into a number of sub-categories, as well as rolling the major categories up into aggregate scores that are then used for the comparative Radar chart plots.

EMA RADAR SCORES

PROFILE SCORESSUB-CATEGORIES

Deployment & Admin

Architecture & Integration

Cost Advantage

Resource Efficiency

Solution Impact

Vendor Strength Vendor Strength

QU

ES

TIO

NN

AIR

ES

KP

I SC

OR

ES

SUB CATEGORIES

Ease of Deployment

Support & Services

Price

Licensing Model

Maintenance Costs

Architecture

Integration & Interoperability

FunctionalityFeatures

Vision and Strategy

Financial Strength

Customer Validation

Partnerships/Channel

Figure 2: ANPM 2013 Radar evaluation model

Behind each of the sub-categories were multiple questions and points of consideration. At the lowest level, several hundred scoring variables were collected and evaluated for each vendor solution reviewed.

The three major axes for final presentation of vendor relative results are resource efficiency, solution impact, and vendor strength.

Architecture & Integration Criteria for ANPM SolutionsFollowing are the criteria used in this research report for evaluating the alternatives, methods, and approaches used in designing and architecting ANPM solution, as well as efforts and capabilities to integrate those solutions with other management platforms and systems:

•Design: As mentioned above, there are several basic approaches to monitoring application performance from the network perspective. An ANPM solution may be broad or narrow in this regard, and may be designed to be more or less real-time. It may also go beyond application-specific metrics to collect other collateral and supportive data. These aggregate design points establish the scope and depth of application performance data that can be processed and delivered by an ANPM solution.



PAGE 6

• Scalability: The basic need for any ANPM solution to support collection and storage of large volumes of performance metrics goes without saying; however, solution scope in terms of throughput capacity as well as distributed coverage are important points for consideration as potential implementers assess the best match for their particular needs.

• Breadth of environments and applications supported: While smaller managed environments may be well standardized and simple, most infrastructures have at least some mix of LAN and WAN technologies in play. The same can typically be said regarding the number and type of applications that may be present and are expected to be visible via an ANPM solution. In general, the broader the better, so that barriers to coverage and visibility are minimized, recognizing of course that there exists a mainstream of network and application technologies that must, at minimum, be supported.

• Integration and interoperability: While some ANPM solutions will come tightly integrated into a multifunction, multi-capacity suite of management tools, most will not live in a homogeneous environment. Consequently it is very important that ANPM solutions be able to integrate and interoperate with products and technologies from other vendors that fulfill complementary functions. Of particular interest within this research are integrations between ANPM solutions and event/fault management systems, service management systems, and CMDB/CMS solutions. Also of interest is whether or not open APIs are available for integration with any other IT or non-IT applications.

Functionality Criteria for ANPMFollowing are the primary feature/functionality evaluation criteria used as part of this research and analysis:

•Application discovery/recognition: If a network performance management solution is to be application-aware, one of the most important aspects of that solution is the way in which it identifies those applications. Important in this category is a range of choices and options, so that the many different subtle and unique identifiers of various types of applications can be accommodated.

•Metrics and measurement: Performance management systems generally gather a wide range and large volume of performance metrics. Presented in this category are those that are most important for characterizing application activity from the network perspective. Since the focus of this research is application awareness, traditional network-centric health measures are not included.

•Virtualization/cloud: Mixed environments of physical, virtual, and cloud infrastructure are fast becoming the norm rather than the exception. This group of measures is designed to assess how well an ANPM solution has been adapted to extend into and include monitoring of virtual networks and traffic within internally or externally hosted virtual server/compute infrastructures.

•Capacity planning: One of the many uses of application-aware data is reality-based planning, whereby capacity monitoring and changes to network capacity are informed decisions made in the full context of understanding how the network resources are actually being used in production. In particular, recognizing the influence of individual and groups of applications and the contributions they make is paramount for reducing both infrastructure cost as well as operational risk. We tested here for the ability to provide trending reports, predict exhaustion, and directly or indirectly support “what-if ” scenario analysis.



PAGE 7

•Alerting/alarming: When things go wrong, and performance problems are recognized, it is essential that personnel be notified of the situation. It is also important not to set off lights and sirens too often – today’s interconnected and interdependent IT infrastructures already generate enough event and alert chatter, without even considering what might be added via performance monitoring. Basic performance alerts and alarms need to recognize short-term and long-term patterns, as well as include as much information as possible to assist subsequent investigation and diagnosis.

•Analytics/advanced analysis: The leading edge of management technologies apply automated, intelligent analysis to the data collected by monitoring systems. Such capabilities can deliver better early recognition of performance problems and support for complex infrastructures via advanced time-aware modeling of behavior and via correlation of related metrics.

•Troubleshooting: Whether reacting to a performance problem reported by the help desk or proactively investigating a growing issue that has not yet been recognized by the end-user community, rapid and efficient troubleshooting is perhaps one of the most important objectives of network management and operations. For ANPM solutions, combining various types of data, facilitating workflows, and intuitively presenting data best facilitate troubleshooting.

•Active controls: Beyond monitoring and analysis, some ANPM solutions will deliver the ability to take direct actions in response to recognized existing or potential problems. The extent of active controls can be quite broad, ranging from intrinsic, direct capabilities to scripting to triggering actions within other management tools.

• Security and user management: ANPM solution will have visibility into detailed user activity and potentially proprietary/private data, and thus must provide access controls. While this does not constitute security management per se, ANPM solutions are often able to recognize potential security events – some solutions will be designed or optimized for this parallel purpose while others will not.

• Ease of use: Navigating, consuming, and sharing the insights gained from an ANPM solution can be difficult or easy, depending on solution design. This section reviews the ways in which an ANPM solution facilitates information presentation and collaboration via consoles and reports.

Resource Efficiency for ANPMThe criteria for resource efficiency spans two topical areas – Deployment and Administration and Cost Advantage. Following are the primary evaluation criteria used as part of this research and analysis within each of those areas:

Deployment & Administration• Ease of Deployment – This includes a number of measures meant to indicate how easy or difficult

it is to put a particular ANPM solution into the production environment and begin to draw value from it for operational purposes. As such, this section addressed three areas:

◦ Implementation cost – Specific questions assessed time to receive initial reports, time to achieve complete functionality, and the percentage of the solution cost which is typically required for professional deployment services. Also included in this part of the assessment were questions regarding product deployments models – software vs. hardware vs. virtual appliances (or other) for ANPM central servers as well as for distributed ANPM instrumentation/collectors, plus how complete, or “out of the box” the solution is as delivered by the ANPM supplier. Highest ratings were given for rapid deployment and low (or zero) need for professional services, as well as for those offering flexibility and multiple options in product deployment models.



PAGE 8

◦ Staff training – In this section, the breadth of training options as well as the length of time expected for administrators to reach basic and advanced proficiency were considered, as well as whether or not the ANPM technology provider offered formal certification programs. Top marks were given to those with broad training options, short learning curves, and existing certification opportunities.

◦ Deployment impact – Many ANPM solutions can be deployed without any disruption to the managed environment, but others will require downtime for software installation, network tapping, or other deployment steps. Some require scheduled network downtime, or a maintenance window in production operations. This measure assesses the impact each ANPM solution has on the monitored network during deployment, with preference given to those with lesser impact.

• Support and Services – An important part of any management solution is the facilities made available by the technology supplier to support initial rollout as well as ongoing production use. In this section, we evaluated several specific areas of interest for supporting and servicing ANPM solutions:

◦ Customer Support – This area investigated the variety of customer support offerings, methods for reporting product issues, and diagnostic information gathered at the point of failure. Also of interest was the presence of organized user community groups, which can act as a powerful supplemental resource to technology users. Highest ratings in this category were given to those with broad support offerings and a well-developed and well-organized user community.

◦ Professional Services - Within the range of ANPM offerings included in this study, some require significant professional services to fully deploy, while others require virtually none. In this category, we gave the highest rankings to solutions that could be deployed with minimal efforts or cost.

• Ease of Administration – Once an ANPM solution has been deployed, focus turns towards ongoing configuration and administration, to ensure that the system remains fully functional and that the maximum value can be achieved. In this section, we investigated several categories that helped to illuminate each ANPM solution’s administration facilities:

◦ Ease of administration – Here, the intent was to assess how much time was required by operations staff to keep the ANPM solution up and running, with preferential ranking granted to those requiring the lightest touch.

◦ Update process – When it comes time to apply a patch or upgrade to an ANPM solution, two questions are predominant. What will be the impact to my continuity of monitoring? And how can large numbers of distributed instrumentation devices be updated efficiently? Highest scores in this section were given to solutions that had a means to transparently apply updates without interrupting coverage and to those that included (where applicable) features for en masse updates of remote instrumentation.

◦ Testing/migration accommodation – While not universally the case, many IT shops require pre-deployment shakedowns of new or upgraded technologies (including management tools) before rolling them out into the production environment. Our interest here was to determine whether or not ANPM solution providers gave discounts for test labs and short-term migration project use of their products. Highest scores were given to those who charged lesser licensing fees (or none at all) for such uses.



PAGE 9

Cost AdvantageThe other major aspect of resource efficiency is the purchase, ownership, and licensing costs of the ANPM solution. These are representing in the following groupings:

• Solution Pricing – Purchase price is often the most immediately visible aspect of resources required to procure an ANPM solution. EMA evaluated solution pricing in three ways:

◦ First, we sought to understand the entry-level list price, including all product elements that each vendor cited as relevant to their ANPM solution. This provides insights into the absolute minimum budget required to obtain the solution.

◦ Next, we considered the average net (discounted) purchase price for customers deploying that full ANPM solution. This revealed how much most organizations would likely need to plan for if they were to truly get started with something other than a pilot deployment.

◦ Finally, we asked for list price quotations against a moderately scaled NetFlow-only ANPM scenario, a large scale packet-inspection-only ANPM scenario, and a hybrid ANPM scenario. For the packet-inspection and combo scenarios, availability and cost of long-term streaming packet storage was also taken into consideration. This provided insights into how licensing costs changed as the managed environment needs grew beyond entry level.

• Licensing Model – There are a number of means by which ANPM solutions are licensed, most commonly by link, interface, or appliance. But other options are available as well, such as subscriptions, usage, and even enterprise-wide. We assessed all of the options made available for each solution, giving scoring preference for those providing the broadest and most flexible set of licensing model options.

•Maintenance Costs – A significant component of overall total cost of ownership and operations can be found in ongoing product support and maintenance fees. We researched fees for both the lowest and highest support and maintenance tier offerings, giving preference to solutions that offered the lowest fees as a percentage of product list price. The best score was given to solutions that did not require any maintenance costs, either because they were bundled into purchase price (for the long term, not simply initial warranty periods) or because subscription pricing included maintenance and support.

Vendor Strength for ANPMThe criteria for vendor strength is measured across a number of factors that include:

•Vision/strategy – This section looked at each ANPM solution provider’s ability to clearly define and articulate its market position and strategy in comparison to its core business as well as other vendors in the ANPM market sector.

• Financial strength – Part of ANPM evaluation is the assessment of strength and staying power of the technology vendor. This section assessed each solution provider vendor’s current stability and financial outlook.

• Partnerships/channel – The best solutions surround themselves with supporting “friends,” whether they are complementary technology providers or “feet on the street” to distribute, deploy, and support the ANPM. This section evaluated the breadth and depth of technology and channel partners for each ANPM solution and vendor/provider.



PAGE 10

•Customer validation – As part of the survey process, we ask each ANPM solution provider to identify at least three customer references with whom EMA can speak, in order to validate features, functionality, and overall customer experience. The quality and strength (and in some cases quantity, lack thereof ) of customer references that fit the ANPM deployment model play an important role in EMA’s overall evaluation of each vendor and solution.

ANPM Study ResultsThis study included detailed analysis of 24 vendors and their ANPM solutions. Over 40 end-user reference calls were made as part of the validation process specifically for this study, and those findings were added to dozens of additional ANPM solution user experiences accumulated over the past 12 months as part of other EMA research projects, activities, and inquiries.

General FindingsThe field of 24 ANPM solutions and the study process produced a number of general findings that are worth revealing as pretext to the EMA Radar chart results:

• Solutions could be grouped into two major clusters: those focused on, designed for, and priced for Small-Medium enterprise organizations, and those aligned more specifically to needs of Large enterprises and communication Service Providers (SPs). This does not mean that solutions have not been successfully sold/deployed/employed in any variety of organization sizes and types, rather that in EMA’s analysis, the “center of gravity” for deployments and success reside most clearly in one or the other group. Our final results evenly divide the field into these two subgroups, with 12 ANPM vendor/solutions in each.

•Our key inclusion criterion was the requirement of NetFlow/xFlow analysis, packet analysis, or both (hybrid) as ANPM data sources. We found that all of the Large/SP ANPM solutions were hybrid in nature, whereas all but two of the Small/Medium solutions were based solely on one approach or the other.

•The strongest use case for ANPM solutions continues to be troubleshooting. Without exception, ANPM references contacted during this study cited troubleshooting as an essential way in which they used the solutions. A majority also used their ANPM tools for capacity planning – some in a systemic, ongoing manner and others on an ad hoc basis, to do spot assessments in support of capacity reviews and new capacity requests. Only about a third of practitioners actively used these solutions for sustained monitoring, and when they were it was usually for gathering data to be presented as part of daily/weekly reports. Less than one in ten were using alerts/alarms from their ANPM tools as part of production monitoring, though many more indicated that this was either planned or under consideration.

•The field of ANPM solutions and vendors continues to expand. New NetFlow/xFlow-centric solutions are coming available, as are new packet monitoring solutions. Several such new entries are covered in this report, having appeared in the market since the 2010 edition of this study.

•Many broader network management solutions have been adding ANPM features as a contributing portion of a larger value proposition. There are three such solutions covered in this EMA Radar – Zyrion, Help/Systems, and AccelOps, though many others that could have been covered declined to participate/contribute, since ANPM is a small portion of their stories.



PAGE 11

• For the first time in this edition of the study, a number of solutions have been brought to the ANPM table based on a combination of NetFlow/xFlow coverage that has network management heritage and packet-based monitoring that has an APM heritage. Such Real User Monitor (RUM) or End User Experience (EUE) tools do indeed inspect packet streams to pick out users, the applications they are using, and key metrics regarding their response time and estimated experience. Three such solutions, offered by HP, Compuware, and Dell, are included in this report.

• Lastly, from a purely technical perspective, we were struck by how many managed environments in which NetFlow collection and analysis was being used have switched over to NetFlow v9. Just over a year ago EMA research indicated that NetFlow v9 was still in very early stages of adoption, being used by less than 20% of networks. Our ad hoc assessment as part of this research indicates that penetration is much higher now, easily approaching and perhaps exceeding 50%.



PAGE 12

EMA Radar Map for ANPM Large/SP DeploymentsThe 2013 EMA Radar for ANPM shown in Figure 3 shows how twelve of the 24 ANPM solutions studied, that have proven large enterprise and service provider deployments, ranked in comparison to each other, in terms of resource efficiency (x axis) and solution impact (y axis). The size of the “bubble” indicates relative measures of vendor strength. In order to be included in the Large/SP Deployments group, solutions must have the highest architectural scalability design/capacity and the installed base for the products must have been primarily composed of large enterprise/government (over 10,000 employees) and communications service providers.

2013 EMA Radar for ANPM: Large/SP Deployments

VENDORSTRENGTH

VALUERATING

AVERAGE EXCEPTIONALBASIC

AVERAGE

EXCEPTIONAL

BASIC

RESOURCE EFFICIENCY Lower Time, Effort and Cost

Stro

nger

Fea

ture

s, A

rchi

tect

ure

and

Inte

grat

ion

SO

LU

TIO

N I

MP

AC

T

STRONG VALUE

VALUE LEADER

LIMITED VALUE

TARGETED VALUE

STRONG VALUE

VALUE LEADER

LIMITED VALUE

TARGETED VALUE

CA Technologies

Compuware

Fluke Networks

HP

InfoVista

Lancope

Network Instruments

Riverbed

SevOne

Wild Packets

CA Technologies

Compuware

Fluke Networks

HP

InfoVista

Lancope

Network Instruments

Riverbed

SevOne

Wild Packets

NetScout NetScout

EMC EMC

Distribution of Results – Large/SP DeploymentsThe results shown in Figure 3 indicate a high level of resource efficiency among the majority of solutions. All of the vendor solutions in the Large/SP Deployments are hybrid solutions that include both NetFlow/xFlow analysis and packet inspection. Four solutions – Riverbed, NetScout, Network Instruments and Fluke – reached the Value Leader category, while ranging in solution impact and resource efficiency. CA Technologies, Riverbed, and NetScout achieved the highest solution impact ratings with CA Technologies eking out a marginal lead over the other two. The majority of vendors in this segment cluster together in the Strong Value category with average-to-exceptional solution impact ratings and ranging to some degree in resource efficiency.



PAGE 13

Following are summary highlights of each of the vendor solutions analyzed within the Large/SP Deployments group, organized by value category and, within each category, presented in order of solution impact score, highest to lowest.

Value Leader: Network InstrumentsValue Leaders are those solutions that achieved the best combinations of solution impact and resource efficiency. Four ANPM solutions emerged as Value Leaders within the Large/SP Deployments group.

Network Instruments – Network Instruments continues to expand the breadth of its ANPM solution, built around a central pillar of deep retrospective troubleshooting via the GigaStor and Observer products. Heavier weights assigned to the three core ANPM use cases highlighted the considerable capabilities of the Network Instruments solution, pushing it up into the Value Leader category. The company is enjoying growing interest in its xFlow/IPSLA/SNMP features, as well as its central reporting capabilities.

EMA AnalysisBased on the results of the Radar Report study, EMA believes that the ANPM sector is alive, healthy, and continuing to grow. There are ongoing innovations via new products and new vendors, steadily increased levels of integration between ANPM components and with other management systems, as well as consolidation of suppliers. The other fascinating new dynamic is the trend towards adding product technologies with APM heritage to network-based solutions to cover the ANPM use cases. This is still relatively early in its manifestation, but has already caused considerable confusion in the market sector and will be worthy of close monitoring and analysis going forward.

Broadly speaking there are several key conclusions and observations that can be made about the state of ANPM solutions today, based on this research and our ongoing coverage of the network management market. First off, troubleshooting use cases still dominate as the most immediate and acknowledged value for ANPM solutions, followed by capacity planning and then by sustained operations monitoring. We were frankly shocked by how few were using alarms and alerting from their ANPM products, even though there was wide acknowledgement that such capabilities existed. Finally, given the clear variations in cost, coverage, and completeness, it is EMA’s conclusion that practitioners must continue to strongly consider using combinations of packet-based and NetFlow/xFlow-based ANPM technologies to meet their needs.

Based on the trends identified in this study as well as broader sector influences, there are also some conclusions that can be drawn regarding the future of ANPM solutions. First, we found continued evidence that practitioners and the vendors who serve them value integration of ANPM data and viewpoints into broader, service-oriented IT operations and orchestration. In parallel and in support of this trend, progress continues to be made in applying intelligent analytics to the huge volumes of monitoring data that ANPM systems can generate. This will be essential for helping practitioners capture the live monitoring (and, in particular, alerting/alarming) capabilities of ANPM solutions – by using advanced pre-processing to correctly identify important performance indicators worthy of bringing to the attention of operations, engineering, or planning personnel.

Radar forANPM: Q1-2013

(Large/SP Deployments)



PAGE 14

Finally, the remarkable expansion of NetFlow v9 use brings us to the next threshold of instrumentation choice, where it is becoming technologically possible to replace some of metrics that have been solely available via packet inspection with measurements gathered and transmitted via flow records. But we are merely at this threshold, and true production use of advanced/flexible features in NetFlow v9 and IPFIX is still nascent as of this writing. EMA believes that continued evolution and adoption of advanced flow record technologies will erode the need for packet-based instrumentation in remote, distributed network settings; however, ANPM use cases in the core networks and datacenters will continue to require packet-based approaches, particularly for true real-time monitoring (i.e. communications, high frequency trading) and forensic reconstruction needs.

EMA will continue to closely monitor Application-aware Network Performance Monitoring solutions, how use cases and requirements change, and how vendors and their products respond to meet them. This particular EMA Radar has been earmarked for annual refresh, so the next expected update will be in early 2014. Between now and then, EMA will publish updates as ANPM tools, technologies, and best practices evolve.



PAGE 15

Network Instruments

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

EMA Radar for ANPM: Q1 2013


Functionality

79.83

Deployment &

Administration

62.83

Cost Advantage

70.98

Vendor Strength

78.90

Architecture &

Integration

70.49

Network Instruments

Average

IntroductionNetwork Instruments is one of the original vendors that established the ANPM sector, with a solid footing in the large enterprise market and a growing presence in communication services providers. It is also an example of a solution that began with a pure focus on packet collection and analysis and expanded. When EMA published the first EMA Radar for ANPM in 2010, Network Instruments had just introduced capabilities for SNMP, IPSLA, and NetFlow/xFlow collection, and has since had time to further build out that extended set of features.

For the purpose of this EMA Radar, the following Network Instrument products were reviewed: GigaStor, Observer, Observer Infrastructure (OI), Observer Reporting Server (ORS), Network Instruments Mgt Server, and Advanced Expert Probe. ORS acts as the high-level aggregator and reporting platform for data acquired and synthesized by GigaStor, Observer, and OI. Observer serves as the primary troubleshooting and analysis console, providing advanced application diagnostics and expert analytics. GigaStor adds back-in-time functionality, capturing packets and generating metadata real-time at up to 40 Gb network speeds.

Architecture and IntegrationThe Network Instruments solution scored well in overall architecture measures, particularly with respect to the breadth of network environments and applications supported. The solution also ranked favorably in terms of scalability, and regularly leads the field for the highest speeds supported for packet monitoring and highest capacity offered for streaming packet capture. The solution also scored atop all peers for system continuity and availability.

In terms of interoperability, the solution rated among the best for support of open/custom integration, offering a broad range of APIs and data import/export features.

Radar forANPM: Q1-2013




PAGE 16

FunctionalityAs a long-standing, mature ANPM solution, it was not surprising to find that Network Instruments ranked neck-and-neck with the best for total features and functionality. The solution scored particularly well (top three) in application discovery/recognition, metrics and measurements, analytics and advanced analysis, and troubleshooting. It also notched top marks among peers for operational roles and audiences supported.

Conversations with reference users of the Network Instruments solution most clearly validated the troubleshooting use case, but operators also used the solutions for capacity planning and, to a lesser extent, operations monitoring.

Deployment and AdministrationThe Network Instruments solution fell in the center of its peer group in aggregate measures of deployment and administration, with better-than average results in deployment time and training required, and performing best in measures of implementation cost and support services.

Cost AdvantageOne of the long-standing hallmarks of Network Instruments solutions has been their highly competitive solution price points. Our analysis confirmed this, with the Network Instruments solution coming in at or near the front of its peer group across all of our cost advantage measures, spanning solution pricing, licensing flexibility, and maintenance costs.

Vendor StrengthAs a supplier of ANPM solutions, Network Instruments accumulated one of the best vendor strength scores. The company’s exclusive focus on ANPM, it’s clear core vision and strategy to pursue and develop leading-edge solutions, it’s continuous growth and profitability, and its steadily maturing partner and channel strategies combined to support an excellent result.

Strengths and LimitationsNetwork Instrument’s ANPM strengths are:

•Reference users raved about the solution’s excellence for troubleshooting application performance users – particularly the capabilities for rapidly finding and retrieving historical packet sequences for detailed analysis.

•Reference users also repeatedly mentioned their belief that the Network Instruments solution was, in their view, the most cost-effective option available for advanced packet monitoring.

•The extended breadth of the solution to include NetFlow/xFlow, IPSLA, and infrastructure SNMP monitoring provides an excellent path for broader value delivery.

Network Instrument’s ANPM limitations are:

•While there is no arguing the troubleshooting values of the Observer solution, a minority of adopters have deployed the solution for sustained operations monitoring purposes. The addition of advanced analytics for intelligent alerting/alarming could improve such use case penetration.

•Reference customers would like to see improved systems administration and management capabilities.

About Enterprise Management Associates, Inc.Founded in 1996, Enterprise Management Associates (EMA) is a leading industry analyst firm that provides deep insight across the full spectrum of IT and data management technologies. EMA analysts leverage a unique combination of practical experience, insight into industry best practices, and in-depth knowledge of current and planned vendor solutions to help its clients achieve their goals. Learn more about EMA research, analysis, and consulting services for enterprise line of business users, IT professionals and IT vendors at www.enterprisemanagement.com or blogs.enterprisemanagement.com. You can also follow EMA on Twitter or Facebook.

This report in whole or in part may not be duplicated, reproduced, stored in a retrieval system or retransmitted without prior written permission of Enterprise Management Associates, Inc. All opinions and estimates herein constitute our judgement as of this date and are subject to change without notice. Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. “EMA” and “Enterprise Management Associates” are trademarks of Enterprise Management Associates, Inc. in the United States and other countries.

©2013 Enterprise Management Associates, Inc. All Rights Reserved. EMA™, ENTERPRISE MANAGEMENT ASSOCIATES®, and the mobius symbol are registered trademarks or common-law trademarks of Enterprise Management Associates, Inc.

Corporate Headquarters: 1995 North 57th Court, Suite 120 Boulder, CO 80301 Phone: +1 303.543.9500 Fax: +1 303.543.7687 www.enterprisemanagement.com 2631-NetworkInstrumentsSUMMARY.051613

http://www.enterprisemanagement.com

http://blogs.enterprisemanagement.com

http://twitter.com/ema_research

http://www.facebook.com/pages/Enterprise-Management-Associates-EMA/232626436757941

http://www.enterprisemanagement.com

Documents

Network Instruments EMA ANPM Q1 2013 RADAR Summary