Challenges in Application Performance Management

Challenges in Application Performance Management

December 2005

©2005 TechRepublic, Inc. www.techrepublic.com. All rights reserved.

22 Challenges in Application Performance Management

Executive summary.................................................................................................................................3

Introduction..............................................................................................................................................5

Research methodology...........................................................................................................................5

Respondent demographics ....................................................................................................................5

Key findings .............................................................................................................................................7

How important is application performance? ............................................................................7 Poor application performance: What’s really happening?.....................................................13 How is application performance managed today? ................................................................17

©2005 TechRepublic, Inc. www.techrepublic.com. All rights reserved. 2


Executive summary November 2005

Today, application performance is viewed as a critical aspect of overall enterprise operations. Businesses depend upon application availability to a greater extent now than in recent years, and new challenges emerge daily due to the rise in the number of applications and their level of sophistication. Degraded performance of mission-critical applications can have serious consequences on overall business performance, which ultimately impacts bottom-line profits. Successful companies have to effectively manage application performance in order to deliver high-quality IT services to maintain Service Level Agreements and to follow the IT management best practices set forth in the IT infrastructure Library (ITIL) framework. To examine the state of application management best practices, TechRepublic surveyed over 200 North American IT and business professionals in organizations with 500 or more employees. The study was conducted to explore the degree to which organizations depend upon optimum application availability for managing mission-critical services, as well as to identify the key issues surrounding the causes and consequences of poor application performance. Specifically, we investigated:

• The importance of application performance;

• Current practices in application performance management;

• The causes and consequences of poor application performance.

Among the most compelling findings:

• Optimum application performance is a critical aspect of enterprise operations: The vast majority of respondents (86 percent) indicate that their businesses require a high level of application performance, and can tolerate only a few disruptions before revenue is lost. The three aspects of operations rated as most important were application availability (90 percent), overall business performance (nearly 90 percent), and employee productivity (85 percent).

• Many organizations rely on reactive, human-based measures as the primary means of identifying application performance problems: Over 20 percent of organizations surveyed used only calls to the help desk, or employee/management complaints to alert them to performance problems. And only 15% use integrated monitoring and notification technology solutions to proactively identify problems.

• Poor application performance results in serious problems for business operations: The three aspects of operations most adversely affected are employee productivity (68 percent), application availability (66 percent), and overall business performance (64 percent).

• Application performance degradation occurs with increasing frequency: According to our findings, application performance degradation occurs more frequently than may be anticipated. In particular, 43 percent of respondents reported that application problems occur once a month, while another 20 percent report degraded application performance daily or weekly.

• Sub-optimal application performance creates significant management challenges: In particular, degraded application performance results in poor availability and loss of reliable access to critical resources, poor Quality of Service, inability to handle unexpected peaks in traffic, and high latencies for 60 to 80 percent of the organizations studied.



• The need for network optimization or application management solutions is growing: Over one- third (36 percent) of respondents have already implemented solutions for optimizing application performance, while another 28 percent have plans to do so within the next 12 months.

On balance, the findings from this study highlight the need to take application performance seriously, rather than relegating the management of performance to the bottom of the priority list. A key challenge, of course, is identifying appropriate solutions that allow for a more proactive and integrated means of monitoring the performance of mission-critical applications. Without doubt, organizations can expect increasing demands on their limited network resources, and more traffic translates into a greater likelihood of degraded performance of key business applications. As such, technology decision makers will continue to experience increasing pressure to adopt solutions that provide real-time monitoring and performance management tools.



Introduction The growing number of business applications, along with their increasing sophistication, is presenting significant challenges in the management of enterprise application performance. What’s more, network traffic is also on the rise, which is placing a strain on access to mission-critical applications, and this situation can be exacerbated by distance across sites. The results can be poor application performance, which often leads to degraded application availability, loss of connectivity, and employee productivity, which ultimately can lead to lowered profitability. To more fully understand the issues surrounding application performance management, TechRepublic conducted a survey to explore best practices in managing application availability for mission-critical services, as well as to evaluate the issues surrounding the causes and consequences of poor application performance. Specifically, we examined:

• The importance of application performance;

• Current practices in application performance management;

• The causes and consequences of poor application performance.

Research methodology A survey invitation was sent to registered TechRepublic members between November 1, 2005, and November 10, 2005. The Web-based survey was designed to assess how organizations are handling current application performance challenges. The results reported here are based on the 219 completed surveys collected during the study.

Respondent demographics Survey respondents were selected from among TechRepublic members who met certain selection criteria, specifically those who are:

• Affiliated with organizations based in North America that have 500 or more employees.

• From a cross-section of industries.

• IT and business professionals (e.g., executives, directors, managers, developers).



The following tables contain the breakdown of the respondents by two demographics: organization size and primary job role.

Organization size

Number of employees (all locations combined) Percentage of respondents

500 – 999 32%

1,000 – 4,999 25%

5,000 – 9,999 14%

10,000 – 14,999 7%

15,000 or more 23%

Column total exceeds 100% due to rounding Primary job role

Job role Percentage of respondents

Technical 83%

IT executive 4%

IT director or manager 17%

Network or systems technologist 18%

IT consultant 9%

Developer 18%

Other technical 17%

Business 17%

Executive or senior management 7%

Department head, director, or manager 3%

Other functions 7%



Key findings How important is application performance? The first objective of this study was to determine the level of importance that IT and business professionals place on the performance of mission-critical applications and the delivery of high-performing applications. We began by asking them to rate the overall importance of application performance for their organizations. Then, we took a closer look at specific areas of their organizations’ operations in which optimal application performance plays a fundamental role. The majority of organizations in the sample require high availability for mission-critical applications, and can tolerate little in the way of disruptions before their organizations experience significant loss of revenue. In terms of overall importance, most respondents rated application performance as highly critical (43 percent) or critical (nearly 43 percent) for their organizations. Only a few respondents (under 14 percent) report their organizations do not require an optimal level of application performance. Even so, they indicated that only minor latencies can be tolerated before disruptions in application performance adversely affect operations.

How important is overall application performance to your company?

Highly Critical43%

Critical43%

Non-critical14%



We then investigated more specifically how respondents rated the importance of application performance on select aspects of their organizations’ operations. We asked respondents to rate the following aspects on a scale of 1 (very important) to 7 (very unimportant):

• Overall business performance • Application availability • Bandwidth provisioning • Hardware/server provisioning • Employee productivity • Service Level Agreements (SLAs)

Overall, the top three aspects of operations rated as important or very important were: Application availability (over 90 percent), Overall business performance (nearly 90 percent), and Employee productivity (about 85 percent). These were followed in importance by Bandwidth provisioning (68 percent) and Hardware/server provisioning (68 percent), and finally by SLAs (nearly 60 percent).

Please rate the importance of optimum application performance on the following aspects of your organization's operations, where: 1=Very important 2=Important 3=Somewhat important 4=Neither

important nor unimportant 5=Somewhat unimportant 6=Unimportant 7=Very unimportant.

0% 20% 40% 60% 80% 100%

Application availability

Overall business performance

Employee productivity

Bandwidth provisioning

Hardware/server provisioning

Service Level agreements

1 = Very important 2 = Important

3 = Somewhat important 4 = Neither important nor unimportant

5 = Somewhat unimportant 6 = Unimportant

7 = Very unimportant



We were also interested in discovering the extent to which respondents were experiencing problems with the same critical aspects of operations as a consequence of poor application performance. We asked respondents to rate each of these aspects on a 4-point scale, where 1 indicated very serious problems and 4 indicated no problems. Average ratings for each of these aspects were around 3 on the 4-point scale. That is, most respondents reported that the problems their processes or operations were experiencing were only somewhat serious. In order to determine whether some ordering of problems could be detected in the data, the 4-point scale was dichotomized to reflect two categories: “Serious” (including ratings of somewhat serious to very serious) and “No Problems.” The resulting rating is shown below, with percentages of respondents rating the item as somewhat serious to serious noted:

• Employee productivity – 68 percent • Application availability – 66 percent • Overall business performance – 64 percent • Bandwidth provisioning – 63 percent • Hardware/server provisioning – 58 percent • Service-level agreements – 55 percent

Please rate the extent of any problems your organization might be having with the following processes

or operations, on a scale of 1 to 4, where: 1=Very serious 2=Serious 3=Somewhat serious 4=No problems at all.

0% 20% 40% 60% 80% 100%

Employee productivity

Application availability

Overall business performance

Bandwidth provisioning

Hardware/server provisioning

Service-level agreements

1=Very serious 2=Serious 3=Somewhat serious 4=No problems at all

The two operational aspects that presented the most serious problems were employee productivity and application availability, strongly suggesting that these problems may well have direct negative impacts on the bottom line.



Next we asked respondents to indicate the significance of a number of challenges that often result from poor application performance. Responses were rated on a 5-point scale where 1 = Extremely Significant and 5 = Not Significant at All. Based upon percentage of responses, it was possible to rank order the various challenges that were identified as significant or extremely significant. The challenges are presented below, along with percentage of respondents rating them significant:

• Poor availability/reliability – nearly 80 percent • Poor quality of service – just over 70 percent • Diminished ability or inability to handle unexpected peaks in traffic/transactions – 67 percent • High latency – nearly 60 percent • Poor scalability – 58 percent • Can't effectively reach/support global user/customer base – 54 percent • Lack of reliable security – 52 percent • Diminished ability or inability to monitor performance real-time – just over 50 percent • Diminished ability or inability to support encryption – over 40 percent

Please indicate on a scale of 1-5 the significance of each of the following challenges that can result from poor application performance, where: 1=extremely significant 2=significant 3=Neutral

4=somewhat insignificant 5=not significant at all.

0% 20% 40% 60% 80% 100%

Poor availability/reliability

Poor quality of service

Diminished ability or inability to handleunexpected peaks in traffic/transactions

High latency

Poor scalability

Can't effectively reach/support global user/customer base

Lack of reliable security

Diminished ability or inability tomonitor performance real-time

Diminished ability or inabilityto support encryption

1=Extremely significant 2=Significant 3=Neutral 4=Somewhat significant 5=Not significant at all

What these data show is that by merely improving availability and reliability, quality of service, and providing sufficient bandwidth, over two-thirds of the problems deriving from poor application performance could be solved.



Next, we looked at a key potential driver for the decision to purchase application performance products or services, namely, an increase in application performance. We asked respondents to estimate the percentage of increase in performance that would lead them to purchase an application performance management solution. There was a wide range of estimates for the amount of increase that would be needed to prompt the purchase of application performance products. The largest segment of respondents, 81 percent in all, would require at least a 21 percent increase in performance to justify such purchases. Fully half would require performance improvement levels 31 percent or higher. Just less than one-fourth would demand at least a 51% increase in performance before they would contemplate buying a performance management solution.

How much of a performance increase would it take for you to consider the purchase of application performance products or services?

0% 10% 20% 30% 40% 50%

1%-10%

11%-20%

21%-30%

31%-50%

51%-100%

More than 100%



Finally, we sought a better understanding of what role these respondents have in the decision process for enterprise software solutions so we could put the study results in proper context. Our data show that roughly three-quarters of the sample evaluate, recommend, or specify brands or actually implement, manage, or maintain enterprise software solutions. Another 12 percent authorize the final purchase decision, while about 13 percent classify themselves as end-users. Hence, the results reported here derive from a knowledgeable and experienced sample of IT decision makers, influencers and end-users.

What is your role in your organization with respect to enterprise application solutions?

0% 10% 20% 30% 40% 50%

Authorize the f inalpurchasing decision

Evaluate, recommend,and/or specify

the brand(s) purchased

Implement, manage, and/ormaintain

enterprise softw aresolutions

End-user

In this section, we’ve learned that IT and business professionals place a premium on application performance and we’ve seen which aspects of operations and processes are most critical and, therefore, suffer most from degraded performance. Additionally, we gained a sense for the levels of performance improvement expected by technology decision makers before they value software solutions that accelerate applications or optimize network traffic. The next section looks at what is really happening when application performance degrades.



Poor application performance: What’s really happening? This section of the study characterizes what happens when poor application performance occurs in daily business operations. Specifically, we analyze the frequency with which organizations experience application performance problems, identify the types of applications that are among the poorest performing, and profile the chief causes and consequences of degraded application performance. Organizations experience incidents of significant application performance degradation to varying degrees. Specifically, about 43 percent experience performance degradations about once a month, followed by one-third who experience such problems once a year or less. Surprisingly, nearly 20 percent report experiencing degraded application performance much more frequently; about 14 percent report problems in performance occur once per week, and just over 5 percent experience such problems on a daily basis. Given the importance of application performance for the majority of organizations in our study, a significant minority report degraded application performance occurs rather frequently. This finding highlights an area that needs attention – maintaining adequate application performance. Without question, frequent lapses in application performance adversely affect bottom line profits. Recall our earlier finding that over 80 percent of organizations rated application performance critical or highly critical to their businesses.

About how often does your company experience incidents of significant application performance degradation?

Once a year or less33%

Once a month43%

Once a week14%

Once a day5%

Never5%



Perhaps more revealing—over 40 percent of respondents said that the incidence of application slowdowns has remained unchanged over the past 12 months, while nearly 30 percent indicate that application performance problems have increased during that same time frame. Fewer than 30 percent report a decreased incidence of application slowdowns over the last 12 months. Taken together, these findings indicate that, in general, the frequency with which organizations experience lapses in application performance is unchanged versus the last year, while nearly equal ratios report the situation has worsened and improved. One word of caution, though, when interpreting this data point—recall that two-thirds of respondents report that their organizations have at least somewhat serious problems from poor application performance, in particular lowered employee productivity and poor application availability. When viewed in the context of the problems identified in this study, rating the frequency of slowdowns over the past year as unchanged does not mean the situation is improved or acceptable in terms of application performance metrics. Please estimate how the number of such application slowdowns has changed over the past 12 months?

Increased29%

Decreased26%

Unchanged41%

Don't know4%



We delved deeper into the issues of sub-optimal performance to identify general classes of applications that tend to encounter or engender the poorest performance. Overall, about 43 percent of respondents identified applications that are packaged—off-the-shelf—as among the poorest performing. The next poorest performing are those that rely on the Internet as a communications platform for application traffic (37 percent), followed by applications that are accessed globally (32 percent), and those developed in-house (nearly 30 percent).

Which applications are among the poorest performing in your organization?

0% 10% 20% 30% 40% 50%

Those that are packaged applications

Those that use the Internet as a communications platform for application traffic

Those accessed globally

Those developed in-house

Other

Respondents could select more than one answer.



So what causes application performance to degrade? Our participants report that poor application architecture/logic (nearly 60 percent), and usage overload/inadequate load balancing (53 percent) are the most frequent root causes of poor application performance. In addition, over 35 percent of respondents cite code inefficiencies. An equal percentage of respondents blame a lack of adequate bandwidth (29 percent) and poor application management (29 percent) as the genesis of poor performance.

To the best of your knowledge, what do you think are the main causes that lead to poor application performance in your organization?

0% 20% 40% 60% 80% 100%

Poor application architecture/logic

Usage overload/inadequate load balancing

Code inefficiencies

Lack of adequate bandwidth

Poor application management

We don't have performance issues.

Other


So, poor application performance has serious, measurable consequences and traceable origins—most often bandwidth overload and poor application architecture and/or logic. The more that network resources are tapped, the more likely that application performance problems will continue and perhaps even worsen, if appropriate solutions to these problems are not identified and embraced.



How is application performance managed today? In the final section of the study, we explore the methods or tools that organizations employ to manage application performance. We began by examining the extent to which organizations have implemented network optimization solutions. As illustrated by the chart below, just over one-third (36 percent) of respondents’ organizations have implemented network optimization or application management solutions to improve application performance for end-users. Another 28 percent currently have plans for implementing application performance management solutions within the next 12 months; although about the same percent currently have no plans to implement such solutions at this time. The trend in these responses is clear—a growing number of organizations are moving toward strategies to optimize network performance. Still, a significant minority of respondents (about 27 percent) currently have no plans in place to implement such solutions. This illustrates that a sizeable portion of organizations in our sample are in limbo—stuck between recognizing that these solutions are important and actually taking action to implement them.

Which best describes your organization's current state of application performance?

0% 10% 20% 30% 40% 50%

Implemented network optimization and/or applicationmanagement/acceleration solutions

Have not yet implemented network optimization and/orapplication management/acceleration solutions,

planned in the next 12 months

No plans to implement network optimization and/orapplication management/acceleration solutions

Don't know



The next question explores specific applications that are currently run on enterprise networks. More importantly, we asked respondents which of these applications they consider be most critical to the successful operation of their businesses. Overall, nearly 72 percent of respondents list Enterprise Resource Planning, Accounting, and Order Entry as the most critical business applications that currently running on their networks. At a distant second are Portals, Collaboration, and Content Management applications (just over 50 percent), and Customer Relations Management applications (49 percent). Other critical applications include Supply Chain Management (35 percent), Voice over IP (about 21 percent). Miscellaneous applications include those that are Web-enabled (15 percent) and those that are not specifically Web-based applications (12 percent).

What applications run over your network that you consider critical to your business?

0% 20% 40% 60% 80% 100%

Enterprise Resource Planning, Accounting, Order Entry

Portals, Collaboration, Content Management

Customer Relationship Management

Supply Chain Management

Voice over IP

Other Web-enabled applications

Other




Next, we asked respondents about best practices used to determine the existence of application performance problems. Most of the respondents rely on multiple approaches for determining if there is a problem with application performance. Surprisingly, reactive approaches are used about twice as often as proactive solutions to uncover the existence of performance problems. The largest segment our sample, nearly 80 percent, know a problem exists when calls to the help desk arrive. Another predominant reactive solution is employee complaints (73 percent). For the proactive solutions, monitoring network performance is most common approach, while monitoring application performance and the use of active/passive agent tests are used with similar frequency.

How does your organization today determine that application performance problems exist?

0% 20% 40% 60% 80% 100%

Calls to the help desk

Employee complaints

Personal experience using the network

Management complaints

Monitoring of network performance

Monitoring of application performance

Active/ Passive agent test

Other

Proactive notification methods

Reactive notification methods




Since respondents could select multiple approaches in this question, we wanted to know if there were any firms who used only a single approach to identifying performance issues, and what preferred approach or combination of approaches is used. The following chart depicts the distribution of the various types of proactive monitoring and the percentage of respondent organizations that rely only on reactive measures. Nearly 30% of participants report their organization relies on a combination of network and application performance monitoring (along with one or more human-based methods of monitoring). Only 15 percent rely on network and application monitoring in concert with active-passive agent tests. The biggest surprise is that over 1/5 of our sample report confess that their organization doesn’t use any proactive monitoring solutions.

Distribution of Reactive and Proactive Application Performance Monitoring*

0% 10% 20% 30% 40% 50%

Active-Passive Agent Test

Network performance monitoring/Active-PassiveAgent test

Application Performance monitoring/Active-Passive Agent test

Application Performance monitoring

Network/Application performance monitoring &Active-Passive Agent test

Network performance monitoring

Reactive, human-based monitoring only

Network/Application performance monitoring

% of Respondents

* All the organizations that use proactive solutions alone or in combination with other proactive solutions also employ

one or more of the reactive, human-based approaches to monitoring.



Managing wide area network (WAN) accessibility and performance presents unique challenges to the IT organization and is necessarily a part of managing enterprise-wide application performance. To identify the challenges that organizations are facing in this area, we delved into current practices for ensuring business application performance across WANs. The majority of organizations with WANs use Firewalls (nearly 60 percent) to restrict traffic while nearly 55 percent add bandwidth directly to the WAN to achieve sufficiency. Other strategies include Content filtering (about 40 percent), Monitoring employee traffic (37 percent), Restricting desktop applications (almost 30 percent), and using Multi-protocol Label Switching (MLS; nearly 20 percent).

What steps does your organization take to ensure that business applications over the WAN get the necessary bandwidth to perform effectively?

0% 20% 40% 60% 80% 100%

Firewall

Add bandwidth

Content filtering

Employee traffic monitoring

Restrict desktop applications

Queuing in the router

Use MPLS (Multiprotocol Label Switching)

We do not have a WAN

Other


In this section, we’ve seen that the majority of organizations are either moving toward strategies to optimize network performance or have already done so. With critical business applications increasingly relying on burdened enterprise networks, more proactive approaches are need for detecting and managing slowdowns before service interruptions or application failures occur.



Additionally, this study clearly demonstrates that the chief sources of application degradation are the very factors intrinsic to any growing business today—increasing network traffic and a growing number of business applications that are increasingly complex to manage. Among the areas most affected by poor application performance are those most critical to operational efficiency, namely network reliability, service quality, and high latencies. And finally, these data serve to highlight the importance of and demand for network optimization and application performance management solutions. Such enhancements will increase performance reliability and provide the end-user with consistent access—laying a solid foundation for maximizing productivity and improving profitability.

CNET Network’s TechRepublic Community Research Programs CNET Networks’ TechRepublic Community Research team conducts surveys of the CNET Networks, Inc. and TechRepublic membership on a project basis. Projects are funded by CNET Networks, Inc., and in some cases by vendors who have particular interests in topical areas. In cases where the project has been sponsored by a third party, the Community Research team leads the effort in developing survey questions and has final approval of all questions. The Community Research team conducts all analyses and writes the final report that is subject to CNET Networks’ editorial review. Funding for this project was provided by Akamai, Inc. If you have a topic of interest for either editorial or sponsored research, please e-mail us at [email protected].


Technology

Challenges in Application Performance Management