The Forrester Wave: Application Lifecycle Management Report

Forrester Research, Inc., 60 Acorn park Drive, cambridge, mA 02140 UsA

Tel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.com

The Forrester Wave™: Application Life-Cycle Management, Q4 2012by Tom Grant, ph.D., October 23, 2012

FOR: Application Development & Delivery professionals

Key TaKeaWays

Leaders Build product strategy around aLM Use CasesForrester Wave evaluation Leaders IBM, Rally Soft ware, PTC, CollabNet, Microsoft , and Serena Soft ware off er strong solutions for ALM problems that no single point solution could address. Th eir product strategies focus on such challenges as delivery and the disruptions of Agile adoption, not just throwing new capabilities at the customer.

strong performers and Contenders Find a specifi c audienceAtlassian and Rocket Aldon don’t focus as much on specifi c use cases that span many ALM activities. Instead, they build tools to address smaller but widespread problems, such as release management and developer collaboration. Meanwhile, HP is expanding its ALM off ering beyond Quality Center.

aLM is Going Through a period of Redefi nition and innovationHot ALM issues such as DevOps, embedded soft ware, and requirements defi nition show how the concept of ALM has expanded and changed. During this transition, the seemingly crowded ALM market will benefi t from the innovation that comes from a wealth of competing and complementary product off erings.

© 2012, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please email [email protected]. For additional information, go to www.forrester.com.

FOR ApplIcATIOn DEVElOpmEnT & DElIVERy pROFEssIOnAls

Why Read This RepoRT

In Forrester’s 116-criteria evaluation of application life-cycle management (ALM) vendors, we identified the nine most significant software providers in the category — Atlassian, CollabNet, HP, IBM, Microsoft, PTC, Rally Software, Rocket Aldon, and Serena Software — and researched, analyzed, and scored them. This report details our findings about how well each vendor fulfills our criteria and where they stand in relation to each other to help application development and delivery professionals select the right partner for their software-fueled business innovation efforts.

Table Of contents

innovation drives interest in application Life-Cycle Management

application Life-Cycle Management evaluation overview

The aLM Forrester Wave Finds Many Leaders and different directions

Vendor profiles

supplemental Material

notes & Resources

Forrester conducted lab-based evaluations in september and October 2011 and interviewed nine vendor and user companies: Atlassian, collabnet, Hp, IBm, microsoft, pTc, Rally software, Rocket Aldon, and serena software.

Related Research Documents

The Time Is Right For Alm 2.0+October 19, 2010

The Forrester Wave™: Agile Development management Tools, Q2 2010may 5, 2010

The Forrester Wave™: application Life-Cycle Management, Q4 2012IBm, Rally software, pTc, collabnet, microsoft, And serena software lead The pack, With Hp, Atlassian, And Rocket Aldon Followingby Tom Grant, ph.D.with Kyle mcnabb and Alissa Anderson

2

6

10

14

17

OcTOBER 23, 2012

www.forrester.com

http://www.forrester.com/go?objectid=RES56832



http://www.forrester.com/go?objectid=BIO1887


The Forrester Wave™: Application life-cycle management, Q4 2012 2

© 2012, Forrester Research, Inc. Reproduction prohibited October 23, 2012

iNNoVaTioN dRiVes iNTeResT iN appLiCaTioN LiFe-CyCLe MaNaGeMeNT

Business innovation now drives the application life-cycle management (ALM) market. The contribution that application development and delivery makes to a company’s business goals —making workers more productive, creating engaging customer experiences, and bringing new software products to market more successfully — must be more direct and successful. Software increasingly plays a central role in a firm’s ability to deliver new products and services or exploit new channels. Firms can no longer accept historical gulfs between business and application development and delivery teams as, increasingly, firms now expect to manage application development and delivery as a business and treat it as a competency.1

Unfortunately, everyone has their long list of reasons why they struggle with better managing application development and delivery. The work is creative, and teams must figure out how to work with one another. Tools or techniques that impose an assembly-line model will fail. Application development and delivery still need management, but the tools of management must fit the nature of the creative and collaborative work.

Software and application development work itself is changing. This Forrester Wave™ evaluation examines the most significant developments that affect both ALM strategy and ALM tools. For example, the definition of ALM has stretched to include not just development but also delivery (see Figure 1). Some of the vendors we assess have expanded their tools portfolio to add or enhance their support for release management and other delivery-related activities.

The need to support the business management of application development and delivery underlies this Forrester Wave’s evaluation criteria:

■ Delivering business value, not just driving process efficiency. ALM’s original aim was process improvement. Firms turned to ALM to do a better job of managing project timelines, dependencies, and deliverables; ensure that every requirement could be traced to a real software capability; and test to ensure that the software worked. While process improvement is still important, increasing efficiency in building and delivering software is as important a goal for ALM as proving the value of the software, as it contributes to business outcomes.

■ Supporting individual, team, and institutional management. ALM’s first wave of tools focused on — in truth — application development management (ADM), designed to increase developer and tester productivity. Now, firms seek ALM tools to improve team success while linking these teams to the larger application development and delivery activity within the organization or, in many cases (such as offshore partners), outside of it.

■ Extending life cycles to include delivery. Organizations now recognize that the life cycle of ALM does not stop with check-in or build phases. The real life cycle and process goes on much longer, arguably all the way to the moment the application goes live and even beyond to support continuous feedback.




■ Managing software that gets deployed everywhere. Software is truly everywhere, from data center servers to laptops, from mobile devices to cars and refrigerators. Regardless of where the software winds up, software development and delivery management is still necessary.

■ Integrating across tools. Vendors have reluctantly admitted that customers are not interested in their suite of tools, no matter how impressive or cost effective they might be. Switching costs can be high, and many capabilities, such as issue tracking and source control, are so commoditized that switching makes little sense. Some tools, such as HP Quality Center, are so widely adopted that other vendors must integrate with them in some fashion. The speed of tool innovation in some areas, such as continuous delivery and requirements, outpaces what any single vendor can keep up with, providing another reason for integrating across tools.

■ Using reporting and dashboards as instruments of change. Many application development and delivery organizations aspire to be “learning organizations” and see their investment in ALM tools in that light. The data in an ALM system provides a yardstick for a team’s continuous improvement efforts. Some teams use this information to communicate with outside parties about the shape and value of new Agile and Lean practices. Larger organizations may use the same information to track software value streams across projects and teams. In these and other scenarios, reporting and dashboards provide the ultimate payoff for ALM investments.

Figure 1 ALM Boundaries Expand To Encompass The Software Value Chain

Source: Forrester Research, Inc.60080

Production planningclosed loop

Change management Service management

Portfolio management

Change-awarecontinuous integration

Just-in-timedemand

management

Releasemanagement

Testing andquality assurance

Build andsoftware

con�gurationmanagement

Deployment

Projectmanagement

Production controlclosed loop




aLM Vendors have Taken Many steps Forward, But a Long Road Remains

ALM tools have made significant progress since our last evaluation:

■ ALM vendors have discovered macro differentiators. In past years, ALM vendors did far less in their product strategy and product marketing to clearly differentiate their products. With notable exceptions such as Rally Software, a company that built its ALM strategy around Agile support, ALM vendors had noticeable differences only if you carefully examined the details of their product features. Today, ALM vendors more clearly identify the types of customers they serve and the problems they can help these customers address. Serena Software, for example, tells a very clear story about the challenges orchestrating development and delivery in large IT organizations and how its products help organizations address those challenges.

■ Vendors help improve the efficiency of the larger ALM process. As noted earlier, the boundaries of ALM have expanded to include more of the delivery phase. In response, ALM vendors have expanded their capabilities in this area to remove the inefficiencies that arbitrarily lengthen and complicate the delivery phase.

■ Requirements, a part of ALM, need innovation attention. Recent innovations in requirements tools, such as visualization and social media integration, have increased the depth and accuracy of insights and the speed of collecting them. These innovations have occurred in requirements-focused companies such as Balsamiq Studios, Blueprint Software Systems, iRise, and Visure Solutions. Even though ALM vendors provide requirements tools, they have not kept pace with these innovations.

■ Mobile and embedded development scenarios get insufficient attention. Embedded software developers still get scant attention, with the exception of MKS (PTC) and IBM. Surprisingly, given how hot the mobile development market is, ALM vendors by and large do nothing special for mobile developers, including in areas such as testing where there is a large need for better tools.

■ Collaboration is the lightning that ALM vendors can’t capture in a bottle. Many of the biggest ALM challenges, such as coordinating the work across geographically distributed teams, boil down to collaboration problems. Collaboration presents some extremely vexing puzzles for tools vendors, particularly given how idiosyncratically a specific set of people may collaborate with each other. Still, some basic collaboration features — desktop sharing for team members in different locations and wikis for capturing and updating information, for example — are far less common than expected.




The Market is still open For innovation and New Contenders

The vendors we assessed in this evaluation provide an impressive array of tools with a broad range of capabilities. Nevertheless, they do not represent every possible innovation in the expanding and evolving realm of ALM. The ALM market is still wide open for anyone to devise a better way to address a problem, identify new and unmet challenges, or give a particular class of customers more attention than the bigger, general-purpose vendors can.

Vendors listed in this section were not evaluated but represent various innovative approaches to ALM, demonstrating how much room remains within this market for innovation and specialization:

■ Blueprint focuses on requirements. Some aspects of ALM pose tough challenges that don’t have an easy solution. Smaller vendors focused exclusively on some of these challenges can innovate in ways that larger suite vendors may not. Blueprint, a requirements tools vendor, provides a good example of the important role that these smaller companies with a much narrower focus play in today’s ALM market.

■ Hansoft caters to video game developers. Since the video game market is nearly as large as the movie industry, it should be no surprise that at least one vendor, Hansoft, has identified this market as its core constituency.

■ Polarion tackles difficult compliance challenges. Few of the vendors we evaluated have taken steps to address complex compliance requirements, leaving no small amount of opportunity for a smaller vendor such as Polarion Software.

■ SmartBear Software makes quality the starting point. While many ALM vendors are shifting their attention to later parts of the development and delivery process, SmartBear Software makes those stages the starting point for its product strategy. This strategy includes both code quality improvement through review and testing and quality of service maintained through web monitoring.

■ Tasktop is the Switzerland of integration. Each ALM vendor has its own set of integrations with other tools, both open source and commercial. Each ALM vendor also has its own approach to integration. Unsnarling this tangle sometimes requires going to a neutral third party, such as Tasktop, for cross-ALM integration.

■ ThoughtWorks provides thought leadership in continuous delivery. A real thought leader in continuous delivery, ThoughtWorks provides a modest range of tools but a very established consulting business (primarily aimed at helping with Agile adoption).

■ VersionOne supports Agile. Rally has built Agile and Lean support into its tools, and other vendors such as Microsoft and IBM provide Agile-specific configurations. With Agile




continuing to spread, there is still room for another Agile-focused ALM vendor such as VersionOne to meet the needs of Agile teams in a different way than its competitors.

appLiCaTioN LiFe-CyCLe MaNaGeMeNT eVaLUaTioN oVeRVieW

To assess the state of the application life-cycle management (ALM) market and see how the vendors stack up against each other, Forrester evaluated the strengths and weaknesses of nine top vendors. These companies fell into two broad categories:

■ Large software companies that, among other products, also offer ALM tools. In this category, HP, IBM, and Microsoft maintain a strong position in the ALM market.

■ Smaller software companies that specialize in ALM. Other companies included in this evaluation, such as Atlassian, CollabNet, Rally Software, Rocket Aldon, and Serena Software, have made ALM their exclusive business. Until its recent acquisition by PTC, MKS also fell into this group.

evaluation Criteria Measured Breadth of portfolio and support For Use Cases

After examining past research, user need assessments, and vendor and expert interviews, we developed a comprehensive set of evaluation criteria. We evaluated vendors against 116 criteria, which we grouped into three high-level buckets:

■ Current offering. ALM covers a vast expanse of functionality for developers, business analysts, testers, project managers, and people in a wide array of other roles. Therefore, we developed more than 86 distinct criteria to measure the strength of each vendor’s tools.

■ Strategy. The value that a vendor brings to the table goes beyond just its current tools, so we included 18 criteria that assessed each vendor’s ability to support customer implementations, its commitment to the ALM market, its ALM product strategy, and its corporate strategy.

■ Market presence. We also took into account the size of the footprint each vendor has in the software market. Forrester’s clients want to know how safe a bet it will be that a given ALM vendor will be in business several years from now in order to gauge how safely they can invest in that vendor’s ALM tools.

This year’s ALM evaluation represents a departure from the 2010 Forrester Wave evaluating Agile development management solutions in three ways:2

■ Opening the aperture to encompass all ALM scenarios. The 2010 evaluation focused exclusively on Agile support among ALM vendors. This year, we include both Agile and non-Agile use cases in the criteria.




■ Treating Agile and Lean as a critical test of ALM offerings. At the same time, this year’s evaluation by no means leaves Agile behind. Instead, we treat Agile and Lean as critical tests of an ALM vendor’s offering.

■ Incorporating systems engineering and embedded software. Given the increasing quantity of hybrid products — those incorporating software into physical products ranging from cars to medical devices — we have also included criteria covering software engineering outside of corporate IT and independent software vendors (ISVs).

evaluated Vendors have strong Tools and substantial Market presence

Forrester included nine vendors in the assessment: Atlassian, CollabNet, HP, IBM, Microsoft, PTC, Rally Software, Rocket Aldon, and Serena Software. Each of these vendors has (see Figure 2):

■ Depth and breadth of ALM offering. While there are many commercial and open source point solutions for particular activities (test management, build management, issue tracking, etc.), the title “ALM vendor” goes to companies that do more than provide a single capability (source control, defect tracking, etc.), however good it might be. We expect these tools to cover the majority of ALM use cases, as Forrester defines ALM.

■ A broad ALM customer base. We also looked at the number of customers for a vendor’s ALM products in 2011 (at least 750) as well as the growth in the vendor’s customer base from 2010 to 2011.

■ Definitive revenue from ALM. Finally, the criteria included the revenue from ALM. Once again, we included both the numbers for 2011 (at least $19 million) and the growth since 2010.




Figure 2 Evaluated Vendors: Product Information And Selection Criteria

Source: Forrester Research, Inc.

Vendor

Atlassian

CollabNet

HP

IBM

Microsoft

PTC

Product evaluated

Atlassian OnDemand:JIRAGreenHopperBon�reCon�uenceSharePoint Connector for Con�uenceTeam Calendars for Con�uenceBambooFishEyeCrucibleCloverCrowdBitbucketAtlassian IDE Connector for EclipseAtlassian IDE Connector for IntelliJ IDEAAtlassian IDE Connector for Visual Studio 2008 and 2010HipChatStashSourcetreeJIRA Mobile ConnectJIRA OnDemandGreenHopper OnDemandBon�re OnDemandCon�uence OnDemandTeam Calendars for Con�uence OnDemandSource OnDemand Bundle (FishEye, Crucible, and Subversion)

TeamForgeScrumWorks ProSubversion EdgeLab Management

HP Application Lifecycle Management (HP ALM) 11.5

Rational Collaborative Lifecycle Management Solution (Rational CLM):Rational Requirements Composer 4.0Rational Team Concert 4.0Rational Quality Manager 4.0

Microsoft Visual Studio

Integrity

Product versionevaluated

5.05.102.24.2

1.5.32.34.12.72.7

3.1.62.4.2

Weekly builds3.5 or later7 and later

N/A

Weekly builds1.1

Weekly buildsN/A

Weekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly builds

Weekly builds

6.26.03.02.5

11.5

4.0

2010

10.1

Versionrelease date

June 2012

Weekly builds

Weekly builds

Weekly builds

Weekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly builds

Weekly builds

June 2012May 2012April 2012May 2012

June 2012

June 2012

May 2010

June 2012




Figure 2 Evaluated Vendors: Product Information And Selection Criteria (Cont.)


Vendor

Rally Software

Rocket Aldon

Serena Software

Product evaluated

Rally Community EditionRally Enterprise EditionRally Unlimited EditionRally Quality ManagerRally Support ManagerRally Product ManagerRally Community ManagerRally Time TrackerRally Idea ManagerRally Portfolio Manager

Aldon Lifecycle ManagerAldon Deployment ManagerAldon Community ManagerAldon Report ManagerAldon Agile Manager 1.0

Development ManagerRelease ManagerRequirements ManagerService ManagerDemand ManagerRequest CenterDashboardChangeMan ZMFAgile Planner

Product versionevaluated

Weekly builds

(e) 6.1/(i) 7.6N/A9.5.41.51.0

1.21.02.13.13.13.12.17.12.3

Versionrelease date

Weekly builds

May 2011N/ANovember 2010June 2011January 2011

May 2012

Vendor selection criteria

The vendor provides designated ALM product(s) and a stated ALM road map.

The vendor has more than 750 customers.

The vendor provides tools that cover the majority of ALM use cases, as Forrester de�nes ALM.

The vendor generates more than $19 million in revenue.




The aLM FoRResTeR WaVe FiNds MaNy LeadeRs aNd diFFeReNT diReCTioNs

The vendors we evaluated for the ALM Forrester Wave provide a wide range of capabilities. Superficially, they look very much alike. Project management? Check. Issue tracking? Check. With such feature breadth, you might conclude that ALM is a mature market.

But where are the vendors heading? The answer is, in very different directions. Software may be everywhere, but the form of software development and delivery varies widely across organizations. While this market now has many genuine leaders, they have reached this point only by identifying their core market. Therefore, the market is less mature than it might seem on the surface. When evaluating ALM vendors, it’s important to consider not just the capabilities they provide but also the use cases for which they are optimized.

Nowhere is this differentiation more clear than in the overarching principles that guide the evaluated vendors’ product strategies (see Figure 3):

■ Leaders IBM and Microsoft provide broad ALM capabilities. Both IBM and Microsoft have maintained a strong portfolio of ALM tools designed to support common use cases for software development and delivery. The two companies both provide broad swaths of capabilities for common ALM activities, such as project definition and management, reporting, issue tracking, and task assignment.

IBM and Microsoft come from a more general-purpose perspective on ALM, and both treat Agile practices as a special instance of ALM workflows, metadata, and reporting. Both companies support Agile as a template, a special configuration of their ALM tools.

■ Leaders PTC and Rally Software lead by focusing on specific use cases. In contrast to IBM and Microsoft, both Rally and PTC (originally MKS, before PTC’s acquisition of the company in 2011) have succeeded in the ALM market by finding a niche in which to specialize — for Rally, Agile, and for PTC, compliance. While this approach may limit their ability to support a broader swath of scenarios, their focus gives them the opportunity to provide thought leadership in their respective areas. Rally’s focus on Agile and Lean, for example, has led the company to build top-notch planning capabilities into its tools. PTC’s focus on compliance, including the highly regulated world of embedded software, has made it the only ALM vendor to provide out of the box many of the workflows and reports needed to satisfy regulatory authorities.

■ Leaders CollabNet and Serena lead through delivering differentiated and broad capabilities. Both CollabNet and Serena have tailored their ALM offerings to more-general value propositions and needs. CollabNet sees its opportunity in the cloud, where ALM tools support has been lagging behind IT’s need to manage cloud development, testing, and deployment more effectively. Serena has recognized how important it is for organizations to effectively orchestrate the larger “meta-process” of software development and delivery, which spans the typical




activities (coding, testing, building, etc.) that guide the development efforts of other commercial and open source tools.

■ Strong Performers Atlassian and HP provide focused capabilities. Atlassian and HP show how far an ALM vendor can go with strong support for a piece of the overall life cycle. However capable individual tools such as JIRA and Quality Center may be, they are no substitute for a broader ALM offering. The two companies view this gap differently. Rather than defocus its product strategy, Atlassian prefers to provide a smaller range of strong capabilities than potentially do a mediocre job tackling a wider range of ALM challenges. This strategy makes it easier for Atlassian to deal with one of its biggest challenges, perception as an enterprise-ready vendor, without the added complexity of too many products and features.

In contrast, HP has pushed hard to fill in the gaps in its ALM offering, moving far past Quality Center. The current version, ALM 11.5, provides a very robust set of tools that can compete with other ALM suites on a wide range of capabilities. Some of these new capabilities, such as the executive dashboard features, demonstrate that HP understands many tough ALM challenges and will invest in providing credible, differentiating features to address them. However, these tools are relatively new, and Forrester has not encountered many implementations of them yet. The next year will be critical for HP’s ALM offering in gaining market traction.

■ Contender Rocket Aldon is the operational dark horse of ALM. While most other ALM vendors strive to move down the value chain to the point of release and deployment, Rocket Aldon’s product strategy starts with these later stages of the application life cycle. Automation plays a prominent role in Rocket Aldon’s offerings, as do IT-governance-related features. The Rocket Aldon road map is, to some extent, extending upstream to cover scenarios (for example, product owner review of the backlog) that it currently does not address as thoroughly as production-related activities.

This evaluation of the ALM market is intended to provide a starting point for your own evaluation. We encourage readers to view the detailed product evaluations and adapt the criteria weightings to fit their individual needs through the Forrester Wave Excel-based vendor comparison tool.




Figure 3 Forrester Wave™: Application Life Cycle Management Tools, Q4 2012


Go online to download

the Forrester Wave tool

for more detailed product

evaluations, feature

comparisons, and

customizable rankings.

RiskyBets Contenders Leaders

StrongPerformers

StrategyWeak Strong

Currento�ering

Weak

Strong

Market presence

Atlassian CollabNet

HP

IBMPTC

Rally

Rocket Aldon

Serena

Microsoft




Figure 3 Forrester Wave™: Application Life Cycle Management Tools, Q4 2012 (Cont.)


CURRENT OFFERING Platform support Installing the product and getting going Administration Security Integration and customization Management Change elements Life-cycle traceability Reporting Analytics Running a project General collaboration capabilities Process con�guration

STRATEGY Support for implementations Product strategy Corporate strategy Price Commitment

MARKET PRESENCE Installed base Financial strength Employees Support services Channel partnerships Global presence

Atla

ssia

n

3.473.404.80

4.002.204.803.952.473.632.751.384.754.702.24

2.922.502.904.500.005.00

3.943.304.002.004.105.003.00

Forr

este

r’sW

eigh

ting

50%5%5%

2%3%

10%5%

10%15%

5%10%10%10%10%

50%15%80%

5%0%0%

0%35%

5%0%

35%20%

5%

Colla

bNet

3.514.104.40

4.752.604.903.893.743.263.702.284.302.902.36

3.954.903.705.000.005.00

4.424.203.001.005.004.005.00

HP

3.534.304.20

5.003.804.003.965.003.853.853.362.902.801.18

3.224.502.904.500.005.00

4.955.004.004.005.005.005.00

IBM

4.383.804.00

5.004.605.003.794.484.384.605.004.304.803.20

4.765.004.705.000.005.00

4.514.505.003.004.105.005.00

Mic

roso

ft

3.913.903.60

5.004.204.753.964.743.474.303.204.304.202.56

3.255.002.903.500.005.00

4.323.204.003.005.005.005.00

PTC

4.011.503.60

4.005.004.754.424.213.644.655.003.704.253.31

3.725.003.405.000.005.00

3.232.203.503.004.103.005.00

Rally

Sof

twar

e

3.953.204.60

5.003.405.003.974.754.084.554.284.153.801.26

4.394.104.405.000.005.00

4.263.103.501.005.005.005.00

Rock

et A

ldon

1.970.102.82

1.505.001.052.061.440.661.802.443.101.054.45

2.042.101.904.000.005.00

2.222.201.501.003.201.001.00

Sere

na S

oftw

are

3.623.102.80

3.752.205.004.143.383.384.154.263.253.153.61

3.443.103.405.000.005.00

3.732.902.502.004.105.003.00

All scores are based on a scale of 0 (weak) to 5 (strong).

aLM Vendors have yet To Fully address Many Common Challenges

While many aspects of the ALM tools market have improved substantially in the past two years since the Agile development management Forrester Wave evaluation, other areas remain ripe for innovation and improvement:




■ Testing. With one or two notable exceptions, the vendors we included in this study did not excel at any aspect of testing support. To some extent, the weakness of testing features is characteristic of the ALM tools market, not just the vendors we assessed. However, it’s still worth noting that some specialty vendors have done a better job of connecting testing to other ALM activity. Some requirements tools vendors, for example, provide rapid test generation from requirements, something that many purveyors of bigger ALM suites have yet to implement.

■ Collaboration. While we have seen some progress in this area, that’s damning by faint praise considering how often collaboration is at the core of an organization’s ALM challenges. Take, for example, the very common challenge of collaborating with an offshore partner. While some features, such as feeds or wikis, make it easier for onshore and offshore teams to eavesdrop on each other’s activities, these are not vehicles for communicating product vision, mentoring people on the other team, or suggesting ideas for process improvement.

■ Mobile support. At the risk of stating the obvious, many aspects of mobile development — user experience, testing, deployment, etc. — are significantly different from the elements involved in building middleware. Yet the typical ALM vendor today has done very little to address these needs, despite the continued expansion of the mobile market.

■ Integration. ALM vendors have acknowledged the inevitability of integration with other commercial and open source tools. However, we’re a long way from arriving at a common integration strategy that makes it easier for customers to select ALM tools based on their individual merits outside of integration. Some vendors provide an application programming interface (API) with few or no prebuilt connectors. Others provide plenty of connectors, with perhaps a weaker API for custom-built integrations. The fact that small companies, such as Kovair and Tasktop, have established themselves as third-party providers of ALM connectors is a testament to the weaknesses of the overall ALM market’s integration efforts.

VeNdoR pRoFiLes

Leaders

■ IBM’s offering is becoming more than the sum of its parts. IBM’s position in this year’s assessment of the ALM market is a testament to an important principle: It’s not enough just to have good products — the products, in combination, must support common activities in software development and delivery. In previous years, IBM has certainly provided strong solutions, such as Rational RequisitePro, Rational Team Concert, and Rational DOORS in specific functional areas such as requirements, planning, and project management. However, IBM provided multiple solutions in some of these areas, leading to confusion about what exactly each of these tools was supporting and how they all worked together.




That confusion has largely dissipated with IBM’s current ALM offering. Not only has IBM continued development of its strong suite of products, but it has also stitched them together in a more coherent way. The company has also made clearer the use cases it supports with its tools, such as Agile teams and embedded software development. Because of the company’s strong product and portfolio strategy, it is possible to write about IBM’s ALM offerings “taken as a whole” with far less irony.

■ Rally Software continues its leadership in Agile/Lean. Rally targets a very healthy and growing opportunity: the expanding number of organizations that have adopted Agile and Lean. Rally’s tools are optimized for Agile planning, project management, status reporting, and other actions that happen within and outside sprints. The company’s acquisition of AgileZen, a Lean project management tool, was a natural fit for both Rally and its customers. So too was the addition of Rally Portfolio Manager, a tool for planning, decision-making, and management above the level of an individual project or product.

Rally also continues to provide thought leadership in the Agile and Lean community. However, this focus on Agile has a downside: non-Agile teams will find Rally’s products and services far less attractive than other general-purpose ALM tools. Rally’s leadership rests with its breadth and depth of capabilities for Agile teams, combined with a strong and focused corporate strategy.

■ PTC continues its leadership addressing regulated and “digital products” needs. PTC provides strength where product-based development and compliance intersect. No other company, with the possible exception of IBM, has done as much to support teams of embedded software developers and systems engineers. PTC helps firms address “digital product” development needs — for products such as appliances, vehicles, and medical devices that have software components — while still maintaining support for teams building and delivering software-only products and projects.

PTC’s strength rests on its long-standing support for addressing governance and compliance challenges. Burrowing into this niche has its concomitant cost: a more complex product that is harder to justify adopting for teams that don’t have to satisfy the needs of auditors or government regulators.

■ CollabNet provides a flexible ALM framework optimized for the cloud. As already noted, one of the strengths of CollabNet’s ALM suite is the flexibility with which users can define ALM content. We also see this flexibility pay off when trying to effectively address the needs of complex teams or complex projects, where there is less chance for a one-size-fits-all design.

CollabNet has also invested heavily in supporting cloud development and deployment. Not only are CollabNet’s tools available in the cloud (CloudForge), but they also include features optimized for developing, testing, and deploying cloud systems.




■ Microsoft shortens cycle times. Starting with Visual Studio at the front end and through to Team Foundation Server (TFS) at the back end, Microsoft’s ALM portfolio provides broad life-cycle support. Microsoft’s overarching goal is to shorten cycle times, which requires a combination of individual measures (task management, test automation, etc.).

Microsoft deserves recognition for understanding how central collaboration is to ALM. While the “M word,” management, is part of the ALM acronym, ALM initiatives frequently fail because they emphasize control over collaboration. Features such as visualization via PowerPoint and translating recorded manual testing into automated tests emphasize collaboration — in these two examples, with business users.

■ Serena Software orchestrates IT projects from idea to deployment. Since retooling its ALM strategy, Serena has made impressive advances at both the individual tool and the suite level. Central to its product strategy is “orchestration,” a vision of ALM as an ongoing, rhythmic activity. Not surprisingly, given this vision of a regular flow of activity (conceive, build, test, deploy), Serena has one of the best workflow designers available. Serena’s recent enhancements to its ALM portfolio, such as the Serena Release Manager, provide more support in this end-to-end orchestration model.

strong performers

■ For HP, quality is truly central, but other ALM capabilities are growing. HP Quality Center is the fulcrum on which HP’s ALM strategy rests. HP Quality Center is a ubiquitous part of many organizations’ ALM infrastructure. Consequently, other ALM vendors integrate with HP Quality Center as a matter of course, almost as a price of entry into any potential customer’s shortlist.

HP has improved upon and expanded its ALM capabilities beyond test management and test automation in the two years since the previous Forrester Wave evaluation of this space. HP now provides improved requirements management and traceability. Another example, HP’s Application Lifecycle Intelligence (ALI), offers a broad set of ALM data aggregation and analysis features, with strong reporting and dashboarding components. Given the importance of this information for everything from team-level continuous improvement to executive-level portfolio management, it represents one of many areas where HP has moved quickly to distinguish itself as a serious ALM player.

■ Atlassian focuses on issue tracking and collaboration within teams. Atlassian’s JIRA, a widely popular issue tracking tool, often serves double duty as a requirements management system. The company’s Confluence Wiki is equally popular with development teams. The two products, in tandem, support the various kinds of day-to-day work done within software development teams, which are always looking for ways to better collaborate (for example, through a wiki).




While Atlassian has decided to provide a narrower set of ALM tools than other vendors, it continues to hone the capabilities in the areas where the company believes it can excel, such as providing a marketplace for its developer community to provide plug-ins. The company has made moves into more enterprise-level scenarios, with new capabilities such as Stash, a tool for the management of Git repositories that enterprise customers demand. Atlassian has also made enhancements to its support, sales force, and other aspects of its business needed to gain ground in the enterprise market.

Contenders

■ Rocket Aldon provides strong operational support. Unlike other ALM vendors, Rocket Aldon’s roots lie in software delivery, not software development. Rocket Aldon has focused from the beginning on the operational side, including both automation and reporting. Unlike other vendors, Rocket Aldon is expanding backward, not forward, in the software timeline, adding support for more upstream ALM activities, such as demand management and requirements.

sUppLeMeNTaL MaTeRiaL

online Resource

The online version of Figure 3 is an Excel-based vendor comparison tool that provides detailed product evaluations and customizable rankings.

data sources Used in This Forrester Wave

Forrester used a combination of three data sources to assess the strengths and weaknesses of each solution:

■ Hands-on lab evaluations. Vendors spent one day with a team of analysts who performed a hands-on evaluation of the product using a scenario-based testing methodology. We evaluated each product using the same scenario(s), creating a level playing field by evaluating every product on the same criteria.

■ Vendor surveys. Forrester surveyed vendors on their capabilities as they relate to the evaluation criteria. Once we analyzed the completed vendor surveys, we conducted vendor calls to clarify and confirm their products’ capabilities.

■ Customer reference calls. To validate product and vendor qualifications, Forrester also conducted reference calls with two of each vendor’s current customers.




The Forrester Wave Methodology

We conduct primary research to develop a list of vendors that meet our criteria to be evaluated in this market. From that initial pool of vendors, we then narrow our final list. We choose these vendors based on: 1) product fit; 2) customer success; and 3) Forrester client demand. We eliminate vendors that have limited customer references and products that don’t fit the scope of our evaluation.

After examining past research, user need assessments, and vendor and expert interviews, we develop the initial evaluation criteria. To evaluate the vendors and their products against our set of criteria, we gather details of product qualifications through a combination of lab evaluations, questionnaires, demos, and/or discussions with client references. We send evaluations to the vendors for their review, and we adjust the evaluations to provide the most accurate view of vendor offerings and strategies.

We set default weightings to reflect our analysis of the needs of large user companies — and/or other scenarios as outlined in the Forrester Wave document — and then score the vendors based on a clearly defined scale. These default weightings are intended only as a starting point, and we encourage readers to adapt the weightings to fit their individual needs through the Excel-based tool. The final scores generate the graphical depiction of the market based on current offering, strategy, and market presence. Forrester intends to update vendor evaluations regularly as product capabilities and vendor strategies evolve.

eNdNoTes1 Forrester’s research indicates that the dependence on software to achieve business outcomes will only

continue through the next decade. Therefore, while packaged applications will continue to be important, software development and delivery — for building new applications, customizing packaged systems, or performing custom integrations — will be critical for bridging the gap between software capabilities and business needs. See the January 30, 2012, “BT 2020: To Thrive In The Empowered Era, You’ll Need Software, Software Everywhere” report.

2 The previous Forrester Wave contained many similar questions but focused more narrowly on Agile team support. See the May 5, 2010, “The Forrester Wave™: Agile Development Management Tools, Q2 2010” report.




Forrester Research, Inc. (nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology. Forrester works with professionals in 17 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 29 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit www.forrester.com. 60080

«

Forrester Focuses OnApplication Development & Delivery Professionals

Responsible for leading development and delivery of applications

that support your company’s business strategies, you also choose

technology and architecture while managing people, skills, practices,

and organization to maximize value. Forrester’s subject-matter expertise

and deep understanding of your role will help you create forward-thinking

strategies; weigh opportunity against risk; justify decisions; and optimize

your individual, team, and corporate performance.

AndreA dAvies, client persona representing Market Insights Professionals

About ForresterA global research and advisory firm, Forrester inspires leaders,

informs better decisions, and helps the world’s top companies turn

the complexity of change into business advantage. Our research-

based insight and objective advice enable IT professionals to

lead more successfully within IT and extend their impact beyond

the traditional IT organization. Tailored to your individual role, our

resources allow you to focus on important business issues —

margin, speed, growth — first, technology second.

for more informAtion

To find out how Forrester Research can help you be successful every day, please contact the office nearest you, or visit us at www.forrester.com. For a complete list of worldwide locations, visit www.forrester.com/about.

Client support

For information on hard-copy or electronic reprints, please contact Client Support at +1 866.367.7378, +1 617.613.5730, or [email protected]. We offer quantity discounts and special pricing for academic and nonprofit institutions.

mailto:[email protected]

www.forrester.com

Technology

The Forrester Wave: Application Lifecycle Management Report