Articolo - Un esame sui migliori 7 della BI - inglese

The BI Survey 7 Summary

The BI Survey 7 Summary compiled by BOARD and authorized by Nigel Pendse

The BI Survey 7 - Summary 3

Content

Overview of The BI Survey 7 3

The Sample 4

Products included 5

The Purchase Cycle 7

The BI ownership experience 9

Vendor effectiveness 13

Timescales 16

What goes wrong? 17

Performance at the speed of thought? 19

Overall key findings of the BI Survey 7 20

Conclusion 23

About BOARD International

The BI Survey 7 follows on from six successful editions of The OLAP Survey. As the distinction between pure OLAP and other types

of business intelligence products has blurred, the time had come to widen the scope of the Survey to include other products that

would not normally be regarded as falling into the OLAP category.

The BI Survey provides a detailed quantitative analysis of why customers buy BI tools, what they use them for, how successful they

are and why they eventually abandon them. It is based on the analysis of actual experiences of some 1900 users worldwide. This

year’s Survey also includes analyses of how vendor, user and consultant perceptions vary. As such, it is the largest and most thorough

fact-based analysis of the growing BI market. The BI Survey 7 benefits from the experience of the six previous editions of The OLAP

Survey and the ability to analyze trends based on seven data sets.

After filtering, the Survey was conducted across about 60 countries, with the following geographic split: North America 37.2 percent,

Europe 52.9 percent, and rest of world 9.9 percent. The sample comes from a wide range of vertical markets. No single industry

dominates, but some are obviously more heavily represented than others. Banking, insurance and retail are again the top three

industries, which is no surprise: these industries have long understood the value of BI. Government came much higher than usual

this year, and automotive also had a relatively larger sample. The median revenue of companies surveyed was just over $500m.

Only sub-samples with at least 30 data points are included in analyses, so products with fewer than 30 respondents are generally

not analyzed individually. In descending order, 16 products or groups of products had enough usage to be analyzed individually

throughout this Survey: MicroStrategy, Microsoft AS,TM1, Cognos Analysis, SAP BW/Netweaver BI, Cognos Reporting, BusinessObjects,

Infor (MIS) Alea, BOARD, Panorama NovaView, Information Builders, WebFOCUS, Cubeware Cockpit, Essbase, Microsoft RS, Crystal

Reports and arcplan. Excel PivotTables and Pentaho/Mondrian were also included in some analyses.

BOARD is included in The BI Survey (formerly the OLAP Survey) for the third time now and again with highly impressive results.

88 respondents named BOARD as their primary product. BOARD was therefore included in all detailed product-related analyses.

The following summary shows the most interesting findings of The BI Survey 7, compiled by BOARD International.

Overview of The BI Survey 7

More information about The BI Survey 7

www.BI-Survey.com

The BI Survey 7 - Summary 5The BI Survey 7 - Summary 4

Vertical Markets

The sample comes from a wide range of vertical markets. No single industry dominates, but some are obviously more heavily

represented than others. Insurance, banking, retail and CPG (consumer packaged goods) are again the top four industries, which is

no surprise: these industries have long understood the value of BI.

The Sample

Product Architecture

The BI Survey 7 classifies products into MOLAP (multi-dimensional OLAP) and ROLAP (relational OLAP), Client, CPM and Relational.

This allows many analyses that compare the characteristics of the different architectures.

• MOLAP (multidimensional OLAP databases) products include proprietary multidimensional databases and the data structures

are usually called ‘cubes’ (though they usually have many more than three dimensions). These cubes are pre-defined and

loaded from external sources, such as relational databases, flat files or direct data entry by end-users.

• ROLAP products also provide a multidimensional view and analysis of the data, but this is achieved without first storing

it in a separate proprietary multidimensional database. A combination of complex SQL and an on-the-fly

multidimensional engine are used to generate these multidimensional views. ROLAPs often feature intermediate caches,

so their architecture overlaps MOLAPs.

• ‘Relational’, to cover pure reporting products

• ‘Client’, for more complex front-end tools, with significant application building and capabilities and/or a local cache with

a calculation engine. This category applied whether the products are deployed over the Web or in a client/server environment.

• ‘CPM’ (Corporate Performance Management) for pre-built financial applications used for planning, consolidations, etc.

Some analysts and vendors also call this category BPM or EPB.

In the Survey, BOARD is classified as a MOLAP tool.

“BOARD began as a MOLAP tool which is how it is historically classified. However, since the introduction of its hybrid architecture,

BOARD can be used with MOLAP and ROLAP, with both used in parallel to offer read-write access,” explains Giovanni Grossi, CEO

at BOARD International and adds: “BOARD’s ROLAP option can be used to write multi-dimensional planning data directly into relational

tables, which is a unique feature in the Business Intelligence market. With MOLAP and ROLAP in one single integrated platform

including pervasive Business Intelligence and Corporate Performance Management functions, BOARD is providing everything for

comprehensive reporting, analysis, planning and simulation applications.”

The following Table 2 lists all the ‘primary’ products used by the respondents, and the number answering questions on each.

Products included

TABLE 1 - Industry sector analysis by product (in percent)

Source: Table based on The BI Survey 7


Why organisations choose products

Overall, the primary reasons cited for selecting products were functionality (41 percent) and ease of use for end-users. Query

performance was only cited by 19 percent of buyers, price by 15 percent, corporate standards by 14 percent and data scalability by

11 percent. The most rarely cited reason was the range of server platforms supported. As Figure 1 shows, ease of use for application

builders mattered much more to arcplan, BOARD, Microsoft RS, TM1 and WebFOCUS customers than to Cognos Reporting, Crystal

Reports, Pentaho or SAP BI/BW customers when selecting a product.

Does product choice affect the business benefits?

The BI Surveys use the Business Benefits Index (BBI) to measure the success of BI projects. This is widely applied throughout this

Survey, and is a handy way of assessing, among other factors, which methods of product selection are typically associated with

successful projects.

Figure 2 shows the ranking of selection criteria, based on the BBI of the projects associated with each method. A color-coding for

product, vendor and cost-related criteria is applied, which makes the result very interesting: every product-related criterion had a

higher associated BBI than every vendor- or cost-related criterion.

The Purchase CycleTABLE 2- Products included in the sample


FIGURE 1 - Reason for choosing a product: Ease of use for application builders

Source: Figure by BOARD based on The BI Survey 7


Fore more information about the Business Benefit Index, please see Page 12.

Departments using BI

Another new question was included in The BI Survey 7: “Which business functions use the application(s)? (Select all that apply)”. The

results presented in Figure 4 show that, overall, just over four functions (ie, departments) used BI applications, with Finance well in

the lead, followed by Sales, Top management and Marketing. The Legal department was the least likely to use BI applications.

By product, BOARD was used by the widest range of departments, closely followed by SAP BI/BW and BusinessObjects. Alea was

used by the fewest, mainly Finance. Some products are clearly focused on particular departments: for example, SAP BI/BW, Cubeware

Cockpit, Essbase, TM1 and Alea are heavily used by Finance, with more than 90 percent of respondents reporting that the product

was used by their Finance function.

This section looks at several aspects of BI deployments, including how widely it is deployed, the on-going admin/maintenance

effort and the benefits seen. There are many ways to evaluate BI products technically, but measuring their customers’ success focuses

more on the real purpose for implementation. This allows products that are not technically comparable to be evaluated side by side

in a meaningful way.

Proportion of employees regularly using BI

This year, respondents where asked a new question: “What proportion of all your organization’s employees currently make regular

use of a BI application based on ‘product’?”

Figure 3 shows that BOARD has the most regular users, in average as well as median value.

The BI ownership experience

FIGURE 3 - Proportions of employees regularly using BI

FIGURE 2 - Benefits-driven ranking of selection criteria



Three product-related criteria are tightly clustered at the top, none of which made the top three of three of the actual criteria reported

by the Survey respondents. In fact, the criterion with the highest BBI, the range of server platforms supported, was actually the least

commonly cited criterion in actual projects. Fast performance was close behind, ahead of its fourth place in the reported list and ease

of use for application builders was just behind, whereas it only came sixth in the actual list.

Also striking is the low place occupied by product functionality. This was the most frequently cited reason for choosing products,

not only in this edition of the Survey, but in every one of the previous editions - and yet it ranked only ninth in the benefits table, the

lowest of any product-related reason.


“This reflects BOARD’s programming-free toolkit approach that enables a flexible use throughout the whole company and in the most

diverse functions. Based on one single development tool it is possible to build a huge variety of applications. That is a differentiation

to many standard tools where different modules cover specific functions,” explains Dr. Carsten Bange, managing director at the

BARC Institute.

FIGURE 4 - Departments using BI by product

Resources used to run and administer BI projects

The initial implementation cost is by no means the only people cost linked to BI projects, which also requires efforts to run and

administer. Organizations were therefore asked: “How many people (both internal and external) are involved in running and administering

the application? For example, if one person administers the server with 50% of his/her time, enter 0.5.”

The overall average was 3.328 people (compared to 2.719 last year), and the median 1.5 people (compared to 1.0 last year). Of the

products covered both last year and this, SAP BI/BW was again the product requiring the most people to run and administer resources,

with figures even higher than last year. Similar to the results of last year’s Survey, BOARD still belongs to the products that require

the least amount of administrative work with a median of just 0.5 people as Figure 5 shows.

“The effort to run and administer BI projects is an important success factor. With a median of 0.5 people compared to the average

of 1.5 of all Business Intelligence tools, BOARD shows a very good result,” says Nigel Pendse, author of The BI Survey 7.

Business benefits enjoyed

Achieving business benefits is, after all, the whole purpose of any BI deployment, so it makes sense to compare deployments based

on the extent to which this has been done. This is more important than specific technical achievements. The authors of the Survey

call this combined weighted score the BBI - Business Benefits Index.

The benefits listed in the questionnaire were:

• Saved headcount in IS

• Saved headcount in business departments

• Reduced external IT costs (e.g. hardware, external support and consulting, or software licensing)

• Saved other non-IT costs (e.g. inventory, waste, financing)

• Faster or more accurate reporting

• Increased revenues through better sales and marketing analysis

• Improved customer satisfaction through enhanced product quality and/or service levels

• Better business decisions through more thorough or timely analysis

• Other (specify)

The top four places all went to small vendors, with BOARD coming top. BOARD also scored well last year, but the sample was not


FIGURE 5 - Net people involved in running and administering projects



FIGURE 6 - Business Benefits enjoyed by product (Cumulated weighted scores)

FIGURE 7 - Selection rates in evaluations

Sales success: winners and losers

The reasons why one product is preferred to another are complex and varied. They include price (including bundling with other

products), functionality, performance, vendor relationships, ease of use and possibly the most important of all, vendor sales skills.

This section simply measures how likely organizations are to select a product after evaluating it.

Figure 7 shows that BOARD belongs to the top solutions selected by organizations when choosing a BI product. With an 82.3

percent win chance, BOARD is significantly above the average of 60.0 percent.

“BOARD is not the best known Business Intelligence solution but, when it is included in a software selection process, it comes out

with the highest win rate of 82.3 percent,” acknowledges Nigel Pendse the author of The BI Survey 7.

Vendor effectivenessquite large enough to include it in the table. Most of the big names in BI had BBIs scores below average this year: Cognos (both

Analysis and Reporting), BusinessObjects, Hyperion and SAP. Not only did SAP BI/BW have the lowest BBI of all the products in the

Survey, but the gap between it and the next lowest was wider than between any other pair of products.



Future buying intentions

Respondents have been asked whether they expected to purchase more seats in the future, and 38.4 percent of respondents

overall said they did, compared to only 21 percent who expected not to, while 40.6 percent didn’t know.

“Almost half of our customers plan to buy more BOARD licences in the future and hereby they confirm that the BOARD toolkit is indeed

a great product to work with. BOARD’s concept of one single integrated platform with MOLAP, ROLAP and all BI and CPM functions

integrated for pervasive reporting, analysis, planning and simulation applications is proved right when we see the feedback in this

years’ BI Survey. In times of such a strong market consolidation as it currently happens in the BI and CPM market, it is important for

us to care about customers needs and develop our product accordingly. Other vendors cannot focus on such things but need to

concentrate on integration issues due to recent acquisitions,” outlines Giovanni Grossi, CEO at BOARD International.


FIGURE 8 - Inclination to buy more seats Figure 9 does indeed support the hypothesis that small vendors provide much better product support than do large vendors, with

medium-sized vendors just behind their smaller competitors. It is particularly striking that, compared to large vendors, small vendors

had more than twice as many customers reporting excellent support, while more than twice as many customers of large as compared

to small vendors regarded their support as unacceptably bad. With all the takeovers of small vendors by large vendors that have

recently occurred, does this mean that overall support scores are doomed to fall?

Product support

Once a product is in use, the nature and quality of product support becomes important. All users were therefore asked which types

of product support they used and how satisfied they were. This section compares users’ ratings for the vendors.

BOARD provides one of the best levels of support to its customers, sharing this high position with only one other vendor.

“High ratings in product support not only reflect a superior support process but also the qualities themselves, such as the product

robustness and the speed of development incorporating customer requirements,” says the author of The BI Survey 7, Nigel Pendse.

Customer loyalty - Product abandonment

Customer loyalty is a crucial factor in vendor success, but it is difficult to assess in an objective way. One of the four factors used in

the Survey to measure unhappiness or disloyalty to a product is if its use has been, or is planned to be, discontinued.


FIGURE 9 - Support Quality Ratings (Weighted answers of the users)


FIGURE 10 - Discontinued usage analysis



Slow (and late) implementations are one of the biggest traditional problems with data warehouse and BI applications. The evidence

shows that it is definitely better to go live within three or at least no more than six months. With extended implementation periods,

problems mount up, and the chances of achieving benefits and meeting the project’s goals fall sharply.

Median implementation times summary

The factor that seems to influence deployment times the most is product choice. Figure 11 shows that only four products had median

deployment times below the optimum three months: Cubeware Cockpit, TM1, BOARD and Microsoft Reporting Services. Two others

had medians of exactly three months: Panorama NovaView and Crystal Reports. At the other extreme, the deployment times for SAP

BI/BW were far higher than for any other product, with a median time of almost nine months. WebFOCUS and MicroStrategy also took

relatively longer than others.

Other factors that seem to have a big influence on rollout times:

• Smaller companies roll out their applications faster than medium or large organizations. Apart from possibly having smaller,

simpler projects, the data also suggests that they are less likely to suffer from company politics and changing requirements.

• Business end-users implement their applications much faster than average. Large general-purpose consulting firms

are much slower, and the latter have far more incomplete projects after two and even three years than any other implementers.

• Applications with larger data volumes are somewhat slower to go live. This confirms that it is better to start small,

and expand the application after it goes live.

• MOLAP implementations are much faster than ROLAP, and Windows deployments are much faster than UNIX.

This may be partly a consequence of the larger data volumes typically handled by ROLAP and UNIX.

Problems encountered

BI deployments depend on a complex choreography of people, data and technology. If any of these fails to perform, the problems

soon mount up - and over 70 percent of the respondents identified at least one major problem that had occurred. As in previous

years, problems were placed into three categories:

Regarding reliability BOARD was rated number one out of all the analyzed BI tools, with the fewest problems reported overall. This

can be seen in Figure 12 which shows the percentages of the diverse problems reported.

The products with the fewest data quality problems were those used for planning applications:Alea, BOARD, Essbase and TM1. Perhaps

this is because most of their data is captured within the applications, rather than having to be loaded from transaction applications.

BOARD was ranked second best in the fewest product-related technological problems category. Product-related problems can be:

• Missing product features

• User scalability

• Data scalability

• Slow query performance

• Security weaknesses

• Too hard to use

• Unreliability (bugs, etc.)

In general, BOARD is the product where the most users said that they did not experience any serious problem with their BI solution at

all (see Figure 13).

Timescales What goes wrong?

FIGURE 11 - Median implementation times


TABLE 3 - Problem categories



Complaints about poor query performance

Poor query performance is by far the most frequently reported product-related problem in both The BI Survey and all previous edi-

tions of The OLAP Survey. Poor query performance is a problem for both end-users and technical administrators, and it is one issue

that cannot be covered up or ignored. Even if the administrators wish to forget about it, end-users will be reminded of the problem

with every query they issue, and usage (and therefore the value of the system) will probably drop as a result.

It is therefore worth investigating in more detail than other product issues, as it is apparently the product feature that has the grea-

test impact on the success of a project. Poor query performance not only reduces the productivity of end-users, but it also limits their

numbers, because performance will degrade further as more users access the system. It therefore sets a limit on user scalability,

and is not just a matter of user convenience.

A way to assess the acceptability of performance is to see how many people reported slow query times as a major problem. This is

an assessment of how well products are living up to users’ expectations, rather than a direct measure of performance. Slow perfor-

mance is the biggest single reported problem in any case, so it is worth assessing which products attract the most complaints.

Figure 14 shows that on this measure, BOARD was second, having been best in 2006.

Performance at the speed of thought?

Data latency by product and vendor

As with query times, the Survey analyzed the overall reported build times, as well as the build times by data volume bands. Given the

assumed trade-off between query and build times, one might have expected that products with consistently slow query times would

have fast load/calculate times, and vice versa. Evidently, this is only partially true, because some of the products that were fast with

queries were also fast to load and pre-calculate. For example, TM1, Alea and BOARD were the fastest to load and pre-calculate while

also being among the fastest to handle queries.

MOLAP sites had slightly fewer problems than ROLAP sites. The main difference was with query complaints, where ROLAPs were

significantly worse than MOLAPs.

FIGURE 12 - Technical, Data and People Problems


FIGURE 13 - No significant problems reported


FIGURE 14 - Performance complaints



The selection process:

• BOARD, Cubeware, Panorama and Microsoft AS had the highest win rates of about 80 percent.

• At the other extreme, QlikTech and Corporate Planner had competitive win rates below 25 percent. The now discontinued

BusinessObjects OLAP Intelligence, OutlookSoft (both now owned by SAP) and Oracle BI EE had competitive win rates

of below 35 percent.

• SAP BI/BW buyers had largely corporate rather than product reasons for selecting it: integration with other SAP products

(54 percent vs 21 percent overall), corporate standards (55 percent vs 14 percent), vendor relationship (14 percent

vs 10 percent) and bundling (32 percent vs 13 percent) were the main reasons cited for buying. Conversely, very few

SAP sites chose it for product-related reasons such as functionality (21 percent vs 41 percent overall), ease of use

(five percent vs 36 percent overall) and performance (just one percent of BI/BW buyers vs 19 percent overall).

This was probably just as well, because SAP sites had significantly more complaints about missing functionality,

difficulty of use and query performance than the norm.

Realizing business benefits:

• A weighted Business Benefit Index (BBI) is calculated and used to compare as many factors as possible throughout

the Survey. This can be used by buyers to maximize the business benefits from their own projects.

• BOARD, Panorama, Cubeware, arcplan and Microsoft AS sites were most likely to report that they had realized benefits

(ie, they had the highest BBI).

• SAP BI/BW, Infor Alea and Crystal Reports sites were the least likely to achieve business benefits. SAP BW sites

have consistently reported the lowest business benefits in each of the six years that a BBI was calculated in the Survey.

• Projects led by specialist BI consulting firms were the most likely to realize business benefits, closely followed

by projects led by business end-users, while those led by large general-purpose consulting firms and vendor consultants

were the least likely. This finding is consistent with previous years.

• Another consistent trend is that business benefits were most likely to be achieved if projects went live within three months.

There was a steady fall-off if initial rollouts took longer than this. The small number of projects that took more

than two years to roll out achieved few business benefits.

The power of the brand:

• As in previous years, SAP BI/BW sites had the lowest likelihood of realizing business benefits. They also spent much more

than average on both license and implementation fees, took by far the longest to go into production, had slightly worse

than average shelfware rates, reported worse than average levels of product support and encountered more technical problems

than almost any other product. Its users complained that the product was slow, lacked functionality, was hard to use

and was less reliable than average. All of these were also true in previous years. Despite these very poor results, BW projects

again had much lower than average rates of product abandonment of all products in the Survey (though no longer the lowest).

Overall key findings of The BI Survey 7 • BOARD received the second lowest rate of product abandonment.

• The best-known BI vendors did not provide the best product support. Business Objects and Hyperion were well

below average, and Cognos, Oracle and SAP were also below average. The best support was reported by customers

of small regional vendors like Cubeware and BOARD, with Information Builders, Applix and MicroStrategy also doing well.

Microsoft was just above average.

Purchases:

• Shelfware is inevitable in young sites, as organizations will have purchased seats that they have not yet had time to deploy.

Many of these may be deployed imminently. Very low shelfware rates are not good, as it suggests that the vendor

has made few recent sales. Very high shelfware rates could be a sign that the product does not live up to expectations.

• Of the major products, Cognos and WebFOCUS had shelfware rates of over 40 percent and Panorama NovaView

and SAP BI/BW above 30 percent. In contrast, Cubeware and Alea were below ten percent, and TM1, BOARD and Essbase

were below 15 percent.

Implementation and rollout:

• There was some correlation between project success and consulting spend. The Business Benefits Index (BBI) peaked

for projects with an external consulting spend of $50k to $100k. It declined sharply in projects with fees of over $1m.

• Projects led by business end-users had more business benefits than those led by in-house IT specialists.

Deployment issues and problems:

• Excel PivotTables users had the most product complaints, including data scalability and lack of functionality.

SAP BI/BW was almost as bad, with slow query performance and difficulty of use being the biggest problem areas.

BusinessObjects and Crystal Reports (both now owned by SAP) were only slightly better.

• Cubeware and BOARD sites reported the fewest technical problems.

• ROLAP sites had significantly more technical problems than MOLAP sites, with the most common being query performance.

• Cubeware and BOARD sites were the most satisfied with the quality of product support they received, while BusinessObjects

and Essbase sites were the least satisfied. BusinessObjects support satisfaction, in particular, was very low, and has

declined sharply in both 2006 and 2007. Essbase has been on the same trend, but the satisfaction levels are not yet as low.

• Overall, smaller vendors provided the best support and larger vendors the worst.

• Software cost was the biggest deterrent to wider deployments, with cost of implementation and support in third place.

Slow query performance was the second highest.


The BI Survey 7, the leading independent Survey of real-world BI implementations, provides unique, statistically significant insight

into actual BI implementations and customer experiences with various BI products. It’s an in-depth analysis of essential and unique

information from over a thousand organizations, covering their experiences with both buying and using BI tools, including the problems

they encounter, query performance, product related problems, product support ratings, implementation times, etc. Since the choice

of BI product has a significant impact on overall BI project success as the results of the Survey have shown, it is advisable to accomplish

product evaluations, starting with a close review of the product benchmarks in The BI Survey 7, when embarking on new BI

projects.

Notable results in The BI Survey 7 are the widely varying customer experiences and product results among the BI products. BOARD

was consistently ranked among the top products that have been analyzed in the Survey in the following categories:

• Business Benefits

• Product Support Quality

• Implementation and administration efforts

• Problems mixes and product complaints

• Performance complaints

“BOARD’s consistent high ranking positions in many of the key areas we analysed in the Survey is a remarkable achievement and

isn’t just limited to technical aspects, such as the performance, reliability and support. BOARD also performed very well with regard

to the key business criteria, mainly the business benefit index (BBI), but also rates of conversion from evaluation and buying intentions,

which emphasizes again the solid strength of the product in a wide range of areas.” concludes Nigel Pendse, author of The BI

Survey 7.

ConclusionPerformance issues:

• Not only was query performance the most severe product-related problem, as in all previous Surveys, but it was again

the single most common problem of all. Furthermore, sites that used query performance as a selection criterion

were more successful in business terms than those that did not.

• There was an almost linear, and significant, decline in the business benefits and goal achievements

as query times lengthened.

• TM1 and BOARD users had the fewest performance complaints, and SAP BI/BW had the most.

• The MOLAP median was 4.8 seconds, and the ROLAP median 26.1 seconds.


To learn more about BOARD MIT Management Intelligence Toolkit, please visit

www.board.com

About BOARD International

Founded in 1994, BOARD International is a global leader in the BI and

CPM (Corporate Performance Management) Toolkit space. BOARD

has enabled over 1800 companies worldwide to rapidly deploy BI &

CPM applications in a single integrated environment completely

programming free and in a fraction of the time and cost associated

with traditional solutions.

BOARD provides one accurate, corporate view of your information, fully

integrated with your CPM processes (budgeting, forecasting, planning,

reporting, KPI, score-carding, etc...) and therefore uniquely linking

performance to strategic vision at all levels down to operational detail.

Documents

Articolo - Un esame sui migliori 7 della BI - inglese