Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Free Preview
lk
THE
BI SURVEY 11 SURVEY
The Customer Verdict The world’s largest survey of business intelligence software users
1211
A preview of The BI Survey 12: The Results For more information, visit: www.BI-Survey.com
This document is not to be shared, distributed or reproduced in any way without first purchasing licensing rights and the express prior permission of BARC.
The BI Survey 12 – The Results
2
Table of Contents
Introduction ............................................................................................................................ 4
Business Benefits and the BBI ............................................................................................... 7
BBI by peer group and vendor ........................................................................................... 9
Goal Achievement ................................................................................................................ 17
Deployment .......................................................................................................................... 22
Usage in different company size classes ......................................................................... 22
Number of BI users .......................................................................................................... 24
BI usage by departments ................................................................................................. 25
Product usage for application scenarios .......................................................................... 29
The selection process .......................................................................................................... 33
Evaluation methods ......................................................................................................... 33
Reasons to buy ................................................................................................................ 37
How products perform in the selection process ................................................................ 42
Buying BI software ............................................................................................................... 45
Licenses .......................................................................................................................... 45
Number of users .............................................................................................................. 47
Cost ................................................................................................................................. 48
Implementing BI and Satisfaction ......................................................................................... 54
Implementation time ........................................................................................................ 54
Project scope ................................................................................................................... 58
Support ............................................................................................................................ 61
Recommendation ............................................................................................................. 62
Chosen as standard ......................................................................................................... 65
Problems in BI projects .................................................................................................... 67
Query Performance and Data Volume .................................................................................. 69
Query Performance results and user satisfaction ............................................................. 69
Data volumes handled per product .................................................................................. 73
The BI Survey 12 – The Results
3
Front ends for multidimensional databases .......................................................................... 75
Infor ION BI OLAP Server ................................................................................................ 76
Microsoft SSAS ............................................................................................................... 76
Oracle Essbase ............................................................................................................... 78
IBM Cognos TM1 ............................................................................................................. 79
SAP BW .......................................................................................................................... 80
Trending topics in BI ............................................................................................................. 82
Three-step approach to analyzing trends ......................................................................... 82
The frequent inaccuracy of plans ..................................................................................... 83
User perception of IT and software trends ....................................................................... 84
Status and plans for trending topics in BI ......................................................................... 87
Visual Analysis & Data Discovery .................................................................................... 88
Collaboration ................................................................................................................... 90
Mobile BI ......................................................................................................................... 91
Cloud BI/BIaaS ................................................................................................................ 94
The BI Survey 12 – The Results
4
Introduction
This document provides an overview of the key product-related results from The BI
Survey 12. The Survey provides a detailed quantitative analysis of why customers
buy BI tools, what they are used for, how successful they are and why they
eventually abandon them. This edition has increased the range of products
reviewed, including not only products from well-known BI giants, but also specialist
tools from much smaller vendors and open source vendors.
The BI Survey 12 follows on from ten successful editions of The BI Survey (formerly
The OLAP Survey). It is important to note that this year the numbering of The BI
Survey has changed so that The Survey number now aligns to the year in which the
data was collected.
Based on analysis of real-world experiences of nearly 3,000 respondents, the value
of The Survey is dependent on a sufficiently large, well-distributed and unbiased
sample. The Survey is the largest and most thorough fact-based analysis of the BI
market currently available, using eleven years of experience to analyze market
trends and debunk many myths surrounding the BI industry.
After data cleansing and the removal of responses from participants unable to
answer specific questions on BI tools, we were left with a sample of 1872 end users,
283 consultants and 512 vendor employees. Participants from all over the world took
part in The BI Survey 12. Around 20 percent of respondents had an IT job function,
15 percent were Heads of BI, and 65 percent had various lines of business job titles.
The BI Survey is not based on anecdotal accounts or personal opinions, unlike much
analyst research. The Survey is not intended to be a measure of market shares and
unlike our research publication, ‘The BI Verdict’, it does not include product reviews
or qualitative product comparisons. The BI Survey does not attempt to forecast
future trends, instead providing evidence undermining the reliability of many such
forecasts.
In the past the vast quantity of data collected by The Survey has been presented in
one large document. However, The BI Survey 12 heralds the introduction of several
bitesize documents, each focusing on a specific area of The Survey.
The BI Survey 12 – The Results
5
Detailed information on the sample and Survey methodology can be found in the
‘Sample, Products and Methodology’ document. For advice on selecting, managing
and implementing products into your organization see the document ‘The BI Survey
12: Best Practices’.
Document Description
The BI Survey 12 - The Results Provides an overview and analysis of the most important results of The BI Survey 12
The BI Survey 12 – Trending Topics series
Provides in-depth analysis of the following trending topics: SaaS/Cloud BI, Mobile BI, Big Data, Social BI, Collaboration, Self Service, Analytical Databases
The BI Survey 12 – Vendor Performance Summaries
A series of short executive reports highlighting all the product-related results for each vendor.
The BI Survey 12 – Best Practices
This report features recommendations on how best to choose a BI product and the most important and relevant tool selection criteria.
The BI Survey 12 - Sample, Products, Methodology (this document)
Provides details of the sample and an overview of our methodology including the entire questionnaire and details of our calculation methods.
The BI Survey 12 - KPIs and Dashboards
This document outlines the definitions and calculation methods of the KPIs used in The BI Survey 12.
Figure 1: Overview of The BI Survey 12
The BI Survey Analyzer is an online tool allowing users to carry out their own
analyses of The Survey data. Using results from this year’s Survey, the tool also
offers users the ability to drill down using product and demographic filters.
2012 also sees the exciting introduction of a mobile version of The BI Survey
Analyzer for the iPad. The mobile BI Survey Analyzer focuses on product-related
results based on criteria (KPIs) such as business benefits achieved, implementation
costs, functional usage, product competitiveness, innovation, performance, customer
satisfaction and agility.
The BI Survey 12 – The Results
6
Figure 2: Screenshot of The BI Survey Analyzer
Figure 3: Screenshot of The BI Survey Analyzer iPad App
The BI Survey 12 – The Results
7
Business Benefits and the BBI
The BI Survey asks key questions regarding the benefits of BI projects. Respondents
are asked to indicate the level of achievement gained from a list of eleven potential
benefits. A scoring system is then used to derive a composite weighted score for
each benefit, based on the level of benefit achieved. This system is called the BBI
(Business Benefits Index). For further information on the calculation methods used,
see the ‘Sample, Products and Methodology’ document.
Figure 4 shows the overall breakdown of responses to the Business Benefits
questions. The six levels of achievement, along with their weightings, are shown on
the horizontal axis. Results are RAG (red, amber, green) rated with green indicating
higher scores and red denoting lower scores.
Figure 4: Frequency of Business Benefi ts (n=2116)
Pro
ven
and
quan
tifie
d (4
961)
Pro
ven,
but
not
m
easu
red
(564
3)
For
mal
ly
clai
med
, bu
t not
ve
rifie
d (2
186)
Inf
orm
ally
su
spec
ted
(274
1)
Not
ach
ieve
d (2
495)
Got
wor
se/m
ore
expe
nsiv
e (2
26)
Do
not k
now
(5
024)
Weighting 10 8 5 3 -2 -6 0
Better business decisions 27.9% 39.5% 12.6% 10.7% 1.8% 0.2% 7.4%
Faster reporting, analysis or planning 45.2% 35.5% 7.4% 5.0% 2.0% 0.3% 4.6%
Improved customer satisfaction 17.7% 29.0% 13.8% 13.8% 5.3% 0.3% 20.1%
Improved data quality 29.6% 32.2% 10.6% 10.5% 6.2% 0.5% 10.3%
Improved employee satisfaction 22.8% 33.9% 13.5% 13.7% 5.3% 0.7% 10.1%
Increased revenues 10.3% 13.3% 11.0% 19.2% 9.0% 0.2% 36.9%
More accurate reporting, analysis or planning 33.6% 38.9% 8.9% 8.7% 1.9% 0.4% 7.6%
Reduced external IT costs 16.6% 14.2% 7.5% 11.8% 14.7% 3.4% 31.7%
Saved business headcount 11.4% 9.5% 5.9% 13.2% 25.7% 0.9% 33.4%
Saved IT headcount 11.1% 9.2% 4.9% 9.0% 29.3% 2.9% 33.6%
Saved other non-IT costs 8.3% 11.4% 7.2% 13.9% 16.7% 0.8% 41.7%
The BI Survey 12 – The Results
8
Figure 5 shows business benefits listed in order of the most commonly achieved.
Figure 5: Benefits overview (n= 2116)
Figure 5 Key Findings:
� Overall, ‘Faster Business Intelligence’ rated higher than accuracy and more
than 80 percent of respondents stated that faster reporting, analysis and
planning had been achieved. 72 percent reported that ‘More accurate
reporting, analysis and planning’ had been proven and 67 percent of
respondents, up from 59 percent last year, said that ‘Better business
decisions’ was another benefit likely to be achieved. The ability to make
better business decisions is a highly desirable benefit. However, it is a benefit
that cannot be accurately scoped when developing the business case for a
project. While all BI projects would hope to gain this benefit, few projects
would be cost-justified against the possibility that it ‘might’ one day be
achieved.
� ‘Improved data quality’ is a benefit that can be measured directly. Although
often one of the most difficult factors to achieve in a business intelligence
project, it ranked fourth in the list with ‘Employee satisfaction’ following in fifth
place, a benefit that typically arises when users have fast access to data and
reports.
� Benefits with the lowest level of achievement were ‘Saving headcount in IT
departments’, ‘Saving headcount in business departments’, ‘Saving non-IT
costs’ and ‘Reducing external IT costs’. This is in line with our experience that
companies are still looking to extend their BI capabilities, and are adding
resources to implement and run them. If BI were a mature and saturated
market we would see more investment in replacing existing systems but until
1.60
2.02
2.14
3.03
3.03
5.06
5.93
6.23
6.85
7.12
7.82
0 2 4 6 8 10
Saved IT headcount
Saved business headcount
Saved other non-IT costs
Reduced external IT costs
Increased revenues
Improved customer satisfaction
Improved employee satisfaction
Improved data quality
Better business decisions
More accurate Business Intelligence
Faster Business Intelligence
The BI Survey 12 – The Results
9
this pattern changes we would not expect savings in headcount and costs to
move up the list of benefits.
In contrast to previous results, this year the overall BBI decreased slightly from 4.89
to 4.62. However, we do not think this the beginning of a long-term trend of declining
benefits from BI projects. Instead we suspect that this small variation can be
accounted for by the fact that The Survey sample varies from year to year.
BBI by peer group and vendor
The difference between business benefit achievement according to the type of
product in use is reflected in the following chart. In order to compare product types
we group products into specific peer groups as explained in the ‘Sample, Products
and Methodologies’ document.
Figure 6: Business benefits analyzed by peer group (n=changing bases)
Visual Analysis & Data Discovery tools achieved the best BBI results reflecting their
user friendliness and flexibility in allowing end-users to quickly load data, perform
user analysis and take advantage of advanced visualization features. The user-
friendly nature of these tools relates to business benefits achieved. Small and
Medium Project vendors rank second, Dashboard vendors rank third and
Performance Management products rank fourth. This is a result we have consistently
observed in previous editions of The BI Survey. Specialists, or smaller vendors, tend
to show better BBI results than the larger BI Giants, Large Project vendors and
Enterprise Reporting vendors. There is a straight correlation between the size of the
BI project (measured in data volume used or number of users served) and the size of
the tool vendor. With few exceptions, smaller vendors serve smaller projects and
products from larger vendors are used in larger projects.
4.10
4.134.24
4.41
4.504.61
4.77
4.805.06
0 2 4 6
Enterprise ReportingLarge Project Vendor
BI GiantsIT Giants
OLAP AnalysisPerformance Management
DashboardSmall & Medium Project Vendor
Visual Analysis & Data Discovery
The BI Survey 12 – The Results
10
In our opinion, there are several reasons for the good results achieved by smaller
and specialist vendors:
� Some customers may achieve more business benefits when working with
smaller vendors with whom they have a close working relationship, compared
to the more arm’s-length, impersonal relationship that users tend to have with
large vendors. Survey results show that users usually experience better
vendor support, satisfaction and recommendation with smaller vendors than
with many of the large vendors.
� The prospect of smaller vendors being removed from a company more
quickly if projects are not successful leads to better than average results for
the ones that remain in use. Conversely, large vendors are able to keep
customers with whom they have long-term, strategic relationships, even if
individual projects or products deliver disappointing results.
� Products from small vendors are more likely to be selected in competitive
evaluations. Users expending more effort selecting solutions show a higher
level of benefit achievement. This could be because the extra diligence taken
to ensure that the product fits their needs increases the probability of the
project delivering business benefits. There is also the simple fact that some
projects are managed better than others.
But, for whatever reason, it is clear that the customers of small vendors such as
Yellowfin, Dimensional Insight and Phocas are reporting more business benefits than
those of larger vendors like SAP, Oracle, Microsoft and IBM. That said, several
products from large vendors have above average BBI scores, most notably the
multidimensional databases Oracle Essbase, IBM Cognos TM1 and Microsoft SSAS.
The BI Survey 12 – The Results
11
Figure 7 shows the business benefits scores for all products in the BI Giants peer
group.
Figure 7: Business benefits for the BI Giants peer group (n=949)
The BI Giants peer group includes companies with annual revenues of more than
$200 million and a truly international reach. These tools are usually used in
enterprise scenarios.
Figure 7 Key Findings:
� The multidimensional database Oracle Essbase ranked first in the BI Giants
peer group. All five top ranking products offer in-memory database
technology, while the traditional ROLAP tools from SAP, Oracle and IBM
deliver the least business benefits.
� Three of the four multidimensional databases in this peer group (all except
Infor) appear in the top five, including QlikTech (with its own in-memory
database technology) and MicroStrategy, having recently expanded its
caching technology.
� The correlation between user self service and fast query performance with
business benefits has been evident in previous BI Surveys.
3.333.45
3.553.76
4.114.23
4.284.50
4.564.64
4.654.904.92
0 2 4 6
SAP BO WebIOracle OBIEE
SAP BWIBM Cognos BI
InforMicrosoft Excel
Microsoft SSRSInformation Builders
MicroStrategyMicrosoft SSAS
QlikTechIBM Cognos TM1
Oracle Essbase
The BI Survey 12 – The Results
12
Figure 8: Business benefits for the Enterprise Repo rting peer group (n=555)
The Enterprise Reporting peer group includes products that can provide standard
formatted reporting in a large scale enterprise situation. Arcplan, a very flexible tool
for designing reporting applications, ranks highest in this peer group, followed by the
specialist vendors MicroStrategy and Information Builders. Users of the large
vendors SAP, Oracle and IBM report lower business benefits.
Figure 9: Business benefits for the Dashboard vendo r peer group (n=634)
The Dashboard peer group includes products that are focused on creating advanced
dashboards.
Users of tools from small vendors report the highest business benefit scores in the
Dashboard peer group. This group is led by Yellowfin, a tool with a modern
3.33
3.45
3.55
3.76
4.28
4.50
4.56
5.24
0 2 4 6
SAP BO WebI
Oracle OBIEE
SAP BW
IBM Cognos BI
Microsoft SSRS
Information Builders
MicroStrategy
Arcplan
3.45
3.58
4.50
4.56
4.65
5.19
5.24
5.52
6.31
0 2 4 6 8
Oracle OBIEE
Decisyon
Information Builders
MicroStrategy
QlikTech
BOARD
Arcplan
Dimensional Insight
Yellowfin
The BI Survey 12 – The Results
13
architecture and look and feel, with most of its customers being small companies.
Dimensional Insight, another specialist, comes in second, followed by two more
small vendors: Arcplan and Board.
Figure 10: Business benefits for the OLAP Analysis peer group (n=939)
The OLAP Analysis peer group includes products that support analysis in
dimensional and hierarchical data models. This peer group is crowded and highly
competitive with many vendors scoring good business benefits results. The peer
group is led by Board, followed by three products with very similar results: Targit,
Oracle Essbase and IBM Cognos TM1. As in all previous editions of The BI Survey,
SAP BW users report considerably lower business benefit achievement than any
other product’s users.
3.55
4.11
4.11
4.23
4.35
4.49
4.52
4.56
4.64
4.73
4.90
4.92
4.94
5.19
0 2 4 6
SAP BW
Infor
Pentaho
Microsoft Excel
Evidanza
Cubeware
Jedox
MicroStrategy
Microsoft SSAS
Bissantz
IBM Cognos TM1
Oracle Essbase
TARGIT
BOARD
The BI Survey 12 – The Results
14
Figure 11: Business benefits for the Visual Analysi s & Data Discovery peer group (n=532)
The Visual Analysis & Data Discovery peer group includes products that provide
advanced visualization features. Dimensional Insight rates top in this small peer
group of specialist vendors, followed by Phocas and Tableau. QlikTech, one of the
most successful BI vendors over the last five years and now considered to be a BI
giant, is starting to show similar results to the large vendors it used to comfortably
beat in this area.
Figure 12: Business benefits for the Performance Ma nagement peer group (n=420)
Figure 12 Key Findings:
� The Performance Management peer group includes companies whose
products are predominantly used in planning projects. This peer group shows
above average business benefits results and is led by BOARD followed by
Oracle Essbase and IBM Cognos TM1.
� Jedox and Cubeware hold fourth and fifth places, also scoring excellent
results.
4.65
4.97
5.24
5.30
5.52
0 2 4 6
QlikTech
Cyberscience
Tableau
Phocas
Dimensional Insight
4.11
4.35
4.49
4.52
4.90
4.92
5.19
0 2 4 6
Infor
Evidanza
Cubeware
Jedox
IBM Cognos TM1
Oracle Essbase
BOARD
The BI Survey 12 – The Results
15
Figure 13: Business benefits for the Small and Medi um Project vendor peer group (n=1308)
Figure 13 Key Findings:
� The SME Project vendor peer group includes products that are primarily used
for small and medium sized projects (<500 users). The sample of Yellowfin’s
customers in The BI Survey 12 reveals its clear focus on small companies
with less than 100 employees. Yellowfin appears to serve its customers well,
scoring exceptionally high business benefits, head and shoulders above the
competition.
� Dimensional Insight and Phocas follow in second and third places
respectively.
3.58
4.11
4.11
4.28
4.35
4.49
4.52
4.64
4.65
4.73
4.90
4.94
4.97
5.19
5.24
5.30
5.52
6.31
0 2 4 6 8
Decisyon
Infor
Pentaho
Microsoft SSRS
Evidanza
Cubeware
Jedox
Microsoft SSAS
QlikTech
Bissantz
IBM Cognos TM1
TARGIT
Cyberscience
BOARD
Tableau
Phocas
Dimensional Insight
Yellowfin
The BI Survey 12 – The Results
16
Figure 14: Business benefits for the Large Enterpri se vendor peer group (n=535)
The Large Enterprise Project peer group includes products that are primarily used for
large enterprise projects (>500 users). In the Large Enterprise projects peer group, a
leading group of four products scored excellent business benefits: Arcplan, Oracle
Essbase, MicroStrategy and Information Builders.
3.33
3.45
3.55
3.76
4.50
4.56
4.92
5.24
0 2 4 6
SAP BO WebI
Oracle OBIEE
SAP BW
IBM Cognos BI
Information Builders
MicroStrategy
Oracle Essbase
Arcplan
Next Steps
The BI Survey Analyzer Demo
The BI Survey Analyzer is an interactive online tool which enables you to perform your own custom analysis of the survey data.
Click here to register for an online demo and find out how effective The BI Survey Analyzer can be in helping you to understand the strengths and weaknesses of BI products and trends in the BI market.
Purchase
To purchase The BI Survey 12, visit our website or contact us at:
Email: [email protected]
Tel: +44 (0)20 8133 8987