Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved Putting It All Together: Trends in Business Intelligence Claudia Imhoff, PhD Intelligent

Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved

Putting It All Together:Putting It All Together:Trends in Business IntelligenceTrends in Business Intelligence

Claudia Imhoff, PhDIntelligent Solutions, [email protected]

www.intelsols.comBlog: http://www.b-eye-network.com/blogs/imhoff/


2

Claudia Imhoff

President and FounderIntelligent Solutions, Inc.

A thought leader, visionary, and practitioner in therapidly growing fields of business intelligence andcustomer focused-strategy – Claudia Imhoff, Ph.D., isan internationally recognized expert on analytical CRM,business intelligence, and the infrastructure to supportthese initiatives – the Corporate Information Factory(CIF). Dr. Imhoff has co-authored five highly-regardedand popular books on these subjects and writesmonthly columns (totaling more than 100) for technicaland business magazines.

Email: [email protected]: 303-444-6650


Putting It All Together

Going Beyond Traditional BI – Operational BI Takes the Stage

Data Warehouse Appliances and Analytic Databases – Making Life Simpler

BI Software as a Service – Feeling SaaS-y? Open Source BI – Free Software Anyone?

3


4

The Three Levels of Business Intelligence

Strategic BItimeframe ~ months

Tactical BItimeframe ~ days or weeks

Operational BI timeframe is intra-day


The Three Levels of Business Intelligence

Strategic BI Tactical BI Operational BI

Business focus

Achieve long-term business goals

Manage tactical initiatives to

achieve strategic goals

Monitor & optimize operational business

processes

Primary users

Executives & business analysts

Business analysts,& LOB managers

LOB managers, operational users &

operational processes

Time-frame

Monthsto years

Days to weeks to months

Intra-dayto daily

DataHistorical

dataHistorical

dataReal-time, low-latency,

& historical data

Mode ofoperation

User driven Data centric

User drivenData centric

Event drivenProcess centric

Paradigm Shift

5


6

What is Operational BI*

A set of services, applications and technologies

for monitoring, reporting on, analyzing and

managing the business performance of an

organization’s daily business operations

*From research study. “Embedded BI”, written by Colin White and Judy Davis, www.B-EYE-Research.com


7

Operational BI – Answers to Day-to-Day Business Questions

Information on demand Real-time + historical data Access to SAP, Siebel, Oracle and

BI results

New Data Needs

picked

packed shipped invoiced

picked

packed shipped invoiced

What is my customer’s order status? What can I offer based on customer’s life-time value?

Can I afford to make this move at current margin rates?

What is my current inventory level world wide? Is it sufficient to meet demands?

What is my production yield right now? Am I at par with acceptable standards?Yield

Helps front-line workers

make immediate business decisions

to squeeze out inefficiencies.

Operational BI


8

Real Time Decision-Making*

Operational BI optimizes time latency between when a business event occurs and when an appropriate action is taken The goal – to “right-size” the decision-making cycle Compressing time lag between knowing what is

happening and taking action based on that knowledge Real-time must consider potential trade-off between time-

to-action and business value of actions

* From “Right-Time Business Intelligence: Optimizing the Business Decision Cycle” By Judy Davis, www. B-EYE-Network.com Research paper


Impact on BI Environment

History of BI Extract usable information from operational systems Users, technologies, processes, procedures – all

independent of operations

Now what? Impact on BI environment is significant

Increase in number of users, volume of data, and faster performance

Operational BI – MUST be integrated into the operational environment Requires understanding of operational systems, processes,

procedures, workflows, personnel

9


10


Numbers of users increase significantly Traditional BI rarely supported a few hundred, maybe a

thousand or so users Opening BI up to operational personnel means ramping

up into tens of thousands of users These users have very different interface requirements Means BI implementers must rethink how BI is delivered to

business users

Means tighter and faster connectivity of enterprise decision support environment to rest of the company.


11


Volumes of data increase substantially Detailed intraday snapshots of data are loaded or trickle-

fed into data warehouses Tens of terabytes to hundreds of terabytes are not

unusual storage requirements for operational BI Scalability now a mandatory requirement in any BI

technology Whether in processing and integration of data, storage of massive

volumes, or retrieval of query responses


12


Faster performance Query performance must mimic or emulate response

times in operational systems Sub-second to just a few seconds to return data from a query.

Ability to prioritize queries not only according to their importance but also their response requirements is mandatory success criterion This last feature has stumped many BI implementers and BI

vendors

Must have ability to handle mixed work load gracefully and simultaneously


Getting Started – Assess Reality

First step – perform honest assessment of existing data delivery capabilities – available technologies, maturity of the BI architecture, existing personnel, etc. Combine these with solid understanding of business

requirements for operational BI data

Important to understand which weaknesses discovered in assessment will be exaggerated as you speed up the enterprise

13


Operational BI Requirements

Continuous availability of operational data and BI results Current information from operational systems Integrated with BI data on demand Minimal impact on operational systems performance

Presented in a proactive manner Make decisions – act on information presented Easy to understand and use

Dynamic modeling Ability to change business rules on the fly Show different set of metrics depending on situation

14


15

Picking a Project

Look for workflow activities that have significant impact on costs or revenues Bottlenecks today that can be made more efficient through

use of operational BI

Don’t make big changes to operational processes Just speed up or make more efficient processes you already

have in place You will have to retrain personnel and retool SOPs

Project managers may not realize operational BI application has ramifications beyond project’s immediate boundaries






16


Data Warehouse Appliances

BI and data warehousing technologies continue to evolve and innovate Produce more efficient & cost effective ways to deliver BI Latest innovations are DW and BI appliances

Definition of an appliance* One purpose One package One installation One vendor

17

* From the B–EYE-Research.com paper titled “Data Warehouse Appliances: Evolution or Revolution?” by Colin White, and Richard Hackathorn



All-in-one box that provides a hardware server preconfigured with all software components Designed for a specific purpose – supporting data warehouse

processing

Offers ease of use, simplicity, and compatibility – tested, ordered and delivered as a single system

Simple to understand even though mechanism may be complex

Low cost in terms of TCO

High performance in achieving its purpose

Single point of service provided by single vendor

18



Cost effective solution TCO of a data warehouse appliance is lower because

cost of hardware and software is cheaper Also because simplicity and ease of reduces installation,

administration and support cots Improved usability of a data warehouse appliance means

projects can be developed and deployed faster

Includes popular BI capabilities Interactive dashboards, analysis, reports, alerting, and

data integration

19


20

Sweet Spot for Data Warehouse AppliancesS

ize

of D

ata

Complexity of Workload

Mega- toGigabytes

MultipleTerabytes

Mixed Purpose

Any databasevendor

Data WarehouseAppliances

SpecializedDatabases

(e.g., Teradata,IBM)

Focused Purpose



Pros Immediate visibility &

interaction into business performance

Non-disruptive to existing infrastructure

Faster deployment Low maintenance –

black box

Cons Still some opposition to

use of appliances by IT departments Loss of “control” over

moving parts

DW and BI appliance scalability

Customization to fit each company’s needs

21


Sample Data Warehouse Appliance Vendors

Netezza Teradata DATAllegro (now Microsoft) Sun + Green Plum Sun + Vertica Sun + ParAccel Sun + Kognitio IBM InfoSphere Warehouse

22


Role of Appliances in BI SaaS

Many data warehouse appliance and BI SaaS vendors are forming partnerships Gives SaaS vendors scalability, reliability, performance Gives appliance vendors applications, new markets,

greater exposure Gives customers more confidence that solution is on solid

technological footing Performance Support for multi-tenancy Scalability Applications

23


Analytic Databases

Many are Massive Parallel Processing (MPP) Can use commodity hardware

Many have column-based data organization Limit I/O by putting similar data together – reduces reads

to only columns needed for query Single data type per column allows for significant

compression

Data compression Compression can be optimized for particular data types CPU is not the bottleneck, only I/O is

24


Analytic Databases

Built-in intelligence Allows decompression of only data that must be for query

resolution and ignore all others Is major factor in overall improved performance

Load times remain constant regardless of table size Should also have query times that remain constant

regardless of table size Bottom line – technology must be seamlessly scalable

25


Analytic Databases

Many new vendors on the market (sample): Green Plum Vertica (Michael Stonebraker*) ParAccel (Barry Zane**) Dataupia (Foster Hinshaw**) InfoBright (Warsaw University) Aster Data (Stanford University) illuminate (Former Synerra Systems founders) One well-established vendor: Sybase IQ since 1993

Most are column based, MPP, shared nothing architectures (not all though)

26* Ingres and Illustra founder** Netezza founders


Analytic Databases – Really Fast: TPC-H 1 TB

27


Analytic Databases – Really Fast and Really Inexpensive

28Graph compliments of Jos van Dongen, Tholis Consulting, NL. Numbers are estimated.

Solution Pricing model Price/unit 1 TB solution Remarks

VerticaData Volume

(raw)$ 100,000/TB $ 200,000,-

Based on 5 nodes, $ 20,000,- each

ParAccel Node$ 40,000,-

(+$10,000/TB)$ 310,000,-


ParAccelData Volume

(raw)$ 1,000,-/GB $ 1,250,000,-

From TPC-H publication

InfoBrightData Volume

(raw)$ 40,000,-/TB $ 140,000,-


Dataupia Node $ 19,500/2TB $ 19,500,-You can not buy a 1 TB Satori server

ExaSolData Volume

(active)$ 675 - $1,750

per GB$ 940,000,-

From TPC-H publication


Analytic Databases

Pros Excellent performance Very cost-effective Low maintenance Partnering with

hardware vendors (DW appliance)

Cons Many are small

companies May not handle mixed

work load well New (unknown)

technology for IT

29






30


BI Delivery Models

There are two BI delivery models today On-premises – traditional model Software as a Service (SaaS)

31


On-premises – Traditional Model

Internal IT is responsible for entire environment from first project Find excess capacity on machines Upgrade memory on existing machine for usage Leverage installed end user access tools

Buy smaller platforms that can scale Migrate to bigger box when necessary Use smaller box for data mart(s)

Look into data warehouse appliances for very large, focused BI analytics

32


33

Software as a Service (SaaS)

Characteristics* Secure, flexible, and efficient business processes &

workflows Service level agreements Value-added business services such as analytics & best

practices Extensive use of service-oriented architecture (SOA) to

enable scaling, configurability, and integration Subscription monitoring & usage-based billing

* From www.sandhill.com, “Get Ready for SaaS 2.0” by Bill McNee, Saugatuck Technology, May 8. 2006


Advantages for SaaS Vendors

Vendors support only one platform and one version of the application No need to support multiple operating systems, platforms, and

older versions of the software Decreases development costs significantly

SaaS gives vendor great visibility into how their customers are actually using their software See every move, every feature, every function used by

customers Gives vendor great intelligence on how to build a better

product based on actual usage

34


Advantages for SaaS Vendors

SaaS model gives vendor a predictable cash flow Subscription model is reliable for cash flow estimation Improves start-up estimations and growth track

Vendors don’t get trapped in “feature bloat” No need to keep adding feature after feature to get

customers to buy new versions Create only features that are needed based on actual

customer usage

35


Disadvantages for SaaS Vendors

SaaS produces lower revenues at first than traditional vendor models Must attain critical mass of subscribing customers Vendor must have enough funding to tide them over More time is needed to ramp up to mature status

Higher customer set up costs Traditional vendor model – send customer a CD SaaS vendors must allocate space, set up customer support,

etc. SaaS vendor becomes IT support for their customers (higher

costs for customer service?)

36


Disadvantages to SaaS Vendors

Customers still need ability to integrate SaaS application data with other enterprise data Need mechanism to export data out of SaaS environment Who supplies integration of SaaS data with customer’s

other data? If customer is not SOA-compliant yet, what does this

mean to SaaS model?

37


Reasons for Adoption: Ease of Deployment

This is the SaaS model’s greatest advantage No installation of hardware No installation of software No administration of new versions of either No need for IT expertise in the tool or application

Set up consists of getting a login and password set up for the business users

38


Reasons for Adoption: More Flexibility for Evolving Needs

Perhaps… You can certainly change SaaS vendors quite easily

If you are unhappy with one vendor, changing to another one is about as easy as getting a new login and password

You can influence the direction and R & D of the current SaaS vendor

You can easily add or subtract users You can easily add or subtract functionality It may not be as easy to customize the SaaS offering to

your specific needs

39


Reasons for Adoption: Not Locked into Long Licenses

True Great advantage in BI world where technology is moving

very fast Can switch from one SaaS vendor to another

But watch for cancellation fees And make sure you know what the subscription fee

is based on Reduction or addition of users may be cross price break

threshold Salesforce.com model is typical

40


Considerations for BI SaaS

SaaS – good at supporting particular types of users Highly mobile work force

Field sales personnel Product support specialists at customer sites Telecommuters

Highly geographically disbursed workforce International enterprises Non-office workers (virtual offices) Customer or partners worldwide

Must include support for various mobile devices Phones, mobile PCs, handheld devices, PDAs, etc.

41


42


Ensuring quality of delivered environment Correct mappings, verified data lineage, transformations Sufficient data quality processing Data represented in analytic engine correctly Appropriate presentation of information, e.g.,

personalized dashboards

Scalability of environment Data volumes – small beginnings to 100’s of terabytes? From a few users to 1000’s


43


Performance From simple to complex queries Response times – operational to strategic BI Getting right data to right people at right time

Open Architecture Compliance with best practices? Non-proprietary infrastructures? Integration with existing infrastructure?


44


What does SaaS vendor bring to the table? Best practices Quick start BI components like a library of reports,

analytic calculations, KPIs, etc. Industry-specific knowledge Horizontal business knowledge Support for all employees in all levels of enterprise


BI SaaS

Pros Fixed cost –

subscription model Fixed time Flexibility /

customization Single vendor

responsible for entire environment

Quick ramp up

Cons New paradigm –

nervousness? Can a company maintain

its uniqueness? Loss of “control” over data,

quality, access Vendor’s timeliness in

response to changes Vendor’s industry

knowledge

45


Sample BI SaaS Vendors

LucidEra Xactly Eyeris PivotLink (was

SeaTab) Oco On Demand IQ

Actuate Cognos SAP ERP SAS Business Objects SalesForce.com Dimensional Insight

46






47


Open Source Vendors Face Questions

The myths and doubts: Is there support for open source BI? How many people are really using it? Will it scale? Is it considered enterprise class? Is it only for developers?

These are being overcome… According to Aberdeen*, 25% of survey respondents will adopt open

source BI in next 12 to 24 months CEOs agree – open source is a worldwide growth story in 2008** First nine months of 2007, open source deal flows doubled each

quarter*** Sun’s commitment to open source - $1 Billion for MySQL

48

* Source: “The TCO of Business Intelligence – Open Source Takes on Traditional BI”, www.aberdeen.com ** Source: www.OpenSolutionsAlliance.org *** Source: www.the451group.com


Why Use Open Source?

Price! Open Source software can be downloaded, installed and

operated free of charge Return on investment (ROI) of Open Source model is

good

Open Source software is reliable and scalable Just look at the Internet – its infrastructure relies heavily

on Open Source software Wall Street – 8 of top 10 banks use Open Source

technologies

49


Why Use Open Source?

Open Source community grown to significant size Millions of developers contributing everyday Cost of development is externalized

Ability to adapt or customize Many companies don’t want or need feature bloat

Easy integration and performance New tools for building browser based reports and

dashboards accessible to more people Ad-hoc report designers with drag and drop capabilities Enhanced wizards for custom data source implementation

50


Open Source Offerings

BI projects won’t be consumed by license fees No huge up-front fees to justify before commencing

a project “Safe choice”

Many successful deployments Professional services experts to work with you Professional, public training Support from the experts – project leaders and sponsor

51


Open Source BI Vendors

Proponents of open source BI point out low cost of entry, flexibility and variety of applications available

Opponents believe open source BI lacks functionality needed to succeed right now

Regardless, don’t be fooled by “numbers of downloads”…

Vendors* – Actuate, JasperSoft, Jpivot, Mondrian, Pentaho, SpadoBI

52* For a more complete list, go to http://www.manageability.org/blog/stuff/open-source-java-business-intelligence


Open Source ETL

Open Source ETL ETL alternative follows industry standards for ease of use,

quick deployment, and fit into company’s needs Users download open source ETL code and get started Can collaborate with open source community to share

integrations and extend tool’s functionality Will probably need to buy support and services from

company’s professional services and support Sample Vendors*: Talend, JitterBug, KETL, Pentaho,

Octopus, CloverETL

53* For a more complete list, go to http://www.manageability.org/blog/stuff/open-source-etl


Open Source

Pros Cost effective Easy to install and

deploy Large development

community

Cons How do they make

money? Many are small

companies Some offerings not

truly open source

54


Right Place at Right Time – Get Going!

Once you have your ducks in a row, you are ready to create the proper environment Create an infrastructure that can withstand change – you’ll

need it Pick technologies that support that infrastructure and move you

toward SOA compliance Constantly monitor business community usage Measure ROI and publish it

IT infrastructure should be to information as a power grid is to electricity Information should flow as freely as electricity does

55


QuestionsQuestions

Claudia Imhoff, Ph.D.President

Intelligent Solutions, Inc.www.IntelSols.com

[email protected]

Documents

Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved Putting It All Together: Trends in Business Intelligence Claudia Imhoff, PhD Intelligent