Journey of Business Intelligence

Phuc H. Duong, M.Sc.phuc@it.tdt.edu.vn

CS504049 / Chapter 01

Objectives• The scope of this chapter is to build up an initial understanding of

business intelligence (BI) with definitions.

• This chapter is also aimed at building a basic understanding of how it started and what the initial use cases for it were.

• I will also cover the paradigm shift into the industry business usability use case for business intelligence.

Fall, 2020 CS504049 1

Outline1. Introduction

2. Effective and timely decisions

3. Data, information and knowledge

4. Business intelligence architectures

Fall, 2020 CS504049 2

Introduction

Fall, 2020 CS504049 3

Introduction• Business intelligence is not an alien word for the IT and business

departments within almost all corporations, whether small or big.

• There are already many players with business intelligence products on the market.

• Definition:

• Business intelligence is a set of technologies and means to acquire data in various formats from various inbound data sources and to churn out meaningful reports, or visualizations, or alerts from it, which in turn supports the business in its various aspects and in turn business.

Fall, 2020 CS504049 4

Introduction• Until recently, a data warehouse was the primary or even sole

source of business intelligence.

• However, with changing technologies and high-speed memory-based databases (e.g., HANA), it is even possible to do entirely without a data warehouse.

Fall, 2020 CS504049 5

Introduction• A transactional system could be directly utilized for reporting.

• The division between separate OLTP (online transaction processing) and OLAP (online analytical processing) systems is going away.

• OLTP systems used to have primary responsibility for supporting day-to-day business transactions, with OLAP systems running modeling and reporting exercises on subsets of data extracted into them from OLTP.

• This is data warehousing capabilities and business intelligence.

Fall, 2020 CS504049 6

Introduction• Analytics is a subset of business intelligence which is primarily

related to running statistical algorithms (generic for business process or customized for organizations) in the back end before more complex results are displayed using reports and dashboards.

• Various kinds of analytics are part of the off-the-shelf packages of almost all business intelligence products.

• With the advancement in database technologies and landscape simplification, it is possible to achieve more real-time information via business intelligence.

Fall, 2020 CS504049 7

Introduction• The various usability use cases of business intelligence are as

follows:

• Measurement of performance matrix & KPIs

• Providing a collaborative platform for data integration & exchange

• Visualization of department data

• Knowledge management

• Regulation compliance management

• Planning operation support

• Predictions

Fall, 2020 CS504049 8

Introduction• Increasing the scope of cloud-based software-as-a-service (SaaS)

and infrastructure-as-a-service (IaaS) is helping business intelligence products find their way to customers of all sizes, as they need to pay-as-per-use according to their requirements.

• Due to lower maintenance costs and higher security commitment, even some bigger players have dropped inhibitions and have moved to cloud solutions.

Fall, 2020 CS504049 9

Introduction• The intelligence driven by business intelligence is supported by

multiple types of multi-source and single-source data.

• Just like we demonstrate our intelligence by mixing up multiple sources of present and past data and experiences, business intelligence does the same.

• This is not possible to do with just a single source of information.

• Until now, the concept of learning from experience (thinking) was the missing link, but now this can be achieved using cognitive computing and analytics.

Fall, 2020 CS504049 10

The types of analytics

Fall, 2020 CS504049 11

The types of analytics

Fall, 2020 CS504049 12

• Descriptive Analytics:

• This is about "what has happened".

• This is coming out of the historical data and thus is considered hindsight.

• It is the least complex to achieve, as this is mostly data drilled up or down.

• The value is the least, as it just helps in finding out "what happened" and nothing beyond.

• Goals: reporting, segmentation, detect interesting behavior.

The types of analytics

Fall, 2020 CS504049 13

• Diagnostic Analytics:

• This is about "why this happened"; it's mostly to solve a specific problem.

• This is assisted by techniques like data drill down, data mining, and analysis of correlations.

• Diagnostic analytics is also done mostly using historical data, but it goes beyond the previous in terms of data mining and correlating pattern analysis.

• It takes the historical data beyond just looking at it as it is and converts it into data points to understand "why this must have happened".

• In short, this is insight from the historical data.

The types of analytics• Predictive Analytics:

• This type of analysis shows "what may happen".

• It combines contextual data with behavioral data and determines probable future outcomes (e.g., modern CRM and marketing solutions for understanding customer patterns in order to decide which campaign should be addressed to which customer segment)

• It needs some complex mathematical models to predict using an array of data mixed with the operational data of business.

• Kinds of prediction: regression, classification.

Fall, 2020 CS504049 14

The types of analytics• Prescriptive Analytics:

• This goes one step further and proposes "better paths of actions to avoid less good or bad ones".

• This is not easily available in mature form now, but it is on the verge of getting into that bracket.

• It includes the use of machine learning and cognition.

• It's the most complex of the four and goes beyond to add further technologies to be able to prescribe solutions by thinking about the problem.

Fall, 2020 CS504049 15

Effective and timely decisions

Fall, 2020 CS504049 16

Effective and timely decisions• In complex organizations, public or private, decisions are made on

a continual basis.

• Such decisions may be critical, have long- or short-term effects and involve people and roles at various hierarchical levels.

• Most knowledge workers reach their decisions primarily using easy and intuitive methodologies, which consider specific elements such as experience, knowledge of the application domain and the available information.

• This approach leads to a stagnant decision-making style which is inappropriate for the unstable conditions determined by frequent and rapid changes in the economic environment.

Fall, 2020 CS504049 17

Effective and timely decisions• The main purpose of business intelligence systems is to provide

knowledge workers with tools and methodologies that allow them to make effective and timely decisions.

Fall, 2020 CS504049 18

Effective and timely decisions• Effective decisions:

• The application of rigorous analytical methods allows decision makers to rely on information and knowledge which are more dependable.

• As a result, they can make better decisions and devise action plans that allow their objectives to be reached in a more effective way.

• Indeed, turning to formal analytical methods forces decision makers to explicitly describe both the criteria for evaluating alternative choices and the mechanisms regulating the problem under investigation.

• Furthermore, the ensuing in-depth examination and thought lead to a deeper awareness and comprehension of the underlying logic of the decision-making process.

Fall, 2020 CS504049 19

Effective and timely decisions• Timely decisions:

• Enterprises operate in economic environments characterized by growing levels of competition and high dynamism.

• Therefore, the ability to rapidly react to the actions of competitors and to new market conditions is a critical factor in the success or even the survival of a company.

Fall, 2020 CS504049 20

Effective and timely decisions

Fall, 2020 CS504049 21

Benefits of a business intelligence system

Data, information and knowledge

Fall, 2020 CS504049 22

Data, information and knowledge• The difference between data, information and knowledge can be

better understood through the following remarks.

• Data:

• Generally, data represent a structured codification of single primary entities, as well as of transactions involving two or more primary entities.

• For example, for a retailer data refer to primary entities such as customers, points of sale and items, while sales receipts represent the commercial transactions.

Fall, 2020 CS504049 23

Data, information and knowledge• Information:

• Information is the outcome of extraction and processing activities carried out on data, and it appears meaningful for those who receive it in a specific domain.

Fall, 2020 CS504049 24

Data, information and knowledge• Knowledge

• Information is transformed into knowledge when it is used to make decisions and develop the corresponding actions.

• Therefore, we can think of knowledge as consisting of information put to work into a specific domain, enhanced by the experience and competence of decision makers in tackling and solving complex problems.

Fall, 2020 CS504049 25

The role of mathematical models

Fall, 2020 CS504049 26

The role of mathematical models• A business intelligence system provides decision makers with

information and knowledge extracted from data, through the application of mathematical models and algorithms.

• In some instances, this activity may reduce to calculations of totals and percentages, graphically represented by simple histograms, whereas more elaborate analyses require the development of advanced optimization and learning models.

Fall, 2020 CS504049 27

The role of mathematical models• The rational approach typical of a business intelligence analysis

can be summarized schematically in the following main characteristics:

• The objectives of the analysis are identified and the performance indicators that will be used to evaluate alternative options are defined

• Mathematical models are then developed by exploiting the relationships among system control variables, parameters and evaluation metrics

• "what-if" analyses are carried out to evaluate the effects on the performance determined by variations in the control variables and changes in the parameters

Fall, 2020 CS504049 28

The role of mathematical models• The adoption of mathematical models also affords other

advantages, which can be appreciated particularly in the long term:

• First, the development of an abstract model forces decision makers to focus on the main features of the analyzed domain, thus inducing a deeper understanding of the phenomenon under investigation.

• Furthermore, the knowledge about the domain acquired when building a mathematical model can be more easily transferred in the long run to other individuals within the same organization, thus allowing a sharper preservation of knowledge in comparison to empirical decision-making processes.

Fall, 2020 CS504049 29

The role of mathematical models• The adoption of mathematical models also affords other

advantages, which can be appreciated particularly in the long term:

• […]

• Finally, a mathematical model developed for a specific decision-making task is so general and flexible that in most cases it can be applied to other ensuing situations to solve problems of similar type.

Fall, 2020 CS504049 30

Business intelligence architectures

Fall, 2020 CS504049 31

Business intelligence architectures

Fall, 2020 CS504049 32

A typical business intelligence architecture

Business intelligence architectures• Data sources:

• In a first stage, it is necessary to gather and integrate the data stored in the various primary and secondary sources, which are heterogeneous in origin and type.

• The sources consist for the most part of data belonging to operational systems, but may also include unstructured documents, such as emails and data received from external providers.

Fall, 2020 CS504049 33

Business intelligence architectures• Data warehouses and data marts:

• Using extraction and transformation tools known as extract, transform, load (ETL), the data originating from the different sources are stored in databases intended to support business intelligence analyses.

Fall, 2020 CS504049 34

Business intelligence architectures• Business intelligence methodologies:

• Data are finally extracted and used to feed mathematical models and analysis methodologies intended to support decision makers.

• In a business intelligence system, several decision support applications may be implemented, including:

• multidimensional cube analysis;

• exploratory data analysis;

• time series analysis;

• inductive learning models for data mining;

• optimization models.

Fall, 2020 CS504049 35

Business intelligence components

Fall, 2020 CS504049 36

Business intelligence components• Data exploration:

• At the third level of the pyramid we find the tools for performing a passive business intelligence analysis, which consist of query and reporting systems, as well as statistical methods.

• These are referred to as passive methodologies because decision makers are requested to generate prior hypotheses or define data extraction criteria, and then use the analysis tools to find answers and confirm their original insight.

Fall, 2020 CS504049 37

Business intelligence components• Data mining:

• The fourth level includes active business intelligence methodologies, whose purpose is the extraction of information and knowledge from data.

• These include mathematical models for pattern recognition, machine learning and data mining techniques

Fall, 2020 CS504049 38

Business intelligence components• Optimization:

• By moving up one level in the pyramid we find optimization models that allow us to determine the best solution out of a set of alternative actions, which is usually fairly extensive and sometimes even infinite.

Fall, 2020 CS504049 39

Business intelligence components• Decisions:

• The top of the pyramid corresponds to the choice and the actual adoption of a specific decision, and in some way represents the natural conclusion of the decision-making process.

• Even when business intelligence methodologies are available and successfully adopted, the choice of a decision pertains to the decision makers, who may also take advantage of informal and unstructured information available to adapt and modify the recommendations and the conclusions achieved using mathematical models.

Fall, 2020 CS504049 40

Cycle of a business intelligence analysis

Fall, 2020 CS504049 41

Cycle of a business intelligence analysis

• Analysis:

• During the analysis phase, it is necessary to recognize and accurately spell out the problem at hand.

• Decision makers must then create a mental representation of the phenomenon being analyzed, by identifying the critical factors that are perceived as the most relevant.

• The availability of business intelligence methodologies may help already in this stage, by permitting decision makers to rapidly develop various paths of investigation.

Fall, 2020 CS504049 42

Cycle of a business intelligence analysis

• Insight:

• The second phase allows decision makers to better and more deeply understand the problem at hand, often at a causal level.

• The information obtained through the analysis phase is then transformed into knowledge during the insight phase.

• Decision:

• During the third phase, knowledge obtained as a result of the insight phase is converted into decisions and subsequently into actions.

• The availability of business intelligence methodologies allows the analysis and insight phases to be executed more rapidly so that more effective and timely decisions can be made that better suit the strategic priorities of a given organization.

Fall, 2020 CS504049 43

Cycle of a business intelligence analysis

• Evaluation:

• Finally, the fourth phase of the business intelligence cycle involves performance measurement and evaluation.

• Extensive metrics should then be devised that are not exclusively limited to the financial aspects but also take into account the major performance indicators defined for the different company departments.

Fall, 2020 CS504049 44

Development of a business intelligence system

• The development of a business intelligence system can be assimilated to a project, with a specific final objective, expected development times and costs, and the usage and coordination of the resources needed to perform planned activities.

Fall, 2020 CS504049 45

Development of a business intelligence system

Fall, 2020 46

Development of a business intelligence system

• Analysis:

• During the first phase, the needs of the organization relative to the development of a business intelligence system should be carefully identified.

• This preliminary phase is generally conducted through a series of interviews of knowledge workers performing different roles and activities within the organization.

• It is necessary to clearly describe the general objectives and priorities of the project, as well as to set out the costs and benefits deriving from the development of the business intelligence system.

Fall, 2020 CS504049 47

Development of a business intelligence system

• Design:

• The second phase includes two sub-phases and is aimed at deriving a provisional plan of the overall architecture, taking into account any development in the near future and the evolution of the system in the mid term.

• Planning:

• The planning stage includes a sub-phase where the functions of the business intelligence system are defined and described in greater detail.

Fall, 2020 CS504049 48

Development of a business intelligence system

• Implementation and control:

• First, the data warehouse and each specific data mart are developed. These represent the information infrastructures that will feed the business intelligence system.

• Moreover, ETL procedures are set out to extract and transform the data existing in the primary sources, loading them into the data warehouse and the data marts.

• The next step is aimed at developing the core business intelligence applications that allow the planned analyses to be carried out.

• Finally, the system is released for test and usage.

Fall, 2020 CS504049 49

Overview of the main methodologies

Fall, 2020 CS504049 50

Portfolio of available methodologies in a business intelligence system

Fall, 2020 CS504049 51

END OF CHAPTER