Business Intelligence

Business IntelligenceBusiness Intelligence

Lecture 15

OutlinesOutlines

Why integrate BI into your smart client application?

Integrating Data Mining into your application

Integrating Reporting into your application

What Is Business Intelligence?What Is Business Intelligence?

“Business intelligence (BI) is a broad category of business processes, application software and other technologies for gathering, storing, analyzing, and providing access to data to help users make better business decisions”

Why integrate BI into your app?

BI makes applications smarter!

Your customers get more value from their data

Big payoff with small amounts of coding

Data mining life cycleData mining life cycle

What Is SQL Server Data Mining?What Is SQL Server Data Mining?

Data Mining platform for analysis and deployment of data mining solutions

Nine algorithms developed in conjunction with Microsoft Research

Tools for the creation and comprehension of models

Familiar API’s and query language for embedding data mining into intelligent applications

Mining ModelMining Model

What Does Data Mining Do?IllustratedWhat Does Data Mining Do?Illustrated

DMDMEngiEnginene

Data Data To PredictTo Predict

DMDMEngiEnginene

Predicted DataPredicted Data

Training DataTraining Data



DB dataDB dataClient dataClient dataApplication dataApplication data

DB dataDB dataClient dataClient dataApplication dataApplication data““Just one row”Just one row”

Intelligent ApplicationsIntelligent Applications

Make decisions without codingData Mining algorithms learn business rules directly from the data, freeing you from trying discover and code them yourselves

Customized for each clientData Mining learns the rules from the client’s data – resulting in logic that is automatically specialized for each individual client

Automatically update themselvesAs your client’s business changes, so do the factors that impact their business. Data Mining allows your application logic to be automatically updated through a simple processing step. Applications do not need to be rewritten, recompiled or redeployed, and are always online – even during processing

Data Data MartsMarts

CRM

LOB

ERP

SourceSourceSystemsSystems

Integrated Integrated Enterprise Enterprise

BI BI PlatformPlatform

Data AnalysisData Analysis(OLAP, Data(OLAP, Data

Mining)Mining)

Familiar,Familiar,PowerfulPowerfulBI ToolsBI Tools

ClientClient PortalPortal

DevicesDevices

EnterprisEnterprisee

ETLETL

Third Party Third Party ApplicationsApplications

EnterprisEnterprisee

ReportinReportingg

Information Information DeliveryDelivery

Data Data VisualizationVisualization

GeospatialGeospatial

Project Project managementmanagement

Reporting & AnalysisReporting & Analysis

Business Business Performance Performance ManagementManagement

DataDataWarehousWarehous

ee

BusinesBusiness s

AnalysiAnalysiss

EnterprisEnterpriseeReportinReportingg

PerformancPerformance e ManagemeManagementnt

Data acquisition Data acquisition from source from source systems and systems and integrationintegrationData Data transformation transformation and synthesisand synthesis

Data Data enrichment, enrichment, with business with business logic, logic, hierarchical hierarchical viewsviewsData discovery Data discovery via data miningvia data mining

Data Data presentation presentation and distributionand distributionData access for Data access for

the massesthe masses

ReportReportAnalyzeAnalyzeIntegrateIntegrate

SQL Server 2005

SQL Server 2005 launches November 7, 2005SQL Server 2005 launches November 7, 2005

Example: before Integration ServicesExample: before Integration Services

Call centre data: semi structured

Legacy data: binary files

Application database

ETL Warehouse

Reports

Mobiledata

Data mining

Alerts & escalation

•Integration and warehousing require separate, staged, operations.•Preparation of data requires different, often incompatible, tools – difficult to audit.•Reporting and escalation is a slow process, delaying smart responses – sequential operations.•Heavy data volumes make this scenario increasingly unworkable.

Handcoding

StagingText Mining

ETL Staging

Cleansing &

ETL

Staging

ETL

Example: with Integration ServicesExample: with Integration Services

Call centre: semi-structured data

Legacy data: binary files

Application database

Alerts & escalation

•Integration and warehousing are a seamless, manageable, operation.•Sourced, prepare and load data in a single, auditable process.•Reporting and escalation can be parallelized with the warehouse load.•Scales to handle heavy and complex data requirements.

SQL Server Integration Services

Text miningcomponents

Customsource

Standardsources

Data cleansingcomponents

Merges

Data miningcomponents

Warehouse

Reports

Mobiledata

Enterprise ETL platform

High performance

High scale

More trustworthy and reliable

Best in class usability

Rich development environment

Source control

Visual debugging of control flow and data

Great range of transforms out-of-the-box

Highly extensible

Custom tasks

Custom enumerations

Custom transformations

Custom data sources

Enterprise BI TodayEnterprise BI Today

datadatawarehousewarehouse

(DW)(DW)

DatamartDatamart

DatamartDatamart

Data ModelData Model

Reporting Reporting Tool (1)Tool (1)

MOLAPMOLAP

MOLAPMOLAP


ToolToolData SourceData Source

OLAP OLAP Browser (2)Browser (2)

Online Analytical Online Analytical Processing (OLAP) Processing (OLAP) Browser (1)Browser (1)


DWDW

DatamartDatamart

DatamartDatamart

MOLAPMOLAP

MOLAPMOLAP

Enterprise BI – A Messy RealityEnterprise BI – A Messy Reality








Duplicate Duplicate ModelsModels

OLAPOLAPversusversusReportingReporting

Duplicate Duplicate DataData

Relational Versus OLAP ReportsRelational Versus OLAP Reports

Feature Relational OLAP

Flexible schema

Real time data access

Single data store

Simple management

Detail reporting

High performance

End-user oriented

Ease of navigation and exploration

Rich analytics

Rich semantics

DWDW

DatamartDatamart

DatamartDatamart


BI ApplicationsBI Applications

MOLAPMOLAP

MOLAPMOLAP





Reporting Reporting Tool (2)Tool (2)UDMUDM

Enterprise BI – With A UDMEnterprise BI – With A UDM

Carol M. Rice

What is UDM? Not an approved abbreviation so please call out instead. Address throughout the rest of the slides.

DWDW

DatamartDatamart

DatamartDatamart


BI ApplicationsBI Applications

MOLAPMOLAP

MOLAPMOLAP





Reporting Reporting Tool (2)Tool (2)UDMUDM

A single dimensional model for all OLAP analysis and A single dimensional model for all OLAP analysis and Relational reporting needsRelational reporting needs

Enterprise BI – With A UDMEnterprise BI – With A UDM

Analysis ServicesAnalysis Services

Unified Dimensional ModelIntegrating relational and OLAP views

Advanced Business IntelligenceKPIs, MDX scripts, translations, currency…

Web servicesNative XML/A

Critical Server Abilities

Pro-active cachingBringing the best of MOLAP to ROLAP

Value of Data MiningValue of Data Mining

5 new algorithms

Graphical tools/wizards

12 embeddable viewers

Tightly integrated with AS, DTS, Reporting

Integration with Web/Office apps

SQL Server 2005SQL Server 2005

OLAP

Reports (Adhoc)

Reports (Static)

Data Mining

Business Knowledge

Easy Difficult

Usability

Rel

ativ

e B

us

ine

ss V

alu

e

New Set of AlgorithmsNew Set of Algorithms

Decision TreesDecision Trees ClusteringClustering Time SeriesTime Series

Sequence Sequence ClusteringClustering

AssociationAssociation Naïve BayesNaïve Bayes

Neural NetNeural Net

Introduced in SQL Server 2000

Mining Model AlgorithmsMining Model Algorithms

Microsoft Decision Trees

Microsoft Clustering

Microsoft Naïve Bayes

Microsoft Time Series

Microsoft Association

Microsoft Sequence Clustering

Microsoft Neural Network

Microsoft Linear Regression

Microsoft Logistic Regression

Analysis ServicesAnalysis ServicesServerServer


Data Mining AlgorithmData Mining Algorithm DataDataSourceSource

Server Mining ArchitectureServer Mining Architecture

Your ApplicationYour Application

OLE DB/ ADOMD/ XMLAOLE DB/ ADOMD/ XMLA

DeployDeploy

BI Dev BI Dev StudioStudio (Visual (Visual Studio)Studio)

AppAppDataData

Programmatically Creating ModelsProgrammatically Creating Models

Analysis Management Objects

Object model for administrative tasks

Data Mining Extensions (DMX)

CREATE MINING MODELCREATE MINING MODEL TargetMailDTTargetMailDT(CustID(CustID LONG KEYLONG KEY,,Gender Gender TEXT DISCRETETEXT DISCRETE,,CommuteDist CommuteDist TEXT DISCRETETEXT DISCRETE,,Education Education LONGLONG CONTINUOUSCONTINUOUS,,……BikeBuyerBikeBuyer LONG DISCRETE PREDICTLONG DISCRETE PREDICT))USINGUSING Microsoft_Decision_Trees Microsoft_Decision_Trees

INSERT INTOINSERT INTO TargetMailDTTargetMailDT(CustID, Gender, CommuteDist,(CustID, Gender, CommuteDist, Education, …, BikeBuyer) Education, …, BikeBuyer)OPENQUERYOPENQUERY ([My Data Source],([My Data Source], ‘SELECT CustID, Gender, ‘SELECT CustID, Gender, ComDist, Education, … ComDist, Education, … BikeBuyer’ BikeBuyer’))

Programmatically Creating ModelsProgrammatically Creating Models

Analysis Management Objects

Object model for administrative tasks

Data Mining Extensions (DMX)

Dynamic mining using client data

CREATE SESSION MINING MODELCREATE SESSION MINING MODEL TargetMailDTTargetMailDT(CustID(CustID LONG KEYLONG KEY,,Gender Gender TEXT DISCRETETEXT DISCRETE,,CommuteDist CommuteDist TEXT DISCRETETEXT DISCRETE,,Education Education LONGLONG CONTINUOUSCONTINUOUS,,……BikeBuyerBikeBuyer LONG DISCRETE PREDICTLONG DISCRETE PREDICT))USINGUSING Microsoft_Decision_Trees Microsoft_Decision_Trees

INSERT INTOINSERT INTO TargetMailDTTargetMailDT(CustID, Gender, CommuteDist,(CustID, Gender, CommuteDist, Education, …, BikeBuyer) Education, …, BikeBuyer)@InputRowset@InputRowset


Local Mining ArchitectureLocal Mining Architecture

ADOMD.Net/OLE DBADOMD.Net/OLE DB

Local Analysis Services Local Analysis Services (msmdlocal)(msmdlocal)

Model FileModel FileMining ModelMining Model

Decision Tree/Clustering algorithmsDecision Tree/Clustering algorithms

RetrievRetrieve Datae Data

DataDataSourceSource

AppAppDataData

Why Integrate Reporting Services Into Your App?Why Integrate Reporting Services Into Your App?

Process data efficiently: filtering, sorting, grouping, aggregations, etc.

Present data in a variety of ways: lists, tables,charts, matrices

Add visual appeal to data: fonts, colors, border styles, background images, etc.

Enable interactive reports: collapsible sections, document map, bookmarks, interactive sorting, etc.

Support printing and print preview

Export to multiple formats (Microsoft Office Excel, PDF, XML)

Integrate with managed report server environment: security, caching, scheduling, delivery, etc.

Bottom Line: More functionality, less codeBottom Line: More functionality, less code

Reporting Services In Visual Studio 2005Reporting Services In Visual Studio 2005

Microsoft Visual Studio 2005 includesReporting Services functionality in standard editions and above

Window Forms and ASP.NET report viewer controls make it easy to embed Reporting Services functionality in applications

Integrated report designer for designing and embedding reports in language projects

Freely redistributable with no run-time restrictionsReport Server mode requires a SQL Server 2005 license

Report ServerReport Server

SQL Server Catalog (Metadata)SQL Server Catalog (Metadata)

Web Services APIWeb Services API

Report Processing and RenderingReport Processing and Rendering DataDataSourceSource

Report Server ModeReport Server Mode


Report ViewerReport Viewer(Remote Mode)(Remote Mode)

PublishPublishReportReport

ReportReportDesignerDesigner(Visual (Visual Studio)Studio)


Report ViewerReport Viewer(Local Mode)(Local Mode)

Report ProcessingReport Processing

Local ModeLocal Mode

Report Report DefinitionDefinition

ReportReportDesigneDesigne

rr(Visual (Visual Studio)Studio)

EmbedEmbedReportReport

Retrieve Retrieve DataData

Report Report DataData

DataDataSourceSource

Designing In Local ModeDesigning In Local Mode

Report creation is integrated with Windows or Web language project (Add New Item…)

Integrates with Visual Studio data toolsBuild report from project data sources (XSD)

Supports Data Sources window

Supports Data Source Configuration Wizard

Supports TableAdapter Configuration Wizard

Full InteractivityExpand/Collapse sections

Drillthrough reports, including drillthrough from charts

Interactive sorting

Introduced with SQL Server 2000

Open, extensible enterprise reporting solution

Report authoring, management, delivery

Office System integration

VS.NET development environment

SQL Server 2005 enhancements

Integration with AS, DTS, management tools

Developer enhancements

Improved report interactivity

Rich end-user reporting

Reporting Services

Reporting ManagerReporting Manager

Rich Client Printing Multi-Valued Parameters

Date Picker Parameters

Sorting

Export to Excel, PDFExport to Excel, PDF

E-mail DeliveryE-mail Delivery

Graphical Report LayoutGraphical Report Layout

Report BuilderReport Builder

A new ad-hoc report design tool for Reporting Services

Targeted at business users who want to find and share answers to interesting questions

Driven from a business model of the data so users do not need to understand the underlying data structures

Fully integrated with Reporting Services and delivered in SQL Server 2005

Report Builder ClientReport Builder Client

Users download and launch the client from Report Manager (new or existing report)

Finished reports can be saved on the server

Report Builder Model DesignerReport Builder Model Designer

For editing models used by Report Builder

Auto-generation of models from data sources

Application ExamplesApplication Examples

Healthcare

Builds models predicting health degradation in citizens

Sends reports to physicians of top patients likely to get sick

Alert Governments with predictable decease

Marketing

Sales Forecast

Targeting client

Traffic

Real-time predictive traffic info delivered to web and mobile devices

Everything…………………………….

ResourcesResources

msdn.microsoft.com/sql

SQL Server Data Mining

www.SQLServerDataMining.com

SQL Server Forums

http://forums.microsoft.com/msdn

Documents

Business Intelligence