How to Achieve common semantics/vocabulary on Business ... · How to Achieve common semantics/vocabulary on Business information in a Data ... SAP SALES- CPO Sal ... Ext er nal Syst

How to Achieve common semantics/vocabulary on Business information in a Data warehouse

Lasse Bache-Mathiesen

2

Business Information Management

Copyright © 2013 Capgemini. All rights reserved.

Business Information Service Center | April 2013

Capgemini Business Information Mangement

Capgemini BIM Norway

130 Employees

Located in Oslo, Stavanger, Bergen

and Trondheim

Capgemini BIM Global

7000 consultants

Lasse Bache-Mathiesen

Cand. Mag.. Mathematics UIO

25 years in the BI and reporting area

Employments

• Norsk Data

• Merkantil Data

• Sysdeco

• Affecto

• Capgemini

3




CONTENTS

1. Business Intelligence

2. History of Data Warehouse – ETL problem

3. Data Governance and organization

4. Developing the logical model

5. The implementation

6. Supporting functions

7. Supporting the target architecture

4




Business Intelligence

Interaction between

Value extracted from

Information

Co

mp

eti

tive

Ad

va

nta

ge

“I can explore my data, but is it correct?”

“How am I doing

vs. goals?”

“What is my best

opportunity?”

“I have a portfolio of appropriate

options available, at the moment of

contact”

BI Maturity model

5




5 © 2007 Capgemini - All

rights reserved

Information Overload

Manage Mater ial

Plan Supply Chain Establish Supply Chain

Manage Orders

W PCS

W ei ght s &

M easur es

I nf o'

VI S

TSV ( TM S)

TSV

Tr uck &

Reci pi ent

l i st s

TRAFFI C /

CUSTO M S

TRAF/

TRAFFI C

TAD/ TDC -

Excel W kb

TAD / TDC

Suppl i er

Schedul es

SUPPLI ER

ADDRESS

DATA

SUPPLI ER

St andar d

I nt er f ace

SQ A

SM ART-

PCS

SM ART PCS

SM ART -

PCS & M M S

SM ART -

M M S

SM ART

Shi ppi ng( C

KD

Bar code)Shi ppi ng

SERT

SECURI TY

SYSTEM S

Secur i t y

Dat abase -

Local

SECURI TY

DATABASE

Scanner s

SAP

SALES- CPO

Sal es

Pl g. &Pr odn

Rel . S/ m 's

Sal es O r der

Pl anni ng

SALES

RS

RO LS

RO C

Ref er ence

Tabl e

G M E I nv.

Pl t . t o DUNs

PSS/ G ES -

Cent r al

PSS/ G ES

PSS

Bar code

Appl i cat i on

PSS

PRO M AS

Ser ver

PRO M AS

Host

PRO M AS

PRO DUCTI O

N PEO PLE

PRO D

O RDER

CHANG ES

I N G M UD

PRI M O

PREP

( PURCHASE

PAYABLES)

PREM I D

PO M S

PLANT M I NI

PI S

PI BUS

Per Box

Docum ent at

i on

PASO S -

I NVO I CI NG /

par t / box

PASO S

Pai nt ShopPai nt

Pr ocess

Pl anni ng

Packi ng

Sl i ps

Packi ng Sl i p

Syst em

Packi ng

Sheet s

P/ DUNS

NCT

NW S

( CO SY)

NW S

NSC

PREFEREN

CI NG

NSC O RDER

CHANG ES

I N O PEL

NO RC

NAO -

O r der s

( Local )

M PCE- PM S

M PCE - PP

M PC Vehi cl e

Schedul i ng

M PC Tel ex 3

M PC Tel ex 1

M PC St or es

M odel Code

Tabl e

M odel Code

Schedul e

M M S

M M DB

M G O Specs

( Assy +

Com p )

M G O

REQ UI REM E

NTS

M G O

M O DEL

CO DE

TABLE

M G O

I NVENTO RY

M G O

CO M PO NEN

T BO M

M G O BO M -

Local Pkg.

Pl ant

M G O BO M

M G O

ASSEM BLY

BO M

M G O

M DPS -

Cent r al

M DPS

M f g.

Engi neer i ng

M f r . Pr i ce

Fi l e DBM AI S

M AI NFRAM

E PI S

I NTERFACE

LPL+LFM S+

C

KREBU

( Account s

Payabl e)

KREBU

I ES

I E

HO DLM AYE

R

G Q TS

G PS

CO NTRACT

S

G PS

ADDRESS

G PS

G PDS

( PDI S)

G PDS

G M UD - EPL

G M * M O VE

G M * DRI VE

G ener al

Assem bl y

Fr ei ght I nf o'

FLEX

Fi n. Syst em

- Local SAP

FI NANCI AL

SYSTEM

Fi nal

Assem bl y

ECO S

ETS

ESQ ES

ESPS

ESO / CO P

ESO / AAS

ESO

EPI CS-

Russ. &

G M * DRI VE

EPI CS

( PDC)

EPI CS

( O HS)

EPI CS ( ERS)

EPI CS

( BCS)

EPI CS

EI SI S

EFFY

EDS

O per at i ons

ECO S/ ADL -

Local

ECO S /

ALDL

ECO S

Cust om s

Cust om er

CTRG

CSI DS

CO SS-

O RDERI NG

& SCHED.

CO SS -

I NVO I CI NG

CO SS

( Local )

CO SS

CO SCO

CO P

Conveyor

Cont r ol

Syst em s

Cockpi t

ECO S

CKD

O RDERS-

ASSEM BLY

PLANTS

CKD Host

BO M

Pr ocessi ng

CKD

Packi ng Sl i p

Syst em

CKD

BARCO DI N

G ( S&I )

CKD

Car dBoar d

Box Li st

CAM AS 2. 0

( Si m ul at i on)

CAM AS 2. 0

( Cor e

M odul e)

CAM AS 2. 0

CAM AS

CACBP&FU

Book

Conver si on

Body Shop

BI LLI NG

SYSTEM S

Bar Code

and CKD

Shi pm ent s

AVI

G M UD &

FAM I LY

ENG I NES

AAS

# 2 2 4

# 2 6 0

#1

06

# 2 4 7

#1

70

#1

77

#1

41

#1

23

# 6 7

# 1 1 8

#1

29

# 1 2 6

# 1 3 1

#2

07

#6

6

# 6 5

# 6 0

#6

3

#6

9

# 1 1 9

# 1 1 4

# 1 2 1

# 1 2 0

# 1 1 5

#1

16

# 3 5

# 3 4

# 9 6

#2

10

# 2 1 2

# 3 3

#2

73

#4

#2

51

#2

52

# 1 9 8

# 2 1 5

# 2 1 6

# 1 3 6

# 1 3 8

# 7 2

# 9 0

# 5 3

# 5 2

# 5 4

# 5 6

# 5 5

# 5 8

# 5 9# 5 7

#4

2

# 4 5

#4

1

# 4 3

#4

4

#3

9

#4

0

#2

11

# 2 0 9

# 2 4 1

# 2 0 6

# 2 0 5

#2

1

# 1 3 0

# 2 4 0

# 2 7 0

# 2 7 1

# 1 1 2

# 1 8 9

#2

20

# 1 5 3

# 1 8 7

#9

4

#1

59

#1

60

#1

54

#1

01

#1

8

# 1 7 1

#1

86

#1

85

# 3

#1

72

# 1 7 3

#1

68

# 1 7 9

# 1 8 2

#7

0

#7

# 2 0 4

# 2 0 3

# 2 0 2

#1

51

#1

50

# 1 4 5# 1 4 6

#1

48

#1

43

#1

33

#1

34

#1

05 # 1 1 0

#1

04

# 1 1 1

#1

09

# 1 0 8

#2

30

# 1 3 7

#2

7

#2

6

# 2 3

#2

2#

20

#1

2

#6

4

# 1 2 2

# 2 6 9

# 1 1

# 1 0

# 2 2 5

# 2 6 1

# 2 2 2

#7

7

#4

8

#5

1

#5

0

#4

9

#8

7

#7

8

#7

9#

86

#9

3

#2

21

#2

58

#2

53

#2

62

#2

23

# 9 2 # 9

1

# 8 3 # 7

6

# 1 2 8

#6

8

# 8 1

#1

55

#2

57

# 1 8 8

# 1 8 3# 1 8 4

# 1 8 1

# 1 8 0

#1

78

#5

# 1 2 7

# 2 6 8

#2

8

#2

48

#2

63

# 2 4

# 2 5

#2

54

#2

18

#2

36

#8

2

#7

5

# 2 3 7

#2

33

# 2 2 9

#2

32

#2

55

#2

66

# 1 9 0

#2

19

#1

25

#2

72

#2

67

# 2 1 3

# 2 5 0

# 2 1 4

# 1 3# 1 4

# 1 5

# 2 3 5

# 2 2 7

# 2 6 5

#2

26

# 1 9 7

# 1 9 6

# 1 9 5

# 1 9 4

#2

01

#2

00

#1

99

# 6 1

# 8 4

# 8 9

# 2 5 6

#1

61

#2

28

#6

2

# 1 9 3

# 9

# 8# 2 0 8

#2

59

#1

91

#1

6 # 1 7

#2

17

# 7 4

#2

45

#2

46

# 2 4 4

#2

43

# 1

#1

24

#1

58

# 1 4 0

#1

49

# 1 4 7

#1

44

# 9 9

#1

62

# 1 6 6# 1 6 5

# 1 6 3# 1 6 4

# 1 6

7

# 1 5 6

# 1 5 7

#1

75

#1

76

# 1 5 2

# 9 7

#7

3

#1

92

#2

49

#2

31

#2

34

#7

1

# 8 8

# 1 1 3

#4

7

#4

6

#8

5

#9

5

#9

8

#2

64

#1

69

# 2 3 9

# 1 0 0

#3

0

# 3 7

#3

2

# 2 9

# 3 6

#3

1

#3

8

# 1 0 3

# 1 0 2

#8

0

#1

35

# 1 3 9

# 1 3 2

#6

# 1 9

# 1 0 7

#1

17

#1

74

# 2 3 8

#2

# 2 4 2

# 1 4 2

GME - PRODUCE PRODUCT (PP) AND PRODUCTION CONTROL & LOGISTICS (PC&L) APPLICATIONSSystems Engineering - Europe

March 1999.

Fi l e Name : Mai denhead Server S:\ Publ i c\ Produce Product DB\ Visio-Inputs&Outputs\ Overal l -PP-Landscape. vsd

P

R

O

D

U

C

T

I

O

N

C

O

N

T

R

O

L

&

L

O

G

I

S

T

I

C

S

A

P

P

L

I

C

A

T

I

O

N

S

I

N

T

E

R

F

A

C

E

S

P

R

O

D

U

C

E

P

R

O

D

U

C

T

A

P

P

L

I

C

A

T

I

O

N

S

I

N

T

E

R

F

A

C

E

S

I ndex to Appl icati ons: I ndex to Busi ness processes:

Ext er nal Syst em s M G O G PDS CKD G PS G Q TS ESQ ES SM ART PRO M AS CAM AS EPI CS

PC&L 1

Devel op

Suppl y Chai n

St r at egy

PC&L 2

Devel op

Capaci t y Pl an

PC&L 3

Devel op Pr oduct

Pr ogr am

Capaci t y Pl an

PC&L 4

Devel op I nbound

Com m odi t y Fl ow

Pl an

PC&L 5

Est abl i sh

M at er i al Sour ce

No t e :

On ly t h e f o llo win g GME PP a n d PC& L a p p lic a t io n s wh ic h we r e t h e f o c u s o f t h is s t u d y , h a v e b e e n ma p p e d a c c o r d in g

t o t h e b u s in e s s p r o c e s s d e f in e d a b o v e : GPDS , CS I DS , MMDB, GPS , PRI MO, PS S , PROMAS , PI BUS , S MART ,

POMS , COP, MGO, AAS , EPI CS , FLEX, MMS , CAMAS , MAI S , GQT S , a n d CKD. T h e o t h e r e x t e r n a l s y s t e ms a r e

la id o u t s o a s t o imp r o v e v is ib ilt y a n d k e e p in g in v ie w t h e ir in t e r f a c e s wit h t h e s y s t e ms in f o c u s .

PC&L 1 PC&L 2 PC&L 3 PC&L 4 PC&L 5 PC&L 6

PC&L 6

Est abl i sh

I nbound M at er i al

Packagi ng

PC&L 7

Est abl i sh

I nbound

Logi st i cs

PC&L 7 PC&L 8

PC&L 8

Devel op I nbound

M at er i al Fl ow

Pl an

PC&L 9

Ensur e

Pr oduct i on

Readi ness

PC&L 9

PP 1

Producti on Control

& Logi stics

Processes

Produce Product

Processes PP 1

Devel op G r oup

M odel Year

Pr oduct i on Pl an

PP 2

PP 2

Devel op G r oup

M ast er Schedul e

PP 3

Devel op Pl ant

Pr oduct i on

Schedul e

PP 3

PP 4 PP 5 PP 6 PP 7 PP 8 PP 9 PP 10

M oni t or

Pr oduct i on

Shi p and Del i ver

Pr oduct

Schedul e

M at er i al

Tr anspor t

M at er i al t o Poi nt

of Use

Assess Suppl i er

Per f or m ance

M oni t or

I nvent or y

M anage

Di sposi t i on of

Packagi ng

PP 4 PP 5

PP 6 PP 7 PP 8 PP 9 PP 10

High Level Business Processes

IT systemes in Silos

Tailored point to

point integration

Ny systems with

ovelapping

funtionality

No visibility

Need of

information

High Complexity

Legacy

systemes

No agility

6




CONTENTS








7




Data Analysts

Internal data sources

External data sources

DATA

DATA

DATA

DATA

Data

Warehouse

Data

copy

Boardroom

Balanced Scorecards

Performance Management

The data warehouse

8




Data Warehouse

Kimball

Hub and Spoke

Inmon

Enterprise Data

Warehouse

9




Layered structure Creatating the ETL problem

Operational

Environment Data Warehouse Environment Business Intelligence

Environment

E

T

L

D

W

Meta

data

Data

Warehouse

layer (DW)

Relational/

Dimensional Dimensional

BI layer

(BI)

E

T

L

B

I

2 3 4 6 7

8

Data Source

layer (DS)

Reports

Users

Analysis

1

Maintenance

Staging

layer (SA)

E

T

L

S

A

ODS

Operational

Data

Store E

T

L

D

W

Relational/

Source like

Source like

Datamart

layer (DM)

E

T

L

D

M Dimensional

5

10




BI Appliances Hadoop

Expensive dedicated HW

Built for performance

Designed for high volumes (e.g. 10s of TB)

High availability

Initially developed using Relational Data bases

Very mature solutions (skills, SW, HW, administration)

Designed for modelled and structured data

Business As Usual ways to design, build and deliver

Teradata, Exadata, Netezza, HANA...

Commodity PCs

Built for extreme scalability (Batch oriented)

Designed for extreme volumes (10s of PB and more)

Very high availability

Initially developed for web ranking

Not as mature

Hadoop = Data is distributed over many machines

MapReduce = Computing is distributed and executed

where data is (grid solution)

11




Data Analysts

Internal data sources

External data sources

DATA

DATA

DATA

DATA

Virtu

aliz

atio

n

La

ye

r Boardroom

Balanced Scorecards

Performance Management

Data Federation

12




In-memory is changing the game

An in-memory appliance

40 x86 cores, 1TB of RAM

For only 100 K EUR !

Performance improvement means:

1 to 10 ratio: 10’’ and 20’’ become instantaneous

1 to 100 ratio: 2 minutes become 1 second

1 to 1000: 2 hours are only 10 seconds

48 hours process should run in 3 minutes !

13




Backoffice

Mortgages

Data

Warehouse

Integration Broker

Call Center Internet Intermediairies

Operational

Datastore

Use of messages

Backoffice

Insurance

CRM BI Server

How can we support this kind real time architecture

14




14

Common Logical Information Model

Data is translated to and from a exchange format that is based on the Canonical Data Model, Canonical Message Model, Canonical Document Type etc.

15




CONTENTS








16




Data governance requires a transfer of responsibility from the IT system- to

the data-dimension:

Data Governance

+

16

Data governance

17




The Data governance organisation

BIS management (Data governance steering

group)

Data Governance (Operational team)

CFO (DG Executive sponsor)

Manager,

Data governance

Data stewards Head of

disciplines

Stakeholders The Data governance team

Business-areas

Support-areas

IT

1

7

18




CONTENTS








19




Distinction Between Data

Master Data

Examples

• Customer

• Product

• Account

• Organization

• Employee

• Hierarchies ++

Transactional data

Examples

• Financial transactions

• Incident

• Step in process

• Contact Event

• Trade order

19

20




Project plan

Subject 1 Subject 4 Subject 7



21




Conceptual Model – More detailed

22




CONTENTS








23




Model Layers

23

24




CONTENTS








25




Data governance tool: Business glossary

25 Effective data governance processes and tools

26




Data Quality Services

26

27




CONTENTS








28




TJE

NE

ST

ER

METADATA

ME

TA

LA

G

mobilt

DATA-

KILDER BI-

TJENESTER

DATAVAREHUS

FOR- OG

MELLOMKAMMER

DATAVAREHUS-

KJERNE DATATORG

Stjerne-

skjema

Kuber

Analyse-

data

nav.no

PORTAL

NÆR SANNTID

HISTORISERING

navet

andre

kanaler

Rapporter

Simulering/prognoser

Statistikk og analyser

Styringsinformasjon

META-

DATA

FIM

Operativt

datalager Kø

…

Eksterne

Operative

data

DATA

Eksport

Interne

Kode verk

Reference Architecture

29




29

Common Logical Information Model

Data is translated to and from a exchange format that is based on the Canonical Data Model, Canonical Message Model, Canonical Document Type etc.

The information contained in this presentation is proprietary.

© 2013 Capgemini. All rights reserved.

Rightshore® is a trademark belonging to Capgemini.

www.capgemini.com

http://www.facebook.com/Capgemini

http://www.linkedin.com/company/capgemini

http://www.twitter.com/capgemini

http://www.youtube.com/capgemini

http://www.slideshare.net/capgemini

Documents

How to Achieve common semantics/vocabulary on Business ... · How to Achieve common semantics/vocabulary on Business information in a Data ... SAP SALES- CPO Sal ... Ext er nal Syst