Embed Size (px)
Citation preview
1
Loading data from SAP BW into Microsoft Analysis Services using Open Hub Service Hermann Däubler, Program Manager, Microsoft Corporation
Summary The following paper should help to get a quick start on how to load data from SAP BW
into an external Microsoft Analysis Services MOLAP cube using the BW Open Hub
Service. Unlike using SAP .NET Connector to access one of the BW APIs it’s not
necessary to write any code. Open Hub Service is the officially recommended way to
unload data from SAP BW into external data targets.
Whenever you extract data from the mySAP BI environment into 3rd-party target
systems your SAP BW Enterprise Data Warehouse functions as an ‘Open Hub Platform’.
Since extracting data from SAP BW by any means requires additional licensing, please
make sure to contact your SAP representative.
Applies to SAP BW 3.0B, Basis SP 39, ABAP SP 39, BW SP 21 Microsoft SQLServer2000 Microsoft Data Analyzer 3.5
Keywords Open Hub Service, Microsoft Analysis Services
Level of difficulty Technical consultants, Developers
Contact This document is provided to you by the Collaboration Technology Support Center Microsoft, a joint team from SAP and Microsoft that drives interoperability. For feedback or questions you can contact the Collaboration Technology Support Center at [email protected] or [email protected]. Please check the .NET interoperability area in the SAP Developer Network http://www.sdn.sap.com/sdn/developerareas/dotnet.sdn for any updates or further information.
Collaboration Technology Support Center – Microsoft – Collaboration Brief
June 2005
2
This document is a common publication by SAP and Microsoft (“Co-Editors”) who have both contributed to its content and retain respective rights therein. The information contained in this document represents the current view of the Co-Editors on the issues discussed as of the date of publication. Because the Co-Editors must respond to changing market conditions, it should not be interpreted to be a commitment on the part of the Co-Editors, and the Co-Editors cannot guarantee the accuracy of any information presented after the date of publication. This document is for informational purposes only. NEITHER OF THE CO-EDITORS MAKES ANY WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of the Co-Editors. Either Co-Editor may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from the respective Co-Editor(s), the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, any example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, email address, logo, person, place or event is intended or should be inferred. © 2005 Microsoft Corporation. All rights reserved. © 2005 SAP AG. All rights reserved.Microsoft, Windows, Outlook, and PowerPoint and other Microsoft products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Microsoft Corporation. SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
3
Contents Summary .............................................................................................................1 Applies to ............................................................................................................1 Keywords.............................................................................................................1 Level of difficulty ................................................................................................1 Contact ................................................................................................................1 Contents ..............................................................................................................3 Introduction.........................................................................................................4 SAP BW Demo Cube 0D_DECU.........................................................................5
Activating demo cube 0D_DECU ............................................................................................ 5 Loading data into the demo cube 0D_DECU ......................................................................... 5 Issues loading the demo cube 0D_DECU .............................................................................. 6
Open Hub Service ...............................................................................................7 Unload data into relational database tables .....................................................7 Loading data into MS Analysis Services ........................................................10
Define a data source and a cube in Analysis Services ...................................................... 10 Define dimension tables in Analysis Services .................................................................... 11 Define and load a cube in Analysis Services ...................................................................... 12 Query the data in the Analysis Services cube .................................................................... 12
Unloading Hierarchies from SAP BW..............................................................13
Unload a SAP BW hierarchy to a flat file ............................................................................. 14 Load hierarchy into SQLServer2000 .................................................................................... 15 Load hierarchy into Analysis Services ................................................................................ 19
Conclusion ........................................................................................................20 Limitations.........................................................................................................20 References ........................................................................................................21
4
Introduction It might be useful in certain customer scenarios to extract data from SAP BW and load it
into an external data target like a MOLAP ( Multidimensional OLAP ) cube for special
data analysis.
The sample is based on a SAP demo cube which is part of the business content in SAP
BW. Therefore it should be possible for every SAP BW 3.0B customer to setup the same
environment which was used for the sample.
Microsoft Analysis Services 2000 which is bundled with SQLServer 2000 was used as
the external data target. BW 3.0B was also running on SQLServer 2000. Open Hub
Service is integrated in the BW GUI. It offers two default destinations to unload the data :
tables in the RDBMS and flat files. Therefore it’s also possible to extract data from a
BW installation on UNIX ( doesn’t matter which RDBMS ) and load it into Microsoft
Analysis Services on a Microsoft Windows server.
5
SAP BW Demo Cube 0D_DECU To use exactly the same test environment which was used for the sample it’s
necessary to activate the 0D_DECU demo cube.
Activating demo cube 0D_DECU The cube can be activated in the administrator workbench ( transaction RSA1 )
under section “Business Content” :
Figure 1 : demo cube activation
Loading data into the demo cube 0D_DECU When the demo cube is activated you will find several .CSV load files in the SAP work directory. Use the appropriate InfoSources and InfoPackages to load the master data and the InfoCube ( fact table ) :
6
Figure 2 : load files for demo cube
Issues loading the demo cube 0D_DECU SAP Note 370397 provides some hints regarding loading the SAP demo cubes. There
were two issues related to loading the .CSV files. A few files contained some additional
character after the last data row which confused BW. A simple workaround was to open
the .CSV file in Notepad and delete any additional ‘newline’ or whatever at the end ( just
use DEL or BACKSPACE until the cursor stands directly next to the last visible
character in the data file ).
The second issue was more difficult to solve. For whatever reason the order of the
columns in two .CSV files ( master data for 0D_SOLD_TO and trans data for the fact
table ) was different than the transfer structure which was generated. The solution was
to change the order of the columns in the DataSource/Transfer Structure.
For 0D_SOLD_TO it has to be :
0D_SOLD_TO, 0D_COUNTRY, 0D_INDUSTRY, 0LONGITUDE, 0LATITUDE,
0ALTITUDE, 0SRCID, 0PRECISID, 0D_REGION, 0D_CITY, 0D_STREET, 0D_PHONE
For the fact table it has to be :
D_CO_CODE, D_DIS_CHAN, D_DIV, D_MATERIAL, D_PROD_HIE, D_SALE_EMP,
D_SALE_ORG, D_SOLD_TO, D_VERSION, D_INV_QTY, SALES_UNIT, D_COST,
DOC_CURRCY, D_NETVLINV, D_TAXAMOUN, CALDAY, D_VTYPE
7
Open Hub Service Open Hub Service can be invoked either directly as a transaction in the BW GUI
( RSBO ) or using the menu item “Tools” in the administrator workbench ( transaction
RSA1 ) :
Figure 3 : invoke Open Hub Service
For further details check the Open Hub Service documentation ( see references at the
end ). It’s pretty easy to unload InfoObjects, InfoCubes and ODSObjects this way.
Unload data into relational database tables Open Hub Service offers two different kinds of default destinations : .csv files and
database tables. The following sample works with database tables. Four objects
from the demo cube were unloaded :
0D_CO_CODE ( InfoObject texts ) -> unloaded into table /BIC/OHIFSDEMOCO
0D_SOLD_TO ( InfoObject texts ) -> unloaded into table /BIC/OHIFSDEMOST
0D_DECU ( InfoCube ) -> unloaded into table /BIC/OHIFSDEMOCU
ODSDECU ( ODS Object ) -> unloaded into table /BIC/OHIFSDEMOOD
The ODSObject contains data from the 0D_DECU InfoCube and was created the same
8
way as described in the “ODS BAPI” white paper. Open Hub Service automatically
added ‘/BIC/OH’ at the beginning of the table names.
Now as the data resides in relational tables it’s possible to access the data with simple
SQL. In theory it would be possible to access the BW tables in the RDBMS directly
without the detour thru Open Hub Service. But this is absolutely unsupported as it would
require a full understanding of the BW metadata layer which isn’t fully documented
besides the fact that it’s not guaranteed that the internal structures look the same in
future BW releases. The following screenshot shows the access to the tables created by
Open Hub Service using Microsoft Query Analyzer. It’s a simple join between the
unloaded ODSObject and the two unloaded InfoObjects CO_CODE and SOLD_TO to
get the text data. The result is the same as the sample queries in the “OLAP BAPI” and
“ODS BAPI” white papers.
Figure 4 : SQL access to unload tables created by Open Hub Service
The following SQL query returns the same result for the unloaded InfoCube or the
unloaded corresponding ODSObject. Which means that Open Hub Service creates a
“flat” structure out of the InfoCube schema ( no dimension tables any more ). In case
the texts for the master data are needed it’s necessary to join the appropriate master
text tables.
9
select T2.TXTMD, T3.TXTSH, sum(D_COST) as sum_cost, sum(D_NETVLINV) as sum_net, sum(D_INV_QTY) as sum_qty from [/BIC/OHIFSDEMOCO] T2, [/BIC/OHIFSDEMOST] T3, -- [/BIC/OHIFSDEMOOD] T1 [/BIC/OHIFSDEMOCU] T1 where T1.D_CO_CODE between '1000' and '2000' and T1.D_SOLD_TO between '0000002100' and '0000002500' and T1.D_CO_CODE = T2.D_CO_CODE and T1.D_SOLD_TO = T3.D_SOLD_TO group by T1.D_CO_CODE, T2.TXTMD, T1.D_SOLD_TO, T3.TXTSH order by T1.D_CO_CODE, T1.D_SOLD_TO
For performance reasons it would be of course better to do the grouping first and then
join the text information as the “group by” usually reduces the result set a lot after the
join. There are different solutions possible e.g. using a view or a temp table to store the
query result including the “group by” and then joining with the text data. SQLServer2005
will offer a new syntax which allows to do it within one SQL statement :
WITH master_text( CO_CODE, SOLD_TO, sum_cost, sum_net, sum_qty ) AS ( select T1.D_CO_CODE, T1.D_SOLD_TO, sum(D_COST), sum(D_NETVLINV), sum(D_INV_QTY) from [/BIC/OHIFSDEMOOD] T1 where T1.D_CO_CODE between '1000' and '2000' and T1.D_SOLD_TO between '0000002100' and '0000002500' group by T1.D_CO_CODE, T1.D_SOLD_TO ) SELECT T2.TXTMD, T3.TXTSH, sum_cost, sum_net, sum_qty FROM master_text, [/BIC/OHIFSDEMOCO] T2, [/BIC/OHIFSDEMOST] T3 WHERE master_text.CO_CODE = T2.D_CO_CODE and
10
master_text.SOLD_TO = T3.D_SOLD_TO ORDER BY master_text.CO_CODE, master_text.SOLD_TO
The following screenshot shows the usage of this new syntax running the beta2 version
of SQLServer2005 :
Figure 5 : new “with ….” syntax in SQLServer2005 ( beta2 )
Loading data into MS Analysis Services
Define a data source and a cube in Analysis Services Before loading the data it’s necessary to define a data source in Analysis Services and
to define the cube structure. As the data was unloaded into normal relational tables
within SQLServer 2000 just choose the Microsoft OLE DB Provider for SQL Server and
enter the required connection parameters including the BW database :
11
Figure 6 : define a data source in Analysis Manager
Define dimension tables in Analysis Services Once the connection works we select the unloaded master data text tables as dimension
tables. The Analysis Manager window which lists all available tables in the BW database
takes some time to show up as there are so many entries. Look for the tables which
were created by Open Hub Service :
Figure 7 : define the ‘company code’ dimension in Analysis Manager using a wizard
On the next screen choose both columns as levels and then finish the task by giving the
dimension a name.
12
Define and load a cube in Analysis Services After both dimensions are defined ( CO_CODE and SOLD_TO ) define a cube. Select a
fact table ( e.g. /BIC/OHIFSDEMOCU – unloaded InfoCube from SAP BW ) and build a
cube design consisting of one fact table and two dimensions :
Figure 8 : define a cube in the Cube Editor of Analysis Manager
Now process the dimensions and finally the cube :
Figure 9 : process the cube which was defined in Analysis Manager
Query the data in the Analysis Services cube In the end the MOLAP cube is ready to accept MDX queries. This can be tested using
the MDX sample application which comes with Analysis Services :
13
Figure 10 : run a MDX query on the newly created cube using the MDX sample application
And of course every BI / OLAP frontend tool which supports Microsoft Analysis Services
works too. The following screenshot shows Microsoft Data Analyzer as an example :
Figure 11 : using Microsoft Data Analyzer to run the same query as before without typing MDX
Unloading Hierarchies from SAP BW It’s currently not possible to unload hierarchies with Open Hub Service. SAP offers an
ABAP report ( Z_SAP_HIERARCHY_DOWNLOAD ) which provides this functionality.
The challenge is to find it. One option is to go to the SAP Developer Network ( SDN )
and search for “Download Hierarchy”. Then you should get a list of links which includes
the PDF document with the title : “How to Download a Hierarchy to a Flat File” from
May 12, 2004. The document includes a description how to use the ABAP report as well
as the source code.
14
The ABAP report unloads the data in a specific format. Afterwards it’s necessary to load
the flat file in a way which fits for the new data target – e.g. Analysis Services.
Unload a SAP BW hierarchy to a flat file To show how the procedure basically works a simple parent-child hierarchy was created
in SAP BW. It’s a list of 10000 “artificial” phone numbers. The InfoObject is called
TELNR2 and the corresponding hierarchy is called TELNR2H. The first level in the
hierarchy is the country code ‘++49-(0)’. The second level consists of 100 area codes :
‘++49-(0)0001-0000’ … ‘++49-(0)0100-0000’. The third and last level contains 100
phone numbers for every area code. Therefore the hierarchy table has 10101 rows.
The following screenshot shows the display of the hierarchy in BW ( Administrator
Workbench -> Modeling -> InfoObjects ).
Figure 12 : display hierarchy TELNR2H in the administrator workbench
Figure 13 shows the screen of the ABAP report Z_SAP_HIERARCHY_DOWNLOAD.
The minimum values which have to be entered are : InfoObject name, hierarchy name
and the name of the flat file. The report can be started with transaction SE38.
15
Figure 13 : run the ABAP report Z_SAP_HIERARCHY_DOWNLOAD using transaction SE38
Load hierarchy into SQLServer2000 Once the hierarchy is unloaded into a flat file it can be loaded into Analysis Services.
As the data source for the test cube in Analysis Services was defined to connect to the
database HTG in SQLServer2000 it’s necessary to load the hierarchy data back to
SQLServer2000 first. This is done via Microsoft Data Transformation Services ( DTS )
which comes with SQLServer2000. Choose “Text File” as the data source and “Fixed
field” as the file format.
The unload file which was created by the report Z_SAP_HIERARCHY_DOWNLOAD
includes the following fields. In the DTS wizard you have to define the positions where
every field ends as it doesn’t contain delimiters :
NodeID Numc 8 - end at position 008
InfoObject Char 30 - end at position 038
NodeName Char 32 - end at position 070
Link Char 1 - end at position 071
ParentID Numc 8 - end at position 079
ChildID Numc 8 - end at position 087
NextID Numc 8 - end at position 095
DateTo Char 8 - end at position 103 ( not selected for unload )
DateFrom Char 8 - end at position 111 ( not selected for unload )
LeafTo Char 32 - end at position 143 ( not selected for unload )
16
LeafFrom Char 32 - end at position 175 ( not selected for unload )
Langu Char 1 - end at position 176 ( 96 without DateTo …LeafFrom )
Descr. Short Char 20 - end at position 196 ( 116 without DateTo …LeafFrom )
Descr. Med Char 40 - end at position 236 ( 156 without DateTo …LeafFrom )
Descr. Long Char 60 - end at position 296 ( 216 without DateTo …LeafFrom )
Figure 14 : define fixed field format of the load file in the DTS wizard
As the destination choose the BW database. DTS will automatically create a table using
the name of the unload file.
WARNING : there might be an issue with the last field in every row of the flat file. The
simple DTS wizard might get problems if the value of the last field doesn’t always have
the same length. It would be necessary to add blanks manually.
Figure 15 shows the table structure in Query Analyzer which DTS created.
17
Figure 15 : destination table created by DTS
To avoid the issue with the fixed length fields at the end of every row it’s possible to
modify the source of the ABAP report. Using transaction SE38 you can change the
field separator in report Z_SAP_HIERARCHY_DOWNLOAD. Just look for the call of
function “GUI_DOWNLOAD”. In the “EXPORTING” section change the value for the
“write_field_separator” from “space” to ‘X’. This will add TABs between the fields.
Figure 16 : change of field separator setting in ABAP report Z_SAP_HIERARCHY_DOWNLOAD
18
After this code change you can choose file format “Delimited” in the DTS wizard. Then
there are no problems loading the data :
Figure 17 : define delimited field format of the load file in the DTS wizard
Now the destination table which the DTS wizard created looks different. All columns are
of type varchar(8000). One more step was taken to make the handling of the hierarchy
easier in Analysis Services. A simple parent-child hierarchy in Analysis Services needs a
minimum of three columns : nodename, parent key and a unique identifier for all
members. A table – called ‘PARENT_CHILD’ - was created which includes just these
three columns. Then the values of the columns Col003 ( Nodename ), Col001 ( NodeID )
and Col005 ( ParentID ) out of the DTS destination table were inserted into the new
table. You can see the corresponding SQL statements in the following screenshot.
19
Figure 18 : create and fill a simple parent-child table which fits perfectly for Analysis Services
Load hierarchy into Analysis Services After all this preparation it’s trivial to load the hierarchy data into Analysis Services. Just
create a parent-child dimension in the dimension wizard and choose the database table
‘PARENT_CHILD’ which was created before. Select the appropriate columns as shown
in the following screenshot.
Figure 19 : define a parent-child dimension in the Analysis Manager
In the “Dimension Editor” the data of the hierarchy can be viewed under the Data tab.
Figure 20 shows that the hierarchy out of SAP BW finally arrived in Analysis Services.
20
Figure 20 : view the data of the newly created parent-child dimension in Analysis Manager
Conclusion Open Hub Service is the official interface in SAP BW to load data into external data
targets like a Microsoft Analysis Services MOLAP cube. There are other APIs which
could be used ( e.g. OLAP BAPI, ODS BAPI, OLE DB for OLAP, XMLA ). But these
are mainly reporting APIs which were not built for unloading mass data and don’t
provide features like delta load. In theory it would be possible to access the SAP BW
database tables directly ( e.g. using SQL or ODBC ). Of course this is absolutely
unsupported besides the fact that it requires an understanding of BW internals which are
not fully documented. Moreover it’s not guaranteed that these internal structures look the
same in future BW releases. Once the data was unloaded with Open Hub Service it
doesn’t matter any more how to access it.
Limitations As described in the paper it’s currently not possible to unload hierarchies from BW thru
Open Hub Service. A separate ABAP report ( Z_SAP_HIERARCHY_DOWNLOAD ) is
needed as well as some additional steps to finally load the hierarchy into the new target.
NetWeaver04 ( BW 3.5 ) provides an API for Open Hub Service which can be accessed by external programs. This allows to automate the process.
For these API’s a certification scenario to extract data via Open Hub Service to a 3rd party tool. For details see the following link scenario BW-OHS. http://www.sdn.sap.com/sdn/icc.sdn?page=business_intelligence.htm
21
References OpenHubService :
http://help.sap.com/saphelp_nw04/helpdata/en/ce/c2463c6796e61ce10000000a114084/frameset.htm Overview Analysis Services :
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/olapdmad/aggettingstart_80xj.asp
