Connecting Power BI to TDV - community.tibco.com · 1 Professional Services Global Headquarters 3303 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 +1 800-420-8450 Fax:

1

Professional

Services

www.tibco.com

Global Headquarters 3303 Hillview

Avenue Palo Alto, CA 94304

Tel: +1 650-846-1000 +1 800-420-8450 Fax: +1 650-846-1005

TIBCO Software empowers

executives, developers, and

business users with Fast Data

solutions that make the right data

available in real time for faster

answers, better decisions, and

smarter action. Over the past 15

years, thousands of businesses

across the globe have relied on

TIBCO technology to integrate their

applications and ecosystems,

analyze their data, and create real-

time solutions. Learn how TIBCO

turns data—big or small—into

differentiation at www.tibco.com.

Connecting Microsoft Power BI to TIBCO Data Virtualization

Project Name

Release 2.0

Date April 2018

Primary Author

Sam Carroll

Document Owner

Tony Young

Client

Document Location

Purpose Provides the steps necessary to make

TIBCO Data Virtualization resources directly queryable in Microsoft’s Power BI

cloud offering.

Connecting Power BI to TDV

© Copyright TIBCO Software Inc. 2 of 30



Revision History

Version Date Author Comments

1.0 February 2018 Sam Carroll Initial revision

2.0 April 2018 Deane Harding Updated with TIBCO branding

Approvals This document requires the following approvals. Signed approval forms are filed in the project files.

Name Signature Title Company Date of Issue Version

Distribution This document has been distributed to:

Name Title Company Date of Issue Version

Related Documents This document is related to:

Document File Name Author

TDV Client Interfaces Guide

TIB_tdv_7.0.6_ClientInterfacesGuide.pdf TIBCO DV Engineering



Copyright Notice COPYRIGHT© TIBCO Software Inc. This document is unpublished and the foregoing notice is affixed to protect TIBCO Software Inc. in the event of inadvertent publication. All rights reserved. No part of this document may be reproduced in any form, including photocopying or transmission electronically to any computer, without prior written consent of TIBCO Software Inc. The information contained in this document is confidential and proprietary to TIBCO Software Inc. and may not be used or disclosed except as expressly authorized in writing by TIBCO Software Inc. Copyright protection includes material generated from our software programs displayed on the screen, such as icons, screen displays, and the like.

Trademarks All brand and product names are trademarks or registered trademarks of their respective holders and are hereby acknowledged. Technologies described herein are either covered by existing patents or patent applications are in progress.

Confidentiality The information in this document is subject to change without notice. This document contains information that is confidential and proprietary to TIBCO Software Inc. and its affiliates and may not be copied, published, or disclosed to others, or used for any purposes other than review, without written authorization of an officer of TIBCO Software Inc. Submission of this document does not represent a commitment to implement any portion of this specification in the products of the submitters.

Content Warranty The information in this document is subject to change without notice. THIS DOCUMENT IS PROVIDED "AS IS" AND TIBCO MAKES NO WARRANTY, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING BUT NOT LIMITED TO ALL

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. TIBCO Software Inc. shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance or use of this material.

Export This document and related technical data, are subject to U.S. export control laws, including without limitation the U.S. Export Administration Act and its associated regulations, and may be subject to export or import regulations of other countries. You agree not to export or re-export this document in any form in violation of the applicable export or import laws of the United States or any foreign jurisdiction.

For more information, please contact:

TIBCO Software Inc. 3303 Hillview Avenue Palo Alto, CA 94304

USA



Table of Contents 1 Introduction ............................................................................................................... 6

1.1 Purpose ..................................................................................................... 6

1.2 Assumptions .............................................................................................. 6

1.3 Audience ................................................................................................... 6

2 Connect Power BI to TDV ......................................................................................... 7

2.1 Environment Configuration ........................................................................ 8

2.1.1 Provision or Install SQL Server 2017 .............................................................................. 8 2.1.2 Install Data Management Gateway for Power BI ............................................................. 8

2.2 Prove Connectivity to a Data Source....................................................... 10

2.2.1 Connect a Data Source to the Gateway ........................................................................ 10 2.2.2 Build a Power BI Desktop Report to Connect to the Data Source .................................. 13

2.3 Install and Configure the TDV ODBC Driver ........................................... 17

2.4 Create SSIS Package ............................................................................. 17

2.5 Deploy SSIS Package to SSISDB ........................................................... 19

2.6 Configure SQL Server to Allow SSIS Packages to be Executed as a View ...... 21

3 Build a Power BI Desktop Report to Connect to the Published SQL Server View .. 28



1 Introduction

1.1 Purpose The purpose of this document is to provide the steps necessary to make TIBCO Data Virtualization (TDV) resources directly queryable in Microsoft’s Power BI cloud offering.

The document provides guidance to overcome challenges with the current limitations of the Power BI data management

system. Currently, ODBC-sourced report queries are not allowed to be executed live from the Power BI cloud offering. i.e. when a user connects to a report in Power BI, they will be querying a version of the data loaded to Power BI on a schedule if it isn’t a ‘Direct Query’ source for Power BI. The current list of Direct Query sources can be seen here: https://docs.microsoft.com/en-us/power-bi/desktop-directquery-data-sources.

1.2 Assumptions This document assumes the following tools are available to the reader and that the reader has sufficient knowledge of these tool.

• SQL Server 2017 (minimum Developer Edition)

o SQL Server Integration Services

o Database Engine

o Client Tools

� SQL Server Management Studio

• Microsoft Visual Studio Professional 2017

o SQL Server Data Tools (SSDT) for Visual Studio 2017

• Microsoft Office 365 Enterprise License

• Microsoft Power BI Desktop

• Microsoft Data Management Gateway

1.3 Audience This document is intended to provide guidance to the following users:

• TIBCO Professional Services

• TDV Administrators

• TDV Developers

• Microsoft Power BI Developers



2 Connect Power BI to TDV

The end goal for this document is to get to a situation where users can build a dynamic SSIS package that will appear as a SQL Server database view that will, in real-time, query TDV from a report in Power BI Cloud, like you see here:

The diagram below shows the traditional, albeit oversimplified, Power BI workflow:

The following steps provide the necessary details to allow Power BI reports to query TDV resources in real-time. To achieve this requires use of some SQL Server tools to facilitate dynamic querying of TDV.

Power BI Cloud

Data Management

Gateway Server

TDV SQL Server

Direct Query (Live)

Scheduled Refresh



2.1 Environment Configuration There are two main environmental elements to configure; a SQL Server instance and a Data Management Gateway.

2.1.1 Provision or Install SQL Server 2017 Provision a SQL Server VM in the cloud provider of your choice or installer SQL Server 2017 developer edition from the download site here:

https://www.microsoft.com/en-us/sql-server/sql-server-downloads

After downloading make sure to install the database engine, integration services, and the client tools. Below is a screenshot of a working database setup:

2.1.2 Install Data Management Gateway for Power BI IMPORTANT NOTE: In order to install the Data Management Gateway for Power BI you must have an enterprise

license for Microsoft Office 365.

1. Download the gateway from this link: http://go.microsoft.com/fwlink/?LinkID=820925



2. You will need to run the installer and login with the email address associated with your Power BI enterprise account.

3. You’ll be prompted to enter the password for your account:



4. Once the installation is complete, you will see the status of your gateway:

2.2 Prove Connectivity to a Data Source After you have installed the data management gateway, it is advisable to run a quick test by connecting the gateway to a data source via the Power BI Web Portal. In this document we show testing using a SQL Server instance running on the same machine as the data management gateway; this isn’t a requirement, the gateway can connect to any data source it has the ability to reach on the network. More details on how gateways work can be found here:

https://docs.microsoft.com/en-us/power-bi/service-gateway-onprem-indepth

2.2.1 Connect a Data Source to the Gateway After you have installed

To add data sources to Power BI Web Portal, do the following:

1. Select Manage gateways from the Settings menu



2. Select your gateway and click Add Data Source

3. Give your data source a name and select SQL Server. Remember, in this first test case we want to connect directly to a standard SQL Server table just to ensure the connection is working properly.



4. Enter the credentials for the database. In this example we are connecting to a database called Dummy.

IMPORTANT NOTE: Basic credentials are sufficient for this initial test, however you MUST use WINDOWS

Authentication later on when we setup the TDV SSIS query views! If you don’t use WINDOWS Authentication, you will receive an ambiguous error message about not being able to read the column metadata from the SSIS package.



2.2.2 Build a Power BI Desktop Report to Connect to the Data Source After connecting to the gateway, it is advisable to build a simple Power BI Desktop report to connect to the SQL Server instance we associated with the gateway.

Side note: When building the report on your desktop in Power BI Desktop, you don’t have to specify anything to let it

know to use the data management gateway when you upload. Microsoft automatically determines if your source has been added to the data management gateway and will notify you if it is available for refresh in Power BI directly.

1. Make sure you download and install Power BI Desktop:

https://powerbi.microsoft.com/en-us/desktop

2. Launch Power BI Desktop

3. Log in with your Microsoft Office 365 enterprise account associated with Power BI Cloud

4. Connect to SQL Server



5. You need to enter your login credentials, in this example we are using basic SQL Server authentication, but REMEMBER: when publishing a TDV view to SQL Server you MUST use WINDOWS Authentication.



6. Create a basic report in Power BI Desktop

7. Publish the report to Power BI Cloud. Note: If you haven’t logged in to Power BI Cloud you will be prompted to

complete the login process in order to complete the publishing.

8. After completing the publish step, you will notice the report has been configured to use an enterprise gateway. If we hadn’t set up this source in the steps above then you would see a different message here.



9. Log in to Power BI, you should be able to see a query coming to SQL Server live at runtime during the Power

BI login process. To confirm this, do the following:

a. Start SQL Server profiler to profile the queries incoming to the SQL Server when Power BI is connecting

b. Connect to Power BI. Note: The report we published previously now shows up under our workspace:

c. We can open our report and refresh it



d. The SQL profiler shows the requests coming through from Power BI into SQL Server.

2.3 Install and Configure the TDV ODBC Driver The TBV ODBC driver must be present on the host where SSIS will be running. Typically, developers build these packages on their local workstations and DEPLOY to a SQL Server instance.

The deployment target will need to have the same DSN entries, and ODBC driver installed. After you’ve successfully

installed and configured the ODBC driver for TDV, create a system DSN that you will use as a connection to TDV resources.

After you create the DSN you can install/configure Visual Studio to begin development work on the SSIS package to allow ‘streaming’ TDV published resources as a SQL Server view, as described in the next section.

2.4 Create SSIS Package The next step of the process is to install SQL Server Development tools for Visual Studio. SSDT will allow you to build a SSIS package.

Once installed, create a new ‘Integration Services’ project in Visual Studio. Your end Visual Studio project will be comprised of a ‘Data Flow Task’ with an ODBC/ADO.net source going to a ‘Data Streaming Destination’.



In the example flow shown below, you will notice we have a data conversion task that is converting a text datatype to a varchar(255). We could have changed this in TDV, but we didn’t want to go through the process of creating new views simply to reference the published SYS_REQUESTS views in this particular example.

After building your SSIS package and saving it to a file, make sure you test it to ensure it executes successfully:



2.5 Deploy SSIS Package to SSISDB Now that we have an SSIS package defined and tested, the next step is to deploy the SSIS package to the SSISDB running on the SQL Server instance we configured earlier.

IMPORTANT NOTE: During installation you must have check ‘Integration Services’ features for SSIS to be available.

Create the SSIS Catalog by right clicking the SSIS database and select ‘Create Catalog’. Once done, deploy the visual studio project to the SSISDB:.

1. Select the package file that you want to deploy

2. Select where in the Integration Services catalog you are deploying the package to:



3. Review and confirm your choices

4. The results of the package deployment will be displayed to you. Once the package is successfully deployed you can close this window.



2.6 Configure SQL Server to Allow SSIS Packages to be Executed as a View After package deployment is successful, we can begin the configuration of SQL Server to allow SSIS packages to be

executed as a view.

For this step to work, we must log into the SSIS instance and configure an option for the ‘SSISOLEDB’ provider that is nested under SqlServerInstance/Server Objects/Linked Servers/Providers’. The property we need to modify is called ‘Allow inprocess’.

You need to check the radio button next to this option, hit OK, and restart the MSSQL service for the instance we are using:



After the above option is set, we are now ready to publish our TDV SSIS package as a SQL Server view. To do this, we need to launch the ‘Data Feed Publishing Wizard’:

1. After launching the Data Feed Publishing Wizard, you will be presented with this introduction page, click Next.

2. In the Package Settings dialog, browse to and select the SSIS package that you want to publish as a view in the SQL Server database



3. The package will be displayed in the Path property. You can then click Next (not shown in the following screenshot):

4. In the Publish Settings dialog, make sure you set the database to whatever database you want the SSIS package to show up under. The process will create a new view named whatever you provided under the database name you input.



NOTE: Be sure the setting Use32BitRuntime is set to true

Click Next

5. On the Validation dialog the validation process will run. Once it is complete, click Next



6. On the Summary dialog, click Publish.



7. When the publishing process has completed you will be presented with the Finish dialog. Click Finish.

8. You can now execute a query against the published view:





3 Build a Power BI Desktop Report to Connect to the Published SQL Server View

Now that we have configured SQL Server to allow our SSIS package to be executed as a view, the next step is to make sure that you added the SQL Server data source to the Power BI Management Gateway like we did in section 2.2.1 of this document.

Once that is done, create a Power BI report that connects to the SQL Server view we just published.

IMPORTANT NOTE: Make sure you choose the Use my current credentials option when connecting to the SQL

Server database! Power BI will get stuck in an infinite exception cycle stating “you cannot currently use ‘alternate

credentials’ for Direct Query sources.

After your report is built, publish it to Power BI and query it from there:



If we check the SQL Server Profiler we will see our Windows Authenticated user come through and execute our packaged query:

Also, in TDV Manager we see the request from the SSIS package query the SYS_REQUESTS view we are calling:



NOTE: There is currently no way to have Power BI append where clauses directly to the TDV statements. You could

parameterize the SSIS package to accept them when you create the view on SQL Server (I.E. select * from

ssisview(inputparam1, inputparam2) and have them mapped inside of SSIS, but there is no way to have the input query dynamically generated, since SSIS requires the metadata about the source input before it will let you create the data streaming destination.