Interconnect session 3498: Deployment Topologies for Jazz Reporting Service

PDD-3498: Deployment Topologies for Jazz Reporting Service

Rosa Naranjo ([email protected])Unleash the Labs – IBM

rhnaranjo.wordpress.com, @rnjazz

Ernest Mah ([email protected])Jazz Reporting Service Architect – IBM

Agenda

• Jazz Reporting Service Architecture– Diagram– Components Explained

• Deployment Topologies• Two Phases to Reporting

– Factors Affecting Data Collection• Strategies for high data volume

– Factors Affecting Report Execution• Strategies for high user load

• Q & A

1

Jazz Reporting Service ArchitectureDiagramComponents Explained

Jazz Reporting Service Architecture – 4Q2015

3

Report Builder(Query mgmt,OOTB

Reports)

LQEDOORS NG

RQM

RTC

Other

DWDCC

TRS

Reports

Gadgets

Spreadsheets

Embedded

CognosTools (BI Server, Data Manager, Framework

Manager)

OOTB/Custom Cognos Reports

OOTB/Custom Cognos Reports ++

OOTB Birt Reports (RQM, RTC)

CLM OfferingCLM OfferingInsight 1.1.1.7

Jazz Reporting Service Built in Reports (e.g. Quick Planner)

Jazz Reporting Service – Components explained

Report Builder• Guided, self service reporting

authoring for mainstream reports• Report management and sharing• Visualization (bar, line, pie) created

and sent to browser for rendering• Queries Data Warehouse or

Lifecycle Query Engine for Data Warehouse

Data Collection Component• Collects data from RTC, DNG, RQM

and feeds into the Data Warehouse• Schedule ODS (Operational Data

Store) and Metrics Collection Jobs• Parallel loading technology• Offloads the work of storing data

into the Data Warehouse from each application (Java ETLs from CLM 5.0.x and prior)

• ODS collection based on deltas4


Data Warehouse• Data source for Report Builder

queries• Database instance supported by

DB2, Oracle, …

Lifecycle Query Engine• Collects data from RTC, DNG, RQM

and feeds into a local disk based index

• Data source for Report Builder • Managing collection intervals from

the CLM applications.• Parallel loading technology• Collection based on deltas• Essentially like DCC + Data

Warehouse5


LDX – Links Index• Not a reporting component• Used only if you are interested in

configurations• Configuration enabled projects now

store directed 1 way links• Service used by the tools to help

locate inbound links from other tools

6


Cognos BI Server• Advanced visualizations and report

needs• Requires a Cognos BI expertise• Allows advanced CLM and other

data source ETLs

ALM Cognos Connector• Component of Jazz Reporting

Service to allow Cognos Data Manager ETLs to collect from CLM

• Installed into the Cognos BI Server

7

Deployment Topologies

Example Deployment Topologies

• High level overview– https://jazz.net/wiki/bin/view/Deployment/StandardTopologiesOverview

• Example topologies with hardware and supporting software– https://jazz.net/wiki/bin/view/Deployment/RecommendedALMDeploymentTopologies6

9

Example Deployment Topologies - Departmental

• Small team and grouped single-server deployments

• Requires less hardware

10

Example Deployment Topologies – Enterprise

• Production or medium-sized to large-sized teams and multiple server (or distributed) deployments

• Flexible application per server deployment

11

Example Deployment Topologies - Federated

• Very large enterprises who tend to deploy an ALM solution per product line or organizational division

• Enterprise wide view with rollup reporting across solution required

12

Example Deployment Topologies - Federated

13

Two Phases of ReportingData Collection and ReportingStrategies for Increasing Scale

Two Major Phases in Jazz Reporting Service

• Data Collection– Data Warehouse

• DCC requests changes from CLM apps• Changes are sent to the Relational DB

– Lifecycle Query Engine• LQE requests changes from CLM apps• LQE stores and indexes the information locally

• Reporting– Run queries against the relational database or LQE– Create final report result by combining data from queries and visualizing

them into tables, line chart, pie chart, bar charts

15

Comparison of DCC/Data Warehouse and LQE

DCC/Data Warehouse LQECollection interval 15 minutes 1 minuteConfiguration management Not Supported SupportedSupport new data in future CLM now, DOOR9

futureYes

Enterprise Scale Yes ImprovingQuery Language SQL SPARQLReady to use / Ready to copy reports

Mature Initial set

16

Factors affecting data collection performance

• Initial data population– Total number of artifacts across the applications you would like to report on

• Ongoing data population– Frequency of change across all connected applications

• Components involved– Data Warehouse - DCC, Relational DB– LQE - LQE

17

Factors affecting report execution

• Number of users running reports• Number of reports running• Quantity of data returned in the reports

• Components involved– Data Warehouse – Report Builder, Relational DB, Cognos BI– LQE – Report Builder, LQE

18

Strategies for Large Data Collection

• Consider separating data collection into logical related project groupings for handling frequent reports at those levels– Data warehouse or LQE per grouping

• Enterprise wide reporting still required?– Data warehouse or LQE across the enterprise– Minimize number of requests against this larger dataset– Use appropriate filters to grab data specifically to what is needed

19

Strategies for Large Number of Users

• Separate Report Builder Servers to group related reports together• Increase cache timeout levels in Report Builder

– Data less fresh, but can handle more users

• LQE based data– Utilize LQE horizontal scaling support to handle more query requests– Increase cache timeout in LQE

20

Lifecycle Query Engine (LQE)

Data Warehouse

Tracked Resource Set

(TRS)

Data Collection Component

(DCC)

IBM Rational Team Concert

IBM Rational Quality Manager

IBM Rational DOORS NG

Multiple Report Builders

Report Builder

Report Builder

LQE

LQE

Tracked Resource Set

(TRS)IBM Rational Team Concert

IBM Rational Quality Manager

IBM Rational DOORS NG

Horizontal Scaling - LQE

Report Builder

Existing Reverse Proxy Server

Reporting Components – Sample Specs**

• For standalone deployment - JRS Report Builder or DCC– 64-bit RHEL– 2 core– 8 GB RAM

• Report Builder + DCC Combined– 64-bit RHEL– 4 core– 16 GB RAM

• Standalone LQE - https://jazz.net/wiki/bin/view/Deployment/LifecycleQueryEngineBestPractises– 64-bit RHEL (Version 7+)– 16 core– 64 GB RAM– SSD

23**Note: Recommendations are a starting point, data volume and user activity greatly affect requirements

STG Deployment (IBM Internal deployment of JRS)

1. How many deployments of JRS / DCC do you have?

We have only one JRS and one DCC deployment which retrieves data from 3 RQM instances, 12 JTS instances, 17 RTC instances and 4 DOORS instances.

2. How many registered users do you have per deployment?

We currently have 8984 registered users in the deployment (although only a subset of them uses JRS directly).

3. What issues with respect to scale / performance do you know about with these deployments?

We have encountered just one significant performance issue related to the traceability report in JRS. This issue was resolved by adding indices to the RIDW database.

4. DCC ODS Schedule: 10 minutes

24

STG Deployment: Reporting Server Specs

1 shared server for DCC and JRS

64-bit RHEL on an 8-way blade AIX server

2.9 GHz processors

64 GB RAM

25

SDAD Deployment (IBM Internal deployment of JRS)

1. How many deployments of JRS / DCC do you have?

1 CLM enterprise environment

2. How many registered users do you have per deployment?

3000 registered users (300 CLM practitioner floating license). Typically 180 users worldwide accessing the system across 20 projects.

3. What issues with respect to scale / performance do you know about with these deployments?

Had issues with database performance. Moved from a single DB2 repo to separate instance for each application. (assuming DW is still consolidated).

4. DCC ODS Schedule

every 120 minutes, Data Mart once per day at 7:30 EST

26

SDAD Deployment: Reporting Server Specs

Separate server for DCC and JRS

JRS Server

Linux xSeries, 2 CPU(s), 8 GB Memory, 80 GB SAN Storage

DCC Server

Linux xSeries, 4 CPU(s), 16 GB Memory, 120 GB SAN Storage

27

Questions

28

Notices and Disclaimers

29

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Notices and Disclaimers Con’t.

30

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank YouYour Feedback is Important!

Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your

smartphone, laptop or conference kiosk.