Upload
sebastien-goiffon
View
120
Download
1
Embed Size (px)
Citation preview
Track Calculation Engine Changes & Regressions in SAP BusinessObjects
Bruno Masek, VP, GB&SMITHAmy Betten, Director of Sales, GB&SMITH
• How do you track calculation engine changes? Alt+Tab ? Print Reports and put on top of each other 2 screens Asking Business Or no checking
STATE OF THE ART?
REGRESSION TESTING
WHEN?• Migration• Service Pack / Patch• Univers Update (UNV —> UNX)• ETL / Data-warehouse changes• Etc
3.1 Webi 4.2 Webi
3.1 Deski 4.2 Webi
4.1 SP74.1 SP5
ROI
• In house checking => Safety• No more manual checking • Automated and Scheduled process • Can be used by Business, QA & IT
ROI
USE CASE
While POC, a service pack was implemented 4.1 SP4 to 4.1 SP6. SP6 bugs + regressions were found due to Sybase DB upgrade.Upgrade was stopped, until Bind checked all reports for calculation engine changes/regressions & all issues fixed.Risk management was included in the loop/process and permanent automated regression testing became mandatory.
BI4.1 SP6 and Sybase upgrade
CENTRAL BANK
USE CASE
While POC, administrator ran the automated regression tool live on a few financial reports that had been checked by consultants. Admin found on some reports with data coming from same database : $1Billion margin in 3.1 and $1.2Billion in 4.1. Customer estimated 40+hours were needed to check certain reports each quarter due to service packs.
BI4.1 Quality check
REAL-ESTATE COMPANY
USE CASE
During a migration with very tight budget, City of Chicago implemented automated calculation engine/regression testing after report clean-up and conversion. As a result $200,000 were saved and regressions were found even on reports 100% converted.
Migration 3.1 to 4.2
CITY OF CHICAGO
USE CASE
500 reports (Webi-Deski-Crytal) complex reports needed to be checked. 250 man days were needed with the Alt+tab method. Automated process was implemented taking 15 days and finding 5% Calculation engine changes/regression.
Quality check of 500 reports
INSURANCE COMPANY
USE CASE
One of the largest company in the world. 10% Regressions found.
4.1 to 4.2 Migration
LARGE OIL & GAS COMPANY
USE CASE
4.1 had been implemented for a while. Automated calculation engine changes was implemented. They found out that for years they were regressions in 3.1 that were fixed in 4.1.
3.1 to 4.2
HUGE TELECOM COMPANY
USE CASE
Organization checks regressions randomly, part of QA control. Regressions found between Dev / Prod due to a table change.
DEV and PROD
HEALTHCARE ORGANIZATION
12,000 Documents730 Users
70 Universes
DETAILED REGRESSION TESTING USE CASE
SAP BusinessObjects XI3.1 SP7to
SAP Business Intelligence 4.2 SP2
Federal Agency
1. RUN THE 360BIND EXPORTS & COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM DOCUMENTS
1. Run Exports and Comparisons: 1.5 hours
EXPORT 1
Run Exports and Comparisons 1.5 hours
Create an Export Task
Select Documents
Run the 360Bind Exports and Comparisons 1.5 hours
Add Documents
Run the 360Bind Exports and Comparisons 1.5 hours
Select Frequency
Run the 360Bind Exports and Comparisons 1.5 hours
Select an Export Categoryor Create a New one
Run the 360Bind Exports and Comparisons 1.5 hours
Populate the Prompts
and Schedule!
Run the 360Bind Exports and Comparisons 1.5 hours
EXPORT 2
Export TaskXI 3
Run the 360Bind Exports and Comparisons 1.5 hours
Select the first export task
Export it into a Zip file
Import TaskBI 4
Run the 360Bind Exports and Comparisons 1.5 hours
Import the task saved
and Run it
COMPARISON
Schedule Comparison Job
Run the 360Bind Exports and Comparisons 1.5 hours
Select Export Category
Run the 360Bind Exports and Comparisons 1.5 hours
Working folder you exported the reports
Select Export Results
Run the 360Bind Exports and Comparisons 1.5 hours
Select Comparison Options
Run the 360Bind Exports and Comparisons 1.5 hours
1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM DOCUMENTS
2. Identify all Kinds of Regressions: 1 hour
Data - Structure - Images - Style28/28 documents with regressions
Identify all Kinds of Regressions1 hour
1. Vertical Alignment & Font Families
Identify all Kinds of Regressions1 hour
XI 3.1 BI 4.2
Data - Structure - Images - Style24/28 documents with regressions
Identify all Kinds of Regressions1 hour
2. Image: Label font, Scales and Layout
Identify all Kinds of Regressions1 hour
XI 3.1 BI 4.2
2. Image: Label font, Scales and Layout
Identify all Kinds of Regressions1 hour
XI 3.1 BI 4.2
Data - Structure - Images - Style17/28 documents with regressions
Identify all Kinds of Regressions1 hour
3. Data: Missing Decimals (48.32 48.00)
Business Expenses Details (in %)
Percentage Total< 100%!!!
Identify all Kinds of Regressions1 hour
XI 3.1 BI 4.2
3. Data: Error DIV/0
Business Expenses Details (in %)
Identify all Kinds of Regressions1 hour
XI 3.1 BI 4.2
1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM DOCUMENTS
3. Analyze root cause of Data Regressions: 1 hour• Document: “State A14 - Expenses Details”• Variable Analyzed: “C Fch Ttc”
• Variable is using “Round()” function included in “FormatNumber()” function: FormatNumber(Round(…))
Analyze root cause of Data Regressions1 hour
Analyze root cause of Data Regressions1 hour
Working fine in XIR3.1 SP7 but no longer in BI 4.2 SP2 due to Calculation Engine Modification
1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM DOCUMENTS
4. Run impact analysis on all the Documents: 0.5 hour
Identify all documents using “FormatNumber(Round(…))” formulas in variables or cells
596 Documents probably impacted by the Regressions
IMPACT ANALYSIS
Run impact analysis on all the Documents0.5 hour
• Identify which one are really used:
262 Documents to Fix for the Migration
28 documents troubleshot between versions:
• Service proposal from a BI consulting company= 3 days to test the 28 documents with manual comparison
• 360Bind and 360Eyes:1. Run the 360Bind exports and comparisons : 1.5 hours2. Identify all kinds of regressions : 1 hour3. Analyze root cause of regressions : 1 hour4. Run impact analysis on all platform documents : 0.5 hour= 4 hours to identify all the 28 document regressions and the impact on the 12,000 platform documents
FIND CALCULATION ENGINE CHANGES
EASIER, FASTER, CHEAPER & SAFER MIGRATION
Backup, promote & version BO content
Schedule & burst your BO reports
Explore & analyze BO metadata and licenses
Automate report qualificationNon-regression testing
XIR2
XI3
BI4
Secure your BO deployment
Streamline and Extend SAP BusinessObjects
STREAMLINE SAP BUSINESSOBJECTS
ABOUT GB&SMITH
• 1,000,000 administrated users• Won EY entrepreneur of the
year and other famous prices• Continuously evolving
company and tools
• More than 500 customers• In more than 30 countries• Small to big deployment
(100-92000 users)
• 70 employees• Very reactive support team
(98% renewal rate)• 50% in R&D
THEY TRUST US
Take a look at our solutions on: www.youtube.com/360suite
Let’s work together
QUESTIONS?