Regression Testing OBIEE without the fuss

  • View
    103

  • Download
    0

Embed Size (px)

DESCRIPTION

Just because OBIEE is presented to users through a web browser that doesn't mean the regression testing has to take place there too. This presentation will demonstrate how you can efficiently and effectively test the bulk of changes made to your system without going anywhere near the front end. Where front end testing is needing, it demonstrates the concept of visual regression testing using the latest tools. With hands-on examples of the tools involved this will be a practical presentation giving anyone interested in the topic a headstart in practical implementation. Video available here: http://ritt.md/regression-testing-video

Transcript

  • 1. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing OBIEE Without the Fuss Robin Moffatt Principal Consultant, Rittman Mead

2. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Smart Regression Testing Do better OBIEE development and deliver it more rapidly 3. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Why Regression Test? Confidence 4. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Why? Confidence Agility in development Happier users 5. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What? Regression Testing is not just a prolonged exercise on a Gantt chart, just before go live Smart Regression Testing can be run on demand, targeting the change made and assuring quality during development 6. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Detect Problems Early Fresh in the developers mindEasier to x ! Fewer surprises during big test cyclesMore honest & realistic project status reports ! Better overall QA for the systemHappier users 7. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Save Time Manually checking for regressionsslow & error-pronelimited test coverage boring! ! Automated regression testingfast & reliablemuch greater test coverage 8. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead OBIEE Conceptual Overview Analysis Analysis Data Sources Dashboard RPD BI Server -------------------- Sending query to database named orcl (id: ), connection pool named Connection Pool 01 ! select sum(T117.FCAST_COST_AMT) as c1, T107.PROD_TYPE_DESC as c2, T127.MONTH_YYYYMM as c4 from GCBC_SALES.TIMES T127 /* Dim_TIMES */ , [] 9. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What Could Possibly Go Wrong? DataAre the numbers still correct? FunctionalityDo dashboards still run without error?Do dashboards still render in the same way? SecurityDataVisible Dashboards 10. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops I broke it (1) - the RPD RPD may still pass consistency check - but do you know if you broke existing reports and dashboards? Cleanup / RefactoringDevelopmentsBugxes 11. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops I broke it (2) - OBIEE front end Cleanup / Refactored calculationsPresentation Catalog changesApplication Role changes 12. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops I broke it (3) - Database Refactored designsView -> TableTable -> MViewIndexesPartitioning 13. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What Makes a Good Regression Test? Easy Better tests Fast Flexible Targeted Scalable 14. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead The OBIEE stack Request Response BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] User WebLogic Server OBIEE system components Data Source(s) Network Network Not all components listed, eg cluster controller -------------------- SQL Request, logical request hash: 810510fe SET VARIABLE QUERY_SRC_CD='Report',SAW_SRC_PATH='/shared/Test/test1 jan/feb';SELECT 0 s_0, "Sales - Fact Sales"."Dim Products"."Product Type" s_1, "Sales - Fact Sales"."Dim Times"."Month" s_2, SORTKEY("Sales - Fact Sales"."Dim Times"."Month") s_3, "Sales - Fact Sales"."Fact Sales"."Cost" s_4 FROM "Sales - Fact Sales" C1 C2 C3 C4 ---------- -------------------- -------------------- ---------- 4636.93 Bread Assortments FEB-2000 200002 4583.53 Bread Assortments JAN-2000 200001 3445.16 Bread Clubs FEB-2000 200002 3542.49 Bread Clubs JAN-2000 200001 4735.63 Cold Drinks FEB-2000 200002 4565.21 Cold Drinks JAN-2000 200001 4604.19 Gifts & Baskets FEB-2000 200002 [] ------------------------------------------------------------------------------------------- s_0 s_1 s_2 s_3 s_4 ------------------------------------------------------------------------------------------- 0 Bread Assortments JAN-2000 200001 4583.53 0 Bread Clubs JAN-2000 200001 3542.49 0 Cold Drinks JAN-2000 200001 4565.21 0 Gifts & Baskets JAN-2000 200001 4608.01 0 Hot Drinks JAN-2000 200001 5768.62 [] Sending query to database named orcl (id: connection pool named Connection Pool 01 ! select sum(T117.FCAST_COST_AMT) as c1, T107.PROD_TYPE_DESC as c2, T127.MONTH_DESC as c3, T127.MONTH_YYYYMM as c4 from GCBC_SALES.TIMES T127 /* Dim_TIMES */ , GCBC_SALES.PRODUCTS T107 /* Dim_PRODUCTS */ , GCBC_SALES.SALES T117 /* Fact_SALES */ 15. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Where Can We Detect Problems? User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in Analysis request Logical SQL Physical Query x n Query results x n DatasetRendered Analysis RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] 2 3 45 1 Logical Physical Front End 4 1 3 5 2 16. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing Opportunities OBIEE Logical modelDoes the RPD still work and return the same results for a given logical query? Physical DatabaseDoes the Physical SQL that OBIEE generates still run? OBIEE front endFrom the point of view of an end user, does it look right? 17. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] PhysicalSQL Dataset Regression Testing Database Changes 18. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing Database Changes If only the DB has changed, then just test the DB. Dont complicate things by involving the stack further up. nqquery.log / Usage Tracking SQL*Plus So long as the query still runs, and returns the same data, youre guaranteed that nothing will change in OBIEE 19. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Logical OBIEE Regression Testing User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] Logical SQL Dataset 20. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Logical SQL in OBIEE Analysis Logical SQL Resultset via nqcmd Analysis results BI Server RPD 21. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 1. Tool automagically extracts Logical SQL for all analyses in targeted dashboards or analyses 22. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 2. Tools runs captured Logical SQL to generate the baseline data 23. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 3. Developers make their RPD changes and deploy 24. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 4. Tool runs regression test to check both data and Logical SQL match the baseline 25. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing the OBIEE Front End User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] HTML, JS, CSS, PNG 26. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing the OBIEE Front End Front End testing is complex!! Two options to make sense of HTML, JavaScript, CSS, images: Programatically - Parse server responseVisually - Render server response 27. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Front End test methods: Parsing vs Rendering ParsingMakes pass/fail tests easier-A string is present, or it is not! Higher rate of false negatives-Its easier to miss problems! High development & maintenance cost-Need to reverse-engineer DOM-Often relies on non-deterministic internal IDs to accurately pinpoint assertions 28. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Front End test methods: Parsing vs Rendering RenderingMore accurate-Can detect dierences not expected or explicitly coded for More prone to false positives-Lots of things change on a page for valid reasons (dates, alerts, etc) Still requires DOM knowledge but less reliance on internal IDs 29. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Visual Regression Testing Before After Test Result: Fail Suitable for testing front-end changes, such as:Refactoring Analysis formulae into the RPDPresentation Catalog restructuring 30. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead DEMO 31. www.r