Embed Size (px)
Citation preview
Agile Testing of Business Intelligence
‘Cinderella 2.0’
Armando Dörsek (Verified) &
Iris Groenewoudt (Ordina)
Nordic Testing Days
6/6/2013
| 6
Programme
About … Us
The Customer
Background Information Business Intelligence
Agile vs Waterfall Projects
Customer Organization
Issues & Lessons Learned
| 7
About us
Armando Dörsek Verified Testing Services
Sr Test manager
15 years experience (finance, retail, government)
Iris Groenewoudt Ordina
Sr BI consultant
15 years experience (finance, travel industry, care)
7
| 8
About the case
Recent customer
Dutch Top 3 Health Insurance Co.
3 brand names, 2.1 mio customers (2011)
In : EUR 4.998.132.000
Out : EUR 4.684.321.000
Strategic Goal: from Insurance Co. to “Health Organization”
Corporate wide DWH & BI Project
8
| 9
DWH & BI
Background Information
| 10
Definition of DW/BI
A data warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management's decision making process.
Business intelligence usually refers to the information that is available for the enterprise to make decisions on.
| 11
Voorbeeld
Note: no actual production data.
| 12
Source Systems
Enterprise Datawarehouse
Staging Area
Data Marts
Semantic Layer
Reports, Dashboards, Analyses
CRM Call C. Backoff. Etc.
Domain Dept.
DW/BI Pyramid
Sources
Customer Situation
Report
Report
SQL:
Select
* from
SQL:
Select
* from
OHI BI OHI BO DWH DB
Schemas
Proprietary
Report Report
Integrated ERP, coming with a solution for Business Intelligence. Extended with
bespoke DWH to enable answering the necessary questions
Universe
Universe
Files
Sources
DW & BI Testing
Report
Report
Universe
SQL:
Select
* from
SQL:
Select
* from
OHI BI
Universe
Validating ETL processes with Test Cases, checking totals etc.
Testing end user products: right, in time, usable etc.
OHI BO DWH DB
Schemas
Auditing Files Files
| 15
Waterfall vs Agile
Background Information
| 17
Waterfall vs Agile
Waterfall Project Agile Project
| 18
Agile Manifesto
Individuals and interactions
… over …
processes and tools
Working software comprehensive documentation
Customer collaboration contract negotiation
Responding to change following a plan
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more.
| 19
Scrum
Three Roles
Four Ceremonies
Three Artifacts
| 26
User Stories
"As a <role>,
I want <goal/desire>
so that <benefit>"
| 27
Planning Poker
| 28
Board
Story Todo Thinking Doing Verify Done
| 32
Burndown Chart
Bron: http://www.brightgreenprojects.com/
| 33
The “Mango” Project
Agile Testing in DW & BI
| 34
Agile (Scrum) Datawarehousing
Project Challenge #1 ‘1001 Cinderellas want to go to the ball…’
| 37
Agile (Scrum) Datawarehousing
Project Challenge #1 ‘1001 Cinderellas want to go to the ball…’
How to prioritize the Product Backlog with so many clients?
PIT
| 38
And then…
| 39
Day after day, sprint after sprint
| 40
Source Systems
Enterprise Datawarehouse
Staging Area
Data Marts
Semantic Layer
Reports, Dashboards, Analyses
CRM Call C. Backoff. Etc.
Domain Dept.
DW/BI Pyramid
Right, in time and complete?
| 42
Product Risk Analysis (PRA)
Waterfall Projects Agile Projects
| 46
The Team is Responsible
| 47
Everyone responsible for everything?
| 48
Distributing Testing Activities
ETL & Database
Semantic Layer
Reports
UT, ST & UAT
Approach/Intensity
| 51
Testing: Roles, Tasks, Activities
Testing is a Role Defining the Testing Strategy
Compare Architect and Project Management Roles
System Testers
Professional Testers for Test Specification and Execution
Testing the Integrated Product
Acceptance Testing
Bridging the Gap, together with Information Analists
Testing is a Task, too: Everyone tests
Whoever can help out, given the priority
Testing Activities are Distributed According to Matrix: Unittest through Acceptancetest (proof)
Regression Testing
Need for clear
Unit Testing instructions
“Typical” Testers’ Activities
| 52
The Verify Column
| 53
Issues
| 54
Issues
Heartbeats Timing issues because of
Surroundings (release schedules of source systems)
Development User Stories in Teams
Developing vs Testing
| 55
Issues (continued)
Complexity of DWH Dependencies regarding Meta Model, EDW Data Model
Delivering Value vs Preparing for the Future
Not unique for
Agile BI?
| 56
Issues (continued)
Test Base Documentation Created during the Sprint (not beforehand as with
Waterfall)
Misinterpretation of “Working Software over Comprehensive Documentation” is easily done
Creating Testcases proves Difficult without HQ Test Base
Definition of Done
User Stories
Not unique for
Agile BI?
| 57
Acceptance Absence of End Users in the Sprint Team
Absence of Maintenance Team in the Sprint Team
Issues (continued)
| 59
Issue: Functional vs Technical
Functional Stories
Direct End User Orientation
Technical Stories
Technical Debt
Architecture
| 60
Issues (continued)
Test Data and -Environment Slow, partially due to using Production Volumes
Short Time for Integration Tests
High Frequency of Regression Tests
Who’s in Charge (when it comes to it?)
Testing End User Functionality SMART Criteria needed
Can sometimes tested late in the Sprint only
Data Models > Universes > Reports
Demo’s with Production-Like Data are deemed essential to End Users
| 66
Lessons Learned
| 67
Lessons Learned
Agile Scrum: Delivers Most urgent products first, not state-of-art
Meanwhile: think of the near future
Sr End User vs PIT Business Intelligence = Company Wide
All Customer Groups need to be involved
and decide “who’s first”
Final Acceptance can become an issue End users need to be involved throughout the sprint
This can be tough (day to day vs project activities)
| 68
Lessons Learned
Functional Maintenance Can become an issue in several ways
Functional maintenance team needs to be involved
This can become an issue (day to day vs project)
Maintenance by Sprint Team needs to be avoided
It’s about People Need for pros, experienced people
Need for agility
It’s about standards Agility vs Rigidity
Experienced people need standards to adhere to
| 69
Lessons Learned
Technology Learn to use tools in use by the team to “build in quality”
Regression testing using tools Start early, or else …
Test Data Management Start early: it pays off for all
| 70
Questions?
+31623054054
@adorsek
