Upload
vodan
View
268
Download
1
Embed Size (px)
Citation preview
Introducing Pentaho reporting solutions for CERN’s administrative dataGabriele Thiede - CERN
06.03.2018 1
? !
What is CERN?
• CERN = "Conseil Européen pour la Recherche Nucléaire"
• Founded in 1954
• Situated in Switzerland, Geneva
• 12 founding Member States: Belgium, Denmark, France, the Federal Republic of Germany, Greece, Italy, the Netherlands, Norway, Sweden, Switzerland, the United Kingdom, and Yugoslavia
• Mandate:Establish a world-class fundamental physics research organization in Europe
CERN Member States – 64 years later
The Activities of CERN
• Probing the fundamental structure of the universe
• Study the basic constituents of matter - the fundamental particles -and the forces acting between them
• Constructing and running particle accelerators and detectors
How CERN is financed
• Budget: ca. 1 Mrd. CHF per year
• Calculation of contributions based on GDP (gross domestic product) of each Member State.
• Additional contributions from Associate Member States, Associate Member States in the pre-stage to Membership etc.
People at CERN
Total: 21.840
• Members of the personnel (17.540)o Employed members of the personnel (3.440)
o Associated members of the personnel (14.100)
• Contractor’s staff + Temporary Labour (4.300) 16%
64%
20%
Employed members of the personnel
Associated members of the personnel
Administrative data and reporting
Administrative Domains at CERN
9
The Classics
Human resources Finance Purchasing Logistics
Extras
Training Access Transportation Accomodations
Social security
Health insurance Pension Fund School Fees Unemployment benefits
Host states
Customs formalities Removals Licence plates
Work and residence
permits
06.03.2018
Infrastructure – current situation
3/7/2018 10
Infrastructure – coming soon
11
• Analyser
• Report Designer
• C-Tools
Pentaho
PDI
06.03.2018
Time Frame
12
• Start: 2016
• End: 2018
= 2,5 years for• Replacement of data warehouse solution
(responsible service: Infrastructure section)
• In parallel: Migration of existing reporting(responsible service: Reporting section)
06.03.2018
Migration of existing Reporting
13
• Ca. 380 existing Business Objects reports
• Covering all administrative domains
• From small & simple to huge & complex reports with over 80 tables and 20 graphs
• Self service reports, scheduled reports
• For internal and external clients
06.03.2018
Resources for migration
14
• 3 permanent staff (avg. 1,87 FTE/year)• Migration in addition to regular work
• Temporary help, short term contracts (5 persons = avg. 2,2 FTE/year)• Dedicated to migration, only
06.03.2018
Project Organisation
15
Inventory Migration Plan
• List of all used reports
• Main characteristics
• Frequency of usage
• Data sources
• Client(s)
• Priorities• Complexity
• Availability of DWH tables
• Preferred Pentaho solution
06.03.2018
Pentaho reporting solutions
16
• Pentaho Analyser• Predefined reports (scheduled + on demand)
• Analyser dashboards
• Ad-hoc analysis on cubes
• Report Designer (scheduled + on demand)
• CTools dashboards
06.03.2018
Requirements for report content
Provision of data in DWH
Implementation of new report
Organisation of work with clients
17
Analysis of needs
Inventory of BO reports
Discussion and identification of solutions
Investi-gation of solutions
TestImplementa
tionUseRelease
06.03.2018
Interim Balance
Where are we now?
19
• 4 temporary resources trained, 1 in training
• 17 dashboards developped
• Ca. 280 reports migrated
• Backlog: reports for training and recruitment
• Many new technical competencies acquired
06.03.2018
Our experience with Pentaho (1/3)
20
Pentaho Analyser for specialised users
Comfortable tool to establish ad-hoc data analysis
Not self explanatory, training necessaryOccasional users get lost and refer to reporting service
Consequences: Desired effect (more autonomous users) did not materialize
06.03.2018
Our experience with Pentaho (2/3)
21
Access right management of Analyser
Separate access rights to report folders and to cubes
Access right to corresponding cube necessary to executeAnalyser reportNo possibility to disable layout panel and list of available fields
Consequences: For users without access to cube ony Report Designer reports possible
06.03.2018
Our experience with Pentaho (3/3)
22
Pentaho‘s strenghts and weaknesses
High acceptance of end users:- Attractive reports and dashboards with Pentaho; Analyser
Developpers see room for improvement:- Report Designer cumbersome- Pentaho components patch work, missing integration- Poor scheduling and search functions and graphs
Consequences: no use of Interactive reports; inhouse development of scheduler and search tool; Excel workaround for graphs
06.03.2018
1st Lesson learned
23
Parallel work on infrastructure and reporting
User need drivenReporting serves as validation
Time consuming, delaysFixes of infrastructure impact reporting (rewrite of queries)
Recommendation: Infrastructure („data lake“ like) before reporting
06.03.2018
2nd Lesson learned
24
Temporary short term resources for reporting
Working on Pentaho is attractive, many interested candidates
Business knowledge (important but complex!) is lackingTraining adds workload to permanent staffKnowledge transfer at departure
Recommendation: avoid short term resources
06.03.2018
3rd Lesson learned
25
Version control and documentation
• Preserve and transfer of knowledge
• Coordinate team work
• Use of tools like GIT and Sourcetree, JIRA
06.03.2018
Conclusion
26
Migration aslearningprocess
Migration as chance
• Technical knowledge aboutPentaho
• Reporting techniques
• Project management
• Massive tidy up
• Synergy effects
• Standardisation and harmonisation
• Improvement of data and reporting quality
06.03.2018
Thank you for your attention