Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
ATLASRucioDBcharacteristics(basedontheOracleRDBMS)
Ruciocommunityworkshop,Oslo,Feb-March2019GanchoDimitrov(CERN)
Aboutme
1
• MemberoftheATLASdatabasegroupsince2006
• Acquired(some)DBrelatedknowledgethroughoutthelast15+years
• Mainfocuson:Ø datamanagementØ databaseschemadesignØ databaseperformancetuning
• CertifiedinOracleRDBMS
• RegularspeakeratOracleUserGroupcommunityconferencesinUK,BulgariaandGermany
RucioCommunityWorkshop,Oslo,Feb2019
Rucioarchitecture
2RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
4
Designedin2013
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
5
61tables
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
6
74indices
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
7
5triggers
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
8
3functions
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
9
12procedures
RucioCommunityWorkshop,Oslo,Feb2019
Ruciodatabaseschema
10
17jobs
RucioCommunityWorkshop,Oslo,Feb2019
Tablesegments
11
Physicallyorganised basedonRucio’s scopeunit
scopedata=tablepartition**separatesegment(providesflexibilityinsetofoperations)
RucioCommunityWorkshop,Oslo,Feb2019
Rucioscopegeneration
12
In2014:5140scopes(systemstartup)
2015-2018=~400peryear
RucioCommunityWorkshop,Oslo,Feb2019
Currently
13
About7000scopesaremappedto34500table
partitions
RucioCommunityWorkshop,Oslo,Feb2019
DIDS(DataIDentifierS)*
14
1014million(AVGrowsize:220bytes)
*Theprimaryaddressableunitofdata(files,datasets,containers)
RucioCommunityWorkshop,Oslo,Feb2019
DIDSreplicasontheGrid
15
1106million(AVGrowsize:150bytes)
RucioCommunityWorkshop,Oslo,Feb2019
Rucio<-->DBinteraction
16
SQLcommandsgeneratedby
SQLAlchemy ORM*
*Object-RelationalMapperRucioCommunityWorkshop,Oslo,Feb2019
DBsessions
17
1100-1400DBsessions15-25activesessions
RucioCommunityWorkshop,Oslo,Feb2019
Rucioinsert/update/deleterate
18
~2-4Kexec/sec
RucioCommunityWorkshop,Oslo,Feb2019
RucioDBqueriesrate
19
~4-8Kexec/sec2Mblockreads(15GB)/sec
RucioCommunityWorkshop,Oslo,Feb2019
DBserverCPUusage
20
~30-40%
RucioCommunityWorkshop,Oslo,Feb2019
RucioDBvolumegrowth2016-2019
21RucioCommunityWorkshop,Oslo,Feb2019
April-June2019
22
Excitingperiod:
TestandvalidationonOracle18c
DBserverupgradefrom11gto18c
RucioCommunityWorkshop,Oslo,Feb2019
Spring2020
23
ExpectedtohavenewDBhardware
RucioCommunityWorkshop,Oslo,Feb2019
RucioDBschemaextension
24
EnrichRucioDIDSwithmetadatastoredinJSON*
*CERNcomputingseminaronDBbuilt-inJSONfeaturesplannedfor15th May2019
RucioCommunityWorkshop,Oslo,Feb2019
Take-homemessages• RucioDBsetuponOracleRDBMSkeepsonperformingwellseveralconsecutiveyears.•WorkforsmoothDBserverupgradeisnecessary• PlannedextensionofthecurrentDBschematosupportDIDSmetadatainJSON
SpecialthankstotheRucioteamforthecollaborativeworkintheyears.
WelltunedDB=goodnight’ssleepforall!25RucioCommunityWorkshop,Oslo,Feb2019