Upload
eavan
View
85
Download
0
Tags:
Embed Size (px)
DESCRIPTION
ALFA - a common concurrency framework for ALICE and FAIR experiments. Mohammad Al-Turany GSI-IT/CERN- PH. ALFA. Common layer for parallel processing . Common algorithms for data processing. Common treatment of conditions database. Common deployment and monitoring infrastructure. - PowerPoint PPT Presentation
Citation preview
ALFA - a common concurrency framework for ALICE and FAIR
experiments
Mohammad Al-TuranyGSI-IT/CERN-PH
M. Al-Turany, CWG13 meeting 2
ALFA
April 25, 2014
• Common layer for parallel processing.
• Common algorithms for data processing.
• Common treatment of conditions database.
• Common deployment and monitoring infrastructure.
M. Al-Turany, CWG13 meeting 3
Current status of FairRoot
April 25, 2014
5/18/13 M. Al-Turany, ACAT 2013, BeijingFlorian Uhlig ROOT Users Workshop, Saas Fee
Root
TEve
ROOT
IO
TGeo
TVirt
ualM
CCint
TTre
e …
Proo
f
Gean
t3
Gean
t4Ge
nat4
_VM
C
Libraries
…
VGM
FairRoot
…Run
Man
ager
IO
Man
ager
Runt
ime
DB DB
Inte
rface
Even
t Di
spla
y
MC
Appl
icatio
n
Mod
ule
Dete
ctor
Task
Mag
netic
Fie
ld …
Even
t Ge
nera
tor
CbmRoot
PandaRoot AsyEosRoot
R3BRoot SofiaRoot MPDRoot
FopiRoot EICRoot
M. Al-Turany, CWG13 meeting 4
What we went to do with FairRoot
April 25, 2014
ROOT
Gean
t3
Gean
t4Ge
nat4
_VM
C
Libraries
…VG
M
Runt
ime
DB
Mod
ule
Dete
ctor
Mag
netic
Fie
ld
Even
t Ge
nera
tor
CbmRoot
PandaRoot
AsyEosRoot
R3BRoot SofiaRoot
MPDRoot
FopiRoot EICRoot
MC
Appl
icatio
n
Tran
spor
t (F
airM
Q)
Build
ing
confi
gura
ion
Test
ing
Fair
DB
CMak
eZe
roM
Q
M. Al-Turany, CWG13 meeting 5
How is it with ALFA and FairRoot?
April 25, 2014
ROOT
Gean
t3
Gean
t4Ge
nat4
_VM
C
Libraries and Tools
…
VGM
Runt
ime
DB
Mod
ule
Dete
ctor
Mag
netic
Fie
ld
Even
t Ge
nera
tor
MC
Appl
icatio
n
Fair
MQ
Build
ing
confi
gura
ion
Test
ing
Fair
DBCM
ake
Zero
MQ
DDS
BOOS
T
Prot
ocol
Bu
ffers
FairRoot
ALFA
CbmRoot
PandaRoot
AsyEosRoot
R3BRoot SofiaRoot
MPDRoot
FopiRoot EICRootAliRoot6 (O2)
????
6
Current Status• The Framework delivers some components which can
be connected to each other in order to construct a processing pipeline(s).
• All components share a common base called Device• Devices are grouped by three categories:– Source:
• Data Readers (Simulated, raw)
–Message-based Processor: • Sink, Splitter, Merger, Buffer, Proxy
– Content-based Processor: • Processor
M. Al-Turany, CWG13 meeting April 25, 2014
M. Al-Turany, CWG13 meeting 7
The Dynamic Deployment System (DDS) Should:
• Deploy task or set of tasks• Use (utilize) any RMS (Slurm, Grid Engine, … ),• Secure execution of nodes (watchdog),• Support different topologies and task
dependencies• Support a central log engine• ….
April 25, 2014
See Talk by Anar Manafov on Alice Offline week (March 2014)https://indico.cern.ch/event/305441/
M. Al-Turany, CWG13 meeting 8
Test Cluster
• 8 machines from DAQ lab with SLC6-64– aidrefma01-08
• FairRoot + FairSoft are installed – /local/home/cwg13/install/FairRoot
• Tutorial3 can be used as a demonstrator
April 25, 2014
M. Al-Turany, CWG13 meeting 9
Some remarks
• Mixture of SL6.5 and SL6.4– Binary compatible but I had to switch off the test
in config.sh• /etc/issue is not available which also require
a manual change for FairRoot
• Is there any way to export the display from these machines?
April 25, 2014
M. Al-Turany, CWG13 meeting 10
First test
• Two Samplers – aidrefma03 and aidrefma08
• Six Processors – aidrefma04 and aidrefma05
• One File writer (sink)– aidrefma02
April 25, 2014
M. Al-Turany, CWG13 meeting 11
Processor1startTestDectorProcessor1.sh
ID=“301"processorTask="FairTestDetectorMQRecoTask"numIoThreads="1"inputSocketType="pull"inputRcvBufSize=$buffSizeinputMethod="connect"inputAddress="tcp://aidrefma08:5565"outputSocketType="push"outputSndBufSize=$buffSizeoutputMethod="connect"outputAddress="tcp://aidrefma02:5568"
/local/home/cwg13/install/FairRoot/build/bin/testDetectorProcessor$dataFormat $ID $processorTask $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress $outputSocketType $outputSndBufSize $outputMethod $outputAddress
April 25, 2014
M. Al-Turany, CWG13 meeting 12
Processor2startTestDectorProcessor2.sh
ID="401"processorTask="FairTestDetectorMQRecoTask"numIoThreads="1” // How many IO threads ZeroMQ should use.
//it is relevant only after 1 GB/s per output.inputSocketType="pull"inputRcvBufSize=$buffSize // Output Buffer size (in # of messages for ZeroMQ, in bytes for nanomsg).inputMethod="connect"inputAddress="tcp://aidrefma08:5565"outputSocketType="push"outputSndBufSize=$buffSize outputMethod="connect"outputAddress="tcp://aidrefma02:5568”/local/home/cwg13/install/FairRoot/build/bin/testDetectorProcessor$dataFormat $ID $processorTask $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress $outputSocketType $outputSndBufSize $outputMethod $outputAddress
April 25, 2014
M. Al-Turany, CWG13 meeting 13
SamplerstartTestDetectorSampler.sh
ID="101"inputFile=”~/install/FairRoot/example/…/data/testdigi_TGeant3.root"parameterFile=“~/install/FairRoot/example/…/data/testparams_TGeant3.root"branch="FairTestDetectorDigi"eventRate="0"numIoThreads="1"outputSocketType="push"outputBufSize=$buffSizeoutputMethod="bind"outputAddress="tcp://*:5565"
/local/home/cwg13/install/FairRoot/build/bin/testDetectorSampler$dataFormat $ID $inputFile $parameterFile $branch $eventRate $numIoThreads $outputSocketType $outputBufSize $outputMethod $outputAddress
April 25, 2014
M. Al-Turany, CWG13 meeting 14
FileSinkstartFileSink.sh
ID="601"numIoThreads="1"inputSocketType="pull"inputRcvBufSize=$buffSizeinputMethod=“bind"inputAddress="tcp://*:5568"
/local/home/cwg13/install/FairRoot/build/bin/fileSin$dataFormat $ID $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress
April 25, 2014
M. Al-Turany, CWG13 meeting 15
The Test
April 25, 2014
Sampler
Sampler
Processor1
Processor1
Processor1
FileSink
April 25, 2014 M. Al-Turany, CWG13 meeting 16
Digits Hits
TClonesArray
Payload Payload
TClonesArray
Payload Payload
TClonesArray TClonesArray
Sampler
Processor
Sink
From digits to hits with :
M. Al-Turany, CWG13 meeting 17
Example: Sampler
April 25, 2014
M. Al-Turany, CWG13 meeting 18
Example: Processor
April 25, 2014
M. Al-Turany, CWG13 meeting 19
Example: FileSink
April 25, 2014
M. Al-Turany, CWG13 meeting 20
The Test
April 25, 2014
Sampler
Sampler
Processor1
Processor1
Processor1
Processor1
Processor1
FileSink
Processor1
The processors are “connecting” on both input and output, thusThey can be added (removed)On the fly
M. Al-Turany, CWG13 meeting 21April 25, 2014
M. Al-Turany, CWG13 meeting 22
FileSink
April 25, 2014