Upload
kalkin
View
30
Download
0
Embed Size (px)
DESCRIPTION
ALICE DAQ future detector readout. Filippo Costa ALICE DAQ. October 29, 2012 CERN. Outline. Current status and evolution LS2 upgrade Architectural issues Development process Present R&D activities. Trigger – DAQ – HLT. PDS. Rare/All. CTP. L0, L1a, L2. BUSY. BUSY. LTU. LTU. DDL - PowerPoint PPT Presentation
Citation preview
Filippo CostaALICE DAQ
ALICE DAQfuture
detectorreadout
October 29, 2012 CERN
Filippo Costa, CERN 2
• Current status and
evolution
• LS2 upgrade
• Architectural issues
• Development process
• Present R&D activities
29/10/2012
Outline
Filippo Costa, CERN 329/10/2012
Trigger – DAQ – HLT
GDC TDSM
CTP
LTU
TTC
FERO FERO
LTU
TTC
FERO FERO
LDCLDC
BUSY BUSY
Rare/All
Event Fragment
Sub-event
Event
File
Storage Network (8GB/s)PDS
L0, L1a, L2
L0, L1a, L2
360 DDLs
D-RORCD-RORC
EDM
LDC
D-RORC D-RORC
Load Bal. LDC LDC
D-RORC D-RORC
HLT Farm
FEPFEP
DDL
H-RORC
10 DDLs
10 D-RORC
10 HLT LDC
120 DDLs
DADQM
DSS
Event Building Network (20 GB/s)
430 D-RORC
175 Detector LDC
75 GDC30 TDSM
18 DSS60 DA/DQM
75 TDS
Archiving on Tapein the ComputingCentre (Meyrin)
Filippo Costa, CERN 429/10/2012
DDL current status (Oct ‘12)
DETECTOR # DAQ DDL # HLT DDLACORDE 1 0
EMCAL 24 24
FMD 3 3
HMPID 14 0
MTRK 20 20
MUON-TRG 2 2
PHOS 12 12
PMD 6 0
SDD 24 24
SPD 20 20
SSD 16 16
T0 1 1
TOF 72 0
TPC 216 216
TRD 13 13
V0 1 1
ZDC 1 1
TOT DETECTORs
17
TOT # DAQ DDLs
485
TOT # HLT DDLs
353
Filippo Costa, CERN 529/10/2012
LS2 Online Upgrade2x10 or 40 Gb/sFLP
DAQ and HLT10 or 40
Gb/s
FLPEPN
FLP
ITS
TRD
Muon
FTP
L0L1
FLPEMCal
EPN
FLPTPC
DataStorage
FLP
FLPTOF
FLP
FarmNetworkPHOS
Trigger Detectors
~ 2500 DDL3s10 Gb/s
L0
DataStorage
EPN
EPN
StorageNetwork
RORC3
RORC3
RORC3
RORC3
RORC3
RORC3
RORC3
RORC3
∞CLK
Filippo Costa, CERN 609/11/2012
LS2 upgrade requirements (LoI)
Detector MAX R/O rate (kHz)
(pp and Pb-Pb)
Event Size Pb-Pb after ZS (MB)
ITS Continuous 0.2
TPC Continuous 1.0
TOF 200 – 400
TRD 27 – 100 0.2
EMCal 50
Muon 5The rate for heavy-ion events handled by the online systems up to permanent data storage should be increased up to 50 kHz (with a safety factor of 2) corresponding to roughly two orders of magnitude, compared to the present system.
Data compression will reduce the input peak data throughput of 1 TByte/s to an average recorded data output of 80 GB/s to a local data storage and 12 GB/s to the computing center.
Filippo Costa, CERN 709/11/2012
Data compression 1Cluster finder (RORC3)
TriggerLevel 0,1
TriggerLevel 2
Event-Building Network
Detector
Digitizers
Front-end Pipeline/Buffer
Readout Buffer
Sub-event BufferFirst-Level Processor (FLP)
Data compression 2Event-Building
Event-building andProcessing Node (EPN)
Decision
Detector Data Link (DDL3)
Trigger & DAQ logical model
Decision
LHC Clock
Distribution of functionsas presented in the LoI
FastTrigger
Processor
DetectorElectronics
Online System
LHC Clock
Filippo Costa, CERN 809/11/2012
TriggerLevel 0,1
TriggerLevel 2
Event-Building Network
Detector
Digitizers
Front-end Pipeline/Buffer
Readout Buffer
Sub-event BufferFirst-Level Processor (FLP)
Event-Building Data compression 2
(EPN)
Decision
Detector Data Link (DDL3)
Trigger & DAQ logical model
Data compression 1Cluster finder
Decision
Alternatives scenarios exist:e.g. cluster finder as part ofthe detector readout
LHC Clock LHC Clock
9
Initial requirements in the LoI. Carry on with functional requirements and R&D in
parallel Refine detectors functional requirements for DCS,
TRG, DAQ in view of the detector TDRs (2013) Online R&D to develop prototypes in view of
Online/Offline TDR (2014)o DDL2:
• Prototype characterization• Production for the TRD and the HLT
o DDL3• Technology selection
o Online dataflow demonstrator• Detector readout, FLP, network, EPN
Development process
09/11/2012
Filippo Costa, CERN
Filippo Costa, CERN 10
Higher reliabilityReduced Support
Issues well defined
Common hardware
and protocol
29/10/2012
Benefit of the commonality
So far we used a common protocol and hardware (SIU – DDL – RORC) for the read-out of all detectors.Lots of benefits:• having the same data transmission
protocol for all the detectors reduces specific debugging sessions.
• It encourages knowledge sharing between the detectors.
• It creates the “standard” in a custom protocol.
We are planning to follow the same idea in the future upgrades, using a common protocol for sending data to the DAQ system.
Filippo Costa, CERN 1129/10/2012
DDL SIU evolution
SIU1 SIU2 SIU3
1 ch @ 2 Gb/sACTEL FPGA
(CORE cost 560 CHF)
6 Gb/sXILINX / ALTERA /ACTEL
FPGA(CORE cost 0 CHF)
10 Gb/sXILINX / ALTERA / ACTEL
FPGA(CORE cost 0 CHF)
Custom DDL protocol Custom DDL protocol(same protocol but faster)
• Custom DDL 10 Gb/s• Ethernet @ 10 Gb/s• PCIe over fibre
RUN1LS1
RUN2LS2
RUN3
Det. Read-Out FPGA
SIU IP CORE
Det. Read-Out FPGA
SIU IP CORE
Filippo Costa, CERN 1229/10/2012
RORC evolution
RORC1 RORC2 (aka C-RORC) RORC3
TBD2 ch @ 2 Gb/s
PCIe gen.1 x4 (1 GB/s)ALTERA FPGA
12 ch @ 6 Gb/sPCIe gen.2 x8 (4 GB/s)
XILINX FPGA
12 ch @ 10 Gb/sPCIe gen.3
ALTERA / XILINX
Custom DDL protocol Custom DDL protocol(same protocol but faster)
• Custom DDL 10 Gb/s• Ethernet @ 10 Gb/s• PCIe over fibre
RUN1LS1
RUN2LS2
RUN3
Filippo Costa, CERN 13
Run 2 - DDL1 &
DDL2
Run 1 - DDL1
29/10/2012
Speed of the link
backward compatible
Same DDL protocol
No need to change the
readout hardware if not needed.
Transition DDL1 to DDL2
+
Filippo Costa, CERN 1429/10/2012
R&D DDL3
Evaluationprocess
DDL custom protocol
UDP Ethernet 10 Gb/s
PCIe over fibre
Other good options
DDL3
Data transmission protocols are under evaluation, for the time being no final decision has been taken yet.Each protocol has different pros and cons, tests started already now, soon to come a reasonable decision.
Filippo Costa, CERN 1529/10/2012
UDP Ethernet 10Gb
The evaluation for UDP Ethernet data transmission protocol has already started.Preliminary tests have been performed together with RD51 collaboration(Hans MULLER, Alfonso TARAZONA MARTINEZ).A test system has been prepared:
• 1 SRU board with a VIRTEX6 , 10 Gb IP OPENCORE.
• 1 Machine (DELL server power Edge r 720) with 10 Gb/s port embedded in the motherboard.
Continuous readout, no external trigger system, no timeout between 2 packets.
Filippo Costa, CERN 1629/10/2012
UDP Ethernet
10 Gb/s embedded in the motherboard
XILINX VIRTEX 6
DDL optical fibre
SFP+ 10 Gb/s
Filippo Costa, CERN 1729/10/2012
UDP Ethernet
0 1 2 3 4 5 6 7 8 90
200
400
600
800
1000
1200
1400
Data throughput vs. packet size
packet size (kB)
PEA
K d
ata
thro
ughput
(MB
/s)
0 1 2 3 4 5 6 7 8 90
50
100
150
200
250
300
350
400
Acquisition rate vs. packet size
packet size (kB)
Acquis
itio
n r
ate
(kH
z)
100 kHz
Filippo Costa, CERN 1829/10/2012
UDP Ethernet (pro/con)
Easy to implement in FPGA (IP cores available for 10 Gb/s).Fast and light protocol (no overhead for handshaking).Allow different configurations, point to point or network with routers.Reduce the hardware needed by the detector team to build test system, they can test their readout system using a standard Ethernet port of the PC.
Is it rad tool ? We (ALICE DAQ+RD51) are evaluating different solutions.SmartFusion2: XGXS/XAUI Extension (to implement a 10 Gbps (XGMII) Ethernet PHY interface)
Not reliable (but software checks can increase the reliability, backpressure algorithm implemented in UDP DATE).High CPU consuming, moving data from the Ethernet port to the memory, but different companies are already addressing the issue, PLDA and Solarflare.IP core license costs for 10 Gb/s can be expensive and not portable outside CERN, but OPENCORE can be a solution for that.
Filippo Costa, CERN 19
Det. readout electronics
29/10/2012
TTCrq/rx
All the detectors in ALICE receive the trigger messages through the TTCrx chip or the TTCrq board.Some of them will integrate the functionalities in the readout electronics.The code will be implemented in the FPGA, so there is no need anymore of the TTCrq board or TTCrx chip to be installed on each readout card, removing the problem of spare components.
TTC VHDL
FPGA
Filippo Costa, CERN 2029/10/2012
Radiation, how to beat it
Use of radiation tolerant FPGA:ACTEL (www.actel.com).
Using partial reconfiguration of FPGA reloading part of the firmware when:• SEU is detected, • periodically
CSABA SOOS “SEU effects in FPGA How to deal with them?”http://www.google.it/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&sqi=2&ved=0CEoQFjAD&url=http%3A%2F%2Findico.cern.ch%2FgetFile.py%2Faccess%3FcontribId%3D8%26resId%3D2%26materialId%3Dslides%26confId%3D56796&ei=zeeIUOzUFtCQswaRyIFQ&usg=AFQjCNFjtJxDqKSGVzYv6pD-x-2b4yfD4w
Thanks!