Detector front-end electronics Eventbuilder network
Eventbuilder PCs (software LLT) Eventfilter Farm up to 4000 servers
Eventfilter Farm up to 4000 servers UX85B Point 8 surface subfarm
switch TFC 500 6 x 100 Gbit/s subfarm switch Online storage Clock
& fast commands 8800 Versatile Link 8800 Versatile Link
throttle from PCIe40 Clock & fast commands 6 x 100 Gbit/s ECS
Online upgrade status - Niko Neufeld2
Slide 3
Full event-building of every bunch-crossing (40 MHz) no
bottle-neck for event-building in the system DAQ, ECS and TFC
relying on the same universal hardware module PCIe40 New DAQ with
very challenging I/O in the event-builders and the network
Cost-effectiveness dictates very compact system concentrated in new
data-center, connected via long- distance links to the detector FEs
ECS and TFC continued smoothly from current system (modulo the
significant changes in the TFC due to the trigger-less nature of
the read-out) Online upgrade status - Niko Neufeld3
Slide 4
Most compact system achieved by locating all Online components
in a single location Power, space and cooling constraints allow
such an arrangement only on the surface: containerized data- centre
Versatile links connecting detector to readout-boards need to cover
300 m Online upgrade status - Niko Neufeld4
Slide 5
CPPM, Bologna, CERN (supporting)
Slide 6
PCIe40: the universal hardware module, a PCIe Gen3 x 16 card
with up to 48 optical transceivers TELL40: a PCIe40 with the DAQ
firmware with up to 48 GBT receivers SOL40: a PCIe40 with the
ECS/TFC firmware (replacing SPECS and various TFC modules) with 48
GBT transceivers
Slide 7
Online upgrade status - Niko Neufeld7 slides by JP.
Cachemiche
Slide 8
Online upgrade status - Niko Neufeld8
Slide 9
9
Slide 10
10 slide by F. Pisani
Slide 11
Online upgrade status - Niko Neufeld11
Slide 12
Bologna INFN and U., CNAF, CERN
Slide 13
Performances tests performed at CNAF with a test bed similar to
the CERN one Exploiting the best performances required some tuning
Bind processes according to NUMA topology and switch off power
saving modes Very close to saturation 52.5 Gbit/s!!! Online upgrade
status - Niko Neufeld13 A.Falabella et al
Slide 14
Extensive tests need to be done on a bigger cluster We aim at
the new CINECA Galileo TIER-1 cluster Possible to test on a scale
similar to the LHCb Upgraded DAQ network The cluster is in
production from the last week of January 2015 First tests in few
weeks managed by CNAF team ModelIBM NeXtScale Cluster Nodes516
Processor 2 8-core Intel Haswell 2.40GHz per node RAM 128 GB/node,
8 GB/core Network InfiniBand with 4x QDR switches Online upgrade
status - Niko Neufeld14
Slide 15
CERN (thanks to technical coordinatioe team for their
help!)
Slide 16
In real conditions First test loop-back of AMC40 Soon with
front-end prototype Watch out for bit-errors / verify optical
margin Use a MiniDAQ setup Online upgrade status - Niko
Neufeld16
Slide 17
PCIe40 Patch cord court MPO(2m-5m) Cble longue distance (300m)
MPO-MPO Adaptateur Cassette MPO vers 12x LC ou SC Cble longue
distance (300m) Patch panel rack en souterrain x12 Config 1: Config
2: (repartition de charges) Fan Out court (2m-5m) PCIe40 MPO-MPO
Adaptateur A B slide by L. Roy Online upgrade status - Niko
Neufeld17
Slide 18
Online upgrade status - Niko Neufeld18
Slide 19
Online upgrade status - Niko Neufeld19
Slide 20
no error (12 links for 4 weeks) over 700 m More links and
different tests to follow Online upgrade status - Niko
Neufeld20
Slide 21
Annecy, CPPM, CERN + sub-detector experts
Slide 22
TFC & ECS firmware (SOL40) DAQ firmware (for MiniDAQ and
PCIe40) Prototype ECS for MiniDAQ Global firmware framework
