View
219
Download
1
Tags:
Embed Size (px)
Citation preview
T2K Time Projection Chambers Front-end Electronics –
Experience Return
P. Baron, H. Bervas, D. Besin, D. Calvet, T. Chaleil, C. Coquelet, X. de la Broïse, E. Delagnes, F. Druillole, A. Le Coguie, E. Monmarthe, J-M. Reymond,
E. Zonca
DSM/IRFU, CEA Saclay, France
O. Ballester,
IFAE, Barcelona, Spain
[email protected] 2 Saclay 2-3 December 2009
Plan
Presentation of the T2K experiment
Architecture of TPC readout electronics
Principal components
Current status
Experience return
[email protected] 3 Saclay 2-3 December 2009
Tokai to Kamioka (T2K) experiment
Main Physics Goal: neutrino oscillation
• μ disappearance for improved accuracy on 23
• e appearance to improve sensitivity to 13
50 kT water
[email protected] 4 Saclay 2-3 December 2009
T2K Time-Projection Chambers
TPC features
• Resolution goal: 10% for p < 1 GeV/c
• Double wall design; inner wall is the field cage
• Total ~9 m2 instrumented with 72 Micromegas amplification modules segmented in ~7 mm x ~10 mm pads
• Custom readout electronics
1 of 3 TPCs shown
1728 pad (36 x 48)Micromegas module
34 cm x 36 cm
CalibrationGas system Electronics; on-line software
1 m
2 m
[email protected] 5 Saclay 2-3 December 2009
TPC Readout Challenges
Environment
• Embedded front-end: limited access, need low power
• No radiation, low magnetic field (0.2 T), Japanese underground facility
Highly segmented detector
• 124.000 channels over 9 m2 of instrumented area
→ Massive replication of identical modular building blocks
Extreme burstiness of data
• 124.000 channels; sampling: 12-bit 33 MHz → 50 Tbps
→ beyond capability of fully digital solution for power budget target
Large raw event size but modest average dataflow• 50 Tbps during ~15 µs drift time → 90 MBytes
• Allowable event size for storage: ~250 KBytes
→ data reduction of ~400 to achieve in real time• Spill repetition period: ~3.5 s; Cosmic calibration: 20 Hz max.
• DAQ average rate: ~250 KBytes × 20 Hz = 5 MByte/s
→ commercial computers and networking techniques
[email protected] 6 Saclay 2-3 December 2009
Plan
Presentation of the T2K experiment
Architecture of TPC readout electronics
Principal components
Current status
Experience return
[email protected] 7 Saclay 2-3 December 2009
Logical Read-Out Flow
Architecture principles
• Custom front-end ASIC; analog memory (Switch Capacitor Array)
• ADC + digital buffer mounted close to the detector
• Multiple optical fibers send data to off-detector concentrator cards
• Interface to common DAQ via standard gigabit Ethernet network
Pre-amp and shapers
Samplers and analog memory buffers
Analog to digital conversion
Digital buffer – zero-suppression
Data concentrator Clock/Trigger fanout
~124.000 channels
1728 ASICs
On
-det
ecto
r
72 Optical fibers
~4 Tbaud*/s peak*1 baud = 12 bit
~2 ms retention max.
34 Gbaud/s peak
400 Gbit/s peak
~1-10 Gbit/s averaged
Shared DAQ~0.1-1 Gbit/s
432 FrontEnd Cards
18 DataConcentrator Cards
72 Front-EndMezzanines cards
Off
-det
ecto
r
Global Clock/TriggerGigabit Ethernet
[email protected] 8 Saclay 2-3 December 2009
Architecture Implementation3 TPCs
1 m
2,5
m
2,5 m
Network
TCP/IP
PC Linux
DCCs
PrivateEthernet
GigabitEthernet
DAQ control
DetectorB
DetectorA
Global trigger
1 of 6 TPC end-plates(12-modules)
Outside magnetInside magnet
2 of 12 (or 18) DataConcentrator
Cardsx 6
12 duplexOptical fibres
1 of 72 modules
Front End MezzanineCard (FEM)
288 channelFront End Card (FEC)
1728 pad Micromegas plane
Slow controlnetwork
Optical fiberto/from DCC
Low voltagePower supply
1 of 1728 Front-End ASIC “AFTER”
72 channel x 511 time bucketsSwitched capacitor array
[email protected] 9 Saclay 2-3 December 2009
Plan
Presentation of the T2K experiment
Architecture of TPC readout electronics
Principal components
Current status
Experience return
[email protected] 10 Saclay 2-3 December 2009
ASIC “AFTER” Functional Diagram
AFTER
511 cells
SCAFILTER
100ns<tpeak<2us
CSA
1 channel
x72(76)
76 to 1BUFFER
SCA MANAGERSLOW CONTROL
Serial Interface
W / R
Mode
CK
CK
ADC
TEST
In Test
120fC<Cf<600fC
Power Supply Reference Voltage Reference Current
Asic Spy Mode
CSA;CR;SCAin (N°1)
Power
On Rese
t
Design features
• 72 channels x 511 analog memory cells; Fwrite: 1-50 MHz; Fread: 20 MHz
• 4 Charge Ranges (120 fC; 240 fC; 360 fC & 600 fC)
• Supports positive or negative input signals
• 16 Peaking Time Values (100 ns to 2 µs)
Asic
For
Tpc
Electronic
Read-out
[email protected] 11 Saclay 2-3 December 2009
The AFTER chipDesign facts
• 0.35 µm CMOS - 500.000 transistors
• 7.8 x 7.4 mm die – 160-pin 0.65 mm pitch LQFP
• single 3.3V supply - 8 mW/channel
Performance
• Integral non-linearity < 1.2% full range; ENC: 350 electrons (no load)
• Stored charge degradation after 2 ms retention: 0.18 LSB
→ All specifications met on first silicon!
AFTER chip used to readout TPC Micromegas and FGD Silicon PM
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 15000
DAC bin
Inte
gral
Non
_Lin
earit
y =
(dat
a - f
it)/fi
tmax
(%)
100ns
2µs
[email protected] 12 Saclay 2-3 December 2009
AFTER chip test and production
Production of the AFTER chip
• 5300 chip produced; 4750 OK; Yield: 89%
• 1800 chips delivered to T2K (TPC + FGD + monitoring chamber)
LABVIEW Test Software
Protection 1 Protection 2 No Protection
AFTER Test Card
Test bench
• AFTER test card served as pre-prototype of Front-End Card
• Xilinx Virtex 2 Pro eval. kit as pre-prototype of FEM board for readout
• DAQ and Analysis with Ethernet PC and LabView
[email protected] 13 Saclay 2-3 December 2009
Front-End Card (FEC)
Features
• All design concepts validated on AFTER test board
• 1 FEC reads out 288 channels – 6 layer PCB
• Consumption: 1 A – 4 W
Quad-channel12-bit ADC AD9229
Passive protection Circuits
4 AFTER chips
3.3V regulatorsSilicon ID chipV - I –T monitor
+4V input
Four dual row 80-pin 1.27 mm pitch connectors to detector
PulserClock fanout
PhotoMos relay
80-pin connector to FEM
[email protected] 14 Saclay 2-3 December 2009
Main functions
• Drive 6 FECs and aggregate data produced (1728 channels, 5.7 Gbps)
• Buffer one event (raw data), i.e. ~10 Mbit
• Deliver data to DCC upon request: raw data or zero-suppressed (one programmable threshold per channel)
• Configuration and slow control, voltage, current, temperature monitoring
Front-end Mezzanine (FEM) card
ZBT SRAM2 Gbps optical transceiver
CANbus slow control (RJ45)
FPGA
µC
FEC#1
Voltage regulators
Voltage regulators JTAGPower In
ConnectorFEC#0Buffers FEC#2
FEC#3 FEC#4 FEC#5
[email protected] 15 Saclay 2-3 December 2009
Detector Module Read-out Electronics
72-channel ASIC
Quad-channel ADC
digital Front-endMezzanine card (FEM)
Optical Transceiver
FPGA
80-pin connector
288-channel analogFront-End Card (FEC)
1728-pad detector plane
Slow-controlNetwork - CANbus
Fiber toDCC
Low voltagepower
Materializing the concept
• 3 year development of ~6 equivalent persons from ASIC specifications to working readout of one detector module
[email protected] 16 Saclay 2-3 December 2009
Plan
Presentation of the T2K experiment
Architecture of TPC readout electronics
Principal components
Current status
Experience return
[email protected] 18 Saclay 2-3 December 2009
Integration of TPC#0 and #1 at TokaiPhotos: Claudio Giganti
[email protected] 19 Saclay 2-3 December 2009
Latest news from T2K nd280 m
TPC #2
• Tested at Triumf and now in the clean room at Tokai
• Installation in pit on Dec’ 17th
→ Fully commissioned TPCs by mid-Jan’10 for closing magnet
TPC #0 and TPC #1
• Installed in the magnet; DAQ commissioning in progress
• Awaiting final gas mixture to power central cathode and detector HV
→ Detector commissioning starts Dec. 7th
The nd280 collaboration announced the observation of the first beam neutrino candidate (in the INGRID detector) two weeks ago!
[email protected] 20 Saclay 2-3 December 2009
Plan
Presentation of the T2K experiment
Architecture of TPC readout electronics
Principal components
Current status
Experience return
[email protected] 21 Saclay 2-3 December 2009
Contribution of Irfu to T2K nd280m
Electronics
• ASIC design, production test, readout electronic cards (design and production supervision)
Physics
• Simulation and experimental data analysis of MicroMegas detectors
Detector engineering and integration
• Design of Micromegas detectors, production coordination, QA
• Cooling mechanics and structure, cabling, services (HV, LV, fibers…)
Firmware and software
• Embedded firmware and software in the front-end, slow control, DAQ for detector test bench, firmware and software for DCCs, LV control, etc.
Project management
• M. Zito co-leader of TPC project with D. Karlen (Canada), A. Delbart, D. Calvet coordinator of “Micromegas modules” and “Readout Electronics”
→ Project relies on the diversity of competence at Irfu, teams above critical size and adequate funding
[email protected] 22 Saclay 2-3 December 2009
Project Difficulties Production
• First batch of wafers of the After chip did not pass QA
→ 2 months delay to get After chips
• First batch of FECs failed: humidity stored in After chip package during test requires extended baking procedure
→ more than 2 months to understand the problem and find workaround
• Defects on 10% FEM for first batches (broken via below the BGA)
no consensus reached between designer, PCB manufacturer, and assembly company on exact cause (coating on vias?).
→ three month delay: defect analysis, modifications to PCB, new batch
• Could not solder some connectors on FEMs using lead free process
Front-end production took 1 year while 6 months had been planned
[email protected] 23 Saclay 2-3 December 2009
Project Difficulties (con’t)
Back-end electronics
• DCC prototype board did not become functional
Unanticipated contribution of Irfu to back-end HW/FW/SW and DAQ.
→ Back-up solution based on customized commercial evaluation boards deployed by common effort of Triumf/Irfu/Lpnhe.
CommercialXilinx Virtex 4 ML405
Evaluation board
CustomClock board
CustomOptical
Transceiverboard
FGD DCC Crate
[email protected] 24 Saclay 2-3 December 2009
What really helped
Commercial FPGA evaluation boards
• Inexpensive, readily available, customizable hardware
• Can develop and debug firmware/software on proven hardware– Firmware/software development before a dedicated board is available– Simplifies dedicated board debugging because firmware/software already
tested and only migration issues need to be solved
• Common platform for collaborative development among various group
→ ~45 Xilinx ML405 boards purchased by the TPC-FGD groups! Used at Saclay, Paris, Rennes, Geneva, Barcelona, Hamburg, Tokai, Vancouver
$995.00Part Number:
HW-V4-ML405-UNI-G
[email protected] 25 Saclay 2-3 December 2009
Summary
• What?
A 124.000 channel readout system for the 3 TPCs of the T2K experiment
• How?
A new 72-channel ASICs based on a 511 time-bucket SCA
Custom made readout cards at the back of each 72 Micromegas detectors
Extensive use of commercial FPGA evaluation boards
• Principal merit?
High density, low power: ~16 mW/channel (2 kW for 124.000 ch.); ~3 €/ch.
• Summary and Status
5 year development effort from concept to installation on site
→ TPC commissioning at J-PARC starting in the next few days