Simple ideas on how to integrate L2CAL and L2XFT ---> food for thoughts

Simple ideas on how to integrate L2CAL and L2XFT

---> food for thoughts

Ted

May 25th, 2007

Outline

Comment on the final L2 upgrade commission efforts

Some basics on how FILAR/S32PCI64 works The problem with L2CAL and L2XFT integration due

to large data package size Ideas on possible quick solutions

Comment on the final commissioning efforts

We have a wonderful team: the L2 United team A strong/powerful team in the past two weeks, even with two key people

(the SPLs, Vadim&Gene) on vacation It has been a real pleasure working with them in the past two weeks:

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.




are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.







people who spent most of their time on final commission in the past ~ two weeks



The L2 tough ladies

S32PCI64: single channel FILAR

The four-channel FILAR

S32PCI64: With transmitter firmware: --> Solar With Receiver firmware: --> single channel FILAR

Four-channel FILAR (Rx)

http://hsi.web.cern.ch/HSI/s-link/devices/FILAR



CAL1

CAL2

CAL3

CAL4

Reces

SVT

Muon/XTRP/L1

Old cluster

XFT1

XFT2

XFT3

L2 Decision Block:Conceptual view

To L2TS PulsarL2D + (TL2D if L2A)

Control Nodeprocess

New L2 decision

node

Spare node

SolarFilar

Filar

Filar

Inside the Box

decision node

Spare node

data words

CAL1 == 104

CAL2 == 104

CAL3 == 156

CAL4 == 104

Reces

SVT

Muon/XTRP/L1

Old cluster

XFT1 > 120

XFT2 > 120

XFT3 > 120

So what is the the problem then? Both new data paths have large data volume: L2XFT (grow with luminosity, could be very large) + L2CAL (fixed length)


Control Nodeprocess

data packagesfighting over 2 PCI buses, when traffic jam occurs,

data wait in the input FIFO

Input FIFO could overflow IFdata packages too large---> missing data words

--> CPU detects this --> L2DTO

Spare node



CAL1CAL2RECESCAL3

SVTMuon/XTRP/L1XFT3

XFT1XFT2Old ClusterCAL4

The actual setup with beam had problem when both L2CAL+L2XFT are included show up as XFT data package corruption --> L2 DTO no problem when only L2CAL is included

PCI bus 1

PCI bus 2



CAL1 == 104CAL2 == 104RECESCAL3 == 156

SVTMuon/XTRP/L1XFT3

XFT1XFT2Old ClusterCAL4 == 104

with L2CAL only, i.e. without XFT:--> do not expect FIFO overflow problem

PCI bus 1: on this solar bus,the worst case is likely when L2A with TL2D bank and scaler info sending out from solar… up to ~ 1K words SLINK package… During the time this package is transferringover PCI bus 1, the largest CAL3 data has to wait in the input FIFO. But the maximal number of input events is 3 (since 1 is already on its way L2Aed). 3 x 156 = 468 < 512 FIFO depth… impossible tofill up the FIFO.

PCI bus 2: worst case is 4 L1A in a row…The largest data package is CAL4 == 104

CAL4: 104 x 4 = 416 < 512

No problem

Can we solve this problem?

The Pulsar-based system is very flexible zero-suppression, fast abort, enable back-pressure, use

dedicated PC for dedicated buffer event … etc

Many ways to deal with this, but I will ONLY talk about the simplest possible solutions…(with the potential to be done within weeks, not months).

Will only show two simple ideas today (1) let large XFT data have higher priority over PCI bus (2) use another PC for the XFT data, to do fast-abort or

data suppression, ROI (Region-Of-Interest)…

How to let XFT data packages have higher priority over the PCI bus?

Current way of PCI handling for each data channel: Send requests for up to 4 (effectively) buffer events at begin of run This means that not only the current event package, but also packages from next

event(s) could compete over the PCI bus if they arrive early: in other words, whoever arrives early gets the bus…

This made sense in the past kuz algorithm time is almost zero, and all data path had small size

How to let largest data path have highest priority? This can be done by simply playing tricks with the request FIFO:

Only send requests for up to 4 buffer events for largest data paths at the beginning of run. Do not send request for all other (smaller) data paths

For each event, only after the largest packages arrived to PC memory, send request for the rest of data paths, but only for the current event. Before that, all other smaller data packages will have to wait in their input FIFO: not allowed to fight over the PCI bus.

This should reduce the possibility of FIFO overflow, but may not fix the problem in a fundamental way (as luminosity increases), and likely delay L2CAL path.

But it is the simplest thing to do…this will be done anytime soon…

data words

CAL1 == 104

CAL2 == 104

CAL3 == 156

CAL4 == 104

Reces

SVT

Muon/XTRP/L1

Old cluster

XFT1 > 120

XFT2 > 120

XFT3 > 120

There is simple way to possibly solve the problem:use the spare PC to receive XFT data and do “zero-suppression”in software … then send to main PC …This sounds more complicated than it actually is …


Control NodeProcesscontrolsboth PCs

L2 decisionnode

Simple implementation:

• Send empty package if no need forStereo tracking

• Send aw xft data when needed• (based on L1 bits)• Could also do ROI

More involved implementation:

Do 3D tracking right here for tracks about certain Pt…. your wild ideas here ,,,

Solar

Filar

Filar

(Single channel)Filar

data words

CAL1 == 104

CAL2 == 104

CAL3 == 156

CAL4 == 104

Reces

SVT

Muon/XTRP/L1

Old cluster

XFT1 > 40

XFT2 > 160

XFT3 > 160

Load balancing

Spare PC could use 2 single channel Filars (2048 input FIFO depth) to receive the largest XFT packages…The decision node could also use single channel Filar to receivethe xft package.


Control NodeProcesscontrolboth PCs

L2 decisionnode

Solar

Filar

Filar

(Single channel)Filar

Solar

Filar(single channel)Filar(single channel)Filar The spare PC

This all looks more complicated than it is--- so what is really involved then?

Step 1: how to ask the existing control node process to also control the spare PC? Daniel’s design of the control node can naturally handle multiple PCs (a L3 based design) In Daniel’s words: “just use the CardEditor to change number of PCs from 1 to 2, then input the spare PC IP address”.

Step 2: what do we have to change for the existing code for L2 decision node? Nothing, if one simply use the spare PC to do fast abort based on L1 bits It doesn’t know whether the XFT package is from a PC or a Pulsar merger

Step 3: move XFT fibers from the decision node to the spare node, as well as the copy of muon/xtrp/L1 fiber.

Step 4: could use single channel Filars if so desired (software is the same) Step 5: copy over the decision node software to the spare PC, remove most of the

algorithm stuff, do fast-abort here. If the event doesn’t need stereo tracking, send empty package. Otherwise, send the combined raw data.

Later on, could add more functions for zero-suppression, such as ROI. Or, if you really want, simply do stereo tracking right here in spare PC… e.g. for muons … Your wild idea here… We may want to order 2 more PCs soon, as spares…

System commissioning is fun …

I am leaving tomorrow, for two weeks This is just one of the ideas I have had Didn’t want to talk about it, but young people kept asking Meant as food for thoughts, to encourage young people to brainstorm

Our system is more flexible than we think

About commissioning:

To have a system more or less working is not hard, To have zero error is hard, To keep it that way is even harder…

Whatever you do, think about how to win the peace afterwards…

Documents

Simple ideas on how to integrate L2CAL and L2XFT ---> food for thoughts