Rob Lambert, NIKHEFCore Soft, 24th July 20121 Decoder DB framework (savannah task #19106)savannah task #19106 R. Lambert (Massive thanks to Gerhard and

Rob Lambert, NIKHEF Core Soft, 24th July 2012 1

Decoder DB framework(savannah task #19106)

R. Lambert

(Massive thanks to Gerhard and others for letting me rant at them continuously about this very annoying task)

https://savannah.cern.ch/task/index.php?19106

A Simple Decoder


DecoderRawBankDataObject (Container)

Our Decoders


Client Alg

RawEvent

TES

Decoder

BankA

BankB

BankC

/Event

DataObject (Container)



(Private?) Tool (Handle?)

(Public?) Tool (Handle?)

Each subdetector follows acompletely different strategy.

Data On Demand??

DecodeRawEvent.py + various .opts files

Existing Decoding


Pros Cons

<100 lines of options Uses old-style .opts

Leave all logic to the C++ Forces us to leave all logic to the C++

Easy to re-configure one decoder Impossible to logically reconfigure many decoders

Sets up DoD Forces us to use or interrogate DoD

DoD locations are unsafe to reconfiguration

Tools outside of DoD unknown

Sets up potentially unused algs

Requirements Don’t re-write any decoder C++

Don’t overwrite any current default C++ decoder behaviour

Don’t force the updating or removal of existing user/SD scripts

Handle all the very different subdetector mechanisms

Allow use of the RawEventFormat database

Allow both user *and* high-level reconfiguration

Simplify decoders so they again look like:


DecoderBankADataObject (Container)

Decoder Class Many iterations and re-implementations later…

Hides all the subdetector differences behind one interface


1. New “configurabloid” class

class Decoder(object): “”” A sort of configurabloid which adds itself to a database of possible decoders “”” FullName=“” # the full gaudi name of the low-level configurable Active=False # flags this as an alg to be configured, writing to the TES Banks = [] #list of banks to decode Inputs={} #list of input locations, or {Property:value} dict Outputs={} #list of output locations, or {Property:value} dict

… def setup(self): “”” Configure and return a true gaudi configurable “””

Decoder DB Many iterations and re-implementations later…

Note that inputs/outputs are _read_ from the property if no override of the default is specified… but defaults are incorrect


2. Translate everything in DecodeRawEvent.py to decoder objects

DecoderDB={}

#===========VELO===========Decoder("DecodeVeloRawBuffer/createVeloClusters", active=True, banks=["Velo"], inputs={"RawEventLocations" : ["Other/RawEvent","DAQ/RawEvent"]}, outputs={"VeloClusterLocation" : None}, properties={"DecodeToVeloClusters": True,"DecodeToVeloLiteClusters":False}, conf=DecoderDB)

…

Decoder Conf Many iterations and re-implementations later…

Note how very simple this configurable can be once the subdetector differences are hidden


3. Instrument new configurable to setup the DoD svc if needed

from DAQSys.Decoders import DecoderDB

class DecodeRawEvent(ConfigurableUser): ""“Simple configurable to handle raw event decoding """ __queried_configurables__ = [RawEventFormatConf]

__slots__ = { "DataOnDemand" : False #Add decoding into DoD? , "Sequencer" : None #Add decoding to this sequence? , "DecoderDB" : None #if none, then decoder_db is used , "OverrideInputs" : None #use raw event version to override input locs }

Use case #1 Drop-in replacement for DecodeRawEvent.py

Note that for L0, $L0TCK/L0DUConfig.opts should also be loaded… but that’s not really the same sort of “decoding”


from Gaudi.Configuration import *importOptions(“$STDOPTS/DecodeRawEvent.py”)

from Configurables import DecodeRawEventDecodeRawEvent().DataOnDemand=True

Use case #2 Decode only one bank, in my own sequence

E.g. to bind members in the trigger sequencers


from DAQSys.Decoders import DecoderDB as ddbfrom DAQSys.DecoderClass import decodersForBank

mySeq.Members+=[d.setup() for d in decodersForBank(ddb,“Velo”)]

Use case #3 Use new RawEventFormat to override input locations

E.g.: To re-run Moore >v20r2 on stripped data, and/or make sure default Moore looks first in DAQ/RawEvent!


from Configurables import DecodeRawEvent

DecodeRawEvent().OverrideInputs=“S20”

Use case #4 Decode a second time with slightly different options

E.g.: To decode multiple TCKs in the same DaVinci job


from DAQSys.Decoders import DecoderDB as ddb

newDec=ddb[“OTTimeCreator”].clone(“OTTimeCreator/Ot2”)newDec.Properties[“OutputLevel”]=42newDec.overrideInputs(“Other/RawEvent”)newDec.overrideOutputs(“Other/OTTimes”)

Annoyances Default RawEventLocations doesn’t hold the default

RawEventLocations… gaaaah!!


… same with ODINDecodeToolOutput!

Advantages Configure time introspection:

The database knows which inputs are required and which outputs exist

Spot configuration mistakes The database is smarter than the options file, it spots logical

problems, such as:

Both algs decode LiteClusters. If you run Lite *then* full you will be unable to “put” new output


VLClusters = VLRawBankDecoder("createVLClusters")VLClusters.DecodeToClusters = TrueVLLiteClusters = VLRawBankDecoder("createVLLiteClusters")VLLiteClusters.DecodeToLiteClusters = TrueVLLiteClusters.DecodeToClusters = False

Being Smarter The RICH has a nice configuration framework, which I can

also use in this database:

Which implies Chris’ module could actually add itself to my database. Anything “active” in the database will be configured.


from Configurables import RichToolst=RichTools().smartIDTool()#ensure it's a public tool...tname=t.getFullName().replace("/","/ToolSvc.")

Decoder("Rich::DAQ::RawBufferToRichDigitsAlg/RichRawEventToDigits", active=True, banks=['Rich'], outputs={"RichDigitsLocation":None}, properties={"DecodeBufferOnly":False}, publicTools=[tname], conf=DecoderDB)

t2=RichTools().rawDecoder()t2name=t2.getFullName()

Decoder(tname, active=False, privateTools=[t2name], conf=decoder_db)

Decoder(t2name, active=False, inputs={"RawEventLocations":["Rich/RawEvent","DAQ/RawEvent"]}, conf=DecoderDB)

Conclusions A drop-in replacement for DecodeRawEvent.py exists

This was not an easy task!

Due to the many subdetector differences a configurabloid wrapper class provided the simplest implementation

DAQSys: .DecoderClass, .Decoders, .Configuration

Suggestions:1. Move defining default locations to the constructor!!

2. Homogenize the subdetector decoding C++, names and design

3. Move “options” logic from C++ python

4. Use a service to decode all BanksDataObject(Container), without using the TES, since we never write out these containers (reducing CPU cycles and memory consumption)


End Backups are often required


Documents

Rob Lambert, NIKHEFCore Soft, 24th July 20121 Decoder DB framework (savannah task #19106)savannah task #19106 R. Lambert (Massive thanks to Gerhard and