Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation

Status and future perspectives of the

analysis language (eDSL) for HWC and LHC Operation

Kajetan FuchsbergerTE-MPE TM, 2014-02-27

On behalf of the TE-MPE-MS Software Team:D. Anderson, M. Audrain, M. Dragu, J.-C. Garnier,

A. Gorzawski, M. Koza, K.H. Krol, A. Moscatelli, K.Stamos,P.C. Turcu

2Kajetan Fuchsberger TE-MPE TM, 2014-01-20, Analysis Language (eDSL) for HWC and Operation

Content

Motivation

Status & Usage

Behind the Scenes

Outlook & Visions


Warning!Vi

siona

rity

Tem

pera

ture

100°

50°

0°

Visionarity Temperature might be High!

(But not unrealistic ;-)


Content

Motivation

Status & Usage

Behind the Scenes

Outlook & Visions


eDSL ?

eDSLDomain

Specific Language

embedded

Hwc; Systems; Accelerators; Signals;


Why a DSL at all? Currently:

E.g. writing a PM module (or an analysis for a test) is pure coding (Java/LV). A lot of knowledge required, which distracts from the real logic: Where to get the data from? What tools to use to display it? What algorithms to apply?

A lot of code duplication. E.g.: E.g. ArrayUtils in PM modules Every application again connects in its own way to the Logging DB,

PM, reads files....

DSL focuses on the logic of the domain:


Why (Java) embedded? Integration within the Accelerator SW Architecture Java

We are not language designers, but software developers.

Reuse: All the native language features of Java, while providing the

required readability for non-programmers. Tools for writing and verification of analysis scripts (eclipse). Strongly typed; with the help of an external library, it was

easy to enforce the correct usage of physical units.

It provides the necessary possibility to constrain the user input to the available functionality.


Where could it be used? Test Analysis within AccTesting (origin)

PM modules

Pre/Post operational checks

periodic calls + reports: E.g. Splice measurements LHC statistics

…?

Any Java Application!

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


What about a better name?

ZeldaZuper Easy Language

for Data Analysis

Brainstorming….

GENIAL

Generic Integrated Analysis Language

Analysis Language for Operation and Hardware

Commissioning


Content

Motivation

Status & Usage

Behind the Scenes

Outlook & Visions


Actual Status

Courtesy: M. Audrain

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Full Automation of PNO.d1 and PCC.1

See Scott & Zinur‘s presentation

Executed ~2000 times within an HWC campaign

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


What are we missing? More signals from post mortem

(E.g. QPS buffers)

Circuit parameters (deferred scalars)

Calculations:

All logging variables (code generation)

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


How to test? - Framework On our (SW) side:

http://sonar.cern.ch See also Cristina‘s recent Presentation:https://indico.cern.ch/event/293574/


How to test? – Module Developer

Use a set of known cases for which the module passes or fails.

To be automated (bamboo)!


Systematic Checks against old data


Integration in AccTestingTo come:Add possibility to display arbitrary signals


(mis ;-) use for data extraction Do not care, where data comes from!

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Content

Motivation

Status & Usage

Behind the Scenes

Outlook & Visions


Behind the scenes

request

„compile“

Data

Result

Only extract necessary data!


The Tree …

Courtesy: Kamil Krol


Resolving….

Main Principle: Ask the Modules, to resolve the biggest subtree possible in one go.


Content

Motivation

Status & Usage

Behind the Scenes

Outlook & Visions


A more abstract view Requirements for data Analysis:

Data incompleteness (e.g. a beam positions along the ring: added/removed BPMs)

Data invalidity: (e.g BPMs might be misbehaving) Mathematical operations: (e.g.

average/addition/scaling) Physical units: (Different units in storage ->

conversion left to the user) Error propagation: left to the user

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Anything out there?

Courtesy:K. Stamos, D. Anderson

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Tensorics® Everything is a tensor.

Common Operations.

Direct and deferred Usage.

Choosable strategies(with useful defaults): Error propagation Data incompleteness Units

To be released as Open Source soon.

Courtesy:A. GorzawskiVi

siona

rity

Tem

pera

ture

100°

50°

0°


The dispatcher – possible Improvements

Caching

All operations are Stateless (basically mathematical functions) easy to parallelize.

E. Montesnitsalis: UFO search as an example Use Case

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Blockers & Wishes ;-) Java 8: Closures!

Would allow us to make the language more dynamic and easily extensible.

Will be production released on 18th March. Unfortunately blocked by CO. Will not come

before end of Run 2! Cluster!

To evaluate different options (parallelization, Scalability)

Useful also for different other R&D: PM storage, distributed file systems, test environments, R&D for future protection systems.

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Only measured data? Circuit Models?

Accelerator Models (MadX)?

Acquisitions (live data)

…

Beam Commissioning?Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Summary & Outlook Ready (from language point of view) to analyse PNO.d1 and

PCC.1

Next features: More Signals, Circuit parameters (scalars), Calculations.

Switch to Tensorics® when ready.

Optimization (Caching, Parallelization…) to come?… as soon as we can get some cluster!

Can be use in other fields (Models? Big Data analysis? Supervision? Statistics…)

Wishes/dreams: Java8 and a Cluster ;-)

Visio

narit

y Te

mpe

ratu

re

100°

50°

0°Visio

narit

y Te

mpe

ratu

re

100°

50°

0°Visio

narit

y Te

mpe

ratu

re

100°

50°

0°Visio

narit

y Te

mpe

ratu

re

100°

50°

0°Visio

narit

y Te

mpe

ratu

re

100°

50°

0°


Growing a Language http://www.youtube.com/watch?v=_ahvzDzKdB0

Guy Steele ‚Growing a Language‘(Keynote OOPSALA Conference, 1998)

„If one lets the users help do the work, growth can be quick!“

„if many persons work side by side and the best work is added with care and good taste, a great deal can be added in short time.“

http://www.youtube.com/watch?v=_ahvzDzKdB0

http://www.youtube.com/watch?v=_ahvzDzKdB0


Thanks a lot for your Attention!


Resolving…. Ask each resolver: can resolve ?

If a resolver can resolve the given node remember.

If none of the resolvers can resolve the node, then the same procedure is recursively invoked with all the children of the node (e.g. node B in the example).

Else (at least one potential candidate resolver), the dispatcher stops the iteration through the tree in this branch and does not try to resolve the child nodes anymore (in the initial state of the example, this would happen for node C).

As soon as the iterations through all branches are the dispatcher selects one of the candidate resolvers per node, invokes it and rebuilds the tree with the resolved node.

After all resolvers return, the whole algorithm is looped starting at item (1), until all nodes (including the root node) are resolved.


The Tree …

Courtesy: Kamil Krol

Documents

Status and future perspectives of the analysis language ( eDSL ) for HWC and LHC Operation