Data Concentrator - SVD data multiplexer with FPGA-based

Preview:

Citation preview

Data ConcentratorSVD data multiplexer with FPGA-based tracking

Tracking-Meeting Bayrischzell

Jochen Dingfelder, Carlos Mariñas, Michael Schnellschnell@physik.uni-bonn.de

University of Bonn

July, 19th 2012

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 1 / 21

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 2 / 21

Motivation

Motivation

CopperSystem

DataConcentrator

ROI

optical links

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 3 / 21

Hardware Status

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 4 / 21

Hardware Status

Hardware Solution

• Advanced Mezzanine Card (AMC)• Redesign with 4 SFP connectors

(Zhen-An)→ 11 cards• Received during DEPFET meeting

in Seeon• Given back to Zhen-An for fixing

backplane problems• Start testing, when it returns

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 5 / 21

Hardware Status

Simplified Block Diagram

FD

FD

FD

FD

4:1Mux

FD

FD

FD

FD

4:1Mux

Framer

11:1Mux

FD

FD

FD

FD

4:1Mux

Decoder

Scheduler &MMU

4 GB DDR 2Memory

Trackreconstruction

EthernetFrame

GPU ProtocolFramer

AmplitudeAnalysis

Collection

AMC "Datcon"

ATCAGPU Farm

FD

FD

FD

Framer

Framer

.

.

.

AMC "Collector"

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 6 / 21

FPGA-based tracking

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 7 / 21

FPGA-based tracking

Data Flow Diagram Plan

General tracking module interconnection (can also be distributed)

Conformal

transformation

SVD strip-dataCoordinate

translator

Hough

transformation

ROI

creator

Track

creation

Intercept

finder

ATCA

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 8 / 21

FPGA-based tracking

Coordinate Translation

LUT for basic location lookup of eachladder

With strip ID, strip pitch in u-vdirection and DSP of FPGAdetermine precise position

Alignment constant for each ladder inu-v direction

Constant for sagging correction withLUT of a x2 function

Start

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 9 / 21

FPGA-based tracking

SVD Clustering idea

+ Take center and size of cluster, saves space and resources− Additional type of data needed

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 10 / 21

FPGA-based tracking

General Testing Platform

Userlandtrack generation and strip ID calculation

Kerneltransmission to PCIe

over DMA map

PCIe 1xtransmission to FPGAand init processing

FPGAprocessing and

back transmission

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 11 / 21

Simulation

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 12 / 21

Simulation

Real 2D Hit

500 Background hits (∼ 1 percent occupancy)

5 generated tracks

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2

y

x

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 13 / 21

Simulation

Hough Space

Corresponding Hough Space:

-30

-20

-10

0

10

20

30

-1.5 -1 -0.5 0 0.5 1 1.5

d

φ

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 14 / 21

Simulation

FHT Example With Less Background

30 background hits and 2 tracks in Hough space

-30

-20

-10

0

10

20

30

-1.5 -1 -0.5 0 0.5 1 1.5

d

φ

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 15 / 21

Simulation

FHT Example With Less Background

30 background hits and 2 tracks in Hough space with intercept regions

-30

-20

-10

0

10

20

30

-1.5 -1 -0.5 0 0.5 1 1.5

d

φ

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 15 / 21

Simulation

FHT Example With Less Background

Zoom to one of the intercept regions

-1.35

-1.3

-1.25

-1.2

-1.15

-1.1

-1.05

-1.28 -1.278 -1.276 -1.274 -1.272 -1.27 -1.268

d

φ

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 15 / 21

Simulation

Definitions

A few later used definitions:

Efficiency:

ε := number of correct found trackstotal number of real tracks ε ∈ [0,1]

→ For all later shown plots: ε = 1!!

Purity:

p := number of found trackstotal number of real tracks p ≥ 0

→ p ≥ 1

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 16 / 21

Simulation

Track Purity With FPGA Algorithm Simulation

0.8

1

1.2

1.4

1.6

1.8

2

100 150 200 300 400 500

# r

eco

nst

ruct

ed

tra

cks

/ #

genera

ted

tra

cks

Background Hits

1 generated Track2 generated Tracks3 generated Tracks4 generated Tracks5 generated Tracks

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 17 / 21

Simulation

Execution Time

Running on Core i7 2600 @ 3.8 GHz (with fixed affinity + turbo boost):

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

100 200 300 400 500 600 700 800 900 1000

Tim

e [

s]

Hits

CreateHough Trafo

Intercept FinderIntercept Finder (Vertex constrained)

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 18 / 21

Simulation

Visualization Software

For visualizationfull VXD (SVD,PXD) with activearea and stripand pixelstructure

Markingbackground,tracks, clusters...

Presentation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 19 / 21

Simulation

Summary and Status

2D Simulation of the SVD (r-ϕ plane only), r-z plane in progress

Track reconstruction efficiency is high, but...tracks with Gaussian smearing only (2 strip pitches)

No multiple scattering, energy loss, magnetic fieldinhomogeneities...

→ Switch to the basf2 framework

• Done, but only with local coordinates of sensors.

• Hough transformation only feasible with up to 1 − 2 percentoccupancy

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 20 / 21

Simulation

Thank you for your attention!

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Technical Specifications

Technical Specification on I/O

Input:Up to 42 (32)optical-fibre links

1.5 Gbps max

On average110 Mbpsexpected

Dynamicbandwidth

Output:6.25 Gbps overbackplane orSFP+

440 Mbpsaverage datarate

Dynamicbandwidth

→ Total: 4.6 Gbps ≈ 550 MiB/s

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Idea

Tracking Options

Hough Transformation to findstraight and arc tracks (afterconformal transformation)

Can be fast and efficientimplemented in the FPGA overFast Hough Transformation (FHT)

First 2D resulsts of the next slides

Open for other algorithm, likesector-neighbour finding cellularautomaton or Kalman-Filter -1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

x

Houghtransformation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup FPGA Implementation

Trigonometric Functions and Memory Requirements

Need trigonometric functions like sine and cosine e.g. forcoordinate transformation

CORDIC: Basic Lookup table for nodes and a shift-addimplementation of approximation functionPure LUT: Large Lookup table for every value (high memoryrequirements)

FPGAs’ DSPs used as Multiplier/Divider unit.

Block Memory needed to translate strip IDs into coordinates andROI creation

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup FPGA Implementation

Status of the Different Blocks in Simulations

FD

FD

FD

FD

4:1Mux

FD

FD

FD

FD

4:1Mux

Framer

11:1Mux

FD

FD

FD

FD

4:1Mux

Decoder

Scheduler &MMU

4 GB DDR 2Memory

Trackreconstruction

EthernetFrame

GPU ProtocolFramer

AmplitudeAnalysis

Collection

AMC "Datcon"

ATCAGPU Farm

FD

FD

FD

Framer

Framer

.

.

.

AMC "Collector"

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Multiplexer Theory

Multiplexing

Three general types of multiplexing:Frequency Division Multiplexing (FDM)

Time Division Multiplexing (TDM)

Statistical Multiplexing

Because of dynamic bandwidth “Statistical Multiplexing” would bemost suitable

Demultiplexing is trivial, if only multiplexing of 32 bit words isensured

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Multiplexer Theory

Aurora overview

Xilinx Aurora protocol as physical link layer over GTP

Takes care of every “low level” operation like clock correction,symbol decoding and 8B/10B Encoding

Easy data transmission/framing over locallink

Error detection and initialization

Status wires for channel and lanes

Lane: One communication lineChannel: Concentration of one or more lanes

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Simple Hough Trafo in 2D with straight tracks

Hits in 2 D

Want to find straight tracks i : yi = m · xi + a in this mess

→ Go to Hough-Space: a = −m · xi + yi

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

xMichael Schnell (University of Bonn) Data Concentrator July, 19th 2012 22 / 21

Backup Hough-Transformation

Hough-Space

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Find interception of 5 Tracks around zero!

Found 2: y1 = 0.6 · x ; y2 = 1.5 · xMichael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

With tracks

Found 2:y1 = 0.6 · xy2 = 1.5 · x

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

x

tr 1tr 2

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Coordinate Transformation

This is bad for tracks close or parallel to y-axes (m = inf.)!

Make a coordinate transformation like: r = x · cos(Φ) + y · sin(Φ)

0

0.5

1

1.5

2

0 0.5 1 1.5 2

y

x

trackr

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Implementation in FPGA?

Use Fast Hough Transformation!

Divide and Conquer type algorithm

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Implementation in FPGA 2

Vary with minimum and maximum size of a unit area the tolerance

Efficient and highly parallelisable, a sector for each unit

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Implementation in FPGA 3

Also support of different shapes

One more to go...

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Backup Hough-Transformation

Implementation in FPGA 4

Error comes for “free”

Done... finally

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Michael Schnell (University of Bonn) Data Concentrator July, 19th 2012 21 / 21

Recommended