Embed Size (px)
Citation preview
, a rad-tol
FPGA-based WorldFIP agent
E. Gousiou, P.Alvarez, E.van der Bij, G.Penacoba, J. Serrano | CERN
TWEPP 2011
nan FIP
Outline
o Project Introduction
o Functionality & Features
o Design Validation
o Conclusions
Outline
o Project Introduction
o Functionality & Features
o Design Validation
o Conclusions
WorldFIP, microFIP & nanoFIP (I)
o WorldFIP is a real-time fieldbus used at CERN's LHC for a variety of control systems: Cryogenics, Power Converters, Quench Protection, Beam Instrumentation, Radiation Monitoring, Survey
o More than 10000 WorldFIP client nodes (agents) and 200 WorldFIP master nodes installed in the LHC
o WorldFIP was selected because of the good performance of its agents under radiation
4
FIELDBUS
…WFIPagent
WorldFIPMaster
Userlogic
WorldFIP Architecture
sensor
WFIPagent
Userlogic
actuator
WFIPagent
Userlogic
sensor
WorldFIP, microFIP & nanoFIP (II)
5
o However, in 2009 Alstom decided to phase out WorldFIP support
o Moreover, the latest batches of WorldFIP agents, the microFIP chipsets, were found less radiation tolerant
o Finally, it was decided to in-source this technology at CERN
o The first phase of in-sourcing concerns the most critical part, the WorldFIP agents
is the replacement of microFIP. It implements a subset of microFIP’s functionality.
It is a radiation tolerant FPGA-based chip that acts as an agent for the communication over the WorldFIP fieldbus.
nan FIP
o The development has been divided to work packages that have been distributed among CERN groups and industry
o Until recently, Alstom was the main provider of WorldFIP technology
Open Hardware
6
o nanoFIP is part of the Open Hardware Repository
o Public specification, design files and production files
o Focus on peer reviews
o Better hardware
o See also “Open Hardware for CERN’s Accelerator Control Systems”, Erik van der Bij
Outline
o Project Introduction
o Functionality & Features
o Design Validation
o Conclusions
WorldFIP services:• Data consumption & Broadcast data consumption (up to 124 bytes)
• Data production (up to 124 bytes)
• Communication in 3 speeds: 2.5 Mbps, 1 Mbps, 31.25 Kbps
Functionality & Features
8
Master
User
n
consumption
Functionality & Features
9
WorldFIP services:• Data consumption & Broadcast data consumption (up to 124 bytes)
• Data production (up to 124 bytes)
• Communication in 3 speeds: 2.5 Mbps, 1 Mbps, 31.25 Kbps• microFIP: 120 bytes for shared produced and consumed data
Master
User
n
production
Functionality & Features
10
Simple interface with the user:
• Data transfer over an integrated memory orUser n
WISHBONE MEMORY
Master
User
n
WorldFIP services:• Data consumption & Broadcast data consumption (up to 124 bytes)
• Data production (up to 124 bytes)
• Communication in 3 speeds: 2.5 Mbps, 1 Mbps, 31.25 Kbps• microFIP: 120 bytes for shared produced and consumed data
Simple interface with the user:
• Data transfer over an integrated memory orUser
WISHBONE MEMORY
• Data transfer in 16 in, 16 out lines (no need for memory access)
Functionality & Features
11
Master
User
n
• microFIP: proprietary interface
WorldFIP services:• Data consumption & Broadcast data consumption (up to 124 bytes)
• Data production (up to 124 bytes)
• Communication in 3 speeds: 2.5 Mbps, 1 Mbps, 31.25 Kbps• microFIP: 120 bytes for shared produced and consumed data
n16 bit DATA BUS
JTAG feature
• Efficient way to remotely reprogram the user FPGA
• 3’ for Actel A3P400; 1.5’ for Xilinx XC5VFX70T
Functionality & Features
12
User
UserUserTAP
Master
User
n
Simple interface with the user:
• Data transfer over an integrated memory or
• Data transfer in 16 in, 16 out lines (no need for memory access)
• microFIP: proprietary interface
TAP
JTAG
• microFIP: no reprogramming feature
WorldFIP services:• Data consumption & Broadcast data consumption (up to 124 bytes)
• Data production (up to 124 bytes)
• Communication in 3 speeds: 2.5 Mbps, 1 Mbps, 31.25 Kbps• microFIP: 120 bytes for shared produced and consumed data
n
WISHBONE MEMORY
16 bit DATA BUS
n
Radiation tolerant design
13
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
Radiation tolerant design
14
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
o TMR of the flip-flops & memories
o Fail-safe state machines
FPGA area usage
Radiation tolerant design
15
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
o TMR of the flip-flops & memories
o Fail-safe state machines
FPGA area usage
Radiation tolerant design
16
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
o TMR of the flip-flops & memories
o Fail-safe state machines
FPGA timing
Radiation tolerant design
17
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
o TMR of the flip-flops & memories
o Fail-safe state machines
FPGA timing
Radiation tolerant design
18
Component Selection
o Actel ProASIC3 family
o Flash-based & reconfigurable
o Configuration cells do not exhibit SEUs
o Immune to SELs for the LHC
o Measure TID > 300 Gy > 10 LHC years
o Proven performance in radiation environments (ALICE, nQPS, NASA).
Mitigation Techniques
o Simplification of specifications
o TMR of the flip-flops & memories
o Fail-safe state machines
o Various reset possibilities
User
nPoR
Rst I
n
Rst O
ut
Reset Frame
Outline
o Project Introduction
o Functionality & Features
o Design Validation
o Conclusions
Design Validation
20
Design validation with independent simulation test bench
Test board for functionality & radiation testing developed by external company
Extensive 5-day VHDL code review by 5 design experts from 2 different CERN groups
19 boards running continuously since May 2011
Design Validation – Test Board
21
Userlogic
FieldTR Fiel
drive
Master
FIELDBUS
nan FIP
Master
nan FIP
Design Validation – Test Board
22
Userlogic
FieldTR Fiel
drive
FIELDBUS
Cons
Master
Design Validation – Test Board
23
Userlogic
Fieldrive
FIELDBUS
nan FIP
FieldTR
Cons
Prod
Control Room
Design Validation – Test Board
24
Userlogic
FieldTR Fiel
drive
Master
FIELDBUS
nan FIP
Control Room
Design Validation – Test Board
25
Userlogic
FieldTR Fiel
drive
nan FIP
Master
FIELDBUS
Control Room
Design Validation – Test Board
26
Userlogic
FieldTR Fiel
drive
nan FIP
Master
FIELDBUS
Control Room
Master
Design Validation – Test Board
27
Userlogic
FieldTR Fiel
drive
9V 50m
FIELDBUS
nan FIP
Control Room
Master
Design Validation – Test Board
28
Userlogic
FieldTR Fiel
drive
RS 232 50m
9V 50m
FIELDBUS
nan FIP
Radiation Testing Campaigns
Large scale tests: Cross section estimation
29
Preliminary tests: qualification of test setup and first understanding of possible failures
Extra tests: Irradiation of the nanoFIP, FIELDRIVE & the FIELDTRStudy of the effects of high temperature while irradiating
30
o 2 nanoFIP chips
o Use of the entire Produced and Consumed memory of nanoFIP
o 5ms and 500ms macrocycles
PSI facility 230 MeV p+ beam
p+ beam
Preliminary Radiation Tests at PSIApril 2011
p+ beam
31
o Correct frames exchange until ~400 Gy; no SEE
o At ~400 Gy no frame was being received by the Master; chips not responding
o Soft reset/ power cycle not able to recover the functionality
o Several hours later without radiation the chips had annealed and were fully functional
Preliminary Radiation Tests at PSI – nanoFIPApril 2011
“The instabilities were always accompanied by an increase of the current in the FPGA core from 1 to 33 mA.”
Radiation-Tolerant ProASIC3 FPGAs Radiation Effects ACTEL report
nanoFIP consumption DUT a [mA]nanoFIP consumption DUT b [mA]
FIELDRIVE consumption DUT a [mA]
32
o 1 set of FIELDRIVE/ FIELDTR
o 5 ms macrocycle
o Testing stopped at 400 Gy due to beam time expiration
o No error appeared throughout the testing
10 % current consumption increase
Preliminary Radiation Tests at PSI – FIELDRIVEApril 2011
Large Scale Radiation Tests -Target Cross SectionScheduled for Nov 2011
PSI facility, p+ 230MeV
Irradiation of 10 devices
33
2e9 p+/cm2/ Gy
400 Gy ProASIC3 lifetime σ nanoFIP ~ 1e-13 cm2
< 10 SEE / year2000 nanoFIPs in the LHC
LHC
σ nanoFIP ~ 1e-13 cm2
Outline
o Project Introduction
o Functionality & Features
o Design Validation
o Conclusions
Conclusions
35
Rigorous design process
Exhaustive testing
Wide collaboration across the organization and with the industry
Expected demand at CERN > 2000 chips
The public nature of the design has attracted a European company for train infrastructure
nanoFIP project status
Extras
Considerations
37
o During the reprogramming process, the normal gateway tasks will be stopped
o Reprogramming will only be used without LHC beam
o ProASIC3 devices can be reprogrammed before the accumulation of ~100 Gy
o During the reprogramming process, the normal gateway tasks will be stopped
o Reprogramming will only be used without LHC beam
o ProASIC3 devices can be reprogrammed before the accumulation of ~100 Gy
JTAG programmer SW
38
.svf
TMS TDI TDOTMS TDI
Lib(X)SVFISC
Master
JTAG programmer SW
39
.svf
TMS TDI TDOTMS TDI
TMS TDI TMS TDI…
Lib(X)SVFISC
Master
JTAG programmer SW
40
.svf
TMS TDI TDO
TMS TDI CRC FESCtrlFSS
TMS TDI TDO
Lib(X)SVFISC
TMS TDI TMS TDI …
Master
FIELDBUS
JTAG programmer SW
41
.svf
TMS TDI TDO
TMS TDI TMS TDI TMS TDI … CRC FESCtrlFSS
Lib(X)SVFISC
user
Master
JTAG Controller HW
42
user
FieldTR Fiel
drive
FIELDBUS
Master
JTAG Controller HW
43
user
FieldTR
Fieldrive
Master
Cons
FIELDBUS
JTAG Controller HW
44
user
FieldTR
Fieldrive
Master
TCK
TDO
Cons
ProdTDITMS
FIELDBUS
TAP
JTAG Controller HW
45
user
FieldTR
Fieldrive
Master
Cons
Prod
TCKTMSTDITDO
FIELDBUS
TAP
TDO
nanoFIP vs. microFIP
46
microFIPuserlogic
sensor Signal Conditionersensor Signal Conditioner
is:
is not:
o Backwards compatible for the user
sensor Signal ConditionerMaster
o Tailored to users’ needs
o Common use of the chip and centralized support
o rad-tol by design
o nanoFIPs and microFIPs can co-exist under the same Master
o Expected demand >2000 components
nan FIP
nan FIP
nanoFIP vs. microFIP
47
nanoFIPuserlogic
sensor Signal Conditionersensor Signal Conditionersensor Signal Conditioner
Master
is:
is not:
o Backwards compatible for the user
o Tailored to users’ needs
o Common use of the chip and centralized support
o rad-tol by design
o nanoFIPs and microFIPs can co-exist under the same Master
o Expected demand >2000 components
nan FIP
nan FIP
Project Organization & Some History
ALSTOM-CERN contract with CERN purchasing ALSTOM’s design information. (2008)
Concerns for the long-term availability of ALSTOM’s components; WorldFIP Taskforce set up. (2006)
Project divided in different Work Packages: (2009)
WP1: microFIP code preliminary interpretation (B. Todd, TE/MPE & E. van der Bij)
WP2: project management documentation for the in-sourcing (E. van der Bij)
WP3: functional specifications for microFIP’s replacement (E. van der Bij)
WP4: rewrite & extend microFIP VHDL code
WP5: write new code (P. Alvarez & E. Gousiou)
WP6: test bench creation (G. Penacoba, TE/CRG)
WP7: design of a board for functional and radiation tests (HLP, France)
WP8: Radiation tests (CERN RadWG EN/STI & E. Gousiou)
Taskforce conclusions: No technological alternative & in-sourcing of WorldFIP technology. (2007)
48
WorldFIP Frames
Communication throughput for 1Mbps:
FSS2 bytes
Ctrl1 byte
Id2 bytes
CRC2 byte
FES1 byte 8 bytes * 8 bits* 1 us
FSS2 bytes
Ctrl1 byte
Data124 bytes
CRC2 byte
FES1 byte
130 bytes * 8 bits * 1us
Master -> nanoFIP
nanoFIP -> Master
1.1 ms for 124 data-bytes= 0.9 Mb/s
Master -> nanoFIP
nanoFIP -> Master
turnaround time 10 us 10 us
FSS2 bytes
Ctrl1 byte
Id2 bytes
CRC2 byte
FES1 byte
FSS2 bytes
Ctrl1 byte
Data2 bytes
CRC2 byte
FES1 byte
10 us138 us for 2 data-bytes= 0.1 Mb/s
49
turnaround time
Project Status
Majority voter circuit:
50
