Review, May 5, 2004Lawrence Berkeley National Laboratory DOM MB Hardware CDR and PRR, May 5, 2004 Gerald Przybylski Lawrence Berkeley National Laboratory

Review, May 5, 2004 Lawrence Berkeley National Laboratory

DOM MBHardware

CDR and PRR,

May 5, 2004

Gerald PrzybylskiLawrence Berkeley National Laboratory


Once upon a time, in 1996…• Dave Nygren, Buford Price, Francis Halzen & JPL

- Use Stewart Kleinfelders ATWR (SCA)• Two DOMs, deployed Jan ’97

(Author recruited to take ‘em to the pole that year for deployment) • Built around a PIC, a One-Time, EPROM

Programmable, Microcontroller, with ADC• Data: Push Mode, FSK modulation• Marginal stability,

but it produced!


SwAmp vs. JPL DOMTypical AMANDA Analog OM, JPL Digital optical module

7.5us 132ns

Thanks to

Tim Miller and Doug Lowder


Lessons Learned• PMT Waveforms ARE Interesting• Software Underestimated• One Time Programmable CPU Worrisome:

- No Post Deployment Evolution- Front Loaded Software Development- Not Enough Memory or Speed

• Event Rate Limit by Slow ADC in PIC (dead-time)

• JPL Version Not Designed to be Scalable• Get to Network ASAP


Scaling up:for AMANDA

String 18:40 Optical Modules, January 15, 2000


Hybrid:Fiber & Digital

Compromises/problems…

+EIM and Noise Storm Immunity

+100 PE linear dynamic range

-Fiber ~7 PE dynamic range (without tricks)

-Fiber Attrition

-PMT HV Power Supply problems


Design Drivers for IceCube• Pulse Fidelity/Waveform Quality• Wide Dynamic Range• Event Timing Accuracy• Low Dead-time• Low Power Consumption• Adequate CPU and Memory• Remote Reprogramability, Software, and Firmware• Off-Board Interfaces: PMT Power, and Flasher Board• Cable Sharing: Bidirectional, Multi-Drop, Power, NO Fiber• Long Lifetime/High Reliability/Cost Tradeoff• Self Identification – No mapping files• Built-in Calibration and Monitoring Features• IceTop & In Ice DOMs Identical


FPGAand

SOPC

CPU

CPLDFlash Flash

PMT Power

SDRAM

SDRAM

ATWD

ATWD

fADC

DAC

Monitor& Control

LPF

LC

x16

x2

x0.25

FlasherBoard

Pulser

DACs & ADCs

Corning Frequency Ctl (was Toyocom)

4Mb 4Mb

16Mb

16Mb

+/-5V, 3.3V, 2.5V, 1.8V

64 Bytes

Delay

Trigger (2)ADC

Oscillator

20 MHz

40 MHzMUX

(n+1)

(n–1)

DOR

OB-LED

1 5

4 8

x 2.6 x 9

10b

10b

10b

10b

8b

32b

16b

8b

8b, 10b, 12b

DPRam

1 megabaud

DC-DC

ConfigurationDevice

8Mbit

Digital Optical Module Block Diagram

GTP, LBNL, 26 Mar, 2004


Rev4.2


Design Methodology• Schematic Entry (Mentor)• Layout Tools (Mentor)• Simulations of Hardware subsystems• Simulations of Firmware• Testing in full scale prototype; 4 Major Revisions• Prototyping in isolation• Trade Studies• Interface and Envelope Coordination with UW• Circuitry Reviews; Reliability Reviews; HALT;

HASS• Framework Testing


For more information:Hardware:

See http://rust.lbl.gov/~gtp/DOMContact Gerald Przybylski

FirmwareContact Thorsten Stezelberger

TestingContact Azriel Goldschmidt

SoftwareContact Chuck McParlandContact Arthur Jones

All of whom are at LBNL


Part II: Subsystems

• Power conversion and conditioning

• Memory: Boot Flash, Flash, and SDRAM

• Configurable Logic: FPGA and CPLD

• Communications: DOM to Surface, LC

• PMT Signal Acquisition

• Local Clock

• Control, Monitoring and Forensics


Waveform Quality

Digitize at the PMT Anode to Overcome Dispersion in Transmission Line

- PMT Waveform risetime of a few ns- Cable risetime from 1us to 2us- Improve noise immunity- High ATWD sampling rate, 128 Samples


Wide Dynamic Range

• x0.25, x2, x16 gain inputs to ATWD- Achieve around 14 bits from 10 bit part

• 40 count SPE = 5mV into ATWD channel 0

• 800 PE = 1V into ATWD channel 2

• PMT saturates at 5 to 6V into 50 ohms - Divided by 4 at input of ATWD channel 2- System limited by PMT saturation!


Event Timing Accuracy

• Synchronous Triggering- Requires Delay Line, but- Simplifies Noise Removal- 25ns Course Clock- ~ 3ns ATWD sample- < 1ns resolution from curve fitting

• Clock Phase Tracking- < 2ns jitter in 2us rise-time pulse


Low Dead Time

• 128 ATWD samples per hit = ~ 420ns- Up to 4 Channels- ~50us to digitize and store data

• Two ATWDs per OM- Acquire into B while Digitizing A- or Deployed spare…

• Acquire fADC data for > 4000ns @ 40 msps• Digitizing when Local Coincidence “Tag” is present


Low Power Consumption

• High Performance, Low Power Amplifiers, Comparators, and Multiplexers

• High Performance, Low Power Data Converters- ATWDs, fADCs, and Communications DAC

• Coolrunner II CPLD• ARM Hard Core CPU/FPGA• Laptop SDRAM Main Memory• High Efficiency Switcher Power Supplies


Adequate CPU and Memory

• SOPC with ARM CPU and 400K Gate FPGAReplaces 256 Pin FPGA + 256 Pin CPU- Wider, Faster Bus… no bottle-necks- Improved speed – power- Bus to FPGA Bridge - for general I/O- FPGA to Bus Bridge - for Event DMA- Dual Port Ram - for Communications


Remote Reprogrammibility

• FPGA as Configurable Peripheral Controller- Communications Rx Tx- PMT Event Acquisition- Calibration Controller

• CPLD as Glue Logic- Mediates Rebooting- Off-Board Peripheral Interface and Firewall- Serial DAC and ADC interfaces


Off-Board Interfaces

• HV Control Pin-out Doubled-up - Enhance the reliability of Ribbon Cable - Digital Control and Read-out Interface

• 48 Pin Flasher Interface- Power-Down mode- Just Another Memory Mapped Peripheral- Firewalled through the CPLD- Event initiation on command from FPGA- Flasher Pulse Current Monitor


Cable Sharing• AMI Modulation, demonstrated in String 18• RAPCal, demonstrated in String 18

- Round Trip- Drift Tracking

• Two DOMs per DOM Hub Channel + Half the Cables, saving $$$, and Flights+ Half the DOM Hubs, saving Power and Space- Double the Data Rate- Higher Firmware Complexity- Higher Cable Complexity- Higher Deployment Complexity


Cost and Reliability• Industrial Grade, or Better

-40°C to +85°C …- Mil-Spec too costly, and narrow selection

• Vetted Component Manufacturers• Qualified PCB maker, and Board Loader• HALT and HASS Testing• Manufacturers Reliability Information Studied• STF Testing • Component Cost ~ 10% to 15% more

- e.g… 10 x Cost for Corning Oscillator


Self Identification• 64 bit Unique Digital ID in Each Flash Chip• Configuration Building Software

- Executed when DOMs are powered up• Plug and Play

- A or B identified by Jumper during manufacture.- Any DOM may be assigned A or B- Any A DOM and any B DOM may be installed on any pair. - Cable pairs may occupy any DOM-Hub Input

• Ident in PMT HV System, and in Flasher Board


Built In Calibration Pulsers• Front End Pulse Generator

- Generates PMT like signals• On-Board Single LED Pulser

- Optimized for narrow pulses- PMT Transit Time Calibration Tool

• Daughter Board: Bright, Long Range LED Pulser- Geometry Calibration Tool- Shower Simulator

• Crude arbitrary waveform pulser circuit


Built-in Monitoring/Forensics• 24 slow ADC channels monitoring

- Voltages- Currents- Temperature- Pressure

• 8 fast signal monitor inputs- Clock x1, Clock x2, - Communications input, - Upper and lower Local Coincidence, - LED Pulse Monitors

• Post deployment failure investigation


IceTop “vs.” IceCubeOne OM per Pair doubles data rate.

Insulation moderates extreme temperature (-80°C)

Dual Range operation spans air shower energy range.

Active, critical involvement in meeting needs.


The End


XA Block Diagram

CPU Side

(Stripe)

__________

“PLD” Side


GlossaryADC, Analog to Digital ConverterAMI, Alternate Mark InversionATWD, Analog Transient Waveform DigitizerATWR, Analog Transient Waveform RecorderARM, Advanced Risk MachineCPLD, Complex Programmable Logic DeviceCPU, Central Processing UnitDMA, Direct Memory AccessDOM, Digital Optical ModuleFPGA, Field Programmable Gate ArrayFSK, Frequency Shift KeyingHALT, Highly Accelerated Life TestingHASS, Highly Accelerated Stress ScreeningJPL, Jet Propulsion LaboratoryLED ,Light Emitting DiodePCB, Printed Circuit BoardPIC, Programmable Intelligent Computer. PMT, Photomultiplier TubePSK, Phase Shift KeyingRAPCal, Reciprocal Active Pulsing CalibrationSCA, Switched Capacitor ArraySOPC, System on a Programmable ChipSTF, Simple Test FrameworkSwAmp, Swedish Amplifier for Analog Optical Modules over Twisted Pair

Documents

Review, May 5, 2004Lawrence Berkeley National Laboratory DOM MB Hardware CDR and PRR, May 5, 2004 Gerald Przybylski Lawrence Berkeley National Laboratory