Experience with the ATLAS LAr optical link

Experience with the ATLAS LAr optical linkThe ATLAS LAr optical link system.Experiences:in the R&D phase.In the integration phase.In the production and QA phase.In early operation phase.The OTx problem and a possible fix as of now.What I would do for future optical links.Jingbo YeDepartment of PhysicsSMUDallas, TexasDisclaimer: this talk only reflects my own experience and understanding of the optical link, not the official opinion of ATLAS LAr.

LAr o-link.Lessons to me.The Dual OTx.Thoughts.

The ATLAS LAr Optical Link. 1The G-Link based optical link for ATLAS LAr:

ORxGLink RXFPGAA total of 1524 channels installed. SMUX: an ASIC based on DMILL, level translating, 2:1 multiplexing.GLink: COTS, operates outside of spec, specially ordered from Agilent. Water cooled on both Feb and ROD.OTx, ORx: VCSEL/PIN based assembly produced by IPAS in Taiwan, with COTS driver and amplifiers.All FE components were qualified for 360 Krad. ORxGLink RXFPGA8 channelsBE -- RODA collaboration of 5 institutions: CPPM, IPAS, ISN, KTH and SMUThe project lasted for 6+ years:1998 - 2004


The ATLAS LAr Optical Link. 2The link TX on the FEB

~ 400 mmOTxGlink TXSMUXThe OTx is a custom assembly with VCSEL in an ST connector. There were quite a few issues of this assembly in production in Taiwan and QA at SMU. We are now having problems with the VCSELs at a few percent level. Overall this single fiber per FEB design does not answer the reliability requirements from physics (physicist), especially in an environment where no access is possible for maintenance.


The ATLAS LAr Optical Link. 3The link RX on the ROD

8 channel optical link integrated on one ROD motherboard.

Challenges:Heat dissipation; Noise, cross talks; ORx sensitivity.

OTx power4 to 7 dBmRequired atORx 17 dBmLink optical power budget


Lessons That I LearnedIn the R&D phase.Giga bit per second serial data transmission is non-trivial, especially in radiation environment, where parts are not readily available from Digikey.com.Fiber optics and the electrical/optical interface have a lot of issues that need special expertise. The above two require experts and special equipment. The price is very high if one only has people train-on-the-job; and much higher if one loses those trained-on-the-job.A system level design will certainly be much better than LEGO building. But this requires people with expertise of such a system to generate a system specification, in particular:Clock jitter requirement.Optical power budget, TX power and RX sensitivity. Reliability requirement and the information from components. R&D should end with a production ready design and prototype. Extending R&D into production will cause problems in system reliability.


Lessons That I LearnedIn the integration phase.Integration of the link with up- and down-stream electronics needs to be investigated before hand. Debugging may be very difficult if that has not been thought about before the integration.A clearly defined and communicated system specification will help the integration. A carefully designed mezzanine board may actually help in integration and operation.


Lessons That I LearnedIn the production and QA phase.The production volume is very small (about 2000 units for each part). We spent a lot of time to learn the process, such as burn-in of the OTx. We learned the hard way that optical coupling and connector assembly are not a job for amateurs. We should now stay away from connectors such as ST and SC, especially those connectors are put on manually. We invented our QA process that seemed to be okay, except that We were doing reliability measurements during and after the production. In the end, we found out that at least one serious issue with the OTx escaped our attention. Overall QA and system reliability are big issues we often do not actually know much about. A way out: build in redundancy.


Lessons That I LearnedIn early operation phase.Do not open your Champagne too soon. The problem with the OTx: the VCSELs from TrueLight fail at a few percent level in the past ~1 year of operation. We tried many things (humidity, magnetic field, ESD, EOS, defects in VCSEL fabrication) to understand the cause of that. This exercise is beyond our knowledge, equipment, and often very expensive. As of now, we still do not have a definite answer to the cause(s) of our problem, although we excluded several things and narrow down to ESD, EOS and fab. defects: Please see Stefan Simions talk. Lesson: HiRel is not easy. Unless you are very confident about all the parts, a design with redundancy may be a way around.


Lessons That I LearnedYet we at least have one variable that seems to have strong correlation with the failure mechanism: the VCSEL optical spectrum (OS).So far all failed VCSELs (after we started to measure the OS) have narrow OS. J.ParsonsS.Simion


Lessons That I Learned

We know ESD produces narrower OS.

We have not yet understood it from physics point of view.We are waiting for the time to tell us what to do:Replace only OTx with narrow OS. OrReplace all OTx with a new VCSEL, and different driving circuit.Replace all OTx with a new design of dual optical channels for redundancy. Forward ESD at 500V, 700V, 900V and 1100V Reverse ESD at 200V, 300V, 400V and 500V


The present OTx:The proposed dual-VCSEL OTx:The Dual OTx, a Fix With All the Constraints we Have Now.


Dual OTx Test Results .1


Dual OTx Test Results .2


Some Thoughts for Future LinksMulti-Giga bit serial data transmission is not trivial. We should not lose the expertise we now have. The entry price is very high. We do have special requirements, such as radiation tolerance, no accessibility for maintenance, etc, that cannot be met by a commercial product and design. But this does not mean that we should always go for a custom design and build. We should only do what we can do, and what we are good at, and borrow as much as possible from industry. This is the concept/beauty of the Versatile Link project.


AcknowledgementsMany work is actually done by Andy and Kent at SMU. All credit should go to them. If any statements here are wrong or offensive, I take full responsibility.

Documents

Experience with the ATLAS LAr optical link