Description and Evaluation of theFAST-NetSmart Pixel-Based Optical InterconnectionPrototype

MICHAEL W. HANEY , MEMBER, IEEE, MARC P. CHRISTENSEN, MEMBER, IEEE,PREDRAG MILOJKOVIC, MEMBER, IEEE, GREGG J. FOKKEN, MARK VICKBERG,BARRY K. GILBERT, FELLOW, IEEE, JAMES RIEVE, JEREMY EKMAN, MEMBER, IEEE,PREMANAND CHANDRAMANI, AND FOUAD KIAMILEV , MEMBER, IEEE

Invited Paper

The design, packaging approach, and experimental evaluationof the free-space accelerator for switching terabit networks(FAST-Net) smart-pixel-based optical interconnection prototypeare described. FAST-Net is a high-throughput data-switching con-cept that uses a reflective optical system to globally interconnecta multichip array of smart pixel devices. The three-dimensionaloptical system links each chip directly to every other with adedicated bidirectional parallel data path. In the experiments,several prototype smart-pixel devices were packaged on a commonmultichip module (MCM) with interchip registration accuraciesof 5–10�m. The smart-pixel arrays (SPA’s) consist of clusters ofoxide-confined vertical-cavity surface-emitting lasers and pho-todetectors that are solder bump-bonded to Si integrated circuits.The optoelectronic elements are arranged within each cluster on acheckerboard pattern with 125-�m pitch. The experimental globaloptical interconnection module consists of a mirror and lens arraythat are precisely aligned to achieve the required interchip parallelconnections between up to 16 SPA’s. Five prototype SPA’s wereplaced on the MCM to allow the evaluation of a variety of inter-chip links. Measurements verified the global link pattern acrossseveral devices on the MCM with high optical resolution andregistration. No crosstalk between adjacent channels was observedafter alignment. The I/O density and efficiency results suggest

Manuscript received September 13, 1999; revised March 10, 2000. Thiswork was supported by the Defense Advanced Research Projects Agency(through the Air Force Wright Laboratory) under Contract F33615-97-C-1054.

M. W. Haney, M. P. Christensen, and P. Milojkovic are with GeorgeMason University, Fairfax, VA 22030 USA.

G. J. Fokken, M. Vickberg, and B. K. Gilbert are with the Mayo Founda-tion, Rochester, MN 55905 USA.

J. Rieve is with the University of North Carolina at Charlotte, Charlotte,NC 28223 USA.

J. Ekman, P. Chandramani, and F. Kiamilev are with the University ofDelaware, Newark, DE 19716 USA.

Publisher Item Identifier S 0018-9219(00)05253-1.

that a multi-terabit/s switch module that incorporates globaloptical interconnections to overcome conventional interconnectionbottlenecks is feasible.

Keywords—Optical interconnections, smart pixels.

I. INTRODUCTION

A. Background

Future multiprocessor computing and communicationssystems must overcome the limitations of traditional metallicbuses and backplanes to achieve high overall performance.Smart-pixel arrays (SPA’s), in which high-density silicon(Si) electronics are integrated with two-dimensional (2-D)arrays of high-speed gallium arsenide (GaAs) microlaser/de-tector arrays, show promise as an enabling technology fornew interconnection architectures that avoid interprocessorcommunications bottlenecks. The free-space accelerator forswitching terabit networks (FAST-Net) concept [1]–[3] usessmart-pixel-based free-space optical interconnects (FSOI’s)to effect a multiprocessor interconnection fabric. Theconcept provides internal bisection bandwidths (BSBW’s)[4] in the Tbit/s regime while incorporating enough flex-ibility to handle a wide range of interconnect topologies.Previous analytical results showed that such an approachmay provide performance advantages over metallic andmicrooptical approaches [5]. It has been shown that the highinternal BSBW of theFAST-Netapproach can be efficientlypartitioned using topological transformations afforded bythree-dimensional FSOI to map any type of point-to-pointinterconnection requirement [6] onto the fabric.

0018–9219/00$10.00 © 2000 IEEE

PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000 819

Fig. 1. In theFAST-Netconcept, a multichip smart pixel array is linked to itself in a global opticalinterconnection pattern using a lens array and mirror. The optical rays represent data paths by whichclusters of VCSEL’s and detectors are imaged onto similar clusters on different chips. Each chip isconnected to every other in this manner. An interface at the edge of the multichip substrate enableshigh bandwidth into, and out of, the module. The insert depicts a magnified view of the I/O pattern forone of the SPA chips—partitioned into 16 clusters of VCSEL’s (dots) and photodetectors (squares).

B. FAST-Net Concept Overview

The FAST-Netapproach is depicted in Fig. 1. An arrayof SPA’s is packaged on a common planar substrate, suchas a multichip module (MCM) or printed circuit board(PCB). The SPA array is linked to itself though an opticalsystem comprising a lens array and mirror. Each SPA is ahybrid Si-GaAs device in which the Si electronic chip isbump-bonded to a GaAs optoelectronic array of emittersand detectors. The GaAs IC contains an interleaved arrayof vertical-cavity surface-emitting lasers (VCSEL’s) andMSM photodetectors (PD’s). The CMOS chip containsthe drivers, receivers, and digital logic associated with therouting, electronic I/O, and computational elements of thearchitecture. The optoelectronic I/O elements are arrayed ina grid of clusters of VCSEL’s and photodetectors. Previousexperiments demonstrated sufficient optical resolution andregistration across the entire multichip array to accom-modate element-to-element spacing as small as100 mwithin clusters [2], [3]. Each cluster may eventually containmany VCSEL’s and photodetectors operating at rates of1Gbit/s—leading to a large aggregate bandwidth betweeneach pair of chips in the array. With proper optical design,there is potential for a massive amount of internal BSBW inthe system depicted in Fig. 1. For example, if each of the 16chips in Fig. 1 contained 1024 VCSEL’s and photodetectorsrunning at link rates of 1 Gbit/s, the aggregate optical inputand output contributed by each chip would be 1 Tbit/s. With16 of these smart pixel arrays linked in a fully connectedpattern, half of their aggregate bandwidth crosses any

bisection boundary. Therefore, the BSBW of the opticalmodule would be 8 Tbits/s.

In previous research, a series of laboratory trials wereconducted to validate key optomechanical aspects of perfor-mance and packaging for the approach. An optomechanicalalignment method was refined using photolithographicmasks to emulate the VCSEL/PD arrays [7]. These initialexperiments proved that high registration and resolutioncould be achieved with theFAST-Netglobal optical system.The interconnection system was shown to register a well-fo-cused simulated multichip VCSEL/PD array to within

10 m across an 10 cm plane. Another key step inthe experimental demonstration of theFAST-Netconceptwas the incorporation of a multichip array of VCSEL/PDarrays [1]. These experiments validated the ability of theoptical system to be aligned to multiple active VCSEL’sand photodetectors that are precisely positioned in theplane while achieving high overall optical efficiency andlow optical crosstalk [2], [3]. The results proved that theresolution and registration of theFAST-Netsystem weresufficient to handle the simultaneous requirements of largeinterchip distances and the small interelement spacing of theVCSEL/PD arrays.

The focus of the present experiments is on the incorpo-ration of a packaged multichip array of the first generationof fully functional SPA’s into an optomechanical prototype.To this end, theFAST-Netexperiments integrate elementsof advanced electronics, optoelectronics, mechanics, andoptomechanics—with each contributing to the overall

820 PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000

packaging and interconnection performance considerations.Section II describes the various packaging/alignment ele-ments used in the experimental prototype. These includethe multichip SPA packaging approach, smart-pixel arrayfunctionality, and global optomechanical subsystem. Theexperimental results from the first operation of the prototypeare presented in Section III. These include the implemen-tation of the high-registration-accuracy SPA-on-MCMpackaging technique, the operation of the fully integratedVCSEL-based SPA’s, and the integration of a multichiparray of the prototype SPA devices into the global opticalinterconnection system. Section IV is the conclusion, whichsummarizes and highlights the significance of the results anddiscusses the next steps in the development of theFAST-Netconcept for data-switching applications.

II. FAST-NET PROTOTYPEDESCRIPTION

A. Multichip Packaging Approach

The currentFAST-Netdemonstration system incorporatesan MCM that contains up to 16 precisely positioned SPA’s.Since postassembly SPA-to-SPA alignment is fixed, strin-gent SPA-to-SPA alignment tolerances (for all six degrees offreedom) were required during the assembly process. Initialefforts evaluated the feasibility of putting fiducial markson the MCM and aligning each SPA to its correspondingfiducial marks. However, marks with the required precisionwere available only in the higher cost deposited thin-filmMCM technologies, and one of the goals of this effort wasto simplify (and lower the cost of) the electrical packaging.Therefore, a method was developed whereby the firstSPA was placed on the MCM with rough tolerances andsubsequent SPA’s were placed with high-precision accuracyrelative to the first SPA. By relaxing the MCM tolerancesand by spreading the application-specific integrated circuit(ASIC) perimeter bond pad I/O to the largest pitch possibleon the SPA, laminate MCM (MCM-L) became a viablepackaging solution. By using “micro-vias” on the top layer,the 11.8-mil-pitch SPA I/O could be further flared out to a15-mil pitch that was compatible with the four inner layersthat utilized standard PCB technology and through-holevias.

A 62-mil-thick MCM is used. It is compatible with anAMP Mictor 767006-4 edge mount connector, which hascontrolled-impedance signal pins on a 25-mil pitch on boththe top and bottom edges of the MCM (12.5-mil effectivesignal pitch). In Fig. 2, the Mictor connector couples to a ver-tical adapter board that is the interface between the high-den-sity Mictor connector on the MCM and test equipment ca-bles. The six-layer MCM design utilizes two inner routing(i.e., signal) layers that have minimal coplanar and broadsideelectrical coupling. This is important since the two signallayers approached 100% routing density on the connectorside of the MCM. All SPA-to-SPA data paths are optical.However, a considerable number of data and control sig-nals are routed independently to each SPA to allow flexi-bility in characterizing the various optical paths. Fig. 2 shows

Fig. 2. FAST-NetMCM assembly, showing the substrate andelectronics interface. The substrate is shown with 16 inactivesurrogate CMOS chips attached.

a mechanical assembly of the MCM (populated with sur-rogate SPA’s) along with the adapter board (the verticallymounted PCB) and the aluminum base plate assembly. Byimplementing this adapter board approach, test cables canbe attached to the adapter board and removed as one unitif MCM modifications were needed (i.e., SPA’s may be in-crementally added as they became available). Also, this ap-proach gives the option of designing an adapter board thatwould self-test the populated MCM. High-speed signal inter-connects on the adapter board and SPA are 50-differentialpairs designed to operate at up to 1-Gbit/s PECL levels. Con-trol lines are single-ended and designed to operate at 50-MHz3.3-V CMOS levels. The top layer of the MCM is gold plated(to be compatible with wire bonding of the SPA’s) with sol-dermask openings around each SPA to allow for electricalprobing in close proximity to the SPA’s, if needed.

B. Smart-Pixel Devices

The high smart-pixel I/O density of theFAST-Netapproach requires VCSEL’s that have lower power dis-sipation, higher efficiency, better temperature stability,and greater array uniformity and yield than previouslyconsidered proton-implanted devices. Recently devel-oped oxide-confined 850-nm VCSEL’s have exhibitedthe required performance [8]. This new generation ofVCSEL’s has low threshold currents of0.3 mA ( 10% ofproton-implanted VCSEL’s), increased wall-plug efficiencyof 22% ( 350% of proton-implanted VCSEL’s), improvedwafer-scale device uniformity (50% of the variation foundin proton-implanted VCSEL arrays), and better temperaturestability ( 50% of the variation found in proton-implantedVCSEL arrays) [8].

In theFAST-Netprototype SPA’s, the VCSEL’s and PD’sare arranged into 16 clusters of four pairs of emitters and de-tectors, as depicted in Fig. 1. Each cluster is part of a unitcell that has dimensions of 500 500 m . Within the unitcell cluster the elements are spaced by 250m horizontallyand 125 m vertically. The unit cells in theFAST-NetSPA’sare arrayed in a 4 4 grid with a center-to-center spacing of

HANEY et al.: DESCRIPTION AND EVALUATION OF THEFAST-NETSMART PIXEL-BASED OPTICAL INTERCONNECTION PROTOTYPE 821

Fig. 3. Schematic illustration of theFAST-Net smart pixelintegration [2], [8] and MCM packaging approach.

Fig. 4. Layout ofFAST-NetASIC. The central region of the chipcontains area pads for bump bonding the GaAs optoelectronic arrayon top of the device, as well as the associated driver and receivercircuits. The outer lower regions contain four crossbars for linkinginputs and detectors to outputs and VCSEL’s. The central lowerportion contains the control circuits for the ASIC.

1 mm. Fig. 3 depicts the hybrid integration [9] and multi-SPApackaging approach used in theFAST-Netexperiments. Theintegration approach uses a transparent superstrate to pro-vide a robust support for the GaAs VCSEL’s and MSM PD’s.As shown in the figure, the devices are processed to providesolder-bump contacts to the Si-ASIC—thus completing thesmart pixel structure. In theFAST-Netprototype, an MCMthat is capable of holding up to 16 SPA’s is used.

Fig. 4 is the layout of the mixed-signal CMOS ASIC usedin theFAST-NetSPA prototypes. The chip is the source/sinkfor data to/from the GaAs VCSEL/PD device onto which itis bump bonded. The ASIC performs switching, data pat-tern generation, synchronization, and detection. The devicewas fabricated by the MOSIS foundry service in the HP14G

Fig. 5. Schematic depiction of alignment marks and area padperimeter of theFAST-NetASIC (not to scale). The square perimeterdelineates the maximum extent of the GaAs array in each direction.The centering of the pattern in the array will cause it to extend intwo of these directions greater than the others.

0.5- m technology. The design incorporated standard cellsas well as several custom analog driver and receiver circuits.

As there is uncertainty in the position of the VCSEL/PDpattern on the final diced GaAs device, theFAST-NetASICmust be large enough to maintain the CMOS perimeter padframe around the GaAs die for all possible pattern configura-tions. Fig. 5 shows the alignment marks (in the corners) andGaAs placement parameters of the CMOS design.

The driver circuit uses the same circuit topology as in thefirst FAST-Netdemonstration [1]–[3]. In this demonstration,the driver maximum current output is 10 mA. The actualVCSEL devices will nominally operate at lower drive levels;the 10-mA capability was a conservative design measure.The CMOS receivers in this design are based on a provencircuit topology [10].

C. Optomechanical Packaging

Fig. 6 is a drawing of the optomechanical interconnectionmodule and attached MCM baseplate used for the prototypedemonstration. The optomechanical assembly is mountedabove the MCM/baseplate assembly on four posts andcomprises three main parts: lens insert, insert holder, andmirror holder with mirror. The lens insert holds 16 lenses(mounted within individual barrels) fixed in a commonplane on a regular rectangular grid at a spacing determinedby the pitch of the SPA’s. The 4 4 lens array measuresabout 7 7 cm in its plane. The 16 f/1.1 lenses used in theprototype have a low-aberration seven-element design andare matched to within 1% in focal length. The lens insertfits into the insert holder, which is held on four vertical postsby set-screws. This allows adjustment of the height of the

822 PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000

Fig. 6. Schematic diagram ofFAST-NetMCM, adapter board, and optomechanical module.

insert holder above the MCM plane such that the VCSEL/PDarrays on the SPA’s are in the back focal plane of the lenses.The lens-array insert assembly can be moved laterally androtationally by a sufficient amount to achieve alignment withthe SPA MCM. This feature was provided so that globaltranslation and rotation of the MCM could be compensated.The individual lens holders on the insert assembly providecompensation for translational positional misalignmentof any individual SPA chip on the MCM, as well as forthe tolerances in the diameter of the lens barrels. Thesemisalignments could result either from slight inaccuraciesof the pick-and-place machine used to mount the SPA’s orfrom thermally induced SPA positional shifts that occurduring the heating and cooling phases of the epoxy curingprocess. Since the SPA thickness may vary from die to die,and the focal lengths of the lenses are not perfectly matched,the height of each lens may also be individually adjusted inthe lens holder assembly. This “fine tuning” of lens heightis used to assure that each SPA is positioned precisely in theback focal plane of its respective lens. Fig. 7 is a photographof the lens array and its mechanical assembly.

The mirror is positioned above the lens array to fold thesystem back upon itself. The mirror is located approximately

Fig. 7. Photograph of theFAST-Netlens-array assembly as viewedfrom the top when the mirror is removed. One of the lenses isremoved to reveal the MCM substrate underneath.

HANEY et al.: DESCRIPTION AND EVALUATION OF THEFAST-NETSMART PIXEL-BASED OPTICAL INTERCONNECTION PROTOTYPE 823

10 cm above the MCM plane, indicating that the total free-space path length for any of the VCSEL/PD links is at least20 cm. The height of the mirror can be adjusted by slidingit along the four posts and then fixing its position with setscrews. The resulting optomechanical assembly has an ap-proximately cubic shape, due to the steep angles of rays inthe f/1.1 optical system and the folded optical paths due tothe mirror. Each lens has a field of view of approximately 8

8 mm to accommodate the placement of the SPA’s. SPA’sare placed within this field of view at a position that resultsin a fully interconnected system, i.e., every SPA is connectedto every other through the optical system.

III. EXPERIMENTAL RESULTS

A. Smart-Pixel MCM Assembly

The FAST-NetMCM assembly was conducted in a pro-totyping lab where each step of the process is performedby manual or semiautomatic means. The MCM assemblyprocess included the following steps: manual solder pastedispense and reflow for the AMP Mictor edge connector,manual conductive epoxy dispense and pick-and-place forall passive components, and precision die placement usinga modified flip chip bonder (Karl Suss FC150) for the crit-ical placement of the individual SPA’s.

During a typical flip-chip assembly, the die was presentedin a face-down position relative to the MCM substrate towhich it was to be bonded. Before die placement and bondingoccurs, alignment was accomplished by inserting a verticalsplit-field microscope between the die and MCM substrate.Since this assembly required the SPA’s to be placed in aface-up configuration, modifications to the bonder’s controlsoftware and die handling system were required. The modi-fied bonder now uses separate prealignment steps for the dieand substrate that were aligned relative to a set of cross hairspresent on the bonder’s video alignment system.

The first SPA to be placed was roughly aligned to align-ment marks on the MCM. Subsequent SPA’s were then lo-cated by first aligning to the reference SPA, then offsettingto a predetermined- position for final location and die at-tach to the MCM.

The SPA array consists of individual hybrid bump-bondedgallium arsenide/silicon SPA’s placed on a 4 4 arrayusing a PCB as the MCM substrate. With this multimaterialstackup, the major contributor to relative die movement, dueto thermal expansion from ambient to operating temperature,lies within the MCM substrate. Determination of die bondtarget positions required compensation for this movementduring adhesive cure and later operational temperatureranges. Since normal manufacturing tolerances of PCBtechnologies make it impossible to create alignment markswith micrometer resolution, two preplaced die were usedto characterize the MCM’s thermal expansion propertiesover these temperature ranges. Measurement results basedon tracking relative movement of alignment marks on thesetwo test die were used to estimate the required offset for theremaining bond sites on the MCM. The remaining die bond

Fig. 8. FAST-Net MCM assembly. Six of the 16 sites arepopulated with prototype smart pixel devices. The interchipregistration accuracy is<10�m across the substrate.

operations yielded a postbond alignment accuracy within5 m for die directly adjacent to the reference die and lessthan 10 m for a further removed location. The initial diepositions used for this characterization were chosen basedon testing requirements that resulted in the reference dielocation at the edge of the assembly rather than in the center.

The SPA alignment and die bonding operation was con-ducted at a stage temperature of 80C. With this temper-ature, the preplaced adhesive allows a 10-min window forthe manual multistep alignment and die attach process to becompleted. In order to minimize die and substrate expan-sion/movement normally experienced during a temperaturerampup from alignment to cure temperature, the stage tem-perature was fixed. This resulted in the long cure time afterSPA placement. The die and substrate were each held in placeby vacuum until the adhesive was completely cured. This ap-proach was designed to give maximum control at each step ofthe process. The addition of an automatic dispensing systemand a pattern recognition vision system, for automatic align-ment, will greatly speed up this process for the productionenvironment.

Given the experimental results using the modifiedflip-chip bonder, along with characterization of a fullypopulated MCM assembly, and moving the reference dieposition to the center of the MCM substrate, the alignmenttolerances of additional assemblies should fall within the2–5 m range. Early indications also showed that the align-ment tolerances are consistently off in the positiveandnegative directions. Taking into account this additionalrepeatable error should add additional improvement, and thedescribed approach should be capable of achieving 2-malignment and bonding accuracy.

Fig. 8 is a photograph of theFAST-NetMCM. Six of the 16chip sites were populated with prototype smart-pixel devicesat various levels of integration. As described above, the in-terchip registration accuracy achieved was5 m for neigh-boring devices and 10 m for devices that were widelyseparated. Neighboring devices are1.7 cm apart, while the

824 PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000

Fig. 9. Closeup photograph of one of theFAST-Netsmart pixelarrays on the MCM with all functioning VCSEL’s illuminated.The 500-�m-thick transparent superstrate causes double images ofthe lower VCSEL clusters to be visible through the side and topsimultaneously.

most widely separated devices are5.2 cm apart. The fol-lowing section details the experimental results for the testsof the prototype SPA’s.

B. Smart-Pixel Tests

The smart-pixel tests for the demonstration described inthis paper occurred in three phases. In the first phase, anelectrical-only ASIC was placed on the MCM. This phasefacilitated the debugging of the demonstration I/O and thesystem control, and validated the ASIC functionality. TheASIC was designed with a loopback mode, which connectedthe digital output of the detector to the digital input of theVCSEL driver. This made possible the verification of theentire signal path except the optical transmission. This wasdesigned to provide a baseline for optical interconnectioncomparisons. When the ASIC-only tests were completed,the second phase incorporated a single VCSEL-only SPAand two detector-only SPA’s on the MCM. Their positionsare shown in Fig. 8. This test allowed for the evaluationof two separate simultaneous optical links (between theVCSEL-only and the two detector-only SPA’s) with theguarantee that there was no on-chip transmitter and receivercoupling. Finally, two full SPA’s were added to the MCM.A picture of one of the full SPA’s is shown in Fig. 9. A4-mm-square sapphire superstrate is centered on the 6-mmCMOS ASIC that is in turn wire-bonded to the MCM.Decoupling capacitors are evident on the MCM around theperiphery of the SPA. A magnified view of a single cluster ofthe full SPA is shown in Fig. 10. The GaAs arrays containeda repeating checkerboard pattern, of which only selectedclusters of devices were used in this demonstration. Thedevices used as a cluster are labeled in Fig. 10.

C. Interconnection Module Characterization

Previous characterization of the optical system with testmasks and wire-bonded VCSEL/PD arrays [1]–[3], [7] vali-dated the interconnection module’s ability to achieve the re-

Fig. 10. Magnified view of a single cluster of VCSEL’s Theelement separation is 250�m along the long dimension (containingVI and V2) and 125�m along the orthogonal dimension. Theassociated four-element PD array is located in the darker strip to thelower right of the VCSEL array.

Fig. 11. Photograph of theFAST-Net optical interconnectionmodule, consisting of the smart pixel MCM assembly (shown inFig. 2) attached to the optical interconnection subassembly.

quired registration and resolution across the multichip pat-tern of VCSEL/PD clusters. In the current experiment, the in-terconnection module is configured to be integrated with theSPA-array-on-MCM subassembly shown in Fig. 8. Fig. 11is a photograph of the completeFAST-Netinterconnectionmodule.

When fully populated, theFAST-Netoptical interconnec-tion module would implement 136 (16 15 1)simultaneous chip-to-chip links between the 16 chips on theMCM. Each of these 136 bidirectional links contains eight

HANEY et al.: DESCRIPTION AND EVALUATION OF THEFAST-NETSMART PIXEL-BASED OPTICAL INTERCONNECTION PROTOTYPE 825

VCSEL’s and eight detectors in this demonstration module.In this prototype module, each SPA has an aggregate band-width capability of input and output totaling 16 Gbit/s (64VCSEL’s at 250 Mbit/s). Therefore, even at this conservativedata rate, if this prototype module was fully populated, itwould contain 128 Gbit/s of BSBW. In an eventual system,each interchip parallel link would contain128 VCSEL’sand detectors (64 bidirectional links), resulting in a BSBWof 8 Tbit/s at a data rate of 1 Gbit/s. For the current imple-mentation, there were only enough fabricated hybrid SPA’sto partially populate the module. The candidate deviceswere evaluated for functional VCSEL’s and detectors, andpositions within the array were chosen to maximize thenumber of overall links. Fig. 8 shows the partially populatedMCM utilized in this experiment. The MCM contains oneelectrical-only ASIC, one VCSEL-only hybrid SPA, twodetector-only SPA’s, and two full VCSEL/detector SPA’s.In this partially populated module, there are 11 chip-to-chiplinks, some bidirectional and some unidirectional, whereeach direction of a parallel link between SPA’s contains fourVCSEL’s and four detectors.

Over 1/3 of the possible chip-to-chip links in the partiallypopulated module were verified to be aligned. All of the in-spected links demonstrated digital receiver triggering. No op-tical crosstalk was observed between parallel links within agiven chip-to-chip interconnection. Optical confinement wasobserved to be as tight as previous work with this approachhad indicated [3]: approximately 90% of the incident lightwas captured by a 50-m-square detector.

IV. CONCLUSION

In summary, theFAST-Netprototype was successfullydemonstrated. It incorporated a multichip array of the firstgeneration of fully integrated VCSEL-based smart pixelsinto a global optical interconnection system that is capableof implementing a high-BSBW fabric. Several smart pixeldevices were packaged on a common MCM substrate witha registration accuracy of 10 m—sufficient to maintainthe required interchip optical alignment of closely spacedclusters of VCSEL’s and photodetectors. A macro-opticalglobal system achieved the multichip cluster links withoutmeasurable optical crosstalk. Each smart-pixel device’soptoelectronic array was an 8 8 array of VCSEL/pho-todetector pairs arranged in tightly packed clusters of fourpairs each. The interelement pitch met the requirementsfor high-density smart-pixel arrays. The successful inter-chip interconnections of VCSEL clusters with intraclusterelement spacing of 175 m suggests that the eventualhigh-density SPA technology, projected to achieve1000VCSEL/PD pairs per cm, will be readily incorporatedwith the FAST-Netapproach. With each of these 16 chipsrunning at link rates of 1 Gbit/s, the aggregate optical inputand output contributed by each chip would be 1 Tbit/s.Therefore, the BSBW of the optical module would be 8Tbits/s.

The ability of theFAST-Netmodule to link each smart-pixel chip in a fully connected high-BSBW network sug-

gested that the concept will be ideally suited to low-latencyapplications. In theFAST-Netapproach, only one stage ofoptical interconnection is needed between any pair of nodes.The remainder of the interconnection function is maintainedin local intrachip metallic links. TheFAST-Netconcept thusoptimally combines global optical and local metallic fabricelement to achieve the minimum latency. This feature willbe especially significant for applications requiring internodetraffic with very small message sizes.

The successful integration of optomechanics, chip pack-aging, and SPA integration into a complete module is acritical step in the validation of theFAST-Netconcept.The strength of the concept stems from the use of asinglemultichip SPA substrate. The reflective global FSOI systemleverages the single multichip plane approach by removingthe excess degrees of freedom (DoF) that would be presentbetween each pair of planes of a multiplane architecture.The FAST-Netapproach minimizes the DoF necessary toalign the optomechanical module and thereby facilitates anautomated alignment procedure—critical for the eventualmanufacturing of such optoelectronic systems.

The high internal bisection bandwidth of theFAST-Netmodule may be exploited only through the incorporation ofan effective interface, through which high-speed nodes canaccess the high BSBW core via the internal routing logic onthe SPA’s. Future experimental validation of theFAST-Netconcept will focus on this interface bandwidth issue. Theincorporation of a high-density interface will be facilitatedby the macro nature of theFAST-Netmodule, which is wellmatched to several high-density board-edge interface tech-nologies. As integrated VCSEL-based smart-pixel SPA andrelated interface technology matures,FAST-Netwill exploitthem to provide Tbit/s BSBW capacities for future multipro-cessor systems.

REFERENCES

[1] M. W. Haney, M. P. Christensen, P. Milojkovic, J. Ekman, P.Chandramani, R. Rozier, F. Kiamilev, Y. Liu, M. Hibbs-Brenner,J. Nohava, E. Kalweit, S. Bounnak, T. Marta, and B. Walterson,“FAST-Netoptical interconnection prototype demonstration pro-gram,” inProc. SPIE, vol. 3288, Jan. 1998, pp. 194–203.

[2] , “FAST-Netoptical interconnection prototype demonstration,”in Proc. Int. Topical Meeting Optical Computing, June 1998, pp.568–571.

[3] M. W. Haney, M. P. Christensen, P. Milojkovic, J. Ekman, P. Chan-dramani, R. Rozier, F. Kiamilev, Y. Liu, and M. Hibbs-Brenner,“Multi-chip free-space global optical interconnection demonstrationusing integrated arrays of vertical cavity surface emitting laser andphotodetectors,”Appl. Opt., vol. 38, no. 29, pp. 6190–6200, Oct.1999.

[4] F. T. Leighton,Introduction to Parallel Algorithms and Architec-tures; Arrays, Trees, Hypercubes. San Mateo, CA: Morgan Kauf-mann, 1992.

[5] M. W. Haney and M. P. Christensen, “Performance scalingcomparison for free-space optical and electrical interconnectionapproaches,”Appl. Opt., vol. 37, no. 14, pp. 2886–2894, May 1998.(Special Issue on Optics in Computing).

[6] M. P. Christensen and M. W. Haney, “Two-bounce optical arbitrarypermutation network,”Appl. Opt., vol. 37, no. 14, pp. 2879–2885,May 1998. (Special Issue on Optics in Computing).

[7] R. R. Michael, M. P. Christensen, and M. W. Haney, “Experimentalevaluation of the 3-D optical shuffle interconnection module ofthe sliding banyan network,”J. Lightwave Technol., vol. 14, pp.1970–1978, Sept. 1996.

826 PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000

[8] Y. Liu, “Smart pixel module development for free space optical in-terconnect,” inProc. Int. Topical Meeting Optical Computing, June1998, pp. 528–531.

[9] Y. Liu, M. Hibbs-Brenner, R. A. Morgan, J. Nohava, B. Walterson,T. Marta, S. Bounnak, E. Kalweit, J. Lehman, D. Carlson, and M.Wilson, “Integrated VCSELs, MSM photodetector, and GaAs MES-FETs for low cost optical interconnects,” inOptics in Comput., vol.8, Mar. 1997, pp. 22–24. paper SMB3.

[10] A. V. Krishnamoorthy, J. E. Ford, K. W. Goossen, J. A. Walker, A.L. Lentine, S. P. Hui, B. Tseng, L. M. F. Chirovsky, R. Leibenguth,D. Kossives, D. Dahringer, L. A. D’Asaro, F. Kiamilev, G. F. Aplin,R. G. Rozier, and D. A. B. Miller, “Photonic page buffer based onGaAs MQW modulators bonded directly over active silicon CMOScircuits,” Appl. Opt., vol. 35, no. 14, pp. 2439–2448, May 1996.

Michael W. Haney (Member, IEEE) receivedthe B.S. degree in physics from the University ofMassachusetts, Amherst, in 1976, the M.S. de-gree in electrical engineering from the Universityof Illinois, Urbana, in 1978, and the Ph.D. degreein electrical engineering from the CaliforniaInstitute of Technology, Pasadena, in 1986.

From 1978 to 1986, he was with General Dy-namics, where his work ranged from the develop-ment of electrooptic sensors to research in pho-tonic signal processing. In 1986, he joined BDM

International, Inc., where he became a Senior Principal Staff Member andthe Director of Photonics Programs. In 1994, he joined George Mason Uni-versity as an Associate Professor of electrical and computer engineering.His research activities are focused on the application of hybrid optoelec-tronics to free-space optical interconnection architectures for switching andsignal-processing applications. He has contributed to more than 50 journaland conference papers in optical information processing. He has chaired orcochaired several technical conferences.

Dr. Haney is a previous Chairman of the IEEE Communications Society’sTechnical Committee on Interconnections within High-Speed Digital Sys-tems. He is a Fellow of the Optical Society of America and a member of theIEEE Communications and Lasers & Electro-Optics Societies.

Marc P. Christensen (Member, IEEE) receivedthe B.S. degree in engineering physics from Cor-nell University, Ithaca, NY, in 1993 and the M.S.degree in electrical and computer engineeringfrom George Mason University, Fairfax, VA, in1998. He is currently pursuing the Ph.D. degreein electrical and computer engineering at GeorgeMason University.

From 1991 to 1997, he was with BDM Inter-national, where his work involved the design andimplementation of acoustooptic signal processors

and free-space optical interconnection architectures. In 1997, he joined Ap-plied Photonics, where he currently develops VCSEL-based smart pixel ar-chitectures. He has contributed to more than 30 journal and conference pa-pers in optical information processing.

Mr. Christensen is a member of OSA. He has served on the programcommittee for the IEEE-sponsored Workshop on Interconnections withinHigh-Speed Digital Systems for the past five years.

Predrag Milojkovic (Member, IEEE) receivedthe diploma in electrical engineering from theUniversity of Belgrade, Yugoslavia, in 1978.He is currently pursuing the Ph.D. degree inelectrical and computer engineering at GeorgeMason University, Fairfax, VA.

From 1978 to 1988, he was with ElectronicIndustry Corp., Belgrade. From 1988 to 1993,he was with Institute for Microwave Techniquesand Electronics, Belgrade, where he worked onthe design of microwave systems and devices. In

1996, he joined BDM Federal, where he conducted research in free-spaceoptical interconnects. He is currently with Applied Photonics, Inc., wherehis current research interests include architectures for free-space opticalinterconnection networks and optical systems for such networks. He hascontributed to seven technical conference papers and journal articles.

Mr. Milojkovic is a member of OSA.

Gregg J. Fokkengraduated from Bemidji StateUniversity in 1987.

Since joining the Special Purpose ProcessorDevelopment Group (SPPDG) at the MayoFoundation in June 1987, he has been involved indesign; electromagnetic, circuit, and functionalmodel development and simulation; layout; test;and analysis of integrated circuits and integratedcircuit single and multichip packaging. He hasworked closely with multigigahertz, semicustomintegrated circuit technologies including com-

plementary metal–oxide semiconductors, silicon-on-insulator, galliumarsenide, indium phosphide, and silicon germanium. Mr. Fokken has servedin a lead role for computer-aided design support for integrated circuit andpackaging design, has been involved in laboratory automation, and hasserved as a Project Engineer on numerous projects. He is currently leadingthe SPPDG simulation team and has research interests in design and signalintegrity analysis of multigigahertz digital and mixed-signal integratedcircuits, packaging, and subsystems.

Mark Vickberg received the A.S. degree in engi-neering curriculum from Rochester Communityand Technical College, Rochester, MN, in 1998.He is currently pursuing the B.S.E.E. degree atthe University of North Dakota.

From 1984 to 1986, he was a Service Techni-cian with Movie Systems, Inc. His activities in-cluded service and design of single- and multiple-dwelling microwave, satellite, and cable TV an-tenna distribution systems. From 1986 to 1992, hewas a Research Technician with Honeywell, Inc.,

with primary assignments in packaging and RF testing of GaAs MMIC andhybrid circuits. He is currently an assistant Engineer in the Special PurposeProcessor Development Group, Mayo Foundation, Rochester, MN. His cur-rent work activities emphasize high-precision assembly development for op-tical backplane free-space interconnect systems and flip chip developmentfor multiple MCM and semiconductor technologies used for microwave andmillimeter-wave digital communications circuits.

HANEY et al.: DESCRIPTION AND EVALUATION OF THEFAST-NETSMART PIXEL-BASED OPTICAL INTERCONNECTION PROTOTYPE 827

Barry K. Gilbert (Fellow, IEEE) received theB.S. degree in electrical engineering from PurdueUniversity, West Lafayette, IN, and the Ph.D.degree in physiology and biophysics (minors inelectrical engineering and applied mathematics)from the University of Minnesota, St. Paul.

He is a Staff Scientist in the Departmentof Physiology and Biophysics of the MayoFoundation, Rochester, MN, and Director of theSpecial-Purpose Processor Development Group.His research interests include the development of

algorithms for the real-time analysis of wide bandwidth image and signaldata; the design of specialized signal-processing computers to execute thesetasks; the development of computer-aided design tools to allow the timelydesign of high-complexity digital signal processors; the advancement ofhigh-performance integrated circuit technologies, such as gallium arsenideand indium phosphide, that can be used to assemble very-high-performancesignal processors; and the development of advanced electronic packagingtechnologies such as MCM’s, which will be capable of supporting digitalintegrated circuit-based processors operating at gigahertz system clockrates.

James Rieve received the M.S.E.E. degree(magna cum laude)from the University of NorthCarolina at Charlotte, where he is currentlypursuing the Ph.D. degree.

From 1996 to 1998, he was a Member ofthe Technical Staff at the MicroelectronicsDivision, Lucent Technologies, Allentown, PA.His research interests center on optoelectronic ICdesign for high-speed networks and distributedcomputing systems.

Jeremy Ekman (Member, IEEE) received thebachelor’s and master’s degrees in electrical en-gineering from the University of North Carolinaat Charlotte. He currently is pursuing the Ph.D.degree at the University of Delaware, Newark.

He is working in the area of CMOS VLSI andsystems design for free-space optically intercon-nected systems.

Premanand Chandramani received the B.E.degree in electrical and electronics engineeringfrom Annamalai University, India, in 1995and the M.S.E.E degree from the Universityof North Carolina at Charlotte in 1999. He iscurrently pursuing the Ph.D. degree in electricalengineering at the University of Delaware,Newark.

His work is focused on optoelectronic VLSI ICdevelopment.

Fouad Kiamilev (Member, IEEE) receivedthe B.A. degree in computer science and theB.S.E.E. (Hons.) degree, the M.S.E.E degree,and the Ph.D. degree in electrical engineeringfrom the University of California at San Diegoin 1988, 1990, and 1992, respectively.

He is currently an Associate Professor in theElectrical and Computer Engineering Depart-ment at the University of Delaware, Newark.Over the past eight years, he has established aresearch group focused on optoelectronic VLSI

IC development.Dr. Kiamilev is a member of OSA. He received the Office of Naval Re-

search Graduate Fellowship.

828 PROCEEDINGS OF THE IEEE, VOL. 88, NO. 6, JUNE 2000