Real-Time Acoustic Modeling for Distributed Virtual Environments

ThomasFunkhouserPrincetonUniversity

PatrickMinPrincetonUniversity

Ingrid CarlbomBell Laboratories

Abstract

Realisticacousticmodelingis essentialfor spatializingsoundindistributed virtual environmentswheremultiple networked usersmove aroundand interact visually and aurally in a sharedvir-tual world. Unfortunately, currentmethodsfor computingaccurateacousticalmodelsarenot fastenoughfor real-timeauralizationofsoundsfor simultaneouslymovingsourcesandreceivers.In thispa-per, we presentthreenew beamtracingalgorithmsthatgreatlyac-celeratecomputationof reverberationpathsin a distributedvirtualenvironmentby taking advantageof the fact that soundscanonlybegeneratedor heardat thepositionsof “avatars”representingtheusers.Thepriority-drivenbeamtracingalgorithmperformsabest-first searchof acell adjacency graph,andthusenablesnew termina-tion criteriawith whichall earlyreflectionpathscanbefoundveryefficiently. Thebidirectionalbeamtracingalgorithmcombinessetsof beamstracedfrom pairs of avatar locationsto find reverbera-tion pathsbetweenthemwhile requiringsignificantlylesscompu-tationthanpreviousunidirectionalalgorithms.Theamortizedbeamtracingalgorithmcomputesbeamsemanatingfrom box-shapedre-gions of spacecontainingpredictedavatar locationsand re-usesthosebeamsmultiple times to computereflectionspathsas eachavatarmovesinsidethebox. Cumulatively, thesealgorithmsenablespeedupsof approximatelytwo ordersof magnitudeover previousmethods.They areincorporatedinto a time-critical multiprocess-ingsystemthatallocatesitscomputationalresourcesdynamicallyinorderto computethe highestpriority reverberationpathsbetweenmoving avatarlocationsin real-timewith gracefuldegradationandadaptive refinement.

Key Words: Virtual environmentsystems,virtual reality,acousticmodeling,auralization,beamtracing.

1 Introduction

Distributedvirtual environment(DVE) systemsincorporatecom-putergraphics,sound,andnetworkingto simulatetheexperienceofreal-timeinteractionbetweenmultipleusersrepresentedby avatarsin a sharedthree-dimensionalvirtual world. They allow a userto“explore” informationand“interact” with otherusersin the con-text of avirtual environmentby renderingimagesandsoundsof theenvironmentin real-timewhile the user“moves” throughthe 3Denvironmentinteractively. Exampleapplicationsfor DVE systemsincludecollaborative design[4], distributedtraining[35], telecon-ferencing[29], andmulti-playergames[27].

A difficult challengein implementingaDVE systemis to renderrealisticsoundsspatializedaccordingto thevirtual environmentin

real-timeoneveryparticipatinguser’scomputer. Soundwavesorig-inatingat a sourcelocationtravel throughtheenvironmentalongamultitudeof reverberationpaths,representingdifferentsequencesof reflections,transmissions,anddiffractions.Thedifferentarrivaltimesandamplitudesof soundwaves traveling alongthesepathsprovide importantauditorycuesfor localizationof objects,separa-tion of simultaneousspeakers(i.e., the“cocktail partyeffect”), andsenseof presencein avirtual environment[11].

The goal of our work is to build a DVE systemin which mul-tiple userscancommunicatewith eachotherin a 3D virtual worldwith realisticspatializedsound.Usergeneratedsoundsandavatarmovementsaretransmittedvia network messagesto anaudioserverwhich spatializessoundsaccordingto impulseresponsesencodingthe reverberationscomputedfor every pair of avatars(seeFigure1). The new researchchallengeis to develop geometricacousticmodelingalgorithmsto computereverberationpathsbetweenev-ery pair of avatarlocationsin real-timeasthey move througha 3Denvironment.

� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �� � � � � � � � � � � � � � �

Client A

NetworkClient B

Client D

Client C

AudioServer

Figure1: Distributedvirtual environmentsystem.

Acoustic modelingalgorithmsfor DVE systemsmust be effi-cient, adaptive, predictive, and asynchronousso that appropriateimpulseresponsesarereadyfor usewithoutdelayasavatarsmove.Sincethe numbersand locationsof avatarsinhabiting the virtualenvironmentcanbehighly variable,it is notpracticalto preallocatea fixed amountof computationalresourcesto the updateof rever-berationpathsbetweenany pair of avatars. Someavatarsmay bemoving very closelyto oneanotherwithin thesameroom,requir-ing impulseresponseupdatesatmorethan10Hzto avoidnoticeablelocalizationartifacts,while othersmay be separatedby greatdis-tancesandrequirelessfrequentand/orlessaccurateacousticsim-ulations(peoplecandetectangulardifferencesin soundsourcelo-cationsseparatedby 5 degrees[3]). Similarly, the quality of thecomputedresultsshoulddegradegracefullyas the computationaldemandsincrease(e.g.,whenmoreusersinhabittheenvironment),andit shouldrefineadaptively asmoreresourcesbecomeavailable(e.g.,whenall theusersremainstationary).Timecriticalalgorithmsmustbeusedto guaranteeupdateratesandallocatecomputationalresourceseffectively andadaptively so that the overall quality ofcomputedimpulseresponsesis thebestpossible.

In this paper, we describegeometricbeamtracing algorithmsto compute� reverberationpathsbetweenmoving avatarlocationsatveryhighrates.Theprimaryresearchcontributionsareembodiedinthethreemethodswe introducefor acceleratingcomputationof re-verberationpaths:1) priority-drivenbeamtracing,2) bidirectionalbeamtracing,and3) amortizedbeamtracing. We have integratedthesemethodsinto a time-critical computingframework to builda completeDVE systemincorporatingrealistic imageryandspa-tialized soundfor real-timecommunicationbetweenmultiple net-workedusers.

2 Previous Work

Therehasbeendecadesof work in off-line acousticmodelingforapplicationssuchasconcerthall design.Previousmethodsincludepathtracing[22], imagesourcemethods[1, 5], boundaryelementmethods[26, 33], andbeamtracing [8, 18]. Surveys of work inthis areaappearin [3, 21, 23]. In general,currentoff-line systemscomputereverberationpathsfor a smallsetof pre-specifiedsourceandreceiver locations,and they allow interactive evaluationonlyfor precomputedresults. Unfortunately, it is usuallynot possibleto storeprecomputedimpulseresponsesor reverberationpathsoverall possibleavatarlocationsfor useby adistributedvirtual environ-mentsystem,asthestoragerequirementsof thisapproachwouldbeprohibitive for all casesexceptvery simpleenvironmentsor verycoarsesamplings.

Therehavealsobeenmany advancesover thelastdecadein dis-tributedvirtualenvironmentssystemssupportingvisualinteractionsbetweennetworked usersin a shared3D virtual environment(anearly exampleis Reality Built for Two [4]). The most commonexamplesincludemulti-playergames(e.g.,Quake [27]), militarybattlesimulations(e.g.,NPSNET[35]), andmulti-userchatenvi-ronments(e.g., Sony’s CommunityPlace[29]). Theseprogramsdisplayimagesin real-timewith complex global illumination andtexturesto producevisuallycompellingimmersiveexperiences.

Ontheotherhand,therehasbeenrelatively little progressin real-timeacousticmodeling.Currenton-linesystemsgenerallyconsideronly simplegeometricarrangementsandlow-orderspecularreflec-tions. For instance,the Acoustetron[13] computesonly first- andsecond-orderspecularreflectionsfor box-shapedvirtual environ-ments,while video gamesprovide spatializedsoundwith ad hoclocalizationmethods(e.g., paneffects), ratherthanwith realisticacousticmodelingmethods. Almost all DVE systemsattenuatesoundwith distance,andmany supportuser-specified“regionsofinfluence”(e.g.,ellipsoids)for eachsoundsource[16]. However,to quotethe1995NationalResearchCouncilReportonVirtual Re-ality ScientificandTechnologicalChallenges[12], “current tech-nologyis still unableto provide interactive systemswith real-timerenderingof acousticenvironmentswith complex, realistic roomreflections.”

The new algorithmspresentedin this paperare basedon thebeamtracingmethoddescribedin [14]. To review, beamsaretracedfrom the positionof eachstationarysoundsourcealongpathsoftransmissionandspecularreflectionvia adepth-firsttraversalof anadjacency graphof polyhedralcells (seeFigure2). Thealgorithmstartsin thecell containingthesourcewith abeamrepresentingtheentirecell (labeled‘D’). Then,it recursively tracesconvex pyrami-dal beamsthroughintersectedcell boundariesinto adjacentcells,incrementallytrimmingthebeamsby convex polygonsat traversedcell boundaries(� , � , � , � , and � ), andmirroring the beamsat re-flectingcell boundaries(� and� ). Therecursionof eachdepth-firsttraversalis capturedandstoredin a beamtree,representingall thetracedreverberationpathsemanatingfrom thesource.Later, duringaninteractivesession,theprecomputedbeamtreesareusedto gen-eratereverberationpathsto a moving receiver position. For everybeamcontainingthe receiver, a reverberationpath is constructed

by iterative intersectionwith the reflectingcell boundariesstoredwith theancestorsof thecorrespondingbeamtreenode(asshownin Figure2).

A

B

CD

E

S

p

u

tsR

Ip

Ioo

So

SopFigure2: Beamtracing.

Thekey ideain this previousmethodis that it is possibleto usean off-line precomputationto constructa datastructure(a beamtree)encodingpotentialreverberationpathsfrom eachstaticsourcelocationandusethatprecomputeddatastructureto computerever-berationpathsto a moving receiver quickly for auralization.Sincetheoff-line beamtracingalgorithmtakesup to 50secondsfor eachnew sourcelocation[14], this methodis not directly applicableforcomputingreverberationpathsin real-timebetweenmovingavatarsin adistributedvirtual environment.Otherdatastructuresproposedfor interactive updatesof global illumination solutionsare bestsuitedfor handlingchangesto materialsandgeometryin primarilydiffuseenvironments[6, 10], andthey wouldgenerallynotperformwell for viewpointchangesin highly specularenvironments.

The focus of our new work is developing beamtracing algo-rithmsthatcanbeusedto computereverberationpathsin real-timewith updateratessuitablefor auralizationin distributedvirtual en-vironmentapplications(upto 100Hz).Achieving thisgoalrequiresaccelerationof currentbeamtracingmethodsby two or threeordersof magnitude.

3 Overview of Approach

Our approachis to take advantageof the fact that DVE systemsmustonly computereverberationpathsbetweenavatarlocations,1

andnot betweenall pointsin 3D space.Sinceavatarsarelocatedat a discretesetof positions,andthey tendto move alongcontin-uouspaths,beamtracingalgorithmsfor DVEs canutilize directedsearchesand temporalcoherenceto find reverberationpathsveryefficiently. In thispaper, weproposethefollowing threebeamtrac-ing algorithmsthatexploit theseproperties:

� Priority-driven beam tracing usespsychoacousticallymoti-vatedpriorities andterminationcriteria to traceonly beamsthat can representthe most significant reverberationpathsfrom oneavatarlocationto another.

� Bidirectional beam tracing combinestwo sets of beamstracedfrom different avatar locationsto find reverberationpathsefficiently.

� Amortized beam tracing utilizes beamstracedfrom a re-gion of spaceto computereverberationpathsquickly for asequenceof nearbyavatarlocations.

The following three sectionscontain detaileddescriptionsofthesenew algorithms,while Sections7 and 8 describehow theyareintegratedinto ouradaptive, real-timeDVE system.

1We extend the notion of an avatar to include anything that can be asourceor receiver of sound.

4 Priority-Driven Beam Tracing

The first algorithm is motivated by priority-driven searchmeth-ods. For a given avatar location, all reverberationpathsare notequallyimportant.Somepathsarepsychoacousticallyvery signif-icant (e.g.,directpathsto otheravatars),while othersfollow com-plex sequencesof reflectionstowardsemptyregionsof space.

Our priority-driven beamtracingalgorithmexploits knowledgeof avatarlocationsto computeonly themostsignificantreverbera-tion pathsefficiently. Specifically, the algorithmconsidersbeamsin best-first order, ratherthandepth-firstorderasin previousbeamtracingalgorithms(e.g.,[14]). As beamtreesareconstructed,theleaf nodesarestoredin a priority queue,andthe highestprioritynodeis iteratively poppedfor expansionat eachstep. The benefitof this approachis that the mostsignificantbeamsareconsideredfirst, enablingmethodsbasedon adaptive refinementanddynamicterminationcriteria.

Thefirst issuein implementingthepriority-drivenalgorithmis toassignrelativeprioritiesto differentbeamtreenodes.Following ac-ceptedpracticeof theacousticsliterature,wepartitionthereverber-ationpathsrepresentedby thebeamsinto two categories:1) earlyreflections,and2) latereverberations.Early reflectionsaredefinedastheonesthatarrive at thereceiver within someshortamountoftime, � , afterthemostdirectsound,while latereverberationscom-prisetherest(� ������������� ����� [3, 17]). Sincelocalizationinthe humanbrain is mostsensitive to the early reflections,andthelate reverberationsarecharacterizedby many pathsarriving fromall directionsafterbeingmultiply scatteredby many surfaces,ge-ometricacousticmodelingsystemsgenerallycomputeonly earlyspecularreflections,while late reverberationsanddiffractionsaremodeledwith statisticalapproximations(seeFigure3). In our cur-rent system,we usethis approach,assigninghigher priorities tobeamtreenodesrepresentingshorterreflectionpaths.

Am

plitu

de

Time Delay

Late Reverberations

Early Reflections

DirectSound

Te

Figure3: Impulseresponse.

Thesecondissueis how to guideourpriority-drivenalgorithmtofind earlyspecularreflectionpathsefficiently. Previousalgorithmshave tracedbeamsin depth-firstorderup to someterminationcri-teriabasedon themaximumnumberof reflections,maximumpathlength,and/ormaximumattenuation.Oneproblemwith this ap-proachis that it is impossibleto selecta priori a terminationcrite-rion that guaranteesfinding all early specularreflectionpaths. Ingeneral,the length of the longestearly reflectionpathcannotbepredetermined,sinceit dependson the lengthof the shortestone,which is notknown until it is found.

Priority-orderedsearchhelpsusovercomethis problem.We as-sign the priority , ������� , of eachbeamtreenode,� , to the lengthof theshortestpathfrom thebeamsourceto the last traversedcellboundary, !���"� , plus the lengthof theshortestpathfrom thatcellboundaryto theclosestavatarlocation, #$���"� (seeFigure4). Since�����"� underestimatesthelengthof any paththrough� to anavatar,wecanbeassuredthatall earlyspecularreflectionpathsarefoundifwe terminatethetraversalwhenthevalueof �����"� for all nodesre-mainingin thepriority queuecorrespondsto anarrival timeat least� laterthanthemostdirectpathfoundto everyavatarlocation.

Source

Closest Avatar

Last TraversedCell Boundary

f(B) = g(B) + h(B)

h(B)

g(B)%

Figure4: Thepriority heuristic,������� , underestimatesthedistancetraveledsofar, !���"� , plusthedistanceto theclosestavatar, #$���"� .

Our implementationof this approachis closely relatedto theclassical&(' algorithmfromtheartificial intelligenceliterature[15].Thedifferenceis that thesearchfor eachavatarlocationshasmul-tiple goals(i.e., the early reflectionpathsto all otheravatar loca-tions). Thus,we mustrememberthe shortestpath to eachavatarwhenwe find it, andwemustrememberfor whichavatarswe havealreadyconsideredall potentialearlyreflectionpaths.

The primary advantage of the priority-driven beam tracingmethodis that it avoids geometriccomputationsfor many beamsrepresentinginsignificant reverberationpaths,and thereforeit isableto computethe significantonesmorerapidly. It canalsobeusedto allocateresourcesdynamicallyin a real-timesystem(seeSection7). The disadvantageis that there is extra overheadincomputingprioritiesandmaintainingpriority queuesof beamtreenodes.

5 Bidirectional Beam Tracing

The secondalgorithm usesa bidirectionalapproachto combinebeamtreestracedindependentlyfrom two differentavatarlocationsto find reverberationpathsbetweenthem.

The primary motivation for a bidirectionalapproachis that thecomputationalcomplexity of beamtracingalgorithmsgrows expo-nentiallywith increasingreflections.Consequently, tracingonesetof beamsup to ) reflectionsusually takes far longer thantracingtwo setsof beamsup to )+*,� reflections.

A secondmotivation is that DVE systemsmustfind reverbera-tion pathsbetweenall pairsof avatars. In this situation,unidirec-tional approachesare inherentlyredundant,sincebeamsmustbetracedfully from all exceptoneavatarlocationsto insurethatrever-berationpathsarefoundbetweenall pairs. But thenalmosteveryreverberationpathis tracedtwice, oncein eachdirection. With abidirectionalapproach,we canavoid this redundantwork by com-bining beamstracedfrom one avatar location with beamstracedfrom anotherto find thesamereverberationpathsmoreefficiently.

Bidirectionalpathtracingapproacheshave beeninvestigatedfordecadesin ray tracing(e.g.,[7, 24, 34]), radiosity(e.g.,[20, 28]),andotherfields (e.g.,[9, 25]). Generally, for every ray found im-pingingupona surface,a datastructureassociatedwith thesurfaceis both updatedandqueriedto computethe energy traveling for-wardsand backwardsalong the ray. The main challengeof thebidirectionalapproachis to find adatastructurethatefficiently andaccuratelyrepresentsthe directionalenergy radiatingfrom everysurfaceof the environment. Previous datastructuresfor bidirec-tionalpathtracinghavemostlybeenbasedonstoringdiscreteradi-ancesamples(e.g.,“illumination maps”[2] and“radiositytextures”[19]), andthusthey suffer from aliasingartifacts.An importantob-servation is thatbeamtreedatastructures[18] areanobject-space

representationof thedirectionalenergy radiatingfrom eachsurface,andthe- y arewell-suitedfor implementingabidirectionalapproach.

Our bidirectionalbeamtracing algorithm computesreverbera-tion pathsby combiningbeamtreestracedindependentlyfrom dif-ferentavatarlocations.Thekey contributionof ouralgorithmis themethodwe usefor determiningwhich beams,�". and ��/ , tracedindependentlyfrom avatarlocations,0 . and0 / , combineto repre-sentviablereverberationpaths.Webaseourmethodon thefollow-ing observations,whichapplyfor commonraypropagationmodelscomprisingspecularreflections,diffusereflections,transmissions,anddiffractionsover locally reactingsurfaces.2

� Condition A: Thereis a reverberationpathif �". contains01/(seeFigure5a).

� Condition B: Thereare(usuallyaninfinite numberof) rever-berationpathscontaininga diffusereflectionat surface 2 ifboth ��. and �"/ intersectthe sameregion of 2 (seeFigure5b).

� Condition C: There is a reverberationpath containing astraight-linetransmissionthroughsurface 2 if: 1) both � .and � / intersectthe sameregion of 2 , 2) � . intersectsthevirtual sourceof �"/ , and3) �"/ intersectsthevirtual sourceof� . (seeFigure5c).

� Condition D: Thereis areverberationpathcontainingaspec-ular reflectionatsurface 2 if: 1) both � . and� / intersectthesameregionof 2 , 2) � . intersectsthemirroredvirtual sourceof �"/ , and3) �"/ intersectsthemirroredvirtual sourceof �".(seeFigure5d).

� Condition E: There is a reverberationpath containing adiffractionat an edge3 if: 1) � . and � / both intersectthesameregionof 3 (seeFigure5e).

To acceleratedeterminationof theseconditions,we constructlistsof beamtreenodesintersectingeachcell andfaceof thespatialsubdivision asthebeamsaretraced.We traversetheselists to de-termineefficiently whichpairsof beamtreenodespotentiallycom-bineto representviablereverberationpaths,avoidingconsiderationof all 45��4"6879�:*,� pairwisecombinationsof tracedbeams.First,foreachpair of beamtreenodesconsidered,we checkto seeif bothnodesareeithertheroot or a leaf of their respective beamtrees.Ifnot, thepair canbeignored,asit surelyrepresentsa reverberationpath that will be found by anotherpair of nodes. Otherwise,wecheckthebeamsintersectingeachcell containinganavatarto seeifthey satisfyConditionA. Wecheckpairsof beamsintersectingthesametransmissive faceto seeif they satisfyConditionC. Wecheckpairsof beamsintersectingthe samereflectingfaceto seeif theysatisfyConditionsD. Finally, we usethefirst nodemeetingoneofthesecriteria to computean underestimatingdistanceheuristictoanotheravatarlocation,which canbeusedto aid earlyterminationwhensearchingfor early reflectionpathsin an integratedbidirec-tionalandpriority-drivenbeamtracingalgorithm.

As comparedto unidirectionalbeamtracingmethods,themainadvantageof our bidirectionalalgorithmis thatpathswith up to Rreflectionscanbefoundby combiningtwo beamtreesrepresentingupto ; . and; / reflections,respectively, where; .=< ; / 6>7?@; .SinceACB$D < ACBCEGF(F�ACB for most A , fewer beamsmustbetraced.Themaindisadvantageis thatextra bookkeepingandprocessingisrequiredto combinebeamsfrom differentbeamtrees.

2Sofar, wehave implementedthemethodsonly for specularreflections.

P2

P1P1’

B1

(a)ConditionA

S

P2

P1P1’

P2’

B1 B2

DiffuseReflection Region

(b) ConditionB

S

P1P1’

P2’

B1

B2

Virtual Source of B2 Transmission

Point

P2

Virtual Source of B1

(c) ConditionC

S

P2

P1P1’

P1’’

P2’

P2’’

B1B2

MirroredVirtual Source of B2

MirroredVirtual Source of B1 Specular

Reflection Point

(d) ConditionD

P2

P1

B1

B2

Diffraction Edge

(e)ConditionE

Figure5: Combiningbidirectionalbeams.

6 Amortized Beam Tracing

Thethird algorithmacceleratescomputationof reflectionpathsbe-tweenmoving avatarsby amortizingthecostof asinglebeamtrac-ing computationoverseveralreceiver locations.

Thepathstakenby peopleasthey move througha virtual envi-ronmentgenerallyexhibit a large amountof spatialandtemporalcoherence.Peopleusuallydo not jump from point to point arbi-trarily. Rather, they tend to move along continuouspaths. Userinterfacesmayevenimposeboundson their speeds,rotations,anddirectionsof movement.Moreover, a person’s two earsarenearbyoneanotheratfixedrelativepositions.By utilizing temporalcoher-enceandspatialconstraintswecanmodelreverberationpathsmoreefficiently for a sequenceof receiver locationsthanfor a setof un-relatedlocations.Specifically, weextendthenotionof a beamtreeto includeconservativebeamstracedfrom aregionof space,ratherthanjust a point. Sincethesebeamsover-estimatethesetof rever-berationpathsemanatingfrom any point within theregion,we canusuallyre-usethemfor a sequenceof receiver locationsinsidetheregion to identify potentialreverberationpathsmorerapidly thantracingbeamsfrom scratchfor every avatarmovement.

Ourapproachis motivatedby algorithmsoriginally proposedforconservative visibility determination(visibility is tracing0th-orderpaths)[30,32]. Thesealgorithmsdonotsolvethevisibility problemexactly. Instead,they simplyreducethesizeof thepolygonsetto beprocessedby alaterhidden-surfacealgorithm(e.g.,z-buffer). In thesameway, a conservative beamtreedoesnot exactly representthereflectionpathsfor any point. Instead,it encodesa supersetof thepossiblesequencesof reflectionsandtransmissionsfrom all pointswithin its sourceregion. Thus,to find exact reflectionpathsfromsuchapoint to anotherpoint,wemustonly checkarelatively smallsetof potentialpolygonsequencesencodedin thebeamtreeto seewhich of themadmitsa “valid” reflectionpath. Theserelativelysimplecheckscanbeexecutedvery quickly for eachpair of avatarlocations,while themuchlongertime requiredfor beamtracingis

amortizedover multipleavatarmovements.OurH amortizedbeamtracing algorithm is integratedwith the

priority-driven and bidirectionalmethodsas shown in Figure 6.Fromtop to bottom,we first usemotionpredictionto updatea setof sourceregionsthatwe expectto containfutureavatarlocations.Then, in the secondstep,we traceconservatively over-estimatingpolyhedralbeamsin priority order from eachsourceregion andstorethem in a conservativebeamtree. Next, for every pair ofsourceregions,we useour bidirectionalmethodsto combinetheconservative beamtreesto form a setof polygonsequences, eachof whichdescribeshow aray travelingfrom oneregion to theothercanpotentially reflectoff and/ortransmitthroughan orderedlistof polygons. Finally, for eachtime an avatar moves within oneof the sourceregions,we generateexact reflectionpathsto everyotheravatar simply by processingthe appropriateset of polygonsequences.This laststepis theonly onerepeatedfor every avatarmovement,andit is usuallyvery fast.

Priority−drivenBeam Tracing

PathGeneration

Reverberation Paths

AvatarPositions & Orientations

MotionPrediction

Source Regions

ConservativeBeam Trees

Bidirectional Sequence Construction

Polygon Sequences

CellAdjacency Graph

Acoustical BRDFs

CellAdjacency Graph

Asynchronous (~10 Hz)

On−line (~100 Hz)

Figure6: Amortizedbeamtracingphases.

The key challengesin implementingthis algorithmare: 1) se-lectingeffectivesourceregions,2) efficiently tracingasetof tightlyover-estimatingbeamsemanatingfrom a region of space,3) form-ing polygonsequencesby combiningbeamstracedfrom two re-gionsof space,and4) checkingpolygonsequencesquickly to seeif they admitvalid reflectionpathsfor aspecificpairof avatarloca-tions.

First,to selecteffectivesourceregions,wemustbalancecompet-ing goals.Eachoneshouldcontainasmany futureavatarlocationsaspossiblesothat its conservative beamtreecanbere-usedmanytimes.However, they cannotbetoolarge(e.g.,awholeroom).Oth-erwise,thebeamstracedfrom theregionare“too over-estimating,”whichnotonly slowsdown thebeamtracingcomputation,but alsoputsundueprocessingburdenon the subsequentsteps. Our cur-rentmethodupdatesasourceregionassociatedwith eachavatarbypredictingits motion with a second-degreepolynomialcurve anddetectingcollisionswith obstaclesin theenvironment.Thesourceregionis alwaysanaxis-alignedboxwith user-specifieddimensions(“Box Size”). We arecurrentlyextendingthis methodto adaptthesizeof eachsourceregionaccordingto thelocationsof otheravatarsandthegeometriccomplexity of thelocalenvironment.

Second,to tracebeamsfrom a box-shapedsourceregion effi-ciently, weover-estimatethesetof pathsemanatingfrom thesourceby a polytope comprisingthe intersectionof at most six halfs-paces,with two halfspacesboundedby planesparallel to eachofthe X, Y, andZ axes. Eachof the threepairsof halfspacesforma wedgeboundingthe beamin one dimension,and the intersec-tion of the threewedgesform a polytopeover-estimatingthe po-tentialreverberationpathsfrom any pointwithin thesourceregion.

As cell boundariesaretraversedin thebeamtracingalgorithm,thehalfspacesare updatedincrementallyby a gift-wrapping step, inwhich theplaneboundingeachhalfspaceis rotatedaroundan ex-tremaledgeof thesourcebox until it hitsanextremalvertex of thecell boundary, asshown in Figure7. Although this beamtracingmethodis slightly moreover-estimatingthanpreviously describedapproximations(e.g.,[32]), it is very fast,requiringconstanttimepercell boundary, andit providesafairly tight over-estimatefor thepathstracedin typicalvirtual environments.

Source Box

PlaneNormal

Extremal Edge

Extremal Vertex

CellBoundary Polygon

Figure7: Conservative beamtracinggift wrappingstep.

Third, to form polygonsequences,we extendthe bidirectionalmethoddescribedin Section5 to combinebeamswhosesourcesareboxesratherthanpointsandto constructpolygonsequencesratherthan specificreverberationpaths. In particular, for eachpair ofbeamtreenodesmeetingoneof theconditions,A-E, listedin Sec-tion 5 (with beamsourcesextendedto includeaxis-alignedboxes),weconstructapolygonsequenceby concatenatingtheorderedlistsof reflectionsandtransmissionsat cell boundariesencodedin theancestornodesof thetwo beamtrees,asshown in Figure8. Everypolygonsequenceconstructedin thiswayprovidesarecipefor howa ray potentiallycantravel from a point in onesourceregion to apoint in theother.

a

c

b

d

e

a

b

e

d

c

Beam Tree for Box B

Beam Tree for Box A

PolygonSequence

a b c d e

cA

ValidPath

InvalidIntersection

B

Figure8: Constructinga polygonsequenceby linking beamtreenodes,anddeterminingits validity for specificpairsof avatarloca-tionsby checkingpolygonintersections.

Finally, to find pathsbetweena specificpair of avatarlocations,0 . and0 / , wesimplycheckeachpolygonsequence,2 , associatedwith the appropriatepair of sourceregions to seewhethera raytravelingfrom 0 . travelingalongtheprescribedsequenceof reflec-tions and transmissionscan possiblyreach 0 / . We first traversethe polygonsequencein backwardsorder to constructa stackofmirror imagesof 0 / , where0�I/ correspondsto theimageresultingfrom mirroring 01/ over thelast J of the 4 reflectingpolygonsin thesequence.Then,we constructthereverberationpathby traversingthepolygonsequencein forwardsorder, while iteratively checking

ConservativeBeam Tree (back)

Priority−DrivenBeam TracingProcesses

ConservativeBeam Tree (front)

Swap Bidirectional SequenceConstruction Processes

PolygonSequences (back)

Swap PathGeneration Processes

PolygonSequences (front)

Reverberation Paths (back)

Swap AudioConvolutionProcesses

Reverberation Paths (front)

MotionPrediction Viewpoint

Reset

Spatialized Audio

Source Audio

Prio

rity

Que

ue

Figure9: Multiprocessingsystemorganization.

to seewhetherthe ray startingfrom 0 I. to 0�KML I/ intersectseverypolygon, 2 I , in thesequence(seeFigure8). If so,we have foundavalid reflectionpathfrom 0 . to 0 / along 2 (shown asa greenlinein Figure8). The lengthof the pathis given by thedistancefrom0 . to 0 KML ./ , andtheattenuationof soundtraveling alongthepathis given by the productof the attenuationsat eachpolygonin thesequence.Otherwise,if a ray fails to intersectoneof thepolygons(e.g., the red line in Figure8), the sequenceis determinedto bein the over-estimatingportionof theconservative beams,andit isignored.

Oneadvantageof conservative beamtracingis that the costofcomputingeachbeamcanbe amortizedover all pointswithin itssourceregion. However, tracing beamsfrom a larger region ofspaceis generallymoreexpensive. So, thesizesof sourceregionsmustbechosencarefully to maximizetheeffective speedups.An-otheradvantageis that beamtreesandpolygonsequencescanbeconstructedasynchronously(or even off-line), andthenreflectionpathscanbe foundat very high rates.This featureis exploited inthemultiprocessingsystemdescribedin thenext section.

7 Time-Critical Multiprocessing

Motivatedby the principlesof time-critical computing,we havedevelopedanadaptive, real-timemultiprocessingsystemthatusesmultipleasynchronouslyexecutingprocessesandallocatescompu-tationalresourcesdynamicallyin orderto performthehighestpri-ority beamtracingcomputationsin a timely manner. We expandthenotionof priority-drivenbeamtracingto includemultiple pro-cessorsworking togetheron thebeamtreesfor all avatarsat once.Ratherthanupdatingthebeamtreefor eachavatarindependently,thesystemcomputesbeamtreenodesin globalpriority order, andthusthesystemcantrade-off computationalresourcesbetweendif-ferentavatarsdynamically. For instance,if many avatarsenterthevirtual environment,availableresourcesfor computingbeamtreenodesfor eachavatardecreases,and the quality of computedre-verberationpathsdegradesgracefully. Conversely, when avatarsleave theenvironmentor becomestationary, thequality of spatial-ized soundsis adaptively refined. In the limit, if thereis just oneavatarin theenvironment,all availableprocessorsconcurrentlyex-pandits beamtreenodesin priority order.

Ourmultiprocessorimplementationexecutesasshown in Figure9. Eachdatastructure(oval) is storedasapairof “front” and“back”buffers. Justas in computergraphics,the backbuffer is usedforupdates,while otherprocessesaccessthe front buffer. Updatestoall datastructuresaresynchronizedby mutualexclusionlocksthatallow multiple readers,but only a singlewriter. As a result,a highdegreeof concurrentprocessingis possible,asmutualexclusionisrequiredfor front buffersonly whenthebuffersareswapped.

As every avatarmovesthroughthe environment,a motion pre-diction processdetermineswhen to swap an avatar’s beamtrees(e.g.,whenit hasmoved significantly). Whenthe beamtreesareswapped,the new “back” beamtreeis emptiedandthenprepared

for expansion:a new sourceregion is assignedanda root nodeiscreatedandput on thepriority queue.Asynchronouslyandcontin-uously, multiplebeamtracingprocesseswork onexpanding“back”beamtreenodesin global priority order. Eachprocessiterativelypopsa nodefrom thepriority queue,computesthechildrenof thenode,andpushesthemontothepriority queue.Meanwhile,multi-ple bidirectionalpolygonsequenceconstructionprocessesmonitorthearrival of new beamtreesin thefront buffer of any avatar. Whenonearrives,the“front” and“back” buffersof theappropriatepoly-gonsequencesareswapped,andnew “back” buffersof polygonse-quencesareupdatedfrom thenew beamtree. Similarly, eachtimean avatar moves, or a new polygonsequencebecomesavailable,pathgenerationprocessesaccessthepolygonsequencesto form re-verberationpaths.Finally, sourceaudiosignalsarespatializedviasoftware convolution with binaural impulse responsescomputedfor eachavatar. The responsesarederived by addinga pulseforeachcomputedreverberationpath,wherethe delay is determinedby NO* P , whereN is the lengthof thepath,and P is thespeedofsound,andtheamplitudeis setto 7�*:�Q�R7 <�S:TVUMW �X&"*9N , whereW isthe angleof arrival of the pulsewith respectto the normalvectorpointingout of the ear, and & is the productof all the frequency-independentreflectivity and transmissioncoefficients for eachofthereflectingandtransmittingsurfacesalongthecorrespondingre-verberationpath.

The nicestfeatureof this implementationis that it provides aframework in whicheachcomponentof thesystemcan“do thebestit can” without slowing down the othercomponents.In particu-lar, the pathgenerationprocessescontinueto computereverbera-tion pathsevenwhenanavatarmovesout of its beamtree’s sourceregion.This featureprovides“extrapolation”of reverberationpathsfrom nearbypointsandis critical to providing a seamlessauditoryexperiencewhenthebeamtracingprocessesbecomeoverloaded.

8 Experimental Results

We have implementedthe algorithmsdescribedin the precedingsectionsin C++ and integratedtheminto a DVE systemsupport-ing communicationbetweenmultiple usersin a virtual world withspatializedsound. Our currentimplementationsupportsspecularreflectionsandtransmissionsin 3D polygonalenvironments,anditrunsonPCsandSGIsconnectedby a100Mb/sTCPnetwork.

The systemusesa client-server design. Eachclient providesan immersive audio/visualinterfaceto the sharedvirtual environ-ment from the perspective of one avatar. As the avatar “moves”throughthe environment,possiblyunderinteractive usercontrol,imagesandsoundsrepresentingthe virtual environmentfrom theavatar’s simulatedviewpoint are updatedon the client computerin real-time. Communicationbetweenremoteuserson differentclients is possiblevia network connectionsto the server(s). Anyclient cansendmessagesto theserver(s)describingupdatesto theenvironment(e.g.,the positionandorientationof avatars)andthesoundsoccurringin the environment(e.g.,voicesassociatedwith

avatars).Whena server receivesthesemessages,it processesthemto determineY which updatesare relevant to which clients, it spa-tializesthesoundsfor all avatarswith thebeamtracingalgorithmsdescribedin theprecedingsections,andit sendsappropriatemes-sageswith updatesandspatializedaudiostreamsbackto theclientssothatthey mayupdatetheiraudio/visualdisplays.

To evaluatethe effectivenessof our new beamtracingmethodsin the context of this system,we executeda seriesof experimentswith asingleserverspatializingsoundsonanSGIOnyx2 with four195MHzR10000processors.In eachexperiment,weuseddifferentbeamtracingalgorithmstocomputespecularreflectionpathsfromasourcepoint(labeled‘A’) to eachof thethreereceiverpointslabeled‘B,’ ‘C,’ and‘D’ in the3D modelshown in Figure10.

AB C

D

Figure10: Testmodelgeometryandavatarlocations.

8.1 Priority-Driven Beam Tracing Results

We testedthe relative benefitsand costsof priority driven beamtracingby runningaseriesof testsusingthefollowing threediffer-entbeamtracingalgorithmsbasedon differentsearchmethodsfortraversingthecell adjacency graphanddifferentterminationcrite-ria:

� DF-R: Depth-first searchup to a user-specifiedmaximumnumberof reflections.

� DF-L: Depth-first searchup to a user-specifiedmaximumpathlength.

� P: Priority-drivensearch(ouralgorithm).

In eachset of tests,we computedall early specularreflectionpaths(� = 20ms)from asourcepoint (labeled‘A’) to oneof threereceiver points(labeled‘B,’ ‘C,’ and‘D’) in the3D modelshownin Figure10. The depth-firstsearchalgorithms,DF-R andDF-L,wereaidedby oraclesin thesetests,astheterminationcriteriawerechosenmanuallyto matchthe exact maximumnumberof reflec-tions,R, andthemaximumpathlength,L, respectively, of knownearlyreflectionpaths,whichwerepredeterminedin earliertests.Incontrast,thepriority-drivenalgorithm,P, wasgivenno hints,anditusedonly thedynamicterminationcriteriadescribedin Section4.

The bar chart in Figure11 shows the wall-clock times(in sec-onds)requiredto find all early specularreflectionpathsfor eachcombinationof thethreereceiverpointsandthethreebeamtracingalgorithms.Althoughall threealgorithmsfind exactly thesamesetof earlyreflectionpathsfrom thesourceto eachreceiver, thecom-putationtimesfor thepriority-drivenapproach(thebluebars)werebetween2.6and4.3timeslessthanthenext best.Thereasonis thatthepriority-drivenalgorithmconsidersbeamsrepresentingearliestpathsfirst andterminatesaccordingto ametricutilizing knowledgeof the receiver location,andthusit avoidscomputingmostof the

uselessbeamsthattravel longdistancesfromthesourceand/orstrayfar from thereceiver location.

DF−LDF−R P

B

10

20

30

40

50

60

Bea

m T

raci

ng T

ime

(sec

onds

)

DF−LDF−R P

CDF−LDF−R P

D

2.0 1.3 0.5

60.1

6.01.8

60.2 61.6

14.1

Figure11: Beamtracingtimeswith differenttraversalalgorithmsandterminationcriteria.

The relative value of the priority-driven approachdependsonthegeometricpropertiesof theenvironment.For instance,all earlyreflectionpathsto the receiver point ‘B,’ which wasplacedin thesameroomasthesource,requiredlessthanor equalto 3 specularreflections,andthe longestpathwasonly 623 inches.Theserela-tively tight terminationcriteriawereableto boundthecomplexitiesof thedepthfirst searchalgorithms,sothespeedupof thepriority-drivenalgorithmis only around2.6xover thenext best.In contrast,for receiver point ‘D,’ someearlyreflectionpathsrequiredup to 7specularreflections,andthelongestearlyreflectionpathwas1046inches. In this case,thepriority-driven algorithmis far moreeffi-cient(speedupis 4.3x)asit directsthebeamtracingsearchtowardsthereceiverpointalmostimmediately, ratherthancomputingbeamsextendingradially in all directions.

Thebenefitsof directedsearchareshown visually in Figure12,which containstwo imagesof 10,000beams(pink) tracedin a 2Dmapof Bostonfromasourcepoint(square)to twodifferentreceiverlocations(circles).In thiscase,thedistanceheuristicdirectsbeamstowardseachreceiver locationenablingit to find complex reflectionpaths(blue).

Source

Receiver

Source

Receiver

Figure 12: Visualizationof beamstracedfor different receiverpointswith priority-drivenbeamtracing.

8.2 Bidirectional Beam Tracing Results

To testtherelativebenefitsandcostsof thebidirectionalbeamtrac-ing algorithmdescribedin Section5, we rana seriesof testswithcomparableunidirectionaland bidirectionalbeamtracing imple-mentationson an SGI workstationwith a 195MHz R10000pro-cessor.

In eachsetof tests,we computedall specularreflectionpathsfrom a sourcepoint (labeled‘A’) to one of threereceiver points

(labeled‘B,’ ‘C,’ and‘D’) up to a specifiedmaximumnumberofreflectionsZ (‘R’) in the 3D model shown in Figure 10. The uni-directionalalgorithmconstructeda singlebeamtreecontainingallpathswith up to ; specularreflectionsfrom thesourcepoint, andthenit reporteda specularreflectionpathfor eachbeamcontain-ing thespecifiedreceiver point. In contrast,thebidirectionalalgo-rithm constructedtwo beamtreesfor eachsource-receiver pair, thefirst containingbeamsup to ;[*,� < 7 specularreflectionsfrom thesource,and the secondcontainingbeamsup to ;[*,� specularre-flectionsfrom thereceiver. Thetwo beamtreeswerecombinedtofind all Rth-orderspecularreflections.In this experiment,theuni-directionalandbidirectionalalgorithmsfind exactly thesamesetofpathswith up to ; specularreflections.Thegoalof theexperimentis to determinewhichalgorithmtakeslesstotalcomputationtime.

Table1 containsstatisticscollectedduringthesetests.Fromleftto right, the first column (labeled‘P’) lists which receiver pointwas used. The secondcolumn (labeled‘R’) indicatesthe maxi-mumnumberof specularreflectionscomputed.Then,for boththeunidirectionalandbidirectionalalgorithms,therearethreecolumnswhich show the times(in seconds)requiredto computethe beamtrees(“BeamTime”), find the reflectionpaths(“PathTime”), andthesumof thesetwo (“Total Time”). Finally, the last column(la-beled“Speedup”)liststhetotaltimefor unidirectionalbeamtracingalgorithmasa ratioover thetotalbidirectionalbeamtracingtime.

Unidirectional BidirectionalBeam Path Total Beam Path Total Speed

P R Time Time Time Time Time Time UpB 3 2.02 0.01 2.03 1.04 0.03 1.07 1.9

4 5.79 0.03 5.82 2.55 0.10 2.65 2.35 15.01 0.07 15.08 4.23 0.50 4.73 3.56 31.53 0.14 31.66 8.02 1.31 9.33 3.97 60.26 0.24 60.50 11.95 4.43 16.39 5.08 100.82 0.41 101.22 21.12 9.52 30.64 4.8

C 3 2.03 0.01 2.03 0.96 0.01 0.98 2.14 5.81 0.01 5.82 2.49 0.04 2.54 2.35 14.83 0.02 14.86 3.92 0.20 4.12 3.86 31.38 0.05 31.42 7.82 0.54 8.37 4.07 60.82 0.08 60.90 11.23 1.97 13.20 5.48 100.89 0.14 101.03 20.56 4.13 24.69 4.9

D 3 2.03 0.00 2.03 0.62 0.01 0.62 3.34 5.81 0.01 5.81 2.17 0.03 2.20 2.75 14.94 0.01 14.95 2.47 0.12 2.59 6.06 31.88 0.02 31.90 6.24 0.29 6.53 5.17 60.31 0.04 60.35 7.10 0.92 8.02 8.58 100.68 0.06 100.75 16.27 1.83 18.10 6.2

Table1: Bidirectionalbeamtracingstatistics.

Theresultsof thisexperimentshow thetrade-offs of thebidirec-tional approachvery clearly. First, comparingthe “Beam Times”in Table1, we seethat the bidirectionalalgorithmspendssignif-icantly lesstime tracingbeamsthanthe unidirectionalalgorithm.This is becauseit constructsbeamtreeswith lessdepth,therebyavoiding theworstpartof theexponentialgrowth. A visualizationof thiseffectappearsin Figure13,whichshowsacomparisonof 2Dbeamstraced(a)unidirectionallyand(b) bidirectionallyto find thesamereflectionpaths(shown in blue).Notehow many morebeamsaretracedin theunidirectionalcase.Second,comparingthe“PathTimes,” weseethatthebidirectionalalgorithmspendssignificantlymoretime generatingreverberationpaths.This is becauseit mustperformintersectiontestsfor eachpair of beamsreflectingoff thesamesurface. Overall, thebidirectionalbeamtracingalgorithmissignificantlyfasterthantheunidirectionalalgorithm,with speedupsrangingfrom 4.8xto 6.2xfor 8 specularreflectionsin theseexperi-ments.

P1

P2

(a)Unidirectional(59,069beams)

P1

P2

(b) Bidirectional(15,078pink beams+ 24,665bluebeams)

Figure13: Visualizationof beamstracedto find specularreflec-tion pathsbetweenpointsP1andP2with (a)unidirectionaland(b)bidirectionalmethods.Thebeamsemanatingfrom locationP1areshown in pink, while theonesfrom P2areshown in cyan.Thebluelinesrepresentspecularreflectionpaths.

8.3 Amortized Beam Tracing Results

We studiedthe trade-offs of the amortizedbeamtracingapproachdescribedin Section6 by runninga setof testsin which conserva-tive beamsweretracedfrom sourceregionswith varying sizestofind specularreflectionpathsbetweena sourcepoint (labeled‘A’)andoneof the threereceiver points(labeled‘B,’ ‘C,’ and‘D’) inthe3D modelshown in Figure10.

Duringeachtest,cubeswith width “Box Size”wereconstructedaroundboththesourcepoint andoneof thereceiver points. Then,thefollowing threestepswereusedto computespecularreflectionpathsbetweenthetwo points.

1. Traceconservative beamsfrom both sourceregionsup to 3specularreflections.

2. Combinethe beamsto form polygon sequencescontainingpotentialpathswith up to 5 specularreflections.

3. Processthepolygonsequencesto findall valid5th-orderspec-ular reflectionpathsfrom thesourcepoint to thereceiver.

Table2 showstimes(in seconds)measuredduringeachtest.Thefirst column (“P”) lists the receiver point, and the second(“BoxSize”) indicatesthe sourcebox width in inches. The next fourcolumnsshow thetimesrequiredfor beamtracing(“BeamTime”),polygonsequenceconstruction(“SeqTime”), andpathgeneration(“Path Time”), respectively. The column labeled“Amort Time”representsthe averagewall-clock time requiredfor the amortizedalgorithmto computeall 5th-orderspecularreflectionpathsfromthe sourceto the receiver at even intervals separatedby 1/2 inch,while the lastcolumn(labeled“Speedup”)lists thespeedupof theamorizedbeamtracingalgorithmascomparedto thebidirectionalalgorithmdescribedin theprevioussection(i.e., “Box Size”= 0).

Scanningdown the rows of the table,we seethat the timesre-quiredfor all threestepsincreasewith larger sourceregions,firstslowly, andthenveryrapidly. Thisgrowthpatternmatchesthenum-berof beamstracedin eachtest.As thesourceregiongrows larger,a largerportionof thespacecanbereachedby reflectionpaths,and

Box Beam Seq Path AmortP Size Time Time Time Time SpeedupB 0 4.2 0.48 0.004 4.691 1.0

2 4.6 0.57 0.006 2.593 1.84 5.1 0.64 0.006 1.423 3.38 5.9 0.81 0.008 0.840 5.616 8.9 1.48 0.013 0.651 7.232 83.3 5.89 0.054 2.787 1.7

C 0 3.9 0.20 0.001 4.116 1.02 4.3 0.23 0.001 2.264 1.84 4.6 0.27 0.002 1.214 3.48 5.4 0.39 0.005 0.730 5.616 8.2 0.73 0.012 0.561 7.332 13.9 3.07 0.049 0.531 7.8

D 0 2.5 0.12 0.000 2.597 1.02 2.7 0.12 0.000 1.405 1.84 2.8 0.14 0.001 0.739 3.58 3.4 0.20 0.001 0.448 5.816 5.2 0.35 0.004 0.344 7.532 10.1 1.55 0.021 0.364 7.1

Table2: Amortizedbeamtracingstatistics.

more beamsmust be traced(seeFigure 14). Accordingly, morepolygonsequencesareconstructed,andthey all mustbeprocessedfor eachpathgenerationstep. We mustbe carefulnot to choosesourceregionsthat areso big that the resultingbeamtreeandthelist of polygonsequencesaregrosslyover-approximating.

The benefitsof conservative beamtracingarereapedby amor-tization as the beamtreesandpolygonsequencesare re-usedformoreandmorenearbyavatarlocationsasthesourceregionsgrow.Theresultof this trade-off betweenextra processingversusamor-tization is shown in the right-mostcolumn of the table (labeled“Speedup”).In this experiment,we seethat thecompetingfactorsarebalancedmostadvantageouslywhenthe box sizeshave widtharound16 inches.In thiscase,theconservative beamsdonotover-estimateexact onesby too much,andthusthey requireonly 2.1xmoretimeto compute,while thebenefitsof amortizationcanbere-alizedover 16 distinctavatarlocationsseparatedby 1/2 inchastheavatarwalksfrom thecenterof thebox, resultingin aneffective7xspeedup.

(a)Box Size= 0(4,686beams)

(c) Box Size= 32(12,242beams)

Figure14: Beamstracedfor upto 2 reflectionsfrom sourceregionsof differentsizescenteredatpoint ‘A.’

8.4 Time-Critical Multiprocessing Results

We studiedthe time-critical multiprocessingaspectsof our sys-temby runningtestswith multiple avatarsmoving simultaneouslythrough a virtual environment (shown in Figure 15) while log-ging statisticsregardingthe system’s performance.During thesetests,eachof four avatarsmovedautonomouslyalongapathwhich

causedit to passverycloseto otheravatarssometimes,andto wan-deraloneat others.

(a)T = 65 (b) T = 90

Figure15: Visualizationof adaptive real-timebeamtracing(circlesrepresentnumberof beamtreenodes).

Figure16containsaplot indicatingthenumberof nodesin eachof four avatars’beamtreesduringatestexecutingonanSGIOnyx2workstationwith 4 195MHz R10000processors.Eachof thefourcurves representsa singleavatar, and the vertical position of thecurve at every time stepindicatesthe amountof computationthesystemis expendingupdatingthebeamtreefor theavatar. Figure15showsthestatusof thesimulationat (a)T=65and(b) T=95.Forvisualizationpurposes,the radiusof the circle aroundeachavatarin theseimagesmatchesthecorrespondingheightin theplot.

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

40 60 80 100 120

Bea

ms

Per

Ava

tar

Time (seconds)

Figure16: Plotof numberof nodesin thebeamtreefor eachavatarshowing adaptive real-timebeamtracing.

Theplot clearlyshows theadaptive andtime-criticalnaturesofourmultiprocessingsystem.In somesituations,suchastheoneoc-curringatT=65,two avatarsarein closeproximity, while theothersareoff by themselves.Accordingly, thesystemallocatesmostof itsresourcesto computingbeamtreesfor thosetwo avatars(blueandpurple)asthey interact(i.e., theblueandpurplelinesarerelativelyhigh). At othertimesin thesimulation,differentavatarsareinter-actingandthesystemadaptsdynamically. Yet, thetotal numberofbeamstracedfor all avatars(i.e., thesumof theheightsfor all linesin the plot) remainsapproximatelyconstantacrossall time steps.This featurereflectsthe time-criticalnatureof our real-timealgo-rithm.

9 System Limitations

Our systemis a researchprototype,andit hasseveral limitations.First, the 3D modelmustcompriseonly planarpolygonsbecausewe do not modelthe transformationsfor beamsasthey reflectoffcurvedsurfaces.Nor do we tracebeamsalongpathsof refraction,diffraction,or diffusereflection,which arevery importantacousti-cal effects. However, we think that the conservative beamtracingapproachmaybeanappropriateframework for extendingour sys-temto handlethesesituations.

Second,our methodsare only practicalfor coarse3D modelswithout\ highly facetedsurfaces,suchas the onesoften found inacousticmodelingsimulations. The difficulty is that beamsgetfragmentedby cell boundariesas they are tracedthrougha celladjacency graph. For this reason,we are not optimistic that ourimplementationcanbe easilyadaptedto modellight transportforphotorealisticimagesynthesis.However, the algorithmsprobablycanbeappliedeffectively in otherapplicationareasconcernedwithsimulationof wavephenomena(e.g.,radiofrequency propagation).

Third, themajoroccludingandreflectingsurfacesof thevirtualenvironmentmust be static throughthe entire execution. If anyacousticallysignificantpolygonwereto move, the cell adjacencygraphwouldhave to beupdatedincrementally.

Finally, our implementationdoesnot includesophisticatedmod-elsfor acousticalreflectancedistributionfunctionsor for direction-ality of audiosourcesandreceivers.For instance,wecurrentlyrep-resentthe reflectanceof eachsurfacewith an angle-independentandfrequency-independentabsorptioncoefficient. However, sincewe computereverberationpathsexplicitly, addingtheseauraliza-tion featuresis relatively straight-forward.

10 Conclusion

In thispaper, wehavedescribedthreebeamtracingalgorithmswell-suitedfor geometricacousticmodelingin adistributedvirtual envi-ronmentsystem.Thesealgorithmsoffer two importantadvantagesoverpreviousapproaches:1) they aremuchfaster, and2) they sup-port real-timecomputing.

To summarizethespeedupresults,thepriority-drivenalgorithmacceleratessearchesfor early reflectionpathsby 2.6x – 4.3x; thebidirectional approachruns 4.8x – 6.2x faster than a compara-bleunidirectionalalgorithm;and,amortizedbeamtracingachievesspeedupsof 7.2x – 7.5x. Moreover, our multiprocessingsystemachievesnearlylinear speedupsfor up to at leastfour processors.Overall, the speedupachieved combiningall thesealgorithmsisapproximatelytwo ordersof magnitudeover previous beamtrac-ing systems.

Our adaptive multiprocessingsystemutilizes thesenew beamtracingalgorithmsto supportmany featuresessentialfor areal-timesystem,including time-critical updates,gracefuldegradation,andadaptive refinement.Moreover, thesystemperformsasynchronousbeamtracing computations,while reverberationpathsare gener-ated,and even extrapolated,at very high updaterates. Overall,our systemis ableto matchtheaccuracy of many previousoff-lineacousticmodelingsystemswhile executingin real-time.

Acknowledgements

TheauthorsthankGaryElko, MohanSondhi,Jim West,andPerryCookfor theirvaluablediscussions.Wewouldalsoliketo acknowl-edgeNadiaMagnenat-Thalmann,DanielThalman,andEric Peta-jan for useof theimagesin Figure1.

References[1] Allen, J.B., andD.A. Berkley, ImageMethodfor Efficiently Simu-

lating Small-RoomAcoustics,J. Acoust.Soc.Am., 65, 4, Apr 1979,943-950.

[2] Arvo, James.Backward Ray Tracing.Developmentsin RayTracingCourseNotes,SIGGRAPH86,1986.

[3] Begault,Durand,3D Soundfor Virtual RealityandMultimedia,Aca-demicPress,1994.

[4] Blanchard,C., S. Gurgess,Y. Harvill, J. Lanier, A. Lasko, M. Ober-man,andM. Teitel, Reality Built for Two: A Virtual Reality Tool.ACM SIGGRAPHSpecialIssueon 1990Symposiumon Interactive3D Graphics, (Snowbird, Utah),1990,35-36.

[5] Borish,Jeffrey. Extensionof theImageModelto ArbitraryPolyhedra.J. Acoust.Soc.Am., 75,6, June,1984,1827-1836.

[6] Briere, Normand,and PierrePoulin, HierarchicalView-DependentStructuresfor Interactive SceneManipulation,ComputerGraphics(SIGGRAPH96),83-90.

[7] Chattopadhyay, Sudeb,andAkira Fujimoto,Bi-directionalRayTrac-ing,ComputerGraphics1987(Proceedingsof CGInternational’87),Springer-Verlag,Tokyo, 1987,335-343.

[8] Dadoun,N., D.G.Kirkpatrick,andJ.P. Walsh.TheGeometryof BeamTracing.Proceedingsof theSymposiumonComputationalGeometry,Baltimore,June,1985,55-61.

[9] Davison,B. NeutronTransportTheory. OxfordUniversityPress,Lon-don,1957.

[10] Drettakis,George,andFrancoisSillion. Interactive Updateof GlobalIllumination Using a Line-SpaceHierarchy. Computer Graphics(SIGGRAPH97),1997,57-64.

[11] Durlach,N.I., R.W. Pew, W.A. Aviles, P.A. DiZio, andD.L. Zeltzer.Virtual EnvironmentTechnology for Training (VETT). Report No.7661,Bolt, Beranek,andNewmann,Cambridge,MA, 1992.

[12] Durlach,N.I, andA.S. Mavor, editors,Virtual RealityScientificandTechnological Challenges, National ResearchCouncil Report,Na-tionalAcademyPress,Washington,D.C.,1995.

[13] Foster, S.H.,E.M. Wenzel,andR.M. Taylor. Real-timeSynthesisofComplex AcousticEnvironments.Proc. IEEEWorkshopon Applica-tionsof SignalProcessingto AudioandAcoustics,1991.

[14] Funkhouser, ThomasA., Ingrid Carlbom,GaryElko, GopalPingali,MohanSondhi,andJimWestA BeamTracingApproachto AcousticModeling for Interactive Virtual Environments.ComputerGraphics(SIGGRAPH‘98), Orlando,FL, July, 1998,21-32.

[15] Hart, P.E., N.J. Nilsson, and B. Raphael,A Formal Basis for theHeuristicDeterminationof Minimum CostPaths,IEEETransactionsonSSC, Vol. 4, 1968.

[16] Hartman,Jed,JosieWerneck,VRML2.0Handbook, Addison-Wesley,ISBN 0-201-47944-3,August1996.

[17] Hartmann,W.M., Listening in a Room and the PrecedenceEffect,Binaural and Spatial Hearing in Real and Virtual Environments,editedby RobertH. Gilkey andTimothyR. Anderson,LawrenceErl-baumAssociates,1997.

[18] Heckbert,Paul,andPat Hanrahan.BeamTracingPolygonalObjects.ComputerGraphics(SIGGRAPH84),18,3, 119-127.

[19] Heckbert,Paul. Adaptive RadiosityTexturesfor Bidirectional RayTracing.ComputerGraphics(SIGGRAPH90),24,4, 145-154.

[20] Immel,David S.,MichaelF. Cohen,andDonaldP. Greenberg. A Ra-diosity Methodfor Non-DiffuseEnvironments.ComputerGraphics(SIGGRAPH85),19,3, 133-142.

[21] Kleiner, Mendel,Bengt-IngeDalenback,andPeterSvensson.Aural-ization – An Overview. J. Audio Eng. Soc., 41, 11, Nov 1993,861-875.

[22] Krockstadt,U.R. Calculatingthe AcousticalRoomResponseby theUseof a RayTracingTechnique, J.SoundandVibrations,8,18,1968.

[23] Kuttruff, Heinrich RoomAcoustics, 3rd Edition, Elsevier Science,London,England,1991.

[24] Lafortune,E.P., andY.D. Willems,Bi-directionalpathtracing,Com-puGraphics, Alvor, Portugal,1993,145-153.

[25] Lewins,Jeffery. Importance, TheAdjointFunction:ThePhysicalBa-sisof Variational andPerturbationTheoryin TransportandDiffusionProblems.PergamonPress,New York, 1965.

[26] Moore, G.R. An Approach to the Analysisof Soundin Auditoria.Ph.D.Thesis,Cambridge,UK, 1984.

[27] Quake, id Software,Mesquite,TX, 1996.[28] Smits,Brian, JamesR. Arvo, andDavid H. Salesin.An Importance-

Driven RadiosityAlgorithm. ComputerGraphics(SIGGRAPH92),26,2, 273-282.

[29] Sony Corporation,CommunityPlaceBrowserManual, 1996.[30] Teller, Seth.,andCarloSequin, Visibility Preprocessingfor Interac-

tive Walkthroughs,ComputerGraphics(SIGGRAPH91), 25, 4, 61-69.

[31] Teller, SethComputingtheAntiumbraCastby anAreaLight Source.ComputerGraphics(SIGGRAPH92),26,2, 139-148.

[32] Teller, SethVisibility Computationsin DenselyOccludedPolyhedralEnvironments.Ph.D.thesis,ComputerScienceDivision(EECS),Uni-versityof California,Berkeley, 1992.Also availableasUC BerkeleytechnicalreportUCB/CSD-92-708.

[33] Tsingos,Nicolas, and Jean-DominiqueGascuel.A GeneralModelfor Simulationof RoomAcousticsBasedOn HierarchicalRadiosity.TechnicalSketches,SIGGRAPH97VisualProceedings, 1997.

[34] Veach,Eric, andLeonidasGuibas,BidirectionalEstimatorsfor LightTransport,Fifth EurographicsWorkshopon Rendering, Darmstadt,Germany, June,1994,147-162.

[35] Zyda, Michael J., David Pratt, JohnFalby, Chuck Lombardo,andKristen Kelleher, TheSoftwareRequiredfor the ComputerGenera-tion of Virtual Environments.Presence, 2, 2 (March1993),130-140.