� � �������� ����������������������� ���������� �"!#������$ %�&���������!��#%������ $���'�()��*�+,�" ��(��.-)/ �10�2�3�4

IncrementalEvaluatorsforRemoteAttributeGrammars

JohnTangBoyland1 5 2Departmentof Electrical EngineeringandComputerScience

University of Wisconsin-MilwaukeeMilwaukee, WI, USA

Abstract

Remoteattributegrammarsuseobjectswith separatelydefinedfieldsto inducedirectnon-local dependenciesin attribute grammars.Fieldsof anobjectmaybereadremotelyfromwhereit is created,andspecial“collection” fieldsmaybewrittenremotelyaswell. Buildingon earlierwork which shows that remoteattribute grammarscanbe scheduledstatically,thispapershows how they maybeimplementedincrementally. Thestaticscheduleis usedto ensureanobject’s fieldsaredefinedbeforethey arereadandthatwenever re-evaluateanattributemultiple timesperedit-cycle. Dynamicdependenciesareusedto markremoteusesitesasaffectedwhenafield is changed.Theresultis anefficientandpracticalincrementalevaluation.

1 Introduction

Remoteattributegrammarsaredefinedandmotivatedin anearlierpaper[2] fromwhich Fig. 1 is taken. Like a classicalattributegrammar, a remoteattributegram-marhassynthesizedattributes(computedby thenodeto which they areattached)andinheritedattributes(computedby the parentof the nodeto which they areattached).For example(line 7 ff), everyexpressionnodein thetreehasascope,assignedby its context, anda shape, its type. To classicalattribute grammarswe addglobal collectionsandobjectswhich canhave normal (locally assigned)fieldsandcollectionfields.Collectionsaregivenaninitial valueandacombinationfunction to combineall the assignmentsthat occur. For example,Fig. 1 declares

1 Work supportedin part by the NationalScienceFoundation(CCR-9984681)and the DefenseAdvancedResearchProjectsAgency andRomeLaboratory, Air ForceMaterielCommand,USAFundercontractF30602-99-2-0522. The views andconclusionscontainedhereinare thoseof theauthorsandshouldnot be interpretedasnecessarilyrepresentingthe official policiesor endorse-ments,either expressedor implied, of the NationalScienceFoundation,DefenseAdvancedRe-searchProjectsAgency, RomeLaboratory, or theU.S.Government.2 Email:boyland@cs.uwm.edu

c6

2002Publishedby Elsevier ScienceB. V.

798;:=<�>=?A@

1 global msgs B {} with ( C )23 block, decls, decl, stmts, stmt4 inh scope5 type6 syn shape7 expr8 inh scope9 syn shape

1011 program D block12 block.scope = ROOT_SCOPE1314 block D "begin" decls stmts "end"15 local scope =16 { col decls B {} with ( C ),17 enclosing = block.scope }18 decls.scope = scope19 stmts.scope = scope2021 decls D2223 decls D decls decl24 decls1.scope = decls0.scope25 decl.scope = decls0.scope2627 decl D id ":" type ";"28 local d =29 { shape = type.shape,30 col used B false with (or)}31 decl.scope.decls B {<id,d>}32 if not d.used then33 msgs B {id ++ " is unused"}34 endif35

36 type D "integer"37 type.shape = INTSHAPE38 type D "string"39 type.shape = STRSHAPE4041 stmts D4243 stmts D stmts stmt44 stmts1.scope = stmts0.scope45 stmt.scope = stmts0.scope4647 stmt D block ";"48 block.scope = stmt.scope4950 stmt D expr ":=" expr ";"51 expr1.scope = stmt.scope52 expr2.scope = stmt.scope53 if expr1.shape /= expr2.shape54 then msgs B {"type mismatch"}55 endif5657 expr D intconstant58 expr.shape = INTSHAPE59 expr D strconstant60 expr.shape = STRSHAPE6162 expr D id63 local decl64 decl = lookup(id,expr.scope)65 expr.shape = decl.shape66 if decl = NOT_FOUND then67 msgs B {id ++ " undeclared"}68 else69 decl.used B true70 endif

71 function lookup(id,s)72 if s = ROOT_SCOPE then73 result = NOT_FOUND74 else75 local d = fetch(id,s.decls)76 if d = NOT_FOUND then77 result = lookup(id,s.enclosing)78 else79 result = d80 endif81 endif

Fig. 1. Usinga remoteattributegrammarfor nameresolution

a globalcollectionof messages(line 1) with anemptysetfor initial valueandsetunion for the combinationfunction. Every block nodecreatesa scopeobjectwith two fields (line 15): the setof decls is collectedfrom elsewhere,but theenclosing scopeis sethere.We see(line 31) thatevery declarationaddsa pairof nameandobjectto thedecls of thescope.Theobjecthasashape setlocallyanda booleancollectionindicatingwhetherit wasused anywhere. If it wasnotused,weaddamessage(line 33) to theglobalcollectionmsgs. At thepointwhereanexpressionis in theform of anidentifier(line 64), we look up this identifierinthescope,checkthatanactualdeclarationis foundandthenmark thedeclarationasused.

Collectionfields andglobal collectionsmayhave multiple definition sites;all

2

798;:=<�>=?A@

(partial) definitionsarecombinedinto a singlevalueusingthecombinationfunc-tion. Unlikethenon-localdependenciesof JohnsonandFischer[9] andVorthman’sDR threads[18], informationcanflow from the ‘reference’site backto the ‘dec-laration’ site. The valuethat onegetswhenreadingthe collectionfield or globalcollection is the final value; intermediatevaluesarenot exposed. In particulararulesuchasi E i + 1 is acyclic dependency, notanimperativeupdate.

Remoteattribute grammarsare relatedto higher-order attribute grammarsinthat both permit compoundobjectsto be transmittedthroughthe tree,but in ourcase,the objectsare transmittedwith identity andwith attributes. Semantically,eachfield of the objecthasa separatesetof dependencies;packagingvaluesto-getherdoesnot causeany useof a field to dependon all otherfieldsof theobject.In contrast,in thehigher-ordercase,subtreesarereferentiallytransparent;any useof thetreedependstransitively on anything usedto constructthetree.Subtreesdonot carry attribute values;they areattributedat the point they areused. In high-orderattributegrammars,collectionfieldsobviouslyhaveno analogue.

Unlike Gorel Hedin’s doorattributegrammars[6] which requireschedulingtobe doneby hand,remoteattribute grammarscanbe automaticallyscheduled[2].Schedulingis accomplishedthroughthegenerationof controlattributesthatcarryno valuebut ensurea properordering,in particularthat fields are(fully) definedbeforebeingused. For instance,whenschedulingthe remoteattribute grammarin Fig. 1, the systemgeneratesa synthesizedattribute of decls indicating thatdeclarationsbelow this point have beenaddedto the contour, and generatesaninheritedattributeof stmts that indicatesthat thescopeobjectis fully populatedwith declarations.Thenthesystemconnectsthesetwo generatedattributesin theproductionfor block. In other words, rather than relying on manuallyaddedcontrolattributesasin KastensandWaite’s useof imperative symboltablesin anattributegrammar[11], we usethedeclarative semanticsof the (remote)attributegrammarto constraintheschedule.

However, eventhougha remoteattributegrammaris declarative,animplemen-tationbasedon controlattributesis not declarative andthuscannotbedirectly in-crementalized.One way forward would thus be to identify somereal valuestobe transmittedand then usestandardincrementaltechniques,perhapsextendingthosefor higher-orderattributegrammars.Theproblemwith this approachis thatthe scaleof affectedattributesO F�GAFFECTEDGIH is muchgreater, asdescribedinfurtherdetail in Section2.4.

Thealternateapproachis to keepan imperative implementationandusesomeotherway to let changesbe propagated.This papershows how remoteattributegrammarsscheduledusingautomaticallygeneratedcontrolattributescannonethe-lessbe incrementalizedthroughthe useof selecteddynamicdependencies.Themethodsarerelatedto thetechniquesfor dealingwith aggregatesusedby HooverandTeitelbaum[8]; thesimilaritiesanddifferencesarepointedout in thecourseoftheexposition.

JohnsonandFischerhandleremotedependenciesincrementallyby staticallydeterminingapriority orderof subtrees[9], andsometimesnoconsistentordercan

3

798;:=<�>=?A@

befound. Hoover [7] handlesall non-circularattributegrammarsthroughan“ap-proximate”topologicalorder, sometimesleadingto attributere-evaluation.Theo-retically, theremaybeanexponentialnumberof re-evaluations,but in practicere-evaluationsneveramountto morethanhalf of all evaluations.Vorthmann[18] usesdynamicschedulingfor all attributesin a tree with Declaration-Reference(DR)threadswhich permitattributesto flow from thedeclarationto thereference.It isnot clearhow the algorithmworks if DR threadsareupdatedasevaluationpro-ceeds.Theschedulingalgorithmin this paper(like thatof JohnsonandFischer)isstaticandonly acceptsa subclassof attributegrammars(orderedremoteattributegrammars).

LIGA [10], partof theEli compilerconstructionsystem[5], hasanotationthatcanreducethenumberof copy rules. And Hoover hasshown how copy rulescanbe bypasseddynamically[7]. Both of thesetechniquesaremostly orthogonaltothe extensionof remoteattribute grammarsandtheir scheduling.Oneoverlapisthat the useof objectsreducesthe numberof “parallel” copy rulesneeded,sinceonecaninsteadpackagetheattributesinto objects.

In the following sectionwe describehow to implementa staticallyscheduledremoteattributegrammar, first withoutincrementalityandthenwith incrementality.Thenext sectionthendescribestheimplementationplatform. Section4 concludesthepaper.

2 Incremental Visit-Sequence Evaluators

We build on thetechniquefor incrementalvisit-sequenceevaluationproposedandimplementedby Maddox[12], who refinesearlierwork by Reps[14]. Theresult,whichhandlesmultiplesubtreereplacement(or equivalently, multiple edit sites)isessentiallyanalogousto thetechniqueof SwierstraandVogt [17]. Wedescribethiscommontechniquein this section.

2.1 Visit-SequenceEvaluators

A visit-sequenceevaluatorusesa protocolof theattributesfor every nonterminal:a sequenceof pairsof setsof attributes J I1 K S1 L�K�M�M�MNK J In K Sn L . Given any O -orderedattributegrammar, onemay find a singleprotocolfor eachnonterminalsuchthateachattribute may only dependon attributesmentionedearlier in the total orderresultingfrom flatteningtheprotocolinto a singlesequence.Visit sequenceevalu-atorscanbedefinedfor any non-circularattributegrammar;herewelimit ourselvesto thesimple O -orderedcase.

Evaluationof thetreeworksthroughtheuseof recursive visit procedures. Fora productionp of nonterminalX, we have onevisit procedurefor eachpair in theprotocolfor X. A visit procedurefor p andpair J Ii K Si L acceptsvaluesfor theinher-itedattributesIi andcomputesthesynthesizedattributesin Si. Eachvisit procedureconsistsof a sequenceof attributeevaluationsfor this productionandcallsto visitproceduresof thechildren.

4

798;:=<�>=?A@

Sinceeachnodeis called from a context aware only of its nonterminal(notits production),this techniquemapsnicely ontoan object-orientedlanguagewithabstractclassesfor eachnonterminaland concreteclassesfor eachproduction.Specifically, for a nonterminalX with protocol J I1 K S1 L�K�M�M�MPK J In K Sn L , we have thefollowing class(hereT F aH meansthetypeof attributea):

abstract class X extends Node {abstract Tuple<T F aHQG a R Si> visit_i(T F aH a G a R Ii); G 1 S i S n

}

And for aproductionp of theform X0 T X1 M�M�M Xm, wehave thefollowing concreteclass:

class p extends X0 {Xi _i; G 1 S i S nT F aH a; G a R A F X0 H (all attributesof X0)T F l H l; G l R Lp (local attributesfor p)p(Xi c_i G 1 S i S n) { _i=c_i; G 1 S i S n }Tuple<T F aHUG a R Si> visit_i(T F aH _a G a R Ii){

a = _a; G a R Iibodyof visit procedure i for preturn (a G a R Si);

} G 1 S i S n}

In thispaper, weassumeall attributesarestoredin thetree.Storingattributesmakesincrementalityeasier.

2.2 Maddox’s IncrementalVisit-SequenceEvaluators

UsingMaddox’stechnique(modeledonReps’multiple-edit-sitecoordinationstrat-egy [14]), we short-circuita visit if the incominginheritedattributesarethesameandthesubtreerootedat this nodeis thesameaswhentheearlierevaluationtookplace.

In essence,wememoizethevisit procedurewith respectto thetreestate;if thetreeis unchangedandtheinheritedattributesarethesame,thetraversalresultscanbe fetchedfrom the treewhich servesasa cache. The treemight be changedintwo differentways.Most obviously, it couldhavebeeneditedsincethelasttime itwasvisited.Another, moresubtleway, is thatit couldhavereceivednew attributesin a previous visit. This intuition is formalizedin the “visit-tree” incrementalitytechniqueof Saraiva, SwierstraandKuiper[15,16], in which attributesarepassedbetweenvisitsbyproducingaspecializedtreefor thefollowingvisit whichincludestheneededattributevaluesdirectly.

Maddox’s techniqueachievesroughlythesameeffect imperatively. In ordertodetecttreechanges,headdsa bit to every nodecalledthe “subtreemodified” bit.This bit is setin a nodeandall its ancestorswhenever thenodeor an attribute in

5

798;:=<�>=?A@

it changes.The bit is clearedwhenthe last visit of a nodecompletes.We avoidsettingthebit in a nodewhenit is alreadyset,but this changedoesnot affect theasymptoticcomplexity in abalancedtree:

class Node {boolean SUBTREE_MODIFIED = true;...void markSubtreeModified() {

if (!SUBTREE_MODIFIED) {SUBTREE_MODIFIED = true;if (parent != null) parent.markSubtreeModified();

}}

}

Theresultingvisit procedurestake thefollowing form:

Tuple<T F aHQG a R Si> visit_i(T F aH _a G a R Ii){

if (SUBTREE_MODIFIED || a != _a G a R Ii) {a = _a; G a R Iibodyof visit procedure i for pSUBTREE_MODIFIED = i VW n;

}return (a G a R Si);

} G 1 S i S n

Thetimecomplexity for updatingattributesafterachangeis O F�GAFFECTEDGYXGEDIT ANCESTORSGIH whereAFFECTED is the set of all attribute valuesthatneedto be re-evaluated,andEDIT ANCESTORSis the setof all nodeson pathsto the root from edit changes.The latter termcanbe aslarge asO F nH wheren isthetotalnumberof nodesin thetree,but Maddoxobservesthatif the(incremental)parserusesbalancedrepresentationfor sequencesandif we assumethat for cog-nitive reasonsa programmerwill not generatedeeplynestedstructuressuchasthefollowing:

f1(f2(f3(...(fn())...)))

then GEDIT ANCESTORSG will have a practicallimit of O F k lognH wherek is thenumberof edit sitesandn is thesizeof tree.

2.3 Visit Proceduresfor a RemoteAttributeGrammar

In this section,we assumewe have a remoteattributegrammarscheduledwith thehelpof (automaticallygenerated)controlattributes,andthat the resultis in the O -orderedclassof attributegrammars.Withoutlossof generality, weassumeall fieldsarecollectionfields; laterwe will show how thecaseof locally assignedfieldscanreceive a simplertreatment.First we examinea non-incrementalimplementation

6

798;:=<�>=?A@

of the visit procedures.In the following sections,we discussincrementalimple-mentation.

As well aslocals,we have theobjectsaswell. As well asregularattributeas-signments,we have objectreadsandwrites. For simplicity we assumeeachobjectis of typeAGObject which hasa field for each f R F , alongwith a getteranda(partial)setter. Wealsoprovideacollectionfunctionwhichis calledafterall partialsetsandbeforeany gets:

class AGObject {AGObject(Node owner) { ... }T F f H f = defaultfor f; G f R FT F f H get_ f() { return f; } G f R Fvoid partial_set_ f(T F f H partial) { ... } G f R Fvoid collect_ f() { ... } G f R F

}

(In theprototype,objectsaretypedandthereis aseparateclassfor eachobjecttyperatherthanassigningspacefor every field in every object.) The remoteattributegrammarspecifieswhereobjectsarecreatedin thetree.A referenceto theobjectsis storedin thenodeaswith a local attribute:class p extends X0 {...AGObject o = new AGObject(this); G o R Bp (objectsfor p)

}

An objectfield write v. f E w is implementedas

v.partial_set_ f(w);

At thepoint wheretheschedulersaysthefield hasreceivedall partialwritesandisreadyfor reads,weaddtheline:

v.collect_ f();

This function needdo nothingif the partial setterkeepsthe valueup to date,buthasan importantpurposewhenwe move to the incrementalimplementation.Anobjectfield readv = w. f is implementedas

v = w.get_ f();

Otherwise,everythingremainsthesameaswith aclassicalattributegrammar. Thisis essentiallytheresultthatwasreportedin our earlierwork [2].

2.4 Pure IncrementalImplementation

Next we turn to the incrementalview. In this sectionwe examine(andreject)apurelyfunctionalapproachto implementation.

In our earlierwork [2], the control attributesweremotivatedby “pretending”they carriedtheactualvaluesof the fields of the objectstransmittedin their baseattributes.Oneway to implementa remoteattributegrammaris thusto transform

7

798;:=<�>=?A@

thecontrolattributesinto value-carryingattributes.Thenonecanusethetechniquein theprevioussectionto scheduletheattributegrammar.

Ondisadvantagehereis thatthis transformationincreasesthenumberof value-carrying attributes. We also needfunctionsto comparethe old and new valuesof the transformedcontrol attributes,eachof which may representan unboundednumberof objectfields. Hash-consingallows comparisonsto be constanttime atthecostof slowing downobjectconstruction.Furthermore,theequalitycheckswilloftenfail. For instance,if wechangetheparametertypeof oneglobalfunction,thegeneratedattributescarryingthetypesof entitiesin theenvironmentwill havenewvaluesin almostevery nodein the tree. Even if we get the “optimal” result ofO F�GAFFECTEDGIH incrementality, thisvalueis boundedonly by thetotalnumberofattributesin thetree(includinggeneratedattributes).

The sameproblemsoccur in higher-orderattribute grammarswhich usepureincrementality. Saraiva, Swierstraand Kuiper give a techniquethat an attributegrammarwriter canuseto amelioratethis situation: at the entry to a block, thesymboltableis “projected” to only that setof identifiersthat areusedwithin theblock. Thenchangesto globalsnot usedwithin theblock arenot propagatedintothe block. This techniqueis not directly applicableto remoteattribute grammarssincethecontrolattributesareautomaticallygeneratedandscheduled.But it seemsplausiblethataprojectiontransformationcouldbecarriedoutautomatically.

Usinghigher-orderattributegrammartechniquespresentsamoreseriousprob-lem. Puttingall the informationtogetherinto a singleaggregatevaluegreatlyin-creasesthenumberof (spurious)dependencies,possiblycausinganorderedremoteattribute grammarto appearcircular when viewed as a higher-orderedattributegrammar. Indeedthat is what happensif one attemptsto expresscollection at-tributesin suchasystem.

The differencesseenbetweenremoteattribute grammarsand high-orderat-tributegrammarsaredueto theway objectsarehandled.For high-orderattributegrammars,treesare pure values; they may be implementedas directedacyclicgraphs(DAGs),andnever containcycles. For remoteattributegrammars,an ob-ject’s fieldsareassignedafter it is created,thuspermittingcyclic structures.Con-siderthecaseof the typeof a formal parameterof a global function. In a higher-orderattributegrammarview of thesystem,thescopeatsomepoint in theprogramis asinglevaluethatincludesall theinformationvisible in thescope,includingthetype of the formal parameter. In the remoteattribute grammarview, the scopeisanobjectwith a field thatrefersto anotherobjectrepresentingthefunctionwith afield thatincludestheformalparameter’s type.

In theformersystem,changingtheformal’s typemeanswehavecreatedanewscopevalue,andasa resulthave a largenumberof affectedattributes.In thelattercase,no suchchangeis observed; the scopeobject is not changedby the changein parametertype.This treatmentof objectsgreatlyreducesthenumberof directlyaffectedattributes. However, onemuststill updatean attribute that accessestheformal’s typethroughthescopeobject,eventhoughthescopeobjectis unchanged.Thefollowing sectionwill describeour techniquefor doingthis.

8

798;:=<�>=?A@

Z\[�]_^)`a[bZ\[�c�dZ\[�]_^)`a[befZ\gh`h[

i"j)k#`hZ\[�[b]_^)d)ghlm[�d[�n�c�ohj)c�`hgh^)pU^)Z\d)[�Z

Fig.2. Threestagesin theincrementalevaluationof atreewith asinglechangeto anobjectfield write wherethefield is usedin two places.

9

798;:=<�>=?A@

2.5 ImperativeIncrementalImplementation

Inspiredby Hoover andTeitelbaum’s work on aggregates,we track dependenciesonobjectfieldsdynamically. Figure2 givesanexampleof how this is doneandwillbereferredto in thefollowing discussion.Thebasicideais to keeppointersfromaplacein thetreewherea field is written (on theextremeleft in our exampletree)to theplacein thetreewheretheobjectbeingwrittenwascreated(thetrianglenearthe middle of the tree). We alsokeeppointersfrom the objectcreationlocationsto the placesin the treewherethe fields areread(thereare two in our example,triangleson theright of thetree).

Aswith theincrementaltechniquefor classicalattributegrammars,editsitesaremanagedby markingall ancestorsto the root. In theexample,this is shown by adarkeningof thetreeedgesfrom theeditsite(justone,ontheextremeleftmostnodeof the tree). Incrementalevaluationworks asbefore,visiting only thosesectionsof thetreewherethetreeis changedor inheritedattributevalueshave changed.Inour examplefigure,in orderto simplify thepresentation,we assumethat inheritedattributeshave not changed.In thetop partof theexample,we seethatevaluationmakesits waydown to thetriangleon thelower left.

If wefind thataremotefieldwritehasbeenaddedremovedorchanged,wemarktheobject’senclosingnodeasaneditsitebycallingmarkSubtreeModified().This is shown in themiddle picture: themiddle triangleis marked asan edit site(edgesaredarkened).Thisactioncausestheincrementalevaluationto visit thistree(which otherwisewould have beenskipped).Thetotal orderon attributesensuresthatwe never markobjectsthathave alreadybeenskipped(or moreprecisely, thatwill not bevisitedlaterin thetotal order).

At thepoint in theschedulefor collectinga field of anobject,we checkif anobject’sfield valueis changed.If so,all nodeswith remotefield readsof thisobjectaremarkedasedit sites.Thuswe usethecontrolattributesonly to ensurethatanobject’s field is visited after the field writes andbeforethe field reads,we do notactuallypassinformationin them.Thetotalorderingpreventstheexponentialityofnaivechangepropagation;weneverre-evaluateanattributemorethanoncein everyedit cycle. Furthermore,the systemnever copiesobjects: identity is maintainedso we do not suffer from the problemsof hierarchicalattribute grammarsystemswhichstoreattributesdirectly in thetree[3].

Our methoddiffers from Hoover andTeitelbaum’s finite functiontechniqueinthesameway that it differs from purefunctionalimplementationsof higher-orderattributegrammars:theaggregateobject(by default) doesnot dependon its parts.Our object referencesstay the sameeven as the fields changeand so far fewerattributesareaffectedaftera change.

We now examinethestructuresthathelp trackobjectfieldsdynamically. Firstevery partial definition of a collectionfield will be storedin an appropriatecon-tainer. Thesepartial definitionswill thenbe placedin a collectionobjectcreatedfor thecollectionfield itself. Wewill usedifferentclassesfor partialdefinitionsandcollectionobjectsdependingon thetypeof thecollectionfield (which includesits

10

798;:=<�>=?A@

initial valueandits combinationfunction). Thecollectionobjectfor a typeT hasthefollowing form:

class Collection_T {...Collection_T(Node owner) { ... }void insertPartial(Partial_T p) { ... }void removePartial(Partial_T p) { ... }T getValue() { ... }boolean hasChanged() { ... }

}

Thecollectionobjecttakesthenodeto which this objectbelongssothat it canbemarked whenthe setof partialschanges.The insertionfunction doesnothing ifthe partial is currently in the collection. Otherwiseif the partial is alreadyin adifferentcollection,it is first removedfrom there.ThegetValue methodreturnsthe fully combinedvalueandthehasChanged methodreturnstrue if this valuehaschangedsincethelasttimehasChangedwascalled.In general,thecollectionobjectwill maintainan unorderedheap(linearizedfully-balancedbinary tree)ofpartial definitionsand incrementallymaintainthe collectionvaluewith O F logpHcallsto thecombinationfunction(wherep is thenumberof partialdefinitions).

Thepartialdefinitionclassfor a typeT hasthefollowing form:

class Partial_T {T partial;Collection_T collection;...Partial_T() { collection = null; }void setValue(T p) {

if (p == partial) return;removeSelf();partial = p;

}T getValue() { return partial; }void removeSelf() {

if (collection != null)collection.removePartial(this);

}}

Thepartialdefinitionobjectis alsomodifieddirectly by thecollectionobject,set-ting thecollection field andother(unspecified)fields indicating its locationin thecollectionandpartially combinedvalues.Thepartialdefinitionremovesit-self beforechangingits value,thussimplifying the work of the collectionobjectwhich needonly updatethe combinedvalueafter insertionor removal of partialdefinitions.

11

798;:=<�>=?A@

Next we turn to thedynamicdependenciesfor remotefield reads.Whenafieldof anobjectis usedat a remotenode,we recorda dynamicdependency so that ifthefield changes,thenodeis markedasthoughanedit happenedthere.Eachfieldmay be usedin multiple places,andso we keepa doubly-linked list of dynamicdependencies(Use nodes)for every field. The list startswith a sentinel. Thisstructurepermitsauseto beaddedor removedfrom thedependency list in constanttime.

class Use {Use prev=this, next=this;Node use_site;Use(Node use) { use_site = use; }void insertAfter(Use n) { ... }void removeSelf() { ... }void noteChange() { // called on sentinelwhile (next != this) {

next.node.markSubtreeNotified();next.removeSelf();

}}

}

ThenoteChange methodin the sentinelremovesall theotherusenodesin thelist, but they will reinsertthemselvesif theremotereadis acteduponagain.

We changethepartial_set andget functionsto take a partialdefinitionobjectanda useobject,respectively. Thecollect functionis changedto notifyusersof any change:

class AGObject {qCollection_T F f H f;Use uses_ f = new Use(null);void set_partial_ f(Partial_T F f H partial) {

f.insert_partial(partial);}void collect_ f() {

if ( f.hasChanged()) uses_ f.noteChange();}T F f H get_ f(Use u) {

uses_ f.insertAfter(u);return f.getValue();

} rsG f R FAGObject(Node owner) {

f = new Collection_T F f H (owner); G f R F}

}

Whenthe collectionobjectnoticesa changein the setof partial values,it marks

12

798;:=<�>=?A@

the ownerasmodifiedto ensurethat evaluationwill proceedto this point. Whenevaluationreachesthecollectionpoint for anobject’s field; we call thecollectfunction to seeif thecollectionhaschanged.A collectedfield may changemorethanonce,but thenotificationhappensat mostonceperre-evaluationsession.

We generatepartialdefinitionobjectsfor eachremotefield write andgenerateuseobjectsfor eachremotefield readoccurringin therulesof aproduction:

class p extends X0 {...Partial_ f p_r = new Partial_ f(); G r W F v. f E wH9R Rp

Use u_r = new Use(this); G r W F v = w. f H9R Rp

...}

Thenwhena rule r W v. f E w getsscheduled,wegeneratethefollowing:

p_r.setValue(w);v.set_partial(p_r);

Thissetsthepartialdefinition’svalueandinsertsit into thecollection.Recallthatifthevaluein thepartialdoesnotchangeandthepartialwasalreadyin thecollection,neitheroperationwill haveany effect.

Similarly whenschedulingr W v = w. f , wegeneratethefollowing code:

v = w.get_ f(u_r);

If theusewasalreadyin a differentuseslist, it will beremovedfrom that list firstbeforebeinginsertedin thenew list.

Unlike most incrementalattribute grammaralgorithms,we requireaccesstothe subtreesremoved during incrementalparsingso that we can remove partialdefinitionscomingfrom the deletedsubtree.We alsoremove usesalthoughthisstepis not necessaryfor correctness.

class Node {...void remove() {} // called when this subtree is removed

}

Thenfor everyproductionp of theform X0 T X1 M�M�M Xm, weoverridethis method:

class p extends X0 {...void remove() {

_i.remove(); G 1 S i S np_r.removeSelf(); G r W F v. f E wH9R Rp

u_r.removeSelf(); G r W F v = w. f H9R Rp

}}

13

798;:=<�>=?A@

Therestof the incrementalvisit procedureimplementationis thesameasthatdescribedearlierin Section2.2. In thenext section,wewill examinewaysin whichwecanavoid somedynamicdependencies.

2.6 AnalysisandImprovements

Thetimecomplexity for anupdateusingthesemodificationsof Maddox’algorithmis O F�GAFFECTEDG logM XtGAFFECTEDANCESTORSGIH whereM is the maxi-mumnumberof partialdefinitionsof any field in thefully evaluatedtree,andAF-FECTEDANCESTORS is the set of all nodeson pathsto the root from placeswhereattributesmustbere-evaluated.Thelog termcomesfrom insertionsandre-movalsin thebalancedtreesof partialdefinitions.Supposefor instancethateverynodehasa rule that writes the samefield of the sameobject. Thenupdatingat asinglenodewith no downstreamattributeswill requirecomputingthe combinedvalueonabalancedtreeof sizen.

For many combinationfunctions,onecando betterthanthis. For instance,ifthecombinationfunction is logical disjunction,thenwe cankeeponly the “true”partials,andrecomputethe result in constanttime after any change.Anotherex-ampleis addition,which hasan inverseoperation;onecanimplementremoval bysubtraction.If thecombinationfunction is bagunion,we canlink thepartialbagstogetherinto theresult.

If we permitout-of-linefunctionsto accessobjectfields(asin our example,orin dynamicattribute grammars[13]) the worst caseincreasesarbitrarily becauseonemay have an unboundednumberof uses(andeven partial definitionsif oneuses“procedures”[1]). In Fig. 1, the lookup function examinesthedecls andenclosing fields of the scopeobject passedto it. Eachof theseusesleavesbehinda dynamicdependency. Furthermore,reifying dynamicdependencieshasaspacecostaswell asa timecost.

Thus maintainingdynamicdependenciescan be expensive for someheavilyused(or frequentlydefined)fields. Onemayperhapsbewilling to live with extracomplexity at the time of generatingan environment,but run-timecostsfor non-local dependenciesexactedwhenusingthe environmentmustbe kept low. Herewewill briefly describesomeimprovementsthatcanbemade.

For thosefieldsthatareonly assignedlocally, wecanarrangefor thedefinitionsto be scheduledas local attributeswithout the machineryfor partial definitions.Although this improvementhasonly a constantfactor effect, we expect it to bewidely applicable:mostobjectfieldsarelikely to beassignedatthepoint theobjectis created.In our runningexample,thetypeof adecl is availableat thepoint it iscreated.

A further improvementcan be madefor a subsetof the fields assignedonlylocally, thosewhosedefinitionscan be scheduledin the samevisit that the ob-ject is createdwithout any interveningvisits to subtrees.For suchfields,we mayassumethat the object’s creationdependson them. In particular, the objectwillget a new identity if the field is different;we call fields treatedin this way strict

14

798;:=<�>=?A@

fields.Treatingafield asstrictcoarsensthedependency informationallowing ustoavoid recordingdynamicdependenciesfor thefield. This shift givesimprovementin eitherof two situations:(1) thefield rarely, if ever, changes;(2) whenthefieldchanges,all the other information in the object is liable to changeaswell. Forinstance,the “enclosing” scopefield changesonly whena subtreeis moved to alocationin adifferentscope.Sucha changeradicallychangesnamelookupwithinthesubtree,soit is no lossto assumeall lookupsneedto beredone.

Hoover andTeitelbaum[8] describestwo majorweaknesseswith usingaggre-gatesin incrementalimplementationsof classicalattributegrammars:(1) thelargestructuresarepassedthroughmyriad copy ruleswhich all mustbe updatedon achangeand(2) asinglechangein a largeaggregatecauseseverythingthatdependson anything in theaggregateto be recomputed.Theuseof fieldshasamelioratedboth problems:(1) changinga field (for examplethe type of a declaration)doesnot usuallymeanthe object itself haschanged,and(2) eachfield hasits own setof dependencies.Thus, inasmuchasour attribute grammarcanuseobjectswithfields to representthe aggregateand the individual objectscan take part in fine-graineddependencies,we have solved the two major problemsposedby HooverandTeitelbaum.

But, while we canuseobjectsusefully to structurea symboltableinto its var-ious contours,our examplein Fig. 1 still usesan aggregateto representthe setof local bindings. Thusif any declarationis addedto or removedfrom the setofdecls, everynamelookupon this scopemustberedone.

Wesolve this problemthroughtheuseof apredefinedTable datatypewhoseimplementationhandlesincrementalityin a specialmanner. We modify the at-tributegrammarto collectasingleentrytablefor eachdeclaration:

decl.scope.table E Table(id,{d})

Thelookupfunctionis changedasfollows:

function lookup(id,s)if s = ROOT_SCOPE thenresult = NOT_FOUND

elsecase get(s.table,id) of

//if one or more, make arbitrary choice{x,...} => result = x

| {} => result = lookup(id,s.enclosing)endcase

endif

This wouldhave thesameincrementalperformanceasour original attributegram-mar, but for the specialbehavior of get. A Use nodeis attachedto thetablefield whenthetableis read.Normally thisUse nodewouldbeactivatedwheneverthetableasawholeis changed,but theget operationmovestheUse nodedeeperinto the table, leaving it on the part of the tablededicatedto holding the entriesfor thegivenidentifier. ThustheUse nodewill only beactivated(andthelookup

15

798;:=<�>=?A@

causedto happenagain)if thatpartof thetablechanges.If thetabledid notbeforehave an entry for the identifier, oneis createdwith an emptysetof declarations.Such“holes” (asusedby Maddox,andothers)causea lookup to be redoneif anentry is addedto a scopethatwaspassedover earlier. Assumingenclosing isdefinedasastrict field, thenonly thesedynamicdependenciesmustbeadded.

Hoover andTeitelbaum’s implementationmakesuseof threeextra datastruc-tures: the copy bypasstree, the applicationtreeandthe “alwayspropagate”list.Theapplicationtreeis analogousto ourUse list. (OurPartial treehasno ana-loguesinceclassicalattribute grammarsdo not permit collections.) The “alwayspropagate”list and the copy-bypasstree are usedto avoid the cost of updatingcopy rules,especiallythosein which anaggregateis copiedto multiple placesinthe tree. Whenoneusesobjectsto implementan aggregate,the objectreferencechangesonly if astrict field changesor theobjectitself is new. Thuscopy rulesaremuchlesslikely to beout-of-date.Our experimentsin thefollowing sectionshowthattheremainingcopy ruleshave little cost.

3 Implementation

Theseideashavebeenimplementedin aprototypesubsetAPS[1] to Javacompilerthathandlesconditionalandremotefeaturesontopof classicalattributegrammars.It canbe usedto generatethe incrementalevaluatorsdescribedin this paper. Asa way of comparison,it alsocan generateincrementalimplementationsthat usestatic informationalone,ratherthanselecteddynamicdependencies.Essentiallythe “fiber” attributesare convertedinto valuecarrying attributes. For efficiencyreasons,theseextra attributesdo not carryfield valuesthemselves,but ratherjustthesumof thehashcodesof thevaluethat they represent.This implementationisunsafe(afield maychangewithout thesumof hashcodeschanging);weuseit onlyasa roughlower boundon the time thatwould be takenby a correctpurelystaticimplementation.Our resultsshow thatusingdynamicdependenciesis fasterthanapurelystaticimplementationby comparingagainstthis benchmark.

In theprototype,onespecifieswhich fieldsarecollected,andfor thosewhicharenot,whetherthey shouldbeconsideredstrict. In theexample,only thedeclsandused fields are collected,and we specify the enclosing as strict. Theimplementationusesan efficient techniquefor building bagsof declarationsandmessagesincrementally;a small insertionor deletionis accomplishedin constanttime.

We run the evaluatorson an artificial testcase:an outerblock has1000dec-larationsand100 innerblockseachof which has10 innerdeclarationsand10 as-signmentstatements.Thevariablenamesarechosenat randomfrom a setof over350 words. Table1 describesthe operationsthat aremeasured.We measuretheinitial attributionof thetree,startingfrom anewly createdtree.Thenext operationshows whathappensif we re-runtheevaluatoraftermarkinga certainfixedsetofnodes(chosenarbitrarily)asif they wereedited.In otherwords,wehaveanemptyaffectedset,althoughthesystemthinksall attributesalongthespinesfrom theedit

16

798;:=<�>=?A@Table1

Descriptionof benchmarkoperations

Operation Edit Size(unbalanced) Edit Size(balanced)

Initial 10504 13200

NOPedit 604 45

Changedtype 503 14

New global 603 58

Table2Incrementalityresultsonartificial testcase(unbalanced).

List Table

Static Dynamic Static Dynamic

visits time visits time visits time visits time

Initial 14605 932 14605 858 14605 408 14605 298

NOPedit 2667 72 1116 2 2667 55 1116 2

Changedtype 9006 707 1107 5 9006 156 1107 3

New global 9008 686 5306 715 9008 155 1154 3

sitesareaffected.Thethird operationis changinga certaindeclarationin themid-dle of the list of 1000declarationsfrom type integer to string. The lastoperationinvolvesinsertinganew declarationin themiddleof this list.

Weruntheevaluatorsin two differentsituations:whenthetreesareunbalanced(left heavy trees,as in the examplein this paper)or whenthe treesarebalanced(using a slightly modified grammar). The unbalancedtree hasa height of over1000nodes,whereasthebalancedtreehasaheightof only 19nodes.Table1 givestheedit size(in termsof thenumberof nodesthatarebetweenanedit siteandtheroot, inclusive) for bothsituations.

Table 2 gives the run-time using two measuresfor four different evaluators.For eachevaluator, we give the numberof visits andthe time in millisecondstoperformthe incrementalupdate.Thetimeswereobtainedon a lightly loadedSunUltra 10 with JDK 1.4.0andaretheaverageof tenrunsaftertwo runsareignored.For eachrun, we createa new treeandthenmeasureall four operations.On theleft, we have the resultsusingtheoriginal attribute grammar(wheneachcontourhasa list of objects). On the right, we usethe modifiedattribute grammarusingthe tableabstractionthat permitsdynamicdependenciesto be attachedto the setof bindingsin the tablecorrespondingto the identifier being looked up. In eachcase,wecomparethepurelystaticimplementation(“Static”) with theonethatusesdynamicdependencieson topof astaticschedule(“Dynamic”). Table3 repeatsall

17

798;:=<�>=?A@Table3

Incrementalityresultson artificial testcase(balanced).

List Table

Static Dynamic Static Dynamic

visits time visits time visits time visits time

Initial 19099 847 19099 773 19099 435 19099 302

NOPedit 3121 71 67 0 3121 22 67 0

Changedtype 11213 719 73 2 11213 153 73 0

New global 11217 652 5042 658 11217 150 541 6

theseexperimentswith balancedtrees.Theinitial evaluationshave thesamenumberof visits for boththepurelystatic

case,aswell astheoneusingselecteddynamicdependencies.Whetheror not weusea list or a tablealsomakesno differencein thenumberof visits. (Thenumberof visits is increasedin thebalancedcasebecauseof thegreaternumberof nodes.)Thepurelystaticimplementationhoweveris slowerbecauseof theadditionalvalue-carryingattributes.In thedynamiccase,thegeneratedattributeshaveanaffectonlyon schedules;they do not carryvalues.The tableimplementationsarealsofasterthan the list implementationsbecausea hashtable of 1000 items permitsfasteraccessthana list of thesamesize.

In theNOPedit case,we still have a largenumberof visits in theunbalancedcasebecauseof thespinemarking.We alsoseethatdynamicdependenciespermitusto avoid redoingwork. Thesebenefitsareespeciallyvisible in thebalancedcasesincethenumberof visitsarecutdrastically. Evenin theunbalancedcase,thevisitsthatareredonewhendynamicdependenciesareusedarecheaper.

The third case,changingthe type of oneof the global variables,againshowsthe benefitsof dynamicdependencies.In the purely staticcase,the incrementalupdatelosesprecision;it only determinesthat oneor moreglobal variableshavenew types.Almostthewholetreemustbere-evaluated.Ontheotherhand,dynamicdependenciesgive the neededprecision;only usesof the changedglobal needtoberevisited. Balancedor unbalanced,usinglists or tables,theevaluatortakesfiveor fewer milliseconds. Sincethe resolutionof the timing is the millisecond,thedifferencesareprobablynot significant.

The last case,addinga new global variable,shows how the tableabstractionpermitsthesedependenciesto be even moreprecise. Without the help from thetable,thedependenciesaretoocoarseto beany betterthanthepurelystaticcase.

Thesenumbersallow oneto draw sometentative conclusions.Usingdynamicdependenciespermitsgreaterprecisionandhencebetterperformingin incremen-tality, especiallywhencombinedwith a specialimplementationof tables.Interest-ingly, althougha balancedtreegivesoneasymptoticallybetterperformance,even

18

798;:=<�>=?A@

a treewith over 10000nodesis not big enoughto show muchbenefit.Essentiallythis is becausetheextra visits thattakeplaceareextremelycheap,whereastheun-avoidablevisitsperformnon-trivial tasks.Thenumberof visitscanbeapoorproxyto evaluationtime.

This work is only preliminaryandinvestigative. In future work, I would liketo experimentwith morekindsof changesandwith largerlanguagedescriptions.Iamalsointerestingin integratingtheprototypewith a softwareenvironmentwithfine-grainedversioningsuchastheFluid project[4].

4 Conclusion

This papershows how remoteattribute grammarscanbe implementedincremen-tally throughcooperationbetweentwo techniques:u A staticscheduleensuresthatfieldsarefully definedbeforebeingused;u Dynamicdependenciesensurethatremoteusesof changedfieldsaremarkedas

edit sites.

Without the staticschedule,edit sitesdeterminedthroughdynamicdependenciesmight bemarked“too late”, that is, afterevaluationhadalreadybedone.Withoutthedynamicdependencies,thestaticschedulewouldn’t know whetherit couldskipasubtreeor not,sinceit might haveusesof changedfields.

The preliminary resultsshow that this cooperationpermitsedits suchas theadditionof a globalvariableor thechangingof thetypeof a variableto betrackedpreciselythrougha large program. The precisionenablesa 50-fold speedupinincrementalevaluationin anexamplewith 10000nodes.

Acknowledgement

I thankWilliam Retert,JoaoSaraivaandtheanonymousreviewersfor theirhelpfulcomments.

References

[1] Boyland,J. T., “DescriptionalCompositionof CompilerComponents,” Ph.D.thesis,University of California, Berkeley (1996),availableas technicalreportUCB//CSD-96-916.

[2] Boyland, J. T., Analyzingdirect non-local dependenciesin attribute grammars, in:K. Koskimies,editor, CompilerConstruction:7th InternationalConference, CC’98,LectureNotesin ComputerScience1383 (1998),pp.31–49.

[3] Carle,A. andL. Pollock,On theOptimality of Change Propagation for IncrementalEvaluationof Hierarchical AttributeGrammars, ACM Transactionson ProgrammingLanguagesandSystems18 (1996),pp.16–29.

19

798;:=<�>=?A@

[4] Chan,E. C., J. T. Boyland andW. L. Scherlis,Promises:Limited specificationsforanalysisandmanipulation, in: Proceedingsof theIEEE InternationalConferenceonSoftware Engineering(ICSE’98) (1998),pp.167–176.

[5] Gray, R. W., V. P. Heuring, S. P. Levi, A. M. Sloaneand W. M. Waite, Eli: Acompleteflexible compiler constructionsystem, Communicationsof the ACM 35(1992),pp.121–131.

[6] Hedin, G., An overview of door attribute grammars, in: P. A. Fritzson, editor,Proceedingsof CompilerConstruction,5th InternationalConference, CC’94, LectureNotesin ComputerScience786 (1994),pp.31–51.

[7] Hoover, R., Dynamically bypassingcopy rule chains in attribute grammars, in:ConferenceRecord of theThirteenthACM Symposiumon Principlesof ProgrammingLanguages(1986),pp.14–25.

[8] Hoover, R. andT. Teitelbaum,Efficient incrementalevaluationof aggregate valuesin attribute grammars, in: Proceedingsof the ACM SIGPLAN’86 SymposiumonCompilerConstruction(1986),pp.39–50.

[9] Johnson,G. F. and C. N. Fischer, A meta-language and systemfor nonlocalincrementalattributeevaluationin language-basededitors, in: ConferenceRecord ofthe Twelfth ACM Symposiumon Principlesof ProgrammingLanguages (1985),pp.141–151.

[10] Kastens,U., Attributegrammarsin a compilerconstructionenvironment, in: H. AlblasandB. Melichar, editors,AttributeGrammars,ApplicationsandSystems.InternationalSummerSchool SAGA Proceedings, LectureNotesin ComputerScience545 (1991),pp.380–400.

[11] Kastens,U. andW. M. Waite,Modularity andreusabilityin attributegrammars, ActaInformatica31 (1994),pp.601–627.

[12] Maddox, W., “IncrementalStatic SemanticAnalysis,” Ph.D. thesis,University ofCalifornia,Berkeley (1997).

[13] Parigot,D., G. Roussel,M. JourdanandE. Duris, Dynamicattribute grammars, in:H. KuchenandS. D. Swierstra,editors,InternationalSymposiumon ProgrammingLanguages, Implementations,Logics and Programs(PLILP ’96), LectureNotes inComputerScience1140, Springer-Verlag,Berlin, 1996pp.122–136.

[14] Reps, T., “Generating Language-BasedEnvironments,” Ph.D. thesis, CornellUniversity(1982).

[15] Saraiva, J., “Purely FunctionalImplementationof AttributeGrammars,” Ph.D.thesis,Departmentof Computer Science,Utrecht University, The Netherlands(1999),ftp://ftp.cs.uu.nl/pub/RUU/CS/phdtheses/Saraiva/.

[16] Saraiva, J.,D. SwierstraandM. Kuiper, Functionalincrementalattributeevaluation,in: D. A. Watt,editor, CompilerConstruction:9thInternationalConference, CC2000,LectureNotesin ComputerScience1781 (2000),pp.279–294.

20

798;:=<�>=?A@

[17] Swiestra,D. and H. Vogt, Higher order attribute grammars, in: H. Alblas andB. Melichar, editors,Attribute Grammars, Applicationsand Systems.InternationalSummerSchool SAGA Proceedings, Lecture Notes in Computer Science 545,Springer-Verlag,Berlin, 1991pp.256–296.

[18] Vorthmann,S. A., Coordinatedincrementalattribute evaluation on a DR-threadedtree, in: P. Deransartand M. Jourdan,editors, Attribute Grammars and theirApplications. International Conference WAGA Proceedings, Lecture Notes inComputerScience461 (1990),pp.207–221.

21