Compositional Modeling for Complex Spatial ReasoningTasks*

Kyungsook Han and Andrew Gelseykshan©cs.rutgers.edu ge.lsey1~cs.rutgers.edu

Departmentof Computer ScienceRutgersUniversity

New Brunswick, NJ 08903U.S.A.

Abstract

Reasoiungabouta complex physical systemgenerallyrequiresthe creationand executionof a model of thesystem, the creationof wluch in turn dependson thetypes of knowledge availablefor the physical systemand their representation. Such a model is normallycreated by the person studying the system. Despitethe considerabletime andeffort spent,a hand-craftedmodel is often error-prone. Modifying a hand-craftedmodel to solve a similar problem about other physi-cal systemsis also (hificult, and may take more timetItan building a new model for the systems. We (IC-scribe a niethod which uses first principles to auto-matically create models and simulators for spatiallycomplex motions. This method solves severalprob-lems with existing Al modeling work on motion by:(1) explicit handling of vector quantities and framesof reference; (2) simultaneous handling of multipleequations (algebraicor differential, linear or nonlin-ear); and (3) declarative,algorithm-neutral represen-tation ofphysics knowledge. The method hasbeen im-plementedin a working program called ORAcLE andtestedin the domains of mechanicaldevicesand sail-boats. Experimental resultsshow that ORAcLE ~S Ca-pable of generatingcorrectmodels of several(lifferenttypesof physical systemsif enough domain knowledgeis available.

Introduction

Spatial reasoningproblemsareconsideredin a varietyof areas,but different areashavedifferent spatial rea-soning tasks. In computervision, for example,recog-nition of familiar objectscan be the main spatial rea-soning task. Our focus in this paperis on modelingphysicalsystemsfor reasoningaboutspatiallycomplexmotion of the systems. By spatially complex motionwe meanthe motion of multiple moving objectsin ar-bitrary configurationsin three dimension. Motion inthree dimensionis much more difficult to understand

~This researchwa.spartiallysupportedby theAdvancedResearchProjectsAgency (ARPA) and National Aeronau-tics and SpaceAdministration under NASA grant NAG2-645 and by the National ScienceFoundationthrough grantCCR-9209793.

anddealwith than themotion in oneor two dimensionbecausemodeling sucha motion involves reasoninginvector space.

There are several works on modeling motion, hutspatiallycomplexmotions did not getmuchattention.Somequalitativephysicsapproacheshavebeenusedtomodel motion. But they focus on developingrepresen-tations for physicalsystemsand reasoningabout thesystemswithin the representations,and their capabil-ity of reasoningabout motion is limited to simple mo-tions only. Consider, for example,a spring with oneend attachedto a fixed point and the other end at-tachedto ablock. What happensif theblock is pulledandrotatedfrom its equilibrium position andreleased,as illustrated in Figure la? This spring-block systemis different from a linear harmonicoscillator,which isacommontextbook exampleoften used in qualitativephysics research. It is well known that the harmonicoscillator has one degree of freedom, i.e., displace-ment of the block from its equilibrium position, andits motion is oscillatory on a straight line. However,predicting the behavior of the spring-blocksystem inFigure la is not assimple as the harmonicoscillator.Manyqualitativephysicsapproacheswhich can modelthe harmonic oscillator (Forbus 1984; Kuipers 1986;Struss 1988; Weld 1988; Williams 1986) cannot han-dle this spring-block system. The primary reasonforthis difficulty can he attributed to the fact that qual-itative valuesare not adequatefor spatially complexproblemsand that they lack the ability to reasonex-plicitly about vector quantitiesand moving frames ofreference. Someresearchin spatial reasoningor corn-mercialmechanicssimulatorshavepowerfulalgorithmsto model motion, but they incorporateknowledgeofphysicalphenomenadirectly into algorithmsanddiffi-cult to reuseor extendthem to solvesimilar problemsabout otherphysical systems.

The work of this paper is motivated by two goals.The first goal is to automatethe model fornmlationand simulation processfor complexmotion. The sec-ond goal is to make the modeling processas generalas possible so that common domain theoriescan besharedand reusedinsteadof being duplicated. As we

~iii~~(a)

I ~---R----::H(c)

Figure 1: (a)1 block attached to a spring. The block is pulled and rotated from its equilibrium position andreleased. (h) 2 blocks connectedby 1 spring. Both blocks are pulled in opposite directions and released. (c) 3blocks connectedby 2 springs. The middle block is pulled directly to the side and released.

Figure2: Stars~4Stripes,winnerof the 1987 America’sCup competition.

will show later, we handleexplicitly vector quantitiesand referenceframes, and construct a model from aset of model fragments. The model fragmentsrepre-sent the fundamentalphysics knowledgein a declara-tive andalgorithm-independentway, andarereusedinbuilding modelsof different typesof physical systems.

The remainderof this paperdiscussesa frameworkfor automaticallycreating and simulating behavioralmodelsof moving physicalsystems,andillustrates theframeworkusing thesinglespring-blocksystemin Fig-ure la as a running example. We also discuss theepistemologicaladequacyof theframeworkfor broaderclassof physical systems,and addressrelated issuessuchas: Can the modelingsystemof the singlespring-block system be used to predict the behaviorof moregeneral spring-block systemssuch as Figure lb andFigure lc, or different typesof physicalsystemssuchas a sailboat shown in Figure 2? Do we needa sep-aratemodeling systemfor every physicalsystem? Orthe samemodeling systemjust with more knowledgewill suffice to model them?

Framework for Model Building andSimulation

In this section we describe the framework of our model-ing systemcalled ORACLE, implemented in the math-ematical manipulation languageMaple (Char et at.

1991).

Ontology and RepresentationThe principal elementsof ORACLE’S ontology are en-tities, phenomena,model fragments,andmodels,eachrepresented in a frame (Minsky 1975). Art entity is aphysical object which either constitutes a physical sys-tem by itself (i.e., primitive object) or is a part of aphysical system (i.e., compositeobject). The proper-tiesof anentity areexpressedasvariablesin equations.The block entity, for example,haspropertiessuch asposition and velocity. An entity is representedin aframe with slots for the properties. Facetsallowed ina slot are value, form, range, if.needed, and if_added.The value facet is initially set to null hut will he as-signeda vector, scalar, string, set, or any other expres-sion as it becomesknown. Theformfacet distinguishesthe slot type (e.g., scalaror vector) and is consultedwhen the system creates a new Maple variable nameduring the problem solving process. For example, ifthe system is asked to compute the position of a blockhl, aset of new variables {hlx(t), bly(t), hlz(t)} willhe created for the position vector and used in equa-tions. The range facet specifies a valid range of theproperty value if it is known. The if_needed facet orif_added facet holds the procedure call, invoked whena slot value is needed or added. The if_needed proce-dureof the velocity slot in the examplebelowsaysthatvelocity is derivable from position.

block [AKOrigid...body,position(t)=Evaluesanull, forxn~=[x(t) ,y(t) ,z(t)]],velocity(t)=[value=null, form’~Cu(t),v(t),w(t)I,

±f_neededEderive_velocity, position(t)JJ,- (other slots not shown) - . . - I

A phenomenonis a process which changes one ormore properties of anentity in a physical system. Force

from a spring, for example, is a phenomenon whichcan change the position and/or orientation of an entitywhich is attached to the spring.

A modelfragmentis a characterization of a physical

phenomenonby a set of entities, variables, assump-tions, and equations. There may be more than onemodel fragments for a single phenomenon, each withdifferent assumptionor approximation. The equationsof a model fragment are applicable when the corre-

(b)

sponding phenomenonoccurs. The spring force, forexample,exerted on an object attachedto end2 of alinear spring with lineardampingis representedas fol-lows (syntaxslightly modified for readability):

Springforce2’~[phenomenon=’sprirtgforce at end2’,entit ies=Es=linear_dantped_spring],variables’~’[ks [force_const],

b~’sEdamping_coeff],e1(t)~’s[end1(t)], e2(t)=s[end2(t)].l=sCrest...length], f(t)”sCforce2(t)]],

equations=Ef(t)=—k*(11e2(t)—el(t)I I—l)*(e2(t)—el(tfl/I e2(t)—el(t)b*diff((i 1e2(t)—el(tfl —l)*(e2(t)—el(tfl/I 1e2(t)—el(t) II ,t)]]

It saysthat s is alinear damped spring, k is a forceconstantof thespring, b is adampingcoefficient,el(t)ande2(t) are the positionvectorsof endl andend2, 1is the rest length,andf(t) is the spring force at end2.

e2(t) — el(t) is the vector norm representing thelength of the spring at time t, e2(t) — el(t) —l isthe signed length change from the rest length, and(e2(t) — el(t))/ e2(t) — el(t) M is a unit vector withdirection from endi to end2.

A modelis acompositionof model fragmentsappli-cable to a physical system in a particular situation.Simulation is the execution of a model.

The motion of an entity at any instant can be de-scribed by a set of ordinary differential equationsinthe twelve components of four vectors: position, orien-tation, velocity, and angular velocity.’ The differentialequations are usually nonlinear and do not have a solu-tion in closed form, so they must be solved by numericintegration. For a moving entity, ORACLE constructsamodel with the twelve components of the four vectors(position, orientation, velocity, and angular velocity)as statevariables, which take numeric values duringsimulation.

The state variables of each subpart of an entity areinitially defined in the local reference frame, which is

assumed to be fixed to the entity. Then each subpartdefined in its local reference frame is translated androtated by having its reference frame redefined in acommoninertial referenceframe. The system choosesthe common inertial referenceframefrom local refer-enceframes which are not accelerated.If there is nosuchreferenceframe(i.e., all thelocal referenceframesarenoninertial), it introducesa new inertial referenceframe. If there are several inertial referenceframes,the choiceis arbitrary.

‘The degreesof freedom of a moving entity are six in-steadof twelve becausethe velocity function amid thean-gular velocity function arc derivableby differentiating theposition and orientationfunctions, respectively. The mo-tion of a physical system with ii subpartscan be charac-terizedby maximum12n statevariableswith 6n degreesof

AlgorithmORACLE takesas input a descriptionof a physicalsys-tem in terms of the entities of the system, any con-straints to he satisfied,and the propertiesof the enti-ties (i.e., variables)whosevaluesare going to he com-putedby modeling andsimulation. As output, it pro-ducesamodel of themotionof thesystemandthevari-able valuesobtainedby solving the model. The algo-rithni of ORACLE consistsof threephases:(1) problemanalysis, (2) model creation, and (3) model execution.In the first phase, ORACLE represents each entity of aproblem statement in a frame by copying a classframeand filling in slots for property values specified in the

Algorithm 1 ORACLE’S top-levelalgorithm

Problem Analysis Analyzeaproblemstatement.

1. Analyze entities, and create frames of the entitiesanda set INIT of initial-value conditions.

2. For each constraint, determine its type and representthem in equations.

3. Analyze variables and generate a set DRVD of dif-ferential equations.

Model Creation Searchfor relevantmodelfragmentsandcomposeabehavioralmodel with them.

1. For each entity B of the problem statement

For each model fragment MF indexed by the“mf” slot of B

If MF has not been instantiated for BAND every variableof MF eithercorrespondsto an entity property or variableof the input or can he derivedfrom themAND the assumption(if any) of MF doesnot violate any entity property or constraintof the input

Put MF in a list MFS.

2. model M = DRVD

3. #equations= #equations(M)4. retry: For eachmodel fragmentMF in MFS

(a) InstantiateMF for theproblem.(h) M = M u {MF}(c) #equations= #equations(M)(d) If #equations= #variables, do model execu-

tion.5. Print the dead-endsituation, and quit.

Model Execution Solve the model M either analyt-ically or by numeric simulation.

1. Determinethe typesof equationsof the model andsolvethem with INIT for thevariables.

2. If avalid solution is obtained,print the model andsolutions,andquit.

3. If a valid solution is not obtained, retract the mostrecentMF from the model and go to retry.

freedom.

problem statement. It also transformsvector quanti-ties expressedin the local referenceframes into thosein the inertial referenceframe,formulatesinitial condi-tions, andexecutesif_addedproceduresin the slots. Amodel fragmentspecifying forceson a componentof acompositeobject is instantiatedby if_addedproceduresin thethis phase.After constraintsareanalyzed,vari-ablesareexaminedto determineif their valuesareal-readyknownin their slot valuesor derivablefrom othervariables. In the secondphase,additional model frag-ments which have not been instantiated are retrievedand a model is constructed from them. In the finalphase,the constructed model is solved for the prob-1cm. If ORACLE runs out of potentially relevant modelfragments before it finds a valid solution, it prints thesituation, asks more information, and quits. The top-

level algorithm of ORACLE is outlined in Algorithm 1.

ExampleWeillustrate how ORACLE works with the spring-blocksystem of Figure in. Suppose the following problemdescription is given as an input. There is no particularconstraint in this problem and the system is asked tocompute the four vector variables (twelve variables incomponent form) of the block.

antities[bl=[block, massl,principal_moments_of _inertia=’[1/6,1/6, 1/6],position(O)~’[3,O,O]orientation(O)=[Pi/4,Pi/2,O],velocity(O)”[O,O,O],ang_velocity(O)C0,O,OJ],

s1’~[spring, force_constlO,damping_coeff=1/1O, rast_length3/2,endl(t)[O,O,O], end2(t)”bl[—1/2,O,O]],

sb”[composite_object, parts’~{b1,s1}]];constraints[ I;variables=Cbl Eposition(t)], bi [oriantation(t)],

bl[velocity(t)] , blEang_velocity(t)]J;

For eachentity hi, si, and sb, a frame is createdandthegiven propertiesof the entitiesarerecordedintheir slot values. The if_addedprocedurein the end2slot of si computesthespring force actingon hi usinga model fragment Springforce2and recordsthe valuein the force slot of hi. The position of end2 in theinertial referenceframeis computedfrom atranslationand arotation of the local referenceframe of bi. Theinitial conditionsof theblockarealsoformulated, Hereare the Maple variable namesORACLE assignsfor thisexample.

INIT =

{blx(O)ss3, bly(O)0, blz(O)0,blphi(O)=O, bltheta(O)”Pi/2, blpsi(O)=O,blu(O)=O, blv(O)”O, blw(O)=O,blomegal(O)0, blomega2(O)0, blomuega3(O)0}

None of the four state variables of hi can be as-signed a value simply by looking at slot values of hi,but the velocity and the angular velocity functionscan be derived by differentiating the position and the

orientation functions, respectively,accordingto theirif_neededfacets. The systemgeneratestrivial differen-tial equationsfor the velocity andangularvelocity bythe proceduresattachedto the if_neededfacets.

DRVD =

{blu(t)diff(blx(t),t),b1v(t)’~diff(b1y(t),t),b1w(t)’~diff(b1z(t),t),blomegal(t)=diff(bltheta(t),t)*cos(blphi(tfl+

diff(blpsi(t),t)*sin(bltheta(tfl*sin(blphi(t)),blomega2(t)diff(bltheta(t),t)*s±n(blphi(tfl—

diff(blpsi(t),t)*sin(bltheta(tfl*cos(blphi(t)),blomega3(t)diff(blphi(t) ,t)+

diff(blpsi(t) ,t)*cos(bltheta(t))}

Now ORACLEfocuses on finding equations for theposition and the orientation. The equations for themcannot be derived from other variables since they arebasic variables, so ORACLElooks for relevant model

fragments. It examines model fragments, indexed bythe mf slot of theblock. ORACLEdecides that Newton2and Euler are potentially relevant because the entities(solid and rigid body, respectively) of the model frag-ments are superclass of a block and the equations ofthe model fragments contain at least one variable ofthe problem. Model fragments of Newton2 and Eulerare as follows.

Newton2 Cphenomenon’ Newton’s second lawof motion’,

entities=[r=solid],variables=[f(t)r[net_force(t)]

p(t ) ‘6r [momentum(t) I]assuinptions[ 3,equations=[f(t)=diff(p(t) , t)]]

Euler[phenomenon ‘time—dependency ofang_velocity’,

entities [brigi&body],variables=[Qmega(t)b[aitg_velocity(t)],

M(t)=b[ang_momentum(t)],T(t)a~b[net_torque(t)J]

assumptions[ 3,equationss6s[add(diff01(t), t)

crossprod(Omega(t),N(t) ) )=T(t)]

The entity and variable names of the model frag-

ments are instantiated as those of the problem andtheyaresubstitutedin theequationsof themodelfrag-ments. The angular momentumis derived from prin-cipal momentsof inertia and angularvelocity by theif_neededprocedurein the aug_momentumslot. Like-wise, the net torqueis derivedfrom force andpositionvector of the point at which the force acts.

/ I,i!,(t) \M(t) = ( I

2~l2

(t) ), T(t) = ~r x f(t)\ 1

311

3(t) J

The principal moments of inertia (Ii, 12, 13) and theposition vector (~-)of the spring-attachedpoint can beassignedfrom the information of t,he problemdescrip-tion. The angularvelocity is oneof thestatevariables

Figure3: Plots of the 12 state variablesof the block hi as functions of time. The first column showsthe positionvector, the secondcolumn the orientation vector, the third column the velocity, and the last column the angularvelocity.

Figure 4: Motion of theblock hi. Spring not shown.

Figure 5: The kinetic, potential, and total energyof thesingle spring-block systemas functions of time during thesinmlation.

I)

z,, 1J~

H H hH

KE PE

askedby the problem,and its functions arederived inDRVD. The value of force 1(t), which hascomputedusinga model fragmentSpringforce2(shown earlier insection ), is availablein the force slot of bi, and sub-stituted in the equationsof Newton2and Euler.

Thesystemhasnow total 12 equationsin componentforms (6 from themodel fragmentsand6 from DRVD)plus 12 initial conditionsfor 12 unknowns. Severalofthedifferential equationsarenonlinear,andwhen OR-ACLE attemptsto solvethe model analytically, it doesnot find asolution in closedform. ORACLE thensolvesthedifferential equationsby numericsimulation. ORA-CLE displaysthesimulationresultby showingthestatevariablesas functionsof time usinggnuplot (Figure 3).Animationof the movingblock is shown(Figure 4) us-ing PADL-2 solid modeling system (Hartquist 1983).Figure 4 contains several animation scenes superim-posed. Note that the motion of the block is much morecomplex than that of the linear harmonic oscillator.

The kinetic energy,potential energy, andtotal energyof the system are also displayedaspart of validationcriteria of the results (Figure 5). The total energy inFigure 5 decreasesover time dueto the nonzerodamp-ing coefficientof the spring of the problem statement.

Extension to Broader Class of PhysicalSystems

Multiple Spring-block Systems

The previoussectionshowedhow ORACLE predictsthebehavior of the single spring-block system. Can themodeling system of the single spring-blocksystem heused to predict the behavior of the multiple spring-blocksystemssuch as Figure lb and Figure ic? Theanswer is “yes”. The multiple spring-block systemshaveadditional entities and phenomena,but they aresimply the multiple occurrencesof the sametypes asthesingle spring-blocksystem. Havingalreadyenoughknowledgerepresentedin general form to handle thesingle spring-block system, ORACLE can handle themultiple spring-block systemswith no change. Theway it solves theproblem is the same.It computesthepositionsof endsof eachspring in the inertial refer-ence frame by transformingthe local referenceframeof its associatedblock, and derives differential equa-tionsfor thevelocity andangularvelocity of theblocksfrom the proceduresattachedto the if_neededfacets.It then instantiatesmodel fragmentsof Spring force,Newton2 and Euler, and composesa model. Noticethat model fragmentsharingoccurswithin the modelsbecauseeachof thosemodel fragmentsis instantiatedmore than once for different entities. The result of theexecutionof the models indicate that although noneof the blocksareinitially rotated, both blocks of Fig-ure lb andtheendblocks of Figure ic rotateaswell astranslatedue to spring forceswhich arenot parallel totheradiusvectorsof thepoints to which the springsareattached.If the springdampingis ignored(i.e., damp-

ing_coeff = 0), the middle block of Figure ic showstranslational motion only, hut it shows both transla-tional and rotationalmotions if the spring damping isconsidered(damping_coeff~ 0). Figure 6 showspartofanimation scenesfor the casewhen the spring damp-ing is considered. In fact ORACLE is able to handlemultiple rigid bodiesconnectedby springs in arbitrarypositions and orientations becausethe way of identify-ing relevantmodel fragmentsandcomposing them isI1ot restricted by the number of entities or their con-nections.

Sailboat in FluidA sailboat is a compositeobject whose driving forcecomes from the differential motion of air over water.Before we model the sailboat, we can ask the samequestion as before. Can we usethemodelingsystem ofthe spring-block systems to predict the behavior of a

sailboat in fluids? The answer is “yes”, provided thatthe modeling system has enough domain knowledgeto handlethe problem. We do not need to build adifferent modeling system. A modeling system withthe same algorithm and the same model fragments plusadditional model fragments and entities can predict the

behaviorof the sailboat.New classesof entitiesaddedto theknowledgebase

are fluids (water and air) and lifting surfaces (hulland sail). The sailboat, water, and air entity havetheir own referenceframes,which moveas their enti-ties move. New phenomenainclude hydrodynamicandaerodynamic forces, each with two components (liftand drag), and skin friction. A single model fragmentis used to representboth hydrodynamicand aerody-namicfrictional drag forces, and later instantiated forthem. Likewise, a single model is used to representboth hydrodynamicandaerodynamiclift forces.

FDrag=[phenomenon=’frictional drag force onan object in fluid’,

entities=[s=physical_objoct, f”fluidJ,variables=[FD=sEfdrag(t)],

v=strel.fluid_speed(t)]fd~’s[rel_fluid_direction(t)]Pas[parasitic_area],rho~’f[densityJ],

assumptions~’[ 3,equations[FD=1/2*Pa*rho*v2*fd]]

Lift”[phenomenon’ lift and lift induced force onan object in fluid’,

entities=[s=physicalobject, ffluid],variables[LFs[lift(t)J,

L~’s [lift _magnitude(t)],vs [rel_fluid.speed(t)Jfd=s [rel_fluid_direction(t)]pds [perpendicular.rel_fluid_dir(t)J,Cas[effective.capture_area],rho=f [density]],

assumptions”[s[rel...fluid_speed(t)] > 03equations[LFL*pd+L2/(2*Ca*rho*v2)*fd]]

For the hull, the rd_fluid_speedis the speedof theboat relative to water. For the sail, therd_fluid_speed

Figure 6: Motion of themultiple spring-blocksystemsin Figure lb and Figure ic. Springsnot shown.

Figure 7: Directionsof forcecomponentsonasailboat,adaptedfrom (Letcher, 1976)

is the speedof the boat relative to air. The magnitudeanddirectionof thehydrodynamicforcesacting on thehull dependon the rd_fluid_speedof the hull, and theaerodynamicforcesdependon thereLfluid..speedof thesail. rd_fluid_direction is an anglebetweenthe direc-tion of fluid andthe directionof an object, representedin angle. Drag forces actingon the hull havecompo-nents opposite to the direction of its rd_fluid_speed.Lift forces on the hull are perpendicularto the direc-tion of its rd_fluid_speed. Similarly, drag forces onthe sail havecomponentsopposite to the direction ofits reL.fluid_speed,and lift forces on the sail are per-pendicularto the direction of its rel..fluid_speed. Liftforceson thehull andsail of asailboatare horizontal,not vertical.

The directions of the force componentsare summa-rized in Figure 7. In the figure, wateris assumedto beat rest (i.e., speed= 0) and V~,~ and A denotethewind speed,sailboatspeed,andcourseanglefrom thewind direction, respectively.

Notice that themodel fragmentLift hasanonemptyassumption slot, sa~ring that the relative fluid speedmust be positive. When both the fluid and the ob-ject are at rest or the fluid hasthe samespeedas theobject in the samedirection, the relative fluid speedbecomeszeroand theequationsof the model fragment

cannot be defined due to zero denominator. Thus wehave another model fragment of lift with a differentassumptionslot; it says that lift force is zero when therelative fluid speed is zero.

Lift_at_zero_speed=[phenomenon”lift and lift induced

force on an object in fluid’,entities~’[s”physical_object, f=fluid],variables=[L=s[lift(t)]

svs [rel_f].uid_speed(t)]],assumptions=[s[rel_fluid_speed(t)] = 03equations [L[13 =0,

L [23=0,L [33=03]

During problem solving, ORACLE automaticallychooses between the two model fragments of lift bychecking their assumptions. At present, the kinds ofassumptions ORACLE can processare confined to thealgebraicpropertiesof entities, suchasnumericrangesor arithmetic expressions.

After the entities and model fragments are added,

ORACLE can solveseveraltypesof problemson a sail-boat,but we will focuson onetype of problemin thissection. Supposethat a sailboatis headingin the an-gle of 49 degreesfrom the direction of wind at uniformspeed16.9 ft/sec and that water is at rest. The sys-tem is askedto computethesailboatspeedwhich willbalanceall the forcesinvolved.

ORACLE first infers all the forces on the sailboatfrom the forcesactingon its components,hull andsail.It instantiatesthe model fragmentsFDragandLift foreachof them and records the summation of them inthe netIorce slot of the sailboat.

F = ~ (FDrag~+ Lift~)

iC (hull, sail)

It then searchesfor a model fragment which relatesforces with speed, and finds the model fragment ofNewton2. It substitutes the equationsof the forcesin the equationof Newton2, F(t) = d(p(t))/dt. Sincethe problemstatesthatall the forcesarebalanced,thenet force on the sailboatmust be zero, implying themomentum p(t) is constant. The right hand side ofthe equationbecomeszero from the constantmomen-tum, resulting in an algebraicequation. However,the

hull liftVt

1614

120

0

Figure 8: The sailboatspeedasa function of time.

problem is under constrainedin the sensethat totalnumber of equationsin componentform is 2 (Fx=0,Fy=O, Fz becomesatrivial equation0=0) but the to-tal number of unknownsin the equations is 3 (boatspeed, sail lift magnitude, and hull lift magnitude).ORACLE prints the situation, asking for further in-formation. The user provides an additional equation,O(Fx)/O(sail_Iift.magnitude)= 0, by making asimpli-fying assumptionthat thesail is controlledas to max-imize the sailboatforce in the direction of boat head-ing. The equationsaresolvedalgebraically,producingasolution, boat speed= 15.7 ft/sec. The solution ischeckedagainsttherangefacet of thespeedslot of theboat,which saysthat value of the boatspeedmust bein the rangeof [0 .. infinity]. Sincethe solutionof 15.8is includedin the range,it is returnedto the userasavalid solution. However, if thereare severalsolutions,only thosewithin thevaluesspecifiedby therangefacet(if any) are selectedas valid solutions. When the in-formation in the rangefacet is not sufficient to choosea correct solution from multiple solutions, additionalor alternativeprocedurefor filtering correct solutionsis to use the result of the numeric simulation of itscorrespondingdiffr~rentialmodel, as we will show be-low. A correct solutionof analgebraicmodel describ-ing a behaviorin a steadystate must agree with thenumericsimulationresultof its correspondingdifferen-tial model. This procedureof validating thesolutionofan algebraicmodelagainst thenumericsimulation of adifferentialmodel hasnot beenautomatedin the cur-rent implementationof ORACLE, and the usershouldtry both models to comparetheir results.

The previous exampleshowed how ORACLE coni-posesa model to compute the sailboat speed in theequilibrium stateof forces. If we areinterestednot onlyin suchaspeedbut also in how the boatarrives at thespeed,startingfrom zerospeed,theboat speedmust hecomputedasafunction of time. Relevantmodel frag-mentsare retrievedandinstantiated in asimilar way.In this case,however, the net force on the sailboatisnot necessarilyzero all the time becausethe boat ac-

celeratesuntil it reachestheequilibriumstateof forces.Therefore, the right hand side of theequationof New—ton2 doesnot becomezero, hut stays as d(p(t))/dt.Sincep(t) = d(m - v(t))/dt, ORACLE solves the differ-ential equation,F(t) = d(m - v(t))/dt for v(t) by nu-meric simulation. A plot of the simulation result inFigure 8 showsthat thesailboatultimately acceleratesto the same speedas the one predicted by the alge-braic method, thus confirming the algebraicsolution.Also noticethat themodel fragmentNewton2usedformodeling the spring_blocksystemsis reusedfor mod-eling thesailboatand that model fragmentsLift andFDragaresharedby hull and sail.

Issues in Scaling UpThere are several problemsraised in scaling up ORA-

CLE not only to broaden the types of physical systemsto he modeled but also to model a physical systemwith a large number of componentsand phenomenainvolved.

First, the size of a model generated by the currentversion of ORACLEcan restrict scaling up either be-causeof practical limitations of solving a huge modelor becausesolving a huge model takes too much timeto be useful. Table 1 showsthesizesof themodelsandthe times for formulating and solving the models forthe examples shown in this paper. The model sizesofspring-blocksystemswith different configurations from

the examplesin Table 1 are about the same as thoseof the spring-block systems with the same number ofblocks and springs in Table 1, that is, the model sizesof spring-blocksystemsareindependentof their config-urations. As we can see in Table 1, the size of a modelis not directly proportional to the number of modelfragmentsinstantiated. Rather it is proportional tothe numberof variablesspecifiedin the problemstate-ment, or to the number of unknownsin theequationsofthe model (#unknowns includes #variables and newlygenerated variables in the equations).

As describedearlier, a model contains aset of vari-ables,a set of assumptions,a set of namesof modelfragments,anda list of equations.The equationsof amodel are compositionof the equationsof the modelfragmentsinstantiated to construct the model. Al-though the equations of most model fragmentsexist ina very short, simple form beforeinstantiation, theymaybecomevery long andcomplexafter they are instanti-atedfor theparticularentities andphysicalphenomenain the problem. This explains the largevariations inthe sizesof modelswith similar numberof model frag-ments; the big size of a model is attributed to the long

equations of instantiated model fragments. Even fora same model fragment, the equations after instantia-tion can he very different in their sizes depending onfor which variables they are to be solved and which

variables of the equations are known.Figure 9 showsthegrowth rate of themodelsizeand

the growth rate of the timne for modeling and simula—

0 20 40 60time (sec)

80 100

example #variables #unknowns #model fragments #lines of modelmodel fragments(#instantiation)model_gentime modeLsol time display_savetime total time

algebraicmodelof boat

1 3 5 30FDrag(2), Lift (2), Newton2(1)4.900 4.616 4.917 14.333

differentialmodelof boat

1 3 6 31Newton2 (1), FDrag (2), Lift (2), Lift_at_zero_speed (1)4.700 18.700 4.716 28.116

oneblocksystem

12 12 3 206Newton2(1), Euler (1), Springforce2(1)14.583 19.800 14.833 49.216

twoblocksystem

24 24 6 2010Newton2(2), Euler (2), Springforcei (1), Springforce2(1)108.533 118.283 110.433 337.249

threeblocksystem

36 36 10 5330Newton2(3), Euler (3), Springforcei (2), Springforce2(2)339.350 264.616 344.767 948.733

Table 1: Size of amodel andtime for formulatingandsolving it for eachof the ORACLE examples.#variahlesis thenumber of variablesin componentform, specified in the problemstatement;#unknowus includes#variablesandnewly generated variablesin the equations;#model fragmentsis the total numberof mqdel fragmentsinstantiatedfor themodel; #lines of model is acountof lines of the model; model_gentime is CPU time for generatingamodel;modeLsol time is CPU time for solving the equationsof amodel; display_savetime is CPU time for displayingtheresultof solving the model andsavingall the results; the units of the CPU times are in seconds.

6000

5000

4000

3000

2000

1000

00

Figure 9: The size of a model andthe time for formu-lating andexecutingthe model. For the count of linesandtimes of amodelwith 3 unknowns,theaverageval-uesof the 2 modelswith 3 unknowns(algebraicmodelof boat,differentialmodel of boat) areused. The unitsof the model sizesa~ein the line countsof modelsandthe units of the timesare in seconds.

tion as a function of the number of unknownsof thea model. The size of a model is measuredin termsofthe number of lines of the model. When the numberof unknownsincreasesfrom 3 to 12, themodel sizein-creasesfrom 30.05 lines to 206 lines, which is almost7 times. This is becausethe spring-blockexample in-volves much morecomplex computationwhich resultsin long equations.It is alsonotablethatwhen thenum-berof unknownsincreasesfrom 12 to 24, themodel sizeincreasesfrom 206 lines to 2010 lines, about 10 times,andthat whenthe numberof unknownsincreasesfrom24 to 36, the model size increasesfrom 2010 lines to5330 lines. Given limited data, the growth rate of amodel is roughlyquadraticin thenumber of unknowns.The total time for generatinga model and solving itis also proportional to the numberof unknownsof themodel, but the growth rate of time is not as fast asthat of the model size, asshownin Figure 9.

Someof the largemodelshavelong equationswhichcannotbe simplified further in their natureunlesswedecide to produce approximatemodels instead. How-ever,somemodelsmayhesimplified without losing ac-curacyof their predictionsby doingadditionalprocess-ing on the equationsinsteadof applying Maple-builtinsimplification functions. Reformulatingthemodel gen-eration and solving processof ORACLE is anotherdi-rection to considerin order to efficiently constructandsolve a large model. Restricting ORACLE to a cer-tain type of physical system is anotherway to scaleup the modeling system to handle a complex physi-cal system with a large number of componentsand

5 10 15 20 25 30 35 40#unknownsof theequationsof amodel

phenomenainvolved. For example, if we want a spe-cial purposemodeling systemfor spring-blocksystemsonly, we can makethemodeling systemgenerateasin-gle model that works for any number of blocks. Asthe numberof blocks increases,the model would needmoredatastoragebut themodel itself doesnot becomelarger. This kind of approach to scaling up a model-ing systemhasadvantageof beingable to model andsimulate a complex physical system without the sizeproblemof ageneratedmodel, but hasdisadvantageoflosing the breadthof physicalsystemscoveredby themodeling system.

Second, selecting relevant model fragmentswouldbecomea more important issue in scalingup. Includ-ing additionalpropertiesof anobject, suchaselectricalor thermal properties,in the descriptionof an entityas slots doesnot causea problem in selectingmodelfragmentsbecausewhen ORACLE determinesthe rele-vanceof amodel fragmentto agiven problemit checkswhether eachvariable of the model fragmentis men-tioned asanentity propertyor variableof theproblemstatementor derivable from them. The difficulty is in

selectingamodelfragment amongmultiple modelfrag-mentswith sameor similar variablesbut with differentassumptions. In the current implementation,modelfragmentsareindexedby the mfslots of entities,withmore frequently relevant ones first, and thereforethesearch processis sensitiveto the order of the modelfragments. We maywant a more efficient method fororganizingmodel fragmentswhich will facilitate iden-tifying andretrievingrelevantmodel fragments.

The third problem in scaling up is related to thesize of a generatedmodel. As amodel gets bigger, itwould becomemore difficult to understandthe modelor to validate its solution. Having ORACLE providean explanationof its solution, or checkingthe solutionagainstthat of an approximatemodel or experimentaldatawill help understandor validate the model.

Related WorkFalkenhainerand Forbus (1991) describe a form ofcompositional modeling where a device model is au-tomatically formulated by composinga setof relevantmodelfragmentswhichareinitially obtainedby match-ing the termsof a query to adomain theoryand thenelaborated later. There are severaldifferencesbetweenour work andtheirs. In ORACLE we distinguishmodelfragmentsfrom entities; model fragmentsareused fordescribingphysicalphenomenaandentities for objects(both compositeand primitive); model fragmentsareindexedby the “mf” slot of an entity. In Falkenhainerand Forbus’ approach,model fragmentsare used fordescribing all the phenomena,objects, and devices,and are organizedinto mutually exclusivesets calledassumptionclasses. When the classcondition holds,one and only one of the assumptionsassociatedwiththeclassmust hold andthemodel fragmentcontainingthat assumptionmust be includedin a model. Once

themodelfragmentswith appropriateassumptionsareselected,the processof instantiating the model frag-mentsand assembling them is straightforward. Whilea compositeobject in ORACLE can consistof anyhet-erogeneous parts, a unique minimal covering of partstakenfrom a singlepart-of hierarchyis requiredto ex-ist in their approachto generatea simplest possiblemodel. They do not have a capability of handlingdetailed structural relationsamong parts and choos-ing appropriatereferenceframes for parts,and there-fore cannothandle complex motions such as motionof multiple objects. For behavior generationFalken-hainerand Forhususeeitherqualitativesimulation byQPE (Forbus 1990) or quantitativesimulation by nit-meric simulation, whereaswe usenumeric sinmlationor analytic method.

In (Nayak 1992), Nayak describesa method to con-struct a device model by selecting an appropriatemodel for eachcomponentof the device using struc-tural, behavioral,andexpectedbehavioralconstraints.In his system, a model is formulated by composing asetof model fragments,asin ours. However,theusesofthemodels producedby the two systemsaredifferent.While ORACLE constructsamodel to predict motionsof physicalsystems,his systembuilds a model to ex-plain causal relationsbetweenparametersof a device.Another differenceis that he usesorderof magnitudereasoningfor behaviorgenerationwhile we usenumericsimulation. His orderof magnitudereasoningmethodis restrictedto generatingthe behaviorat afixed pointin time, but we can predict the behaviorchangingwithtime as well as the behaviorat afixed point in time.

The SIGMA system developedat NASA Ames Re-searchCenter(Keller and Rimon 1992)is atool whichaids ascientist-userin building a model. After thein-teraction with the user, it producesamodel specifiedin dataflow graphand executesthemodel to computea unknownquantity. Like ORACLE, SIGMA organizesandrepresentsdomain knowledgein frame structures.However, it is a user-assistantsystem rather than anautonomousmodel-building system, and hasseveralre-strictions in constructingandexecutingamodel, whichORACLE doesnot have. For example,multiple quanti-ties cannotbe computedat the sametime becauseitcannotsolvemore than oneequationssimultaneously,and the types of equationsare restricted to algebraicequationsor first-order ordinarydifferentialequations;model fragmentscannothe put togetherin an arbitraryorderdueto thestrict hackchainingcontrol strategyofits model building process. It converts the input val-ues into a common,consistentset of scientific units,hut does not have a provision to transforma vectorquantity measuredin one referenceframe to another,which is necessaryin dealingwith moving objects.

The MSG system developedby Ling et at. (1993)generatesmathematical models for analyzing heattransfer behavior. The approachof the MSG systemto building amodel is similar to that of ORACLE in the

sensethat it is compositional.However, therearesev-eral differences. While ORACLE focuses on modelingphysicalsystemsinvolving motion, MSG models phys-icalsystemsinvolvingheatflow. Therefore,thedomainknowledgethe two systemsuseare different. A seconddifferenceis that ORACLE representstheknowledgeex-plicitly in general,declarative form, but much of theknowledgethat the MSG system usesis embeddedinthesystemaspartof itsalgorithm. A third differenceisthat, while ORACLE generatesamodel andthensolvesthe equationsof the model to predict the behaviorofagivenphysicalsystem,MSG presentlydoesnot solvetheequationsof its generatedmodel.

Another relevant line of work concernsmodel selec-tion (Addanki ci at. 1991; Weld 1992), or model sim-plification (Yip 1993;Falkenhainer1993), rather thanmodel generation.

Yet anotherrelatedworksconcernsimulation gener-ation insteadof modelgeneration.The SIMLAB systeni(Palmer andCremer1991)producesa simulatorfroma user-providedphysics model. Given a mathemati-cal model of a physical phenomenonand instructionsfor solving theresulting equations,SIMLAB transformsthe model into an executablesimulation code to ana-lyze the phenomenon.However, the userstill has theburdenof creatingthe mathematicalmodel. The pro-grambuilt by Berkooz ci at. (1992) is similar to SIM-LAB. It is basicallya compiler for translatingdiffer-ential equationsexpressedin mathematicaland pro-gramming constructs into an executablecode. TheSINAPSE system (Kant 1992) also automaticallytrans-forms a given model into a program in desired lan-guage, though again the human usermust createtheinput model.

A numberof mechanicaldevicesimulatorsarecoin-merciallyavailable,suchasADAMS (Dawson1985),andDADS (Haug 1989). Theseprograms,like most simula-tors, incorporatephysicsknowledgesuch asNewton’slawsof motiondirectly into algorithmsratherthan rep-resentingthem explicitly. The simulatorsinclude pow-erful algorithmsfor forming andsolving the equationsof motions for a wide variety of mechanisms,but lackthe flexibility that ORACLE hasto explicitly instantiategeneralmodel fragmentsin particular situations.

PreviousAl researchin spatial reasoningabout me-chanical devices (Faltings 1987; Gelsey 1989; 1994;Joskowiczand Sacks1991) ha,s devoted considerableattention to reasoningabout contacts between solidbodies,aproblemORACLE doesnot presentlyaddress.Like thecommercialsimulators,theseprogramsincor-porateknowledgeof physicalphenomenadirectly intoalgorithmsrather thanattemptingto explicitly instami-tiate generalmodel fragmentsin particular situations,as ORACLE does.

Future WorkThere are several directions in which the work de-scribed in this paper can he extended. In the cur-

rent implementation of ORACLE, there are only cer-tain classes of entities and model fragmentsavailablein theknowledgebase. Adding more model fragmentsand entities would expand the types of physical sys-temscoveredby ORACLE. It would also he a valuabletest for the extensibility of the system.

Problems which do not involve new physical phe-nomena, but require model fragments with differ-ent assumptionsor representationswill involve minorchanges. For example, the two model fragments forspring forces presently assumealinearspring with liii-ear damping. To model nonlinearsprings, we need toaddanew modelfragmentwith different equationsandassumptions. The typesof springs should be consid-ered when an if_addedprocedurechoosesbetweenthetwo model fragmentsof spring forces.

Somespatial reasoningproblemscan be solved byqualitative interpretation of the quantitative modelsproduced by ORACLE. For example,qualitative de-scription of motions (such as translational,rotational,oscillatory, or tumbling) can be easily obtained bypostprocessingthe simulation results of the models.Coverageof spaceof a moving object, any regularityof the coverageover time (suchas monotonically de-creasingcoverageof adampedspring),or possiblecon-tact/collision with other moving objects (intersectionof the coveragesduring sametime intervals) can alsohe producedby postprocessingthesimulation results.

When somethinggoeswrong during problemsolving(e.g.,amodel cannothe solved dueto fewer equationsthan unknowns), ORACLE currently prints the dead-end situation, asksfor further information, andquits.The userhas to figure out the causeof the problemandrerunthe programwith newinformation. In orderfor ORACLE to suggest possible directions to fix theproblem, it must havea capabilityof reasoningaboutequationsand unknowns.

Conclusion

The model of generalmotion in three dimension is dif-ficult to formulateby hand but important becauseofits relevanceto many practical applications, includ-ing computergraphics,robotics, anddesign.We havemade important progress in automating the model—building processfor physical systems with multiplemoving objects in arbitrary configurations by devel-oping a new methodwhich usesbasic domain knowl-edge. The method ha.s beenimplementedin a work-ing program called ORACLE and testedin the domainsof mechanicaldevicesandsailboats. Given a descrip-tion of aprobleminvolving a movingphysicalsystem,ORACLE automatically identifies relevant model frag-ments,instantiatesthemfor theparticular entitiesandphysicalphenomenain the problem,composesthe in-stantiatedfragments to form a model, and simulatesthe model to solvetheproblem. Knowledgeof physicalphenomenais representedwith model fragmentswhichcan he sharedand reusedby many models. Most of

the knowledgeis just thesamefundamentalequationsthat appearin anystandardmechanicstextbook,withtheir implied semanticsof vectorsandframesof refer-ences. Starting with the most basic, simple conceptsin the domain of mechanics,ORACLE can still gener-ate a powerful model for complex motions. This is anew methodwhich solvesseveralproblemswith exist-ing AT modeling work on motion by: (1) explicit han-dling of vector quantities andframesof reference;(2)simultaneoushandling of multiple equations(algebraicor differential, linearor nonlinear);and(3) declarative,algorithm-neutralrepresentationof physicsknowledge.

As discussedearlier, therearemanyprogramsdevel-opedfor reasoningaboutmotionof mechanicaldevices.However,the programsdo not solveproblemsin agen-eral methodfrom basic physicsprinciples, hut rely onspecificmethodsspecializedfor certainclassesof prob-lems. Letcher (1976), for example,usesa numericalprocedureadaptedfrom Newton-Raphsoniteration tofind the optimumsailboatvelocity with the maximumcomponentin the wind direction, or the sailboatve-locity whichwill balanceall theforces. ORACLE solvesthe samesailboatproblemswithout requiringspecial-purposeproblemsolving methods,as it does for othermechanicaldevices.

References

S. Addanki, R. Cremonini, and 3. 5. Penberty. Graphsof Models. Artificial Intelligence,51:145—177,1991.

G. Berkooz, P. Chew, 3. Cremer, R. Palmer, andR. Zippel. Generatingspectral method solvers forpartial differential equations. Technical Report 92-1308, Cornell University, 1992.

B. W. Char, K. 0. Geddes, G. H. Gonnet, B. L.Leong, M. B. Monagan,and S. M. Watt. MapleVLanguage ReferenceManual. Springer-Verlag,NewYork, 1991.

G. Dawson. The Dynamic Duo: Dram and Adams.Computersin MechanicalEngineering,March 1985.

B. Falkenhainerand K. D. Forbus. Compositionalmodeling: finding the right model for the job. Artifi-cial Intelligence, 51:95—143, 1991.

B. Falkenhainer. Ideal physical systems. In Proc.of the 11th National Conferenceon Artificial Intelli-gence,pages600—605, 1993.

B. Faltings. Qualitative Place VocabulariesFor Mech-anisms in Configuration Space. PhD thesis,Univer-sity of Illinois at Urbana-Champaign,July 1987.

K. D. Forbus. QualitativeProcessTheory. ArtificialIntelligence, 24:85—168, 1984.

K. D. Forbus. The Qualitative ProcessEngine. InD. S. Weld andJ. de Kleer, editors, Readingsin Qual-itative Reasoningabout Physical Systems,pages 220— 235. Morgan Kaufmann, 1990.

A. Gelsey.AutomatedPhysicalModeling. In Proc. ofthe 11th International Joint Conferenceon ArtificialIntelligence,pages 1225—1230, 1989.

A. Gelsey. Automated reasoningaboutmachines.Ar-

tificial Intelligence, 1994. to appear.G. Hartquist. Public PADL-2. IEEE ComputerGraphics and Applications, pages 30—31, October1983.

E. 3. Hang. ComputerAidedKinematicsand Dynam-ics of MechanicalSystems,VolumeI: BasicMethods.Allyn and Bacon,Boston, etc., 1989.

L. Joskowiczand E. P. Sacks. ComputationalKine-matics. Artificial Intelligence, 51:38i — 416, 1991.

E. Kant. Codesynthesisfor mathematicalmodeling.In WorkingNotesof AAAI Fall Symposiumon Intel-ligent Scientific Computation,pages54—59, 1992.

R. M. Keller andM. Rimon. A Knowledge-basedSoft-wareDevelopmentEnvironmentfor Scient ific Model-Building. In Proc. of the 7th Knowledge-BasedSoft-ware Engineering Conference,1992.

B. Kuipers. QualitativeSimulation. Artificial Intelli-gence, 29:289—388, 1986.

3. 5. Letcher, Jr. Optimum windward performanceof sailing craft. Journal of Ilydronauiics, 10:140—144,1976.

S. R. Ling, L. Steinberg, and Y. Jaluria. MSG:A Computer System for Automated Modeling ofHeat Transfer. Artificial Intelligence for Engineer-ing Design, Analysis and manufacturing, 7(4):287—300, 1993.

M. Minsky. A Frameworkfor RepresentingKnowl-edge. In The Psychologyof Computer Vision, pages211 — 277.McGraw-Hill, New York, 1975.

P. P. Nayak. AutomatedModeling of Physical Sys-tems. PhD thesis, Stanford University, 1992. STAN-CS-92-i443.

R. 5. Palmerand3. F. Cremer. SIMLAB: Automati-cally creatingphysicalsystems.TechnicalReport 91-1246,Cornell University, 1991.

P. Struss. Global Filters for Qualitative Behaviors.In Proc. of the 7th National Conferenceon ArtificialIntelligence,pages275 — 279, 1988.

D. S. Weld. ComparativeAnalysis. Artificial Intelli-gence, 36:333—374,1988.

D. S. Weld. Reasoningaboutmodel accuracy. Artifi-cial Intelligence,56:255—300,1992.

B. C. Williams. Doing Time: Putting QualitativeReasoningon Firmer Ground. In Proc. of the 5thNational Conferenceon Artificial Intelligence,pages105 — 112, 1986.

K. M. Yip. Model Simplification by Asymptotic Orderof MagnitudeReasoning.In Proc. of the11th NationalConferenceon Artificial Intelligence,pages 634—640,1993.