Research ArticleMultitasking Planning and Executing of Intelligent Vehicles forRestaurant Service by Networking

Jun Zhang123 Zhixian Chen123 Ying Hu12 Jianwei Zhang4

Zhenhua Luo125 and Xuehui Dong125

1Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Xueyuan Avenue 1068Shenzhen University Town Shenzhen 518055 China2The Chinese University of Hong Kong Shatin Hong Kong3Shenzhen College of Advanced Technology University of Chinese Academy of Sciences Xueyuan Avenue 1068Shenzhen University Town Shenzhen 518055 China4TAMS University of Hamburg Vogt-Kolln-Straszlige 30 22527 Hamburg Germany5Shenzhen Graduate School Harbin Institutes of Technology HIT Campus Shenzhen University Town Shenzhen 518055 China

Correspondence should be addressed to Ying Hu yinghusiataccn and Jianwei Zhang zhanginformatikuni-hamburgde

Received 19 December 2014 Accepted 2 March 2015

Academic Editor Ching-Hsien Hsu

Copyright copy 2015 Jun Zhang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Indoor intelligent vehicles have been applied widely in restaurant service where they are usually demanded to work for long periodfor multiple tasks and face the challenges of uncertainty efficiency and working online In this paper we propose an approachof multitasking planning and executing by networking for intelligent vehicles working for restaurant service As to increase theefficiency of accomplishing multiple tasks traditional HTNmodel is adapted to present the property of executing in parallel for theplan As working online the new inserted task is decomposed to get its individual plan which is thenmerged into the global plan byoptimization modelled as a maximal weight clique problem through evaluating the relations among actions and optimizing basedon maximal cost saving first Additionally the protected states and goal states of an action are monitored in execution monitoringwhile action executing Once exception occurs the replanning based on partially backtracking would repair it Moreover with themechanism of cooperation by networking vehicles can allocate tasks share information and cooperate for execution monitoringFinally the test and demonstration of the experiment for drink service in restaurant environment show the feasibility and theimprovement on the efficiency of multitasking

1 Introduction

Today indoor intelligent vehicles sometimes also namelyrobots have been applied more and more widely not only inlogistics warehouses [1] but also in other public areas thatis restaurant [2ndash4] official building [5] and hospitals [6] fordelivery surveillance and information inquiry Particularlyin restaurant domains shown in Figure 1 the vehicles arealways demanded to deal with multitask in a long periodwhere multiple tasks are performed concurrently and effi-ciently

Since the tasks may be too complex and the scale of theworld is too large [7 8] they may also work by distributed

networking as multiagent system with heterogeneity whereeach of them can work as an individual equipped withdifferent capability for different job with others and shareinformation with each other via network communicationAdditionally the working environments are usually uncon-structed and dynamic in these domains in this conditionplanning and executing for multitasking of the vehicles turnsto be too complex under uncertainty And there are twochallenges to be taken up

One is that the vehicles have to plan multiple tasksin the world with partially observable information In anunconstructed and dynamic world the vehicles may hardlyacquire complete information required for tasks planning

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2015 Article ID 273825 14 pageshttpdxdoiorg1011552015273825

2 International Journal of Distributed Sensor Networks

Figure 1 Intelligent vehicle serving in restaurant

with its own sensors for example to deliver a specifiedobject the location of the target object may be unknownsince it is not inside the searching range or has been movedUnfortunately some unreachable informationmay be the keyfor the planning and the planner cannot get a feasible andefficient plan for the task without it

Another challenge is that the vehicles have to plan andexecute online in dynamic world In the real application forexample restaurant the tasks for the vehicles normally can-not be preset in advance and they may come stochastically atany timeWhen a new task comes the vehiclemay beworkingfor other tasks at this time the new task has to be insertedinto the current task list and treated properly to get a moreefficient plan rather than waits until the current tasks arefinished Additionally since the states of world may changedynamically by external events exception may rise and leadto failure of the task when executing the plan To recover thefailure vehicles usually have to reconfigure its status make anew plan and maintain it online

In the literatures many studies have put their effect torise to the challenges On the one side to figure out theuncertainty previous works have shown their methods ininformation gathering planning with incomplete informa-tion [9 10] Moreover Markov decision processes (MDP)[11] and its expansion partially observable MDP (POMDP)[12 13] are applied to predict the states of the world andmakedecision during planning In [14] Kaelbling and Lozano-Perez described an integrated strategy for task and motionplanning based on planning in belief space and showedits robustness and flexibility in the mobile manipulationproblems with substantial uncertainty On the other side forplanning and executing online the methods of executionmonitoring and replanning have been developed in [15 16]As an alternative view continuous planning [17] provides apractical approach to planning online in dynamic world [18]In [19 20] an approach based on continual planning wasproposed to enable the agents planning and acting in dynamicenvironment where the concept of assertions is introducedto describe why and when an agent should switch betweenplanning and acting

Indeed probability theory minimizes the uncertainty ofthe world and the mechanism of both execution monitoringand continual planning guarantees the achievement of thetask However the probability approaches are with highcomputational cost In an alternative opinion if the futurealways cannot be predicted precisely by applying probability

inference then the simplified way is to make the uncertaintyunder the monitoring of sensors

Thanks to the development of the sensor networkingsensor communication technology has been employed forintelligent vehicles In [21ndash23] the authors provided thelow-cost solutions for indoor localization especially forcolocalization in a networking environment In these studiessensor networking plays an efficient role in extending thereachability and enhancing the robustness of the vehiclesHowever in thesemethods the sensor network has to be builtby deploying the sensors distributed in thework environmentwhich would be a tough work in an unconstructed world

Instead of building a distributed sensor network in theenvironment cooperating in multiagent system provides analternative view to plan under uncertainty Micalizio andTorasso presented a distributed approach for monitoring anddiagnosing the execution of a multiagent plan in partiallyobservable environment [24] However the approach mostlyaimed at recovering the exception of plan execution ratherthan preventing the failure of planning

In this paper aiming for multitasking problem of intelli-gent vehicles in restaurant environment it is assumed that thevehicles can share information in a multiple vehicles networkwith heterogeneity and the main idea is to handle theuncertainty and planning online through the cooperation ofnetworking Based on this idea we propose a novel approachto enhance the robustness and improve the efficiency ofmultitasking by integratingmultitasking planning executingand networking The features of the approach and ourcontributions are summarized as the following

(i) extending the model of HTN planning for multi-tasking planning under uncertainty and adapting thedecomposing algorithm for the extended model

(ii) developing the optimizer for parallelizing plans formultiple tasks in which the actions optimization ismodelled as a maximum weighted clique problem byevaluating the relations among actions and searchingthe action clique with maximal cost saving

(iii) Presenting the method for integrating executingmonitoring and replanning in dynamic environ-ment with which the execution can be monitoredwith the protected states which must be kept fromchanging during the action execution and goal statesIn addition the algorithm of replanning based onpartially backtracking is developed

The paper is structured as follows Section 2 introducesthe overall approach In Section 3 task decomposing isintroduced Followed with Section 4 the optimization forplan parallelizing is presented After that in Section 5 themethod for plan executing and monitoring is introduced Atlast but not least the experiments for restaurant service aredemonstrated and the result is discussed in Section 6 and theconclusion and future work are summarized in Section 7

2 Overall Approach

21 Control Architecture The design of the control architec-ture for intelligent vehicles networking has a very important

International Journal of Distributed Sensor Networks 3

impact on the robustness of the system and several differentarchitectures have been described in [25] However differentfrom most traditional multiagent systems in this studyeach vehicle in the network is supposed to be responsiblefor particular tasks and equipped with specified capabilitiesdifferent from others At the meantime each vehicle isexpected to have more authorization of local control so thatnew team members are facilitated to join in network Forexample in an automated restaurant there are two vehicles ormore particularly one is a waiter working for menu serviceanother one is for delivering food and clearing tables andif there are others they may provide reception service oreven others Each vehicle may take on responsibility fordifferent jobs in serving for guests and work as individualwith high autonomy as well as communicating with otherswhen necessary

To realize both robustness and flexibility of the systemas a hybrid architecture the distributed robot architecture(DIRA) [26] is adapted to our system Besides allowingautonomy in individual the DIRA also facilitates explicitcoordination among robots and employs the layered archi-tectures which are popular with single vehicle systems Basedon DIRA the control architecture is shown in Figure 2

In this architecture intelligent vehicles connected in net-work can interact with each other via their own networkinginterface with which cooperation requests are published andsubscribed and each vehicle has a complete layered archi-tecture with a loop of planning executing and monitoringon task level (see in the blue dashed line) Taking the vehiclefor delivery service as an example the vehicle works onlineand waits for recalling Once a new task is generated by taskservice and inserted to task list five processing phases areinvolved to accomplish it

In task decomposing (TD) phase the initial states will bechecked whether there is complete information for planningDue to the partially observability of the world informationmay be incomplete and undetermined In this case the vehi-cle hands out a request of state acquisition to the interfacewhere the request is published to the network Then theundetermined states would be acquired by the cooperationfrom other vehicles If the known states are sufficient forplanning TD is recalled to decompose the inserted task byusing hierarchical task net (HTN) and its individual plan isobtained

In plan parallelizing (PP) phase the individual plan of theinserted task is merged into the global plan for all processingtasks and the actions from each individual plans will beparalleled for multitasking However plan parallelizing willprocess only one layer each time and generate a combinedaction Only if the combined action is successfully executedor new task is inserted the parallelization will go ahead tonext layer This will save the computation consumption andincrease the real-time performance

In plan executing (PE) phase the combined action in theglobal plan will be performed in parallel where the actionsare transformed into executing commands and sent to thecontrollers at the same time the protected states and the goalstates are delivered to the execution monitoring

Menu service

NetworkSurveillance

Networking interfaceTask service State service

tn+1

Taskdecomposing

Replanning

Delivery service

Planparallelizing

Successful

Actionexecuting

Executionmonitoring

Controllers Sensors

Figure 2 The control architecture

In execution monitoring (EM) phase the protected statesand goal states will be monitored on semantic level withthe sensors Likewise those states are also published tothe state service so that other vehicles can cooperate formonitoring them If there is no destroyed protected state andall goal states of an action are achieved the action executionis successful and PE is recalled to execute next actionsOtherwise if either any protected state is destroyed or thegoal states cannot be achieved in regular time the executionfails and the replanning tries to repair it As to increase theefficiency for replanning a partially backtracking algorithmis applied to minimize the cost of replanning

Furthermore the controllers and the sensors togetherconstitute a route of reactive control for action execution inbehavior layer The controllers include the mobile controlrobot arm control and pan-unit control while the sensorsinclude the laser binocular vision camera and gyroscopeAnd the geometric planners including motion planning andpath planning are supposed to be implemented in behaviorlayer

22 Cooperating by Networking The cooperation amongintelligent vehicles by networking includes two capacitiestask service and state service as shown in Figure 2There taskservice allocates the tasks for the vehicles in network whilethe state service provides the sharing states and publishes therequest of state acquiring and monitoring

For long term service the vehicles in a restaurant have towork online Through task service a vehicle can publish newtasks to other vehicles as well as receiving new tasks publishedfrom others For example when the vehicle for menu servicegets an order from guest then it will publish a new task totask service and others will receive and check whether thenew task is suited for performing If the new task is suited for

4 International Journal of Distributed Sensor Networks

a vehicle it will be inserted to the task list of this vehicle andothers will ignore it

On the other hand as discussed previously the worldthe vehicles are working in is usually under uncertaintyand dynamic so that the information for TD and EMis often incomplete for individual vehicle To handle thisproblem the state service enables the cooperation among thevehicles in network to share new states and publish requestof state acquiring On one side in TD before the task isdecomposed the current states have to be checked whetherthey are sufficient to decompose the task if there is lackof information the request of acquiring this information ispublished to state service and others can help to completethis information once the information is obtained it will beshared to state service and the vehicle that needs it will beable to continue working On the other side in EM duringexecution monitoring the protected states and goal stateshave to be monitored online For an individual vehicle tomonitor the change of the environment and execute an actionat the same time sometimes it is tough and inefficient In thissituation with the state service other vehicles can be involvedto monitor these states

23 Action Knowledge of Restaurant Domain To model thedomain of restaurant service theHTNmodel [11] is expandedand the SHOP2 syntax [27] is employed to described theknowledge of the restaurant domain As one of the mostfamous planning method HTN model provides a syntaxof defining the methods by controlling the decomposing oftasks and avoiding the irrelevant searching branches and thisspeeds up significantly the task decomposition [28]Howeverfor cooperating and monitoring of the execution the actionknowledge should be able to present how to cooperate andwhat tomonitor and it can be formalized as planning domainas follows

Definition 1 (planning domain) A planning domain 119863 is atuple 119863 = (119878 119874119872) where 119878 = (119904

0 1199041 1199042 ) 119874 = (119900

0 1199001

1199002 ) and 119872 = (119898

0 1198981 1198982 ) are the limited sets

respectively of states operators and methods

Similar with the traditional definition planning domainincludes the states operators and methods And the threeelements are defined respectively as Definitions 2ndash4

Definition 2 (state) A state 119904 is an atomic sentence as the formof (predicate term etc) which describes a fact of the world

Even though the traditional HTN model is weak forhandling uncertainty by assuming the knowledge aboutthe working world is complete we keep this property toguarantee the high performance on planning efficiency andfigure out the uncertainty through an exertional modular ofstate acquisition detailed in the following section Differentfrom the belief state the determined state indicates a factFor instance (at vehicle1 waypoint1) indicates that a vehiclenamely vehicle1 is at a way point namely waypoint1 and(vehicle vehicle1) indicates that vehicle1 is a vehicle

Definition 3 (operator) An operator 119900 is a 5-tuple

119900 = (name (119900) precond (119900) add (119900) delete (119900)

protected (119900))

(1)

in which name(119900) precond(119900) add(119900) delete(119900) andprotected(119900) are respectively the set of the name precondi-tions positive effects negative effects and protected states

As the introduction of what should be monitored duringaction execution the protected(119900) describes the states thathave to be protected and cannot be changed while executingthe action In Box 1 an example of operator (move basefrom to) is given There the head contains the namemove base and parameters ldquofromrdquo and ldquotordquo which is witha question symbol to distinguish from instance The otherelements of operator are listed follow the head particularlythe protected states (available to) and (passable from to)indicate that to guarantee the execution of (move base fromto) the target location to must be available and the waybetween from and tomust be passable during the execution

Definition 4 (method) A method 119898 is a 4-tuple

119898 = (name (119898) task (119898) precond (119898) subtask (119898)) (2)

in which name(119898) is the name of the method task(119898) is thegeneralization of nonprimitive task precond(119898) is a set ofliterals for the preconditions of the method and subtasks(119898)

are a set of ordered subtasks of 119898

A method describes the prescription of decomposing anonprimitive task into subtasks And an example methodnamely (serve drink drink guest) is defined in Box 2

In Figure 3 the plan hierarchy for the task ofserve drink(drink1 guest1) is shown in which 5 methodsand 2 operators are involved There the red arrows indicatethe executing order for the subtasks in the same layer andthe grown boxes present the protected states of respectiveprimitive action

3 Task Decomposing

31 Problem Modelling Similar to traditional HTN a plan-ning problem for multitasking is defined as Definition 5

Definition 5 (planning problem) A planning problem 119875 ismodelled as a 3-tuple

119875 = (119863 1198780 119879) (3)

in which119863 is the planning domain 1198780= (1199041 1199042 119904

119899) is a set

of initial states and 119879 = (1199051 1199052 119905

119899) is a list of target tasks

A planning problem provides the action knowledge ini-tial states and the target tasks for the planner and with themthe solution will be found if there is one In traditional HTNplanning the solution is defined as plan 120587 = ⟨119886

0 1198861 1198862 ⟩

where 119886119894is a primitive action and 120587 is a sequence of total

ordered actions In the definition the actions in 120587 can only be

International Journal of Distributed Sensor Networks 5

Serve_drink(drink1 guest1)

Get_drink(drink1 servicedesk)

Put_drink(drink1 table1)

Drive_to(servicedesk)

Drive_to(table1)

Navigate_from_to(stoparea servicedesk)

Navigate_from_to(servicedesk table1)

Navigate_from_to(table2 table1)

grasp_place(drink1 servicedesk tray1)

grasp_place(drink1 tray1 table1)

move_base(stoparea servicedesk)

Protected(available servicedesk)(passable stopareaservicedesk)

move_base(servicedesk table2)

Protected(available table2)(passable servicedesktable2)

move_base(table2 table1)

Protected(available table1)(passable table2 table1)

Protected(at servicedesk)(available tray1)(on drink servicedesk)

Protected(at table1)(in drink tray1)(available table1)

Figure 3 Plan hierarchy for serving a drink

(operator (move base from to)( precondition

((location from) (location to) (passable from to)(at vehicle from) (available to))delete((at vehicle from) (available to))add((at vehicle to) (available from))protected((available to) (passable from to))

)

Box 1 An example of operator

executed one by one correspondingwith the ordersHoweverformultitasking it cannot represent the situation of executingactions in parallel To provide a plan feasible formultitaskinga concept of action clique and the new definition of plan areintroduced

Definition 6 (action clique) An action clique is a pair

AC = (119860 119862) (4)

in which 119860 = 1198861 1198862 119886

119894 is a set of actions 119886

119894partially

ordered and 119862 is a set of constraints among actions

(method (serve drink drink guest)( preconditions(drink drink) (guest guest)(at guest table1) (on drink table2) subtasks(get drink drink table2) (put drink drink table1)]

)

Box 2 An example of method

Definition 7 (plan) A plan is

120587 = ⟨AC1AC2 AC

119899⟩ (5)

which is a total ordering sequence of action clique AC119894

A plan is a hybrid construct in which actions are orderedin two layers In the plan layer the action cliques are totalordered there an action clique AC

119894is available only if the

preconditions of all actions inside are satisfied after theperformance of AC

119894minus1 However in clique layer actions are

partial ordered and can be executed in parallelIn the whole planning process the plan will appear in two

different forms to store and reuse them the task tree and plantree are defined

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

2 International Journal of Distributed Sensor Networks

Figure 1 Intelligent vehicle serving in restaurant

with its own sensors for example to deliver a specifiedobject the location of the target object may be unknownsince it is not inside the searching range or has been movedUnfortunately some unreachable informationmay be the keyfor the planning and the planner cannot get a feasible andefficient plan for the task without it

Another challenge is that the vehicles have to plan andexecute online in dynamic world In the real application forexample restaurant the tasks for the vehicles normally can-not be preset in advance and they may come stochastically atany timeWhen a new task comes the vehiclemay beworkingfor other tasks at this time the new task has to be insertedinto the current task list and treated properly to get a moreefficient plan rather than waits until the current tasks arefinished Additionally since the states of world may changedynamically by external events exception may rise and leadto failure of the task when executing the plan To recover thefailure vehicles usually have to reconfigure its status make anew plan and maintain it online

In the literatures many studies have put their effect torise to the challenges On the one side to figure out theuncertainty previous works have shown their methods ininformation gathering planning with incomplete informa-tion [9 10] Moreover Markov decision processes (MDP)[11] and its expansion partially observable MDP (POMDP)[12 13] are applied to predict the states of the world andmakedecision during planning In [14] Kaelbling and Lozano-Perez described an integrated strategy for task and motionplanning based on planning in belief space and showedits robustness and flexibility in the mobile manipulationproblems with substantial uncertainty On the other side forplanning and executing online the methods of executionmonitoring and replanning have been developed in [15 16]As an alternative view continuous planning [17] provides apractical approach to planning online in dynamic world [18]In [19 20] an approach based on continual planning wasproposed to enable the agents planning and acting in dynamicenvironment where the concept of assertions is introducedto describe why and when an agent should switch betweenplanning and acting

Indeed probability theory minimizes the uncertainty ofthe world and the mechanism of both execution monitoringand continual planning guarantees the achievement of thetask However the probability approaches are with highcomputational cost In an alternative opinion if the futurealways cannot be predicted precisely by applying probability

inference then the simplified way is to make the uncertaintyunder the monitoring of sensors

Thanks to the development of the sensor networkingsensor communication technology has been employed forintelligent vehicles In [21ndash23] the authors provided thelow-cost solutions for indoor localization especially forcolocalization in a networking environment In these studiessensor networking plays an efficient role in extending thereachability and enhancing the robustness of the vehiclesHowever in thesemethods the sensor network has to be builtby deploying the sensors distributed in thework environmentwhich would be a tough work in an unconstructed world

Instead of building a distributed sensor network in theenvironment cooperating in multiagent system provides analternative view to plan under uncertainty Micalizio andTorasso presented a distributed approach for monitoring anddiagnosing the execution of a multiagent plan in partiallyobservable environment [24] However the approach mostlyaimed at recovering the exception of plan execution ratherthan preventing the failure of planning

In this paper aiming for multitasking problem of intelli-gent vehicles in restaurant environment it is assumed that thevehicles can share information in a multiple vehicles networkwith heterogeneity and the main idea is to handle theuncertainty and planning online through the cooperation ofnetworking Based on this idea we propose a novel approachto enhance the robustness and improve the efficiency ofmultitasking by integratingmultitasking planning executingand networking The features of the approach and ourcontributions are summarized as the following

(i) extending the model of HTN planning for multi-tasking planning under uncertainty and adapting thedecomposing algorithm for the extended model

(ii) developing the optimizer for parallelizing plans formultiple tasks in which the actions optimization ismodelled as a maximum weighted clique problem byevaluating the relations among actions and searchingthe action clique with maximal cost saving

(iii) Presenting the method for integrating executingmonitoring and replanning in dynamic environ-ment with which the execution can be monitoredwith the protected states which must be kept fromchanging during the action execution and goal statesIn addition the algorithm of replanning based onpartially backtracking is developed

The paper is structured as follows Section 2 introducesthe overall approach In Section 3 task decomposing isintroduced Followed with Section 4 the optimization forplan parallelizing is presented After that in Section 5 themethod for plan executing and monitoring is introduced Atlast but not least the experiments for restaurant service aredemonstrated and the result is discussed in Section 6 and theconclusion and future work are summarized in Section 7

2 Overall Approach

21 Control Architecture The design of the control architec-ture for intelligent vehicles networking has a very important

International Journal of Distributed Sensor Networks 3

impact on the robustness of the system and several differentarchitectures have been described in [25] However differentfrom most traditional multiagent systems in this studyeach vehicle in the network is supposed to be responsiblefor particular tasks and equipped with specified capabilitiesdifferent from others At the meantime each vehicle isexpected to have more authorization of local control so thatnew team members are facilitated to join in network Forexample in an automated restaurant there are two vehicles ormore particularly one is a waiter working for menu serviceanother one is for delivering food and clearing tables andif there are others they may provide reception service oreven others Each vehicle may take on responsibility fordifferent jobs in serving for guests and work as individualwith high autonomy as well as communicating with otherswhen necessary

To realize both robustness and flexibility of the systemas a hybrid architecture the distributed robot architecture(DIRA) [26] is adapted to our system Besides allowingautonomy in individual the DIRA also facilitates explicitcoordination among robots and employs the layered archi-tectures which are popular with single vehicle systems Basedon DIRA the control architecture is shown in Figure 2

In this architecture intelligent vehicles connected in net-work can interact with each other via their own networkinginterface with which cooperation requests are published andsubscribed and each vehicle has a complete layered archi-tecture with a loop of planning executing and monitoringon task level (see in the blue dashed line) Taking the vehiclefor delivery service as an example the vehicle works onlineand waits for recalling Once a new task is generated by taskservice and inserted to task list five processing phases areinvolved to accomplish it

In task decomposing (TD) phase the initial states will bechecked whether there is complete information for planningDue to the partially observability of the world informationmay be incomplete and undetermined In this case the vehi-cle hands out a request of state acquisition to the interfacewhere the request is published to the network Then theundetermined states would be acquired by the cooperationfrom other vehicles If the known states are sufficient forplanning TD is recalled to decompose the inserted task byusing hierarchical task net (HTN) and its individual plan isobtained

In plan parallelizing (PP) phase the individual plan of theinserted task is merged into the global plan for all processingtasks and the actions from each individual plans will beparalleled for multitasking However plan parallelizing willprocess only one layer each time and generate a combinedaction Only if the combined action is successfully executedor new task is inserted the parallelization will go ahead tonext layer This will save the computation consumption andincrease the real-time performance

In plan executing (PE) phase the combined action in theglobal plan will be performed in parallel where the actionsare transformed into executing commands and sent to thecontrollers at the same time the protected states and the goalstates are delivered to the execution monitoring

Menu service

NetworkSurveillance

Networking interfaceTask service State service

tn+1

Taskdecomposing

Replanning

Delivery service

Planparallelizing

Successful

Actionexecuting

Executionmonitoring

Controllers Sensors

Figure 2 The control architecture

In execution monitoring (EM) phase the protected statesand goal states will be monitored on semantic level withthe sensors Likewise those states are also published tothe state service so that other vehicles can cooperate formonitoring them If there is no destroyed protected state andall goal states of an action are achieved the action executionis successful and PE is recalled to execute next actionsOtherwise if either any protected state is destroyed or thegoal states cannot be achieved in regular time the executionfails and the replanning tries to repair it As to increase theefficiency for replanning a partially backtracking algorithmis applied to minimize the cost of replanning

Furthermore the controllers and the sensors togetherconstitute a route of reactive control for action execution inbehavior layer The controllers include the mobile controlrobot arm control and pan-unit control while the sensorsinclude the laser binocular vision camera and gyroscopeAnd the geometric planners including motion planning andpath planning are supposed to be implemented in behaviorlayer

22 Cooperating by Networking The cooperation amongintelligent vehicles by networking includes two capacitiestask service and state service as shown in Figure 2There taskservice allocates the tasks for the vehicles in network whilethe state service provides the sharing states and publishes therequest of state acquiring and monitoring

For long term service the vehicles in a restaurant have towork online Through task service a vehicle can publish newtasks to other vehicles as well as receiving new tasks publishedfrom others For example when the vehicle for menu servicegets an order from guest then it will publish a new task totask service and others will receive and check whether thenew task is suited for performing If the new task is suited for

4 International Journal of Distributed Sensor Networks

a vehicle it will be inserted to the task list of this vehicle andothers will ignore it

On the other hand as discussed previously the worldthe vehicles are working in is usually under uncertaintyand dynamic so that the information for TD and EMis often incomplete for individual vehicle To handle thisproblem the state service enables the cooperation among thevehicles in network to share new states and publish requestof state acquiring On one side in TD before the task isdecomposed the current states have to be checked whetherthey are sufficient to decompose the task if there is lackof information the request of acquiring this information ispublished to state service and others can help to completethis information once the information is obtained it will beshared to state service and the vehicle that needs it will beable to continue working On the other side in EM duringexecution monitoring the protected states and goal stateshave to be monitored online For an individual vehicle tomonitor the change of the environment and execute an actionat the same time sometimes it is tough and inefficient In thissituation with the state service other vehicles can be involvedto monitor these states

23 Action Knowledge of Restaurant Domain To model thedomain of restaurant service theHTNmodel [11] is expandedand the SHOP2 syntax [27] is employed to described theknowledge of the restaurant domain As one of the mostfamous planning method HTN model provides a syntaxof defining the methods by controlling the decomposing oftasks and avoiding the irrelevant searching branches and thisspeeds up significantly the task decomposition [28]Howeverfor cooperating and monitoring of the execution the actionknowledge should be able to present how to cooperate andwhat tomonitor and it can be formalized as planning domainas follows

Definition 1 (planning domain) A planning domain 119863 is atuple 119863 = (119878 119874119872) where 119878 = (119904

0 1199041 1199042 ) 119874 = (119900

0 1199001

1199002 ) and 119872 = (119898

0 1198981 1198982 ) are the limited sets

respectively of states operators and methods

Similar with the traditional definition planning domainincludes the states operators and methods And the threeelements are defined respectively as Definitions 2ndash4

Definition 2 (state) A state 119904 is an atomic sentence as the formof (predicate term etc) which describes a fact of the world

Even though the traditional HTN model is weak forhandling uncertainty by assuming the knowledge aboutthe working world is complete we keep this property toguarantee the high performance on planning efficiency andfigure out the uncertainty through an exertional modular ofstate acquisition detailed in the following section Differentfrom the belief state the determined state indicates a factFor instance (at vehicle1 waypoint1) indicates that a vehiclenamely vehicle1 is at a way point namely waypoint1 and(vehicle vehicle1) indicates that vehicle1 is a vehicle

Definition 3 (operator) An operator 119900 is a 5-tuple

119900 = (name (119900) precond (119900) add (119900) delete (119900)

protected (119900))

(1)

in which name(119900) precond(119900) add(119900) delete(119900) andprotected(119900) are respectively the set of the name precondi-tions positive effects negative effects and protected states

As the introduction of what should be monitored duringaction execution the protected(119900) describes the states thathave to be protected and cannot be changed while executingthe action In Box 1 an example of operator (move basefrom to) is given There the head contains the namemove base and parameters ldquofromrdquo and ldquotordquo which is witha question symbol to distinguish from instance The otherelements of operator are listed follow the head particularlythe protected states (available to) and (passable from to)indicate that to guarantee the execution of (move base fromto) the target location to must be available and the waybetween from and tomust be passable during the execution

Definition 4 (method) A method 119898 is a 4-tuple

119898 = (name (119898) task (119898) precond (119898) subtask (119898)) (2)

in which name(119898) is the name of the method task(119898) is thegeneralization of nonprimitive task precond(119898) is a set ofliterals for the preconditions of the method and subtasks(119898)

are a set of ordered subtasks of 119898

A method describes the prescription of decomposing anonprimitive task into subtasks And an example methodnamely (serve drink drink guest) is defined in Box 2

In Figure 3 the plan hierarchy for the task ofserve drink(drink1 guest1) is shown in which 5 methodsand 2 operators are involved There the red arrows indicatethe executing order for the subtasks in the same layer andthe grown boxes present the protected states of respectiveprimitive action

3 Task Decomposing

31 Problem Modelling Similar to traditional HTN a plan-ning problem for multitasking is defined as Definition 5

Definition 5 (planning problem) A planning problem 119875 ismodelled as a 3-tuple

119875 = (119863 1198780 119879) (3)

in which119863 is the planning domain 1198780= (1199041 1199042 119904

119899) is a set

of initial states and 119879 = (1199051 1199052 119905

119899) is a list of target tasks

A planning problem provides the action knowledge ini-tial states and the target tasks for the planner and with themthe solution will be found if there is one In traditional HTNplanning the solution is defined as plan 120587 = ⟨119886

0 1198861 1198862 ⟩

where 119886119894is a primitive action and 120587 is a sequence of total

ordered actions In the definition the actions in 120587 can only be

International Journal of Distributed Sensor Networks 5

Serve_drink(drink1 guest1)

Get_drink(drink1 servicedesk)

Put_drink(drink1 table1)

Drive_to(servicedesk)

Drive_to(table1)

Navigate_from_to(stoparea servicedesk)

Navigate_from_to(servicedesk table1)

Navigate_from_to(table2 table1)

grasp_place(drink1 servicedesk tray1)

grasp_place(drink1 tray1 table1)

move_base(stoparea servicedesk)

Protected(available servicedesk)(passable stopareaservicedesk)

move_base(servicedesk table2)

Protected(available table2)(passable servicedesktable2)

move_base(table2 table1)

Protected(available table1)(passable table2 table1)

Protected(at servicedesk)(available tray1)(on drink servicedesk)

Protected(at table1)(in drink tray1)(available table1)

Figure 3 Plan hierarchy for serving a drink

(operator (move base from to)( precondition

((location from) (location to) (passable from to)(at vehicle from) (available to))delete((at vehicle from) (available to))add((at vehicle to) (available from))protected((available to) (passable from to))

)

Box 1 An example of operator

executed one by one correspondingwith the ordersHoweverformultitasking it cannot represent the situation of executingactions in parallel To provide a plan feasible formultitaskinga concept of action clique and the new definition of plan areintroduced

Definition 6 (action clique) An action clique is a pair

AC = (119860 119862) (4)

in which 119860 = 1198861 1198862 119886

119894 is a set of actions 119886

119894partially

ordered and 119862 is a set of constraints among actions

(method (serve drink drink guest)( preconditions(drink drink) (guest guest)(at guest table1) (on drink table2) subtasks(get drink drink table2) (put drink drink table1)]

)

Box 2 An example of method

Definition 7 (plan) A plan is

120587 = ⟨AC1AC2 AC

119899⟩ (5)

which is a total ordering sequence of action clique AC119894

A plan is a hybrid construct in which actions are orderedin two layers In the plan layer the action cliques are totalordered there an action clique AC

119894is available only if the

preconditions of all actions inside are satisfied after theperformance of AC

119894minus1 However in clique layer actions are

partial ordered and can be executed in parallelIn the whole planning process the plan will appear in two

different forms to store and reuse them the task tree and plantree are defined

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 3

impact on the robustness of the system and several differentarchitectures have been described in [25] However differentfrom most traditional multiagent systems in this studyeach vehicle in the network is supposed to be responsiblefor particular tasks and equipped with specified capabilitiesdifferent from others At the meantime each vehicle isexpected to have more authorization of local control so thatnew team members are facilitated to join in network Forexample in an automated restaurant there are two vehicles ormore particularly one is a waiter working for menu serviceanother one is for delivering food and clearing tables andif there are others they may provide reception service oreven others Each vehicle may take on responsibility fordifferent jobs in serving for guests and work as individualwith high autonomy as well as communicating with otherswhen necessary

To realize both robustness and flexibility of the systemas a hybrid architecture the distributed robot architecture(DIRA) [26] is adapted to our system Besides allowingautonomy in individual the DIRA also facilitates explicitcoordination among robots and employs the layered archi-tectures which are popular with single vehicle systems Basedon DIRA the control architecture is shown in Figure 2

In this architecture intelligent vehicles connected in net-work can interact with each other via their own networkinginterface with which cooperation requests are published andsubscribed and each vehicle has a complete layered archi-tecture with a loop of planning executing and monitoringon task level (see in the blue dashed line) Taking the vehiclefor delivery service as an example the vehicle works onlineand waits for recalling Once a new task is generated by taskservice and inserted to task list five processing phases areinvolved to accomplish it

In task decomposing (TD) phase the initial states will bechecked whether there is complete information for planningDue to the partially observability of the world informationmay be incomplete and undetermined In this case the vehi-cle hands out a request of state acquisition to the interfacewhere the request is published to the network Then theundetermined states would be acquired by the cooperationfrom other vehicles If the known states are sufficient forplanning TD is recalled to decompose the inserted task byusing hierarchical task net (HTN) and its individual plan isobtained

In plan parallelizing (PP) phase the individual plan of theinserted task is merged into the global plan for all processingtasks and the actions from each individual plans will beparalleled for multitasking However plan parallelizing willprocess only one layer each time and generate a combinedaction Only if the combined action is successfully executedor new task is inserted the parallelization will go ahead tonext layer This will save the computation consumption andincrease the real-time performance

In plan executing (PE) phase the combined action in theglobal plan will be performed in parallel where the actionsare transformed into executing commands and sent to thecontrollers at the same time the protected states and the goalstates are delivered to the execution monitoring

Menu service

NetworkSurveillance

Networking interfaceTask service State service

tn+1

Taskdecomposing

Replanning

Delivery service

Planparallelizing

Successful

Actionexecuting

Executionmonitoring

Controllers Sensors

Figure 2 The control architecture

In execution monitoring (EM) phase the protected statesand goal states will be monitored on semantic level withthe sensors Likewise those states are also published tothe state service so that other vehicles can cooperate formonitoring them If there is no destroyed protected state andall goal states of an action are achieved the action executionis successful and PE is recalled to execute next actionsOtherwise if either any protected state is destroyed or thegoal states cannot be achieved in regular time the executionfails and the replanning tries to repair it As to increase theefficiency for replanning a partially backtracking algorithmis applied to minimize the cost of replanning

Furthermore the controllers and the sensors togetherconstitute a route of reactive control for action execution inbehavior layer The controllers include the mobile controlrobot arm control and pan-unit control while the sensorsinclude the laser binocular vision camera and gyroscopeAnd the geometric planners including motion planning andpath planning are supposed to be implemented in behaviorlayer

22 Cooperating by Networking The cooperation amongintelligent vehicles by networking includes two capacitiestask service and state service as shown in Figure 2There taskservice allocates the tasks for the vehicles in network whilethe state service provides the sharing states and publishes therequest of state acquiring and monitoring

For long term service the vehicles in a restaurant have towork online Through task service a vehicle can publish newtasks to other vehicles as well as receiving new tasks publishedfrom others For example when the vehicle for menu servicegets an order from guest then it will publish a new task totask service and others will receive and check whether thenew task is suited for performing If the new task is suited for

4 International Journal of Distributed Sensor Networks

a vehicle it will be inserted to the task list of this vehicle andothers will ignore it

On the other hand as discussed previously the worldthe vehicles are working in is usually under uncertaintyand dynamic so that the information for TD and EMis often incomplete for individual vehicle To handle thisproblem the state service enables the cooperation among thevehicles in network to share new states and publish requestof state acquiring On one side in TD before the task isdecomposed the current states have to be checked whetherthey are sufficient to decompose the task if there is lackof information the request of acquiring this information ispublished to state service and others can help to completethis information once the information is obtained it will beshared to state service and the vehicle that needs it will beable to continue working On the other side in EM duringexecution monitoring the protected states and goal stateshave to be monitored online For an individual vehicle tomonitor the change of the environment and execute an actionat the same time sometimes it is tough and inefficient In thissituation with the state service other vehicles can be involvedto monitor these states

23 Action Knowledge of Restaurant Domain To model thedomain of restaurant service theHTNmodel [11] is expandedand the SHOP2 syntax [27] is employed to described theknowledge of the restaurant domain As one of the mostfamous planning method HTN model provides a syntaxof defining the methods by controlling the decomposing oftasks and avoiding the irrelevant searching branches and thisspeeds up significantly the task decomposition [28]Howeverfor cooperating and monitoring of the execution the actionknowledge should be able to present how to cooperate andwhat tomonitor and it can be formalized as planning domainas follows

Definition 1 (planning domain) A planning domain 119863 is atuple 119863 = (119878 119874119872) where 119878 = (119904

0 1199041 1199042 ) 119874 = (119900

0 1199001

1199002 ) and 119872 = (119898

0 1198981 1198982 ) are the limited sets

respectively of states operators and methods

Similar with the traditional definition planning domainincludes the states operators and methods And the threeelements are defined respectively as Definitions 2ndash4

Definition 2 (state) A state 119904 is an atomic sentence as the formof (predicate term etc) which describes a fact of the world

Even though the traditional HTN model is weak forhandling uncertainty by assuming the knowledge aboutthe working world is complete we keep this property toguarantee the high performance on planning efficiency andfigure out the uncertainty through an exertional modular ofstate acquisition detailed in the following section Differentfrom the belief state the determined state indicates a factFor instance (at vehicle1 waypoint1) indicates that a vehiclenamely vehicle1 is at a way point namely waypoint1 and(vehicle vehicle1) indicates that vehicle1 is a vehicle

Definition 3 (operator) An operator 119900 is a 5-tuple

119900 = (name (119900) precond (119900) add (119900) delete (119900)

protected (119900))

(1)

in which name(119900) precond(119900) add(119900) delete(119900) andprotected(119900) are respectively the set of the name precondi-tions positive effects negative effects and protected states

As the introduction of what should be monitored duringaction execution the protected(119900) describes the states thathave to be protected and cannot be changed while executingthe action In Box 1 an example of operator (move basefrom to) is given There the head contains the namemove base and parameters ldquofromrdquo and ldquotordquo which is witha question symbol to distinguish from instance The otherelements of operator are listed follow the head particularlythe protected states (available to) and (passable from to)indicate that to guarantee the execution of (move base fromto) the target location to must be available and the waybetween from and tomust be passable during the execution

Definition 4 (method) A method 119898 is a 4-tuple

119898 = (name (119898) task (119898) precond (119898) subtask (119898)) (2)

in which name(119898) is the name of the method task(119898) is thegeneralization of nonprimitive task precond(119898) is a set ofliterals for the preconditions of the method and subtasks(119898)

are a set of ordered subtasks of 119898

A method describes the prescription of decomposing anonprimitive task into subtasks And an example methodnamely (serve drink drink guest) is defined in Box 2

In Figure 3 the plan hierarchy for the task ofserve drink(drink1 guest1) is shown in which 5 methodsand 2 operators are involved There the red arrows indicatethe executing order for the subtasks in the same layer andthe grown boxes present the protected states of respectiveprimitive action

3 Task Decomposing

31 Problem Modelling Similar to traditional HTN a plan-ning problem for multitasking is defined as Definition 5

Definition 5 (planning problem) A planning problem 119875 ismodelled as a 3-tuple

119875 = (119863 1198780 119879) (3)

in which119863 is the planning domain 1198780= (1199041 1199042 119904

119899) is a set

of initial states and 119879 = (1199051 1199052 119905

119899) is a list of target tasks

A planning problem provides the action knowledge ini-tial states and the target tasks for the planner and with themthe solution will be found if there is one In traditional HTNplanning the solution is defined as plan 120587 = ⟨119886

0 1198861 1198862 ⟩

where 119886119894is a primitive action and 120587 is a sequence of total

ordered actions In the definition the actions in 120587 can only be

International Journal of Distributed Sensor Networks 5

Serve_drink(drink1 guest1)

Get_drink(drink1 servicedesk)

Put_drink(drink1 table1)

Drive_to(servicedesk)

Drive_to(table1)

Navigate_from_to(stoparea servicedesk)

Navigate_from_to(servicedesk table1)

Navigate_from_to(table2 table1)

grasp_place(drink1 servicedesk tray1)

grasp_place(drink1 tray1 table1)

move_base(stoparea servicedesk)

Protected(available servicedesk)(passable stopareaservicedesk)

move_base(servicedesk table2)

Protected(available table2)(passable servicedesktable2)

move_base(table2 table1)

Protected(available table1)(passable table2 table1)

Protected(at servicedesk)(available tray1)(on drink servicedesk)

Protected(at table1)(in drink tray1)(available table1)

Figure 3 Plan hierarchy for serving a drink

(operator (move base from to)( precondition

((location from) (location to) (passable from to)(at vehicle from) (available to))delete((at vehicle from) (available to))add((at vehicle to) (available from))protected((available to) (passable from to))

)

Box 1 An example of operator

executed one by one correspondingwith the ordersHoweverformultitasking it cannot represent the situation of executingactions in parallel To provide a plan feasible formultitaskinga concept of action clique and the new definition of plan areintroduced

Definition 6 (action clique) An action clique is a pair

AC = (119860 119862) (4)

in which 119860 = 1198861 1198862 119886

119894 is a set of actions 119886

119894partially

ordered and 119862 is a set of constraints among actions

(method (serve drink drink guest)( preconditions(drink drink) (guest guest)(at guest table1) (on drink table2) subtasks(get drink drink table2) (put drink drink table1)]

)

Box 2 An example of method

Definition 7 (plan) A plan is

120587 = ⟨AC1AC2 AC

119899⟩ (5)

which is a total ordering sequence of action clique AC119894

A plan is a hybrid construct in which actions are orderedin two layers In the plan layer the action cliques are totalordered there an action clique AC

119894is available only if the

preconditions of all actions inside are satisfied after theperformance of AC

119894minus1 However in clique layer actions are

partial ordered and can be executed in parallelIn the whole planning process the plan will appear in two

different forms to store and reuse them the task tree and plantree are defined

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

4 International Journal of Distributed Sensor Networks

a vehicle it will be inserted to the task list of this vehicle andothers will ignore it

On the other hand as discussed previously the worldthe vehicles are working in is usually under uncertaintyand dynamic so that the information for TD and EMis often incomplete for individual vehicle To handle thisproblem the state service enables the cooperation among thevehicles in network to share new states and publish requestof state acquiring On one side in TD before the task isdecomposed the current states have to be checked whetherthey are sufficient to decompose the task if there is lackof information the request of acquiring this information ispublished to state service and others can help to completethis information once the information is obtained it will beshared to state service and the vehicle that needs it will beable to continue working On the other side in EM duringexecution monitoring the protected states and goal stateshave to be monitored online For an individual vehicle tomonitor the change of the environment and execute an actionat the same time sometimes it is tough and inefficient In thissituation with the state service other vehicles can be involvedto monitor these states

23 Action Knowledge of Restaurant Domain To model thedomain of restaurant service theHTNmodel [11] is expandedand the SHOP2 syntax [27] is employed to described theknowledge of the restaurant domain As one of the mostfamous planning method HTN model provides a syntaxof defining the methods by controlling the decomposing oftasks and avoiding the irrelevant searching branches and thisspeeds up significantly the task decomposition [28]Howeverfor cooperating and monitoring of the execution the actionknowledge should be able to present how to cooperate andwhat tomonitor and it can be formalized as planning domainas follows

Definition 1 (planning domain) A planning domain 119863 is atuple 119863 = (119878 119874119872) where 119878 = (119904

0 1199041 1199042 ) 119874 = (119900

0 1199001

1199002 ) and 119872 = (119898

0 1198981 1198982 ) are the limited sets

respectively of states operators and methods

Similar with the traditional definition planning domainincludes the states operators and methods And the threeelements are defined respectively as Definitions 2ndash4

Definition 2 (state) A state 119904 is an atomic sentence as the formof (predicate term etc) which describes a fact of the world

Even though the traditional HTN model is weak forhandling uncertainty by assuming the knowledge aboutthe working world is complete we keep this property toguarantee the high performance on planning efficiency andfigure out the uncertainty through an exertional modular ofstate acquisition detailed in the following section Differentfrom the belief state the determined state indicates a factFor instance (at vehicle1 waypoint1) indicates that a vehiclenamely vehicle1 is at a way point namely waypoint1 and(vehicle vehicle1) indicates that vehicle1 is a vehicle

Definition 3 (operator) An operator 119900 is a 5-tuple

119900 = (name (119900) precond (119900) add (119900) delete (119900)

protected (119900))

(1)

in which name(119900) precond(119900) add(119900) delete(119900) andprotected(119900) are respectively the set of the name precondi-tions positive effects negative effects and protected states

As the introduction of what should be monitored duringaction execution the protected(119900) describes the states thathave to be protected and cannot be changed while executingthe action In Box 1 an example of operator (move basefrom to) is given There the head contains the namemove base and parameters ldquofromrdquo and ldquotordquo which is witha question symbol to distinguish from instance The otherelements of operator are listed follow the head particularlythe protected states (available to) and (passable from to)indicate that to guarantee the execution of (move base fromto) the target location to must be available and the waybetween from and tomust be passable during the execution

Definition 4 (method) A method 119898 is a 4-tuple

119898 = (name (119898) task (119898) precond (119898) subtask (119898)) (2)

in which name(119898) is the name of the method task(119898) is thegeneralization of nonprimitive task precond(119898) is a set ofliterals for the preconditions of the method and subtasks(119898)

are a set of ordered subtasks of 119898

A method describes the prescription of decomposing anonprimitive task into subtasks And an example methodnamely (serve drink drink guest) is defined in Box 2

In Figure 3 the plan hierarchy for the task ofserve drink(drink1 guest1) is shown in which 5 methodsand 2 operators are involved There the red arrows indicatethe executing order for the subtasks in the same layer andthe grown boxes present the protected states of respectiveprimitive action

3 Task Decomposing

31 Problem Modelling Similar to traditional HTN a plan-ning problem for multitasking is defined as Definition 5

Definition 5 (planning problem) A planning problem 119875 ismodelled as a 3-tuple

119875 = (119863 1198780 119879) (3)

in which119863 is the planning domain 1198780= (1199041 1199042 119904

119899) is a set

of initial states and 119879 = (1199051 1199052 119905

119899) is a list of target tasks

A planning problem provides the action knowledge ini-tial states and the target tasks for the planner and with themthe solution will be found if there is one In traditional HTNplanning the solution is defined as plan 120587 = ⟨119886

0 1198861 1198862 ⟩

where 119886119894is a primitive action and 120587 is a sequence of total

ordered actions In the definition the actions in 120587 can only be

International Journal of Distributed Sensor Networks 5

Serve_drink(drink1 guest1)

Get_drink(drink1 servicedesk)

Put_drink(drink1 table1)

Drive_to(servicedesk)

Drive_to(table1)

Navigate_from_to(stoparea servicedesk)

Navigate_from_to(servicedesk table1)

Navigate_from_to(table2 table1)

grasp_place(drink1 servicedesk tray1)

grasp_place(drink1 tray1 table1)

move_base(stoparea servicedesk)

Protected(available servicedesk)(passable stopareaservicedesk)

move_base(servicedesk table2)

Protected(available table2)(passable servicedesktable2)

move_base(table2 table1)

Protected(available table1)(passable table2 table1)

Protected(at servicedesk)(available tray1)(on drink servicedesk)

Protected(at table1)(in drink tray1)(available table1)

Figure 3 Plan hierarchy for serving a drink

(operator (move base from to)( precondition

((location from) (location to) (passable from to)(at vehicle from) (available to))delete((at vehicle from) (available to))add((at vehicle to) (available from))protected((available to) (passable from to))

)

Box 1 An example of operator

executed one by one correspondingwith the ordersHoweverformultitasking it cannot represent the situation of executingactions in parallel To provide a plan feasible formultitaskinga concept of action clique and the new definition of plan areintroduced

Definition 6 (action clique) An action clique is a pair

AC = (119860 119862) (4)

in which 119860 = 1198861 1198862 119886

119894 is a set of actions 119886

119894partially

ordered and 119862 is a set of constraints among actions

(method (serve drink drink guest)( preconditions(drink drink) (guest guest)(at guest table1) (on drink table2) subtasks(get drink drink table2) (put drink drink table1)]

)

Box 2 An example of method

Definition 7 (plan) A plan is

120587 = ⟨AC1AC2 AC

119899⟩ (5)

which is a total ordering sequence of action clique AC119894

A plan is a hybrid construct in which actions are orderedin two layers In the plan layer the action cliques are totalordered there an action clique AC

119894is available only if the

preconditions of all actions inside are satisfied after theperformance of AC

119894minus1 However in clique layer actions are

partial ordered and can be executed in parallelIn the whole planning process the plan will appear in two

different forms to store and reuse them the task tree and plantree are defined

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 5

Serve_drink(drink1 guest1)

Get_drink(drink1 servicedesk)

Put_drink(drink1 table1)

Drive_to(servicedesk)

Drive_to(table1)

Navigate_from_to(stoparea servicedesk)

Navigate_from_to(servicedesk table1)

Navigate_from_to(table2 table1)

grasp_place(drink1 servicedesk tray1)

grasp_place(drink1 tray1 table1)

move_base(stoparea servicedesk)

Protected(available servicedesk)(passable stopareaservicedesk)

move_base(servicedesk table2)

Protected(available table2)(passable servicedesktable2)

move_base(table2 table1)

Protected(available table1)(passable table2 table1)

Protected(at servicedesk)(available tray1)(on drink servicedesk)

Protected(at table1)(in drink tray1)(available table1)

Figure 3 Plan hierarchy for serving a drink

(operator (move base from to)( precondition

((location from) (location to) (passable from to)(at vehicle from) (available to))delete((at vehicle from) (available to))add((at vehicle to) (available from))protected((available to) (passable from to))

)

Box 1 An example of operator

executed one by one correspondingwith the ordersHoweverformultitasking it cannot represent the situation of executingactions in parallel To provide a plan feasible formultitaskinga concept of action clique and the new definition of plan areintroduced

Definition 6 (action clique) An action clique is a pair

AC = (119860 119862) (4)

in which 119860 = 1198861 1198862 119886

119894 is a set of actions 119886

119894partially

ordered and 119862 is a set of constraints among actions

(method (serve drink drink guest)( preconditions(drink drink) (guest guest)(at guest table1) (on drink table2) subtasks(get drink drink table2) (put drink drink table1)]

)

Box 2 An example of method

Definition 7 (plan) A plan is

120587 = ⟨AC1AC2 AC

119899⟩ (5)

which is a total ordering sequence of action clique AC119894

A plan is a hybrid construct in which actions are orderedin two layers In the plan layer the action cliques are totalordered there an action clique AC

119894is available only if the

preconditions of all actions inside are satisfied after theperformance of AC

119894minus1 However in clique layer actions are

partial ordered and can be executed in parallelIn the whole planning process the plan will appear in two

different forms to store and reuse them the task tree and plantree are defined

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

6 International Journal of Distributed Sensor Networks

Definition 8 (task tree) A task tree TaskTree(119879119894) is a hier-

archical tree presenting the decomposing architecture of task119879119894

A task tree shows how a task is decomposed into subtasksrecursively until the primitive tasks are achieved There eachnode stands for task which may be a primitive task or anonprimitive task and the children nodes are its subtasks

Definition 9 (plan tree) A plan tree PlanTree(119875) is a reach-ability tree representing the action paths to achieve the goalsof the tasks

A plan tree shows how to accomplish the tasks byexecuting the plan In a plan tree each node is an action oraction clique Normally the initial plan tree containsmultiplebranches and each branch from the root to the leaf is anaction path to achieve the goals of respective tasks Since theplans will be paralleled layer by layer in plans parallelizingthe actions distributed in each branch will be insisted intoaction cliques by optimizing Finally if the tree shrinks into asingle branch it is the final plan tree for all tasks

32 Algorithm for Task Decomposing The goal of TD is togenerate the individual task tree and the individual plan treefor the tasks As initialization a global task tree rooted asldquoTaskTreerdquo is created to store the task tree for each inde-pendent task which is attached as the subtree of ldquoTaskTreerdquoSimilarly a global plan tree rooted as ldquoPlanTreerdquo is createdand every plan tree of each independent task will be attachedto the current action clique as a new child tree

To plan online the planner listens to task service andwaits for new task insertion Once a new task 119905

119899is inserted

it will be the initial member of tasklist and decomposedindependently to get the task tree and plan tree by thefunction of decompose task() shown in Algorithm 1

The decompose task() performs a recursive process todecompose the inserted task At the beginning the endingcondition is checked and once the tasklist is empty it returnsthe tasktree and the plantree Otherwise the activetask whichis always the first element of tasklist is treated in two ways(1) If activetask is an instance of operators a circle blockrepeats the acquiring of sharing state and check preconditionThere check precond() checks whether all preconditions ofactivetask are ready in renewed 119878

0 If not it publishes the

preconditions which are lacking to state service or else thepreconditions are sufficient it breaks the circle And then1198780is renewed as 119878

1015840 by applying activetask And if 1198781015840 is not

false activetask is added to plantree and decompose task()is recalled to decompose next task in tasklist under 119878

1015840(2) If activetask is an instance of methods the functioncheck precond() is implemented as it does in the first con-dition While the preconditions of activetask are met thesubtasks of activetaskwill be added to the node of activetask asits children and inserted to the front of tasklist after deletingactivetask from tasklist after that the decompose task() isrecalled with the new tasklist And this process will beimplemented until the ending condition is satisfied

4 Plan Parallelizing

41 Process of Plan Parallelizing The objective of PP is toparallelize all the individual plans for target tasks so thatthe tasks can be executed more efficiently in parallel and theprocess is shown in Figure 4 and the subprocesses are listedas follows

Description of the Subprocesses

1198791199031 new task is inserted and decomposed

1198791199032 the active AC is executed successfully

1198791199033 the execution for the active AC fails

1198791199001 append the new plan tree to global plan tree

1198791199002 activate the next layer of the global plan tree

1198791199003 replace the failing branches with repaired plan tree

1198791199004 insert the new AC and delete the actions of AC fromrespective branches

There PP will be activated by three triggers and itsrespective operations of plan tree after activating the newlayer the relations among the actions in active layer as wellas the relations between the actions and the executed ACare analyzed in relation analyzing With the relations theoptimization is performed to select actions into new AC andthen it will be inserted into the global plan tree by 119879119900

4

In the following we introduce the details for relationevaluating and actions optimizing

42 Action Relations As the definition of action clique theactions in an action clique must be compatible for executionin parallel hence the relations among actions must beevaluated In literatures the analyzing technique of actionrelations has been introduced in Graphplan [29] and LCGP[30] inspired by these works three relations are defined asfollows

Action Relations

Relation 1 (independence) Consider

del (119886119894) cap [pre (119886

119895) cup add (119886

119895)]

cup del (119886119895) cap [pre (119886

119894) cup add (119886

119894)] = 0

(6)

Relation 2 (allowance) Consider

[del (119886119894) cap pre (119886

119895)] cup [add (119886

119894) cap del (119886

119895)] = 0

[pre (119886119894) cap del (119886

119895)] cup [del (119886

119894) cap add (119886

119895)] = 0

(7)

Relation 3 (retraction) Consider

del (119886119894) = add (119886

119895) or add (119886

119894) = del (119886

119895) (8)

The relations are defined by the interplay between actionsas the equations in Action Relations As the definition

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 7

decompose task(1198780 tasklist tasktree plantree)

if tasklist is emptyreturn(tasktree plantree)

activetask larr tasklist(0)if activetask is an instance of operator

while 1get sharing states and renew 119878

0

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else break1198781015840larr 120574(119878

0 activetask)

if 1198781015840is not Falseplantreeadd child(activetask)tasklistremove(activetask)tasktree plantree larr decompose task(1198781015840 tasklist tasktree plantree)

if activetask is an instance of methodwhile 1

get sharing states and renew 1198780

check precond(1198780 activetaskprecond)

if there are preconditions not readypublish the preconditions which are not ready

else breaksubtasks larr method(119878

0 activetask)subtask

tasklistremove(activetask)tasklist larr insert subtasks in the front of tasklisttasktreeadd children(subtasks)tasktree plantree larr decompose task(119878

0 tasklist tasktree plantree)

Algorithm 1 Decomposing task based on hierarchical decomposing

Tr1

Tr2

Tr3

To1

To2

To3

Relationanalyzing

Actionsoptimizing To4

TriggerTree operationToi

Tri

Figure 4 Process of plan parallelizing

shown the relation between actions is dependent on theirprecondition resource and output effect There 119886

119894and 119886119895are

independentwith each otherwhile the performance of 119886119894does

not delete the preconditions and positive effects of 119886119895 and at

the meantime the performance of 119886119895does not delete those of

119886119894either And if the performance of 119886

119894will neither delete the

preconditions of 119886119895nor produce the negative effects of 119886

119895and

at the meantime the performance of 119886119895will either delete the

preconditions of 119886119894or produce the negative effects of 119886

119894 we

say 119886119894allows 119886

119895 Furthermore a retraction relation between

119886119894and 119886119895is met while the positive effects of 119886

119894is equal to the

negative effects of 119886119895 or vice versa

The criteria to justify whether certain actions or tasks areparallelizable can be summarized as two relations indepen-dence and allowance However the independence is more

restricted than the allowance Different from the definitionin LCGP our definition of allowance is more stronglyconstrained and divided into two cases 119886

119894allows 119886

119895and 119886

119895

allows 119886119894 But in LCGP the allowance means only 119886

119894and 119886

119895

are with allowance relation and this definition includes theindependence The advance of our criteria is that the actionswith allowance relation are ordered and are more appropriatefor actions execution

There the independence and allowance relations arechecked among the actions in the same layer of the plan treewhile the retraction relation is checked between an action incurrent layer and the actions in the latest action clique Andthe respective rules for assembling them in an action cliqueare listed as follows

(i) If actions are independent of each other they canbe put together in an action clique and executedsimultaneously

(ii) If two actions 119886119894and 119886

119895are with allowance relation

they can be put in the same action clique and in case ofindependence relation but with an allowance orderwhich indicates that 119886

119894can be executed before 119886

119895if 119886119894

allows 119886119895

(iii) If there is any action in active layer with a retroactionrelation it should be excluded from the candidates forthe optimization

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

8 International Journal of Distributed Sensor Networks

Besides actions are not with any one of above relationsthey are mutually exclusive and cannot appear in the sameaction clique

43 Optimizing as Maximum Weight Clique Problem Afterthe preselection by analysing the relations among actionsthe actions with independence and allowance relations willbe optimized And we model the optimization as a MWCP(maximum weight clique problem) [31 32] as

119866 = (119860119882 119864) (9)

where

(i) 119860 = 1198861 1198862 119886

119899 is a set of vertices representing

action candidates for action clique(ii) 119882 = 119908

119886119894| 119886119894

isin 119860 is a set of combined weights119908119886119894valued with the time cost for the execution of the

according action and the number of times that thisaction has shown in current layer

(iii) 119864 = (119886119894 119886119895) | 119886119894 119886119895

isin 119860 119894 = 119895 is a set of edgesconnecting the relative actions 119886

119894and 119886119895

In a MWCP the vertices are the actions with indepen-dence relation or allowance relation in the active layer of plantree An edge indicates the relation between two connectedactions where an undirected edge presents an independencerelation and a directed edge presents an allowance relationAn example graph of MWCP is shown in Figure 5 wherethe vertices are named with actions and their weights arenoted on the nodes In the example one independence andsix allowances are included

Since the goal of optimization is to find an action cliqueto reduce the time consumption for multitasking and theglobal optimization is always complicated and needs a hugeconsumption of computation the second best thing is to findthe local optimum As this the rule of the optimization isset asmaximal-cost-saving-first compared with the total timecost to execute all actions involved in the action clique one byone and the formula is as (10) where 119899

119886119894is the amount of that

119886119894appearing in the active layer of plan tree and 119882(119888

119909) is the

cost of action clique 119888119909

119891max (119908119886119894 119888119909) = max(sum

119888119909

(119908119886119894

lowast 119899119886119894

| 119886119894isin 119888119909) minus 119882(119888

119909))

(10)

However since an action clique may contain allowancerelations the actions in clique have to be sorted by allowanceorder And 119882(119888

119909) is equal to the maximum weight for all

action paths in 119888119909 shown in

119882(119888119909) = max(sum

119886119901119896

(119908119886119894

| 119886119894isin 119886119901119896) | 119886119901

119896isin 119888119909) (11)

In particular if there is neither independence norallowance relation among actions in the active layer theoptimization is simplified as

119891max (119908119886119894 119886119894) = max (119908

119886119894lowast (119899119886119894

minus 1) | 119886119894isin 119860) (12)

15 1

11 3

4 1

7 2

6 1

g122

m21

m25

m24

g232

m21 (move_base table2 table1)m25 (move_base table2 table5)m24 (move_base table2 table4)g122 (grasp_place drink1 tray2 table2)g232 (grasp_place drink2 tray3 table2)

Figure 5 An example graph of MWCP

As the rule of optimization the selected actions cliqueof the example in Figure 5 will be AC = (119892122 119892232

11989821) independence (119892122 119892232) allowance (119892122 11989821)

allowance (11989223211989821)

5 Plan Executing and Execution Monitoring

51 Plan Executing After a planned action clique is generatedby plan parallelizing it is executed in PE As an action cliqueconsisted of the actions executable in parallel PE should playtwo roles during the execution (1) to manage and allocate theactions in action clique and (2) to transform the actions intoexecuting command

Algorithm2 shows themain algorithm for executing planThere PE works online and waits for the trigger until a newAC is generated in PP through wait for ac() Once an AC isgot the actions in AC will be executed iteratively until it isempty Since the actions in an AC is executable in parallel butwith allowance constrain in which an action can only be exe-cuted immediately when there is no prefix action of it all thefree actions without prefix action for them are transformedto controllers respectively by tramsform to control (actioni)at once At the meantime their protected states and effectedstates including the positive effect and negative effect arepublished to EM by publish to monitor (actioni) and then thefree actions will be removed from current AC as well

In tramsform to control (actioni) actioni is transformedto control function by recalling the respective controller Forexample an action move base (Table1 Table2) is to movethe vehicle from Table1 to Table2 where Table1 and Table2will be mapping to the corresponding positions in the mapand assigned to the navigation function as its parameters inwhich path planning is called to find a path way and drive thevehicle from the position of Table1 to the position of Table2Likewise the protected states of actioni are published to EMwhere they will be monitored

After transforming the free actions to control PE willwait for the response of EM Once an action is successfullyexecuted it is returned as consummator and removed fromthe prefix list of other actions to free them for executing

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 9

execute plan()AC larr wait for ac()while AC is not empty

for each 119886119888119905119894119900119899119894in AC

if prefix of (119886119888119905119894119900119899119894) is empty

tramsform to control(119886119888119905119894119900119899119894)

publish to monitor(119886119888119905119894119900119899119894)

ACremove(119886119888119905119894119900119899119894)

consummator larr wait for monitor()for each 119886119888119905119894119900119899

119894in AC

if consummator in prefix of (119886119888119905119894119900119899119894)

remove consummator from prefix of (119886119888119905119894119900119899119894)

activate PP to parallel next layer

Algorithm 2 Plan executing

immediately When all actions in AC are executed PE willsend a trigger to activate PP to parallel next layer of the globalplan tree

52 Execution Monitoring In EM the plan execution ismonitored by two ways (1) the protected states have to bekept and never be tampered until the execution is finished (2)the goal states are expected to be realized where the positiveeffect states are shown in current states and the negativeeffected states are deleted from current states

Each state of protected states and goal states is monitoredby a process of the involved sensor in which the sensoracquires the information and confirms the states EM man-ages the processes by calling or killing them and compares theconfirmed states with the monitoring states of each actionThere are three cases

(i) If any protected state of an action has been confirmedto be tampered during executing the action the exe-cution fails and the relative plan must be replanned

(ii) If the goal states are all confirmed to be achievedand the protected states are all kept well during theexecution of an action then the action is executedsuccessfully At this time the action is sent to PE asa consummator and this would activate the processof PE to go ahead to execute the next batch of freeactions

(iii) If the goal states cannot be confirmed in expectedtime then the execution of this action would beconsidered as failure and it will be replanned also

While the execution fails the replanning is activated torecover it As to improve the performance an algorithmbased on partially backtracking is designed to restrict thereplanning depth As the basic principle the replanningalways and iteratively tries to replan the nearest upper task ofthe failure action or task until the new solution is obtained

As shown in Algorithm 3 the corresponding action intask tree can be achieved by Action2Task() in which a dic-tionary maps the actions between plan tree and task treeTheparent task is then replanned If the replanning is successful

K21

2

3

Kitchen

Entrance

Figure 6 Map for the experiment

the repairedplan is got and replaces the failure one waiting fornext process of plan parallelizing by renew plantree() and therepairedtask repairs the failure task tree by renew tasktree()If not the upper parent task is got and replanned recursivelyuntil solution is found or the root is reached with returningfalse

6 Experiment and Result

61 Experiment Environment and Setting The experiment istested in a dynamic environment setting with one entranceand three tables distributed in a laboratory shown in Figure 6which simulates the scenario of drink service in auto-mated restaurant Three facilities two intelligent vehiclesand settled Kinect are involved in the networking of theexperiment each of them has the individual responsibilitybut cooperating with each other for acquiring informationand monitoring execution

For details each facility has the following capability (1)K1 Kinect for surveillance settled at the door way of theentrance can detect and recognize the guest who entersthe service area or goes out from the entrance (2) K2 apioneer intelligent vehicle equipped with Kinect for menuservice can detect and recognize which table the identifiedguest is sitting by and get his or her order for drink (3)K3 another intelligent vehicle equipped a 7 DOF arm and agripper for deliver service can serve the ordered drink to thecorresponding table and recycle the empty drink boxes afterguests leave

To cooperating in network due to the convenience forcommunicating among distributed machines in local areanetwork [33] ROS (robot operating system) is employed forthe development of control systems In addition to enhancethe robustness of the networking we construct a hybridnetwork environment by using wireless data transmissionmodules as the supplement In the experiment communi-cation among all facilities is through one shared ldquoroscorerdquoin ROS shown as Figure 7 while K2 and K3 communicatethrough wireless data transmission

In Figure 8 we can see the flowchart of multiple sensorsOn the left top K1rsquos and K2rsquos computers are connected in thesame local area network among them K2rsquos laptop is runningas a ROS master and other computers are running connect

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

10 International Journal of Distributed Sensor Networks

repaire plan(119878 fAction preAction)fTask larr Action2Task(fAction)solution larr Falsewhile solution = False

fParent larr fTaskupif fParent is the root of TaskTree

return Falserepairedtask repairedplan larr decompose task(119878 fParent repairedtask repairedplan)if repairedtask repairedplan is not False

solution larr Truerenew tasktree(repairedtask)renew plantree(repairedplan)

Algorithm 3 Repair the failure plan by replanning based on partial backtracking

camera_base_link

camera_base_link1

camera_base_link2

camerabase_link3

_tf_laser

openni_tracker

tf

makers_from_tf

amcl_pose

chatter

listener

laser_static_tf_publisher

amcl

odom_publisher

scan

map_server

odom

cmd_vel

scan

map

gyro

move_base

gyro

move_basegoal

move_baseresult

tf_static goal_move_baseexecution_feedback

Camera

execute_plan

camerargbimage_colorcameracamera_nodelet_manager

camerargbimage_raw

surf Object_match_result

face_recognition

surf

Face_recognitionaction_topics face_recognition

face_recognition_client

Figure 7 ROS computation graph for K2

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 11

K2

positionPoint (1sim3)

AmclposeMap

K2rsquos vehicle computer

Localnetwork

Run as ROS master

Kinect

Kinect

K2rsquos laptop

K1rsquos computer

Message

Laser

Gyroscope

Encoderfa

ce_n

ame

xx g

uest talker Wireless data

transmission module 1cc guest point (1sim3)

lzh1guest point (1sim3)Occur (1sim2) point (1sim3)

Wireless datatransmission module 2

K3

Planner K3 Action execution

K1 Many sensors

Figure 8 The flowchart of multiple sensors

with it so that the K1rsquos and K2rsquos computers can share ROStopic with each other They are assembled as a group K1sends topic of ldquoface namerdquo to the group by using its Kinectsensor to detect guests and record their ID And then K2rsquosKinect sensor can detect guests and recognizes their namesat the same time K2rsquos vehicle sensors send back a topic ofldquopositionrdquo to the group Using all these topics K2rsquos laptop canknow the guests and their sitting tables then it sends a topicmessage using K2rsquos wireless data transmission module to K3K3rsquos planner receives the messages and works with them

In experiment K1 monitors the entrance to detect andacquire training images for every new guest Human detec-tion is realized by the skeleton tracking which is the bestknown application of Kinect and the joint positions of aperson standing in front of the camera can be detected andtracked through using the ROS package of openni tracker[34] If a new guest stares at the camera for couple of secondsthe desired number of training images is obtained from thevideo stream and the ID for the new guest is then assigned[35] Finally the acquired images are stored and added to thelist of training imaged and shared with others The process isshown in the images of Figure 9

As a pioneer vehicle K2 is capable of SLAM (simultane-ous localization and mapping) communicating with K1 byldquoroscorerdquo and exchanging information with K3 by wirelessdata transmissions Equipped with the navigation stack ofROS it is able to plan path create a map from laser scan datalocate itself in a map and so forth In Figure 9 K2 starts toroutine patrol to detect and recognize every guest (as shownin Figure 10) confirm their table number through its currentlocation and pose acquire the orders of drink and publishthem to K3 If the K2 detects the drink only but no person isnear the table it will infer whether the guest has gone by theK1rsquos information

Figure 9 Guest lzh1 is detected and recognized by K1

Moreover as to make the experiment easy going but notthe key point of our experiment we design SURF detectorto detect the images pasted on the drink box to classify thedrinks shown in Figure 11 The red circles on the images aregenerated by SURF detector

And as the delivery vehicle K3 is able to execute thedrink identification autonomous moving grasping andplace actions and the tray on K3 is with themaximal capacityof carrying three drinks With multitasking planning K3can treat the multiple tasks as a whole to save the costfor task execution When a new task is requested it willbe inserted to the current task list and the individualplan for it will be parallelized into the global plan Asan example if two persons are coming or leaving at thesame time K3 could serve them in parallel and carry twodrinks at once and deliver to them respectively shown inFigure 12

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

12 International Journal of Distributed Sensor Networks

Figure 10 K2 has detected and recognized Guest lzh1 and cc andprovided their location

Figure 11 Different drink box at different position is detectedprecisely

62 Testing and Result To verify the feasibility and theimprovement on the efficiency of the approach proposed thecomparable testing and results are presented in two cases

Case 1 (serving two guests in parallel by networking) Inthis case K3 working with networking receives a new taskthat Guest lzh1 orders Drink2 while grasping and placing theDrink1 on the tray for serving it to Table1 for Guest cc At thistime the new task is inserted and paralleled into the globalplanAs the result of parallelization two tasks can be executedin parallel and K3 would bring Drink1 together with Drink2and serve them to Table1 and Table2 respectively The planand the executing time are shown in Figure 13 and thewaitingtime of drink service for Guest lzh1 is 217 s

Case 2 (serving two guests by single vehicle) In Case 2 K3 isworking as a single vehicle and it may not be able to detectthe coming of new guest in first time and new service taskhas to wait until the current task is finished This takes muchmore time for the new guest to wait With the same settingand variables of Case 1 the plan and the executing time areshown in Figure 14 and the waiting time for Guest lzh1 is389 s

As the comparison between Cases 1 and 2 the testingwhich is based on the proposed approach and networkingsaves 172 seconds for Guest lzh1 to get the drink service

Figure 12 K3 is serving two drinks for two guests in parallel

grasp_place(drink1 table1)

move_base(table1 table2)

grasp_place(drink2 table2)

Total time 217 s

Case 1detect_recognize

(lzh1)Wait for lzh1rsquos order

grasp_place(drink2 tray)

move_base(kitchen table1)

25 s

35 s

32 s

27 s

33 s

30 s

35 s

Figure 13 Result of Case 1

move_base(kitchen table1)

grasp_place(drink1 table1)

move_base(table1 table2)

detect_recognize( lzh1)

Wait for lzh1rsquos order

move_base(table2 kitchen)

grasp_place(drink2 tray)

move_base(kitchen table2)

grasp_place(drink2 table2)

26 s

33 s

33 s

52 s35 s

59 s

33 s

54 s

34 s

Total time 389 s

Case 2

Figure 14 Result of Case 2

The different bounding colors red and green highlightthe differences actions between two cases There the redcolor presents the actions which are involved in both casesbut with different time costs In two cases the time costfor detect recognize (lzh1) colored with red is differentBecause in Case 1 K2 can use the shared data of lzh1rsquos facefrom K1 during executing detect recognize (lzh1) while inCase 2 K3 has to obtain this data by itself At this pointit shows the advance of our method that the networkingimproves the efficiency on acquiring information And thegreen presents the actions that are involved in either oftwo cases The actions of move base (Table2 Kitchen) andmove base (Kitchen Table2) in Case 2 are not appear in Case1 because K3 has paralleled the two tasks as a global plan And

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of Distributed Sensor Networks 13

it shows the improvement of our approach on the efficiencyof multitasking

7 Conclusion and Future Work

In this paper we proposed an approach for multitasking inrestaurant service of intelligent vehicles by networking Wepresented the adapted method of hierarchical decomposingbased on traditional HTN planning and the optimizationmodelled as maximal weight clique problem for optimizingthe efficiency of multitasking And by integrating the planexecuting and execution monitoring the vehicles can workonline in dynamic environment Additionally as the vehiclesare working in network the mechanism for cooperating bysharing information and allocating task help the vehicles tohandle partially observability and uncertainty in dynamicworld As shown in the experiment of restaurant servicethree facilities cooperated with each other in a hybridnetwork and the comparing results between working bynetworking and working as individual support that theproposed approach has made obvious progress on improvingthe efficiency and robustness of the vehicles

Even though the feasibility of the proposed approach isverified the uncertainty is not really figured out yet Thefuture works will focus on the deeper integration with thecapability for predicting the change of the environment in realworld with much more uncertainty to find the way to endowthe indoor intelligent vehicles with more robustness so thatit can work more unaffectedly and friendly in a human-robotcoexisting world

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Foun-dation of China (no 61210013 and no 61473278) and KeyResearch Program of the Chinese Academy of Sciences(KJZD-EW-TZ-L03)

References

[1] P R Wurman R DrsquoAndrea and M Mountz ldquoCoordinatinghundreds of cooperative autonomous vehicles in warehousesrdquoAI Magazine vol 29 no 1 pp 9ndash19 2008

[2] L Acosta E J Gonzalez J N Rodrıguez et al ldquoDesign andimplementation of a service robot for a restaurantrdquo Interna-tional Journal of Robotics andAutomation vol 21 no 4 pp 273ndash281 2006

[3] Q X Yu C Yuan Z Fu andY Z Zhao ldquoAn autonomous restau-rant service robot with high positioning accuracyrdquo IndustrialRobot vol 39 no 3 pp 271ndash281 2012

[4] Q-X Yu C Yuan Z Fu and Y-Z Zhao ldquoResearch of thelocalization of restaurant service robotrdquo International Journalof Advanced Robotic Systems vol 7 no 3 pp 227ndash238 2010

[5] J Lopez D Perez E Paz and A Santana ldquoWatchBot A build-ingmaintenance and surveillance system based on autonomousrobotsrdquo Robotics and Autonomous Systems vol 61 no 12 pp1559ndash1571 2013

[6] J B Silva C Santos and J Sequeira ldquoDeveloping a timed nav-igation architecture for hospital delivery robotsrdquo in Proceedingsof the IEEE 3rd Portuguese Meeting in Bioengineering (ENBENGrsquo13) pp 1ndash4 February 2013

[7] T Arai E Pagello and L E Parker ldquoGuest editorialmdashadvancesin multirobot systemsrdquo IEEE Transactions on Robotics andAutomation vol 18 no 5 pp 655ndash661 2002

[8] Y U Cao A S Fukunaga and A B Kahng ldquoCoopera-tive mobile robotics antecedents and directionsrdquo AutonomousRobots vol 4 no 1 pp 7ndash27 1997

[9] A Torreno E Onaindia and O Sapena ldquoA flexible couplingapproach to multi-agent planning under incomplete informa-tionrdquoKnowledge and Information Systems vol 38 no 1 pp 141ndash178 2014

[10] P H Tu T C Son and C Baral ldquoReasoning and planning withsensing actions incomplete information and static causal lawsusing answer set programmingrdquo Theory and Practice of LogicProgramming vol 7 no 4 pp 377ndash450 2007

[11] M Ghallab D Nau and P Traverso Automated PlanningTheory and Practice Elsevier 2004

[12] J Ma and B-Q Yin ldquoSimulation optimization of actions ofrobot based on POMDP modelrdquo Journal of System Simulationvol 20 no 21 pp 5903ndash5906 2008

[13] S Candido and S Hutchinson ldquoMinimum uncertainty robotpath planning using a POMDP approachrdquo in Proceedings ofthe IEEERSJ International Conference on Intelligent Robots andSystems (IROS rsquo10) pp 1408ndash1413 Taipei Taiwan October 2010

[14] L P Kaelbling and T Lozano-Perez ldquoIntegrated task andmotion planning in belief spacerdquo International Journal ofRobotics Research vol 32 no 9-10 pp 1194ndash1227 2013

[15] M Fichtner A Grossmann and M Thielscher ldquoIntelligentexecution monitoring in dynamic environmentsrdquo FundamentaInformaticae vol 57 no 2ndash4 pp 371ndash392 2003

[16] F R Noreils and R G Chatila ldquoPlan execution monitoring andcontrol architecture for mobile robotsrdquo IEEE Transactions onRobotics and Automation vol 11 no 2 pp 255ndash266 1995

[17] M E Desfardins E H Durfee C L Ortiz andM JWolvertonldquoA survey of research in distributed continual planningrdquo AIMagazine vol 20 no 4 pp 13ndash22 1999

[18] J S Russell and P Norvig Artificial Intelligence A ModernApproach Prentice Hall 2nd edition 2003

[19] M Brenner and B Nebel ldquoContinual planning and acting indynamic multiagent environmentsrdquo Autonomous Agents andMulti-Agent Systems vol 19 no 3 pp 297ndash331 2009

[20] G-JMKruijffM Brenner andNHawes ldquoContinual planningfor cross-modal situated clarification in human-robot interac-tionrdquo in Proceedings of the 17th IEEE International Symposiumon Robot and Human Interactive Communication (RO-MANrsquo08) pp 592ndash597 Institute of Electrical and Electronics Engi-neers Computer Society Munich Germany 2008

[21] LMing Q Kejie C Fengyu et al ldquoTowards indoor localizationusing Visible Light Communication for consumer electronicdevicesrdquo inProceedings of the IEEERSJ International Conferenceon Intelligent Robots and Systems (IROS rsquo14) pp 143ndash148Chicago Ill USA September 2014

[22] Y Sun M Liu and M Q-H Meng ldquoWiFi signal strength-based robot indoor localizationrdquo in Proceedings of the IEEE

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

14 International Journal of Distributed Sensor Networks

International Conference on Information and Automation (ICIA14) pp 250ndash256 Hailar China July 2014

[23] L Wang M Liu and M Q-H Meng ldquoTowards cloud roboticsystem a case study of online co-localization for fair resourcecompetencerdquo in Proceedings of the IEEE International Confer-ence on Robotics and Biomimetics (ROBIO rsquo12) pp 2132ndash2137December 2012

[24] R Micalizio and P Torasso ldquoMonitoring the execution of amulti-agent plan dealing with partial 30 observabilityrdquo in Pro-ceedings of the 18th EuropeanConference onArtificial Intelligence(ECAI rsquo08) M Ghallab C D Spyropoulos N Fakotakis andN M Avouris Eds pp 408ndash412 IOS Press Amsterdam TheNetherlands 2008

[25] D Kortenkamp and R Simmons ldquoRobotic systems architec-tures and programmingrdquo in Springer Handbook of Robotics BSiciliano and O Khatib Eds pp 187ndash206 Springer BerlinGermany 2008

[26] R Simmons S Singh D Hershberger J Ramos and T SmithldquoFirst results in the coordination of heterogeneous robots forlarge-scale assemblyrdquo in Experimental Robotics VII D Rusand S Singh Eds vol 271 of Lecture Notes in Control andInformation Sciences pp 323ndash332 Springer Berlin Germany2001

[27] R PGoldmanDNau andUKuterDocumentation for SHOP2University of Maryland 2011

[28] Q Yang ldquoFormalizing planning knowledge for hierarchicalplanningrdquo Computational Intelligence vol 6 no 1 pp 12ndash241990

[29] A L Blum and M L Furst ldquoFast planning through planninggraph analysisrdquo Artificial Intelligence vol 90 no 1-2 pp 281ndash300 1997

[30] M Cayrol P Regnier and V Vidal ldquoLeast commitment inGraphplanrdquo Artificial Intelligence vol 130 no 1 pp 85ndash1182001

[31] I M Bomze M Budinich P M Pardalos and M PelilloldquoThemaximum clique problemrdquo inHandbook of CombinatorialOptimization pp 1ndash74 Springer 1999

[32] P R Ostergard ldquoA new algorithm for the maximum-weightclique problemrdquo Nordic Journal of Computing vol 8 no 4 pp424ndash436 2001

[33] Open Source Robotics FoundationRunning Ros acrossMultipleMachines Open Source Robotics Foundation 2009 httpwikirosorgROSTutorialsMultipleMachines

[34] P Goebel Ros by Example Hydro Lulu 2013 httpwwwlulucomshopr-patrick-goebelros-by-example-hydro-volume-1ebookproduct-21393108html

[35] P Ziafati A Face Recognition Package for Ros GitHub 2014httpsgithubcomprocrobprocrob functional

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of