Feasibility Study for the Automation ofCommercial Vehicles on the Example of a

Mobile Excavator

Carsten Hillenbrand1, Daniel Schmidt1, Nureddin Bennett2, Peter Bach3,Karsten Berns1, and Christian Schindler2

1 Robotics Research Lab at the Department of Computer Sciences, University ofKaiserslautern, P.O. Box 3049, 67653 Kaiserslautern, Germany

2 Chair of Design in Mechanical Engineering, University of Kaiserslautern,Gottlieb-Daimler-Strasse 42, 67663 Kaiserslautern, Germany

3 VOLVO Construction Equipment GmbH & Co. KG, 54329 Konz, Germany

Abstract. The automation of mobile working equipment used in farm-ing, forestry and the construction industry has great market potential.Tasks of a highly complex nature could be executed without interventionof the operator. Despite the potential significance of this topic, only veryfew results of international research projects in robotics have heretoforebeen applied in the serial production of commercial vehicles.The joint project AMoBa (Autonomer Mobiler Bagger, AutonomousMobile Excavator) of Kaiserslautern University and Volvo CE aims totransfer knowledge and experience from the field of robotics to the area ofmobile excavators. The objective is the realization of fully autonomousoperation in basic construction activities, such as terrain modeling ortrenching. For this purpose a prototype is being prepared by modifyingan 18t Volvo Mobile Excavator to meet the project’s requirements. It isbeing equipped with electronic interfaces to control all relevant functions;i.e. the hydraulic arm cylinders, thereby employing sensor devices for theacquisition of the kinematics (i.e. travel sensors or angular sensors) andsensors for the acquisition of the environment surrounding the roboticexcavator. For example, 2-D laser scanning devices and 3D cameras.Mechanical and hydraulic simulation models are being created to enablestudies of suitable algorithms for trajectory control of the excavators’kinematics. Algorithms concerning the global task of processing the au-tomation task are developed and tested in the software framework MCA2before they are ported to the prototype. The MCA2 framework com-bines the simulation of the excavator and its environment, including thesimulation of environment sensors. On completion of this project, thealgorithms developed in the simulation will be verified on the prototype.

1 Background

The importance of autonomous machines and assistance functions is perma-nently growing. Leading car manufactures and research institutes attend chal-lenges for fully autonomous cars. In the field of mobile machines autonomousguidance systems for agricultural vehicles are already available on the market.

Referring to this, the visionary objectives are mobile construction machinesthat work like a robot with versatile benefits. For instance the operating costscould be lower due to less manpower requirements and less non-working time.The machines could be used in rough working environments without endangeringany operator. The operating conditions could be adjusted in a way that fuelefficiency or material damage is reduced to a minimum. Regarding the human-machine interface a landscape architect or a construction engineer could designa given environment on a PC. Then, the construction machines could start theirwork autonomously exactly reproducing the architects ideas. Apart from thesevisions autonomous machines will only have market opportunities when theydemonstrate their profitability in customer use and when they have a safetystandard that is comparable to ordinary machines or even higher.

With this background, the pilot project investigates how autonomous func-tions could be realized on a mobile wheeled excavator. This type of machine hasbeen chosen for the project because of the wide range of applications that areeven beyond typical earth moving machinery tasks. Excavators are comparablewith industrial automation robots. The kinematics of the digging apparatus isquite similar and it can be equipped with a variety of tools. Hence, excavatorscan be seen as mobile machining tools.

2 Project AMOBA

The joint project AMoBa (Autonomer Mobiler Bagger, Autonomous MobileExcavator) of Kaiserslautern University and Volvo CE aims to realize fully au-tonomous operation of a mobile excavator. Kaiserslautern University is involvedwith two chairs, the Robotics Research Lab RRLab and the Chair of Design inMechanical Engineering KIMA. The project AMoBa is finacially supported bythe ”Stiftung Rheinland-Pfalz fur Innovation”. Started in August 2008, it willbe supported until end of 2010.

The specific task is to accomplish autonomous terrain modeling and trenchingwith a given VOLVO EW180, an 18 ton state of the art mobile excavator. RRLabhas extensive knowledge in mobile robotics using alternative approaches likebehavior-based control. One very successful example is the autonomously driv-ing vehicle RAVON. In this project, existing knowledge in robotics of RRLabis transfered to an industrial application. KIMA supports the project with itsexperience in commercial vehicles, i.e. the implementation of electro-hydraulicinterfaces and accurate simulation models of the mechanical and hydraulic sys-tems involved.

The project formulation is a first approach into the wide-ranging challengeof autonomous machinery for commercial purposes. The manual controls are tobe altered for digital computer control. However, the current state of the art inmobile excavators is based on pure hydraulic actuation, with very little electron-ics involved. The mechanical stresses on excavators in daily use are massive. Atthe same time, the requirements for reliability are extremely high. Heretofore,suitable sensor systems are to be found to measure the current state of internal

parameters. Even more important is the acquisition of environmental informa-tion with regards to the area surrounding the excavator. All sensory informationhas to be read out and evaluated in real-time. As the next step, suitable controlalgorithms for the two given digging tasks have to be developed. This involvesthe implementation of the pure digging process, bucket-ground interaction, andfinally the execution of the main objective of performing defined landscapingand trenching.

In order to concentrate on the main objective, the driving operation of theexcavator has been excluded. Within this project, the excavator will be standingon its’ outriggers. The environment surrounding the machine will be free frommoving objects including human beings.

3 The challenge for robotics

Today’s standard industry robots operate with high precision at high velocitiesin the production process. Unfortunately, this is only possible as their completemotion is pre-calculated, including velocities and accelerations, and stored asa fixed program. As no variance is allowed in the plan, a static environmentbecomes necessary. The robots are surrounded by enclosed cages in order toprevent human beings from disturbing the manufacturing process.

However, the situation in outdoor robotics is totally different concerning theenvironment. All information about the perimeter and the global position isunknown and has to be aquired. Disturbances are very likely to occur. In thiscase an appropriate solution for controlling the excavation process is required.

Related work concerning the control of autonomous excavators [1] uses taskcentered planning structures defining goals for specific trench excavation tasksbased on real operator behaviors [2]. Goals are divided into different activities.The planning structure is capeable of creating working trajectories ensuringsmooth bucket movements. An additional strategy is implemented for the re-moval of obstacles inside the excavation space. As no complex sensor evaluationtake place, the strategies are only functional up to a limited obstacle size. Others[3,4,5] can be described as parametrized scripted joint control, based on existingknowledge about excavation strategies. Here, a central planner creates the con-trol values from excavation to dumping using a set of parametrized sequentialscripts. However, in case of unexpected disturbances, script parameters have tobe recalculated, which is likely to lack performance concerning external distur-bances. Furthermore, scripts cannot be paused at arbitrary times, which leadsto undesired repetitions of scripted steps or a complete maneuver.

4 Adaptive concept for outdoor robots

Due to disturbances in the area of outdoor robotics, an appropriate solution forcontrolling the highly dynamic excavation process is required. As behavior-basedapproaches have shown their suitability for various applications [6], the same canbe expected for the field of construction equipment. Here, the behavior based

iB2C architecture [7] is used which is implemented in the Kaiserslautern branchof the Modular Controller Architecture MCA21.

4.1 Behavior Based Control Architecture

Behaviors can be described as controlling units, which try to reach and to main-tain a desired goal continuously, like a specific boom elongation or a turningvelocity. As the individual goals may overlap, the behaviors, which are usuallyconnected in a behavior network, may stimulate or inhibit others. The coordina-tion of directly competing behaviors is accomplished by so-called fusion behaviorshaving the same interface as basic behaviors. A further hierarchical level can beintroduced by arranging several behaviors in a behavioral group which works asa behavior itself again.

The development process for iB2C starts with the top-down design stagewhere the given task is decomposed into sub tasks. It is described by means ofthe excavation of a trench, which can be partitioned into the following elements:

1. Make a surface scan with the laser scanners and identify a target excavationposition.

2. Approach the target position with the bucket.3. Scratch the surface deep enough to dig out the desired amount of soil.4. Move the bucket to a target dumping position.5. Dump the soil onto a pile at a given position.

Each of these elements can be decomposed according to the degree of freedomthey influence. These are:

– Turn the torso angle (e. g. surface scanning)– Adjust the bucket extension (e. g. approaching the excavation point)– Adjust the bucket height (e. g. digging deep)– Change the bucket pitch angle (e. g. excavation or dumping of soil)

Removing soil from the surface is performed similar to the strategy of ahuman driver, depicted in figure 1. First, the bucket vertically penetrates thesurface until it reaches a depth of around 20 cm. Then, a scraping behavior isachieved by a combination of adjusting the bucket extension and the bucket anglewhile the bucket height and the torso angle are kept. As a whole, the process isperformed by four behaviors influencing the required degrees of freedom (dof)of the bucket.

1 MCA2 is a framework for robot prototyping and development. Here the KL branchbuilds the background for the system. Resources and information how to use andinstall the system can be found under http://rrlib.cs.uni-kl.de/mca2-kl.

Fig. 1. Idealized desired trajectory of the bucket during the excavation process.First the bucket is lowered until the desired value of 20 cm is reached. Afterwards,it is permanently pulled and turned become to constantly decrease the boomlength and adjust the bucket angle.

4.2 Environment perception

In order to obtain information regarding the environment surrounding the exca-vator, Laser scanning devices are used. Their 3D point clouds are permanentlyupdated during the excavation process and deliver the basis for the surface eval-uation algorithm, visualized in figure 2. It uses one two-dimensional grid foreach point cloud and evaluates the average value of scanned points per cell. If nopoint in the cell can be found, the surrounding cells are used to build an averagevalue where empty cells without surrounding filled ones will get a height in zdirection of zero. Afterwards, the two evaluated grids for the actual and the de-sired surface are compared. A third height difference grid is built by subtractingthe desired surface grid from the actual surface grid. To find possible positionsfor excavations areas in the size of the bucket, around 1.5 m2 in the actual case,with a minimum excavation depth of 20 cm, are located. The one with the mostsurrounding space and the deepest excavation depth will be chosen as excavationarea.

5 Toolchain

5.1 Mechanical Simulation

For the development of appropriate control algorithms, extensive simulationmodels are required. This involves the simulation of the mechanics and the actu-ators and drives of the excavator. The mechanics are modelled in a multi-body-

z

y

x

Actual surfaceActual surface

Desired surfaceDesired surface

Excavation areasExcavation areas

Chosen areaChosen area

Fig. 2. Graphical representation of the surface distance evaluation algorithmusing two grid structures for the actual surface and the desired surface. Thefour possible excavation areas are identified and the first area is chosen as it issurrounded by eight green fields and has the highest excavation depth, i.e. thered area at this position is high.

simulation environment. In this project the models are created in MSC.Adams2.The drives and actuators used in today’s excavators are realized with highlysophisticated hydraulic systems. In the case of the EW180B the hydraulics in-corporate load sensing technology which provides load-independent movementsof the digging arm. The hydraulic system is modeled in LMS.Amesim3, a 1-Dmulti domain system simulation software. Both mechanical simulation and hy-draulic simulation are then connected through co-simulation. This final modelmakes it possible to develop optimized control algorithms for trajectory controlof the digging arm, creating the required interface between computer softwareand the excavator.

5.2 Environment Simulation

Motivated by security reasons and therefore to prevent harm from human be-ings, buildings or the excavator itself, a safe test environment is required. TheMCA2 framework is used to build up an interactive, real-time simulation of theexcavator’s shape and it’s environment. Furthermore it allows the representationand testing of control and perception algorithms including sensor devices. As theexcavator is a highly dynamic system, a physics engine4 is used to simulate the2 MSC.Adams is a commercial software for the dynamic simulation of multi-body-

systems. For further information refer to www.mscsoftware.com.3 LMS.Amesim is a software package for 1 dimensional transient simulation of multiple

domain systems. For further information refer to www.lmsgermany.com4 The Newton Game Dynamics physics engine is an integrated solution for real

time simulation of physics environments. Additional information can be found underhttp://newtondynamics.com.

excavator masses and joints and produces a quite realistic behavior of the wholemachine. Interaction with other dynamic elements like walls or other machinesis also included. As sensor systems like laser scanners or stereo vision camerascan be simulated too, appropriate mounting positions and algorithms for en-vironment perception can be safely evaluated. A screen-shot of the simulationincluding a three dimensional surrounding area is shown in figure 3.

Fig. 3. Screen-shot from the test simulation showing the excavator in the 3D-environment. The yellow vertical bars represent artificial landmarks which areused for determining the actual excavator position via triangulation from theperceived sensor data.

5.3 Hardware Implementation in EW180B Excavator

Control tasks for the autonomous excavator can be divided into high-level tasksand low-level tasks. High-level tasks are characterized by complex and compu-tationally intensive tasks whereas low-level tasks are closed-loop control algo-rithms. Low-level functions require real-time processing with minimal latency.High-level tasks are implemented on powerful pc-based computers. Low-leveltasks are run on specialized micro-controllers. Figure 4 illustrates the differentcomponents of the system and the way they are connected.

Actuators The excavators actuation is realized through a state-of-the-art hy-draulic system. However, all actuators are operated directly by the operator.In order to enable digital closed-loop computer control, all actuators have tobe fitted with electric interfaces. This is realized through electro-hydraulic pilotvalves that work in parallel to the manually operated existing valves.

Sensors ProcessingUnit

Actors

Localisation

GPSUSB

InertialCAN

Environment

Laser rangeEthernet

ToF CameraEthernet

Arm Joints

PositionSSI

PressureAnalog

Computer

Micro-controller

CANPWM

Arm Joints

Pilot Valve

Fig. 4. Control system architecture of the excavator

The digging arm itself has 4 DOF, powered by linear hydraulic cylinders.The superstructure is turned through a rotational hydraulic motor. The bucketitself is equipped with a tilting mechanism which adds another DOF. Summingup, there is a total of 6 DOF used in this project (see figure 5):

– Boom cylinder– Adjust cylinder– Arm cylinder– Swing motor– Bucket tilt cylinder

Each one of these hydraulic actuators is controlled by one pilot valve for eachdirection of movement (FW-BW).

Sensors As shown in figure 4, sensors can be categorized in three groups basedon their purpose in the system:

– Localization Sensors– Environmental Sensors– Kinematic Sensors

Localization is realized through GPS sensors and orientation sensors. Informa-tion regarding the environment surrounding the excavator is gathered throughlaser range measuring devices. Time-of-Flight (ToF) camera systems are usedto gather detailed information about the perimeter of the digging arm or thefilling degree of the bucket. The third category of sensors acquires informationabout the excavator itself. For closed-loop control of its linear actuators, posi-tion feedback is required. This is realized through magnetic restrictive position

Elements:

Two-piece boom

Boom

Superstructure

Sensors:

valve pressure

Rotation angle

Actor:

pilot valves

Sensors:

Cylinder length

Cylinder pressure

Rotation angle

Elements:

Dipper arm

Bucket

Fig. 5. Experimental EW180B Volvo bucket excavator

measuring technology. The rotation of the superstructure is acquired through arotational absolute angle sensor. The control of the pilot valves requires feed-back of the control current and the pilot pressure. Each actuator is operated bytwo pilot valves. Beyond the sensor devices required for closed loop control ofthe digging kinematics, both hydraulic pressures of each actuator in the systemare acquired through pressure transducers. This makes it possible to estimateexternal forces acting on the bucket, i.e. digging forces.

Data Processing Unit for High-Level Tasks The sensors acquiring infor-mation about the environment or position of the excavator produce extensiveamounts of data. Sensor data originating from various sensors has to be eval-uated and combined to become useful for the automation task. This processneeds extensive computing power, which is provided with pc-based computersequipped with powerful cpu’s. The system is modular as lacking computationalpower can be compensated by adding further computers. These communicateusing standard ethernet hardware.

Beyond the evaluation of sensor data, the data processing unit serves variousfurther tasks, i.e.

– Graphical user interface– Manual control interfaces (HMI)– Conversion of sensory data into an environment model– Generation of strategies for the digging task– Processing trajectories of the arm– Behavior based control algorithms obstacle avoidance– Kinematic calculation for each joint motion– Security functions

Low-Level Processing Units Closed-loop control requires low-latency pro-cessing devices that are equipped with required I/O interfaces and which ensurereal-time performance. This is realized through digital signal processors com-bined with logic programmable devices for the connection of different interfaces.The DSP’s are programmed to meet hard real-time specifications. Transducersrequired for closed loop control, i.e. pressure and position sensors, are directlyconnected to the DSP boards. Current control for pilot valves is realized throughpulse width modulated amplifiers also directly connected to the DSP’s I/O in-terfaces.

6 Outlook

The project described in this publication aims to create a first approach to-wards autonomous excavators. After the realization of the objectives autonomoustrenching and landscaping, further works are planned in this field. As the chal-lenges are numerous, the next step will be to incorporate the driving functionof the excavator and the interaction with other construction vehicles, i.e. trucksfor the transportation of digging material. However, besides the feasibility of anautonomous operation of an excavator the second outcome of the project mightbe the usage of partial solutions in assistive systems, which could possibly beported to todays excavators and improve the productivity.

The actual behavior-based software control system uses the described algo-rithms for surface evaluation and localization, based on three dimensional laserscans of the environment, to excavate a trench at a given position. Although alot of research has to be done and more complex perception algorithms have tobe developed, the first steps into the area of autonomous mobile excavators havebeen successfully taken.

References

1. D. A. Bradley and D. W. Seward, “The development, control and operation of anautonomous robotic excavator,” Journal of Intelligent and Robotic Systems, vol. 21,no. 1, pp. 73–97, November 2 2004.

2. Y. Sakaida, D. Chugo, K. Kawabata, H. Kaetsu, and H. Asama, “The analysis ofexcavator operation by skillful operator,” in Proc. of 23rd International Symposiumon Automation and Robotics in Construction, 2006, pp. 543–547.

3. H. Cannon, “Extended earthmoving with an autonomous excavator,” Master’s the-sis, Carnegie Mellon Robotics Instiute, 1999.

4. P. Rowe and A. Stentz, “Parameterized scripts for motion planning,” in IEEE/RSJInternational Conference on Intelligent Robots and Systems, IROS 97, vol. 2, 1997.

5. A. Stentz, J. Bares, S. Singh, and P. Rowe, “A robotic excavator for autonomoustruck loading,” Autonomous Robots, vol. 7, no. 2, pp. 175–186, September 1999.

6. R. Arkin, Behaviour-Based Robotics. MIT Press, 1998.7. M. Proetzsch, T. Luksch, and K. Berns, “Development of complex robotic systems

using the behavior-based control architecture iB2C,” Robotics and Autonomous Sys-tems, vol. 58, no. 1, 2010.