1. Car Racing Competition Daniele Loiacono, Julian Togelius and Pier Luca Lanzi CIG, 17 December 2008, Perth, Australia

2. Goal Learnor design a controller for TORCS that races as fast as possible alone or in the presence of others drivers Spiritual successor of CEC 2007 Competition, direct successor of WCCI 2008 3. Car Racing Competition meets TORCS Morerepresentative of real game AI Better interaction with human players Many good programmed controllers available Challenges Howto make it easy accessible? Not designed for Machine Learning! More similar to a real-world problem 4. 4 5. The Open Racing Car Simulator 6. The Open Racing Car Simulator TORCS is a state of the art open source simulator written in C++ Main features Sophisticated dynamics Provided with several cars, tracks, and controllers Active community of users and developers Easy to develop your own controller OS Support Linux: binaries and building from sources suppo Windows: binaries and limited bulding from sources support OSX: legacy binaries and no building from sources support 7. Competition API To make TORCS more easy to use we developed an API basedTORCS on socket (UDP)Patch Values of sensors and effectors are sent through UDPServer BOT 3 components Torcs PatchUDP Server Bot (C++) Client API (C++ and Java)Client Controlller 8. Sensors Rangefindersto sense Opponents Edges of the track Speed Positionon track Rotation speed of wheels Opponent Sensors RPM Angle with track Distance raced Fuel and damage ... Track Sensors 9. Effectors Basically4 effectors Steeringwheel [-1,+1] Gas pedal [0, +1] Brake pedal [0,+1] Gearbox {-1,0,1,2,3,4,5,6} 10. Effectors Basically4 effectors Steeringwheel [-1,+1] Gas pedal [0, +1] Brake pedal [0,+1] Gearbox {-1,0,1,2,3,4,5,6} 11. The Competitors 12. Submissions 5 entries have been submitted to the competition: Matt Simmerson Independent Researcher Luigi Cardamone Politecnico di Milano Chung-Cheng Chiu Redjava - Academia Sinica Diego Perez - University Carlos III, Madrid Aravind Gowrisankar - UT Austin, USA3 more controllers have been considered Danieles heuristic C++ controller Julians heuristic Java controller WCCI 2008 champion (entered by Matt Simmerson) 13. Matt Simmerson:NEAT Controller Controller implemented as an arbitrary- topology neural network, evolved with the NEAT4j implementation of NEAT Inputs: speed, angle to track axis, wall rangenders, track position, gear, wheel spin, RPM Output: driving and gear selection 14. Politecnico di Milano Evolving a Fast Controller for TORCSUsing NEAT Luigi CardamonePolitecnico di Milano Neuroevolution in TORCS CardamoneLuigi 15. Overview We applied NEAT to evolve a fast controller for theCar Racing Simulator To evaluate the candidate solution we measuredtheir performance on a complete lap The evolved neural controller was then coupled witha gear shifting policy and with a basic obstacleavoidance behaviorNeuroevolution in TORCS Cardamone Luigi 16. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 17. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 18. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 19. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 20. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 21. Sensors and effectors Neuroevolution in TORCS CardamoneLuigi 22. Sensors and effectors The following sensors were used: Track sensors at -90,-60,-30,+30,+60,+90 Frontal sensor: max reading among the frontal track sensors at -10,0,10 Car speed The network controls Steering wheel Gas/Brake pedals Neuroevolution in TORCS CardamoneLuigi 23. Controller design To avoid wasting time with fast but slower controller, we set gas pedal to 1 (the max value) when the car is on a straight (i.e., when the frontal sensor return 100m as reading) In addition, the neural controller does not deal with gear shifting and the scripted policy provided is used instead We also used a very simple scripted policy to avoid at least the opponents that are close and in front of the bot Neuroevolution in TORCS CardamoneLuigi 24. Evaluation of Controllers The fitness is computed on an entire lap as Where Tout is the number of game tics the bot was outside the track Savg is the average speed D is the distance raced As soon as a Tout becomes greater than 500 game tics, the evaluation is immediately stoppedNeuroevolution in TORCS CardamoneLuigi 25. An example of behaviors evolved Neuroevolution in TORCS CardamoneLuigi 26. Aravind Gowrisankar Basedon NEAT Preliminary work 20 27. Redjava (Chung-Cheng Chiu) The driving policy is based on determining which sensor detects the most distant value, and turn the steer according to its angle between driving direction. When the car is in the track and the gear is above 3, then it act in the similar policy as SimpleDriver (but without speed limit). Otherwise, accelerate. (brake is opposite) The policy except steering adopt similar code as SimpleDriver (also the unstuck detection). The code is naive since my original attempt wai to apply a machine learning method but I got bad results, thus I removed that part. 21 28. The Results 29. Submissions 5 entries have been submitted to the competition: Matt Simmerson Independent Researcher Luigi Cardamone Politecnico di Milano Redjava - ??? Diego Perez - University Carlos III, Madrid Aravind - ???3 more controllers have been considered Danieles heuristic C++ controller Julians heuristic Java controller WCCI 2008 champion (entered by Matt Simmerson) 30. Scoring setup A server with 2 Quad-core Xeon 2.66 GHz,8GB RAM, running Fedora Core 6 31. Scoring process: rst stage Scoring process involves three tracks(unknown to the competitors): C-Speedway E-Track6 Wheel 2 Onlya controller at once is tested andperformance is defined as the distancecovered within 10000 game tics 32. First Stage: C-Speedway C-SpeedwayRedjavaLuigi Diego CompetitorsMatt Aravind WCCI Champ Daniele Julian 0 5000 1000015000 20000 Distance Raced 33. First Stage: E-Track 6 E-Track 6 Luigi Redjava Aravind Competitors DiegoMatt WCCI Champ Daniele Julian 0 2000 4000 6000 8000 Distance Raced 34. First Stage: Wheel 2Wheel 2RedjavaLuigiMatt Competitors Diego Aravind WCCI Champ Daniele Julian 0 2250 4500 6750 9000 Distance Raced 35. Comments on rs stage Two clear winners: Redjava Luigi WCCIChamp still competitive in all but Wheel 2 track Matts new entry is outperformed by the WCCI champ in all tracks but Wheel 2 (the most difficult track) 36. The nal stage Who is the best controller in presence of opponents? In the final stage we compared the best entries with the previous champion For each track we run 5 races with random starting grids The final score is computed as the median of the 5 races using the GP2 point system (10 to first, 8 to second, 6 to third AND 1 bonus point for the fastest lap) 37. Final results After 25 races Luigi Cardamone is the winner of theCIG 2008 Simulated Car Racing 38. Summary of results Why did Luigi win this competition? It is not the fastest but the most reliable one: Redjavascontroller is extremely fast but oftenlooses the control of the car It often reaches the max amount of damage orjust crashes into other opponents Better shifting policy? (though both hard-coded) 39. Summary of results Good things: Significant improvements with respect to the previouscompetition! One new learned controller performed significantly betterthan programmed controllers! Hand-coding still very competitive An important step to the actual level of the bestprogrammed controller in TORCS but the distance is stillbig Bad things: Still poor performance in avoiding and overtakingopponents! Many crazy and dirty behaviors to go as fast as possibleis really what we are looking for