RoboSina from Scratch

Submitted as partial Fulfillment for the Requirements of the degree of

Bachelor of Science In

Software Engineering

By

Mostafa Rafaie-Jokandan & Nima Kaviani

1. Introduction

2. Simulation System and Soccer Server

3. Learning Algorithms used in RoboSina

4. Prominent Aspects of RoboSina

5. RoboSina Agent and Learning

6. Suggestions and Conclusion

Introduction

Why RoboCup?An approach to Distributed Artificial IntelligenceDAI vs CAI

Where are we moving toward?

Who were we in RoboSina?

Simulation System & Soccer Server

What is a Simulation System? Why Simulation? Soccer Server

Soccer Server

Match RulesControlled by Soccer ServerControlled by Human

Properties of the Simulated PitchA 105.0 * 68.0 field covered by flagsGoal widths are twice a real one

Some of predefined parameters for Soccer Server

A Simulated Field

Models used in Soccer Server

Movement Model Visual Model Aural Model

Visual Model

(see ObjName Distance Direction DistChng DirChng BodyDir HeadDir) ObjName ::= (p ”Teamname” UniformNumber goalie)

| (g [l|r]) | (b) | (f c) | (f [l|c|r] [t|b]) | (f p [l|r] [t|c|b]) | (f g [l|r] [t|b]) | (f [l|r|t|b] 0) | (f [t|b] [l|r] [10|20|30|40|50]) | (f [l|r] [t|b] [10|20|30]) | (l [l|r|t|b])

Visual Formulas prx = pxt − pxo pry = pyt − pyo vrx = vxt − vxo vry = vyt − vyo Direction = arctan(pry/prx) − ao erx = prx/Distance ery = pry/Distance DistChng = (vrx erx∗ ) + (vry ery∗ ) DirChng = [(−(vrx ery∗ ) + (vry erx∗ ))/Distance] ∗ (180/π) BodyDir = PlayerBodyDir − gentBodyDir − AgentHeadDir HeadDir = PlayerHeadDir − AgentBodyDir − AgentHeadDir

Noise Formulas d’ = Quntize ( exp(quantize(log(d) . quantize – step )), 0.1) Quantize (V , Q) = ceiling(V/Q) * Q

Visual Model

ry2

rx2 pp Distance

Visual Model

9045

Aural Model

(hear Time Sender ”Message”)

Movement Model

Movement Noise Model

Catch Dash & Stamina Model Kick Tackle Turn Move Turn-Neck Point to Focus Say Change View Score

Actions for an Agent

Heterogeneous PlayersDo you know how a body builder differs a normal

human? Acting in an environment with heterogeneous

creatures covers all possible properties Coach

Mapping of DAI to CAI with restricted bandwidth in communication

What was the role of Jose Morinio in a team if we didn’t have a coach?!!!

Learning Algorithms

Definition Decision Trees Artificial Neural Networks

Learning Algorithms-Keywords

Environment Property State Goal Learning Algorithm

Learning Algorithms-Keywords

Propertyi

State1

State2

State3Goal State

Decision Trees

Definition ID3 Basis ID3 Formulas ID3 Algorithm What is C4.5

Decision Trees - Definition

A powerful tool for Inductive Inference Proposing a model for discrete environments Resistance against Noisy Data

?

?

Outlook = Sunny, Wind = Weak, Humidity = HighD=( )

Playing a Soccer Gamein case of having a weather

with conditions D

Decision Trees – ID3 Basis

Occam’s Razor, mid of 14th century

“It is vain to do with more what can

be done with less… entities should

not be multiplied beyond necessity”

Decision Trees – ID3 Formulas

Decision Trees – ID3 Algorithm

Decision Trees – C4.5

A developed software on the basis of ID3 It’s abilities

Avoiding over fitting the data Reduced error pruning Rule post-pruning Handling continuous attributes Handling attribute with different costs Choosing an appropriate selection measure Handling training data with missing attribute values Improving computational efficiency

Neural Networks

Definition A Mathematical Representation BackPropagate Basis BackPropagate Algorithm

Neural Networks-Definition

A mathematical model of human’s neurons It’s abilities

Association, Clustering, Classification Pattern Recognition Generalization, Reference Improvement

Classification on the basis of Learning Algorithm Supervised Unsupervised => Competitive

Neural Networks- Math Representation

Neural Networks- BackPropagate Basis

A supervised learning algorithm Works on the basis of revising Weight

Values Training Example (x, t) where x represents

the entry, and t is the expected output value

ai(l) representes the output gerenrated by the network in the layer named l

x = ai(0)

m is the frequency for the execution of learning algorithm

Neural Networks- BackPropagate Algorithm

Prominent Aspects

System Architecture Localizing an Agent Intercept Dribble Formation Defense System Play With Ball System

System Architecture

Describe the structure of the system's componentsFunctional ArchitectureOperational ArchitectureImplementation Architecture

Brief view to Functional Architecture

System Architecture-Functional Architecture

Localizing an Agent

What is localization?

f

Localizing an Agent

• A new approach used• It deals with points rather than edges• Uses a swipe line to find active edges• A point will be added if :

1. It is a result of an intersection2. It is surrounded by both active

edges of the other polygon• Stops after reaching the end point of a

polygon• It is executed in O(m+n) for 2 polygons• T(n) = 2T(n/2) + O (n)• It is executed in O(nlogn) using divide

and conquer for n polygons

Intercept

As a skills Intercept the ball in at any distance.

As a strategy utility Determine ball ownerPredict ball ownerDetermine who must be intercept ball.Need to communication to enhance its efficiency

Dribble

Running with the ball from one point to another as quickly as possible

Misleading obtrusive defender

Holding ball if it is need .

One , Two

Dribble

Dribble

Mislead

Formation

UsageDetermine Agent position relative to the ball

position New Works

New Floating Formation Adding A Free midfielderImplementing the main formation as 4-2-3-1 Implementing forwards press.

New Floating Formation

The 4-2-3-1 formation is the main formation.

4 2 3

1

New Floating Formation

4

The 4-2-4 formation is a fully offensive formation.

24

Defense System

Skills BlockMark

StrategyOffside linePress

Defense System- Block

Defense System- Mark

Defense System- Offside Line

NO, offside!!!

NO, I lost the ball !!

RoboSina Agent & Learning

Agent learns to ShootNeural Network and training a shoot skillUsing BP and Matlab

Agent learns to PassDecision Trees and training a pass skillUsing ID3 and C4.5

Learning how to Shoot

Problem Definition Mapping the environment to a discrete model

6 input entries 6 output entries

Grabbing relevant properties Using trainer to collect training samples Choosing the proper target to shoot

Making the appropriate Neural Network Modeling the network with matlab

A 2 layer network tgsig as activation function in the first layer logsig as activation function in the second layer

Applying training samples to the network Results

Failed

1st Successful State

Last Successful State

State 19 = 100111

State 28 = 111001

State 24 = 110001

(bx, by)

(gx, gy)

distcb

dircb

100011

bx

by

gx

gy

distcb

dircb

Shoot-Results

Harms Very time consuming method!!

AlgorithmOccurrence in 8

matchesSuccess Num Success Percentage

Computational 141 93 65.9%

Learning 173 126 72.8%

Learning how to Pass

Problem Definition Grabbing relevant properties Using trainer to collect training samples

Making the appropriate Decision Tree using C4.5 to create the tree

Creating pass.names file applying training samples to the tree

Creating pass.data file

Results

Pass-Results

Pass Probability Occurrence in 8 matches

Success Num Success Percentage

85-100% 912 753 78.3%

70-85% 328 187 57.01%

55-70% 168 92 54.7%

Harms Partial Observation causes problems Body direction of the receiver is ignored during the train

RoboSina Project

Honors Championship of the 3rd US-Open RoboCup Competitions,

May 2005, Atlanta, USA 2nd place of the 3rd Iranian Open RoboCup Competitions,

April 2005, Tehran, Iran 5th place of RoboCup World Cup 2004, Lisbon, Portugal Championship of the 2nd US-Open RoboCup

Competitions, May 2004, New-Orleans, USA Championship of the 2nd Iranian Open RoboCup

Competitions, March 2004, Tehran, Iran 10th place RoboCup World Cup 2003, Padova, Italy

Conclusions &

Suggestions

Acknowledgements

Thank you very much for being here

Any Question?

We are ready to Answer