Dr M F Abbod

Using Intelligent Optimisation Methods to Improve the Group Method

of Data Handling in Time Series Prediction

Maysam Abbod and Karishma Dashpande

School of Engineering and DesignBrunel University, West London

Dr M F Abbod

Outline

• GMDH

• Genetic Algorithms

• Particle Swarm Optimisation

• Financial Data

• Prediction Results

• Conclusions

Dr M F Abbod

Introduction

• The GMDH is an algorithm to learn inductively, combinatorial multi-layers for modelling complex systems.

• The method was introduced by A. G. Ivakhnenko in 1966 and several scholars has since developed the theory GMDH for various applications.

Dr M F Abbod

GMDH

An important feature of the algorithm GMDH is providing robust polynomial regression models of linear and non-linear systems.

Dr M F Abbod

Principle of Selection

Ivakhnenko uses the principles of selectivity - "to get plants, for example, with certain properties, there is the first cross and then the first harvest. Later picks up the best plants and it is the second crossing and the second harvest and thus to find a plant that is desired. "

Dr M F Abbod

GMDHGMDH-layers

All combinations of inputs are generated and issued the first layer of the network. The outputs of these are classified and then selected for entry into the next layer with all combinations of selected outlets.

Only those elements whose performance was acceptable survive to form the next layer.

This process is continued as long as each layer (n +1) subsequent produce a better result than the layer (n). When the layer (n +1) is not better as the layer (n), the process is stopped.

Dr M F Abbod

SELECTION 1

SELECTION 2

x1 x2 x3 x4 . . . xn

y1 y2 y3 . . . yn

A11

A12

A13

A1n

A21

A22

A23

A2n

Layer 1 Layer 2

.

.

.

.

.

.

GMDH

Dr M F Abbod

• GMDHEach layer consists of Polynomial Equation generated from combinations of pairs of inputs. Each node is the way Ivakhnenko polynomial which is a polynomial of the second order:

The error we are computed by RMSE and MAPE:

The Choice of Plymomial Eq

%1001

1

n

i i

ii

y

zy

nMAPE

n

iii zy

nRMSE

1

21

ijijjjjiiijjii xaxaxaxaxaay 22

Dr M F Abbod

The Coefficients

Determining the values that can produce the best adjustment of the equation

Dr M F Abbod

Genetic Algorithms

It was developed by Goldberg in 1989.

Genetic Algorithms (GAs) are randomised search and optimisation techniques guided by the principles of evolution and natural genetics

Dr M F Abbod

Genetic Algorithms

• Chromosomes are an encoded representations of the solutions, each gene represents a feature

• A fitness value that reflects how good it is

• A crossover mechanism that exchanges portions between strings

• Mutation plays the role of regenerating lost genetic material

Dr M F Abbod

Particle Swarm OptimisationRules of movement – the formulas:

)()()()1( 21 xgxptvtv

)1()()1( tvtxtx

x

y

Dr M F Abbod

The Data• USD2EURO from 29

Sept, 2004 to 5 Oct, 2007.

• GBP2USD from 29 Sept, 2004 to 5 Oct, 2007.

• www.oanda.com 1.7

1.75

1.8

1.85

1.9

1.95

2

2.05

2.1

Jan-

04

Mar

-04

May

-04

Jul-04

Sep-

04

Nov

-04

Jan-

05

Mar

-05

May

-05

Jul-05

Sep-

05

Nov

-05

Jan-

06

Mar

-06

May

-06

Jul-06

Sep-

06

Nov

-06

Jan-

07

Mar

-07

May

-07

Jul-07

Sep-

07

Date

Exc

hang

e R

ate

ll

0.7

0.72

0.74

0.76

0.78

0.8

0.82

0.84

0.86

0.88

Jan-

04

Mar

-04

May

-04

Jul-04

Sep-

04

Nov

-04

Jan-

05

Mar

-05

May

-05

Jul-05

Sep-

05

Nov

-05

Jan-

06

Mar

-06

May

-06

Jul-06

Sep-

06

Nov

-06

Jan-

07

Mar

-07

May

-07

Jul-07

Sep-

07

Date

Exc

hang

e R

ate

ll

Dr M F Abbod

The Data

• 2 data sets (GBP2USD & USD2EUR)

• 120 Data points

• 100 for training

• 20 for testing

Dr M F Abbod

Training Data Performance

USD2EUR GBP2USD Algorithm type

MAPE RMSE MAPE RMSE

GMDH 0.31024 0.0035959 0.29232 0.0076715

PSO-GMDH gbest 0.30258 0.0035831 0.29009 0.0076768

PSO-GMDH lbest 0.30351 0.0035822 0.29024 0.0076835

GA-GMDH 0.30553 0.0036046 0.29429 0.0076953

GA-PSO-GMDH 0.30198 0.0035611 0.29006 0.0076707

Dr M F Abbod

GMDH

GMDH predictions on testing set for (a) USD2EUR, and (b) GBP2USD

Dr M F Abbod

PSO-GMDH (gbest)

PSO-GMDH gbest model predictions on testing set for (a) USD2EUR and (b) GBP2USD

Dr M F Abbod

PSO-GMDH (lbest)

PSO-GMDH lbest model predictions on testing set for(a) USD2EUR and (b) GBP2USD

Dr M F Abbod

GA-GMDH

GA-GMDH predictions on testing set for

(a) USD2EUR, and (b) GBP2USD

Dr M F Abbod

GA-PSO-GMDH

GA-PSO-GMDH predictions on testing set for

(a) USD2EUR and (b) GBP2USD

Dr M F Abbod

Testing Data Performance

USD2EUR GBP2USD Algorithm type

MAPE RMSE MAPE RMSE

GMDH 0.18276 0.0017923 0.17727 0.0049046

PSO-GMDH gbest 0.16956 0.0017084 0.17718 0.0048962

PSO-GMDH lbest 0.16709 0.0016749 0.17705 0.0049657

GA-GMDH 0.17630 0.0017688 0.17809 0.0050035

GA-PSO-GMDH 0.16537 0.0016672 0.17592 0.0048144

Dr M F Abbod

USD2EUR

0.7

0.71

0.72

0.73

0.74

0.75

0.76

20/0

4/2

007

04/0

5/2

007

18/0

5/2

007

01/0

6/2

007

15/0

6/2

007

29/0

6/2

007

13/0

7/2

007

27/0

7/2

007

10/0

8/2

007

24/0

8/2

007

07/0

9/2

007

21/0

9/2

007

Date

Exchange R

ate

US

D2E

UR

a

Actual GMDH GA PSO lbest PSO gbest GA-PSO

Dr M F Abbod

GBP2USD

1.92

1.94

1.96

1.98

2

2.02

2.04

2.06

2.08

20/0

4/2

007

04/0

5/2

007

18/0

5/2

007

01/0

6/2

007

15/0

6/2

007

29/0

6/2

007

13/0

7/2

007

27/0

7/2

007

10/0

8/2

007

24/0

8/2

007

07/0

9/2

007

21/0

9/2

007

Date

Exchange R

ate

GB

P2U

SD

a

Actual GMDH GA PSO lbest PSO gbest GA-PSO

Dr M F Abbod

Performance Improvements

USD2EUR % improvement

GBP2USD % improvement Algorithm type

MAPE RMSE MAPE RMSE

PSO-GMDH gbest 7.22 4.68 0.05 0.17

PSO-GMDH lbest 8.57 6.55 0.12 -1.24

GA-GMDH 3.53 1.31 -0.46 -2.01

GA-PSO-GMDH 9.52 6.98 0.76 1.84

Dr M F Abbod

Computational Requirements

Algorithm type Number of layers Computation Time (sec)

GMDH 2 0.36

PSO-GMDH gbest 4 73.91

PSO-GMDH lbest 2 312.76

GA-GMDH 4 3694.88

GA-PSO-GMDH 4 3039.18

Dr M F Abbod

Conclusions

• Improvements can be achieved

• Model Complexity and Computational burden

• Parallel Processing (Matlab: Parallel Computing Toolbox)

• Other data sets