Forecasting the Runoff Using Least Square Support Vector Machine

7/31/2019 Forecasting the Runoff Using Least Square Support Vector Machine

1/6

884

Forecasting the Runoff Using Least Square Support Vector Machine

FENG Lijun1,2 LI Shuquan1

1Tianjin University of Finance and Economics, P.R.China, 3002222 College of Urban and Rural Construction, Agriculture University of Hebei, P.R.China, 071001

[email protected]

Abstract: To forecast the runoff of medium and long term is a research difficult problem in naturalscience domain, and that to forecast the runoff accurately is the important foundation of preventingflood and reducing natural disasters and optimizing the management of water resource. For this problem,

we propose a new forecast method based on least square support vector machine in this paper to forecastthe runoff of medium and long term, and we compare the forecast result using least square supportvector machine with the forecast result using artificial neural network (Back propagation, BP) . Theexperiments prove that the method of least square support vector machine has advantages of lower errorin simulation and higher precision in forecast comparing with artificial neural network (Back

propagation, BP).Keywords: Runoff, Least Square Support Vector Machine, Forecast , BP

1 Introduction

To forecast the runoff of medium and long term is a research difficult problem in natural science

domain. Its difficult lies in that hydrology situation is affected by various uncertain factors, such asclimate, weather, mankind activity and geographical environment change

[1]. But forecasting the runoff

accurately is the important foundation of preventing flood and reducing natural disasters and optimizing

the management of water resource. So improving the precision of forecasting runoff of medium andlong term is the research focus at home and abroad all the time.

At present the methods to forecast runoff are numerous. According to technology route, we can

divide these methods into two category. One is to construct the forecast model based on change law ofrunoff, such as time series model, artificial neural network model,wavelet model,pattern regconitionmodel and so on. The other is to construct the forecast model based the relation between runoff and itsinfluence factors, such as plural linear regression model, ANN model, fuzzy mode model and so on. It issucessful for these models to forecast runoff generally. But the important foundation of parameterlearning method of these forecast models is statistics. The traditional statistics theory is used based on

big data sample situation. In fact we are difficult to get big data sample when forecasting runoff. So it isvery necessary to study the statistics theory based on samll data sample. In 1970s, Vapnik began to studythe statistics learning theory based on samll sample and proposed thelearning method of support vector

machine in 1995. This method is a new and different from the learning method of ANN. At present thismethod is used in a lo of research fields and the research result indicates this method is feasible andsuperior. But its application in the fields of hydrology and water resouce has just started. Some

documents indicate that the authors at home studied the flood predication and ground water prediction

etc. using support vector machine method

[2]

.Recently, the overseas scholars have made great progress on the research of statistical learningtheory. This method has many merits in setting up model in the respects of using small sample andknowledge acquirement, and the like. In this paper the author proposed the least square support vectormachine method(LS-SVM) and forecasted the runoff using it.Our main purpose lies in introducing thethinking, characteristic and key place of LS-SVM briefly and offer the new approach for forecasting therunoff.

2 Theory of support vector machine

Project supported by investment of Tianjin teaching committee (No. TJGL06-099)


2/6

885

Support Vector Machine (SVM)[3] is a kind of new machine learning algorithm that has developed

on the basis of statistical learning theory. Based on the principle of structural risk minimization, thisalgorithm can solve the problem of overfitting effectively and has good generality capability and betterclassification accuracy. It is becoming a new study focus of machine learning field after

pattern-recognition and neural network.

2. 1 Vapniks SVM theoryThe theory of SVM was originally put forward by Boser and Guyon and Vapnik

[4]at the

Computational Learning Theory Congress held in 1992. SVM was originally used to find the optimalseparating hyperplane of linear classification problem. The so-called optimal separating hyperplane not

only can be used to separate the data correctly, but also can maximize the margin. Therefore, in regard to

known observation samples ),( 11 yx , ),( 22 yx ),( nn yx Considering two separate classes,

1=iy , we can construct the optimal separating hyperplane to classify the samples. The problem ofconstructing the optimal separating hyperplane can be turned into the following optimization problem:

( ) 221min ww = (1)

s.t. )( bxwy ii + 1 i 1 2 l

The problem above can be transformed into the following dual problem by using Lagrange optimizationmethod:

( ) ===

=l

j

jjijij

l

j

i

l

i

xxyyW

111

),(2

1min (2)

s.t. 0

1

==

i

l

i

iy i 0, i 1 2 l

In which i denotes the Lagrange multipliers.

Solving Equation (2) with constraints Equation determines the Lagrange multipliers, and the optimalseparating hyperplane is given by the following equation,

}),(sgn{})sgn{()( ** bxxybxwxf ii

l

i

ii +=+= (3)

In which sgn () denotes the sign function.

So far the discussion has been restricted to the case where the training sample is linearly separable.

However, in general this will not be the case. In the case where it is misclassification, alternatively amore complex function can be used to describe the boundary. To enable the optimal separating

hyperplane method to be generalized, Cortes and Vapnik (1995) introduced non-negativevariables, i 0. The i is a measure of the misclassification errors. The optimization problem is now

posed so as to minimize the classification error as well as minimizing the bound on the VC dimension of

the classifier. The generalized optimal separating hyperplane is determined by the vector wwhichthat minimizes the functional,

( ) +=l

i

iCww 2

2

1,min (4)

Where C is a given value. The generalized optimal separating hyperplane is nearly the same as to


3/6

886

linearly separable problem, just the constraints Equation turns into 0 i C i 1 2 l.

To non-linear problem, we can transform it to the problem of a high dimensional feature space by the

use of reproducing kernels. The idea of the kernel function is to enable operations to be performed in theinput space rather than the potentially high dimensional feature space. Hence the inner product does notneed to be evaluated in the feature space. This provides a way of addressing the curse of dimensionality.

So the optimal separating hyperplane is transformed to:

}),(sgn{)( bxxKyxf ii

l

i

i += (5)

Where )()(),( jiji xxxxK =

Hence, if we select different kernel function, we can acquire different support vector machines.

2. 2 LS-SVM theory

Suykens J.A.K[5] put Least Squares Support Vector Machine (LS-SVM) forward in 1999. In regard

to some known observation samples ),( 11 yx , ),( 22 yx ),( nn yx Ryi , if we classify these data,

we can construct the following optimization problem:

( ) =

+=n

i

kT

ebwewwewJ

1

2

,, 2

1

2

1,min (6)

s.t. ( ) iiT

i ebxwy ++= , =i 1 2 3 n

Where is the regularization parameter, determining the trade-off between the fitting error

minimization and smoothness.

The solution is obtained after constructing the Lagrangian,

( ) ( ) ( ) }{,,,,1

iiiT

n

i

i yebxwewJebwL ++= =

(7)

Where i is Lagrangian multiplier.Application of the conditions for optimality yields the following

linear system (8):

=

+ y

b

II

IT 01

0

(8)

where 111 K=I [ ]n K21= [ ]nyyyy K21= Mercers condition is applied in

the matrix( ) jiji xxxxK == , (9)

The resulting LS-SVM model for function estimation becomes function (10).

( ) bxxKy il

i

i += =1

(10)

where i , b comprise the solution to the linear system.

In Equation (10), ),( ixxK is the so-called kernel function with which the input vector can be mapped


4/6

887

implicitly into a high-dimension feature space. The most usual kernel functions are polynomial,

Gaussian-like or some particular sigmoid.

3 Forecasting process

In order to forecast the runoff, we constructed the following forecasting process. We may carry onthe work according to this process. This process is as figure 1:

Figure 1. The process of forecast investment risk

In Figure 1, first, we need to construct forecast index system. In other words, we must know whatfactors we should take into consideration in the process of forecasting runoff.As for different region, theinfluence factors selected are dissimilar. So we must select the influence factors scientifically andreasonably according to the different region location when carrying on the forecast work of runoff.Whether we select the influence factors scientifically and reasonably would affect the justness andaccuracy of runoff forecast.

Second, we need to get the LS-SVM training sample. According to theconstructedforecast indexsystem, we collect the historical hydrology data of some region so that we can acquire the observation

sample of LS-SVM. See),( 11 yx

,),( 22 yx

),( nn yx

. In this sample,ix( =i

1 2 3 n

)

is a vector in which each element is a forecast index. iy denotes the forecast result. Different output of

iy reflects different value of runoff.

Third, we need preprocess the LS-SVM training sample, including normalizing the data, and so on.By means of preprocessing the sample data, we may optimize the LS-SVMs capability of study anddecision-making.

Fourth, we forecast the runoff by the LS-SVM model which we have got. By training the

observation sample, we can get the LS-SVM decision-making model ( ) bxxKy il

i

i += =1

. Because

these parameters decide the LS-SVM learning and forecast capability to a great degree, the selection of

hyperparameter and kernel function parameter is very important when we train LS-SVM. After we

get the LS-SVM model ( ) bxxKy il

i

i += =1

, we can input the data preprocessed into the LS-SVM

model. According to the output result of model we can get the value of runoff.

4 Experiments

According to the requirement of the research task, we collect the hydrology data of recent 25yearsfrom some hydrology station. We divide the data into two groups, namely taking the previous 21 years

data as training data and taking the rest 4 years data as test data. In these data, iy denotes the annual

Constructing forecast index system

Getting sample

Preprocessing data

Forecast runoff


5/6

888

runoff value and ix denotes forecast index. 4,3,2,1=i . 1x demotes the total rainfall from December to

November of last year. 2x demotes the total rainfall of January of this year. 3x demotes the total

rainfall of February of this year. 4x demotes the total rainfall of March of this year. Then we analyzed

data using LS-SVM matlab toolbox. To the kernel function, we compare the polynomial kernel, radial

basis kernel, sigmoid kernel and so on. At last we found radial basis kernel fits the problem of runoff

forecast. To kernel parameter2, , we found the forecast result is good when 20.01, 0.05 = = .

In order to carry on method comparison, we analyzed the casualty data using BP algorithm again. Seetable 1. In table1, from sample No. 22 to 25 are test samples and the rest are training samples.

From table1 we draw the conclusion that the method of LS-SVM has the obvious superioritycomparing with the method of artificial neural network. Besides, we are prone to get into localoptimization and we need to determine the network structure and its parameters when using the methodof artificial neural network. Generally to choose the parameter of artificial neural network is moredifficult than LS-SVM. Considering the speed of the computation, the method of LS-SVM is quicker

than artificial neural network.So, the method of LS-SVM has more accurate forecast ability and better application prospect in

hydrology and water resource domain.

5 Conclusions

We use the method of LS-SVM to deal with forecast problem of runoff in this paper. This method

can take full advantage of the data distributing characteristic, and that we dont need previousknowledge and skill when constructing making-decision function. This proposes a new approach for therunoff forecast. From now on we will further our study on how to choose kernel function and itsparameter, thus make the runoff forecast more convenient and accurate.

References

[1]Wang Wensheng, Cao Xuewei, Lei Danfa. Application of Wavelet Network Model to Forecast

Runoff of Geheyan Reservoi, Hubei Water Power, 2005,(4) , pp.10-12 (in Chinese)[2]Liao jie, Wang wensheng,Li yueqing, Huang weijun.,Support Vector machine Method and Its

Tabble 1 The runoff data of some hydrology station and its forecast results comparison of using different

method

No.Real value

(m3s-1)

Forecast results of using LS-SVM Forecast results of using BP

Forecast value(m3s-1)

Forecast error % Forecast value(m3s-1)

Forecast error %

1 22.9 23.667 3.349 29.438 28.550

2 23.4 23.012 1.658 26.290 12.352

3 36.8 32.165 12.595 31.327 15.144

4 22.0 21.301 3.177 19.162 12.900

22* 39.9 36.552 8.391 31.066 22.140

23* 24.6 23.040 6.341 30.840 25.367

24* 20.4 22.796 11.745 17.877 12.365

25* 30.3 33.124 9.320 30.982 2.251


6/6

889

Application to Prediction of Runoff. Journal of Sichuan University.,2006,38(6), pp. 2428(in

Chinese)[3]Corinna Cortes, Vladimir Vapnik, Support vector networks, Machine learning, 1995,20(3), pp.

273297

[4]Vapnik V. Statistical learning theory [M], New York: John Wiley & Sons, 1998.[5]J.A.K. Suykens, J. Vandewalle, Least squares support vector machine classifiers, Neural processing

letters, 1999,9(3), pp. 293300

Documents

Forecasting the Runoff Using Least Square Support Vector Machine