Probability Distribution and Deviation Information Fusion ...downloads.hindawi.com/journals/mpe/2017/9650769.pdf2 MathematicalProblemsinEngineering methods are utilized in SVR [13]

Research ArticleProbability Distribution and Deviation Information FusionDriven Support Vector Regression Model and Its Application

Changhao Fan and Xuefeng Yan

Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of EducationEast China University of Science and Technology MeiLong Road No 130 Shanghai 200237 China

Correspondence should be addressed to Xuefeng Yan xfyanecusteducn

Received 29 June 2017 Revised 25 August 2017 Accepted 30 August 2017 Published 12 October 2017

Academic Editor Xinkai Chen

Copyright copy 2017 Changhao Fan and Xuefeng Yan This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited

Inmodeling only information from the deviation between the output of the support vector regression (SVR)model and the trainingsample is considered whereas the other prior information of the training sample such as probability distribution information isignored Probabilistic distribution information describes the overall distribution of sample data in a training sample that containsdifferent degrees of noise and potential outliers as well as helping develop a high-accuracy model To mine and use the probabilitydistribution information of a training sample a new support vector regression model that incorporates probability distributioninformation weight SVR (PDISVR) is proposed In the PDISVR model the probability distribution of each sample is consideredas the weight and is then introduced into the error coefficient and slack variables of SVR Thus the deviation and probabilitydistribution information of the training sample are both used in the PDISVRmodel to eliminate the influence of noise and outliersin the training sample and to improve predictive performance Furthermore exampleswith different degrees of noisewere employedto demonstrate the performance of PDISVR which was then compared with those of three SVR-basedmethodsThe results showedthat PDISVR performs better than the three other methods

1 Introduction

Since its proposal by Vapnik the support vector machine(SVM) has been used in many areas including both patternrecognition and regression estimation [1 2] The originalSVM is utilized to provide a pair of parameters as a solutionto a quadratic program problem SVM has some advantagessuch as low standard deviation and easy generation as well assome disadvantages such as the redundancy of the regressionfunction and the low efficiency of support vector selectionTo address these disadvantages various improvements to thesupport vector algorithm and its kernel function have beenproposed Suykens proposed least-square support vectorregression (LS-SVR) for a regression modeling problem [34] By transferring inequality constraints to equality con-straints LS-SVR simplifies the solution to quadratic programproblems [5] In the field of regression Smola proposedthe linear programming support vector regression (LP-SVR)model [6 7] LP-SVR has numerous strengths such as the

using of more general kernel functions and fast learningability LP-SVR can control the accuracy and sparseness ofthe original SVR by using the linear kernel combination as asolution approach In addition a new kernel function multi-kernel function (MK) has been introduced into the standardSVM model MK provides lower fault and requires a shortertraining period than the original kernel function Multiple-kernel SVR (MKSVR) is very popular in some systems Yeh etal [8] developed MKSVR for stock market forecasts Lin andJhuo [9] discovered a method to generate MKSVR param-eters for integration into a system that converts the pixelsof a checkpoint into the brightness value Zhong and Carr[10] used theMKSVRmodel to estimate pure and impure car-bondioxide-oilmatrixmetalloproteinases in aCO2 enhancedoil recovery process

The SVR model also has been improved by prior knowl-edge [11 12] There are numerous types of prior knowledgeincluding the average value and monotonicity of the sampledata To appropriately use prior knowledge three types of

HindawiMathematical Problems in EngineeringVolume 2017 Article ID 9650769 11 pageshttpsdoiorg10115520179650769

2 Mathematical Problems in Engineering

methods are utilized in SVR [13] Our team previouslyworked on the monotonous a priori knowledge of sampledata Our monotonous a priori knowledge of the sample datais described by first-order difference inequality constraints ofkernel expansion and additive kernels [14] The constraintsare directly added to kernel formulation to acquire a convexoptimization problem For additive kernels SVMs are con-ducted through the addition of dissociate kernels for everyinput dimensionThese operations confer higher accuracy tothe SVR model in support vector (SV) selection

Inevitably even small noise can debase the accuracy ofthe model Furthermore in some situations part of the noisyinformation may be ten to even dozens of times larger thanthe normal data These outliers introduce bias and inac-curacies to SVR Nevertheless the probability distributionof the sample data is a good indicator of noise From theperspective of the probability distribution of sample datanormal data and data that contain the least amount of noisehave the highest probability in the sample data By contrastdata that contain large amount of noise have relatively smallprobability Thus outliers in the sample data will have thesmallest probability Therefore the probability distribution isthe prior knowledge that helps weaken the influence fromnoise and outliers in the sample data We consider thisinformation to modify our SVR model

This article is structured as follows Section 2 introducesstandard SVR algorithms Section 3 describes the proposedalgorithm that integrates probability distribution informa-tion into the SVR framework Section 4 provides someexperimental results that were obtained from comparing theproposed algorithm with other algorithms Finally Section 5presents some conclusions about the proposed algorithm

2 Review of SVR

To better describe the proposed algorism the mathematicalclarification of the basic concepts of SVR and the usage ofdeviation information should be provided

21 Support Vector Regression (SVR) SVR is originally usedto solve linear regression problems For given training sam-ples X = (x119894 119910119894) | x119894 isin 119877119889 119910119894 isin 119877 119894 = 1 2 119899 fittingaims to find the dependency between the independent vari-able x119894 and the dependent variable 119910119894 Specifically it aims toidentify an optimal function and minimize prospective risk119877(120596) = int 119871(119910 119891(x120596))119889119875(x 119910) where 119891(x120596) is predictivefunction set 120596 isin Ω is the generalized parameters of thefunction 119871(119910 119891(x120596)) is the loss function and

119891 (x120596) = 120596 sdot x + 119887 (1)

is the fitting function [15] Thus the solution of the optimallinear function for SVR is expressed as the following con-straint optimization problem

min 119877 (120596 120585119894 120585lowast119894 ) = 12 ⟨120596 sdot 120596⟩ + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)

120596 sdot x minus 119910119894 + 119887 le 120576 + 120585lowast119894

st 119910119894 minus 120596 sdot x minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899

(2)

where the penalty coefficient C that determines the accuracyof the function fitting and the degree of the error greaterthan 120576 is given in advance Parameter 120576 is used to control thesize of the fitting error the size of the support vector andthe size of the generalization capability Taking into accountthe accuracy of the fitting error the introduction of slackvariables 120585119894 120585lowast119894 becomes necessary Figure 1 in reference [10]illustrates this linear fitting problem

However the previous solution is only for a linear regres-sion problem Nonlinear regression necessitates the kernelfunction in the SVR model [16] The kernel function can beexpressed as follows

K (x z) = ⟨120601 (x) sdot 120601 (z)⟩ (3)

where 120601 is the mapping from a low-dimensional spaceto a high-dimensional space The independent variable x119894becomes a vector that should be mapped to a feature space sothat a nonlinear problem could be changed into a linear prob-lem After introducing the kernel function the new fittingfunction becomes

119891 (x) = 119899sum119894=1

120596119894119896 (x xi) + 119887 = K (xX119879)120596 + 119887 (4)

where the symbol X119879 indicates the transpose of the matrixX

The changing of the fitting function leads to the followingconstraint optimization problem


(120585119894 + 120585lowast119894)

K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894

st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899

(5)

In this constraint optimization problem the length of 120596 andx is n The notion K(sdot sdot) is the kernel function that fulfillsMercerrsquos requirements

The standard SVR is a compromise between structuralrisk minimization and empirical risk minimization In par-ticular for the support vector regression learning algorithmthe structural risk term is (12)⟨120596 sdot 120596⟩ and the empirical riskitem is sum119899

119894=1(120585119894 + 120585lowast

119894) However calculating the structural risk

term (12)⟨120596 sdot120596⟩ requires enormous time and resources [17]Researchers found counting the minimization of the 1-normof the parameter 120596 will reduce the time and resources spent

Mathematical Problems in Engineering 3

on calculationThen the optimization formula turns into thefollowing form

min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(6)

Although the time and resource spent on modeling arereduced there is no considerable difference in the finalaccuracy

22 Support Vector Regression with Deviation Information asa Consideration Traditional SVR does not possess a specialmethod for addressing noise in sample data An efficient wayto weaken noise is to adjust parameters in the SVR modelThese parameters are called hyperparameters in SVRsHyperparameters exert a considerable impact on algorithmperformance The general way to test the performance ofhyperparameters is via the deviation between the model out-put and the sample data [18] The obtained deviation is thencompared with other deviations to select the minimum devi-ation as the final resultThe parameters that correspond to theminimum deviation are the best parameters in the optimiza-tion process Usually this process is conducted using an intel-ligent optimization algorithm such as particle swarm opti-mization (PSO) [19] and genetic algorithm (GA) [20] Thedeviation is set as the fitness function in an intelligent opti-mization algorithm In this section we refer to thismethod asdeviation-minimized SVR (DM-SVR)

In most of the circumstances the deviation betweenthe model output and sample data is represented by thecorrelation coefficient 119903 or the mean square error (MSE)Given vector y as the model output and vector y as thesample output the correlation coefficient r can be expressedas

119903= sum119873

119894=1119894119910119894 minus sum119873

119894=1119894sum119873119894=1

119910119894119873radic(sum119873119894=1

119894

2 minus (sum119873119894=1

119894)2 119873) (sum119873

119894=11199101198942 minus (sum119873

119894=1119910119894)2 119873)

(7)

The formula for mean square error (MSE) is as follows

MSE = sum119873119894=1

(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)

In short if the value of MSE is close to zero and the value of ris close to one that group of parameters will produce the bestperformance

3 Probability Distribution InformationWeighted Support Vector Regression

Although DM-SVR can reduce influence from the noise italso has some weaknesses The main disadvantage of thismethod is the time it spends on training There are manyparameters that need to be optimized in SVR If thereare extra parameters to optimize these works would makethe train process inefficient To solve the uncertainty oferror parameter 120576 we introduce the probability distributioninformation (PDI) into SVR and designate it as PDISVR

31 Probability Distribution of the Output The probabilitydistribution information is the same as the probability distri-bution function and describes the likelihood that the outputvalue of a continuous random variable is near a certain pointIntegrating the probability density function is the proper wayto calculate the probability value of the random variable inthat certain region From the sample data we could set thefrequency of output to appear as different values Then weset frequency as119873(y) where y is the output value vector Let119875(y) be the probability of the samplersquos output Therefore therelationship between 119875(y) and119873(y) can be expressed as

119875 (y) = 119873 (y)sumyisinY 119873(y) (9)

where Y is the range of y Then we can easily obtainthe probability distribution function The next step is theidentification of the probability of every point

32 Optimization Formula with Probability Distribution Infor-mation Weight Once we have obtained the probability dis-tribution of output it should be integrated into the basicSVR model In the basic SVR model the error parameter 120576indicates the accuracy of model fitting by providing an areathat does not have any loss for the objective function How-ever due to the influence of noise some sample data containexcessive noise information If the same parameters 120576 areadapted the performance of themodel is reduced To preventthis situation SVR should be adjusted in accordance withnoise informationWe propose illustrating noise informationthrough the probability distribution of the output Samples inthe regionswith low probability distributions have a relativelylarge proportion of noise For this reason in modeling theregion with higher probability should have a smaller errorparameter than the lower probability regionThus the proba-bility distribution function increases the accuracy of the SVRmodel in the area with the high probability of output

Define the 120576-insensitive loss function as

1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816120576 = 0 if 1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 le 1205761003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 minus 120576 otherwise (10)

where119891(x) is a regression estimation function constructed bylearning the sample and y is the target output value that cor-responds to x By defining the 120576-insensitive loss function theSVR model specifies the error requirement of the estimated


Feature space

0+

minus

Figure 1 Linear problem illustration of a PDISVR model

function on the sample dataThis requirement is the same forevery sample point To avoid this situation the artificially seterror parameter 120576 is divided by the probability distributionvector 119875(y) Figure 1 illustrates the change from a constant120576 to a vector 120576 The distance between two hyperplanes hasbeen modified in some area where the density of the pointsbecomes different Furthermore in the high-density areathe model has a smaller error parameter By contrast inthe low-density area the model has a large error parameterThe density of the output points is directly related to theprobability of the samplersquos output y Therefore the divisionof PDI would make the SVR model emphasize the area witha high density of points This technique can improve overallaccuracy despite sacrificing the accuracy of low-density areasAccording to (9) and (10) the PDISVR can be expressedas

min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)

K (xX119879)120596 minus 119910119894 + 119887 le 120576119875 (119910119894) + 120585lowast119894

st 119910119894 minus K (xX119879)120596 minus 119887 le 120576119875 (119910119894) + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899

(11)

By comparing (11) with the standard form of SVR we cansee that the error parameter 120576 changes in accordance with 119910119894Then the PDISVRmodel will have low error tolerance for thehigh density of points

To further improve the performance of the SVR modelwe consider adding an extra fragment to the PDISVR frame-work The PDISVR model only has a unique error parameter120576 However 120576 is too small to have an obvious impact onthe accuracy of the model Hence we propose an additionalmethod to introduce PDI that is we apply the same operationas that on error parameter 120576 on the slack variable 120585 Given thetreated error parameter 120576 in the PDISVR model we divided

Table 1 Typical parameters for PSO algorithms

Parameter ValueMaximum generation 200Population size 20Cognitive efficient (1199061) 15maximum velocity (119881max) 1Social efficient (1199062) 17Initial inertia weights 09Final inertia weights 04minimum velocity (119881min) 10

the slack variable 120585 by probability distribution information119875(y) Subsequently the final PDISVR model is

min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)

K (xX119879)120596 minus 119910119894 + 119887 le 1119875 (119910119894) (120576 + 120585lowast119894)

st 119910119894 minus K (xX119879)120596 minus 119887 le 1119875 (119910119894) (120576 + 120585119894)120585119894 120585lowast119894 ge 0 119894 = 1 2 119899

(12)

33 Parameters Optimization Based on PSO Normally theperformance of the different SVRs is intensively dependenton the parameters selection A PSO based hyperparametersselecting method in [10] is used in this paper After datasetwas normalized the control parameters of PSO includingmaximum velocity (119881max) minimum velocity (119881min) initialinertia weight (120596min) final inertia weight (120596max) cognitivecoefficient (1206011) social coefficient (1206012) maximum generationand population size should be initialized according to theexperience of operators In our experiment we set thecontrol parameters of PSO based on Table 1 In the followingexperiments there are at most five hyperparameters ofdifferent SVRs that need to be optimized in PSO Theseparameters include penalizing coefficient (C) radial basisfunction kernel parameter (120574) and polynomial degree (d)(120576) in 120576-insensitivity function and mixing coefficient (m) inmultikernel function In our experiments different compar-ative methods adapt different groups of parameters and inorder to search the global optimum reasonably the parameterm is limited in [0 1] 120574 in [10minus2 10] d in [1 10] C in[10 1000] and 120576 in [10minus4 10minus1] During the process ofsearching best parameters by PSO particles update theirpositions by changing velocity and converge finally at a globaloptimum within the searching space In this study the parti-clesrsquo positions are the combination ofm 120574 d C and 120576 whichare denominated as P Then V-fold (V = 5) cross-validationresampling method is applied to validate the performanceof searched best parameters until criterions are met Toevaluate the performance of training process mean squareerror (MSE) is chosen as the fitness function which isformulated as (8) Figure 2 shows the workflow to find theoptimum values of each parameter in models


Start

Prepare andnormalize dataset

Initialize control parameters of PSO including

generation population size

Set parameter P for SVR

V-fold cross-validationaccuracy

NoOr

Testing SVR based on theoptimized parameters

Yes

Predict result using well-trained SVR model

End

[VＧＣＨ VＧＲ] [ＧＣＨＧＲ] 1 2 maximum

Search best value m dC by PSO algorithm

Figure 2 Workflow on hyperparametersrsquo selection

4 Experimental Results

To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage

Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22

41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]

119891 (x) = 119899sum119894=1

[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2

minus ( 119899prod119894=1

119909119894)02

+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)

(13)

In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected

After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5

In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result


0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)

Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE

line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances

42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated

by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed

To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms

As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters

In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples


0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)

Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE

Table 2 Testing results of SVR algorithms with rough error (linear kernel)

Criterion PDISVR SVR HWSVR MKSVR119903

Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998

MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321

Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)




are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely

detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)


PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE

Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)




could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm

43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for

testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33

As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation


Table 5 Details of the experimental datasets

Data Number of instances Number of attributes Dataset information

Combustion side reaction 144 6The combustion side reaction is the process by which

the solvent and reactants are oxidized to carbondioxide carbon monoxide and water

Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption

Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters

Soil samples measured byNear-Infrared Spectroscopy(NIR)

108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)

Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy

efficiency) as a function of building parameters

Crude oil distillation process 30 1 Data related to different components with differentboiling points

Table 6 Comparative results of previous SVR models in real datasets (linear kernel)

Criterion PDISVR SVR HWSVR MKSVR

Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044

Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802

Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772

NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858

Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818

Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628

0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228

Figure 6 Weights of the training sample for PDISVR

through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the

0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054

Figure 7 Weights of the training sample for HWSVR

noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets


Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)



Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235


Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248

Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)



Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion

In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets

Conflicts of Interest

The authors declare that they have no conflicts of interest

References

[1] VN VapnikTheNature of Statistical LearningTheory Springer2000

[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016

[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001

[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002

[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006

[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999

[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002

[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011

[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector


regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014

[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016

[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008

[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015

[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008

[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014

[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016

[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017

[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015

[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017

[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009

[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014

[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008

[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012

[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013

[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013

[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012

[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007

Submit your manuscripts athttpswwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Journal of


Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of


methods are utilized in SVR [13] Our team previouslyworked on the monotonous a priori knowledge of sampledata Our monotonous a priori knowledge of the sample datais described by first-order difference inequality constraints ofkernel expansion and additive kernels [14] The constraintsare directly added to kernel formulation to acquire a convexoptimization problem For additive kernels SVMs are con-ducted through the addition of dissociate kernels for everyinput dimensionThese operations confer higher accuracy tothe SVR model in support vector (SV) selection

Inevitably even small noise can debase the accuracy ofthe model Furthermore in some situations part of the noisyinformation may be ten to even dozens of times larger thanthe normal data These outliers introduce bias and inac-curacies to SVR Nevertheless the probability distributionof the sample data is a good indicator of noise From theperspective of the probability distribution of sample datanormal data and data that contain the least amount of noisehave the highest probability in the sample data By contrastdata that contain large amount of noise have relatively smallprobability Thus outliers in the sample data will have thesmallest probability Therefore the probability distribution isthe prior knowledge that helps weaken the influence fromnoise and outliers in the sample data We consider thisinformation to modify our SVR model

This article is structured as follows Section 2 introducesstandard SVR algorithms Section 3 describes the proposedalgorithm that integrates probability distribution informa-tion into the SVR framework Section 4 provides someexperimental results that were obtained from comparing theproposed algorithm with other algorithms Finally Section 5presents some conclusions about the proposed algorithm

2 Review of SVR

To better describe the proposed algorism the mathematicalclarification of the basic concepts of SVR and the usage ofdeviation information should be provided

21 Support Vector Regression (SVR) SVR is originally usedto solve linear regression problems For given training sam-ples X = (x119894 119910119894) | x119894 isin 119877119889 119910119894 isin 119877 119894 = 1 2 119899 fittingaims to find the dependency between the independent vari-able x119894 and the dependent variable 119910119894 Specifically it aims toidentify an optimal function and minimize prospective risk119877(120596) = int 119871(119910 119891(x120596))119889119875(x 119910) where 119891(x120596) is predictivefunction set 120596 isin Ω is the generalized parameters of thefunction 119871(119910 119891(x120596)) is the loss function and

119891 (x120596) = 120596 sdot x + 119887 (1)

is the fitting function [15] Thus the solution of the optimallinear function for SVR is expressed as the following con-straint optimization problem


(120585119894 + 120585lowast119894)

120596 sdot x minus 119910119894 + 119887 le 120576 + 120585lowast119894

st 119910119894 minus 120596 sdot x minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899

(2)

where the penalty coefficient C that determines the accuracyof the function fitting and the degree of the error greaterthan 120576 is given in advance Parameter 120576 is used to control thesize of the fitting error the size of the support vector andthe size of the generalization capability Taking into accountthe accuracy of the fitting error the introduction of slackvariables 120585119894 120585lowast119894 becomes necessary Figure 1 in reference [10]illustrates this linear fitting problem

However the previous solution is only for a linear regres-sion problem Nonlinear regression necessitates the kernelfunction in the SVR model [16] The kernel function can beexpressed as follows

K (x z) = ⟨120601 (x) sdot 120601 (z)⟩ (3)

where 120601 is the mapping from a low-dimensional spaceto a high-dimensional space The independent variable x119894becomes a vector that should be mapped to a feature space sothat a nonlinear problem could be changed into a linear prob-lem After introducing the kernel function the new fittingfunction becomes

119891 (x) = 119899sum119894=1

120596119894119896 (x xi) + 119887 = K (xX119879)120596 + 119887 (4)

where the symbol X119879 indicates the transpose of the matrixX

The changing of the fitting function leads to the followingconstraint optimization problem


(120585119894 + 120585lowast119894)



(5)

In this constraint optimization problem the length of 120596 andx is n The notion K(sdot sdot) is the kernel function that fulfillsMercerrsquos requirements

The standard SVR is a compromise between structuralrisk minimization and empirical risk minimization In par-ticular for the support vector regression learning algorithmthe structural risk term is (12)⟨120596 sdot 120596⟩ and the empirical riskitem is sum119899

119894=1(120585119894 + 120585lowast

119894) However calculating the structural risk

term (12)⟨120596 sdot120596⟩ requires enormous time and resources [17]Researchers found counting the minimization of the 1-normof the parameter 120596 will reduce the time and resources spent



min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(6)




119903= sum119873

119894=1119894119910119894 minus sum119873

119894=1119894sum119873119894=1

119910119894119873radic(sum119873119894=1

119894

2 minus (sum119873119894=1

119894)2 119873) (sum119873

119894=11199101198942 minus (sum119873

119894=1119910119894)2 119873)

(7)


MSE = sum119873119894=1

(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)





119875 (y) = 119873 (y)sumyisinY 119873(y) (9)







Feature space

0+

minus



min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(11)






min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(12)



Start






NoOr


Yes


End








119891 (x) = 119899sum119894=1

[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2

minus ( 119899prod119894=1

119909119894)02

+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)

(13)





0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)









0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of






min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(6)




119903= sum119873

119894=1119894119910119894 minus sum119873

119894=1119894sum119873119894=1

119910119894119873radic(sum119873119894=1

119894

2 minus (sum119873119894=1

119894)2 119873) (sum119873

119894=11199101198942 minus (sum119873

119894=1119910119894)2 119873)

(7)


MSE = sum119873119894=1

(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)





119875 (y) = 119873 (y)sumyisinY 119873(y) (9)







Feature space

0+

minus



min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(11)






min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(12)



Start






NoOr


Yes


End








119891 (x) = 119899sum119894=1

[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2

minus ( 119899prod119894=1

119909119894)02

+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)

(13)





0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)









0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of





Feature space

0+

minus



min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(11)






min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1

(120585119894 + 120585lowast119894)



(12)



Start






NoOr


Yes


End








119891 (x) = 119899sum119894=1

[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2

minus ( 119899prod119894=1

119909119894)02

+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)

(13)





0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)









0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of





Start






NoOr


Yes


End








119891 (x) = 119899sum119894=1

[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2

minus ( 119899prod119894=1

119909119894)02

+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)

(13)





0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)









0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of





0 1 2 3 4 5 6 7 8 9 10045

05055

06065

07075

08085

09095

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

5

10

15

20

25

30

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(b)

0 1 2 3 4 5 6 7 8 9 100

001

002

003

004

005

006

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(c)

0 1 2 3 4 5 6 7 8 9 100

20406080

100120140160

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(d)









0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of





0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

PDISVRSVR

HWSVRMKSVR

(a)

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

Noise intensity

Stan

dard

dev

iatio

n

PDISVRSVR

HWSVRMKSVR

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005001

0015002

0025003

0035004

0045

Noise intensity

Aver

age v

alue

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

051

152

253

354

45

Noise intensity

Stan

dard

dev

iatio

n

(d)













PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of





PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 10

04

05

06

07

08

09

1

Noise intensity

Aver

age v

alue

(a)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 1002468

1012141618

Noise intensity

Aver

age v

alue

(b)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

0005

001

0015

002

0025

003

0035

Noise intensity

Stan

dard

dev

iatio

n

(c)

PDISVRSVR

HWSVRMKSVR

0 1 2 3 4 5 6 7 8 9 100

50

100

150

200

250

Noise intensity

Stan

dard

dev

iatio

n

(d)

























Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of



















Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802


NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 010531

X = 17Y = 0

X = 50

Y = 000036228



0 10 20 30 40 50 60 70 800

01

02

03

04

05

06

07

08

09

1

Sample number

Wei

ght o

f dat

a

X = 48Y = 027292

X = 17Y = 031429

X = 50

Y = 0075054







Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of








Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482


NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235






Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559


NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073



5 Conclusion




References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of






























Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of











Volume 2014




Journal of











Journal of


Function Spaces






Algebra





Journal of




Documents

Probability Distribution and Deviation Information Fusion ...downloads.hindawi.com/journals/mpe/2017/9650769.pdf2 MathematicalProblemsinEngineering methods are utilized in SVR [13]