A Hybrid Method of CART and Artificial Neural Network for Short-term Load Forecasting

in Power SystemsHiroyuki Mori

Dept. of Electronics EngineeringMeiji University

Tama-ku, Kawasaki 214-8571Japan

hmori@isc.meiji.ac.jp

I. Objective

II. Background

III. Case Studies in CEPCO◦ Proposed Method◦ Simulation

IV. Conclusion

Outline

To construct an Intelligent Model for Short-term Load Forecasting

Input data (Rules, Knowledge, Feature Extraction) Output Data

Cause and Effect of Input and Output Data

I. Objective

Nonlinear Large-

Scale

Dynamical

DiscreteStochast

ic

Random-Like

Quasi Periodical

Time-Varian Para metered

2.1 Complexity of Power Systems

Deregulation

Competition

Distributed Generation

Power

networks

(Maximizing Profit while Minimizing

Risk) Uncertainty

2.2 Recent Complexity

Expert Systems (1980~)

Artificial Neural Net (1987~)

Fuzzy Inference (1990~)

Evolutionary Computation or Meta-Heuristics (1990~)

Multi-Agent Systems (1995~)

Data Mining (2000~)

2.3 Trends on Intelligent Systems in IEEE Power Engineering Society

2.4 Intelligent Systems Expert Systems

ANN

Fuzzy

Meta-heuristics

Multi Agent

Data Mining

Inference

Learning

Classification

Optimization

Distributed Systems

Knowledge Discovery

To Understand Complicated Data with Some Rules

To Extract Important Features That are Known and/or Unknown

To Construct More Reasonable Models/Strategies

2.5 Roles of Data Mining

Load Forecasting

Dynamic Security Assessment

Power System Control Center

Data Profiling of Customers, etc.

2.6 What Is Data Mining Used for in Power Systems?

System Operators

III. Case Studies in CEPCO

To Propose a Hybrid Method of Regression Tree and ANN for Short-term Load Forecasting in Electric Power Systems

To Optimize the Structure of the Regression Tree with TS Globally

To Extract Some Simple Rules From Data Set, i.e., Explain the Relationship between Input and Output Data

3.1 Objective

Japan Sea

CEPCO: Chubu Electric Power Co.

Osaka, Nagoya, Tokyo

Pacific Beach

3.2 Where is CEPCO?

Large Factories (Toyota Gr.)

High Humidity

3.3 Regional Features

Kalman Filtering (Toyoda, ‘70) Regression Model (Asbury, ‘75) ARIMA Model (Hagan, ‘77) Expert System (Rahman,’88) ANN (El-Sharkawi, ’91) Fuzzy Decision Model (Park, ’91) Fuzzy Neural Net (Mori, ‘94) Simplified Fuzzy Inference (Mori, ‘96) Chaos Time Series Analysis (Mori, ‘96) DM-Based Approach (Mori, 2001)

3.4 Conventional Methods on SLF

Regression Model

ANN Model

Neuro-Fuzzy Model

Fuzzy Inference Model

3.5 Experience of CEPCO on SLF Model

To enhance the Model Accuracy

◦ To Minimize the Maximum Errors

To clarify the Relationship between Input and Output Variables

◦ To Validate Their Own Rules

◦ To Find out New Rules

3.5 Requirements of Operators for SLF Models

To Play a Key Role in Power System Operation and Planning

◦ To Smooth ELD and UC

◦ To Make Profit through Deregulated and Competitive Power Markets

◦ (insert equation)

3.6 Short-term Load Forecasting (SLF)

Learning Data Fuzzy ANN y

Learning Data Preprocessor Predictor y

Prediction Model with Preprocessing Technique

3.8 Hint of Prediction Method

Learning Data

Classifier

Cluster 1

Cluster 2

Classifier

Cluster 3

3.9 Proposed Method 1

ANN1y ANN2y AnnMy

Regression Tree as a DM Tools (To Find Out Important Rules)

Open Issue

To Focus on Globally Optimal Classification Rather Than Locally Optimal or Locally Quasi-optimal One

3.10 Classification as Preprocessor

Data Mining

◦ To Discover Important Rules in Large Data Base

Data Mining

◦ Pattern Recognition

◦ Fuzzy Theory

◦ Decision Tree, etc.

◦ (insert cahrt of Split and Root Node)

3.10 Outline of DM

Growth◦ Minimization of Error after Splitting◦ R(n)=V(n)/V0◦ R(n):Error of Node n◦ V(n): Variance of Learning Data Belonging to Node

n◦ V0: Variance of All Learning Data

Pruning◦ Simple Structure of Regression Tree

Error Estimate◦ Cross-Validation Method

Procedures of Regression Tree

△R(s,t)=R(t)-R(tL)-R(tR)

Where, R(s,t): Degree of Error Reduction in Case where Attribute s at node t, s: Attribute, t: Parent Node, R(t): Sum of Squared Error of Parent Node, R(tL(r)): Error of Left-Side (Right-Side) Child Node

(Insert Chart)

Constructing the Tree

(insert equation)

Where, r: Error, rcv(*): Cross-Validation error, Standard Deviation of Cross-Validation Error, Pruned Tree Number

Pruning

Decision Tree Output Conventional Methods

Classification Qualitative CART, ID3, C4.5

Regression Quantitative CART

Table 1 Difference between Classification and Regression Trees

Drawback of Regression Trees- Classification Accuracy

(Locally Optimal Structure)

Methods Decision Tree Applications Model Structure

Wehenkel, ‘94[A1]

Classification Transient Stability

Local

Rovnyak, ‘94[A2]

Classification Transient Stability

Local

Proposed Regression Load Forecasting

Global

Table 2 Difference between Conventional and Proposed Decision Trees

[A1] Wehenkel, et. Al., “Decision Tree Based Transient Stability Method a Case Study,” IEEE Trans. on Power Systems, Vol. 9, No. 1, pp. 459-469, Feb. 1994[A2] Rovnyak, et. Al., “Decision Tree for Real-time Transient Stability Prediction,” IEEE Trans. On Power Systems, Vol. 9, No. 3, pp. 1417-1426, Aug. 1994.

Definition ◦ Iterative Methods That Have Some Heuristics or

Simple Rules in Search Process

Feature◦ To Aim at Evaluating Highly Accurate Solutions

Typical Meta-Heuristic Methods◦ SA, GA & TS

3.11 Meta-Heuristics

Methods Analogies

Parameters

Solution Accuracy

CPU-Time

Probability

SA Annealing -cooling schedule-temperature

Less Slower X

GA Natural Selection

-population-crossover-mutation

Less Slow X

TS Adaptive Memory

-tabu length

More Fast

Table 3 Comparison of Meta-Heuristic Methods

Adaptive Memory (Tabu List)

Only One Parameter (Tabu Length)

No Use of Random Numbers

Transition Type Algorithm

3.12 Tabu Search (TS)

(insert image)

(a) neighborhood search◦ Red (Fixed Attribute): Blue (Free Attribute) Tabu

List (b) Tabu List

Fig. 15 Concept of Tabu Search

To Construct the Regression Tree with the Globally Optimal Structure

To Combine the Optimal Regression Tree with MLP

Optimal Regression Tree◦ To Assign Input Variables to Split Nodes

◦ To Globally Optimize Combinations of Input Variables with TS

3.13 Proposed Method 2

(insert image)

(a) phase 1, (b) phase 2

V(a), V(b), V(c): Input Variables Used as Split Conditions

Fig. 16 Constructing Process by CART

Locally Optimal Structure

(insert image)

(a) phase 1, (b) phase 2

Fig. 17 Constructing Process of Proposed Regression Tree

Locally Optimal Structure

TS Solution: Splitting Attribute

Cost Function: (insert equation)

(insert graph)

Fig. 18 Transfer of Splitting Attribute to TS Solution

Constructing Tree Structure with TS

Fig. 19 Flowchart of Proposed Regression Tree

Start Set Initial Conditions Generate New Solutions (Combinations of Input Variables) Evaluate Cross-Validation Errors of New Solutions (Calculate Split Value?) (Pruning) Select Best Solution Terminated? Stop

Target: One-Step- ahead Daily Maximum Load Forecasting Learning Data: Summer Weekdays in June to September ‘91-’98

(Except ‘93 for Unusual Weather Conditions) Test Data: Summer Weekdays in June to September ‘99 Size of Initial Tree: 31 Splitting Nodes Tabu Length: 12 Conventional methods: CART-MLP and MLP Table 4 Parameters of MLP

3.13 Simulation

Method Learning Rate

Momentum Term

Iterations

Hidden Unit

Proposed Method

0.01 0.6 10000 5

CART-MLP

0.02 0.1 10000 5

MLP 0.9 0.5 30000 5

No. Input Variables

A Day of the Week d+1

B Predicted Max Temperature

C Predicted Min Temperature

D Predicted Average Temperature

E Predicted Min Humidity

F Predicted Discomfort Index

G Max Load day d

H Dif between max load on days d and d-1

I Dif between avg temp

J Avg of max load

k Avg of avg temp

Table 5 Eleven Input Variables

(Insert graphs)

Fig. 20 Comparison of Errors for Proposed and Conventional Methods

(insert decision tree)

Fig. 21 Example of Split Conditions Close to Root Node

(Insert decision tree)

Fig. 22 Optimal Regression Tree

(Insert decision tree)

Fig. 22 Regression Tree of CART

N(t) Rule

4 T(AV,d+1)> 28.05 CL(md)> 0.845

Note L(md): Max Load on Day d

Table 6 Rule Assigned to Terminal Node 4

Methods Tree (sec) MLP (sec)

Proposed 17760 2.7

CART-MLP 7 4.3

MLP 27.6

Table 7 Computational Time of Each Method

Computer: FUJITSU S-7/7000U Model 45SPECint_rate 95:422 (296MHz)SPECfp_rate 95:561 (296MHz)

(insert graph)

Fig. 24 Comparison of Regression Tree of the Proposed Method and CART-MLP

1. This paper has proposed a Hybrid Method of the optimal regression tree and MLP for short-term load forecasting

2. Tabu Search is used to globally optimize the model structure of the regression tree

3. The simulation results have shown that the proposed method is more effective than CART-MLP in terms of the average and the maximum errors

4. The proposed method allows to clarify the relationship between input and output variables through the systematic rules

V. Conclusion

H. Mori and N. Kosemura, “Optimal Regression Tree Based Rule Discovery for Short-term Load Forecasting,” Proc. Of 2001 IEEE PES Winter Meeting, Vol. 2, pp.421-426, Columbus, USA, Jan. 2001

H. Mori, N. Kosemura, K. Ishiguro and T. kondo, “Short-term Load Forecasting with Fuzzy Regression Tree in Power Systems,” Proc. Of 2001 IEEE International Conference on Systems, Man & Cybernetics, pp. 1948-1953, Tuscon, AZ, U.S.A, Oct. 2001

H. Mori, N. Kosemura, T. Kondo and K. Numa, “Data Mining for Short-term Load Forecasting,” Proc. Of 2002 IEEE PES Winter Meeting, Vol. 1, pp.623-624, New York, NY, USA, Jan. 2002

H. Mori and Y. Sakatani, “An Integrated Method of Fuzzy Data Mining and Fuzzy Inference for Short-term load forecasting,” Proc. Of ISAP (CD-ROM), Limnos, Greece, Aug. 2003

H. Mori, Y. Sakatani, T. Fujino and K. Numa, “An Efficient Hybrid Method of Regression Tree and Fuzzy Inference for Short-term Load Forecasting in Electric Power Systems, “A..Lofti and M.J. Garibaldi (Eds.), “Applications and Science in Soft Computing,” pp.287-294, Springer, Berlin, Germany, Nov. 2003

References