Recurrent Neural Networks for Energy Forecasting

Recurrent Neural Networks for Energy ForecastingNikil Pancha1, Richard Povinelli2

Case Western Reserve University1, Marquette EECE2

Problem Statement• Improve hourly gas and electricity

demand predictions using deep learning

ReferencesMarino, D. L., Amarasinghe, K., & Manic, M. (2016, October). Building energy load forecasting using deep

neural networks. In Industrial Electronics Society, IECON 2016-42nd Annual Conference of the IEEE (pp.

7046-7051). IEEE.

Gers, F. A., Schmidhuber, J., & Cummins, F. (1999). Learning to forget: Continual prediction with LSTM.

Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances

in neural information processing systems (pp. 3104-3112).

Bianchi, F. M., Maiorino, E., Kampffmeyer, M. C., Rizzi, A., & Jenssen, R. (2017). An overview and comparative

analysis of Recurrent Neural Networks for Short Term Load Forecasting. arXiv preprint arXiv:1705.04378.

Why Deep Learning?• No feature engineering

• High model capacities

• Effective at a variety of tasks (computer vision, robotics, etc.)

• Highly nonlinear

Long Short-term Memory (LSTM)• Used to process sequences

• Use previous output, previous state, and current input to predict current output

• Commonly used in natural language processing tasks

• Operated either autoregressively (AR) or sequence to sequence (seq2seq)

Sequence to Sequence• Two separate LSTMs: Encoder and Decoder

• Encoder determines a dense representation of past flow and weather

• Decoder translates encoder output and future weather information to predicted flow

• Most common architecture for machine translation models

AcknowledgementsI would like to thank the NSF for funding this research (NSF Award ACI-1461264), Dr. Factor,

Dr. Dennis Brylow, and Dr. Petra Brylow for running this REU, and the GasDay lab for their

support

Results• Seq2seq performs better than AR at most

horizons

• AR prediction accuracy quickly declines as prediction horizon increases

• Adding an encoder on daily data improves the basic seq2seq model on short horizons

• Regularization is necessary, but some forms are too expensive to apply.

Model WMAPE MAPE MSE

AR 3.926 5.196 3.144

Seq2seq, no daily 3.5886 4.974 2.385

Seq2seq, with daily 3.5404 5.087 2.262

Average Results over 48 hours

Diagram by Eddieantoniooooo, distributed under a CC BY-SA 4.0 license

Recurrent Neural Networks for Energy Forecasting

Documents

Tutorial Training Recurrent Neural Networks

Recurrent Neural Network for Language Modeling Taskfraser/smt_nmt_2016_seminar/08_rnns.pdf · Recurrent Neural Network for Language Modeling Task ... Recurrent Neural Network Language

Deep Multi-State Dynamic Recurrent Neural Networks ...papers.nips.cc/paper/9594-deep-multi-state-dynamic-recurrent-neural... · Deep Multi-State Dynamic Recurrent Neural Networks

Fractional Integrated Recurrent Neural Network (FIRNN) for …€¦ · Fractional Integrated Recurrent Neural Network (FIRNN) for forecasting 1537 patterns with a sequential distortion

Jacek Mazurkiewicz, PhD Softcomputing · Softcomputing Part 3: Recurrent Artificial Neural Networks Self-Organising Artificial Neural Networks . Recurrent Artificial Neural Networks

Using Recurrent Neural Networks to Forecasting of Forex

Deep Q-Learning with Recurrent Neural Networkscs229.stanford.edu/...DeepQLearningWithRecurrentNeuralNetwords... · Deep Q-Learning with Recurrent Neural Networks ... rent neural network

Lecture 6: Recurrent & Graph Neural Networks · 2019. 12. 14. · UVA DEEP LEARNING COURSE –EFSTRATIOS GAVVES RECURRENT NEURAL NETWORKS - 1 Lecture 6: Recurrent & Graph Neural Networks

순환신경망(Recurrent neural networks) 개요

Electricity price forecasting with Recurrent Neural Networks

STOCK MARKET FORECASTING USING RECURRENT NEURAL NETWORK

Lecture 6: Recurrent & Graph Neural NetworksLecture 6: Recurrent & Graph Neural Networks Efstratios Gavves. UVA DEEP LEARNING COURSE –EFSTRATIOS GAVVES RECURRENT NEURAL NETWORKS

Lecture 6: Recurrent Neural Networks - GitHub Pages · Lecture 6: Recurrent Neural Networks Efstratios Gavves. UVA DEEP LEARNING COURSE –EFSTRATIOS GAVVES RECURRENT NEURAL NETWORKS

Forecasting with Recurrent Neural Networks: 12 Tricks · 2 In Section 2 we introduce the so-called correspondence principle for neural networks (Trick 1). For us neural networks are

Medium-term Wind Power Forecasting via Recurrent Neural

Intelligent Forecasting in Multi-Criteria Decision-Makingceur-ws.org/Vol-2608/paper72.pdf · 2020. 5. 19. · convolutional neural networks, recurrent neural networks, decision-making,

Recurrent Neural Network tutorial (2nd)

Pixel Recurrent Neural Networks

Recurrent Neural Networks · Recurrent Neural Networks LING572 Advanced Statistical Methods for NLP March 5 2020 1. Outline Word representations and MLPs for NLP tasks Recurrent neural

Electroencephalogram Classification by Forecasting with ... › eeg › publications › forney_thesis.pdf · ELECTROENCEPHALOGRAM CLASSIFICATION BY FORECASTING WITH RECURRENT NEURAL