Energy Storage and Renewables in New Jersey: Complementary Technologies for Reducing

Our Carbon Footprint

ACEE E-filliates workshop November 14, 2014

Warren B. Powell Daniel Steingart Harvey Cheng Greg Davies

© 2014 Warren B. Powell, Princeton University

Frequency regulation PJM sends charge/discharge signals to generators every 2 seconds to smooth out frequency/voltage variations

1 hour

MW

Solar from PSE&G solar farms Solar from a single solar farm

Solar energy from a single solar farm 1 week

Energy from wind

1 year

Wind power from all PJM wind farms

Jan Feb March April May June July Aug Sept Oct Nov Dec

Energy from wind

30 days

Wind from all PJM wind farms

Wind energy in PJM Total load vs. total current wind (January)

Winter load and solar Total PJM load plus factored solar (January)

Wind energy in PJM Total PJM load plus actual wind (July)

Summer load and wind Total PJM load plus actual wind (July)

99.9 percent from renewables!

Fossil backup

Battery storage

Wind & solar

}20 GW

Lead-acid or lithium-ion

Ultracapacitors

Research challenges How do we control a battery storage system? Challenges include: » Managing a single storage device to handle multiple revenue

streams, over multiple time scales » Controlling a storage system in the presence of a multidimensional

“state of the world” » Controlling dozens to hundreds of storage devices spread around

the grid.

How do we design storage systems? » What type of storage device(s)? » How many are needed? » How should they be distributed across the grid?

How does storage change the economics of renewables?

Revenue streams Frequency regulation Power quality management Battery arbitrage Energy shifting Demand peak management - Many utilities impose charges based on peak usage over a month, quarter or even a year. Peak management for avoiding capacity expansion Backup power for outages

seconds

minutes

hours

days-weeks

weeks-months

months-years

Research goals To design an algorithm that produces near-optimal policies that handle the following problem characteristics: » Responds to predictable time-dependent structural patterns over

hourly, daily and weekly cycles in generation and loads. » Able to simultaneously optimize over multiple revenue streams,

balanced against maximizing the lifetime of the battery. » Able to handle time scales ranging from seconds to minutes, hours

and days. » Handles uncertainty in energy generation, prices and loads. » Handles “state of the world” variables such as weather conditions,

network conditions and prices. » For some applications, we need to scale to large numbers (tens to

hundreds, but perhaps thousands) of grid-level storage devices. » Ability to incorporate forecasts of wind or solar energy, loads, and

weather. » Needs to be computationally very fast.

A storage problem Energy storage with stochastic prices, supplies and demands.

windtE

gridtP

tD

batterytR

1 1

1 1

1 1

1

ˆ

ˆ

ˆ

wind wind windt t t

grid grid gridt t t

load load loadt t t

battery batteryt t t

E E E

P P P

D D D

R R x

+ +

+ +

+ +

+

= +

= +

= +

= +

1 Exogenous inputstW + =State variabletS =

Controllable inputstx =

A storage problem Bellman’s optimality equation

( )1 1( ) min ( , ) ( ( , , )tt x t t t t t tV S C S x V S S x Wγ∈ + += +X E

windtgrid

tloadtbatteryt

E

P

D

R

wind batterytwind loadtgrid batterytgrid loadtbattery loadt

x

x

x

x

x

−

−

−

−

−

1

1

1

ˆ

ˆ

ˆ

windt

gridt

loadt

E

P

D

+

+

+

Managing a water reservoir Backward dynamic programming in one dimension

1 1 1Step 0: Initialize ( ) 0 for 0,1,...,100Step 1: Step backward , 1, 2,... Step 2: Loop over 0,1,...,100 Step 3: Loop over all decisions 0 Step 4:

T T T

t

t t

V R Rt T T T

Rx R

+ + += == − −=

≤ ≤

{ }100

max1

0

Take the expectation over all rainfall levels (also discretized):

Compute ( , ) ( , ) (min , ) ( )

End step 4; End Step 3;

Wt t t t t t

wQ R x C R x V R R x w P w+

=

= + − +∑

*

* Find ( ) max ( , )

Store ( ) arg max ( , ). (This is our policy)

End Step 2;End Step 1;

t

t

t t x t t

t t x t t

V R Q R x

X R Q R xπ

=

=

Managing cash in a mutual fund

( )

1 1Step 0: Initialize ( ) 0 for all states.Step 1: Step backward , 1, 2,... Step 2: Loop over = , , , (four loops) Step 3: Loop over all decisions (a problem if i

T T

t t t t t

t t

V St T T T

S R D p Ex x

+ + == − −

( )

1 1 1

s a vector)ˆ ˆˆ Step 4: Take the expectation over each random dimension , ,

Compute ( , ) ( , )

, , ( ,

t t t

t t t t

Mt t t t

D p E

Q S x C S x

V S S x W w+ +

= +

=( )( )1 2 3

*

100 100 100

2 3 1 2 30 0 0

*

, ) ( , , )

End step 4; End Step 3; Find ( ) max ( , )

Store ( ) arg max ( , ). (This is our policy)

End St

t

t

W

w w w

t t x t t

t t x t t

w w P w w w

V S Q S x

X S Q S xπ

= = =

=

=

∑ ∑ ∑

ep 2;End Step 1;

Dynamic programming in multiple dimensions

Approximate dynamic programming Bellman’s optimality equation » We approximate the value of energy in storage:

( )( )( ) min ( , ) ( , )t

x xt t x t t t t t tV S C S x V S S xγ∈= +X

Inventory held over from previous time period

Approximate dynamic programming We update the piecewise linear value functions by computing estimates of slopes using a backward pass: » The cost along the marginal path is the derivative of the simulation

with respect to the flow perturbation.

Rδ

Approximate dynamic programming Testing on a deterministic problem demonstrates that we can precisely capture optimal time-dependent behavior:

Approximate dynamic programming Benchmarking against optimality on a stochastic model

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Perc

ent o

f opt

imal

Pe

rcen

t of o

ptim

al

Storage problem

Approximate dynamic programming Benchmarking against optimality on a stochastic model

95

96

97

98

99

100

101

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Perc

ent o

f opt

imal

Pe

rcen

t of o

ptim

al

Storage problem

Slide 25

Grid level storage

Slide 26

Grid level storage control

Slide 27

Grid level storage control Monday

Time :05 :10 :15 :20

Slide 28

Grid level storage control Monday

Time :05 :10 :15 :20

Slide 29

Grid level storage control Monday

Time :05 :10 :15 :20

Grid level storage control Approximate dynamic programming (blue) vs. optimal using linear programming (green)

Heterogeneous fleets of batteries

Heterogeneous fleets of batteries A tale of two batteries » Ultracapacitor – High power, high efficiency, low capacity » Lead acid – Lower power, lower efficiency, high capacity

Lead acid

Ultra capacitor

Heterogeneous fleets of batteries Control algorithm adapts to characteristics of each storage device

Time (hours) energy is held in storage device

Cum

ulat

ive

dist

ribut

ion

Handling multiple time scales

0 . . . 1 2 3 3 4 23 24 Daily (hourly increments)

0 5 10 15 20 55 60 . . . Hourly (5-min. increments) ( )1( ) min ( , ) (

tt x t t tV S C S x V Sγ∈ += +X E

( )1( ) min ( , ) (tt x t t tV S C S x V Sγ∈ += +X E

Handling multiple time scales

0 . . . 1 2 3 3 4 23 24

0 5 10 15 20 55 60 . . .

Daily (hourly increments)

Hourly (5-min. increments)

( )1( ) min ( , ) (tt x t t tV S C S x V Sγ∈ += +X E

( )1( ) min ( , ) (tt x t t tV S C S x V Sγ∈ += +X E

Handling multiple time scales

0 . . . 1 2 3 3 4 23 24

0 5 10 15 20 55 60 . . .

Daily (hourly increments)

Hourly (5-min. increments)

There are 43,200 2-second increments in a day, over 300,000 in a week.

( )1( ) min ( , ) (tt x t t tV S C S x V Sγ∈ += +X E

0 2 4 6 8 298 300 . . . 5 min (2-sec. increments) ( )1( ) min ( , ) (

tt x t t tV S C S x V Sγ∈ += +X E

( )1( ) min ( , ) (tt x t t tV S C S x V Sγ∈ += +X E

Solar energy Princeton solar array

What is the value of storage in managing the variability from renewables?

Our model, “SMART-Storage” simultaneously optimizes the ramping of generators as well as storage.

Solar-storage experiments

Load

Storage level

Solar intensity LMP

Sunny day

Cloudy day

Mixed sun/cloud

Mixed sun/cloud

Solar

Energy from storage

Solar = 2.3GW Storage = 12Gwh/600MW

Solar = 23GW Storage = 12Gwh/600-MW

Nuclear Steam

CC Gas turbine Pumped storage

Sunn

y da

y C

loud

y da

y

Solar-storage experiments

23MW 230MW 2.3GW 11.5GW 23GW

23MW 230MW 2.3GW 11.5GW 23GW

Solar capacity

Solar capacity

Load covered by solar %

% of solar used

Solar-storage experiments Some conclusions: » The model will only put energy in storage when storage is the only

way to meet fast variations in generation and loads. » The reason is the losses that are incurred when converting energy

is stored. It is always better to ramp down a generator during periods of high energy generation from wind or solar, than to store the energy and use it better.

» The idea that the conversion losses do not matter when the energy is free is a myth…. It only applies when the total generation from renewables exceeds the total load (which was never the case in our experiments).

Research goals To design an algorithm that produces near-optimal policies that handle the following problem characteristics: » Responds to predictable time-dependent structural patterns over

hourly, daily and weekly cycles in generation and loads. » Able to simultaneously optimize over multiple revenue streams,

balanced against maximizing the lifetime of the battery. » Able to handle time scales ranging from seconds to minutes, hours

and days. » Handles uncertainty in energy generation, prices and loads. » Handles “state of the world” variables such as weather conditions,

network conditions and prices. » For some applications, we need to scale to large numbers (tens to

hundreds, but perhaps thousands) of grid-level storage devices. » Ability to incorporate forecasts of wind or solar energy, loads, and

weather. » Needs to be computationally very fast.

Thank you!

For more information see:

http://energysystems.princeton.edu