prev

next

out of 58

View

273Download

0

Embed Size (px)

Approximate Dynamic Programming Methods forResidential Water Heating

A thesis submitted in partial fulfillment for the degree of Masters of Sciencein the

Department of Electrical Engineering

byMatthew Motoki

December 3, 2015

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Motivation

Why do we need a smarter water heater?

I Energy efficiency is important.

Electricity is expensive. Burning fossil fuels is bad for the environment.

I Can we do better than water heaters with an adjustable set-point?

If so, then are there any provable guarantees that can be made? Theoretically, what is best that we can do?

I The legacy grid is becoming obsolete.

Renewable energy sources are variable and distributed. Energy storage capabilities of water heaters have been fully exploited.

1 / 31

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Problem Formulation

State VariableDefine t {0,t, . . . , (N 1)t}. Define tk = mod(k ,N)t, where k = 0, 1, . . . isthe simulation time stage.

The state x := (T , h) summarizes the information needed to make a decision.

We require T [Tamb,Tmax ]. The temperature at tk is written Tk .The hot water usage history is hk := {(ti ,wi ) | 0 i < mod(k ,N)}, where wk is theintensity of the hot water draw at time tk .

2 / 31

Problem Formulation

Decision VariableThe decision variable is

uk :=

{1, if the water heater is on

0, if the water heater is off.

We assume that the decision uk is constant during the interval [tk , tk+1). A feasibledecision uk u, is one that does not violate T [Tamb,Tmax ]. A policy is amapping from a state into a feasible decision.

3 / 31

Problem Formulation

Hot Water Demand (Disturbance Variable) 1

We model hot water demand as a cyclostationary random process W(t) given by

W(t) := specific heat

Npeoplei=1

Nij=1

F (j),i (T (j),i Tamb

) I{S(j),i t < S

(j),i +D

(j),i

},

where := {shower , bath, . . . , dishwasher} is the set of possible usage events, Npeopleis the number of people in a household, Ni is the number of events of type corresponding to the i th person in the household, and the following are randomvariables:

S(j),i := the start time of E(j),i ,

D(j),i := the duration of E(j),i ,

F (j),i := the flow rate of E(j),i ,

T (j),i := the desired temperature of E(j),i .

4 / 31

Problem Formulation

Hot Water Demand (Disturbance Variable) 2

We can only observe W(t) at pre-specified times t t , therefore, we approximateW(t) using a piecewise linear interpolation

W(t) :=W(tk) +t tk

t[W(tk + t)W(tk)].

for all k = 0, 1, . . . and t [tk , tk + t). The discrete-time analog of W(t) to be theaverage of W(t) over t [tk , tk + t),

Wk :=1

t

tk+ttk

W(t) dt = 12 [W(tk) +W(tk + t)].

We denote particular realizations of W(t) and Wk using w(t) and wk , respectively.We discretize wk {0,w , . . . ,wmax}. We write the conditional probability massfunction of Wk given hk as pWk (wk | hk).

5 / 31

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Problem Formulation

State Equation

The state equation maps the current state xk , current decision uk , and currentdisturbance wk into the next state xk+1 according to

xk+1 = f (xk , uk ,wk) :=(fT (Tk , uk ,wk), fh(tk , hk ,wk)

),

where

Tk+1 = fT (Tk , uk ,wk) := max{Tk rcoolt (Tk Tamb)+ rheatt uk rlosst wk , Tamb

}hk+1 = fh(hk ,wk) :=

{{(tk ,wk)} hk , tk 6= (N 1)t, otherwise,

for all k = 0, 1, . . .

6 / 31

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Problem Formulation

Objective Function

The objective is to minimize over all policies , the following function

J(x0) = limK

EW

[1

K

K1k=0

g(Xk , (Xk),Wk ;

) x0],

= limK

1

K

K1k=0

EW0,W1,...,Wk ,

[g(Xk , (Xk),Wk ;

) x0].where X0 = x0 is given and Xk = f

(Xk1, (Xk1),Wk1

), for all k = 1, 2, . . .

7 / 31

Problem Formulation

Stage Cost

The stage cost is

g (xk , uk ,wk ; ) := gdiscomfort (xk , uk ,wk ;Tmin) + (1 ) goperating (xk , uk) ,

where := {,Tmin} is a customer-defined parameter set, [0, 1] is the relativeweighting of the objectives, and Tmin is the minimum desirable temperature during ahot water use.

Operating Cost

The operating cost is

goperating (uk) :=1

t

tk+ttk

C (t) rating uk dt,

where C (t) is the cost of power and rating is the power rating of the water heater.8 / 31

Problem Formulation

Discomfort CostThe discomfort cost is

gdiscomfort(xk , uk ,wk ;Tmin

):=

1

t

tk+ttk

max{Tmin T (t), 0

} I{w(t) > 0} dt,

where

T (t) := Tk +t tk

t[fT (Tk , uk ,wk) Tk ],

for all k = 0, 1, . . .

9 / 31

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Problem Formulation

Problem StatementFind a feasible on/off policy that minimizes an expected objective cost.

minimize

limK

EW

[1

K

K1k=0

g(Xk , (Xk),Wk ;

) x0]

subject to Xk+1 = f(Xk , (Xk),Wk

), (xk) {0, 1},

Tk [Tamb,Tmax ], for all k = 0, 1, . . .This is a discrete-time, average cost periodic Markov decision problem (MDP).

10 / 31

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Outline

1 Motivation

2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

Prescient Lower Bound (PLB)

5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

6 Conclusion

Methodology

Finite-Horizon Dynamic Programming

The goal is to minimize over all policies , the following function

J(x0) = EW

[gterminal(XM) +

M1k=0

g(Xk , k(Xk),Wk ;

) x0],

where M is the horizon and gterminal is a terminal cost function.

The optimal policy is the minimizer of Bellmans equations

J(xM) = g

Recommended

View more >