Master's Thesis Slides

Embed Size (px)

Text of Master's Thesis Slides

  • Approximate Dynamic Programming Methods forResidential Water Heating

    A thesis submitted in partial fulfillment for the degree of Masters of Sciencein the

    Department of Electrical Engineering

    byMatthew Motoki

    December 3, 2015

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Motivation

    Why do we need a smarter water heater?

    I Energy efficiency is important.

    Electricity is expensive. Burning fossil fuels is bad for the environment.

    I Can we do better than water heaters with an adjustable set-point?

    If so, then are there any provable guarantees that can be made? Theoretically, what is best that we can do?

    I The legacy grid is becoming obsolete.

    Renewable energy sources are variable and distributed. Energy storage capabilities of water heaters have been fully exploited.

    1 / 31

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Problem Formulation

    State VariableDefine t {0,t, . . . , (N 1)t}. Define tk = mod(k ,N)t, where k = 0, 1, . . . isthe simulation time stage.

    The state x := (T , h) summarizes the information needed to make a decision.

    We require T [Tamb,Tmax ]. The temperature at tk is written Tk .The hot water usage history is hk := {(ti ,wi ) | 0 i < mod(k ,N)}, where wk is theintensity of the hot water draw at time tk .

    2 / 31

  • Problem Formulation

    Decision VariableThe decision variable is

    uk :=

    {1, if the water heater is on

    0, if the water heater is off.

    We assume that the decision uk is constant during the interval [tk , tk+1). A feasibledecision uk u, is one that does not violate T [Tamb,Tmax ]. A policy is amapping from a state into a feasible decision.

    3 / 31

  • Problem Formulation

    Hot Water Demand (Disturbance Variable) 1

    We model hot water demand as a cyclostationary random process W(t) given by

    W(t) := specific heat

    Npeoplei=1

    Nij=1

    F (j),i (T (j),i Tamb

    ) I{S(j),i t < S

    (j),i +D

    (j),i

    },

    where := {shower , bath, . . . , dishwasher} is the set of possible usage events, Npeopleis the number of people in a household, Ni is the number of events of type corresponding to the i th person in the household, and the following are randomvariables:

    S(j),i := the start time of E(j),i ,

    D(j),i := the duration of E(j),i ,

    F (j),i := the flow rate of E(j),i ,

    T (j),i := the desired temperature of E(j),i .

    4 / 31

  • Problem Formulation

    Hot Water Demand (Disturbance Variable) 2

    We can only observe W(t) at pre-specified times t t , therefore, we approximateW(t) using a piecewise linear interpolation

    W(t) :=W(tk) +t tk

    t[W(tk + t)W(tk)].

    for all k = 0, 1, . . . and t [tk , tk + t). The discrete-time analog of W(t) to be theaverage of W(t) over t [tk , tk + t),

    Wk :=1

    t

    tk+ttk

    W(t) dt = 12 [W(tk) +W(tk + t)].

    We denote particular realizations of W(t) and Wk using w(t) and wk , respectively.We discretize wk {0,w , . . . ,wmax}. We write the conditional probability massfunction of Wk given hk as pWk (wk | hk).

    5 / 31

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Problem Formulation

    State Equation

    The state equation maps the current state xk , current decision uk , and currentdisturbance wk into the next state xk+1 according to

    xk+1 = f (xk , uk ,wk) :=(fT (Tk , uk ,wk), fh(tk , hk ,wk)

    ),

    where

    Tk+1 = fT (Tk , uk ,wk) := max{Tk rcoolt (Tk Tamb)+ rheatt uk rlosst wk , Tamb

    }hk+1 = fh(hk ,wk) :=

    {{(tk ,wk)} hk , tk 6= (N 1)t, otherwise,

    for all k = 0, 1, . . .

    6 / 31

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Problem Formulation

    Objective Function

    The objective is to minimize over all policies , the following function

    J(x0) = limK

    EW

    [1

    K

    K1k=0

    g(Xk , (Xk),Wk ;

    ) x0],

    = limK

    1

    K

    K1k=0

    EW0,W1,...,Wk ,

    [g(Xk , (Xk),Wk ;

    ) x0].where X0 = x0 is given and Xk = f

    (Xk1, (Xk1),Wk1

    ), for all k = 1, 2, . . .

    7 / 31

  • Problem Formulation

    Stage Cost

    The stage cost is

    g (xk , uk ,wk ; ) := gdiscomfort (xk , uk ,wk ;Tmin) + (1 ) goperating (xk , uk) ,

    where := {,Tmin} is a customer-defined parameter set, [0, 1] is the relativeweighting of the objectives, and Tmin is the minimum desirable temperature during ahot water use.

    Operating Cost

    The operating cost is

    goperating (uk) :=1

    t

    tk+ttk

    C (t) rating uk dt,

    where C (t) is the cost of power and rating is the power rating of the water heater.8 / 31

  • Problem Formulation

    Discomfort CostThe discomfort cost is

    gdiscomfort(xk , uk ,wk ;Tmin

    ):=

    1

    t

    tk+ttk

    max{Tmin T (t), 0

    } I{w(t) > 0} dt,

    where

    T (t) := Tk +t tk

    t[fT (Tk , uk ,wk) Tk ],

    for all k = 0, 1, . . .

    9 / 31

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Problem Formulation

    Problem StatementFind a feasible on/off policy that minimizes an expected objective cost.

    minimize

    limK

    EW

    [1

    K

    K1k=0

    g(Xk , (Xk),Wk ;

    ) x0]

    subject to Xk+1 = f(Xk , (Xk),Wk

    ), (xk) {0, 1},

    Tk [Tamb,Tmax ], for all k = 0, 1, . . .This is a discrete-time, average cost periodic Markov decision problem (MDP).

    10 / 31

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Outline

    1 Motivation

    2 Problem FormulationSystem VariablesState DynamicsObjective FunctionProblem Statement

    3 MethodologyFinite-Horizon DPAverage Cost DP for Periodic MDPsApproximate Dynamic Programming

    Prescient Lower Bound (PLB)

    4 ResultsNumerical Simulations SetupSet-Point MethodsTemperature AggregationUsage History Aggregation

    5 Problem ExtensionsSolar Water HeatingAutomated Demand Response

    6 Conclusion

  • Methodology

    Finite-Horizon Dynamic Programming

    The goal is to minimize over all policies , the following function

    J(x0) = EW

    [gterminal(XM) +

    M1k=0

    g(Xk , k(Xk),Wk ;

    ) x0],

    where M is the horizon and gterminal is a terminal cost function.

    The optimal policy is the minimizer of Bellmans equations

    J(xM) = g