Autonomous Energy Systems: TransportationPresentation from Matt Moniot and Kalpesh Chaudhari at the 2020 Autonomous Energy Systems Workshop on Modeling and Management of Electric Vehicle

Autonomous Energy Systems: Transportation

Brennan Borlaug, Kalpesh Chaudhari, Rob Fitzgerald, Venu Garikapati, Yanbo Ge, Matt Moniot, Clement Rames, Nick Reinicke, Jinghui Wang, Eric Wood

NREL | 2

• Overview of two research areas related to transportation and grid impacts– Learned ride-hailing fleet control, load management– Consensus charging overview

Presentation Overview

NREL | 3

• Overview of two research areas related to transportation and grid impacts– Learned ride-hailing fleet control, load management– Consensus charging overview

• Integration into broader AES simulation framework

Presentation Overview

Ride Hailing Modeling, Managed Loads

NREL | 5

Model Overview, Inputs and Outputs

• Battery size• Fleet size• Occupancy, etc.

• Station power levels• Locations, plugs, etc.

• O-D locations• Pickup times

• Passenger pooling willingness

• $/kWh by TOD

NREL | 6

Model Overview, Inputs and Outputs

• Battery size• Fleet size• Occupancy, etc.

• Station power levels• Locations, plugs, etc.

• O-D locations• Pickup times

• Passenger pooling willingness

Broad Outputs:• Vehicle utilization• Fleet performance• Station economics• … and many more!• $/kWh by TOD

Highly Integrated Vehicle Ecosystem

NREL | 7

Trip Demand:Ride Austin TNC Data

Animation Web link

Fleet Visualization (Austin, TX)

https://i.imgur.com/4UpXQjH.gifv

HIVE 0.4.0+ Model Structure

NREL | 9

Data-Driven Fleet Control

• Version: 0.1.0, heuristic based decision making• Version: 0.4.0: Refactored HIVE in an “RL gym” to enable model training• Exploring opportunities for improved performance beyond heuristics

NREL | 10



Agent“Fleet Manager”Environment

State• Current & upcoming

requests• Current & upcoming

prices to charge• Fleet average SOC• Fleet composition by state

NREL | 11



Agent“Fleet Manager”Environment Action

• % Fleet Charging• % L2 vs DCFC




NREL | 12



Agent“Fleet Manager”Environment Action

• % Fleet Charging• % L2 vs DCFC




Reward• Marginal Income• Request revenue – charging costs

NREL | 13



Agent“Fleet Manager”Environment

Reward• Marginal Income• Request revenue – charging costs

Action• % Fleet Charging• % L2 vs DCFC




NREL | 14

Sample Scenario, Downtown Chicago

• Requests: Two days of request data - one for training, one for test• Fleet: 350 fleet vehicles with 50 kwh battery• Infrastructure 4 fast charging stations, 1 base with slow charging• Charging Costs: variable by time of day, based on data from ComEd• Forecasting: perfect for upcoming prices and requests

NREL | 15

Results, RL-Trained Fleet Manager

N training steps

Rew

ard

($10

00)

NREL | 16

Results, RL-Trained Fleet Manager, cont.

NREL | 17

Future Work, HIVE

• Extended study over 10+ days of request data• Expose fleet manager to more complicated rate structures

(demand charges)• Simulate more constrained scenarios – limited infrastructure,

larger geographic area• Expand scope of state/action space to include control of more

than just charging– Fleet rebalancing

Consensus Charge Control

NREL | 19

Background – Typical Control Hierarchies

Centralized data acquisition

from vehicles

Optimized charge profiles passed to

EVs from centralized node

A) No Control B) Centralized Control

All vehicles permitted to charge as demanded

Node

AES Simulation Framework

Node


Node


NREL | 20

Proposed: Consensus-Based Control

C) Consensus-based Distributed ControlRather than communicating with a centralized node, vehicles communicate amongst each other to develop a charging profile through consensus

Step 1: Communication among EVs Step 2: Aggregate optimized charge profiles from individual EVs

EV

EV

EV

Pev,i

Pev,i Pev,i

EV

EVEV

EVSE

Pev,i Pev,i

Pev,i

Node objective

Pev,i - Charging profile for each EV= min. Pev,iWhere,

NREL | 21

Consensus Control – Deeper Dive

Plim = Grid supply/capacity constraint

Pcs,iPcs,i

Pcs,i

EVSE

EVSEEVSEPaggPcs,i Pcs,i

Pcs,iEVSE

EVSE

EVSE

Step 3: Communication among Charging Stations

Step 4: Aggregate optimized load profilefrom individual charging stations

Step 5: If Pagg

NREL | 22

Consensus Control: Parameters Assumed

Focus: Demand charge mitigation, by flattening the charging profileParameters:

– EVi is the vector specified by (ai, di, ei, Pmax,i)– ai is the arrival time of EVi.– di is the departure time of EVi.– ei is the charging energy demand of EVi. ei (t) =0 if t < ai or t≥di– Pmax,i is the peak charging rate of EVi.– ri(t) is the instantaneous charging rate of EVi. ri (t) =0 if t

NREL | 23

Consensus Control: Performance Evaluation, cont.

0.00

200.00

400.00

600.00

800.00

1000.00

1200.00

100 120 140 160 180 200 220 240 260 280 300

Peak Power vs. No. of Vehicles

Real-time No Control

Real-time Central

Real-time Hierarchical w consensus

0.00

200.00

400.00

600.00

800.00

1000.00

1200.00

100 120 140 160 180 200 220 240 260 280 300

Peak Power vs. No. of Vehicles

24h Horizon No Control

24h Horizon Central

24h Horizon Hierarchical w consensusPe

ak P

ower

No. of Vehicles No. of Vehicles

Peak

Pow

er

NREL | 24


0.00

10.00

20.00

30.00

40.00

50.00

60.00

100 120 140 160 180 200 220 240 260 280 300

Comparison for 24h Horizon

24h Horizon % reduction Central

24h Horizon % reductionHierarchical w consensus

0.00

10.00

20.00

30.00

40.00

50.00

60.00

100 120 140 160 180 200 220 240 260 280 300

Comparison for Real-time

Real-time % reduction Central

Real-time % reductionHierarchical w consensus

No. of Vehicles

% P

eak

Redu

ctio

n

No. of Vehicles

% P

eak

Redu

ctio

n

NREL | 25


050

100150200250300350400450500550600650700750800850900950

1000

100 130 150 200 220 240 260 290

Solution time (24h Horizon)

24h Horizon Central

24h Horizon Hierarchical w consensus

Tim

e (s

econ

ds)

No. of EVs

020406080

100120140160180200220240260280300320340360

100 130 150 200 220 240 260 290

Solution time (Real-time)

Real-time Central

Real-time Hierarchical w consensusTi

me

(sec

onds

)No. of EVs

Transportation Topics, AES Simulation Framework

NREL | 27

Load Shifting Strategy, Comparison

HIVE: “Plug” control• HIVE has no formal charge control, but can load

shift by controlling when vehicles are sent to charge• Fleetwide charging peaks are controlled through

strategic dispatch & charge instructions• Main incentive is to recharge vehicles quickly so

additional passengers may be served

Time of Day

Char

ge L

oad

No grid feedback, static decision-making

NREL | 28

Load Shifting Strategy, Comparison

HIVE: “Plug” control• HIVE has no formal charge control, but can load

shift by controlling when vehicles are sent to charge• Fleetwide charging peaks are controlled through

strategic dispatch & charge instructions• Main incentive is to recharge vehicles quickly so

additional passengers may be served

Consensus: “Charge” control• Consensus control can affect the charging rate during an

event, but has no control over plug-in / plug-out times• Charging peaks are controlled within the confines of a

dwell. Greater dwell time correlated with greater flexibility• Likely to be best integrated at fleet depots where vehicles

have long overnight stays

Time of Day

Char

ge L

oad

Time of Day

Char

ge L

oad

No grid feedback, static decision-making Direct integration with AES framework

NREL | 29

Proposed Electric Vehicle-Grid Integration Framework

• Grid constraints• Electricity price• Additional load data

• Optimized charge schedule• Optimized charging cost• Cost incentive for wait times

• Generation• Storage

values

• Optimized charge schedule for • Demand charge mitigation• Operating cost reduction• Charging stations integrated with building &

renewables• Quantify communication requirements• Enable peer-to-peer transactions for price negotiation

HIVE Electric Vehicle-Grid IntegrationUsing Distributed Control


(Distribution Grid)• Arrival and departure time• Start and end SOC• Energy requirement• Vehicle parameters• Connected charger details• Desired charging cost• Maximum allowable charging

delay

BuildingsRenewables &

Storage

• Controllable loads• Uncontrollable loads • Optimized

load profile

Electric Vehicles Loads DER Grid

This work was funded by the US Department o Energy Vehicle Technologies Office.

NREL | 31

Consensus Control: Performance Evaluation

Title SlidePresentation Overview 1Presentation Overview 2Ride Hailing Modeling, Managed LoadsModel Overview, Inputs and Outputs 1Model Overview, Inputs and Outputs 2Fleet Visualization (Austin, TX)HIVE 0.4.0+ Model StructureData-Driven Fleet Control 1Data-Driven Fleet Control 2Data-Driven Fleet Control 3Data-Driven Fleet Control 4Data-Driven Fleet Control 5Sample Scenario, Downtown ChicagoResults, RL-Trained Fleet ManagerResults, RL-Trained Fleet Manager, cont.Future Work, HIVEConsensus Charge ControlBackground – Typical Control HierarchiesProposed: Consensus-Based ControlConsensus Control – Deeper DiveConsensus Control: Parameters AssumedConsensus Control: Performance Evaluation, cont. 1Consensus Control: Performance Evaluation, cont. 2Consensus Control: Performance Evaluation, cont. 3Transportation Topics, AES Simulation FrameworkLoad Shifting Strategy, Comparison 1Load Shifting Strategy, Comparison 2Proposed Electric Vehicle-Grid Integration FrameworkThanks! Questions?Consensus Control: Performance Evaluation

Documents

Autonomous Energy Systems: TransportationPresentation from Matt Moniot and Kalpesh Chaudhari at the 2020 Autonomous Energy Systems Workshop on Modeling and Management of Electric Vehicle