Design and Simulation Issues for Secure Power Networks as ... · controlled by the system operator and private microgrid operator entities: ... The smart grid concept covers an extensive

Energy Systems Research Laboratory, FIU

Design and Simulation Issues for

Secure Power Networks as Resilient

Smart Grid Infrastructures

Professor Osama A. Mohammed

Energy Systems Research Laboratory

Department of Electrical & Computer Engineering

Florida International University

Miami, Florida

September 21, 2016

ECCE IEEE Energy Conversion Congress & Expo

Milwaukee, WI USA

Prof. O. A. Mohammed, ECCE 2016, Milwaukee, Sept. 21, 2016




• Management of increased levels of distributed and

renewable energy sources. (control challenge)

• Integrating a wide variety of systems governed by different

regulations, different comm. standards, and owned by

different entities. (interoperability challenge)

• The variable nature of renewable energy sources.

(Generation uncertainty)

• Real time energy forecasting and energy management

system for generation and demand balancing. (Demand

uncertainty)

• New reconfigurable distributed architectures with many

micro grids. (Resiliency)

• Distributed optimization and adaptive control (multi agent

systems)

Challenges of integrating distributed resources



• Interoperability between different protocols

and standards and applications (in software

layer).

• Identification of the communication network

and bandwidth required to collect

measurement and control remote sites

(Distributed control).

• Data availability (Delay, corrupted data, denial

of service,…etc.)

• Data security and privacy



Smart Grid Cyber Infrastructure



Smart Meter Security Threats

SMART Meter Vulnerability:• The AMI network is open to external unsecured

environments such as cellular channels, power line

carriers and radio signals.

Cellular,

Power line modem

Radio Signal (900MHZ)

Adversary

• The AMI can provide a communication path

to customer systems such as building

management systems (BMS) through the

customer gateway.

• If the adversary succeeds in penetrating into

the AMI network and pretending to be a valid

smart meter management system, he can

easily send a disconnect signal to millions of

customers.

AMI: Advanced Metering InfrastructureD

isco

nn

ectio

n s

ign

al.

Inco

rrect p

rice

Incorr

ect Load D

ata

ZigBee

WiFi, etc.



Smart meter security threats

Secure Measures

• The network topology should prevent

interaction between customers in the NAN.

• Price signal should be authenticated

• Smart meters use X.509 certificate.

• Most of the smart meters doesn't update the

certificate for life time. Example latest

discovered bug “Heartbleed” in OpenSSL used to

compromise the certificateZigbee, Wifi

X.509 Certificate is an Authentication Protocol

Between Smart Meter and Utility. Uses SSL Certificate

Example: an attack on a

customer appliance



Smart Grid Cyber Infrastructure (FAN Threats)



Area Electric Power System Operator (AESPO) Responsible entity of the host grid Interacts with microgrid agents Leads tertiary level controls Update rate: Minutes Ancillary service, Pricing (Auction)

Microgrid Hierarchical Control

ExampleAESPO Agent

Microgrid Agent

DER Agent

Microgrid Agent

DER Agent

Microgrid Agent

DER Agent

DER Agent

DER Agent

DER Agent

Te

rtia

ry L

eve

l Co

ntr

ol

Age

nt

Co

mm

un

icat

ion

Se

con

da

ry L

eve

l Co

ntr

ol

Age

nt

Co

mm

un

ica

tio

n

We adopted three control levels for

microgrid and active distribution

network hierarchy.

Microgrid Operator Responsible entity of the microgrid Interacts with DER agents Leads secondary level controls Update rate: Seconds Automatic generation control, Optimization

Distributed Energy Resource (DER) Operator Responsible entity of the DER unit Interacts with DER agents Lead primary level controls Update rate: Immediate Power sharing, Protection



Field Area Network (FAN)

• FAN shared multi service IP

network cover Distribution

automation, Integrated

Distributed resources, Demand

Response and field devices

• Based on Broad Band wireless

resources. FAN routers has

WIFI interface for field

technician.

• Data integrity and confidentiality

should be ensured for smart

meter data and field devices.

• If adversary succeed to

compromise FAN router he

could be easily sending

wrong signal to switches or

field devices

NIST reference ModelNIST Publication 1108 Page 35

FAN routers located on the pole



Smart Grid Cyber Infrastructure (WAM Threats)



Security challenges:

• Most of the protocols were developed for efficient data

transmission in isolated control network without

considering the security required for wide spread and open

system.

• Phasor Measurement Units (PMUs) depend on external

clock source which can be spoofed or jammed.

• PMU protocols ( C37.118 and IEEE 1334 ) doesn't support

authentication or data integrity check.



• State estimator can detect

bad data form faulty

meters or communication

errors

• Stealth attack can be

designed to be hidden

from state estimator.

• Several types of stealth

attacks can be performed

against the state estimator

such as (state, framing

and topology attack)

Bad data from faulty

meter

Bad data

Identified

State estimator stealth attack

Bad data not

identified



To design security aware WAM, different factors

should be considered in the communication and

system design such as:

Data authentication (insure the source of the Data)

Data integrity (detect corrupted or changed data)

Proper location of highly secured and encrypt meters

to prevent state estimator attack.

Data mining techniques could be used to detect

altered data.



• Cyber Physical security should not only be

considered in the cyber component but also

the power system network topology should

be designed to be resilient in cases of

attack.

• The control system should be designed to

withstand cyber attack and cyber

component failures.

• Centralized control suffer from single point

of failure problems.

• Successful attack against centralized

control system could lead to serious

damage and loss of service



• Decentralized

control reduce the

risk of single point

of failures and loss

of service.

• Risk of attacking

area and loss of

service still high



• Distributed control minimize

the risk of cyber attack.

• Each node exchange

information and cooperate

with neighbor node to

improve the system stability.

• Attack detection can be

improved by data mining from

different sourcescompletely distributed

multi-agent control



Centralized vs Decentralized Control



Multi Agent Applications in Power Systems

Decentralized control is

established using multi agent

frameworks

Agents interact and cooperate

to achieve a global or local

objective (through an

optimization function)

In future active distribution networks, simultaneous power system operations will be

controlled by the system operator and private microgrid operator entities:

• Frequency and voltage support (Ancillary Service)

• Online DER Scheduling (Optimal Dispatch)

• Market models (Auctions, Dynamic Pricing)

There is a need to perform concurrent control. Multi agent

control applications are required.Agent Entity

Peer-to-peer

Communication



• An agent requires to interact with its environment through sensors and actuators.

• A sensor acquires the data from the outside world and the actuator responds according to the agent’s decision.

Agent Platform

Sensors

Decision Making

Actuators

Envi

ron

men

t

Perception

Action

How Can We Link Power System Physical Objects to Agent Platforms?

• For Actual Multi Agent Field Implementation:

Need to link agent objects to distributed industrial control systems.

C37.118 OPC Server

IEC 61850 MMS OPC Server

Modbus OPC Server

OPC UA Information Exchange Interface

JAVA OPC UA Client Cloud OPC UA Client

IED PMU PLC



Can IEC 61850 Meet Decentralized Control of

Active Distribution Network Demands?

The smart grid concept covers an extensive control, automation and protection applications.

IEC 61850 does not meet all the required forms of monitoring and

information exchange demands.

Active distribution networks require dynamic adjustment of primary, secondary and tertiary control levels.

• Frequency and voltage support (Ancillary Service) • Online DER Scheduling (Optimal Dispatch) • Market models (Auctions, Dynamic Pricing)

Advanced intelligent multi agent frameworks are necessary with a flexible

ability to create tailor-made decentralized control schemes while

allowing the legacy protocols.



FRA

MEW

OR

K is

a p

roce

ss b

y w

hic

h a

gen

t co

des

are

d

evel

op

ed a

nd

per

form

ed in

rea

l-ti

me

po

wer

sys

tem

ap

plic

atio

ns

OPC UAMIDDLEWARE

Cloud Communication

Interface

Clie

nt /

Ser

ver

Java ClientFIPA Messages

IEC 61850Manufacturing Message

Specification (MMS)



• The types and levels of data protection used to

encrypt or authenticate signals should be

coordinated with signal sensitivity and impact on

the system stability.

• The attack detection should rely on physical

system characteristics as well as the cyber

security rules

• Cyber attack countermeasures should consider

the dynamics and the special nature of power

system.



• We need new modeling and simulation tools to capture

the dynamic nature of both cyber and physical

components.

– Integrated HW/SW

• Ability to perform real-time experiments involving

integration of different technologies in one system

(communication, software and physical components)

• Co-simulation tools should provide the ability to test

different types of vulnerabilities and launch attack

scenarios over a real network

• Exploration simulation to run asynchronous simulation

models in order to explore impact of control decision on

real network before execution or detecting system

anomalies.



• Communication middleware is required to

enable information exchange between

different controllers.

• Should provide portability and interoperability

between different system component.

• Should provide time predictable performance,

low latency and overhead to meet the real

time application requirements.

• The communication middleware must support

large system expansion and adding new

types of data.

• The Communication Middleware could be

message centric or data centric



Efforts

saved By

Data

centric

Middleware

Da

ta c

en

tric

is

ch

os

en

to

be

Us

ed

at

the

sm

art

gri

d T

es

t b

ed

Emphasized by Industry

(see Smart grid

Interoperability Panel)



• Data Distribution service is a communication

standard based on data centric and

publisher subscriber approach created by

Object Management Group (OMG).

• Supported by standard Application

Programming Interface API which simplify

integration with different applications.

• Utilize real time publisher subscriber protocol

(RTPS)

• IEC 61850 GOOSE implements RTPS

communication.

(provide interoperability between different vendors)



• No message broker or server which avoid single

point of failure.

Communication

failure

DDS DATA BUS

Client server communication

schema

• Single point of failure

• Low update rate

• High latency

DDS publisher/subscriber

communication schema

• Reliable communication (no single point of failure)

• High update rate

• Low latency (no intermediate message broker)

Tra

nsm

itting n

odes

Transmitting nodes Receiving nodes

Receiv

ing

nodes



Unicast communication• Multiple stream for multiple

destination

• Consume high bandwidth

• Not suitable for remote or

distributed control where multiple

agents need to access the same

data

• Multiple copy of the same data

sent from the source to Each

distention.

• Consume high bandwidth

• Not suitable for Wide Area

Measurement since allocated

bandwidth for remote site

usually low.

• Transmitting multiple stream

add extra processing overhead

on the transmitting nodes and

reduce the update rate

Three copies of the same data



• Single stream for all destination

• Reduce the network bandwidth

• Reduce the processor overhead

• Suitable for low bandwidth link

and remote sites

• Rich set of Quality of Services

profiles QoS (predictable delivery)

• QoS defines the data

transmission priority, life time,

ordering based on time stamp

and allowed latency.

Multi cast communication• Single stream for multiple

destination

• Optimize network bandwidth

• suitable for remote or distributed

control where multiple agents need

to access the same data

Single copy of the data



• Creating a unified global

space for high speed data

sharing between micro grid

assets

• Increasing system resiliency

by implementing distributed

peer-to-peer communication

to avoid single point of

failure

The DDS communication can

be implemented over wired or

wireless networks



Conclusions• The smart grid requires new set of modeling, simulation and

experimental tools for designing, testing and validation.

• The co-simulation platform provides an integrated

environment for testing developed techniques involving

cyber and physical components in order identify full system

behavior.

• The DDS provide real time performance and distributed

architecture which simplify the data exchange and control

implementation.

• The RTPS ensure the interoperability between different

nodes.

• New techniques may be required for devising protection

schemes that utilize energy storage to enhance resiliency

against communication outages.


Documents

Design and Simulation Issues for Secure Power Networks as ... · controlled by the system operator and private microgrid operator entities: ... The smart grid concept covers an extensive