EMS Users Group Meeting 2011 Training Session Interop Testing.pdf · 2011. 9. 20. · Schneider Electric Siemens Schweitzer Engineering Laboratories Toshiba Triangle Microworks ZIV

Systems Integration Specialists Company, Inc.

The Standards Based Integration Company

© Copyright 2010 SISCO, Inc. 1

EMS Users Group Meeting 2011 Training Session Interop Testing, C37-118 Update, IEC 61850-90-5, and IEC Security Overview

September 20, 2011

Philadelphia, PA USA

Ralph Mackiewicz SISCO, Inc. 6605 19 1/2 Mile Road Sterling Heights, MI 48314 Tel: +1-586-254-0020 ext. 103 [email protected] http://www.sisconet.com


Agenda

IEC 61850 Interoperability Testing

IEEE C37.118 Update

IEC Security Activities

IEC 61850 -90-5, Secure PMU communications using IP Multicast


Ground Rules

Have a Question?

Ask a Question As Needed!



© Copyright 2010 SISCO, Inc.

IEC 61850 Interoperability Testing

Herbert Falk

UCA IOP Group Meeting

April 1, 2011

UCAIug IOP Results Report

Herbert Falk

UCA IOP Group Meeting

April 1, 2011

UCAIug IOP Results Report

Topics

• Areas of Testing – Network Infrastructure

– Substation Configuration Language

– Sampled Values

– GOOSE (Generic Object Oriented Substation Event)

– Client/Server

– Time Synchronization - SNTP (added at site)

IEC 61850-90-4 Network Engineering Guidelines

• Test Approach – Multiple switch vendor’s equipment

– Primary purpose to test Rapid Spanning Tree Protocol (RSTP) in the following topologies:

– Single Ring

– Main Ring with 2 Sub-Rings

– Other topologies:

– Main Ring with Mesh

– Single Ring with Integrated Switches

Redundant Port: 2 independent Ethernet ports with 2 different

addresses

Redundant Media: 1 Ethernet port with switched media

Ethernet

Ethernet1 Ethernet2

Switches on loss of Ethernet

link pulses

Primary Back-Up

MAC – 2

IP Addr - 2

MAC – 1

IP Addr - 1

MAC – 1

IP Addr - 1

Redundant Media is Common - Easy to Configure for Redundancy

Redundant Ethernet

Redundant Network Configuration

Ethernet Switch Ethernet Switch

Ethernet Card

Ethernet Switch

WAN WAN

The time to

rebuild MAC

tables after failure

is critical feature

of the switches

Parallel Redundancy Protocol (PRP)

LAN A

LAN B

PRP Header

PDU

Send to both

PRP Cache

PDU

First PRP frame in

is delivered

Emerging Approach Embedded Switching

IED

E-Net1 E-Net2

Switch

IED

E-Net1 E-Net2

Switch

IED

E-Net1 E-Net2

Switch

HSR – High-Speed Redundancy Ethernet uses this kind of approach to

avoid the delay of rebuilding the MAC tables on a failure

Network Infrastructure Participants

– Hirschmann

– RuggedCom

– Siemens

– Schweitzer Engineering Laboratories*

– ZIV

* - unmanaged switch did not participate

in RSTP testing.

Infrastructure IOP Results

• Not all switches interoperated properly.

• Found that all hands are not as quick as others.

• Fiber 1G uplink cables “preferred” over copper.

• Auto-negotiation turned off has a major impact on RSTP performance (can impact recovery by almost 6 seconds).

• In a highly meshed “network” a root bridge failure can cause the network to take up to 20 seconds to recover.

Infrastructure Lessons Learned

• IOP Host IT staffs need to be more involved prior to the IOP.

• Network infrastructure should have been staged prior to IED being plugged in.

• The best laid plans sometimes take too long to configure.

– The full network infrastructure never got fully configured to support the IED/61850 testing as was originally intended.

– Need to investigate how to streamline configuration (maybe an SCL like configuration file for switches).

SCL – IEC 61850-6

• Test Approach: – Exchange of SCL for IED Configuration

• primarily Configured IED Description (CID)

– Exchange of SCL to create Substation Configuration Description(s) for exchange.

Every participant had to participate either as a IED exchange or System exchange. The exception to the rule: Switches are not considered IEDs (yet?).

SCL IOP Results

• No “complete” SCL validating tool exist, XML validation is not SCL validation

• Not able to properly interpret the XSD without reading -6.

• A good percentage of problems have been addressed in ED.2

• There has not been a validated release of the ED.1 XSD + Technical Issues (TISSUE) fixes.

SCL Lessons Learned

• Clarifications/user guide may be useful

• SCL allowed the IOP to come together rather quickly.

IEC 61850-9-2 Sample Values (SV) Process Bus

• Test Approach: – Validate UCA Users Group Usage guide for 9-2LE

– Merging Units and Simulators provided by:

• Alstom Grid

• RTDS Technologies • Schweitzer Engineering Laboratories (SEL)

– Subscribers provided by:

• KETOP

• Alstom Grid

• Schweitzer Engineering Laboratories

SV IOP Results

• Question of SampleSync values (an additional value was added in V3 of UCA 61850-9-2LE but V3 was never published).

• SCL example in the standard is not correct.

GOOSE – IEC 61850-8-1

• Test Approach: – Validate FCD (Functionally Constrained Data –

complex structure) and FCDA (Functionally Constrained Data Attribute – singel value) exchange.

– Validate detection of communication loss and Time Allowed to Live (TAL) processing

– “Test” bit behavior.

GOOSE Participants

• Publishers And Subscribers

– Alstom Grid

– Efacec

– GE

– Prosoft-Systems

– RTDS Technologies

– Schneider Electric

Siemens

Schweitzer Engineering Laboratories

SISCO

Toshiba

Triangle Microworks

ZIV

GOOSE IOP Results

• Many issues clarified in ED.2 • Need to forward a Tissue regarding a transition indication from Test to

Non-Test. • Need to come up with a recommendation in regards to how to handle a

mismatched configuration. • May need to come up with best implementation guidance regarding IEC

61850-7-3 information to be supported so that “common” datatype transformations are readily available.

• Determined that leaf FCDA exchange is the least common denominator that enables interoperability.

Client Server – IEC 61850-8-1

• Test Approach: – Validate FCD and FCDA exchange.

– Validate typical control and reporting patterns

– Transfer and interoperability of transient disturbance files (COMTRADE).

Client/Server Participants

• Clients – ARC Informatique – Efacec – OSIsoft – Prosoft-Systems – Siemens – SISCO – Triangle Microworks – ZIV

Servers Alstom Grid Efacec GE Prosoft-Systems Schneider Electric Siemens Schweitzer Engineering Laboratories Toshiba Triangle Microworks ZIV

Client/Server IOP Results

• Determined “how” to solve the issue of COMTRADE file location and naming. Will need to add specific guidance in IEC 61850 8-1.

• Need to come up with better test methodologies for purging report buffers.

• Should recommend that FCD be preferred for reporting members.

SNTP – IEC 61850-8-1

• Test Approach: – Make sure that SNTP time synchronization worked.

– SNTP source: RuggedCom

– SNTP Clients:

• Alstom Grid SISCO

• Efacec Toshiba

• GE ZIV

• Prosoft-Systems

• Schneider Electric

• Siemens

Results: It worked.

Issues that span technological groups

• Use of VLANs: Network Infrastructure and IEC 61850-8-1

– IEC 61850-8-1 default configuration with VLAN 0 vs the way substations should be implemented.

– IEEE 802.1q and its impact on VLAN usage and text in IEC 61850-8-1.

Impacts: IEC 61850-90-4, IEC 61850-8-1, and IEC 61850-9-2.

More…

• Need IEC 61850-90-4 to be explicit about the impact of not using VLANs and Multicast Filtering.

General Comments • Major benefits for the 61850 suite of standards.

• Allowed vendors to improve their products.

• Utilities/witnesses observed that 61850 is interoperable.

• Encountered issues were typically fringe conditions.

– A high percentage of the executed tests had no issues (on previous slides).

– Most issues were resolved during IOP through system engineering.

• Recommend implementation of ED.1 + Tissues.

• Detailed test result document(s) will be produced.

• Current IOP focused on IEDs. More concentration on system engineering tooling recommended in the future.

It was a WIN:WIN

A Big Thank You goes out to our witnesses

• EDF – France

• Endesa Distribucion - Spain

• EnerNex - USA

• KEMA – Netherlands

• Ketop Laboratories – China

• Prosoft-Systems - Russia

• Red Electrica de Espana - Spain

• Mikronika - Russia

And to the host: EDF

Additional Information • UCA IOP Test Sponsors:

– Kay Clinard at UCA - [email protected] – Randy Lowe at AEP – [email protected] – John Simmins at EPRI – [email protected]

• UCA IOP Test Director: – Margaret Goodrich email – [email protected] – Margaret Goodrich Cell – + 1-903-477-7176

• UCA IOP Vendor Coordinators – 61850 - Herb Falk – [email protected] – 61968-4 – Bruce Scovill – [email protected] – 61968-13 – Eric Lambert – [email protected] – 61968-6 – Nada Reinprecht – [email protected]

– 61968-3 – Jon Fairchild – [email protected]

• IOP Host: – Provided hosting, sponsorship and technical consulting – Eric Lambert – [email protected]

mailto:[email protected]











Questions - Discussion




IEEE C37.118 Update



© Copyright 2010-2011 SISCO, Inc.

Synchrophasor Communications Review

C37.118 Issues

C37.118.1 and C37.118.2 Update




Synchrophasor Communications Review

C37.118 Issues

C37.118.1 and C37.118.2 Update

© Copyright 2010-2011 SISCO, Inc. 2

Outline

Review of IEEE C37.118 (2005)

Reasons for IEC 61850-90-5




Issues of C37.118 (2005) Leading to C37.118.1, C37.118.2 and IEC 61850-90-5

Issues and questions uncovered


IEEE C37.118 consists of:

Majority of the standard deals with Measurement techniques to generate appropriate measured values (e.g. synchrophasor measurements).

Some of the standard deals with a packet format to transfer the information.


Major protocol items uncovered

Address overlap due to aggregation

Command state machines

Behavior for commands sent to an invalid PMU ID

Command interaction

TCP

UDP

UDP Multicast?

Architectural

Bit mask definitions

Conformance Statement


Why UDP Multicast?


(1) Address Overlap – only have 65533 global addresses available

Utility

A

Utility

B

PMU ID=1 PDC

PMU ID=1 PDC

PDC now has 2 PMU IDs=1

PDC or Client can’t

differentiate between

the 2 PMU IDs=1

Problem:

Possible Solutions:

• Use address registry to prevent

duplicates

• Use description strings to provide

uniqueness

• Have PDCs transform addresses


(2) Invalid PMU ID Client Network PMU ID = 2

Normal

Operation

Enable ID=2

Data

Abnormal

Enable ID=1

No Error Responses defined in C37.118

• Client does not know how long to wait for response

Possible Solutions:

• Do nothing

• For TCP Connected PMUs, disconnect if invalid PMU ID received

•UDP?

• Define Error Responses


(3) Command Interaction TCP and UDP

Client Network PMU ID = 2

TCP Based

Command

TCP Response

UDP Based

Command

UDP Response (unicast)

Normal

TCP Based

Command

UDP Response

(unicast or multicast)

UDP Based

Command

UDP Response (multicast)

Allowed?

Ambiguous response profiles makes clients difficult


(4) UDP vs. TCP CONFIG responses do not indicate if data/responses will be sent over UDP or TCP.

If using UDP, is UDP Multicast allowed? (this creates state machine issues for

DISABLE commands).

If using TCP Command/Response/Data, when the connection is terminated data messages stop (this is a good thing?).

When a UDP Client goes down, the UDP Data Messages continue (should they?)

Propose that:

• All commands be sent over TCP

• All command responses (except Data) be over TCP

• Config responses be modified to indicate if Data is being sent over UDP

• Allow UDP Unicast only

• Use TCP-KEEPALIVE or application keepalive to check TCP connection.

•If connection terminates, DISABLE Data.


(5) Architectural – How to handle one Device/multiple PMUs?

PMU/PDC 1 IP Address

1 IP Address Device

PMU

PMU

TCP Connection

Device

PMU

PMU

1 IP Address

Allows one TCP connection to

be used to enable/receive multiple

PMU information. If connection

terminates, all PMU data lost.

Requires a separate connection per

PMU. Minimizes sequencing issues.


MASKING

Assume 1 DIGITAL Channel

Which bit is the mask?


Conformance

What is required by Clients?

What is required support for Servers?

What is the usefulness of Config1?

If a PDC receives a Config1, does it have to have the Config1 responses for all PMUs?

Should it be removed from the standard?


Implementation Issues (Security Related)

Command Command

Vs.

Command Padding Command

What does your PMU do?


Initial Test Rig

Sent an extra “0” byte.

Some PMUs worked OK (e.g. executed the second command)

Others didn’t recover.

Test Rig also sent a single command with x amount of padding that exceeded 65535 bytes.

Same results

Test Rig sent length-1 bytes

Similar results


Implementation Conclusions

Those PMUs that leverage Serial packet re-sync handle the conditions “properly”

Difficult to integrate/achieve interoperability due to different transport profiles/port usage.




IEEE C37.118 and IEC

Resolution process of joint logo request by IEEE (harmonization)


The Nexsus:

IEEE approached IEC for a dual logo (e.g. an IEC and IEEE standard)

IEC responded “NO” since C37.118 conflicted with IEC 61850.

Agreed upon approach was reached:

The measurement techniques/standard part of C37.118 would

become IEEE C37.118.1.

The “packet” format part would have some minor fix-ups and

become IEEE C37.118.2.

IEC 61850-90-5 would be the long term solution to improved

synchrophasor communications.


Project Timeline: Simplified

IEEE

C37.118

Published

2005 2009

IEEE Request

IEC for Dual

Logo

IEEE & IEC

start

JTF to develop

IEC 61850-90-5

2010

IEEE splits

C37.118 into

C37.118.1

C37.118.2

SGIP PAP-13

NIST recommends

IEC 61850 for

Adoption

1st DC of IEC

61850-90-5

balloted

2011

2nd DC of IEC

61850-90-5

to be balloted.

2012

IEC TR

61850-90-5

publication

IEEE

C37.118.1

C37.118.2

complete


IEEE C37.118.1

New C37.118.1 standard covers phasor, frequency, & ROCOF (Rate of Change of Frequency)

Steady-state and dynamic measurement characteristics

Keep existing steady-state requirements –magnitude, phase, frequency variation

Improve steady state requirements definitions

Add measurement requirements under dynamic conditions

Dynamic measurement bandwidth and response time

Modulation, ramp, and step test conditions

Add requirements for frequency & ROCOF measurement

New measurement –required definition & development

Requirements matched to same steady-state & dynamic tests as phasors


PMU Measurement techniques/filters create latencies

Static system measuring can produce measurements at better than reporting rates.

Dynamic system measurements impacted by filter and algorithm of PMUs.

IEEE PSRC May 2010 discussed this issue.

Typical latency (e.g. change in the field to reflection in PMU data)

can be 2.5 seconds (step response).

IEEE PSRC H11 is working on standardizing dynamic measurement

techniques that will further help quantify this.


C37.118.2 Common format

New

New


C37.118.2 Data Frame

Behavioral change

Forces Time Alignment function to be in PDCs


C37.118.2 Data Frame: Interesting Observations

Forces Time Alignment function to be in PDCs, single time quality

No PDC related configuration changes

only PMU

No data quality available, only time quality


C37.118.2 Config 1 & 2

Means fractional part of timestamp must be adaptively computed upon

receipt.

New


C37.118.2 Config 3 (Totally New)

Solves PMU ID

overlap issue. Clients

need to correlate since

not in Data Frame.

Allows more descriptive

names than 16 characters

and allows 61850 FCDA

names to be used.

Geospatial Location

added.

C37.118.1 class of

measurement


More upgrades






Communications Security Concepts Overview




Security Concepts Review

Ralph MackiewiczSISCO, Inc.6605 19½ Mile RoadSterling Heights, MI 48314-1408 USATel: +1-586-254-0020 x103Fax: +1-586-254-0053Email: [email protected]




Security Concepts Review

Ralph Mackiewicz SISCO, Inc. 6605 19½ Mile Road Sterling Heights, MI 48314-1408 USA Tel: +1-586-254-0020 x103 Fax: +1-586-254-0053 Email: [email protected]


Topics

Review of Communications Security Concepts ala ICCP and IEC 61850

Summary of IEC WG15 Status and Activities

Role Based Access Control


General Security Concerns

Appropriate access to information

Restriction of control and configuration ability.

Communication Access Control

Confidentiality


Background

Security is just not an ICCP issue:

FTP

Telnet

HTTP

Others….

For confidentiality (e.g. encryption) the above always

uses SSL/TLS. So does ICCP.

IEC wanted to use well understood and supported

technology for securing the TC57 protocols:

IEC 62351 – Data and Communication Security


Security Objectives for IEC 62351

Assuring only Authorized Access even within a closed private

network

Preventing Eavesdropping by non-trusted entities

Preventing Spoofing/Playback of captured data from non-trusted

entities

Secure and non-secure profiles must be able to co-exist and be

unambiguous

One set of identity management policies required

Same mechanism for all IEC TC57 communications profiles (&

DNP3)

Desire to use mainstream IT methodologies.


The IEC 62351 Specifications

IEC 62351-1 Introduction and Overview

IEC 62351-2 Glossary

IEC 62351-3 TCP/IP Profile How to use TLS

IEC 62351-4 Security for MMS based profiles Includes ICCP-TASE.2 annex) References 62351-3

IEC 62351-5 Security for IEC 60870-5 and derivatives (DNP3)

IEC 62351-6 Security for 61850 References 62351-4

IEC 62351-7 Mgmt Info. Base (MIB) for end-to-end net. Mgmt

IEC 62351-8 Role Based Access Control


IEC 62351 – Data and Communications Security

IEC 62351 specifies only how to use technology to implement security for

TC57 protocols.

It does not specify:

What systems need to be secured

When to use authentication

When to use encryption

How to implement role-based access control (coming for IEC 61850)


Profile of concern for ICCP-TASE.2

Data Link

Network

Transport

Session

Presentation

Application

Ethernet

IP (RFC 791)

ARP (RFC 826)

TCP (RFC 793)

RFC 1006

ISO Transport (ISO/IEC 8073)

Transport Class 0

ISO Session (ISO 8327)

ISO Presentation (ISO 9576)

ASN.1 (ISO/IEC 8824/8825)

ACSE

MMS (ISO/IEC 9506)


Security Tools

Encryption

Encrypting data so that only the 2 communicating

entities are able to understand the data.

Authentication

Using digital signatures to ensure that the entity at

the other end is known and trusted.


Security Technologies Used

Public/Private Key Encryption

Transport Layer Security (TLS)

Needed for Confidentiality

Digital Signatures

Needed to verify authenticity of identification

X.509 Digital Certificate Technology

Public / Private Key


NODE B NODE A

Public Key Encryption

Data

Node B

Public Key Encrypt

Encrypted

Data

Data

Node B

Private Key Decrypt

Encrypted

Data


NODE B NODE A

Y

N

Digital Signatures

Data

Encrypt

Digest

Data +

Signature

Don’t Use

Data

Signature

OK?

Data +

Signature

Node A

Private Key

OK to

Use Data

Node A

Public Key

Create

Digest


What is a Digital Certificate?

A digital certificate is a standardized file format that can be exchanged with communications partners that identifies an entity and contains:

A public key for encrypting data that can only be decrypted by the private key

A unique serial number assigned by the certificate authority

Certificate Authority Signature of the Certificate and algorithm used

The name of the certificate authority

Version of the certificate

Validity dates

Certificate thumbprint/digest and algorithm used

usage, etc.

A private key is included for your own certificate that you install on your own machine. You do not distribute certificates with private keys to others


What is a Certificate Authority?

A certificate authority is an entity that issues certificates.

There is a digital certificate for the CA that includes all the usual certificate

information including the CA’s public key

TRUST is a critical element of the CA:

Accepting a CA certificate means that you trust them to verify that the

information in certificates issued by them is valid

Don’t install certificates from CAs into

your system you don’t trust


Use of Certificate Authority

Calculate Digest/thumbprint/fingerprint of the digital certificate

Compare this to the signature generated by the certificate authority

If they MATCH AND you trust the CA: the certificate was issued to the

entity identified in the certificate by that CA and the public key can be

trusted

If they DON’T MATCH: then something is wrong and you can’t trust the

certificate or any information in it including the public key.


Certificate Authorities

Verisign

Thawte

Certisign

Deutche Telecom

EquiFax

ANYONE can be a CA

Important to Utilities

Power Pools

ISOs

RTOs

Your own company


Data Link

Network

Transport

Session

Presentation

Application

Logical Link Control (ISO 8802)

Media Access Control (ISO 8803)

IP (RFC 791)

ARP (RFC 826)

TCP (RFC 793)

SSL/TLS

RFC 1006

ISO Transport (ISO/IEC 8073)

Transport Class 0

ISO Session (ISO 8327)

ISO Presentation (ISO 9576)

ASN.1 (ISO/IEC 8824/8825)

ACSE (ISO/IEC 8650) + ACSE Authentication Definitions

MMS (ISO/IEC 9506)

Secure Profile for ICCP-TASE.2


Specification Theory

TLS is used to supply encryption and node

authentication.

Authenticates the identity of the computer running the

transport stack, not the applications accessing that stack.

ACSE is used for Application Authentication.

Authenticates individual applications residing on a given

computer.


TLS Encryption Application

Authentication Use

None None Backward Compatible with

current implementations

None Yes

For use over VPN

connections or internal to

control centers

Yes No

Provides encryption and

node level authentication

only.

Yes Yes Full security

Security Modes


TLS Encryption

Asymmetrical Public Key exchange is used to negotiate a secure

encrypted connection at the transport level.

Usually relatively high strength keys are used

1024 bit key length currently specd

Requests for 2048 bit keys

In order to minimize overhead, a symmetrical key (both sides use

the same encryption key) of a smaller size is then exchanged for

continuing communications.


Symmetrical Key Renegotiation

Maximum of every 5,000 packets (configurable).

10 minute time limit (configurable)

Entity that was connected to (called) responsible for

key negotiation.

Avoids protocol deadlocking.

Eliminates possibility of long-term eavesdropping to

break the weaker symmetrical keys.


TLS Cipher Suite

OpenSSL from http://www.openssl.org

Approximately 40 suites are available in OpenSSL

Picked a single suite as mandatory to enable

interoperability:

TLS_DH_DSS_WITH_AES_256_SHA

Several don’t encrypt and are deprecated


CPU Performance Impact of Encryption

Implementation specific

CPU performance related.

MMS Info Rpt

32K PDU

1520 Integer Variables

Every 2 seconds for 10 minutes

System A

Athlon XP 2400+

Windows 2000 Pro

System B

Athlon XP 2500

Windows 2000 Server


Measure Average CPU Utilization

TLS Suite System A System B

None 0.425 0.537

AES 256 0.577 (+35%) 0.758 (+41%)

3DES 0.708 (+66%) 0.931 (+73%)

DES 0.597 (+40%) 0.884 (+65%)


Data Transfer Bandwidth Impact of Encryption

Implementation not expected to have a major impact.

MMS Reads of 100 Vars

1000 Iterations

Observed with Ethereal

System A

Athlon XP 2400+

Windows 2000 Pro

System B

Athlon XP 2500

Windows 2000 Server


Data Transfer Bandwidth Results

Number of

Bytes observed

No Encryption 2,693,846

SSL AES-256 2,742,774 + 1.18%

Percentage

Increase


Impact of Application Authentication

Application Authentication only takes place during association

establishment.

ICCP-TASE.2 consists of long-lived associations

Infrequent application association initiation

No significant impact on application perfromance or bandwidth for

application authentication.

Minimal impact on application association initiation processing.




Security for IEC GOOSE and Sampled Values


Ethernet Multicast Address Using 802.3 Ethertype

Binary encoding of data

GOOSE - Generic Object Oriented Substation Event

Name Type

gocbRef VISIBLE-STRING

timeAllowedtoLive Integer (ms)

datSet VISIBLE-STRING

goID VISIBLE-STRING

T UTC Time

stNum INTEGER

sqNum INTEGER

Simulation BOOLEAN

confRev INTEGER

ndsCom BOOLEAN

numDatSetEntries INTEGER

Data per DataSet Def’n.


GOOSE/GSSE is Reliable Multicast

NON-EXISTENT

RETRANSMIT-

PENDING

SEND

Message

New State: 1.Sequence Number = 0

2.State Number++

3. Reset HoldTimer

HoldTime expired

1. Hold Time Preset ++

2. Start Hold Timer

3. Sequence Number ++


GOOSE/GSSE Traffic

t

Event at t=0

Each line below represents a GOOSE/GSSE message

Hold time increases from until

steady state of ~1/min is reached

State = 1, Seq = 0

State = 1, Seq = 6

State = 2, Seq = 0

State change occurs


Why Ethertype?

Supports Virtual LAN (VLAN) processing by switches.

VLAN enables intelligent 3-layer Ethernet switches to prioritize packets via VLAN Priority.

Enables high priority GOOSE packets to be forwarded sooner than lower priority directed messages (SCADA).

IEC 61850-90-5 adds UDP Multicast profiles


Piloting a Centralized Remedial Action Scheme (C-RAS) with

Emerging Telecomm / Protection Technologies

Piloting a Centralized Remedial Action Scheme (C-RAS) with

Emerging Telecomm / Protection Technologies

Patricia Arons,

Transmission & Interconnection Planning

Southern California Edison Company

March 2, 2007

Wide Area

Network

GOOSE Wide Area Application

Substation-to-Substation and Substation-to-EMS Communication

New Work Item Proposal for IEC TC57 – WG10

Application of VLAN Critical


GOOSE Control Block (GoCB) Services

GOOSE Multicast GOOSE Unicast ACSI Client/Server

From IEC61850-7-2


GOOSE Control Block per 8-1 Component

Name MMS

TypeDescription r/w m/o Condition Comments

GoEna Boolean rw m

GoID Visible-string r m

DatSet Visible-string r m The value of this component shall be of the format of ObjectReference and shall be limited to VMD or domain scoped NamedVariableLists

ConfRev Unsigned r m

NdsCom Boolean r m

DstAddress PHYCOMADDR r m

MinTime Unsigned r o As specified in the SCD file for the GoCB

MaxTime Unsigned r o As specified in the SCD file for the GoCB

FixedOffs Boolean r o As specified in the SCD file for the GoCB

Component Name Data Type m/o Comments

Addr OCTET-STRING m Length is 6 Octets and contains the value of the destination Media Access Control (MAC) address to which the GOOSE message is to be sent. The address shall be an Ethernet address that has the multicast bit set TRUE.

PRIORITY Unsigned8 m Range of values shall be limited from 0 to 7.

VID Unsigned16 m Range of values shall be limited from 0 to 4 095.

APPID Unsigned16 m As defined in Annex C


9-2 Process Bus

Process Bus – Sampled Value Messaging

Merging Unit

A/D A/D Input

Voltages

and

currents

Breaker

Status

Ethernet

Bay

Controller

Protection

Relay

Fault

Recorder

RTU,

etc. Ethernet Ethernet Ethernet Ethernet


SV Message

Ethernet Multicast Address Using 802.3 Ethertype

Binary encoding of data

From IEC61850-7-2


SV Control Block (MSVCB) Services

GOOSE Multicast ACSI Client/Server

From IEC61850-7-2


SV Control Block (MSVCB) MSVCB class

Attribute name Attribute type r/w Value/value range/explanation

MsvCBName

ObjectName

- Instance name of an instance of MSVCB

MsvCBRef ObjectReference - Path-name of an instance of MSVCB

SvEna BOOLEAN r/w Enabled (TRUE) | disabled (FALSE), DEFAULT FALSE

MsvID VISIBLE STRING129 r/w

DatSet ObjectReference r/w

ConfRev INT32U r

SmpMod ENUMERATED r/w samples per nominal period (DEFAULT) | samples per second | seconds per sample

SmpRate INT16U r/w (0..MAX)

OptFlds PACKED LIST r/w

refresh-time BOOLEAN

reserved BOOLEAN

sample-rate BOOLEAN

data-set-name BOOLEAN

DstAddress PHYCOMADDR r

Component Name Data Type m/o Comments

Addr OCTET-STRING m Length is 6 Octets and contains the value of the destination Media Access Control (MAC) address to which the GOOSE message is to be sent. The address shall be an Ethernet address that has the multicast bit set TRUE.

PRIORITY Unsigned8 m Range of values shall be limited from 0 to 7.

VID Unsigned16 m Range of values shall be limited from 0 to 4 095.

APPID Unsigned16 m As defined in Annex C


GOOSE and SV are special

Security may/may not be needed for many

applications

Implementations of secure and non-secure PDUs

(except for Encryption) need to be interoperable.

Becomes a subscriber’s configuration issue if to

expect/require security.

Source just does what it can.


Header GOOSE/SV PDU

Reserved

Current

Header

GOOSE/SV PDU

C

R

C Extended PDU

Extension

Length

Authentication Value

(Digital Signature)

Secure

Basic Idea for GOOSE/SV Authentication


Header GOOSE/SV PDU

Reserved

Current

Header

GOOSE/SV PDU

C

R

C Encrypted PDU

Extension Flags

Secure

Typically, confidentiality is only required for GOOSE over Wide

Area Networks.

Basic Idea for Confidentiality


Issues that arose

Implementation

Asymmetric signatures not fast enough for SV.

Almost not fast enough for GOOSE

Need to convey and Route Synchrophasors

Gave rise to IEC 61850-90-5




Data At Rest Issues


Data at rest concerns

Settings Topology Communication Technology

CIM XML

61850 XML

DNP Soon XML

ICCP Paper (soon

XML)

General Security

Levels of Concern Tamper Protection

Encryption

Work occurring in IEEE PSRC H18


Data in “transition”

Utility A Utility B

(1)

Utility C

(2)

(3)

How can Utility A restrict

what is exported to Utility C?


A look into the future (coming to IEC)

Normal XML

Contents

Access Constraints

XML

W3C

Signature

CIM, 61850,

DNP, ICCP

(new)

Reference

All File Contents

Specific Instances

Specific Instance Attributes

Constraints

By usage*




Role Based Access Control


Roles, Rights, and Operations


IEC 61850 Pre-Defined Roles and Rights


PUSH Model for RBAC Authentication


PULL Model for RBAC Authentication






IEC Security Activities

INTERNATIONAL

ELECTROTECHNICAL

COMMISSION

IEC TC57 WG15 - Cybersecurity

Status & RoadmapSeptember, 2011

Frances Cleveland

Convenor WG15

INTERNATIONAL

ELECTROTECHNICAL

COMMISSION

IEC TC57 WG15 - Cybersecurity

Status & Roadmap September, 2011

Frances Cleveland

Convenor WG15

Mission and Scope of WG15 on

Cybersecurity

Undertake the development of standards for

security of the communication protocols defined

by the IEC TC 57

Specifically the IEC 60870-5 series, the IEC 60870-6

series, the IEC 61850 series, the IEC 61970 series, and

the IEC 61968 series.

Undertake the development of standards and/or

technical reports on end-to-end security issues.

WG15 Status September 2011 3

WG15 Members

60 members

Participants from 20 countries

Argentina

Canada

China

Czechoslovakia

Denmark

Finland

France

Germany

Great Britain

Israel

Italy

Japan

Korea

Norway

Russia

South Africa

Spain

Sweden

Switzerland

USA

4 WG15 Status September 2011

Status of Security Documents, April 2011

IEC 62351: Data and Communications Security

Part 1: Introduction

Part 2: Glossary

Part 3: Security for profiles including TCP/IP

Part 4: Security for profiles including MMS

Part 5: Security for IEC 60870-5 and derivatives

Part 6: Security for IEC 61850 profiles

Part 7: Objects for Network Management

Part 8: Role-Based Access Control

WG15 Status September 2011 6 Issued as Technical Specifications in 2007/2008. Will be updated

Issued as TS in July 2009. MCR issued on Remote Update Key Change.

Issued as TS, released in July 2010

Issued as DTS, March 2011

Coordination with Other Security Activities

NIST Cyber Security Working Group (CSWG) under NIST’s

Smart Grid Interoperability Panel

Cyber security standards assessment – very detailed

assessments – IEC 62351 was included – FERC is reviewing

(http://www.nist.gov/public_affairs/releases/smartgrid_100710.cfm)

IEC TC57 WG15 has a Liaison A with IEC TC65C which will

review and standardize the work of the ISA SP99 Security

Standards

IEC TC57 WG15 has a Liaison D with the IEEE PES PSCC

Security Subcommittee

NERC CIP 002-009 – WG15 has members who are very active

with NERC security activities

Cigré D2.22


http://www.nist.gov/public_affairs/releases/smartgrid_100710.cfm

Completed and Current

Work

Updates & New Work On-Going

Coordination • Parts 1, 2, 3, 4, 5, 6 –

Finalized as TS Standards

• Part 2 (Glossary) can be

found at (http://std.iec.ch/terms/terms.nsf/ByPub?

OpenView&Count=-

1&RestrictToCategory=IEC%2062351-2

)

• Part 7: Network & System

Management – Finalized as

TS in July 2010

• Part 8: Role-Based Access

Control – DTS

• MCR for Part 5 on remote

changing of update keys


As of Sept 2011

• Part 5 Implementation

Specification

for IEC 60870-5 thru WG3

• Security Architecture White

Paper

• Key Management to

become IEC 62351 Part 9

• Edition 2 or Amendments

to Parts 1, 3, 4, & 6

• IEC TC65C WG10

• ISA SP99

• CIGRE D2.22

• EPRI NESCO

• NERC

• Research Labs

• NIST CSWG

• IEEE PSRC

• TC57 WG03

• ISO/IEC 27000

TC57 Security (IEC 62351) Roadmap

http://std.iec.ch/terms/terms.nsf/ByPub?OpenView&Count=-1&RestrictToCategory=IEC 62351-2






Issues

Run into Intellectual Property issues with certain cryptographic suites

Although we have cybersecurity experts, they are very busy

Cybersecurity is a very dynamic, rapidly changing field which is quite new for the power industry

Need rapid development of new standards and updates to existing standards

Need Security Architecture

Need guidelines for end-to-end security

What should be standards and what should be technical reports


INTERNATIONAL

ELECTROTECHNICAL

COMMISSION

Questions? Comments?






Using IEC 61850 for synchrophasor and protection/control messaging over IP Multicast

IEC 61850-90-5




C37.118.2 and beyond

IEC 61850-90-5




C37.118.2 and beyond

IEC 61850-90-5


Use cases documented in 90-5

WAMS/WAMPAC related

WAMS

Situational Awareness

State Estimation and on-line security assessment

Archival of information

WAMPAC

Special protection schemes

Predictive Dynamic Stability

Phenomenon assumption WAMPAC


Use cases documented in 90-5

“regional”/local related

Out-of-step (OOS) protection

Adaptive relaying

Synchro-check

Under-voltage shedding

NASPINET (covered by others)

PDC use case to be added.


Situational Awareness


State Estimation


Archival of Data


Special Protection Schemes


Predictive Dynamic Stability


Phenomenon Assumption


Out-of-step (OOS) protection


Adaptive Relaying


Synchro-check


Under Voltage Shedding


90-5 development asked: Why so many PDCs?

Answer:

C37.118.2 protocol not designed to scale from a communication

perspective.

Time alignment function (it is good and bad).


NASPInet - Requirements

Decided to use IP multicast to address large scale of NASPInet.


The assumption:

C37.118.1 Measurement

techniques produce

measured values that are

synchronized.

Voltage

Vectors

Current

Vectors

Frequency

ROCOF

f

C37.118.2 has adopted

some semantics, but

names of the measurements

are still not standardized. S

yn

ch

ron

ize

d

Me

asu

red

Va

lue

s

61850 has semantics

but no measurement class

(e.g. P and M class).

PPV,PhV

A

Hz

No semantic for ROCOF,

so that needed to be added.

HzRte


What about modeling of P and M class?

Added P and M Class calculation types

so that any Logical Node instance can

be measured in a fashion to C37.118.1.

Means the measurement/calculation

method is not independent of semantic

and independent from mechanism

of value transfer.

IEC 61850 Ed. 2


The multiple degrees of freedom allow:

61850 to convey P and M class data simultaneously.

61850 to convey P and M class data with other data/calculated information (e.g. Average, etc.).

To be conveyed through all of the available FCD/FCDA transfer profiles at 61850’s disposal (reporting, GOOSE, Logs, and Sampled Values).

Allows DataSet construct to be used by “clients” to determine which

values are to be delivered/configured for delivery.


Other information transported by C37.118.2

Geospatial – represented in 61850 ED.2 by Logical Nameplate FCDAs of longitude, latitude, and altitude.

Phase identification already supported in 61850:

TIME_BASE: Only one in IEC 61850.

phsC

phsB

phsA


Time Stamp and Time Quality in IEC 61850


Comparison of major C37.118.2 Services

Config 1 – Exposes what information the server has available. 61850 has self discovery and SCL that furnish this ability.

Config 2 & 3 – Exposes what information is being reported (subset of Config 1) 61850 control blocks/DataSets exposed through self-discovery or SCL.

Header – Intended to provide human readable descriptions of reported information. 61850 names are inherently human readable. Additional descriptions can be made available in the “d” and “du” attributes.


Comparison of major C37.118.2 Services

Commands – Allows enabling and disabling of Data transfer 61850 control block “enable” attributes provide this functionality.

Data – Actually transfers the synchronized measurements. 61850 – So many services to choose from:

Bufferred/Unbufferred Reports - data change (event), periodic,

update triggerred).

GSE Services (GOOSE) – multicast event driven.

Sampled Value Services – multicast stream delivery.

Logs – local historical storage.


To meet the use cases:

Services explicitly specified in IEC 61850-90-5

GOOSE

SV

Reporting and logging are implicitly allowed.


Component Name

MMS TypeDescription

r/w m/o Condition Comments

GoEna Boolean rw m

GoID Visible-string r m

DatSet Visible-string r m The value of this component shall be of the format of ObjectReference and shall be limited to VMD or domain scoped NamedVariableLists

ConfRev Unsigned r m

NdsCom Boolean r m

DstAddress PHYCOMADDR r m

MinTime Unsigned r o As specified in the SCD file for the GoCB

MaxTime Unsigned r o As specified in the SCD file for the GoCB

FixedOffs Boolean r o As specified in the SCD file for the GoCB

GOOSE Control Block (GoCB) and Services

GOOSE Multicast GOOSE Unicast ACSI Client/Server

From IEC61850-7-2


SV Control Block (MSVCB) and Services

GOOSE Multicast ACSI Client/Server

From IEC61850-7-2

MSVCB class

Attribute name Attribute type r/w Value/value range/explanation

MsvCBName

ObjectName

- Instance name of an instance of MSVCB

MsvCBRef ObjectReference - Path-name of an instance of MSVCB

SvEna BOOLEAN r/w Enabled (TRUE) | disabled (FALSE), DEFAULT FALSE

MsvID VISIBLE STRING129 r/w

DatSet ObjectReference r/w

ConfRev INT32U r

SmpMod ENUMERATED r/w samples per nominal period (DEFAULT) | samples per second | seconds per sample

SmpRate INT16U r/w (0..MAX)

OptFlds PACKED LIST r/w

refresh-time BOOLEAN

reserved BOOLEAN

sample-rate BOOLEAN

data-set-name BOOLEAN

DstAddress PHYCOMADDR r


IEC 61850-90-5 has several different profiles

Security Key

Management

Key Distribution

Center (KDC)

Multicast

Route

Determination

Data Transfer


Data Transfer – Session Layer

Session can carry:

Individual GOOSE messages

Individual SV messages

Re-encapulated GOOSE/SV messages

Individual Mngt PDUs

Aggregates (e.g. PDC aggregation function)

of:

GOOSE

SV

Encapsulations

Mngt

IP Multicast services: GOOSE, SV, Tunnel

IP Unicast services: Mngt


Data Transfer – Session Layer Security

Hints regarding Key rotation

Encryption Signature

Algorithm Algorithm

AES-128-GCM

AES-256-GCM

Key management/exchange done

out-of-band through GDOI profile/protocol


Data Transfer T-Profile (IPv4)

UDP

IP

Differentiated Services Code Point

Explicit

Congestion

Notification

802.1Q VLANs and Priority

Port 102


30

Security members and companies

Fernando Alvarez (ABB:Switzerland) – WG10 and WG15 Herbert Falk (SISCO:US) – WG10 and WG15 Steffen Fries (Siemens AG- GTF IT-Security: Germany ) – WG15 Darren Highfill (Utilisec:US)- Security Architect for Southern California

Edison, ASAP-SG, SGIP-CSWG, WG15. Satoshi Ito (Toshiba: Japan)

Denis Parnaland (Schneider Electric –R&D Security Technical Expert :France) – WG15

Maik Seewald (CISCO:Germany) - International: IEC TC 57 WG 10, WG 15, IEEE. National (Germany): DKE 952, DKE 952.0.15

Daniel Thanos (GE – Chief Cyber Security Architect of GE Digital Energy:Canada) – WG15


Selected for Extension

The KDC – Evaluated existing technologies


Group-Based Key Management (GBKM) Available Protocols and Approaches – GDOI (I)

Group Domain of Interpretation (GDOI) – Standards Track RFC 3547

Enhances IKE for group based communication, has two phases

GDOI Phase 1 = IKE (Internet Key Exchange) phase 1 for authentication of members toward group

controller to establish a security association (main mode 3 handshakes); RFC states phase 1 can be

any protocol providing Peer Authentication, Confidentiality, and Message Integrity

GDOI Phase 2 = distribution of key encryption key for protecting application specific keys, also used for

re-keying of the data security protocol SA. (2 handshakes) (re-keying initiated from server)

Support push and pull model for group keys; the push model reuses the phase 1 established SA (thus

the pull model would simply restart)

Support of different payloads:

Identity used to characterize the group selection


GDOI Purpose

Symmetric Key distribution

Enables message authentication code (ie. Signature) and encryption

Key distribution center (KDC) , GDOI server could be in the PMU/PDC


Group Domain of Interpretation (GDOI): Phase 1

Utilizes client certificate exchange to establish identity

Asymetric keys are used to establish a secure path betweeen 2 nodes for exchange of key information.

Symmetric keys used to encrypt TCP/IP packets.

Similar to how TLS is used for ICCP-TASE.2


GDOI Phase 2

Once access to the KDC is authenticated, subscriber requests a policy for a security association (SA) to an IED:

Type of communications (GOOSE or SV)

Data Set being transmitted


What is a Policy Request? It is a request to obtain policy and key information regarding a particular “group”.

GDOI groups, prior to IEC 61850-90-5, were destination IP address based. For IEC 61850-90-5 groups

needed to be further qualified:

161 (d) Length Object Identifier Payload as defined by Object Identifier General Format

Ethernet

GOOSE SV

Tunnel

UDP

GOOSE SV

MMS

Routable GOOSE/SV

Reservation for 62351-6

Reservation for Future use

Version Dest Mac DataSetRef

Version Dest IP

Version Dest IP

Content is part

of policy determination

Others

DataSetRef


What is a Policy Request?

It is a request to obtain policy and key information regarding a particular “group”.




Ethernet

GOOSE SV

Tunnel

UDP

GOOSE SV

MMS

Routable GOOSE/SV




Version Dest IP

Version Dest IP

Content is part


Others

DataSetRef

>128 user defined type of policy.

IEC 61850 chose 161 as it is

unassigned


What is a Policy Request?

It is a request to obtain policy and key information regarding a particular “group”.




Ethernet

GOOSE SV

Tunnel

UDP

GOOSE SV

MMS

Routable GOOSE/SV




Version Dest IP

Version Dest IP

Content is part


Others

DataSetRef

Object Identifier defines 90-5 or others that

may be used in future and share the

161 type.

The 90-5 OID under the WG15 recognized

root. Maintained by WG15/


GDOI Phase 3

Assuming the client is authorized to access, the KDC responds with GDOI Security Association Payload (SA):

The Current Key Encrypting Key (KEK) in use by the PMU/PDC

KEK is a symmetric key used to authenticate data received by the client that is in current use by the PMU/PDC

Next KEK that is to be used

Time remaining on current KEK

Client receives the IEC 61850-90-5 payloads separately using IP Multicast and authenticates using the KEK.

Must occassionaly reinstate GDOI phases to keep keys up to date.


Policy Request generates a SA Payload that contains:

RFC Defined Hdr

ID = 161

Object Identifier

Current Key Info Key Type Key Remaining Lifetime

Next Key Info

Auth Alg.

Key Type Key Remaining Lifetime Auth Alg.

Won’t be returned if client’s certificate expires prior to expiration of current key.

Key Download Payload very similar!


Where should the KDC function be placed?

In the device

External to

device

No redundancy required. Can only serve information for

the device.

Redundancy required. Can serve information for

the device.

IEC 61850-90-5 SCL modifications allow either approach to be described.


90-5 also recognized:

No way for a C37.118.2 client to configure a server for what data needs to be delivered.

90-5 makes use of SCL.

Did not want to re-develop measurement techniques.

References C37.118.1

Need to support streaming and events (based upon use cases).

Need to support other data besides synchrophasor measurements.


How to migrate from C37.118 to IEC 61850-90-5


IEC 61850-90-5

Allows for transmission of time aligned and non-time aligned information (e.g. multiple PDU transmission support).

Use of UDP/IPv4/IPv6 allows for the use of multicast addresses,

Should allow for “late” information to be delivered.

Will support event driven messaging and streaming.


Other features being discussed…

Needs to provide substation-to-substation and substation-control center

Designed for control center-to-control center

Does not require/expect time alignment to be provided by PDCs or other intermediate systems

Needs to be able to support 120 samples/cycle (might need 240/cycle)

Security

Application level digital signature on data to detect tamper and to provide “chain” of trust capability.


Assumptions

Intermediate Systems (e.g. PDCs and Phasor Gateways)

Provide up/down sampling

May not provide time alignment function

Implication: Applications/System designers must provide a time

alignment function.






Thank You

Ralph Mackiewicz

SISCO, Inc.

6605 19½ Mile Road

Sterling Heights, MI 48314-1408 USA

Tel: +1-586-254-0020 x103

Fax: +1-586-254-0053

Email: [email protected]