Systems of Systems: Cybersecurity Vulnerabilities and Opportunities

Applied Computational Intelligence Lab & Trustworthy Systems Lab

University of Missouri - Rolla

Systems of Systems: Cybersecurity Vulnerabilities and Opportunities

Applied Mathematics for Deregulated Electric Power Systems:

Optimization, Control, and Computational IntelligenceCrystal City, November 2003

Donald Wunsch, ACIL DirectorAnn Miller, TSL Director

Acknowledgements Funding

– NSF– Sandia– Boeing– MK Finley Professorship– Cindy Tang Professorship

Senior Personnel– Ganesh Kumar

Venayagamoorthy– Ron Harley– Daryl Beetner– Danil Prokhorov– Raonak Uz-Zaman– Frank Harary

Personnel – Narayan Vishwanathan– Amit Agarwahl– Sam Mulder– Wenxin Liu– Nian Zhang– Alexander Novokhodko– Xindi Cai– Rohit Dua– Hu Xiao– Rui Xu– Brian Blaha– Paul Pigg– Arvind Rapka Nath– Qiang Yao– Kevin Bollum– Anjaya Shrestra– Karthik Balasubramanian– Pinar Demircan– Daniel Treat– Ian Downard– Eyad Salah Tagiedin– Ganesh Sridharan– Jason White– Krishnaprasad Balasubramanian– Dayle Majors– Nartaj Lakshminarasimhan– Siddarth Panchal– Robert Wayne Denier– Tongquan Wei– Jimish Doshi– Ravikiran Sharda

“system of systems”– Grown/evolved by adding components not

initially designed to be part of the system– Interdependencies not easily identified

Potential for cascading failures Potential for hidden robustness

Systems of Systems: Interdependencies

Trustworthiness Testing Market Demands Complexity Safety Life-Cycle Model Integration

Issues in Systems ofSystems

19841984 8686 9090 9292 9494 9696 199819988888

11

1010

100100R

ecom

men

ded

disk

spa

ce, M

B

Math package 1Math package 2Math package 3

Moore’s L

aw

Source: IEEE Spectrum, January 1998

Complexity: Software Size Growth

Complexity: Software Size Growth

Complexity: Interdependencies

A graph representing almost 6 million lines of computer code. The graph contains approximately 33 thousand nodes and 34 thousand relations.

Source: NATO Report on Visualization, 1999.

Memory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environmentMemory managementFile directory Access

I/O PrimitivesProcess Primitives

Process environment

Normalised Failure Rate, %0 5 10 15 20 25 30

LINUX

NT

Win-2000

Win-CE

Failure Rates – System Calls

(Source: Carnegie Mellon, CS Dept.)

Cascading failures Opportunities for errors Control, Communication, IT

– Pres. Commission on Critical Infrastructure Protection

– Particularly EMS & SCADA Voltage Collapse

Effects of Complexity and Growth

High-Consequence

Even brief – expensive– Circuit fab: 20 min = ($30 M)

Recent large disruption caused deaths

Backups no guarantee– Well-known in software safety circles

Therac 25 classic example

At 0903 CST on 18 December 1997, at the Olathe (Kansas City) Air Route Traffic Control Center, a technician routed power through half of the redundant uninterruptible power system, preparatory to performing the annual preventive maintenance on the other half. Apparently the wrong board was pulled.

Complexity: Ripple Effect Example

Results: – Power only out for 4 minutes– Radar and communications working within 17

minutes– However, at least 300 planes were in the Olathe-

controlled airspace; domino effect: hundreds of flights canceled, diverted, or delayed with problems well into the evening.

Complexity: Ripple Effect Example

Not only did the Air Route Traffic Control Center have redundant systems, there were also standby generators and emergency batteries.

Yet, that December morning, these back-up systems were bypassed.

Why?

Back-up Systems Are Not a Guarantee

The back-up systems were bypassed because the system was in a maintenance state.

This particular combination of inputs was not anticipated to occur when the system was in maintenance mode.

Complex Interactions: States and Inputs

Tempting Target

Dramatic growth in number of knowledgeable experts

Potential to insert incorrect data or Denial of Service attacks

High leverage / low risk

Computational Intelligence Tools Can Help

Neural Net Intrusion Detection ADP Robust Controls Combinatorial Optimization for

reconfigurability

Intrusion Detection with Neural Nets

RBFNs can be used for misuse and anomaly detection using sequences of system calls

Data are obtained from 1998 DARPA Intrusion Detection Evaluation program

Also collaboration with Sandia Red Team

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False positives

True

pos

itive

s

accuracy=0.74

RBFNN Generalization on unknown test data

PNN + ADABOOST

Multi-Machine Power System Control

G1 G2

G3

1 2

3

4 5

900 KmExciter

AVR Vref2

Exciter

AVR

Vref1

Ve1 Ve2Vt1 Vt2

900 Km

900 Km

Pref1

1Governor Governor

Turbine Turbine

2

Pref2

Multi-Machine Power System with Conventional Controllers

Multi-Machine Power System with DHP Neurocontrollers

G1 G2

G3

1 2

3

4 5

900 KmExciter

Vref2

Exciter

Vref1

Vf1 Vf2Vt1

Vt2

900 Km

900 Km

Pref11

Turbine Turbine

2

Pref2

Neurocontoller

TDL TDL

P1

Neurocontoller

TDL TDL

P2

Vref1 Vref2

GovernorGovernor

DHP Critic Network Adaptation

PLANTTDL

ACTIONNeural

NetworkMODELNeural

Network

CRITICNeural

Network

CRITICNeural

Network

TDL

MODELNeural

Network

YrefY(t)

A(t))()(

tAtU

(t+1))(

^1tY

)(^

tY

)(^

1tY

TDLTDL

)(^

tY

)(^

1tY TDLTDL )(

^2tY

++)(

)()(

^1tY

1tJ1t

)(

)()(

^1tY

1tJ1t

)()(tYtU

+---

EC2(t)

Terminal Voltage of Generator G2 for a 5% Step Change in its Desired Terminal

Voltage & Operating Point Changed

1 2 3 4 5 6 7 80.98

0.99

1

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

Time in seconds

Term

inal

vol

tage

in p

u

DHP

AVR

Speed Deviation of Generator G2 - Operating Point Changed

0 1 2 3 4 5 6 7 8-1.5

-1

-0.5

0

0.5

1

1.5x 10

-3

Time in seconds

Spe

ed d

evia

tion

of G

1 in

pu

Conventional

Neurocontroller

Traveling Salesman Problem

Great benchmark NP – complete

– Maps to other NP – complete problems Public databases

Big need – get learning capability of NN without brittleness of other techniques.

Paper Method Largest

Instance

Quality

(percent

excess over

optimal )

Test bed

[11] 1st 100 14.6% NS

[13] 1st 100 14% NS

[10] 1st 400 NR NS

[5] 2nd 532 6.8% TSPLIB

[12] 1st 1000 NR NS

[16] 2nd 1000 NR NS

[15] 1st 2392 5% TSPLIB

[17] 2nd 2392 9% TSPLIB

[2] 1st 10000 NR NS

[4] 1st 11849 17.4% TSPLIB

Previous contributions -- disappointing

Clustered Traveling Salesman

Divide problem into clusters using ART in O(n)

Use Lin-Kernighan algorithm for global tour

Use Lin-Kernighan algorithm for local tours

Merge local tours in O(n) time Global operations limited to O(n) time

Algorithm Overview

Read problem from file O(n)

ART O(n lg n)

cluster

cluster

cluster

LK O(k2.2)

LK O(k2.2)

LK O(k2.2)Merge

Clusters O(n)

Result

Implemented in C++ thread-safe code Uses Windows threads for parallelism Operating System-specific code isolated to one

file Should be easy to port to other parallel

systems

Implementation

#cities Tour Length 1P Time 2P Time Vig factor % off Speedup 1000 2.58E+07 0.422 0.281 0.7 10.40% 1.50 2000 3.61E+07 1.031 0.672 0.7 10.64% 1.53 8000 7.14E+07 8.328 4.281 0.72 10.97% 1.95 10000 7.97E+07 11.359 7.297 0.75 10.57% 1.56 20000 1.12E+08 24.641 14.406 0.8 10.53% 1.71 250000 4.00E+08 315.078 209.687 0.92 11.64% 1.50 1000000 7.94E+08 1468.165 986.48 0.97 11.03% 1.49 10000000 2.52E+09 10528.7 0.98 1.27% CONCORDE 1000 2.34E+07 1.670 2000 3.26E+07 3.500 8000 6.43E+07 26.570 10000 7.20E+07 37.620 20000 1.01E+08 84.830 250000 3.58E+08 1379.540 1000000 7.15E+08 9013.53 10000000 2.495E+09 43630.7

1k

4k

8k

10k

20k

50k

85k

150k

250k

1 M

Even better news…

Continued Scaling Results Parallelizability Memory Management

BUT – To Move Beyond

Clear Need for more advanced architectures– Especially to Learn from

Experience Cellular Structures necessary Same with SRNs Therefore, combine them and

ACDs

Recurrent Nets

Obviously achieve dynamic behaviors Possible similarity to adaptive systems

but with fixed parameters Simultaneous recurrent nets particularly

challenging, esp. architectures

•Graph Theoretic Representation

•SRN Necessary (Werbos & Pang, ’96 & ’98)

•Cellular structure – scaling

•Closed form now

•Convergence time now

•Importance of design principles

Generalized Maze Problem

Require for the output node: x16 = (x2 / x1)[min{x6, x5, x4, x3} + 1].

This is a known SRN!

Design from output backward

Output J = (x2/x1) * sum = x16(a,b)

CurrentNode inputs Neighbor

node inputsFeedback inputs

(Occurs at each node (a,b) in maze.)

Product Nodes

+1

/ *

Cellular SRN Structure Complete

Analyze worst-case convergence

WCT = N2 - 2N + N - 3 = N2 - N - 3.

Also true for N x N maze by simple induction proof.

Note that this is convergence in J steps.

Conclusions

Power networks inherit the full range of “systems of systems” issues.

These are amenable to computational intelligence solutions:– Detection– Robust Control– Reconfigurability

Combinatorial Optimization