Next Generation Automation Final

  • Published on

  • View

  • Download

Embed Size (px)


This paper describes a service-oriented architecture for data acquisition and control leveraging some of the latest concepts in software engineering. By addressing the business requirements to support role-based applicationsand services, the platform removes the islands of automation issue and allows a vertical decomposition of the application stack to lower the barrier to entry for new applications


  • 1. Next Generation Automation Effective Platform Design and PracticalImplementation Daniel Evans, Sam Hendley, Adam Crain, John S. CamilleriNCSU Centennial Park Campus Venture IV Building 1730 Varsity Drive, Suite 500Raleigh, NC,,, jcamilleri@greenenergycorp.comKeywords: Automation, SOA, ESB, Messaging 1. KEY CHALLENGES The Electric Utility Industry is under an ever increasingAbstract pressure to support the demands of the Smart Grid. TheThere are a number of challenges in meeting the definedchallenge is complex due to the variety of systems andNIST architecture while driving functions that exist today and those that are beingApplication modularity, integration with existing enterprise considered the smarter models, leveraging existing enterprise architectureand supporting increasing volumes of diverse field dataSome of these challenges are identified as the following:(including meter data, phasor measurements, traditionaltelemetry for operations, power quality and new remote Bridging the operational technologies (OT) andsensing devices) are among some of the key challenges information technologies (IT) to support increasingtoday.interoperability needs Avoid architecture brittlenessThis paper describes a service-oriented architecture for dataRemove the island of automationacquisition and control leveraging some of the latestExponential growth in remote sensing dataconcepts in software engineering. By addressing the Manage the effects of a diminishing workforcebusiness requirements to support role-based applications Harmonization of enterprise data modelsand services, the platform removes the islands ofautomation issue and allows a vertical decomposition of Leveraging/Enhancing the legacy IT investmentthe application stack to lower the barrier to entry for newapplications. The platform supports horizontal scalability for The Guidelines for Smart Grid Cyber Security - NIST IRperformance and is distributed, by design, for reliability 7628 [1] help identify the overall number of systems,and high availability. In addition, this paper addresses the interoperability and guidelines for cyber securitydetailed design issues around support, performance,requirements. The main thrust of this paper is to describe anreliability, security and scalability. open platform approach that addresses the challenges identified above and leverages the NIST IR as a standardFinally, we will showcase a practical implementation of thereference model.basic concepts of data acquisition and control whileleveraging an industry standard enterprise data model to 2. DESIGN PRICINPLESsupport interoperability between heterogeneous systems.In order to address the key challenges, the platform supportsThis platform design shows how todays well definedthe following design principles. With these principles instandards coupled with emerging standards can be leveraged place, barriers to innovation are a modular approach to minimize the customers risk andreduce overall development support for vendors.Grid-Interop Forum 2011

2. Evans, Hendley, Crain, Camilleri2.1. Open Platform abandon their use. The principle use of open source withThe concept of openness is typically focused on a specific this platform is to allow community involvement across theprotocol, standard or programming interface rather than acore platform components and support liberal licensingholistic view of a system. The word open has become apolicies for independent software vendors to use themarketing buzzword with no real meaning. Therefore itplatform.requires a clear understanding of the platform lifecycle toevaluate whether open in the context of the specific 2.3. Cross Platformsystem or platform really is a feature or red herring. In order to support a diverse set of applications, platforms and human resource skill sets the platform should leverageThe main objective of an open platform is to foster adoption architectures and technologies that allow multipleby those entities that can support and employ the platform. languages, operating systems, and client architectures. ThisAdoption is encouraged by common standards that a approach enables another opportunity for innovation bycommunity agrees to use. There are numerous standards increasing the tool sets and approaches of domain experts.that are currently being reviewed by the Smart GridInteroperability Panel. Many of these standards, like IEC61850, DNP3, C22.12,are specific to the electric utility.Other standard bodies like IETF, OASIS, and W3C support2.4. Service Oriented Architecturea vast array of technologies in the software community. TheA service oriented architecture (SOA) provides for modularprinciple of this platform is to leverage informationgrowth capabilities and is especially designed fortechnology standards across the software and industrycontinuous integration support [4]. Figure 1 shows thedomains. publish/subscribe model supported in this platform. The SOA supports modularity and endpoint abstraction. The message broker supports pub/sub and event driven systems. Adherence to this type of architecture promotes composition and reusability. Architecture brittleness is reduced and nicely supports a dynamic business and technical environment such as the evolution to a smarter grid. This principle supports innovation, serviceability, continuous integration. 3. DOMAIN TRIAD DESCRIPTION In order to support continuous integration of business objectives related to managing the electric power system, providing value added solutions to the end customers, and managing the rate of change on business requirements we first haveto identify the domaintriad.Figure 1: Publish/Subscribe Model2.2. Open SourceThe use of open source in commercial software is growing.In many cases, the end customer does not even realize thatsome of the code it is running is based on open source.Many companies leverage stable open source projects tosupport their product. Running stable projects [2] dependson a complex set of community interactions and solidsoftware engineering lifecycle principles. Many of theseprojects rival some of most complex propriety code bases inthe world due to the large number of developers and level ofcomplexity. One key use of open source in this platform isto leverage hundreds of thousands of lines of code fromstable community projects addressing common issues.Licensing options add to the complexity of open source.Figure 2: Domain TriadThe open source initiative [3] maintains a list of approvedlicenses. Some licenses are incompatible with each otherand require users to package their solution differently or For simplicity, we can identify three main domains of systems, business processes and technologies for a Utility.Grid-Interop Forum 2011 3. Evans, Hendley, Crain, Camilleri Back Office systems are dedicated to running the business messages/sec on an 8-core box using Gigabit Ethernet [8].of the utility such as accounting, billing, and account There are a number of other industries adopting AMQP asmanagement. Front Office systems are customer facing the transport protocol for the same reasons required by thesuch as support and new services. Grid Operations refers toUtility industry.the management of the power system. The conventional Brokers and clients communicate with each other by thedomain model from SGIP contains seven domains, but for AMQP specification. Because the specification is open,simplicity the three mentioned above are sufficient for multiple vendor implementations can be deployed to workdiscussions in this paper. Figure 2 shows a simple depiction together which gives the end customer true interoperabilityof thedomaintriad. and the ability to choose best-in-class client and broker solutions avoiding vendor lock-in.To build out a platform architecture that is open, leverages The function of the broker is to receive messages from thebest practices and solutions from other industries, and clients and create routes, queues and generally manage allsupports a robust ecosystem of solution providers requires incoming/outgoing messages of the system. Clients arebasic architecture building blocks to interface across all services and/or applications that use them.three domains. In this paper we will focus primarily on thebuilding blocks for the Grid Operations Domain. 4.2. Configuration Brokers can be configured in clusters for High Availability4. ARCHITECTURE BUILDING BLOCKS Messaging. Clients, also referred to as consumers andIn many cases the IT Enterprise solves integration needs for producers, are really an extension of a simple client serverbusiness systems using Enterprise Integration Patterns [4]. model. When a client publishes messages to the broker, theThe common term for this type of integration architecture is broker, for instance, might configure an exchange withcalled an Enterprise Service Bus (ESB) [5]. An ESB topics for subscription by other clients. For this platform aleverages service oriented architecture (SOA) [6] for simple example is the collection of measurements from animplementing the interaction and communication of services RTU in a substation where the client subscribes to alland applications. The SOA design also supports distributed measurement in that particular substation.computing and promotes an asynchronous message orienteddesign.The information technology fabric used to bridge thedomain triad is called a message oriented middleware. Theheart of the fabric is the Advanced Messaging QueuingProtocol (AMQP) which is an open standard applicationlayer protocol. The deployment models of the brokers and clients are driven4.1. AMQPby the requirements of the architecture. Clustering,AMQP is the heart of the message-oriented middleware federation, message persistence (durability), and otherplatform that bridges the domain triad. AMQP providesimplementation can be tuned to support the deploymentqueuing, routing, message orientation, reliability and The specification [7] calls out Ubiquity, Safety,Fidelity, Applicability, Interoperability, and Manageability The configuration supports how the messages moveas key requirement areas. The open standard originated inthroughout the system, their availability, and reliability. Thethe financial industry. Their objective was to address a payload of the messaging system for this type of systemcritical business need of providing high speed reliablerequires a terse payload envelope to maintain highmessaging with an open standard to avoid vendor-lock in. performance. The payload schema in this platformThis work started in the mid 2000s and AMQP 1.0 is just leverages protocol buffers developed by being ratified underOASIS. 4.3. Protocol BuffersThe requirements of AMQP from the financial industry are Google protocol buffers are licensed under an Apache 2.0applicable to the utility industry as well. The two majorlicense and have C++, .Net, Java and Pythonrequirement areas for supporting ubiquitous messagingimplementations among others. Protocol buffers use awithin the Grid Operations domain are performance andschema that is compiled to multiple language bindings,security.AMQP addresses these requirements in an providing a simple, yet effective, interface. The lightweightabstract fashion. In a cited performance test, a commercialschema format loosely resembles structs in C.implementation of AMQP achieved over 6 millionGrid-Interop Forum 2011 4. Evans, Hendley, Crain, CamilleriThe following example shows how measurement quality is GET, PUT, POST, DELETE) which decouplesencoded as a protocol buffer. In this case, the IEC 61850client server development.definition is used. The reef project uses protocol buffers to Opaque transport - Clients are unaware of whetherdefine service resources and the REST framework. they are directly connected to the end service or a proxy. Proxies can be used for things like load// mirror the iec61850 quality (CIM uses these too)balancing or to enforce security.message Quality { Stateless - Every request contains all of the enum Validity {// No abnormal condition of the acquisition function orinformation necessary to satisfy the request. Any// the information source is detectedserver-side state must be addressable in some way.GOOD = 0;// Abnormal condition ""5. CORE SERVICESINVALID = 1;Within this platform there are a number of services. Wewill focus a set of core services that support the automation// supervision function detects abnormal behavior,functions of the platform. Figure 3 shows the basic// however value could still be valid.// Up to client how to interpret. services of Reef.QUESTIONABLE = 2;}enum Source {// value is provided by an input function from// the process I/O or calculated by applicationPROCESS = 0;// value is provided by input on an operator// or by an automatic sourceSUBSTITUTED = 1;}optional Validity validity = 1 [default = GOOD];optional Source source = 2 [default = PROCESS]; Figure 3: Basic Services of Reefoptional DetailQual detail_qual = 3;// classifies a value as a test value, not to be used for 5.1. Measurementsoperational purposesMeasurements are the basic units of data published by dataoptional bool test = 4 [default = false]; sources which are processed and monitored by the system.Often, measurements are acquired using communications// further update of the value has been blocked by an// operator. if set, DetailQual::oldData should be setprotocols, and are used to represent the state of remote fieldoptional bool operator_blocked = 5 [default = false]; devices. Measurements may also be generated or manually} entered by agents. The basic fields provided by themeasurement service are Value, Quality, Time, and Unit.Measurements enter the system either from communicationfront-ends or directly using the Reef Project APIs. Themeasurement stream is then processed -- configured4.4. REST transformations are applied (scaling, value mapping) andRepresentational State Transfer (REST) is an architecturalside-effects are triggered (event/alarm generation) . Finally,style for distributed systems. REST is commonly thought ofmeasurements are stored in the database and published toas a "nouns and verbs" alternative to Remote Procedure Call the bus. The system also maintains measurement history,(RPC), but it is actually a proper subset of RPC that hasthe chronological stream of previous measurement values .some interesting properties. Factoring a distributed systemin a RESTful style has the following advantages (which can5.2. Commandsalso be thought of as constraints) as applicable to theCommands are the indications agents use to interact withdiscussed platform:and modify the state of the system. Commonly, commandsare tunneled by the communications processors to remote Uniform interface - Service consumers/providers field devices in order to exercise control ove...