Boot Camp - Conformity July 19, 2010 Detroit, USA

Boot Camp - Conformity

July 19, 2010 Detroit, USA

Bruce Muschlitz, EnerNexSlide 2

Boot Camp - Conformity• Overview– What does this Working Group do?– Previous meetings – Knoxville, San Fran, DC– Guiding principles– Terms we use– NIST Activities– Our Activities– Re-organization

Bruce Muschlitz, EnerNexSlide 3

Boot Camp – ConformityGoals of the working group

• Coordinate Task Forces– Edge Conformity– Enterprise Conformity– Security Conformity

• Provide overall guidance• Propose/Review task force deliverables

Bruce Muschlitz, EnerNexSlide 4

Boot Camp – ConformityPrevious Meeting (Knoxville)

• Organized Group• Introduced 61850-10 as one models• Stressed conformance != interoperabilty• Explained abstract vs. detailed tests• Introduced “virtual” test environments

Bruce Muschlitz, EnerNexSlide 5

Boot Camp – ConformityPrevious Meeting (San Francisco)

• Continued Group Organization• Discussed “plug-fest” – won’t do this• Discussed how other do this:– ISO Guide 65– IEC 17011 and 17025– http://www.rabnet.com

• Discussed Product Mark (logo)• Recognized: 61850 Testing, SGIP TCC

Bruce Muschlitz, EnerNexSlide 6

Boot Camp – ConformityPrevious Meeting (McLean)

• Organized Security Conformity• Re-organized by Horizontal teams• Discussed interaction with SGIP TCC• Recognized Edge/ENT might work better as

(Physical) Device/ (Middleware) Interface

Bruce Muschlitz, EnerNexSlide 7

Boot Camp – ConformityGuiding Principles

• Detailed Tests are not defined by UCAIug • Testers shall adhere to the defined tests • Equivalence of testers (no easy testers) • Tester shall produce “full” test results • Testers are free to script the tests

Bruce Muschlitz, EnerNexSlide 8

Boot Camp – ConformityCommon Terms

• Conformance – meets spec?• Interop – plays well with others?• Positive tests – does it work right?• Negative tests – recovers gracefully?• Black Box tests – no inside knowledge• White Box tests – view algorithms

Slide 9

Boot Camp – ConformityNIST Priority Action Plans

Phil Beecher, PG&E

Bruce Muschlitz, EnerNexSlide 10

Boot Camp – ConformityNIST SGIP Standing Committees

• Smart Grid Architecture Committee• Smart Grid Testing and Certification

Committee – TCC committee will learn on our work– TCC will share some duties with out work

Conformity Activities

• Common Glossary• Product Mark presentation• Templates – test cases, use cases• TISSUEs (Technical Issues)• “Conformity Requirements Document”

Re-Organization• Original organization based upon physical

location of devices (distance from enterprise)• New organization based upon interface

functionality (real or virtual)– Real – wired or wireless protocols– Virtual - APIs

Bruce Muschlitz, EnerNexSlide 13

Boot Camp – ConformityBackground Material

• http://osgug.ucaiug.org• http://www.ucaiug.org• http://www.rabnete.com• http://www.iec.ch/helpline/sitetree/

conformity• http://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/

WebHome

Slide 14

Boot Camp – ConformityQuestions?

Phil Beecher, PG&E

Boot Camp

Edge / Enterprise Conformity

Edge / Enterprise Conformity Activity

• Certification Process Reference Manual• Define Abstract Test Cases

What is the CPRM?

• Overview of device and system requirements• Identifies best practice for device and system

protocol design• Defines the process used to define and

maintain the quality of a Certification Program

Guiding Principles

• Open standards based• Clean, layered architecture• Robust certification program• Focussed on application programming interface, not specific

applications• Layered conformance testing• Performance testing considerations• Economically viable

Organisational Requirements

• Identify organisational structure to support a robust certification and interoperability testing program– Program management– Test laboratory qualification– Logo management– Change control– Dispute resolution– Devices and systems

Program Overview

Context (systems)

Context (products)

OpenSG SG Conformity – Security Conformity

July 19, 2010Bobby Brown

Security Conformity Task Force

Security Conformity TF Charter

• Establish security conformance requirements for laboratories desiring to certify smart grid components and systems

• Establish clear scoping boundaries, perform research to identify existing models, and propose a high-level philosophy of approach

• Chair: Bobby Brown, EnerNex representing Consumers Energy, bobby@enernex.com

• Vice-Chair: needed

Work Plan

• Reporting & Communication• High-level Conformity Requirements• Testing Use Cases

Goals

• Provide testing laboratories with best practice for cyber security testing.• Provide environmental and technical considerations for entities

developing internal cyber security testing processes• Develop cyber security tools list and resources for security testing.• Develop test cases• Support funded efforts conducting actual testing to vet and assist in the

development of the testing method, and cyber security metrics.• Develop guidance for suppliers outsourcing cyber security testing –

technical, procedural, communication, reporting, and status.

Conformance Definitions

a) “Is any activity to determine, directly or indirectly, that a process, product, or service meets relevant standards and fulfills relevant requirements.” ISO/IEC Guide 2:2004

b) Conforms if… “has not been proven to be non-conformant with standard x”

Meeting Logistics

• Every Friday at 2:00PM Eastern Time (changing to every other week)

• OPENSG-SGCONFORM-SEC@SMARTGRIDLISTSERV.ORG

Contact bobby@enernex.com for dial-in information and to be added to ListServ

Detroit Face-to-Face Agenda

8-10 AM Thurs July 22nd in Founders B-3

• Review Charter & Work Plan• Old Business• AMI Use Cases• Identify Standards (AMI)

Thank you!

Bobby Brown, EnerNex865-740-2844

bobby@enernex.com

Bruce Muschlitz, EnerNexSlide 34

Boot Camp – Edge ConformityQuestions?