Smart Metering Implementation Programme (SMIP)€¦ · · 2014-12-14Smart Metering Implementation Programme (SMIP) ... 5.1.6 Test Traceability ... valid API Key Web Services N Y

SMKI & Repository Test Scenarios Document

(Working Document)

Page 1 of 46 v1.0

Date: 28/11/2014 DCC Public

Smart Metering Implementation Programme (SMIP)

SMKI and Repository Test Scenarios Document

(SRTSD)

(Working Document)

Version: 1.0

Date: 28/11/2014


(Working Document)

Page 2 of 46 v1.0


Associated Documents This document is associated with the following other documents:

Reference Type

Title & Originator’s Reference Source Release Date

Version

Information Only

BT SMKI_024 Device Certificate Subscribers Interface Specification

BT Work in progress

1.4

Information Only

BT SMKI_030 Workflow User Interface Specification

BT Work in progress

1.5

Related Documents

Document Reference

Document Version /

Status Author Date

1 SD4.1 DCC User Gateway Interface Design Specification

0.8 12/09/2014

2 Self-Service Interface Design Specification

1.0 28/02/2014

3 DCC User Gateway Code of Connection.

V0.9 Aug 2014

4 Smart Energy Code Stage 4 4 17/11/2014

5 Testing Issue Resolution Process Not

published

6 Common Test Scenarios Document 1.5 30/10/2014

7 SRT Approach Document Not

published

8 Registration Authority Policies and Procedures (RAPP) Document

Not published

9 SMKI Code of Connection Not

published

10 SMKI Repository Code of Connection

Not published

11 SMKI Repository Interface Design Specification

Not published

12 SMKI Interface Design Specification Not

published

13 DCC Gateway Connection and Codes of Connection Document

Consultation Closed


(Working Document)

Page 3 of 46 v1.0


Contents

Associated Documents ....................................................................................... 2

Related Documents ............................................................................................ 2

1 Introduction ...................................................................................................... 5

1.1 Purpose ................................................................................................... 5

2 Scope ................................................................................................................ 6

3 Test Sequence .................................................................................................. 7

4 SREPT Procedure ............................................................................................ 8

4.1 RAPP ...................................................................................................... 8

4.2 SMKI & Repository Entry Process Testing ............................................... 9

4.3 SREPT Initiation .................................................................................... 12

4.3.1 Procedural Steps ................................................................................... 12

4.3.2 SREPT Entry Criteria ............................................................................. 15

4.4 SREPT Execution .................................................................................. 16


4.4.2 SREPT Test Suspension/Resumption ................................................... 17

4.4.2.1 Suspension Criteria ........................................................................ 17

4.4.2.2 Test Resumption Criteria ................................................................ 17

4.4.2.3 Disputes regarding Test Suspension/Resumption .......................... 17

4.5 SREPT Completion................................................................................ 19


4.5.2 SREPT Exit Criteria ............................................................................... 20

4.5.3 Quality Gate Review .............................................................................. 20

4.5.4 SREPT Test Completion Certificate ....................................................... 21

5 Appendix A: Test Artefacts ........................................................................... 22

5.1 Party Documents & Reports .................................................................. 23

5.1.1 Test Preparation Document Set ............................................................. 23

5.1.2 Reports and Dashboard ......................................................................... 23


(Working Document)

Page 4 of 46 v1.0


5.1.3 Test Readiness Report (TRR) ............................................................... 23

5.1.4 Test Execution Dashboard..................................................................... 24

5.1.5 Test Completion Report ......................................................................... 25

5.1.6 Test Traceability .................................................................................... 26

6 Appendix B: Test Data ................................................................................... 29

7 Appendix C: Test Scenarios .......................................................................... 30

7.1 RAPP Scenario ...................................................................................... 30

7.1.1 Registration and Enrolment of Organisations and their representatives . 30

7.2 DCC User - SMKI & Repository Entry Process Test Scenarios .............. 31

7.2.1.1 Security Credentials Access Tests to SMKI .................................... 31

7.2.1.2 Security Credentials Access Tests to the Repository ...................... 32

7.2.2 Submission of Certificate Signing Requests .......................................... 33

7.2.3 Collection of Documents and Information from the Repository ............... 35

7.2.4 Query the Repository and Retrieve Documents and Information............ 36

7.2.5 Access Certificate Revocation List ......................................................... 38

7.3 Non-Gateway user - SMKI & Repository Entry Process Test Scenarios 39

7.3.1 Security Credentials Access Tests to SMKI ........................................... 39

7.3.2 Submission of Certificate Signing Requests .......................................... 40

7.3.3 Submit Requests for Repository Content and Access CRLs .................. 42

8 Appendix D: Forms and Templates .............................................................. 43

9 Appendix E: TEST COMPLETION CERTIFICATE ......................................... 44

10 Appendix F: DEFINITIONS ......................................................................... 45


(Working Document)

Page 5 of 46 v1.0


1 Introduction

1.1 Purpose

The purpose of this document is to:

Define the procedural steps to be undertaken by a Party wishing to complete the SMKI and Repository Entry Process Tests (SREPT) and meet the obligations set out in the Smart Energy Code (SEC) 4 in accordance with Section H14;

Set out the SMKI and Repository Entry Process Test Scenarios (SREPTS) that must be tested to use the SMKI and Repository service in accordance with SEC 4 Section L7;

Describe the role and responsibilities with regard to the conduct of SREPT including in;

o Defining Test Scripts o Defining Test Data o Planning the manner in which tests will be undertaken o Executing the tests o Reporting the results of those tests to the Data Communications

Company (DCC) for approval

The Party and the DCC shall use all reasonable endeavours to comply with the timescales that are defined within the procedures in Table 2, Table 3 & Table 4. Likewise, the DCC shall use all reasonable endeavours to meet its obligations with these documents, including compliance with timescales.

In the event that the Party does not comply with the timescales in Table 2, Table 3 & Table 4, the DCC may reschedule the Party test execution and completion date.


(Working Document)

Page 6 of 46 v1.0


2 Scope

Section 7 Appendix C: Test Scenarios of this document sets out the SMKI and Repository Entry Process Test Scenarios as referred to in SEC 4 Section H14.22.


(Working Document)

Page 7 of 46 v1.0


3 Test Sequence

The successful completion of the Registration Authority Policies and Procedures (RAPP) is a prerequisite to acquiring access to the SMKI and SMKI Repository.

On successful completion of the RAPP and becoming an Authorised Subscriber (Party) of the test system, the remaining SREPT scenarios set out in this document may be executed in any sequence the Party chooses.

For clarification:

there is one RAPP process with two possible outcomes

o One for test o One for live

the organisational RAPP process will be undertaken once only in order to verify the organisational id

each individual will be required to verify their identity either for live or for test, unless there are live users also carrying out testing

once the process is complete a user will be provided with their credentials for the specified environment

o Test users will only be able to access the test environment o Live users will only be able to access the live environment


(Working Document)

Page 8 of 46 v1.0


4 SREPT Procedure

This section describes the procedure that must be completed in order for a Party to complete SREPT.

4.1 RAPP

The Test scenario identified to support the RAPP process of SREPT is listed in section 7.1 RAPP Scenario, successful execution of this scenario is a prerequisite to executing the remainder of the SREPT scenarios.

Figure 1 SMKI Credential Issuance Flow


(Working Document)

Page 9 of 46 v1.0


4.2 SMKI & Repository Entry Process Testing

The matrix reflected in Table 1 below provides a mapping of DCC Users and Non-Gateway users to test scenarios.

Detailed test scenarios identified to support the SREPT process are listed in sections 7.2 DCC User - SMKI & Repository Entry Process Test Scenarios and 7.3 Non-Gateway user - SMKI & Repository Entry Process Test Scenarios.

Where a User Role of IS & GS is relevant to the same Party and the User Role appears in the User Role column in Table 1 SREPTS Matrix) below against a Device Certificate Signing Request scenario, that scenario must be undertaken by the Relevant Party against each individual User Role. (i.e. twice (IS, GS))

In all other instances the Relevant Party will only be required to execute a scenario once regardless of User Role.

SMKI Number

Scenario Method of

Access SMKI

SMKI Repository

User Role DCC User

Non Gateway

user

SMKI 01

A Party wishing to establish their organisational and individual identity with the DCC Registration Authority should follow the processes described in the Registration Authority Policies and Procedures (RAPP) document in order to acquire access to the SMKI using the SMKI Portal and also to the SMKI Repository Note: This scenario is a prerequisite and must be successfully executed prior to running any of the follow-on scenarios

Not Applicable N N IS GS ES

ED GT RSA OU

Y Y

SMKI 02 Use security credentials to access the SMKI Portal Portal Interface Y N IS GS ES

ED GT OU Y N

SMKI 03 Use Security Credentials to Access SMKI Service using the SMKI Web Service Interface

Web Services Y N IS GS ES

ED GT OU Y N

SMKI 04 Accessing the SMKI Portal using security credentials Portal Interface

via Internet Y N

IS GS ES ED GT RSA

N Y


(Working Document)

Page 10 of 46 v1.0


SMKI Number

Scenario Method of

Access SMKI

SMKI Repository

User Role DCC User

Non Gateway

user

SMKI 05 Access the SMKI Repository via the Portal Interface using secure log in and password

Portal Interface N Y IS GS ES

ED GT OU Y N

SMKI 06 Access the SMKI Repository via the SMKI Repository Interface using valid API Key

Web Services N Y IS GS ES

ED GT OU Y N

SMKI 07 Access the SMKI Repository via the Secure File Transfer Protocol (SFTP) using username and password

SFTP N Y IS GS ES

ED GT OU Y N

SMKI 08 Access Certificate Revocation List (CRL) using the Portal Interface Portal Interface

via Internet N Y

IS GS ES ED GT RSA

N Y

SMKI 22 Submit an Organisational Certificate Signing Request (CSR) using the Portal Interface

Portal Interface Y N IS GS ES

ED GT OU Y N

SMKI 23 Submit a Device Certificate Signing Request (CSR) via the Portal Interface


OU Y N

SMKI 24 Submit a Batch Device Certificate Signing Request (CSR) via the Portal Interface


OU Y N

SMKI 25 Submit a Device Certificate Signing Request via the Web Service Interface

Web Services Y N IS GS ES

OU Y N

SMKI 26 Submit Organisational Certificate Signing Request (CSR) using the Portal Interface

Portal Interface via Internet

Y N IS GS ES

ED GT N Y

SMKI 27 Submit Device Certificate Signing Request (CSR) via the Portal Interface


Y N IS GS ES N Y

SMKI 28 Submit a Batch Device Certificate Signing Request (CSR) via the Portal Interface


Y N IS GS ES N Y


(Working Document)

Page 11 of 46 v1.0


SMKI Number

Scenario Method of

Access SMKI

SMKI Repository

User Role DCC User

Non Gateway

user

SMKI 29 Download a copy of all ‘In Use’ Certificates via the SFTP Interface SFTP N Y IS GS ES

ED GT OU Y N

SMKI 30 Download a Daily Delta of Certificates via the SFTP Interface SFTP N Y IS GS ES

ED GT OU Y N

SMKI 31 Query SMKI Repository for Organisational Certificate via Portal Interface


ED GT OU Y N

SMKI 32 Query SMKI Repository for Organisational Certificate via Web Service Interface


ED GT OU Y N

SMKI 33 Query SMKI Repository for Device Public Key Certificate via the Portal Interface


ED GT OU Y N

SMKI 34 Query SMKI Repository for Device Public Key Certificate via the Web Service


ED GT OU Y N

SMKI 38 Download Zip files of Anchor Slot Certificates via the Portal Portal Interface

via Internet N Y

IS GS ES RSA

N Y

SMKI 48 Access Certificate Revocation List (CRL) using the Portal Interface Portal Interface N Y IS GS ES

ED GT OU Y N

SMKI 49 Access Certificate Revocation List (CRL) using the Security File Transfer Protocol (SFTP)

SFTP N Y IS GS ES

ED GT OU Y N

Table 1 SREPTS Matrix

Note: The numbering convention reflected in the column titled “SMKI Number” in Table 1 above aligns to the numbering convention used in the SRT Approach document. Any gaps in the numbering sequence are intentional.


(Working Document)

Page 12 of 46 v1.0


4.3 SREPT Initiation

4.3.1 Procedural Steps

The table below sets out the steps that must be undertaken during Initiation of the SREPT by both the DCC and the Party seeking to undertake SREPT and the timeframes within which such steps must be complete.

Ref When Action From To Information Required Method

4.3.1.1 60 working days (WD) prior to commencement of SREPT

Notify DCC of intention to undertake SREPT

Party DCC Party id (s)

Confirmation that notification provided to SECAS, User Role(s)

Test start date

Identity of Test Manager and contact details

By email: as per

Section 8.1 Party

Notification of Intention to Undertake Testing Template

4.3.1.2 Within 2 WD of receipt of the notification 4.3.1.1

Acknowledge request DCC Party Confirmation of:

Party id (s)

Test start date

DCC Test Manager contact

Date for SREPT initiation meeting

By email: as per

Section 8.2 DCC

Acknowledgement of Intention to Undertake Testing Template


Conduct SREPT Initiation Meeting

DCC Party DCC to provide guidance information on conducting SREPT, including clarification of test artefacts requirements and access to test environments

Meeting

4.3.1.4 In each week occurring within the period from 40 WD prior to start of testing

Provide progress report, demonstrating readiness to begin tests

Party DCC Test Readiness Report By email: as per

Section 8.4 Test

Readiness Report Template


(Working Document)

Page 13 of 46 v1.0



4.3.1.5 25 WD prior to start of testing

Provide test artefacts to support conduct of SREPT

Party DCC As agreed with DCC to include:

Test Plan incorporating the Testing Schedule

Requirements Traceability Matrix

Test Scripts

Test Data Plan

By e:mail as attachments

4.3.1.6 By 20 WD prior to start of testing

DCC complete review of test artefacts

DCC Party Details regarding any deficiencies in the test artefacts and a potential revised start date for testing provided – continue from 4.3.1.7

Or confirmation that test artefacts accepted – continue from 4.3.1.9



Party to provide revised documents

Party DCC Revised documents By e:mail as attachments


DCC complete review of revised test artefacts

DCC Party Details regarding any deficiencies in the test artefacts and a revised start date for testing provided – Regress and continue from 4.3.1.7

Or confirmation that test artefacts accepted – continue from 4.3.1.9



(Working Document)

Page 14 of 46 v1.0




1. Review Test Readiness Report and confirm the Entry Criteria for commencing testing in relation to the relevant User Role has been met

2. Confirm Start Date and Test Schedule for execution of tests by Party

DCC

Quality Gate meeting

Party Source: Test Readiness Report, Test Schedule

Output: Confirmation of Party readiness to proceed

Quality Gate review meeting

If there is any outstanding documentation presented at the Quality Gate Review the DCC could either;

Not provide approval of the Test Readiness Report and provide revised start date for testing (Regress in the process to 4.3.1.7); or

Provide provisional approval of the Test Readiness Report (and approval to proceed) with an understanding that the outstanding documentation would be provided before the start of testing;

Table 2 SREPT Initiation: Procedural Steps


(Working Document)

Page 15 of 46 v1.0


4.3.2 SREPT Entry Criteria

Each Party wishing to undertake SREPT must comply with (and provide evidence of complying with) the following criteria prior to entry into SREPT.

Relevant section of DCC Gateway Connection and Codes of Connection Document

DCC to confirm Stage 1 SRT successfully completed

Prior to start of test execution of SREPT scenarios, the DCC to confirm that the Party has successfully completed the RAPP and become an Authorised Subscriber of the test system

(As applicable) The Party must have identified the User Roles for which it wishes to undertake SREPT

All relevant test artefacts (as agreed with the DCC and set out in section 4.3.1.5, and 4.3.1.6) must have been produced by the Party and approved by the DCC. This includes the production of a Requirements Traceability Matrix (RTM) which has been reviewed and agreed by the DCC showing how the tests the Party is planning to execute relate to the SMKI & Repository Entry Process Test Scenarios

(unless it receives confirmation in writing from the DCC to state otherwise) The Party must comply with the procedural steps for initiating SREPT (as set out in Table 2 SREPT Initiation: Procedural Steps above)

The Party can provide evidence to the DCC that an appropriate level of resources are available to support the SREPT process

Pursuant to SEC 4 Section H14.25 where the DCC considers that the Party has not met the Entry Criteria, the DCC may either:

withdraw that parties right to undertake SREPT until such time as the DCC is satisfied that the Party meets the entry criteria; or

reschedule the test start date for the Party. In so doing, the DCC shall provide the earliest practicable alternative date.

Where the DCC is not satisfied that a Party meets the entry Criteria to commence testing, the Party may refer the matter to the Panel, pursuant to SEC 4 Section H14.25.


(Working Document)

Page 16 of 46 v1.0


4.4 SREPT Execution


The table below sets out the steps that must be undertaken during test execution by either the DCC or a Party seeking to undertake SREPT and the timeframes within which such steps must be complete.


4.4.1.1 SREPT Start Date or earlier (if appropriate)

Execute scenario SMKI 01 listed section 7.1 RAPP Scenario

Party DCC Submission of relevant forms As per RAPP document

4.4.1.2 SREPT Start Date or earlier (if appropriate)

Carry out completeness and discrepancy checks

DCC Party Verification of Organisational Identity – approval to execute SREPT scenarios as an Authorised Subscriber (Party)

As per RAPP document

4.4.1.3 In accordance with Test Schedule and 4.4.1.2 completion

Conduct SREPT Party Approved test artefacts As per test artefacts

4.4.1.4 Daily Basis Provide progress report to DCC

Party DCC Test execution dashboard and Daily Testing Issue Report By e:mail as attachment

4.4.1.5 SREPT execution complete

Provide Test Completion report

Party DCC SREPT completion report including: details of Test Scripts executed and Testing Issues resolved

By email: as per

Section 8.5 Test

Completion Report Template

Table 3 SREPT Execution: Procedural Steps


(Working Document)

Page 17 of 46 v1.0


4.4.2 SREPT Test Suspension/Resumption

During the execution of SREPT, the DCC or the Party may determine that there are grounds to suspend testing.

The duration of any suspension is dependent on the severity of the issue that caused the suspension and the estimated time to rectify.

If following a recommendation of a suspension to testing it is proposed that specific areas of testing can be resumed, that decision will be made and agreed by both the DCC and the Party based on available information.

4.4.2.1 Suspension Criteria

Where there is deemed reasonable grounds to do so, the DCC or Party may suspend testing; this may include any of the following;

Application components are not available as scheduled

A high severity Testing Issue prevents further useful testing from proceeding

A significant percentage of planned Test Scripts for a given day fail, taking Testing Issue severity and volume of tests into consideration which would generate root cause analysis to be undertaken to establish the cause. Testing Issues trending should also be used to determine any recommendation. The outcome of any root cause analysis activity may result in testing being suspended

Test Scripts to be executed are in a “blocked” status due to an identified Testing Issue

The Party has failed to comply with the procedural steps for executing SREPT

4.4.2.2 Test Resumption Criteria

Where testing has been suspended, either the DCC or the Party shall produce a test suspension report reflecting the cause of the suspension, and what actions are to be taken by whom and when in order for testing to resume – the ‘Test Resumption Criteria’.

Testing will only resume once the DCC or Party has demonstrated to either the DCC’s or Party’s satisfaction that the Test Resumption Criteria have been met, or where both the DCC and the Party agree that testing should resume.

4.4.2.3 Disputes regarding Test Suspension/Resumption

Any dispute regarding the suspension of testing may be referred to the SEC Panel for determination (which determination shall be final and binding for the purposes of this Code Subsidiary Document).

Where the DCC is not satisfied that a Party has met the Test Resumption Criteria, the Party may refer the matter to the SEC Panel for determination (which determination shall be final and binding for the purposes of this Code Subsidiary Document).


(Working Document)

Page 18 of 46 v1.0


Where a dispute regarding the suspension/resumption of testing is made, testing will not resume whilst the dispute is being heard by the SEC Panel, and/or until the Test Resumption Criteria are met.


(Working Document)

Page 19 of 46 v1.0


4.5 SREPT Completion


The table below sets out the steps that must be undertaken during test completion by either the DCC or Party and the timeframes within which such steps must be complete.


4.5.1.1 SREPT execution complete

Confirm receipt of notification of Test complete (Test completion report)

DCC Party SREPT completion report including: details of Test Scripts executed and Testing Issues resolved received

By email


DCC review completion report and confirm that SREPT concluded or further testing required

DCC Party SREPT completion report and supporting Test Data as requested by DCC 4.4.1.5 refers

Quality Gate review meeting

4.5.1.3 Within 2 WD of successful quality gate review meeting

Confirm Test Complete DCC Party Issue Test Completion Certificate By e:mail attachment

as per Section 9

Appendix E: TEST COMPLETION CERTIFICATE

Table 4 SREPT Completion: Procedural Steps


(Working Document)

Page 20 of 46 v1.0


4.5.2 SREPT Exit Criteria

The following Exit Criteria are to be met prior to a Party’s completion of and exit from SREPT:

All Test Results have been documented by the Party and evidence captured in the Party’s Test Management Tool and available to be provided to the DCC

All test issues identified during a Party’s test execution have been recorded in the Test Management Tool

o any Testing Issue generated by the Party as a result of SREPT has been fixed, verified by retest and closed

o any outstanding Testing Issue will be reviewed to confirm impact and resolution plan

o any outstanding Testing Issue count must not exceed those defined in Table 5 below

Severity Threshold for Outstanding Testing Issues

1 0

2 0

3 0

4 0

5 5

Table 5 Testing Issue Threshold

A Test Completion Report has been created by the Party and approved by the DCC

A quality gate review meeting has been held between the Party and the DCC, with progress approved by the DCC

A Test Completion Certificate has been issued to the Party by the DCC

Pursuant to SEC 4 Section H14.25, where the DCC considers that a Party has not met the exit Criteria, the Party may refer the matter to the Panel.

Where a dispute regarding whether a Party has met the SREPT exit criteria occurs, the SREPT Completion process will not resume whilst the dispute is being heard by the Panel, and/or until the SREPT Exit Criteria are met by the Party.

4.5.3 Quality Gate Review

A quality gate review will be held as part of the process for a Party to complete SREPT.

The quality gate review meeting is expected to be a checklist-driven event at which previously assembled and validated test artefacts generated by the Party will be


(Working Document)

Page 21 of 46 v1.0


presented to the DCC for formal assessment to support an informed decision as to whether the Party can be deemed to have completed SREPT.

A final decision regarding whether a Party has successfully completed SREPT will be provided to the Party no later than 2 working days after the date on which quality gate review meeting is held.

4.5.4 SREPT Test Completion Certificate

The SREPT Test Completion Certificate will be issued by the DCC to the Party.

The Test Completion Certificate will be issued after the quality gate review has concluded that the Party has met the SREPT Exit Criteria.


(Working Document)

Page 22 of 46 v1.0


5 Appendix A: Test Artefacts

The DCC and each Party will be required to produce and maintain a number of documents, dashboards and reports during the testing lifecycle as depicted in Figure 2 Test Documentation Hierarchy Error! Reference source not found.below.

Testing Execution

Quality Gate

Party Test Execution

Dashboard

Partyr Test Completion Report

Testing Completion

Entry Criteria

DCC/Party Test Readiness

Report

DCC Test Completion

Certificate

Party Daily Testing Issue

Report

Testing Initiation

Party Requirements

Traceability Matrix

Party Test Scripts

Party Test Data Plan

Partyr Test Schedule

Document

Party Test Plan Document

DCC SRPT Document

Authority to Proceed Process

Exit Criteria

Figure 2 Test Documentation Hierarchy


(Working Document)

Page 23 of 46 v1.0


5.1 Party Documents & Reports

5.1.1 Test Preparation Document Set

The following documentation must be produced by a Party before Testing commences:

Test Plan

Test Data

Test Schedule


Test Scripts

5.1.2 Reports and Dashboard

Table 6 Test Stage Supporting Documentation Set’ sets out the Reports and Dashboard that a Party must produce to demonstrate progress in preparing for and executing testing.

5.1.3 Test Readiness Report (TRR)

The Test Readiness Report will be produced by the Party and provides the DCC with the on-going capability to assess the progress the Party is making in preparing for testing and therefore the likelihood that test execution will commence on the planned date.

The report must be provided to the DCC by the Party on a weekly basis, commencing 40 working days prior to the start of Testing and must indicate progress against the following criteria:

Previous Test Phase/Stage Exit Criteria (if appropriate)

Party Test Management tool selected and available

Party RAID (Risk, Assumption, Issue and Dependency) log, including, for each risk

Priority (High, Medium, Low)

Action taken

Target close date

Overall RAG status (based on progress to plan)

Party Test Plan produced

Party Test Schedule produced

Party Requirements Traceability Matrix


(Working Document)

Page 24 of 46 v1.0


Total numbers of Requirements identified

Total number of testable requirements documented

Total number of testable requirements in progress

Total number of testable requirements not started

Total number of Requirements deemed not testable

Party Test Script complete to date – to reflect the following breakdown

Planned number of Test Scripts

Total number of Test Scripts produced to date

Total number of Test Scripts in progress

Total number of Test Scripts not started

% Test Data readiness by Party against planned Test Scripts

Readiness of Party Test Resources and Technical (support) Resource

Party test environment readiness – to include

Security code of connection satisfied

User roles identified, available and validated

All interfaces required to support testing validated

5.1.4 Test Execution Dashboard

The test execution dashboard will identify the Party’s progress when executing testing and will be provided in a format specified by the DCC. The dashboard must be updated by the Party and provided to the DCC on a daily basis once testing commences.

The dashboard will include the following details:

Name of Party and Party ids under test

Party Location of testing

Date and time test execution dashboard updated by Party

Total number of tests Party scheduled for execution and projected as a test execution glide path

Actual number of tests executed by Party (by test run) to date reflected on an incremental daily count including test results (passed, failed, blocked, not run)

Party summary of Testing Issues to include


(Working Document)

Page 25 of 46 v1.0


Total number of Testing Issues generated

Counts by status Open, Fixed, Closed etc

Counts by Severity 1, 2, 3 etc

Party Regression Test execution results

o relevant Party’s will be informed of any change to the system, the decision as to whether Regression testing is required to be undertaken as that change lays with the Relevant Party

o where Regression testing has been undertaken by the relevant Party the results of that testing must be reflected in the Test Execution Dashboard

Party summary progress against Exit Criteria

Party Top 5 risks and issues - to include any environment concerns; and

Party Overall RAG status (based on progress against testing schedule)

5.1.5 Test Completion Report

The Party will be required to produce a Test Completion Report and submit the draft to the DCC 10 working days prior to the test completion date. The finalised version of the Test Completion Report will be submitted to the DCC on completion of each test execution activity.

A Test Completion Report template will be provided by the DCC to ensure that all Party reports contain the same level of detail. The report will include:

Party Test approach and Scope of Testing Undertaken

Party Summary of the Test Results

Total number of tests originally scheduled for execution

Total number of tests executed

Displayed by test run to include

Overall results achieved

Passed, Failed, Blocked, Not Run

Any tests not run, blocked or not successfully executed end to end must be supported by an explanation.

Party Summary of Testing Issues


(Working Document)

Page 26 of 46 v1.0


Total number of Testing Issues generated

Total counts by status Open, Fixed, Closed etc

Total counts by Severity 1, 2, 3 etc

5.1.6 Test Traceability

To provide the DCC with a sufficient level of test assurance, all tests executed by each Party undertaking SREPT will be required to demonstrate full traceability as follows:

Each testable requirement captured in the Requirements Traceability Matrix must be linked to one or many Test Scripts

Each Test Script executed must be reflected in one or many test execution cycles

A record of each test executed and the results of that test

Where an executed test generates a Testing Issue;

o Each Testing Issue must be linked to the test that generated the Testing Issue

o Any subsequent retesting to validate a fix of Testing Issue carried out must be linked to the Testing Issue

o Each retest executed must reflect a result achieved as a result of execution


(Working Document)

Page 27 of 46 v1.0


Test Stage Supporting Documentation Set

S. No

Phase Description DCC Responsibility Party Responsibility When/Frequency Entry Criteria

Exit Criteria

Sign-Off Authority

1 Initiation Test Plan Document Review and Approve Produce and maintain Test Stage Entry Quality Gate and updated during execution as required in preparation for Test Stage Exit Quality Gate

Y Y DCC

2 Initiation Test Schedule Review and Approve Produce and maintain Test Stage Entry Quality Gate and updated during execution as required in preparation for Test Stage Exit Quality Gate

Y Y DCC

3 Initiation Requirements Traceability Matrix

Review and Approve Produce and maintain Test Stage Entry Quality Gate and updated during execution as required in preparation for Test Stage Exit Quality Gate

Y Y DCC

4 Initiation Test Scripts Review and Approve Produce and maintain Test Stage Entry Quality Gate and updated during execution as required in preparation for Test Stage Exit Quality Gate

Y Y DCC

5 Initiation Test Data Plan Review Produce and maintain Test Stage Entry Quality Gate and updated during execution as required in preparation for Test Stage Exit Quality Gate

Y N DCC

6 Initiation Test Readiness Review Checklist

Provide Template

Review and Approve

Produce and maintain Test Stage Entry Quality Gate Y N DCC

7 Initiation Test Stage Entry Criteria

Review and Approve Produce Test Stage Entry Quality Gate Y N DCC

8 Execution Test execution dashboard

Review Produce and maintain Test Stage Entry Quality Gate and updated daily during execution in preparation for Test Stage Exit Quality Gate

Y Y DCC


(Working Document)

Page 28 of 46 v1.0


Test Stage Supporting Documentation Set

9 Execution Daily Testing Issue Report

Review Produce and maintain Test Stage Entry Quality Gate and updated daily during execution in preparation for Test Stage Exit Quality Gate

Y Y DCC

10 Execution Test Completion Report Document

Provide Template

Review and Approve

Produce and file Test Stage during execution in preparation for Test Stage Exit Quality Gate

N Y DCC

11 Execution Test Stage Quality Gate Exit Criteria

Review and Approve Produce Test Stage Exit Quality Gate N Y DCC

12 Completion Authority to Proceed Approve Follow Once a Party has successfully passed the Test Stage Exit Quality Gate

N Y DCC

Table 6 Test Stage Supporting Documentation Set


(Working Document)

Page 29 of 46 v1.0


6 Appendix B: Test Data

Following successful completion of the RAPP process the DCC will provide the Party with relevant data (in the form of a token and PIN) to match the level of approval granted. This data will be used to support the SREPT scenarios.

During the development of the Test Data plan there may be additional data items identified that the DCC may be called upon to provide to support the execution of the SREPT Scenarios. (i.e. Organisation and Device Certificates)


(Working Document)

Page 30 of 46 v1.0


7 Appendix C: Test Scenarios

7.1 RAPP Scenario

As a pre-requisite to any individuals becoming an ARO and gaining access to SMKI Services and/or SMKI Repository Services, the organisational identify of Parties must be verified, scenario SMKI 01 supports that requirement.

7.1.1 Registration and Enrolment of Organisations and their representatives

ID SMKI 01

Title Complete Registration Authority Policies and Procedures (RAPP) Process

Description: A Party wishing to establish their organisational and individual identity with the DCC Registration Authority should follow the processes described in the Registration Authority Policies and Procedures (RAPP) document in order to acquire access to the SMKI using the SMKI Portal and also to the SMKI Repository.

Objective:

To prove the following process has been exercised in order to carry out tests within the test environment and test the process of enrolment of a party

o Completion and submission of relevant forms Verification of Organisational Identity Authorisation and Verification of Senior Responsible Officer (s) Authorisation and Verification of Authorised Responsible Officer (s)


(Working Document)

Page 31 of 46 v1.0


7.2 DCC User - SMKI & Repository Entry Process Test Scenarios

The following sub sections contain the SMKI & Repository Entry Process Test Scenarios that are applicable to each prospective DCC User.

The SMKI Repository referred to in the following SREPT test scenarios is the DSP Repository.

7.2.1.1 Security Credentials Access Tests to SMKI

ID SMKI 02

Title: Use security credentials to access the SMKI Portal

Description A Party representative with the user role of Authorised Responsible Officer (ARO) wishes to use their individual security credentials to access to the SMKI Service, through the SMKI Portal Interface via the DCC Gateway Connection, using the security credentials supplied by the DCC.

Objective To prove that the PKI Client Installed on ARO’s computer validates their access details

To prove that a Party authorised representative can use the FIPS token which is registered to them and their organisation when accessing the SMKI Portal Interface via the DCC Gateway Connection

ID SMKI 03

Title: Use Security Credentials to Access SMKI Service using the SMKI Web Service Interface

Description A Party System using the security credentials to access the SMKI Service, through the SMKI Web Service Interface, using the security credentials supplied to the DCC (validated with IKI (Infrastructure Key Infrastructure) certificate).

Objective To prove that the Organisation can access the SMKI Service through the SMKI Web Service Interface using the issued SMKI API

connection and valid API Key


(Working Document)

Page 32 of 46 v1.0


7.2.1.2 Security Credentials Access Tests to the Repository

ID SMKI 05

Title: Access the SMKI Repository via the SMKI Repository Portal Interface using secure log in and password

Description A Party representative with the user role of Authorised Responsible Officer (ARO) wishes to use their individual security credentials to access the Repository using the SMKI Repository Portal Interface via the DCC Gateway Connection.

Objective To prove that the Organisations representative can access the SMKI Repository using the SMKI Repository Portal Interface via the DCC

Gateway connection using the SMKI URL, log in details and password

ID SMKI 06

Title: Access the SMKI Repository via the SMKI Repository Interface using valid API Key

Description The Party system uses the API Key to access the Repository using the SMKI Repository Web Service Interface.

Objective To prove that the Organisation can access the repository using the SMKI Repository Web Service Interface with the issued SMKI API

connection and valid API Key

ID SMKI 07

Title: Access the SMKI Repository via the Secure File Transfer Protocol (SFTP) using username and password

Description The Party system using security credentials provided to the Authorised Responsible Officer (ARO) uses the credentials (provided for that use) to access the Repository using the SMKI Repository SFTP (Secure File Transfer Protocol) Interface.

Objective To prove that the Organisation can access the Repository using the SMKI Repository SFTP (Secure File Transfer Protocol) Interface with

the issued username and password


(Working Document)

Page 33 of 46 v1.0


7.2.2 Submission of Certificate Signing Requests

ID SMKI 22

Title: Submit an Organisational Certificate Signing Request (CSR) using the SMKI Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to submit an Organisational Certificate Signing Request (CSR) and receive Organisational Certificates using the SMKI Portal Interface via DCC Gateway Connection.

Objective

To prove a Party is capable of generating an asymmetric key pair and submit a Certificate Signing Request in the correct format as detailed in the SMKI Interface Design Specification

To prove that Certificate Signing Requests are in the correct format with true and accurate information

To prove that Parties can download their Certificates

ID SMKI 23

Title: Submit a Device Certificate Signing Request (CSR) via the SMKI Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to submit an Ad Hoc Certificate Signing Request (CSR) and receive Certificates for Devices through the SMKI Portal Interface via DCC Gateway Connection.

Objective

To prove a Party is capable of submitting a Certificate Signing Request in the correct format as detailed in the SMKI Interface Design Specification

To prove that Parties can download individual Device Certificates

Note: For GS User roles this scenario will be executed for Gas Proxy Function & Gas Meter Device


(Working Document)

Page 34 of 46 v1.0


ID SMKI 24

Title: Submit a Batch Device Certificate Signing Request (CSR) via the SMKI Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to submit a Batch Certificate Signing Request (CSR) and receive Certificates for Devices through the SMKI Portal Interface via DCC Gateway Connection.

Objective To prove a Party is capable of submitting a batch of Certificate Signing Requests in the correct format as detailed in the SMKI Interface

Design Specification

To prove that Parties can download their Device Certificates in zipped batches

ID SMKI 25

Title: Submit a Device Certificate Signing Request via the Web Service Interface

Description A Party System submits an Individual Device Certificate Signing Request (CSR) and receives Certificates for Devices via the Web Service Interface.

Objective To prove a Party is capable of submitting a Device Certificate Signing Request in the correct format as detailed in the SMKI Interface

Design Specification

To prove that Parties can receive the Device Certificates


(Working Document)

Page 35 of 46 v1.0


7.2.3 Collection of Documents and Information from the Repository

ID SMKI 29

Title: Download a copy of all ‘In Use’ Certificates via the SFTP Interface

Description The Party system using security credentials provided to the Authorised Responsible Officer (ARO) uses the credentials (provided for that use) to download a copy of all in use SMKI Certificates from the SMKI Repository via SFTP (Secure File Transfer Protocol) Interface.

Objective To prove a complete copy of all ‘In Use’ Certificates including OCA and DCA issued Certificates can be downloaded

To prove that the Party accesses the SFTP Interface using a file transfer client that supports SFTP in accordance with the standards in the SMKI Repository Interface Design Specification

ID SMKI 30

Title: Download a Daily Delta of Certificates via the SFTP Interface

Description The Party system using security credentials provided to the Authorised Responsible Officer (ARO) uses the credentials (provided for that use) to download a daily Delta of SMKI Certificates from the SMKI Repository via SFTP Interface.

Objective

To prove a partial/’Daily Delta File’ batch containing ‘In Use’ Certificates published to the SMKI Repository during the preceding twenty four hours can be downloaded up to seven days from publication

To prove that the Party accesses the SFTP Interface using a file transfer client that supports SFTP in accordance with the standards in the SMKI Repository Interface Design Specification


(Working Document)

Page 36 of 46 v1.0


7.2.4 Query the Repository and Retrieve Documents and Information

ID SMKI 31

Title: Query SMKI Repository for Organisational Certificate via the SMKI Repository Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to interrogate the SMKI repository via the SMKI Repository Portal Interface via the DCC Gateway Connection in order to prove that an Organisation Certificate is now contained in the SMKI Repository and that they can retrieve it.

Objective To prove a Party is able to query the SMKI Repository to determine the presence of their Certificate in the SMKI Repository

To prove a Party is able to download the located Certificate

ID SMKI 32

Title: Query SMKI Repository for Organisational Certificate via Web Service Interface

Description A Party System queries the SMKI Repository via the Web Services Interface in order to prove that an Organisation Certificate is now contained within the SMKI Repository and that it can be retrieved.

Objective To prove a Party is able to query the SMKI Repository using selection criteria to determine the presence of their Certificate in the SMKI

Repository

To prove the Party can receive the generated Certificate


(Working Document)

Page 37 of 46 v1.0


ID SMKI 33

Title: Query SMKI Repository for Device Public Key Certificate via the SMKI Repository Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to query via the SMKI Repository Portal Interface via DCC Gateway Connection whether a Device’s public key Certificate is in the SMKI repository.

Objective To prove a Party is able to query the SMKI Repository to determine the presence of a Certificate in the SMKI Repository

To prove a Party is able to download the located Certificate

ID SMKI 34

Title: Query SMKI Repository for Device Public Key Certificate via the Web Service

Description A Party System queries via the Web Service Interface whether a Device’s public key certificate is present in the SMKI repository and that it can be retrieved.

Objective To prove a Party is able to query the SMKI Repository to determine the presence of a Certificate in the SMKI Repository

To prove a Party can receive the generated Certificate


(Working Document)

Page 38 of 46 v1.0


7.2.5 Access Certificate Revocation List

ID SMKI 48

Title: Access Certificate Revocation List (CRL) using the SMKI Repository Portal Interface

Description A Party with the user role of Authorised Responsible Officer (ARO) wishes to access the CRL using the SMKI Repository Portal Interface via the DCC Gateway Connection.

Objective To prove that the Organisation’s ARO can access (download) the CRL using the SMKI Repository Portal Interface via the DCC Gateway

Connection

ID SMKI 49

Title: Access Certificate Revocation List (CRL) using the SecureFile Transfer Protocol (SFTP)

Description The Party system using security credentials provided to the Authorised Responsible Officer (ARO) uses the credentials (provided for that use) to access the CRL using the SFTP (Secure File Transfer Protocol).

Objective To prove that the Organisation’s ARO can access (download) the CRL using the Secure File Transfer Protocol (SFTP)


(Working Document)

Page 39 of 46 v1.0


7.3 Non-Gateway user - SMKI & Repository Entry Process Test Scenarios

7.3.1 Security Credentials Access Tests to SMKI

ID SMKI 04

Title: Accessing the SMKI Portal using security credentials

Description A Non-Gateway user representative with the user role of Authorised Responsible Officer (ARO) wishes to use their individual security credentials to access to the SMKI Service, through the Portal Interface via the Internet, using the security credentials supplied by the DCC.

Objective To prove that the PKI Client Installed on ARO’s computer validates their access details

To Prove the Party authorised representative can access the SMKI Service through the Portal Interface via the Internet using the FIPS token which is registered against them and their organisation


(Working Document)

Page 40 of 46 v1.0


7.3.2 Submission of Certificate Signing Requests

ID SMKI 26

Title: Submit Organisational Certificate Signing Request (CSR) using the Portal Interface

Description A Non-Gateway user with the user role of Authorised Responsible Officer (ARO) for Organisation Certificates wishes to submit an Organisational Certificate Signing Request (CSR) and receive Certificates using the Portal Interface via the Internet.

Objective To Prove that Certificate signing requests are in the correct format with true and accurate information

To prove that Non DCC Gateway users can download their Organisational Certificates

ID SMKI 27

Title: Submit Device Certificate Signing Request (CSR) via the Portal Interface

Description A Non-Gateway user with the user role of Authorised Responsible Officer (ARO) for Device Certificates wishes to submit an Ad Hoc Device CSR and receive the Certificates via the Portal Interface via the secured Internet connection.

Objective

To prove a Non-Gateway user is capable of submitting a Certificate Signing Request in the correct format as detailed in the SMKI Interface Design Specification

To Prove that Certificate Signing Requests are in the correct format with true and accurate information

To prove that a Non-Gateway User can be download their Device Certificates, to confirm the information contained in the resulting Certificate is consistent with the information contained within the corresponding CSR


(Working Document)

Page 41 of 46 v1.0


ID SMKI 28

Title: Submit a Batch Device Certificate Signing Request (CSR) via the Portal Interface

Description A Non-Gateway User with the user role of Authorised Responsible Officer (ARO) for Device Certificates wishes to submit a Batch Device CSR and receive the Certificates via the Portal Interface via the secured Internet connection.

Objective

To prove a Non-Gateway user is capable of generating a device asymmetric key pair and submit a Certificate Signing Request in the correct format as detailed in the SMKI Interface Design Specification

To Prove that Certificate signing requests are in the correct format with true and accurate information

To prove that a Non-Gateway User can resubmit further CSRs following rejection by the DCC

To prove that a Non-Gateway User can download their Device Certificates in zipped batches, to confirm the information contained in the resulting Certificates is consistent with the information contained within the corresponding CSRs

To prove that a Non-Gateway user can reject a Certificate


(Working Document)

Page 42 of 46 v1.0


7.3.3 Submit Requests for Repository Content and Access CRLs

ID SMKI 08

Title: Access Certificate Revocation List (CRL) using the Portal Interface

Description A Non-Gateway user with the user role of Authorised Responsible Officer (ARO) wishes to access the CRL using the Portal Interface.

Objective To prove that the Organisation’s ARO can access the CRL using the SMKI Portal Interface via the Internet

ID SMKI 38

Title: Download Zip files of Anchor Slot Certificates via the Portal

Description A Non-Gateway user with the user role of Authorised Responsible Officer (ARO) wishes to download zip files of anchor slot Certificates via the Portal Interface via the Internet.

Objective To prove Zip files of anchor slot Certificates can be downloaded


(Working Document)

Page 43 of 46 v1.0


8 Appendix D: Forms and Templates

Current versions of templates for the following documents can be found on the DCC Sharepoint site, accessible to authorised parties:

1. Party Notification of Intention to Undertake Testing Template 2. DCC Acknowledgement of Intention to Undertake Testing Template 3. Test Plan Template 4. Test Readiness Report Template 5. Test Completion Report Template

Please contact the DCC, to gain access to this site via [email protected]


(Working Document)

Page 44 of 46 v1.0


9 Appendix E: TEST COMPLETION CERTIFICATE

TEST COMPLETION CERTIFICATE

To: [Party]

From: [DCC]

[Date]

Dear Sirs,

TEST COMPLETE CERTIFICATE

[TEST]: [insert description, to correspond with relevant description]

We confirm that the relevant tests have been executed in accordance with the relevant Test Documents. We confirm that the relevant Exit Criteria have been achieved.

Please note that the granting by the DCC of this Certificate shall not result in a transfer of risk to the DCC in respect of any part of the Party Solution and the Services.

Yours faithfully

[Name]

[Position]

Acting on behalf of the DCC


(Working Document)

Page 45 of 46 v1.0


10 Appendix F: DEFINITIONS

Term Definition Source

Authorised Subscriber

Any Party which has successfully completed the SMKI and Repository Entry Process Tests) in respect of either Certificate Policy may apply to become an “Authorised Subscriber” in accordance with, and by following the relevant procedures set out in that Certificate Policy and the RAPP

Daily Testing Issue Report

The document reporting on the status of any testing issues identified

CRL Certificate Revocation List

CSR Certificate Signing Request

DCA Device Certificate Authority

(DCC) User A Party that has completed the User Entry Process (and, in respect of Services available in accordance with this Code to Users acting only in one or more User Roles, a Party that has completed the User Entry Process for that User Role)

SEC 4

Eligible Subscriber An Authorised Subscriber shall be known as an “Eligible Subscriber” in respect of a Certificate if it is entitled to become a Subscriber for that Certificate

Exit Criteria The criteria that must be satisfied before testing can be considered complete

SEC 4

Install & Commission

The process of installing and commissioning communications hub functions, gas proxy functions and smart meters with the DCC

Non-Gateway user any party that isn’t yet or doesn’t intend to become a DCC Gateway User

DECC

OCA Organisational Certificate Authority

Regression Testing

Testing to ensure that existing functionality of a system is not affected by the addition of new and modified functionality (or any other change to any related part of any System) introduced at any point in the service period (and Regression Test shall be construed accordingly)

RAPP Registration Authority Policies and Procedures


A matrix of defined requirements that provides traceability (linkage) to Test Scripts for the purpose of providing a measurement of test coverage as intended in the relevant specification

SMKI Smart Metering Key Infrastructure

SREPT SMKI and Repository Entry Process Tests

SREPTS SMKI and Repository Entry Process Test Scenarios

SRT SMKI and Repository Testing – A test stage within the System Integration Test (SIT) Phase that is undertaken prior to User Integration Test (UIT) Phase

SRTSD SMKI and Repository Test Scenarios Document (this document)

Test Completion Certificate

A certificate issued by the on successfully completes SREPT as set out in Appendix E: TEST COMPLETION CERTIFICATE


(Working Document)

Page 46 of 46 v1.0


Term Definition Source

Test Completion Report

A document summarizing testing activities and results. It also contains an evaluation of the corresponding test items against Exit Criteria

Test Data Any data constructed for the purposes of undertaking SMKI and Repository Entry Process Testing

Test Data Plan The document that sets out: the size and type/format of data, who is responsible for providing the data; and when the data is required to be available to support test activities in a Test Plan

Test Execution Dashboard

The document summarizing testing activities and results, produced at regular intervals, to report progress of testing activities against a baseline (such as the original test plan) and to communicate risks and alternatives requiring a decision to management

Test Exit Report A report that sets out the results of testing during the Test Stage and demonstrates the manner in which the Exit Criteria have been attained

Test Management Tool

A tool that has the ability to log and track Testing Issues

Test Plan A document describing the scope, approach, resources and schedule of intended test activities within a Test Stage that will be

produced as set out in Appendix 1

Test Result The consequence/outcome of the execution of a test script

Test Readiness Report

A report that when completed provides the capability to assess the status of test preparation and determine the readiness to proceed into test execution

Test Schedule A list of test process activities, tasks or events identifying their intended start and finish dates and/or times and interdependencies

Test Script A document specifying a sequence of actions for the execution of a test

Test Stage A group of test activities that are organised and managed together