DOCUMENT #: GSC15-PLEN-51

FOR: Presentation

SOURCE: ITU

AGENDA ITEM: Plenary 6.10

CONTACT(S): Reinhard.Scholl@itu.int

Conformance and Interoperability Testing

Reinhard SchollITU

Global Standards Collaboration (GSC) GSC-15

Highlight of Current Activities (1/3)

ITU-T SG 11: Lead study group on test specifications

ITU-T SG 15: Has extensive testing ecosystem ITU-T SG 16: its IPTV standards (H-series) was

put to test in ITU-T’s first interop event (20-23 July)• 3 IPTV testing specs approved; others further developed

ITU-T SG 17: Lead study group on languages and description techniques• Also: Q14/17 on Testing methodology, specification language and

Generic test suites for CIT

2

Highlight of Current Activities (2/3)

ITU-T JCA-CIT: Joint Coordination Activity on Conformance and Interoperability Testing• Works with all Study Groups to:

address the topics relevant to conformance and interoperability testing

enhance the quality of ITU-T Recommendations• Eliminate errors, ambiguities unverified behaviours

• Include sufficient detail for implementation and testing

• Include interfaces needed for interconnection and testing

develop methodologies and frameworks for testing

3

Highlight of Current Activities (3/3)

Regional ITU consultations on conformance assessment and interoperability• Quito, 8 July 2010

• Nairobi, 30-31 July 2010

• Sydney, 16-17 Sep. 2010

Pilot conformity database developed ITU Interop started

• IPTV (July in Geneva; Sep. in Singapore; Dec. in India)

• Home networking

4

Strategic Direction WTSA-08 Res. 76 on conformance & interop … supported & complemented by WTDC-10

Res. 47 (ITU-D) ITU Council-09 mandated

• a pilot conformity assessment database • interoperability events • human resources capacity building • assistance in the establishment of test facilities in

developing countries • a business plan for long-term implementation

5

Challenges

Regional ITU consultations on conformance assessment and interoperability are a useful discussion platform, see e.g.• Presentations at Kenya workshop, in particular

TSB’s overview presentation on status of implementation of WTSA-08 Res. 76

6

Next Steps/Actions

Continue implementing steps outlined under “Strategic directions”

Study groups to continue working on test specs Keep momentum of ITU’s IPTV Interop event

• Expand to other ITU standards, in collaboration with partners as appropriate

Populate conformance database

7

Supplementary Slides

8

ITU-T SG 11’s Questions on Testing

ITU-T SG 11: Signalling, protocols and test specs• Q 8/11   Protocol Test Specifications for NGN  • Q 9/11   Monitoring parameters for NGN

protocols  • Q 10/11   Service test specification for NGN  • Q 11/11   QoS tests specification for NGN  • Q 12/11   NID and USN test specification 

ITU-T SG 11 Approved Standards Q.3900 Methods of testing and model network architecture for NGN

technical means testing as applied to public telecommunication networks  

Q.3901 Distribution of tests and services for NGN technical means testing in the model and operator networks  

Q.3902 Parameters to be monitored in the process of operation when implementing NGN technical means in public telecommunication networks  

Q.3903 Formalized presentation of testing results   Q.3904 The scenarios, list and types of tests for TM local and NUT

testing for IMS on the model networks   Q.3910 Monitoring parameters set for NGN protocols   Q.3911 Monitoring parameters set for voice services in NGN

10

11

ITU-T SG 15: “Optical transport networks and access network infrastructures”

Tight collaboration:• Access (DSL, PON):

Broadband Forum

• Optical transport, framing, cross connecting Optical Internetworking Forum (OIF) Metro Ethernet Forum (MEF)

• Home networking HomeGrid Forum Broadband Forum

Nairobi, Kenya, 30-31 July 2010

ITU-T SG 15’s Extensive Testing Ecosystem

Develop conformance testing methodology and framework

Develop interoperability testing methodology and framework

Assist standards developers for applying conformance and interoperability testing methodology to their specific needs

Progress work in the area of Testing and Test Control Notation, version 3 (TTCN-3)

ITU-T SG 17’s Testing languages, methodologies and framework

ITU-T SG 17’s approved standards Conformance Testing Methodology and Framework

X.290 - General Concepts X.291 - Abstract Test Suite Specification X.292 - Tree and Tabular Combined

Notation X.293 - Test Realization X.294 - Requirements on Test

Laboratories and Clients X.295 - Protocol Profile Test Specification X.296 - Implementation Conformance

Statements X Suppl. 4 - Supplement on generic

approach to interoperability testing X Suppl. 5 - Supplement on

interoperability testing framework and methodology

Testing and Test Control Notation version 3 (TTCN-3) Z.161: TTCN-3 core language Z.162: TTCN-3 tabular presentation

format (TFT) Z.163: TTCN-3 graphical presentation

format (GFT) Z.164: TTCN-3 operational semantics Z.165: TTCN-3 runtime interface (TRI) Z.166: TTCN-3 control interface (TCI) Z.167: TTCN-3 mapping from ASN.1 Z.168: TTCN-3 mapping from CORBA

IDL Z.169: TTCN-3 mapping from XML data

definition Z.170: TTCN-3 documentation comment

specification

14

Imp

lem

en

tatio

n o

f the IT

U

Con

form

ity

Pro

gra

mm

e

ITU-T Recs & test suites

Supplier’s conformity

routedecision

Test lab (certification body

responsibility)

1st 2nd or 3rd party accredited lab

(ISO/IEC 17025)

ITU Conformity Database

Test results(ITU-T X.290)

1st partyEvaluation

Supplier’s ConformityDeclaration

(ISO/IEC 17050)

Test results(ITU-T X.290)

Accredited(ISO/IEC Guide 65) Certification body

Evaluation

ConformityCertificate issued by

CertificationBody

Con

form

ity A

ssessm

en

t / C

ertifi

catio

nIT

U C

IP s

erv

ices

Supplier’s Request to ITU

Route 1

Route 2

15

Supplier’s Declaration of Conformity – SDoC 1/2

16

Supplier’s Declaration of Conformity – SDoC- 2/2

Putting to the test: ITU-T’s IPTV suite of standards: H.721 et al.

Geneva, 20-23 July: testing, followed by showcasing Participating companies highly satisfied: 4.5 rating (out of 5) Event highlight on YouTube, ITU website:

http://youtube.com/watch?v=CpJmI_5aQrA

ITU press release: http://itu.int/net/pressoffice/press_releases/2010/31.aspx

Photos on Flickr: http://flickr.com/photos/itupictures

ITU’s 1st Interop Event (1/2)

Test report will be fed into the standards development process

H.721 Conformity Check List

H.762 Conformity Check List

H.770 Conformity Check List

H.750 Conformity Check List

ITU’s 1st Interop Event (2/2)

Testing specs:• HSTP.CONF-H721• HSTP.CONF-H762• HSTP.CONF-H770