Procedia Engineering 29 (2012) 2182 – 2186

1877-7058 © 2011 Published by Elsevier Ltd.doi:10.1016/j.proeng.2012.01.284

Available online at www.sciencedirect.comAvailable online at www.sciencedirect.com

Procedia Engineering 00 (2011) 000–000

ProcediaEngineering

www.elsevier.com/locate/procedia

2012 International Workshop on Information and Electronics Engineering (IWIEE)

Automatic Test Generation System for EFSM-Based Protocols

Ting Shu a,*, Qiangxin Cai b, Lianggui Liua, Yubo Jiaa

aCollege of Informatics and Electronics, Zhejiang SCI-TECH University, China b CECEP Industry Development CO.,LTD, China

Abstract

Protocol conformance testing based on Extend Finite State Machine (EFSM) is an open research topic. How to implement an automatic test generation system for EFSM-specified protocols is a challenging problem. This paper focuses on test generation system implementation techniques. Specifically, a feasible loosely-coupled system architecture and design idea is presented. Firstly, the four main functional modules of this system architecture and the relationships between them are introduced respectively in detail. Then, the entire test generation process of the system, divided into three key phases, is described according to the execution order of the system modules. Next, ATSGEN, an integrated automatic test sequence generation system specified in EFSM, is developed. Finally, a typical test example is given to demonstrate system features.

© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Harbin University of Science and Technology

Keywords: Protocol conformance testing; Test sequences generation; Extend finite state machine; Abstract test suites;

1. Introduction

Protocol conformance test is an experimental activity to judge whether or not a protocol implement under test (IUT) conforms to its specification standard. To avoid semantic ambiguities, the Extended Finite State Machine (EFSM) is introduced to describe specifications formally. Hence, the conformance test method based on EFSM has attracted much attention [1-3]. In the conformance test process, a tester can only stimulate an IUT using a set of inputs and observe the external outputs from the IUT to draw a

* Corresponding author. Tel.: +0-086-0571-86843323; fax: +0-086-0571-86843323. E-mail address: shuting@zstu.edu.cn.

2183Ting Shu et al. / Procedia Engineering 29 (2012) 2182 – 21862 Author name / Procedia Engineering 00 (2011) 000–000

test conclusion. The sequence of the input/output pairs is called as protocol conformance test sequences. Unfortunately, this can be expensive and difficult to generate test sequences manually. Thus, how to develop an automatic test sequences generation system for EFSM-specified protocols has become an urgently problem to be solved.

Several typical methods and test generation tools, such as UPPAAL Tron [4], Phact [5], Conformance Kit [6], have been proposed for generating EFSM-based protocol conformance test sequences. However, these research results are seldom applied in the network communications industries [1]. Therefore, there is a strong need to make more effort by researchers to narrow the big gap between academic results and testing practice. In this paper, we discuss the test generation system implementation techniques. Then, a loosely-coupled system architecture and design idea based on a data center is presented. Based on these methods, an Automatic Test Sequences Generation system (ATSGEN) is developed. The feasibility of the proposed system structure and design idea is shown through a simple test generation experiment.

2. ATSGEN system architecture and design

According to the protocol conformance testing standard ISO9646, we design the general architecture of ATSGEN (shown in Figure 1). The system framework consists of four main functional modules, Protocol Model Construction and Information Extraction (PMCIE), Automatic Protocol Conformance Test Sequence Generation (APCTSG), Protocol Execution Simulation (PES) and Automated TTCN-3[7] Test Case Generation (ATTCG). These modules interact with each other and contribute to accomplish system function. The data interaction center of these modules is a local relational database system. In this way, ATSGEN can be constructed based on the loosely coupled architecture, which contributed to the good extension feature of the system. The following is a detail description of the design idea and the key function of each module.

Fig. 1. ATSGEN system architecture

2184 Ting Shu et al. / Procedia Engineering 29 (2012) 2182 – 2186 Author name / Procedia Engineering 00 (2011) 000–000 3

2.1. PMCIE module

This module is to fulfil two tasks: building the protocol formal model and making the information extraction from an EFSM model. For the former, protocol specification under testing is modelled using the Rational Rose tool with any one of two modes: indirect mode and direct mode. In indirect mode, testers can make ATSGEN run the Rose to build the formal model. In direct mode, testers can also run independently Rose to develop a model. As a result, a protocol model is generated, which is saved as a file with the mdl extension. Then, the file can be imported into ATSGEN for testing preparation.

After the establishment of the formal model, for the latter, we need extract the necessary information from it for the next test sequences generation. This information includes states, variables, transitions and the relations between them. Meanwhile, this module also need support the transition condition analysis and variables computation in the process of the test generation algorithm execution.

2.2. APCTSG module

APCTSG is the core module of ATSGEN. Conformance abstract test sequences can be automatically generated in this module. It implements three types of test generation algorithms [8]: control flow test criteria (CFT), data flow test criteria (DFT), control-flow and data-flow coverage criteria (CDC). For each test criteria, it also supports three testing generation modes respectively: test sequence generation, test generation and intermediate results storage, test generation and simulation. A variety of testing generation modes is intended to provide convenience for testing simulation. In the first operation mode, APCTSG only generate the conformance testing sequence, but not save the intermediate results related with testing simulation into the data center. By reducing the database reading and writing operations, the testing generation efficiency can significantly be improved in this mode. In the other two modes, all the intermediate data are stored into the data center, which is needed for the subsequent testing simulation process. But only in the third mode, ATSGEN will automatically load the testing simulation sub-application immediately after the test sequence generation.

2.3. PES module

In this sub-application, the dynamic characteristics of the protocol EFSM model under testing will be presented in the form of executable directed spanning tree (EDST) [8]. Based on the EDST technique, PES provides two types of testing simulation: Protocol EFSM Execution Simulation (PEES) and Testing Sequences Generation Simulation (TSGS). In PEES, tester can subjectively set the initial state configuration of the EFSM as the root of EDST. And then, an EDST is built step by step with each time a mouse click on a node. By observing the generated EDST, we can trace the execution path of the EFSM and validate the established EFSM. In TSGS, a partial EDST is automatically rebuilt based on the intermediate data of the testing sequence generation process stored in the data center.

2.4. ATTCG module

This module’s main task is that translate the test sequence generated in APCTSG into the abstract test suite (ATS) described by TTCN-3 language. In the test suite execution phase, we need a TTCN-3 compiler to compile the ATS into the executable test suite (ETS). However, TTCG is only responsible for generating the ATS in TTCN-3, not for compiling the ATS into ETS. The compilation of the ATS is to be done by a third party compiler, such as the TTCN-3 compiler of TTworkbench.

2185Ting Shu et al. / Procedia Engineering 29 (2012) 2182 – 21864 Author name / Procedia Engineering 00 (2011) 000–000

3. Test generation process of ATSGEN

As shown in the Figure 2, the entire test generation procedure consists of three main steps. In the testing preparation phase, first of all, a protocol UML model is needed to be built using the Rose. And then, the information related with test generation is extracted from the state machine model in the generated model file. Next, the PMCIE reformulates a new protocol EFSM model in memory. Following with the preparation phase, ATSGEN enters the testing generation phase. In this phase, ATSGEN will automatically generate the conformance testing sequences according to the optional test generation modes and algorithms. After the testing generation, multiple sets of the result test sequences may be generated. In the last phase, testing suites TTCN-3 description phase, these generated test sequences will be translated into the corresponding ATS in TTCN-3 core language.

Fig. 2. Test generation process of ATSGEN

In the process of the testing sequence generation, APCTSG plays an important role in supporting for the model validation and testing sequence generation process supervision. Just after the model built, we can validate the model using the PEES. And then, before the testing sequences converted into TTCN-3, we also can supervise the process of the test sequence generation through the EDST technique in the TSGS. Ether in PEES or TSGS, if an error is found in the process of the simulation, we can continue to modify the corresponding EFSM model or correct the testing algorithm. This process will be repeated until the results meet our requirements.

4. A simple testing generation example

The initiator module of INRES protocol is a widely discussed example in the protocol testing field. So we also choose the INRES as an example to show ATSGEN’s some features.

In testing preparation phase, ATSGEN runs the Rose tool to build a initiator module of the modified INRES protocol model as demonstrated in Fig.3(a), which is saved as the model file “INRES.mdl”. Then, all necessary information will be extracted from this model file to rebuild an EFSM model in the system memory. In testing generation phase, we select the CDC criteria as the conformance test generation algorithm. Then, the state “Disconnect” is selected as the initial state and all context variables in the EFSM model are assigned the corresponding value of zero or false according to the type of each one. Finally, the APCTSG module is stimulated to generate the test results as shown in Fig.3 (b). Due to space limitation, we do not present the process of test suites TTCN-3 description and testing simulation here.

2186 Ting Shu et al. / Procedia Engineering 29 (2012) 2182 – 2186 Author name / Procedia Engineering 00 (2011) 000–000 5

5. Conclusion

This paper introduces the definition of protocol conformance test and test automatic generation system design and implementation techniques. A conformance test generation system, ATSGEN, based on EFSM with a loosely-coupled structure is developed. The structure of ATSGEN is divided into four main modules to illustrate respectively. These modules interact with each other through a data center. The entire workflow of testing generation in ATSGEN is also described step by step. Finally, based on the modified Initiator module in INRES protocol, a test generation example is given to show the feasibility of our system to generate abstract conformance test sequences.

Fig. 3 (a) Initiator module of the modified INRES protocol model in Rose; (b) Results of the testing sequences generation

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61101111 and No. 61002016), Science Foundation of Zhejiang Sci-Tech University（ZSTU）under Grant No.1004839-Y.

References

[1] Lai, R., A survey of communication protocol testing. Journal of Systems and Software, 2002. 62(1): p. 21-46.

[2] Kalaji, A.S., R.M. Hierons and S. Swift, An integrated search-based approach for automatic testing from extended finite state

machine (EFSM) models. Information and Software Technology, 2011. 53(12): p. 1297-1318.

[3] Petrenko, A., S. Boroday and R. Groz, Confirming configurations in EFSM testing. IEEE Transactions on Software

Engineering, 2004. 30(1): p. 29-42.

[4] Larsen K G, Mikucionis M, Nielsen B, et al. Testing real-time embedded software using UPPAAL-TRON: an industrial case

study. EMSOFT 2005 , Jersey City, NJ, USA : ACM, 2005: 299-306.

[5] Feijs L, Meijs F, Moonen J R, et al. Conformance testing of a multimedia system using PHACT. Proceedings of the IFIP

TC6 11th International Workshop on Testing Communicating Systems, Tomsk, Russia: Kluwer, B.V., 1998: 193-210.

[6] Van De Burgt S P, Kroon J, Kwast E, et al. The RNL Conformance Kit. Proceedings of the 2nd International Workshop on

Protocol Test Systems, North Holland: 1989: 279-294.

[7] Etsi ES 201 873-1: The Testing and Test Control Notation version 3, Part1: TTCN-3 Core notation V3.2.1. 2007.

[8] T Shu, SQ Sun, HN Wang, WQ Xu, Test sequence generation of a communication protocol by an heuristic state

configuration exploration. Journal of Beijing University of Posts and Telecommunications, 2009. 32(6): p. 120-124.