Automated Evaluation of Runtime Object States Against Model-Level States

for State-Based Test Execution

Frank(Weifeng) Xu, Gannon University

Dianxiang Xu, North Dakota State University

Overview Introduction Objectives State evaluation infrastructure Case study Experiments/Demo Conclusions

Introduction State- based testing process Evaluation of runtime object states against

the model-level states defined in a state model is critical to state-based test automation. Manually keep track of the state of the running

objects is difficult Mapping from runtime object states to abstract

states is time consuming

Objectives This paper presents a state evaluation

framework to support the automated state-based test execution process. keeping track of the state of running objects mapping the states to abstract states firing a warning message if the abstract state does

not match model level states

Challenges How does the evaluation framework monitor

and collect the states of running objects? How does the monitor device interact with yet

not depend on a particular IUT? How can we automatically map the runtime

object states to abstract states in a state model?

How can the test driver get informed if the actual state and the expected state do not match?

Approach We take advantage of the pointcut mechanism

in aspect-oriented programming and implement the framework in AspectJ.

We demonstrate the framework by a case study We conduct a series of empirical studies to

evaluate the framework by calculating and comparing the total consumed minutes of mapping states and checking the oracle in term of manual and automated execution.

The experiment results show that the evaluation framework is much more effective and efficient than manual evaluation.

State evaluation infrastructure

Case study

Expected state

We need run time objects

pointcut setter(State s): call(void State.set*(*)) && target(s); /** * Advice to get the property change event fired when the setters are called. It's around advice because you need the old value of the property */ after(State s): setter(s) { //get the name of the property. Each state is stored in one //property String propertyName = thisJoinPointStaticPart. getSignature().getName().substring("set".length()); //get expected states String expectedState = s.getState(); //get current states String currentState = (String) ht.get(expectedState.substring(0, expectedState.indexOf("."))) //fire the event if those states are different firePropertyChange(s, propertyName, currentstate, expectedState); }

Figure 7. The pseudo code of a JavaBean fires events if the expected and actual states are different

Experiments Two groups of students

Group 1 students manually monitor and map states

Group 2 student use the framework 5 application

Conclusions We have proposed a novel approach to

automatically evaluation runtime object states to against model-level states for State-Based Test Execution. The framework is able to automatically keep track of

the properties of running objects, converting the properties to corresponding states in

state models and comparing whether or not the states match the expected states.

The framework is essentially an extension of observer design pattern implemented by enterprise JavaBeans. We take advantage of the pointcut mechanism of

AspectJ to facilitate the state monitoring ability.

Discussion We are assuming that all the state models we

are using for testing execution are correct. We consider state abstractions but never

action abstractions (input/methods to call). This makes things easier because these would have to be concretized.