An approach to testing conceptual schemas

Data & Knowledge Engineering 69 (2010) 598–618

Contents lists available at ScienceDirect

Data & Knowledge Engineering

journal homepage: www.elsevier .com/ locate/datak

An approach to testing conceptual schemas

Albert Tort *, Antoni OlivéUniversitat Politècnica de Catalunya (UPC), Campus Nord, Edif. Omega, Jordi Girona 1-3, 08034 Barcelona (Catalonia), Spain

a r t i c l e i n f o

Article history:Received 28 July 2009Received in revised form 8 February 2010Accepted 11 February 2010Available online 19 February 2010

Keywords:Conceptual modelingTestingRequirements validationUML/OCL

0169-023X/$ - see front matter � 2010 Elsevier B.Vdoi:10.1016/j.datak.2010.02.002

* Corresponding author. Tel.: +34 938990106.E-mail addresses: [email protected] (A. Tort), a

a b s t r a c t

Conceptual schemas of information systems can be tested. The testing of conceptual sche-mas may be an important and practical means for their validation. We present a list of fivekinds of tests that can be applied to conceptual schemas. Two of them require schemascomprising both the structural and the behavioral parts, but we show that it is possibleand useful to test incomplete schema fragments, even if they consist of only a few entityand relationship types, integrity constraints and derivation rules. We present CSTL, a lan-guage for writing automated tests of executable schemas written in UML/OCL. CSTLincludes language primitives for each of the above kinds of tests. CSTL follows the styleof the modern xUnit testing frameworks. We describe a prototype implementation of a testprocessor, which includes a test manager and a test interpreter that coordinates the execu-tion of the tests. Tests written in CSTL can be executed as many times as needed.

� 2010 Elsevier B.V. All rights reserved.

1. Introduction

A conceptual schema defines the general knowledge that an information system needs to know in order to perform itsfunctions [1]. Two fundamental quality properties of a conceptual schema are correctness (i.e. the knowledge defined is truefor the domain) and completeness (i.e. all relevant knowledge is defined) [2,3]. The main purpose of conceptual schema val-idation is to ensure correctness and completeness. In this paper we focus on testing as a means for conceptual schema val-idation, and we describe an approach to conceptual schema testing.

Several techniques are used to achieve correct and complete conceptual schemas. These include paraphrasing in naturallanguage [4–7], generation of abstractions and abstracts [8], explanation generation [9–11], constraint acquisition [12] andsimulation and animation [13–17]. The approach to conceptual schema testing described in this paper belongs to the cate-gory of simulation and animation, and it can be used in conjunction with the other techniques. Our approach can also beused in conjunction with verification techniques aimed at checking structural properties or finding out inconsistencies.

Our approach applies modern principles and techniques developed in the software testing field to conceptual schemas.The need for and the importance of software testing are undisputed [18]. We adopt here the precise and concise definitionof testing proposed by Meyer: ‘‘To test a program is to try to make it fail” from which the goal of testing becomes ‘‘to uncoverfaults by triggering failures” [19]. Many other validation and verification techniques are used or are in research and devel-opment, but, in professional practice, testing continues to be the dominant technique.

Currently, most work in conceptual modeling assumes that conceptual schemas are executable, and that therefore theyare software [20–22]. Then, a question naturally arises: how can we test conceptual schemas? In this paper, we try to clarifywhat it means to test conceptual schemas, and we present a language for writing tests of conceptual schemas as a means fortheir validation.

. All rights reserved.

[email protected] (A. Olivé).

http://dx.doi.org/10.1016/j.datak.2010.02.002

mailto:[email protected]

mailto:[email protected]

http://www.sciencedirect.com/science/journal/0169023X

http://www.elsevier.com/locate/datak

A. Tort, A. Olivé / Data & Knowledge Engineering 69 (2010) 598–618 599

Most of the work in software testing assumes that the system under test (SUT) consists of programs (objects, compo-nents) that provide only a set of operations, and testing a SUT means calling those operations with appropriate contextand input parameters, and checking that they return the expected outputs. For example, the recent UML testing profile(UTP) is based on this assumption [23,24] and the same happens in popular testing frameworks like JUnit.

If a conceptual schema were like an ordinary program, then its testing would not be very different from testing a program.However, a conceptual schema is knowledge or, more precisely, it is the general knowledge that an information systemneeds to know about the domain and about the functions it has to perform [1]. An executable conceptual schema can beconsidered a program only when there is a general-purpose information processor (virtual machine) able to behave accord-ing to the structural and behavioral rules defined in the conceptual schema [25]. Consequently, we may find some similar-ities between testing a program and testing a conceptual schema, but there are important differences. The first contributionof this paper is a list of five kinds of tests that can be applied to conceptual schemas regardless of the conceptual modelinglanguage. These kinds of tests are the assertions of (1) the consistency of the information base (IB); (2) the inconsistency ofthe IB; (3) the occurrence of a domain event; (4) the non-occurrence of a domain event, and (5) the contents of an IB state.

Testing conceptual schemas is as important as testing programs in projects that follow the OMG’s model-driven devel-opment (MDD) approach [26,27] when the transformation from platform-independent models (PIM) to platform-specificmodels (PSM) is fully automatic and correct-by-construction. Our PIMs are complete conceptual schemas that include allstructural and behavioral aspects, and the obtained PSMs include all aspects related to database design, integrity constraintsenforcement procedures, derivation rules procedures and the operations that correspond to the behavioral aspects. In theMDD approach, the testing activity is done at the conceptual schema level and, given that the transformation is correct-by-construction, there is no need to test again the outputs from that transformation.

Testing conceptual schemas is also important in those projects in which the quality [2,28–31] of the conceptual schemamust be maximal, even if the transformation to the PSMs will be manual and, therefore, there will be the need to test againthose PSMs. In particular, the maximal quality is an important objective when the conceptual schema is the basis for thedevelopment of several information systems to be done by independent project teams working in parallel, when the concep-tual schema is going to be used as a reference model for several applications, or when the conceptual schema is a metasche-ma to be instantiated in several applications. Testing conceptual schemas may also be useful in conceptual modelingeducation as a means for students to check that their conceptual schemas do in fact represent the knowledge they intendto define.

Even if we deal with complete conceptual schemas, we think that it makes sense to test incomplete conceptual schemaswhile they are developed, as a means of increasing their quality [2]. Even small fragments consisting of a few entity and rela-tionship types, integrity constraints and derivation rules can be tested in order to uncover faults during the early stages ofthe development of a conceptual schema. This fact lays the groundwork for the future development of a test-driven concep-tual modeling methodology, similar to the popular test-driven development [32].

The second main contribution of this paper is the conceptual schema testing language (CSTL), a specialized language forwriting automated tests [33] of executable conceptual schemas written in UML/OCL. CSTL makes it possible to write pro-grams to test conceptual schemas in the style of the modern xUnit software testing frameworks [34], which are successfullyused in many programming projects. CSTL enables the automation of tests of conceptual schemas, that is, the use of softwareto control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test precondi-tions, and other test control and test reporting functions [35].

CSTL includes the usual instantiation, assignment, conditional and iteration statements needed to write test cases, in thestyle of the ASSL language [36], but it also includes built-in constructs that correspond to the elements found in the xUnitframeworks, and the formalization of test assertions [34,37] that make the tests automatically executable as many times asneeded.

The structure of the paper is as follows. In the next section, we briefly review the main concepts and the notation used todefine conceptual schemas. In Section 3, we present the above mentioned five kinds of tests that can be applied to conceptualschemas and the corresponding CSTL statements. Section 4 extends the previous two sections with the temporal constraintsand derivation rules. Section 5 describes the overall structure of the CSTL language. The kinds of tests and the CSTL languageare illustrated by means of application to the conceptual schema of the osCommerce system [38], a popular industrial e-commerce system. All of the examples in this paper are taken from this case study. The full details of the case study canbe found in the report [39]. Section 6 describes our third contribution: a prototype of a conceptual schema testing environ-ment. In this prototype, tests written in CSTL may be automatically executed as many times as needed. Section 7 discussesrelated work. Finally, Section 8 summarizes the conclusions and suggests further work.

2. Basic concepts and notation of the conceptual schemas

In this section, we review the main concepts and notation we have used to define the conceptual schemas under test. Theconcepts and notation used for temporal constraints and derivation rules will be introduced in Section 4. We adopt UML/OCLas the conceptual modeling language, but the ideas presented here can also be applied to schemas in other languages.

A conceptual schema consists of a structural (sub)schema and a behavioral (sub)schema. The structural schema consistsof a taxonomy of entity types (a set of entity types with their generalization/specialization relationships and the taxonomic

600 A. Tort, A. Olivé / Data & Knowledge Engineering 69 (2010) 598–618

constraints), a set of relationship types (either attributes or associations), the cardinality constraints of the relationship typesand a set of other constraints formally defined in OCL [40]. We assume multiple classification (an entity may be an instanceof several types not related by IsA relationships). Entity and relationship types may be base or derived. The population of thebase entity and relationship types is explicitly represented in the information base (IB). If they are derived, there is a formalderivation rule in OCL that defines their population in terms of the population of other types.

The behavioral schema consists of a set of event types. We adopt the view that an event can be modeled as a special kindof entity, which we call event entity [41]. An event entity is an instance of an event type. There are several kinds of eventtypes, but in this paper, for expository reasons, we will only deal with domain event types. Given that there is a direct cor-respondence between events and invocations of system operations, the adaptation of our work to languages that view eventsas invocations of system operations is straightforward [42].

Event types have characteristics, constraints and effects. The characteristics of an event are the set of relationships (attri-butes or associations) in which it participates. The constraints are the conditions that events must satisfy in order to occur.An event constraint involves the characteristics and the state of the IB before the event occurrence. An event may occur inthe state S of the IB if S satisfies all constraints and the event satisfies its event constraints. Each event type has an operationcalled effect() that gives the effect of an event occurrence. The effect is declaratively defined by the postcondition of the oper-ation. We define both the event constraints and the postcondition in OCL.

For domain event types, the postcondition defines the state of the IB after the event occurrence. It is assumed that thestate of the IB after the event occurrence also satisfies all constraints defined over the IB. Therefore, the effect of a domainevent is a state that satisfies the postcondition and all IB constraints. Note that the OCL expressions used in constraints, der-ivation rules and pre/post conditions are without side-effects. These expressions are evaluated over the IB and their evalu-ation cannot change the IB.

Given that we want to deal with executable conceptual schemas, we also need a procedural specification of the method ofthe effect() operation. An execution of an effect() operation can change the IB. A method is correctly specified if the result itproduces always satisfies the postcondition and the IB constraints. UML does not include any particular language for writingmethods [43]. In the work reported here, we write the methods of the effect() operations using a subset of the CSTL.

3. Testing conceptual schemas

In this paper, we adopt the UTP’s terminology and consider that a test case is a ‘‘specification of one case to test the sys-tem including what to test with, which input, result, and under which conditions. . .. A test case always returns a verdict.” Theverdict may be pass, fail, inconclusive or error [24]. We consider that the verdict is Error when the conceptual schema or thetest case is ill-formed (i.e. it is not a valid instance of the corresponding metaschema). Other cases are explained below.

In CSTL, the basic construct for testing conceptual schemas is a concrete test case.Each concrete test case has a name and consists of a set of statements:

test testName {

. . .

assert . . .

}

The last statement of a concrete test case is an assertion, but in general there may be several assertions in the same testcase. The verdict of a concrete test case is Pass if the verdict of all of its assertions is Pass. The objective of the conceptualmodeler is to write test cases whose final verdict is Pass.

The formal definition of the CSTL language syntax is given in Appendix 1. In this section, we describe the syntax and thesemantics of the four kinds of statements related to testing conceptual schemas:

– Statements that update the IB (Section 3.1).– Statements that assert the state of the IB (Sections 3.2, 3.3 and 3.6).– Statements that create domain events (Section 3.4).– Statement that assert the occurrence of domain events (Sections 3.4 and 3.5)

3.1. Updating the information base

When the execution of a test case begins, the IB is assumed to be empty and, during the normal execution of an IS, the IBcan only change due to the occurrence of domain events, and a particular IB state can only be reached by the successfuloccurrence of one or more domain events. However, in testing conceptual schemas, we often need to set up an IB state inde-pendently of the domain events [21]. This happens when the domain events are not available (i.e. they have not yet beenspecified) or when we want to check an inconsistent state that cannot be reached by valid domain events.

CSTL includes six statements that can be used to explicitly set up an IB state in a test case. We describe them below usingexamples based on the schema fragment of Fig. 1.

Category*child

parent1

Fig. 1. A simple schema fragment.


We define that entityID is a new instance of the entity types EntityType1,. . ., EntityTypen with the following statement:

entityID :¼new EntityType1,. . .,EntityTypen;

where entityID must be a new identifier. For example:

cat1 :¼new Category;

Note that we assume multiple classification, and therefore we allow that an entity is an instance of two or more entitytypes not related by generalization/specialization relationships.

Entities can be deleted with the following statement:

delete entityExpr;

where entityExpr is an identifier or, in general, an OCL expression that evaluates to a previously created entity. The dele-tion of an entity implies the deletion of its attributes and the links in which it participates.

To define that the value of attribute att of entity entityID is val (where val is a valid OCL expression) we write:

entityID.att :¼val;

The types of att and val must be compatible; otherwise the verdict of the test case in which the statement appears is Error.Similarly, to define that the entity entityID is related with role role in a binary link (an instance of an association) to one or

more entities given by the OCL expression participants we write:

entityID.role :¼participants;

The types of role and participants must be compatible; otherwise the verdict of the test case in which the statement ap-pears is Error. For example:

cat1.parent :¼cat2;

Often, it is convenient to state in a single statement the creation of a new entity entityID as an instance of entity typeEntityType1 and the assignment of an initial value for its attributes att1,. . ., attn and of its binary links with roles r1,. . .,rm.The syntax is as follows:

entityID :¼new EntityType1 (att1:¼value1,. . ., attn:¼valuen,
r1:¼participants1,. . ., rm:¼participantsm);
where valuei and participantsi are OCL expressions. For example:

cat3 :¼new Category(parent:¼cat1);

Instances of an n-ary UML association Assoc with roles r1,. . ., rn are created with the following statement:

new Assoc(r1:¼entityExp1,. . ., rn:¼entityExpn);

If Assoc is an association class, then the above statement returns the identifier of the instance of that class.

3.2. Asserting the consistency of an IB state

A conceptual schema may include a large number of constraints, and the task of ensuring that all of them are correct is farfrom trivial. Using an appropriate modeling environment, testing the schema may be a practical means of uncovering faultyconstraints. This is done by setting up one or more test cases, each with an IB state that domain experts believe is valid, fol-


lowed by an assertion that the state satisfies all defined constraints. An IB state is called consistent if it satisfies all con-straints defined in the conceptual schema.

In CSTL, the conceptual modeler asserts that the current IB state must be consistent by writing the following statement:

assert consistency;

The verdict of the assertion is Pass if the IB state satisfies all defined constraints. The verdict is Fail if, contrary to what theconceptual modeler expects, the IB is not consistent. When the verdict is Fail two cases are possible: (1) domain experts con-sider that the IB state is indeed invalid or, if it is valid, then (2) the non-satisfied constraint(s) is incorrect. In the former, theproblem lies in the test case, and the conceptual modeler may prefer to change the assertion to assert inconsistency (see be-low). In the latter, the corresponding constraint(s) must be corrected.

As an example, assume the schema shown in Fig. 1 and consider the following test case:

test CategoryCanBeInstantiated{

cat :¼new Category;

assert consistency;

}

The verdict will be Fail because according to the schema, each category should have a parent. If the domain experts con-firm that the IB state just defined is valid (cat is a root category), then the test case has uncovered a schema error, which mustbe corrected by changing the cardinality (multiplicity) of parent to 0..1.

A conceptual modeler may use this kind of assertion not only to check that the constraints defined in the schema behaveas expected, but also to check that each entity type is satisfiable (i.e. it may have a non-empty population). If the conceptualmodeler is able to set up an IB state in which there is at least one instance of entity type E and that state is asserted as con-sistent, then by definition E is satisfiable. If the conceptual modeler is unable to set up such a state, this is not formal proofthat E is unsatisfiable, but in many practical cases it provides a clue that helps to uncover a faulty constraint.

For example, consider again the original schema of Fig. 1 and assume that it is extended with the following invariant:

context Category inv thereMayNotBeCycles:

not self.allChildren() -> includes(self)

where the operation allChildren() returns the whole set of subcategories of the category in the hierarchy [36].Now Category becomes unsatisfiable, and the conceptual modeler is unable to set up a test case with at least one instance

of Category and an assert consistency statement whose verdict is Pass. This result is not formal proof that Category is unsat-isfiable, but in many practical cases it provides a clue that helps the conceptual modeler to uncover a faulty constraint. In thisexample, Category is satisfiable if we make the same change as above: set the cardinality of parent to 0..1.

3.3. Asserting the inconsistency of an IB state

For the set of constraints defined in the schema to be correct and complete, not only must the constraints be satisfied byvalid IB states, but those constraints must also rule out invalid states. Testing the schema may be a practical means of detect-ing missing constraints. This is done by setting up one or more test cases, each with an IB state that domain experts believe isinvalid, followed by an assertion that the state does not satisfy at least one schema constraint.

In CSTL, in order to assert that the current IB state is inconsistent, the conceptual modeler simply writes the followingstatement:

assert inconsistency;

The verdict of the assertion is Pass if the IB state does not satisfy one or more constraints; otherwise, it is Fail. If the verdictof the assertion is Fail then two cases are possible: (1) the IB state is indeed valid or, if it is not, then (2) some constraint is

CategoryInLanguagename : String

Category Language1..***child

parent0..1

Fig. 2. Extension of Fig. 1 with the names of categories.


missing. In the former, the conceptual modeler may prefer to change the assertion to assert consistency (see above). In thelatter, the conceptual modeler must define a new constraint or refine an existing one in order to make it more constraining.

For example, consider the schema of Fig. 2, which extends the schema of Fig. 1 with categories in a multilingual e-com-merce system. Assume that the conceptual modeler writes the following test case to assert that if a category has no name inone language, then the IB state is inconsistent:

test CategoriesHaveANameInEachLanguage{

cat :¼new Category;

english :¼new Language;

catalan :¼new Language;

catInEnglish :¼new CategoryInLanguage

(category:¼cat, language:¼english);catInEnglish.name :¼‘‘Food”;assert inconsistency;

}

Contrary to what it is expected, the execution of the test case fails. This fact reveals to the conceptual modeler that theschema does not enforce that categories have a name in each language. If the domain expert confirms this requirement, thenthe test case has uncovered a schema error, which must be corrected by adding the following constraint to the schema:

context Category inv CategoriesHaveANameInEachLanguage:

self.language = Language.allInstances()Now, the verdict of the above test case is Pass, as desired.

3.4. Asserting the occurrence of domain events

A domain event type is a complex schema element that may have several kinds of defects:

1. The constraints of the event type may not allow the occurrence of valid events.2. The postconditions may not precisely define the intended effect of events.3. The method of the effect operation may produce an IB state that does not satisfy both the postconditions and the schema

constraints.

Testing the schema may be a practical means of detecting those defects. This is done by setting up for each domain eventtype one or more test cases with an IB state and an instance of that event type that domain experts believe may occur in thatstate, followed by an assertion of the (satisfactory) occurrence of that event.

In CSTL, the instances of a domain event type can be created in the same way as those of entity types. If EventType1 is adomain event type, then:

eventId :¼new EventType1;

creates the instance eventId of EventType1, whose characteristics (attribute values, binary links) can be defined as in thecase of entities. Often, it is convenient to define in a single statement the creation of a new event eventId as an instance ofdomain event type EventType1 and the assignment of the value for its attributes att1,. . ., attn and of its binary links with rolesr1,. . .,rm. The syntax is as follows:

eventId :¼new EventType1(att1:¼value1,. . ., attn:¼valuen,
r1:¼participants1,. . ., rm:¼participantsm);
Once the concrete event eventId has been created in a test case in order to assert that it may occur in the current state ofthe IB, the conceptual modeler writes the following sentence:

assert occurrence eventId;

The verdict of this assertion is determined as follows:

1. Check that the current IB state is consistent (as defined in Section 3.2). The verdict is Error if that check fails (events maynot occur in inconsistent IB states).

2. Check that the constraints of the event are satisfied. The verdict is Fail if any of the event constraints is not satisfied.3. Execute the method of the corresponding effect() operation.4. Check that the new IB state is consistent (as defined in Section 3.2). The verdict is Fail if any of the constraints is not satisfied.5. Check that the event postconditions are satisfied. The verdict is Fail if any of the postconditions is not satisfied; otherwise

the verdict of the whole assertion is Pass.


If the verdict from step 1 is Error, then the conceptual modeler must change the IB state in order to make it valid. If theverdict from steps 2, 3 or 4 is Fail, then the event has not occurred as expected by the conceptual modeler. If the verdict fromstep 2 is Fail then the following two cases are possible: (1) domain experts consider that the IB state and event occurrence areindeed invalid or, if it is valid, then (2) the non-satisfied constraint(s) is incorrect. In the former, the conceptual modeler mayprefer to change the assertion to assert non-occurrence (see below). In the latter, the corresponding event constraint(s) mustbe corrected. If the verdict from step 4 is Fail, then the effect method, the event constraints or some schema constraint mustbe ill-specified. If the verdict from step 5 is Fail then either the method of the effect operation or some postcondition is incor-rect: the method may not produce the intended IB state, or the postconditions may be ill-specified.

As an example, assume the extension of Fig. 2 shown in Fig. 3, in which products are classified in a category and manu-factured by a manufacturer. The figure also shows the specification of the DeleteManufacturer event, including the postcon-dition and the method of the corresponding effect operation. The postcondition states that the manufacturer given by theevent (self.manufacturer@pre) must not exist after the event occurrence (in OCL this can be stated by requiring that it isnot an instance of OclAny, the supertype of all types). The only event constraint is that an instance of DeleteManufactureris associated with a manufacturer. Consider, now, the following test case:

test AManufacturerIsDeleted{



cat1InEnglish :¼new CategoryInLanguage(category:¼cat1,language:¼english);

cat1InEnglish.name :¼‘‘Food”;m1 :¼new Manufacturer;

p1 :¼new Product(manufacturer:¼m1, category:¼cat1);dmEv :¼new DeleteManufacturer(manufacturer:¼m1);assert occurrence dmEv;

}

The execution of the test case fails (as detected in step 4) because the occurrence of the event dmEv reaches an invalidstate in which p1 is not manufactured by any manufacturer. There are at least two possible actions that can be performedto make the test case Pass:

1. If products may exist without a manufacturer, then the cardinality of manufacturer is set to 0..1.2. If the domain experts confirm that when a manufacturer is deleted the system must also delete the products it manufac-

tures then the postcondition and the method of the event DeleteManufacturer must be changed to the following:

context DeleteManufacturer::effect()

post theManufacturerDoesNotExistAnyMore:

not [email protected](OclAny)

post theProductsAreDeleted:

[email protected]@pre->forAll(pjnot p.oclIsKindOf(OclAny))

method DeleteManufacturer::effect(){

delete self.manufacturer;

for each p in self.manufacturer.product do

delete p;endfor

}

Fig. 3. Extension of Fig. 2 with products and manufacturers.

A conceptual modeler may use this kind of assertion not only to check that the domain events defined in the schema behave
as expected, but also to check that each domain event type is satisfiable. A domain event type is satisfiable if there is at leastone consistent IB state and one instance of that event type such that the event constraints are satisfied [1]. If the conceptualmodeler is able to set up an IB state and an instance of the event type Ev for which assert occurrence gives the verdict Pass, thenby definition Ev is satisfiable. If the conceptual modeler is unable to set up such an IB state and event, this is not formal proofthat Ev is unsatisfiable, but in many practical cases it provides a clue that helps to uncover a faulty event specification.
As an example, assume that we extend the schema in Fig. 3 with a new event type named AddNewCategory with charac-teristics product and addedCategory. The effect of AddNewCategory is adding a new category to a product. Consider, also, thefollowing event constraint:


context AddNewCategory inv productHasNoCategories:

self.product.category->isEmpty()The conceptual modeler is unable to set up any IB state in which the event constraints are satisfied because the event

AddNewCategory is unsatisfiable.

3.5. Asserting the non-occurrence of domain events

A correct domain event specification must not only accept valid event instances, but also reject invalid ones. An eventinstance is invalid if it may not occur in the domain in the current IB state. Testing the schema may be a practical meansof detecting missing event constraints. This is done by setting up for each domain event type one or more test cases withan IB state and an instance of that event type that domain experts consider may not occur in that state, followed by an asser-tion of the non-occurrence of that event.

In CSTL, in order to assert that the event eventId may not occur, the conceptual modeler writes the following sentence:

assert non-occurrence eventId;

The verdict of this assertion is determined as follows:

1. Check that the current IB state is consistent (as defined in Section 3.2). The verdict is Error if that check fails.2. Check the satisfaction of the event constraints. The verdict is Fail if the event constraints are satisfied and Pass if one or

more event constraints are not satisfied.

If the verdict of the assertion is Fail then two cases are possible: (1) the event is indeed valid or, if it is not, then (2) someevent constraint is missing. In the former, the event may occur in the domain, and the conceptual modeler may prefer tochange the assertion to assert occurrence (see above). In the latter, the conceptual modeler must define a new event con-straint or refine an existing one in order to make it more constraining.

In the example of Fig. 3, if we assume now that a manufacturer cannot be deleted if there are products manufactured by it,then the following event constraint must be added:

context DeleteManufacturer inv manufacturerHasNoProducts:

self.manufacturer.product->isEmpty()With the above event constraint, the following test case will pass (as expected):

test theManufacturerCannotBeDeleted{




catInEnglish.name :¼‘‘Food”;m1 :¼new Manufacturer;

p1 :¼new Product(manufacturer:¼m1, category:¼cat1);dmEv2 :¼new DeleteManufacturer(manufacturer:¼m1);assert non-occurrence dmEv2;

}

3.6. Asserting the contents of an IB state

It is often useful to include in a test case an assertion on the current state of the IB. The purpose may be to check that oneor more derivation rules derive the expected results, or that a navigational expression yields the expected results or that theeffect of one or more domain events implies an expected result in the IB.


In CSTL, to assert that the current state of the IB satisfies a Boolean condition defined in OCL, the conceptual modelerwrites the following statement:

assert true booleanExpression;

where booleanExpression is an OCL expression over the types of the IB and the variables of the test case. The verdict of theassertion is Error if the current state is inconsistent (as defined in Section 3.2). The verdict is Pass if booleanExpression is trueand Fail otherwise. If the verdict is Fail, two cases are possible: (1) booleanExpression should not be True or (2) the derivationrules and/or domain events do not give the expected results. In the former, the conceptual modeler may prefer to change theassertion to assert false (see below). In the latter, the conceptual modeler must change the derivation rules and/or the domainevents specification.

A similar CSTL statement is as follows:

assert false booleanExpression;

Additionally, CSTL includes the following assertions:

assert equals valueExpression1 valueExpression2;

assert not equals valueExpression1 valueExpression2;

As an example, consider again the schema of Fig. 3 and that the derivation rule of the derived attribute numberOfProductsis defined as follows:

context Category::numberOfProducts:Natural

derive: self.product->size()

A conceptual modeler that wants to test that derivation rule may write the following test case:

test NumberOfProductsInACategory{

cat1:¼new Category;



cat1InEnglish.name :¼‘‘Food”;cat2 :¼new Category;


cat2InEnglish.name :¼‘‘Bakery”;cat2.parent :¼cat1;m :¼new Manufacturer;

p1 :¼new Product(manufacturer:¼m, category:¼cat1);p2 :¼new Product(manufacturer:¼m, category:¼cat2);assert equals cat2.numberOfProducts 1;

assert equals cat1.numberOfProducts 2;

}

The verdict of the first assertion is Pass, but that of the second is Fail. The conceptual modeler expects that numberOfProd-ucts includes the products of the children categories, and therefore the result should be 2 (p1 and p2). The derivation ruledoes not derive the expected results because it does not take into account the products of children categories. The test casewill pass if the derivation rule is corrected as follows:

context Category::numberOfProducts:Natural

derive:

let allParents() : Set(Category) =

self.parent -> union(self.parent.allParents())in

Category.allInstances() -> select(c jc.allParents() -> includes(self) or c=self).product->size()


4. Testing conceptual schemas with temporal constraints and derivation rules

Many kinds of temporal constraints have been studied in the literature, and testing them poses particular problems. Inthis paper, we explain how to test schemas that use a representative set of constraints that involve two successive statesof the IB (Sections 4.1 and 4.2), and on creation-time constraints (Section 4.3). We also explain (in Section 4.4) how to testschemas with a particular kind of derived relationship types, which are similar to creation-time constraints. The examples inthis section are based on the schema fragment shown in Fig. 4.

4.1. Temporal constraints on the population of entity types

Two representative constraints on the population of entity types that can be evaluated taking into account two successivestates of the IB are the constant and permanent entity types [1]. An entity type is constant if its population is the same at all times,and permanent if its instances never cease to be instances of it. Most conceptual schemas include several entity types that arepermanent. In the example of Fig. 4, Order is a permanent entity type because its instances never cease to be instances of it.

When a conceptual schema includes one or more temporal constraints of the above kinds, then the semantics of the assertconsistency and assert inconsistency statements defined in Sections 3.2 and 3.3, respectively, must be extended as follows. In atest case, there is a consistent IB state every time that an assert consistency is executed and the obtained verdict is Pass. Notethat such an assertion is executed in each of the assertions described in Section 3, except for assert inconsistency (section 3.3).Once there is a consistent IB state in a test case, the next time that an assert consistency or assert inconsistency is executed inthat test case, the constant and permanent constraints must be evaluated taking into account the current IB state and theprevious consistent IB state.

For example, assume the following test case:

test DeletingAnOrder{

c1 :¼new Customer;

order1 :¼new Order;order1.customer :¼c1;assert consistency;

delete order1;

assert consistency;

}

Fig. 4. Fragment of the osCommerce case study focusing on shopping cart items.


The verdict of the first assertion will be Pass, but that of the second will be Fail and, therefore, the verdict of thetest case will be Fail. If the domain experts confirm that the existing IB state in the second assert consistency is not validthen the conceptual modeler may prefer to change that assertion to assert inconsistency in order to make the verdictPass.

The above constraints can be evaluated in several ways. In our implementation, described in Section 6, we simulate theexecution of an operation between the two IB states. The operation has one postcondition for each temporal constraint. Wemay use the standard OCL in those postconditions in order to ensure that the constraints are satisfied. If E is a constant entitytype, then the postcondition is as follows:

E.allInstances() = E.allInstances@pre()

where @pre is the OCL keyword to refer to the previous state. If E is a permanent entity type, then the postcondition is asfollows:

E.allInstances() -> includesAll(E.allInstances@pre())

4.2. Temporal constraints on the population of relationship types

Two representative constraints on the population of relationship types (attributes, associations) that can be evaluatedtaking into account two successive states of the IB are the constant and permanent relationship types [1]. A relationship typeR(p1:E1, . . .,pn:En) is constant with respect to a participant pi if the instances of R in which an instance ei of Ei participates arethe same during the temporal interval in which ei exists. A relationship type R(p1:E1, . . .,pn:En) is permanent with respect to aparticipant pi if the instances of R in which an instance ei of Ei participates never cease to exist during the temporal interval inwhich ei exists. Most conceptual schemas include several relationship types that are constant or permanent with respect toone of their participants.

In the example of Fig. 4, Order is constant with respect to its participation in Places because the instances of Places inwhich an order participates are the same during the temporal interval in which the order exists. The product and the attri-butes of a shopping cart item are constant because they remain the same during the temporal interval in which that itemexists. Customer is permanent with respect to its participation in Places because the instances of Places in which a customerparticipates never cease to exist during the temporal interval in which the customer exists.

As in the previous case, once there is a consistent IB state in a test case, the next time that an assert consistency or assertinconsistency is executed in that test case, the constant and permanent constraints must be evaluated taking into account thecurrent IB state and the previous consistent IB state. In our implementation, the postconditions that we add to our simulatedoperation are as follows. If R(p1:E1, . . .,p2:E2) is constant with respect to participant p1, then:

E1.allInstances()-> intersection (E1.allInstances@pre)-> forAll(e1je1.p2 = e1.p2@pre)

If R(p1:E1,. . .,p2 :E2) is permanent with respect to a participant p1 and the multiplicity of p2 is 1 or 0..1, then:

E1.allInstances()-> intersection (E1.allInstances@pre)-> forAll(e1je1.p2->includes(e1.p2@pre))

And if the maximum multiplicity is greater than 1, then:

E1.allInstances()-> intersection (E1.allInstances@pre)-> forAll(e1je1.p2->includesAll(e1.p2@pre))

For example, assume the following test case:

test ChangingTheCustomerOfAShoppingCart{

c1 :¼new Customer;

sc :¼new ShoppingCart;

sc.customer :¼c1;fashionTShirt:¼new Product;

fashionTShirt.netPrice :¼10;sci1 :¼new ShoppingCartItem;

sci1.shoppingCart :¼sc;sci1.product :¼fashionTShirt;


assert consistency;

c2 :¼new Customer;

sc.customer :¼c2;assert consistency;

}

The verdict of the first assertion will be Pass, but that of the second will be Fail and, therefore, the verdict of the test casewill be Fail. The customer of a shopping cart has been defined as constant, and therefore it cannot be changed. The postcon-dition that is not satisfied is as follows:

ShoppingCart.allInstances()-> intersection (ShoppingCart.allInstances@pre())
-> forAll(scj sc.customer = sc.customer@pre)
If the domain experts confirm the temporal constraint, then the conceptual modeler may prefer to change the last asser-tion to assert inconsistency in order to make the test Pass.

4.3. Creation-time constraints

A creation-time constraint is a particular kind of temporal constraint that appears several times in most conceptual sche-mas. A creation-time constraint of an entity type E is a constraint that its instances must satisfy only at the time when theybecome an instance of that entity type [44]. As proposed in [44] we define creation-time constraints by means of operationsstereotyped ‘‘iniIC” that must give a True result when the corresponding entity is created.

Consider, as an example, the schema of Fig. 4 and assume that the conceptual modeler adds the following constraint inorder to enforce that the product attributes of the attributes of a shopping cart item must be enabled when that item is cre-ated. The formal specification in OCL is as follows:

context ShoppingCartItem::productAttributesEnabled():Booleanbody: self.attribute.productAttribute->forAll(status = Status::enabled)

where the operation productAttributesEnabled has the stereotype ‘‘iniIC”.In the following test case, the conceptual modeler declares that the schema allows changing the state of a product attri-

bute once it has been used in the creation of a shopping cart item:

test changeProductAttributeStatus{

color :¼new Option;

black :¼new Value;

blackColor :¼new Attribute(option:¼color, value:¼black);carInsurance :¼new Product(netPrice:¼300);blackCarInsurance :¼new ProductAttribute

(product:¼carInsurance,attribute:¼blackColor);

blackCarInsurance.increment :¼80;blackCarInsurance.status :¼Status::enabled;blackCarInsurance.sign :¼Sign::plus;c :¼new Customer;

sc :¼new ShoppingCart(customer:¼c);item1 :¼new ShoppingCartItem(shoppingCart:¼sc);item1.product :¼carInsurance;item1.quantity :¼1;item1.attribute :¼Set{blackColor};assert consistency;blackCarInsurance.status :¼Status::disabled;assert consistency;

}

The verdict of the first assertion is Pass. The verdict of the second assertion is also Pass because the constraint productAt-tributesEnabled is only evaluated when shopping cart items are created.

Creation-time constraints must be evaluated every time an assert consistency or assert inconsistency is explicitly or implic-itly executed in a test case, taking into account the current IB state and the previous consistent IB state, if it exists.


The first time that the assertion is executed, the previous IB state is assumed to be empty. In the example, the con-straint must be evaluated only for the shopping cart items that have been created since the execution of the previousassertion.

In our implementation, the evaluation of a creation-time constraint ct of entity type E is performed by adding the follow-ing postcondition to our simulated operation:

(E.allInstances() - E.allInstances@pre())
-> forAll (ej ct replacing self by e)
In the example, the postcondition becomes the following:

(ShoppingCartItem.allInstances() - ShoppingCartItem.allInstances@pre())
->forAll (scijsci.attribute.productAttribute
->forAll(status= Status::enabled))

4.4. Derived constant relationship types

A particular class of derived relationship type that often appears many in many conceptual schemas is the derived con-stant relationship type, whose instances can be derived when the instances of one of its participants are created, and theyremain fixed during its lifetime [45].

Fig. 4 shows two simple examples: attributes ShoppingCartItem::added, price. Their value is determined when an instanceof ShoppingCartItem is created (using the derivation rules shown at the bottom of the Figure), and those values do not changelater on.

Consider as an example the following test case:

test priceDoesNotChange{

p :¼new Product(netPrice:¼15);c :¼new Customer;

sc :¼new ShoppingCart(customer:¼c);item1 :¼new ShoppingCartItem(shoppingCart:¼sc);item1.product :¼p;item1.quantity :¼1;assert equals item1.price 15;

p.netPrice :¼20;assert equals item1.price 15;

}

The verdict of the first assertion is Pass and the verdict of the second one is also Pass given that the attribute price is de-rived and constant.

Derived constant relationship types must be materialized when the corresponding entity is created, before the evaluationof the constraints. In our implementation, we materialize those types every time an assert consistency or assert inconsistencyis explicitly or implicitly executed in a test case. The materialization needs to be done only for the entities that have beencreated since the execution of the previous assertion.

Fig. 5. Metamodel fragment of the structure of a CSTL program.


5. The CSTL language

In this Section, we describe the structure of the CSTL language. A CSTL program consists of a fixture (may be empty), a set(may be empty) of fixture components, and a set of one or more test cases. The basic construct of CSTL is the concrete testcase. However, CSTL also includes other constructs (such as fixture and abstract test case) that make easier to write tests.Fig. 5 shows a fragment of the metamodel of the structure of a CSTL program and Fig. 6 shows a program example. The for-mal definition of the CSTL language syntax is given in Appendix 1. The complete example of Fig. 6 can be found in [39].

It is assumed that the execution of each test case of a CSTL program starts with an empty IB state. With this assumption,the test cases of a program are independent each other, and therefore the order of their execution is irrelevant. However, thetest cases of a program often have a common initial IB state and variables, and it is practical to define this common part,called a fixture, separately. The fixture is a set of statements that create an IB state and define the values of the commonprogram variables. It is assumed that the execution of a test case starts with the execution of the fixture.

A fixture component is a named set of statements that create a fragment of the state of the IB and define the values of a setof variables. If fc is a fixture component, then a test case may use the following statement:

load fc;

to add to the current state of the IB and to the current variables the fragment defined by fc. In the example of Fig. 6, thereare two fixture components: one that instantiates a shopping cart item with a product without attributes, and another thatinstantiates two items with products having attributes. The same fixture component can be loaded by several test cases.

Fig. 6. CSTL program for testing order confirmation.


In CSTL, there are three kinds of test cases: concrete, abstract and abstract invocation. A concrete test case is a set of state-ments that builds a state of the IB, defines values of its variables, and executes one or more tests of one of the five test kindsdescribed in the previous section. In Fig. 6, an example is the test EmptyShoppingCart, which checks the consistency of the IBstate defined by the fixture. The verdict of this test is Fail, because the shopping cart sc has no items and, according to thecardinality constraints shown in Fig. 4, it should have at least one.

An abstract test case is a parameterized test case intended to be invoked one or more times in the same program. Theparameters of an abstract test case may include fixture components and variables. An example in Fig. 6 is confirmedOrder-Total. The aim is to test that the total amount of the order created when the customer checks out his shopping cart (eventOrderConfirmation) is as the domain expert expects (see [39] for further details). It loads the fixture component received, as-serts that the domain event OrderConfirmation (with characteristic sc) occurs, and asserts that the amount of the order justcreated is expectedTotal.

An abstract test case invocation is the invocation of an abstract test case with the desired values of the parameters. InFig. 6, there are two invocations of the abstract test case. The verdict of the first invocation is Pass and the second verdictof the one is Fail.

The execution of a concrete test case or of an abstract test case invocation always returns a verdict. The verdict is obtainedfrom the verdicts of the test kinds defined in the previous section as follows:

If the verdict of an assertion of a test case is Error, then the execution of the test case ends, and the verdict of the wholetest case is Error.

If the verdict of an assertion of a test case is Fail, then the execution of the test case ends, and the verdict of the whole testcase is Fail.

The verdict of a test case is Pass if the verdicts of all its assertions are Pass.The result of the invocation of a CSTL program also always returns a verdict, which (as in jUnit) is obtained from the ver-

dicts of its test cases as follows:

1. If the verdict of a test case is Error, then the verdict of the program is Error.2. If the verdict of a test case is Fail, then the verdict of the program is Fail.3. When the verdict of all test cases is Pass, the verdict of the program is Pass.

6. The test processor

Fig. 7 shows the relationships between the definition and execution of a conceptual schema and the definition and pro-cessing of its tests. In our implementation, the execution of conceptual schemas is done by the information processor, whilethe execution of test programs is done by the test processor.

We have implemented the information processor reusing USE [46] as much as possible. We have needed to extend USE inorder to be able to deal with several new language features. The most significant extension has been the treatment of thetemporal constraints and derivation rules described in Section 4.

We have developed a prototype of a test processor, which is a program capable of executing CSTL programs. The test pro-cessor consists of three main components: presentation manager, test manager and test interpreter (see Fig. 7).

The presentation manager provides means for writing CSTL programs and for displaying the results of their execution.

Fig. 7. Test processing and conceptual schema execution.


The test manager stores the CSTL programs in order to make it possible to execute their test cases at any time and, inparticular, every time that the conceptual schema is changed (regression testing). When the conceptual modeler requeststhe execution of one or more CSTL programs, the test manager selects the programs and requests that the test interpreterexecute them. The test manager also keeps track of the test results and maintains test statistics.

The test interpreter reads and executes CSTL programs. For each test case, the interpreter sets up the common fixture (ifany), executes the statements of each test case and computes the verdicts. The interpreter invokes the services of the infor-mation processor to create, delete and change entities, attributes and associations of the IB, and also to evaluate OCL expres-sions over the IB.

The assert statements described in Section 3 are executed as follows:

1. Assert consistency. The interpreter requests that the information processor check all static and temporal constraints. Theresult is the set of constraints that are not satisfied. The verdict is Fail if the set is non-empty.

2. Assert inconsistency. The interpreter requests taht the information processor check all static and temporal constraints. Asbefore, the result is the set of constraints that are not satisfied. The verdict is Fail if the set is empty.

3. Assert event occurrence. In the first step, the interpreter requests that the information processor check all static and tem-poral constraints. The result is the set of constraints that are not satisfied. The verdict is Error if the set is non-empty. Inthe second step, the interpreter requests that the information processor check the event constraints. The result is the setof event constraints that are not satisfied. The verdict is Fail if the set is non-empty. In the third step, the test interpreterexecutes the method of the effect() operation corresponding to the event (recall that the methods are written in the CSTLlanguage, but without using assert statements). The test interpreter executes the statements of this method in the sameway as those of the test programs: by invoking the services of the information processor. In the fourth step, the inter-preter again requests that the information processor check all static and temporal constraints. The result is the set of con-straints that are not satisfied. The verdict is Fail if the set is non-empty. Finally, the interpreter requests that theinformation processor check the event postconditions. The result is the set of postconditions that are not satisfied. Theverdict is Fail if the set is non-empty.

4. Assert event non-occurrence: In the first step, the interpreter requests that the information processor check all static andtemporal constraints. The result is the set of constraints that are not satisfied. The assertion is Error if the set is non-empty. In the second step, the interpreter requests that the information processor check the event constraints. The resultis the set of event constraints that are not satisfied. The verdict is Pass if the set is non-empty.

5. Assert IB contents. In the first step, the interpreter requests that the information processor check all static and temporalconstraints. The result is the set of constraints that are not satisfied. The assertion is Error if the set is non-empty. In thesecond step, the interpreter requests that the information processor evaluate the OCL expression(s), and the interpretercomputes the verdict from the results obtained.

Fig. 8 shows the result of the execution of the CSTL program shown in Fig. 6. One test case has passed (the first invocationof confirmedOrderTotal), whereas the other two have failed, and therefore the global verdict is Fail. Note that the test proces-sor indicates the number of the lines where the two tests cases have failed and gives an explanation of the failure in naturallanguage. Many more examples can be found in [39].

Fig. 8. Test processing and conceptual schema execution.


7. Related work

Hierons et al. [47] provides a thorough survey on the state of the art regarding the many relationships between formalspecifications and testing. One such relationship, which has attracted a lot of work, is that specifications can be used to auto-matically derive test cases of the final system and/or of some intermediate artefact [48–54].

However, in this paper we have focused on a different kind of relationship between specifications and testing: specifica-tion validation. Specifications can be validated by using testing techniques that execute them through animation. In general,animation facilities allow users to execute operations of the specification with user supplied parameters, thereby calculatingthe value of the output parameters and the new system state [55]. Note that we do not deal here with the validation of thefinal system or of any intermediate artifact.

The idea of animating conceptual schemas for testing purposes dates back to the mid-80s. [13] describes a conceptual lan-guage (CPL) and a tool that generates a prototype from a CPL schema, which can be tested. The generated prototype makes itpossible to build an IB state, perform consistency checks and ask questions about the contents of the IB. A similar approachwas taken in [14,15], with the PPP and TROLL light language and environment, respectively. The most recent work, and the onemost closely related to our approach is the USE tool [16,36,46]. USE makes it possible to define snapshots (instantiations) ofstructural schemas expressed in a subset of UML/OCL and checking if they are valid instantiations of the schema. In general,the work reported in this paper contributes to the impressive work done so far by dealing with a more expressive language(taking into account advanced structural schema elements and the behavioral schema), and by proposing a complete testinglanguage (CSTL) for writing test cases of conceptual schemas.

Conceptual schema testing can be used in conjunction with other validation techniques such as paraphrasing specifica-tions in natural language [4–7], generation of abstractions and abstracts of specifications [8], and explanation generation [9–11].

Similarly, conceptual schema testing tools can also be used in conjunction with automated reasoning procedures. Therehas been a lot of work on these procedures (a representative set of recent papers is [56–62]. Their general objective is toautomatically check that a conceptual schema has a set of properties. Some of these properties are general and must be sat-isfied by all conceptual schemas, while others apply only to a particular conceptual schema. The most studied general prop-erties are integrity constraints satisfiability, liveliness of an entity or relationship type, non-redundancy of integrityconstraints and operation executability (or, in our terms, domain event occurrence). However, it is well known that the prob-lem of reasoning with integrity constraints and derivation rules in its full generality is undecidable. Therefore, the availableprocedures are restricted to certain kinds of constraints and derivation rules or domains, or they may not terminate in somecircumstances [63]. In contrast, testing conceptual schemas, as proposed in this paper, can be applied to any conceptualschema that includes any kind of OCL constraint or derivation rule and, if there is not an infinite loop in a derivation ruleor method, test cases usually terminate in a very short time. However, the results obtained by testing are not as strong asthose obtained by automated reasoning procedures when they are applicable. Therefore, like it has been observed in the gen-eral field of software [47], testing and automated reasoning procedures are complementary and can be combined in a devel-opment environment.

8. Conclusions

We have shown that testing conceptual schemas may be a means for their validation. We have seen that conceptual sche-mas of information systems can be tested with the goal of ‘‘uncover faults by triggering failures” [19]. We have described thesimilarities and differences between testing conceptual schemas and testing programs.

We have presented a list of five kinds of tests that can be applied to conceptual schemas. Some of these test kinds requireconceptual schemas that include all structural and behavioral aspects. However, we have shown that it also makes sense totest small fragments consisting of a few entity and relationship types, integrity constraints and derivation rules in order touncover faults during the early stages of the development of a conceptual schema [2]. We have shown that our approach canalso be applied to schemas with temporal constraints and derivation rules.

We have presented CSTL, a textual procedural language for writing tests of executable conceptual schemas written inUML/OCL. As far as we know, this is the first proposal of a language for testing conceptual schemas designed in the styleof the modern xUnit software testing frameworks. The main features of the language have been illustrated using examplestaken from its application to the osCommerce case study. We refer the interested reader to the case study report [39] to fullyappreciate the complexity and the need for testing.

We have described an implementation of a test processor that manages and executes CSTL programs. It includes a testinterpreter that coordinates the execution of the tests and invokes the services of an information processor, which we haveimplemented by reusing and extending USE [46]. Tests written in CSTL may be automatically executed as many times asneeded (regression testing).

We believe that our work opens new directions for research and development in conceptual modeling. We mention fourof these new directions here. First, it is necessary to develop one or more methodologies for testing conceptual schemas. Inthis paper we have focused on the testing language and the test processor, which are the prerequisite for the development ofthose methodologies, but we need to study how to use both the language and the processor in professional projects in order


to know how and when to get the maximum benefit from them. In particular, it seems interesting to develop a test-drivenconceptual modeling methodology, similar to the popular test-driven development [28], and to build a catalog of test casepatterns. Second, and similar to the adequacy criteria for program testing (such as, for example, statement or branch cover-age), we need to develop adequacy criteria for conceptual schema testing that measure the degree to which a conceptualschema has been tested. Such criteria are useful for determining which parts of a conceptual schema need more tests. Third,when the transformation from a conceptual schema to a PSM is manual, there is the need of rewriting the conceptual schematest cases at the PSM level. In this context, it would be very interesting to save testing effort by developing automatictransformation procedures of the test cases. Finally, conceptual schema testing should be integrated with other validationtechniques, and the test processor should be integrated with the other tools of a comprehensive development environment[64].

Acknowledgments

We sincerely thank the anonymous referees for their valuable suggestions, which have improved the paper. We wouldalso like to thank the people of the GMC group for their useful comments to previous drafts of this paper. This work has beenpartly supported by the Ministerio de Ciencia y Tecnología under the project TIN2008-00444, Grupo Consolidado.

Appendix A. CSTL grammar

testProgram :testprogram <programID> {fixture fixtureComponent*testCase* }

fixture :statement*

fixtureComponent :

fixturecomponent <fixtureComponentID> {statement* }

testCase :concreteTestj abstractTestj abstractTestInvocation

concreteTest :test <testID> {statement* }

abstractTest :abstract test <abstractTestID> paramList {statement* }

paramList :(parameter [, parameter]* )

parameter :parameterType <parameterID>

type :<oclPrimitiveType>j <entityTypeID>

parameterType :typej Fixture

abstractTestInvocation :test <abstractTestID> parametersAssignment

parametersAssignment :(parameterAssignment [, parameterAssignment]* )

parameterAssignment :<parameterName> :¼expression

expression :<oclExpressionWithVariableIDs>

statement :stateStatement ; j variableStatement ;j assertion ;j controlFlowStatement

stateStatement :entityCreationj entityDeletionj binaryPropertySettingj nAryRelationshipCreationj fixtureComponentLoading

variableStatement :variableDeclarationj variableAssignment

assertion :assertTruej assertFalsej assertEqualsj assertNotEqualsj assertConsistencyj assertInconsistencyj assertDomainEventOccurrencej assertDomainEventNonOccurrence

controlFlowStatement :conditionalj whileLoopj forLoopj forEachLoop

entityCreation : new <entityTypeID> [, <entityTypeID>] * propertiesAssignment?

propertiesAssignment :(propertyAssignment [, propertyAssignment]* )

propertyAssignment :<propertyID> :¼expression

entityDeletion :delete expression

binaryPropertySetting :expression :¼expression

nAryRelationshipCreation :new <assocID> participantsAssignment

participantsAssignment :(participantAssignment [, participantAssignment]+ )

prarticipantAssignment :<roleID> :¼expression

fixtureComponentLoading :load <fixturecomponentID>

variableDeclaration :type<varID>

variableAssignment :[<varID> j varDeclaration] :¼[expression j entityCreation j nAryRelationshipCreation]

assertTrue :assert true expression

assertFalse :assert false expression

assertEquals : assert equals expression expression

assertNotEquals :assert not equals expression expression

assertConsistency :assert consistency

Appendix A (continued)

(continued on next page)


assertInconsistency :assert inconsistency

assertDomainEventOccurrence :assert occurrence <domainEventID>

assertDomainEventNonOccurrence :assert non-occurrence <domainEventID>

assertDomainEventNonOccurrence :assert non-occurrence <domainEventID>

condition :if expression then statement*[else if expression then statement*]*[else statement*]?endif

whileLoop :while expression do statement* endwhile

forLoop :for variableAssignment to expression step expression do statement* endfor

forEachLoop :for each [variableDeclaration j varID] in expression do statement* endfor

Appendix A (continued)


References

[1] A. Olivé, Conceptual Modeling of Information Systems, Springer, Berlin, 2007.[2] O.I. Lindland, G. Sindre, A. Solvberg, Understanding Quality in Conceptual Modeling, IEEE Software 11 (1994) 42–49.[3] D.L. Moody, G. Sindre, T. Brasethvik, A. Sølvberg, Evaluating the quality of information models: empirical testing of a conceptual model quality

framework, in: Proceedings of ICSE 2003, IEEE Computer Society, Washington, DC, USA, 2003, pp. 295–305.[4] C. Rolland, C. Proix, A natural language approach for requirements engineering, in: Proceedings of CAiSE 1993, LNCS, vol. 593, Springer, 1992, pp. 257–277.[5] H. Dalianis, A method for validating a conceptual model by natural language discourse generation, in: Proceedings of CAiSE 1992, LNCS, vol. 593,

Springer, 1992, pp. 425–444.[6] E. Métais, Enhancing information systems management with natural language processing techniques, Data and Knowledge Engineering 41 (2002)

247–272.[7] P.J.M. Frederiks, T.P. van der Weide, Information modeling: the process and the required competencies of its participants, Data and Knowledge

Engineering 58 (2006) 4–20.[8] L. Jesus, R. Carapuca, Automatic generation of documentation for information systems, in: Proceedings of CAiSE 1992, LNCS, vol. 92, Springer, 1992, pp.

48–64.[9] J.A. Gulla, G. Willumsen, Using explanations to improve the validation of executable models, in: Proceedings of CAiSE 1993, LNCS, vol. 685, Springer,

1993, pp. 118–142.[10] A. Olivé, M.R. Sancho, Validating conceptual specifications through model execution, Information Systems 21 (1996) 167–186.[11] J.A. Gulla, A general explanation component for conceptual modeling in CASE environments, ACM Transactions on Information Systems 14 (1996) 297–

329.[12] S. Hartmann, S. Link, T. Trinh, Constraint acquisition for entity-relationship models, Data and Knowledge Engineering 68 (2009) 1128–1155.[13] F. Dignum, T. Kemme, W. Kreuzen, H. Weigand, R.P. van de Riet, Constraint modelling using a conceptual prototyping language, Data and Knowledge

Engineering 2 (1987) 213–254.[14] O.I. Lindland, J. Krogstie, Validating conceptual models by transformational prototyping, in: Proceedings of CAiSE 1993, LNCS, vol. 685, Springer, 1993,

pp. 165–183.[15] M. Gogolla, S. Conrad, G. Denker, R. Herzig, N. Vlachantonis, A development environment for an object specification language, IEEE Transactions on

Knowledge and Data Engineering 7 (1995) 505–508.[16] M. Gogolla, F. Büttner, M. Richters, USE: a UML-based specification environment for validating UML and OCL, Science of Computer Programming 69

(2007) 27–34.[17] A. van Lamsweerde, Requirements Engineering, Wiley, 2009.[18] R.L. Glass, An ancient (but still valid?) look at the classification of testing, IEEE Software 25 (2008) 111–112.[19] B. Meyer, Seven principles of software testing, IEEE Computer 41 (2008) 99–101.[20] E. Insfrán, V. Pelechano, O. Pastor, Conceptual modeling in the eXtreme, Information and Software Technology 44 (2002) 659–669.[21] S.J. Mellor, M.J. Balcer, Executable UML. A Foundation for Model-Driven Architecture, Addison-Wesley, 2002.[22] A. Olivé, Conceptual schema-centric development: a grand challenge for information systems research, in: Proceedings of CAiSE 2005, LNCS, Porto,

Portugal, Springer, 2005, pp. 1–15.[23] P. Baker, Z.R. Dai, J. Grabowski, Ø. Haugen, I. Schieferdecker, C. Williams, Model-Driven Testing: Using the UML Testing Profile, Springer, 2008.[24] OMG, UML Testing Profile, Version 1.0, formal/05-07-07, <http://www.omg.org/cgi-bin/doc?formal/05-07-07> (accessed October 2009).[25] D.A. Jardine, Concepts and terminology for the conceptual schema and the information base, Computers & Standards 3 (1984) 3–17.[26] OMG, MDA Guide, Version 1.0.1, docs/omg/03-06-01, <http://www.omg.org/docs/omg/03-06-01.pdf> (accessed October 2009.[27] O. Pastor, J.C. Molina, Model-Driven Architecture in Practice, Springer, 2007.[28] A. Gemino, Y. Wand, Complexity and clarity in conceptual modeling: comparison of mandatory and optional properties, Data and Knowledge

Engineering 55 (2005) 301–326.[29] M. Genero, G. Poels, M. Piattini, Defining and validating metrics for assessing the understandability of entity-relationship diagrams, Data and

Knowledge Engineering 64 (2008) 534–557.[30] K.D. Schewe, B. Thalheim, Conceptual modelling of web information systems, Data and Knowledge Engineering 54 (2005) 147–188.[31] A. Burton-Jones, V.C. Storey, V. Sugumaran, P. Ahluwalia, A semiotic metrics suite for assessing the quality of ontologies, Data and Knowledge

Engineering 55 (2005) 84–102.

http://www.omg.org/cgi-bin/doc?formal/05-07-07

http://www.omg.org/docs/omg/03-06-01.pdf


[32] K. Beck, Test-Driven Development: By Example, Addison-Wesley, 2003.[33] D. Janzen, H. Saiedian, Test-driven development: concepts, taxonomy, and future direction, Computer 38 (2005) 43–50.[34] E. Gamma, K. Beck, JUnit: A cook’s tour, Java Report 4 (1999) 27–38, available from: <http://junit.sourceforge.net/doc/cookbook/cookbook.htm>.[35] British Computer Society, Vocabulary of terms in software testing, British Standards, BS 7925-1, <http://www.testingstandards.co.uk/bs_7925-

1_online.htm> (accessed October 2009).[36] M. Gogolla, J. Bohling, M. Richters, Validating UML and OCL models in USE by automatic snapshot generation, Software and Systems Modeling 4 (2005)

386–398.[37] J.S. Ostroff, R.F. Paige, D. Makalsky, P.J. Brooke, E-tester: a contract-aware and agent-based unit testing framework for Eiffel, Journal of Object

Technology 4 (2005) 97–114.[38] A. Tort, The osCommerce Conceptual Schema, 2007, available from: <http://www.guifre.lsi.upc.edu/OSCommerce.pdf>.[39] A. Tort, Testing the osCommerce Conceptual Schema by Using CSTL, Research Report LSI-09-28-R, UPC, 2009, available from: <http://www.lsi.upc.edu/

~techreps/files/R09-28.zip>.[40] OMG, Object Constraint Language, Version 2.0, formal/2006-05-01, <http://www.omg.org/spec/OCL/2.0/> (accessed October 2009).[41] A. Olivé, R. Raventós, Modeling events as entities in object-oriented conceptual modeling languages, Data and Knowledge Engineering 58 (2006) 243–262.[42] C. Larman, Applying UML and Patterns, Third ed., Prentice-Hall, 2005.[43] G. Booch, J. Rumbaugh, I. Jacobson, The Unified Modeling Language Reference Manual, second ed., Addison-Wesley, 2005.[44] A. Olivé, A method for the definition of integrity constraints in object-oriented conceptual modeling languages, Data and Knowledge Engineering 59

(2006) 559–575.[45] A. Olivé, Derivation rules in object-oriented conceptual modeling languages, in: Proceedings of CAiSE 2003, LNCS, vol. 2681, Springer, Berlin, 2003, pp.

404–420.[46] M. Richters, M. Gogolla, Validating UML models and OCL constraints, in: Proceedings of UML 2000, LNCS, Springer, York, UK, 2000, pp. 265–277.[47] R.M. Hierons, K. Bogdanov, J.P. Bowen, R. Cleaveland, J. Derrick, J. Dick, et al, Using formal specifications to support testing, ACM Computing Surveys 41

(2009).[48] S. Pickin, C. Jard, Y. Le Traon, T. Jéron, J.M. Jézéquel, A. Le Guennec, System test synthesis from UML models of distributed software, in: Proceedings of

FORTE 2002, LNCS, vol. 2529, Springer, 2002, pp. 97–113.[49] A. Gargantini, C. Heitmeyer, Using model checking to generate tests from requirements specifications, ACM SIGSOFT Software Engineering Notes 24

(1999) 146–162.[50] C. Nebut, F. Fleurey, Y. Le Traon, J.M. Jézéquel, Automatic test generation: a use case driven approach, IEEE Transactions on Software Engineering

(2006) 140–155.[51] P. Santos Neto, R. Resende, C. Padua, A method for information systems testing automation, in: Proceedings of CAiSE 2005, LNCS, vol. 3520, Springer,

2005, pp. 504–518.[52] A. Hartman, K. Nagin, The AGEDIS tools for model based testing, ACM SIGSOFT Software Engineering Notes 29 (2004) 129–132.[53] L. Briand, Y. Labiche, A UML-Based approach to system testing, in: Proceedings of UML 2001, LNCS, vol. 2185, Springer, 2001, pp. 149–208.[54] Y. Zhang, Test-driven modeling for model-driven development, IEEE Software 21 (2004) 80–86.[55] J. Bicarregui, J. Dick, B. Matthews, E. Woods, Making the most of formal specification through animation, testing and proof, Science of Computer

Programming 29 (1997) 53–78.[56] A. Formica, Satisfiability of object-oriented database constraints with set and bag attributes, Information Systems 28 (2003) 213–224.[57] D. Berardi, D. Calvanese, G. De Giacomo, Reasoning on UML class diagrams, Artificial Intelligence 168 (2005) 70–118.[58] M. Jarrar, Towards automated reasoning on ORM schemes, in: Proceedings of ER 2007, LNCS, vol. 4801, Springer, 2007, pp. 181–197.[59] A. Queralt, E. Teniente, Decidable reasoning in UML schemas with constraints, in: Proceedings of CAiSE 2008, LNCS, vol. 5074, Springer, 2008, pp. 281–295.[60] M. Brambilla, C. Tziviskou, An online platform for semantic validation of UML models, in: Proceedings of ICWE 2009, LNCS, vol. 5648, Springer, 2009,

pp. 477–480.[61] A. Queralt, E. Teniente, Reasoning on UML conceptual schemas with operations, in: Proceedings of CAiSE 2009, LNCS, vol. 5565, Springer, 2009, pp. 47–62.[62] V. Strahonja, Modeling legislation by using UML state machine diagrams, in: Proceedings of CCECE’06, vol. 54, 2006, pp. 624–627.[63] A. Queralt, Validation of UML conceptual schemas with OCL and operations, Ph.D. Thesis, Universitat Politècnica de Catalunya, 2009.[64] M. Bouzeghoub, Z. Kedad, E. Métais, CASE tools: computer support for conceptual modelling, in: O. Diaz, M. Piattini (Eds.), Advanced Database

Systems, Techniques and Design, Artech House, 2000, pp. 439–483.

Albert Tort is currently a researcher at the Services and Information Systems Engineering Department of the UniversitatPolitècnica de Catalunya in Barcelona. He is a member of the Information Modeling and Processing (MPI) research group. Heobtained his Computer Science degree in 2008 from the Universitat Politècnica de Catalunya and he received his M.Sc. inComputing in 2009. He is working towards his Ph.D. degree. His main research interests are conceptual modeling of informationsystems and requirements engineering.

Antoni Olivé is a professor of information systems at the Universitat Politècnica de Catalunya in Barcelona. He has worked in
this field during over 20 years, mainly in the university and research environments. His main interests have been, and are,conceptual modeling, requirements engineering, information systems design and databases. He has taught extensively on thesetopics. He has also conducted research on these topics, which has been published in international journals and conferences. Heis a member of IFIP WG8.1 (Design and evaluation of information systems) where he served as chairman during 1989–1994.
http://junit.sourceforge.net/doc/cookbook/cookbook.htm

http://www.testingstandards.co.uk/bs_7925-1_online.htm

http://www.testingstandards.co.uk/bs_7925-1_online.htm

http://www.guifre.lsi.upc.edu/OSCommerce.pdf

http://www.lsi.upc.edu/~techreps/files/R09-28.zip

http://www.lsi.upc.edu/~techreps/files/R09-28.zip

http://www.omg.org/spec/OCL/2.0/

Documents

An approach to testing conceptual schemas