Formal Specification and Verification of Data-Centric Web Services

Iman Saleh, Ph. D.

Assistant Scientist

University of MiamiFlorida

March 15th, 2014

In a Nutshell…

We develop and evaluate a formal model and contracting framework for

data-centric Web services.

We show how this model can enable the formal specification and

verification of Web service compositions.

We demonstrate through both formal proofs and empirical evaluations that

our proposed framework significantly decreases ambiguity about a service,

enhances its reuse, and facilitates detection of errors in service-based

implementations.

2

Part I

Problem Formulation

Introduction

Service-Oriented Architecture (SOA): Applications implemented as

reusable, interoperable, network-accessible software modules commonly

known as Web services

Web Services make use of standardized interfaces to enable loosely-

coupled business-to-business and customer-to-business interactions over

the Web

Service consumers depends heavily on the exposed service interface

specification to invoke a service

4

Data specification challenges within SOA

Service Consumer

Enterprise Gateway

Web Service Description Language (WSDL)

Specification

Capability Specification Documents (in natural

language)

EnterpriseIntranet

UnderlyingApplicationsLogical Data Model and

Business Rules X

X

?

Reasoning & Interpretation of Input/Output messages and formulation of input

Interpreted Input/Output

D

C

A B

5

Developers Feedback

Amazon

“I have met a problem when I was trying to get the salesrank of digital cameras

under the category of Photo. It showed me pretty different salesranks from what

we saw through Amazon Website…”

PayPal

“Can someone explain the precise distinction between 'Completed' and

'Processed' [ as the output of a payment service]?”

FedEx

“You can find information about Web Services on the FedEx Site. But when you

look at the [Developer Guide], it's two pages long and incredibly vague”

6

Part II

Proposed Solution

Proposed Solution

The main goal of our research is to formally model, specify and

verify the data aspect of the data-centric Web services

The formal specification enables verification of service behavior

Our approach is applicable to any data store that exhibits some

structural organization of the data

We use formal methods and design-by-contract techniques to

achieve this goal8

Formal Methods and Design-by-Contract

“Formal Methods used in developing computer systems are mathematically

based techniques for describing system properties. Such formal methods

provide frameworks within which people can specify, develop, and verify

systems in a systematic, rather than ad hoc manner”

• Advantages

• Unambiguous

• Machine-readable

• Abstract

9

Formal Methods and Design-by-Contract

10

Formal Methods and Design-by-Contract

11

Formal Methods and Design-by-Contract

service S requires P ensures Q

• P : precondition of the service S

• Q: postcondition of the service S

12

Research Hypotheses

Main Hypothesis

The formal specification and verification of data-centric Web services

can be facilitated by exposing a formal data contract

Methodology

1. We develop a formal model for an abstract source of data and

develop a contracting framework for data-centric Web services

based on the abstract model

2. We investigate how the data contract can be used to specify and

verify service behavior

13

Modeling a Data-Centric Web Service

class GenericDataModel attribute entity1: Set(GenericRecord1)

attribute entity2: Set(GenericRecord2) ... attribute entityn: Set(GenericRecordn)...

Data source set of Entities Entity set of RecordsRecord set of Attributes class GenericRecord

attribute key: Tkey

attribute attr1: T1

attribute attr2: T2

... attribute attrm: Tm

end GenericRecord

14

The Data Model

class GenericDataModel... operation GenericRecordi findRecordByKey(key: GenericKeyi)

requires (GenericKeyi is the key for GenericRecordi )

ensures (result.key = key and result in this.entityi) or result = NIL

operation Set(GenericRecordi) findRecordByCriteria( values1: Set(Ti1)

values2: Set(Ti2), ...

valuesn: Set(Tin))

requires ( Tij is the type of the jth attribute of GenericRecordi )

ensures rec in result, rec.attrj in valuesj and result in this.entityi

operation GenericDataModel createRecord(gr: GenericRecordi) requires this.findRecordByKey (gr.key) = NIL ensures result.entityi = this.entityi U gr and j ≠ i, result.entityj = this.entityj

operation GenericDataModel deleteRecord(key: GenericKeyi) requires this.findRecordByKey(key) ≠ NIL ensures result.entityi = this.entityi – this.findRecordByKey(key)

and j ≠ i, result.entityj = this.entityj

operation GenericDataModel updateRecord(gr: GenericRecordi) requires this.findRecordByKey(gr.key) ≠ NIL ensures result.entityi = this.entityi – this.deleteRecord(gr.key).createRecord(gr)

and j ≠ i, result.entityj = this.entityj

end GenericDataModel

Data operation Create – Read – Update - Delete

operation Set(GenericRecordi) findRecordByCriteria( values1: Set(Ti1) values2: Set(Ti2),

... valuesn: Set(Tin))

requires ( Tij is the type of the jth attribute of GenericRecordi )

ensures rec in result, rec.attrj in valuesj and result in this.entityi

15

Proposed Methodology

1. Abstracting the service underlying data model as set of records, and

identifying their attributes’ types

2. Implementing the service data model as a class using our abstract data

model as a template

3. Annotating the data model class with invariants that define any data

constraints or business rules

4. Annotating the service interface with formal specifications that are

defined in terms of the data model and data functionalities defined in

step 216

Example: Amazon.com ItemSearch Web Service

ItemInfo[] ItemSearch ( searchIndex: CatString, keywords: String, minPrice: Double, maxPrice: Double,

author: String, artist: String, title: String, availability: AvailString, merchant: String, sort: SortString )

Array of [itemId: Integer, detailPageURL: String, title: String, author: String, artist: String, price: Double]

Enum {Books, CD, DVD, All}

Enum {Available} Enum {price, -price}

17

Example: Amazon.com ItemSearch Web Service

Data Model

class ItemRecord

attribute key: Integerattribute category: { Book, CD, DVD }attribute merchantName: Stringattribute author: Stringattribute artist: Stringattribute title: Stringattribute price: Floatattribute stockLevel: Integer

invariant stockLevel ≥ 0invariant price ≥ 0invariant title ≠ NILinvariant (category = CD or category = DVD) author =

NILinvariant (category = Book) artist = NILinvariant (stockLevel > 0) merchantName ≠ NIL 

end ItemRecord

class ItemSearchDataModel 

attribute itemEntity: Set(ItemRecord) end ItemSearchDataModel

invariant (stockLevel > 0) merchantName ≠ NILinvariant (category = Book) artist = NIL

18

ItemSearch Sample Pre- and Post-Conditions

//The following specification variables are assumed: isdm: ItemSearchDataModel searchIndices ,authors, artists, titles, merchants, prices, stockLevels: Set(String)

// Precondition on input price rangerequires minPrice ≥ 0 and maxPrice ≥ 0 and minPrice maxPrice

//Searching by keywords in the Books category ensures (old(keywords) ≠ NIL) and (old(searchIndex) = Books) Book searchIndices and old(keywords) authors and old(keywords) titles

// Specifying the results in terms of the service inputs and the defined model ensures i, 1 ≤ i < result.length, result[i] { [rec.key, ”http://www.amazon.com”+rec.key, rec.title, rec.author, rec.artist] | rec isdm.findRecordByCriteria(searchIndices, merchants, authors, artists, titles, prices, stockLevels)}

 

//The following specification variables are assumed: isdm: ItemSearchDataModel searchIndices ,authors, artists, titles, merchants, prices, stockLevels: Set(String)

// Precondition on input price range requires minPrice ≥ 0 and maxPrice ≥ 0 and minPrice maxPrice

//Searching by keywords in the Books category ensures (old(keywords) ≠ NIL) and (old(searchIndex) = Books) Book searchIndices and old(keywords) authors and old(keywords) titles

// Specifying the results in terms of the service inputs and the defined model ensures i, 1 ≤ i < result.length, result[i] { [rec.key,”http://www.amazon.com”+rec.key, rec.title, rec.author, rec.artist] | rec isdm.findRecordByCriteria(searchIndices, merchants, authors, artists, titles, prices, stockLevels)}

19

Specification and Verification of Compositions

1. A service provider abstracts the data

source(s) of into a formal data model

2. The data model is used to annotate the

service with a data contract

3. The service consumer consults the

individual contracts to understand the

behavior of each service and construct

a global data contract

4. The service consumer can formally

verify the correctness of the

composition with the global data

contract

Iman Saleh, Gregory Kulczycki and M.Brian Blake, "Formal Specification and Verification of Data-Centric Service

Composition", IEEE International Conference on Web Services, July 2010. Acceptance Rate 15.6%.

Formal Verification Process

Part III

Empirical Evaluations

Experiment I: Improving Service Reuse

Hypothesis

The proposed contracting approach decreases the level of ambiguity about a Web service

behavior in terms of its underlying data interactions and expected output under different

input conditions

23

Web Crawler

Hidden DatabaseData-Centric Web Service

Indexed Data

Query

Data

Crawler Goals:[G1] Identifying services that are candidates for crawling data (read-only).[G2] Maximizing collected data while trying as few queries as possible.[G3] Avoiding queries that results in crawling duplicate data.[G4] Identifying presentation inputs, i.e. inputs that only affect the output format.[G5] Avoiding invalid queries that would generate error pages or no results.

34,992 31,104

19296

3216

1

10

100

1,000

10,000

100,000

L0 L1 L2 L3 L4 L5

Number of Possible Queries Per Specification Level

Iman Saleh, Gregory Kulczycki and M.Brian Blake, "A Reusable Model for

Data-Centric Web Services", 11th International Conference on Software

Reuse, September 2009.

Experiment II: Static Detection of Implementation Errors Using Code Contracts

Hypothesis

The formal code specification and verification facilitates the discovery

of implementation errors at design time. These errors cannot be

detected using a non-verifying compiler

Null Hypothesis

“Formal code specification does not affect the ability to detect

implementation errors.

24

Iman Saleh, M.Brian Blake and Gregory Kulczycki, “Static Detection of Implementation Errors Using Formal

Code Specification”, submitted to the Journal of Empirical Engineering.

Experiment III: Static Detection of Implementation Errors in Data-Centric Web Services

Hypothesis

Our proposed formal model and contracting framework enables

detection of programming errors at design time in data-centric Web

services.

Null Hypothesis

Our formal model and contracting framework does not affect the ability

to detect design-time implementation errors in data-centric Web

services

25

Example

public int BinarySearch(int[]! a, int key) requires a.Length > 0; ensures 0 <= result ==> a[result] == key; { int low = 0; int high = a.Length - 1; while (low <= high) invariant high+1 <= a.Length; invariant low >= 0; { int mid = (low + high) / 2; int midVal = a[mid]; if (midVal < key) { low = mid + 1; } else if (key < midVal) { high = mid - 1; } else { return mid; // key found } } return -(low + 1); // key not found. }

26

invariant high+1 <= a.Length;invariant low >= 0;

Loop Invariant

requires a.Length > 0; Method Preconditionensures 0 <= result ==> a[result] == key; Method

Postcondition

! Non-Null Type

Data SetA set of 17 formally specified data functionalities extracted from a Book Rental application.

27

No. Functionality Description

1 create_book Creates a new book given the book ISBN, title, publisher and category 2 create_category Creates a new book category given the category name 3 create_customer Creates a new customer given the customer name 4 create_publisher Creates a new publisher given the publisher name 5 create_user Creates a new application administrator given a username and password 6 delete_customer Deletes an existing customer given the customer’s name 7 delete_publisher Deletes an existing publisher given the publisher’s name 8 delete_user Deletes an existing administrator given the administrator’s name 9 find_book_by_isbn Searches for a book given an ISBN 10 find_category_by_name Searches for a category given a name 11 find_customer_by_name Searches for a customer given a name 12 find_publisher_by_name Searches for a publisher given a name 13 find_user_by_username Searches for an administrator given a username 14 purchase_book Creates a new purchase given the book ISBN, quantity and price per book 15 rent_book Creates a new rental given the book ISBN, rental days, price per day and

customer id 16 return_book Returns a book given an ISBN 17 update_user_password Updates an administrator’s password given a username

Experiment Design1. A class with all the 17 functionalities is implemented. The class contains variables that represent

the underlying database according to our model.

2. All methods in the class are specified using Spec#. The specification defines how these methods

interact with the data model variables.

3. The specified implementation is verified using Boogie to ensure that it is initially correct with

respect to the specifications.

4. A fault injection tool is used to automatically introduce errors in each class.

5. The verifier is executed on each mutant of each class.

6. Step (3) is repeated for different types of errors and the total number of errors detected using

different specification levels is calculated.

Specification Levels:

• L0: No specification, this level acts as a baseline

• L1: Non-null types

• L2: Non-null types and invariants

• L3: Non-null types and preconditions

• L4: Full specification28

Mutation Operators

29

No. Operator Description Example

1 ABS Replacing a numerical value with its absolute value

int x = y to int x = Math.Abs(y)

2 AOR Arithmetic Operator Replacement a = b + c to a = b – c

3 ROR Relational Operator Replacement while(a < b) to while(a > b)

4 UOI Unary Operator Insertion a = b to a = -b

5 UOR Unary Operator Replacement i++ to i--

Experiment Design

Independent Variable

The Specification Level: Nominal Variable (L0, L1, L2, L3, L4)

Dependent Variable

The Mutation Score: Ratio Variable

Mutation Score = Number of Detected Errors

Total Number of Injected Errors

30

Results and Analysis

31

A total of 134 mutants were generated and formally verified

No. Method Mutation Operators No. Of Mutants

1 create_book UOR 32 create_category UOR 33 create_customer UOR 34 create_publisher UOR 35 create_user UOR 36 delete_customer ABS – UOI 47 delete_publisher ABS – UOI 48 delete_user ABS – UOI 49 find_book_by_isbn ABS – ROR – UOI – UOR 1410 find_category_by_name ABS – ROR – UOI – UOR 1411 find_customer_by_name ABS – ROR – UOI – UOR 1412 find_publisher_by_name ABS – ROR – UOI – UOR 1413 find_user_by_username ABS – ROR – UOI – UOR 1414 purchase_book ABS – AOR – UOI – UOR 1115 rent_book ABS – AOR – UOI – UOR 1116 return_book ABS – AOR – UOI – UOR 1117 update_user_password ABS – UOI 4Total 134

Results and Analysis

32

The Mutation Score achieved at different specification levels

- L0: No specification

- L1: Non-null types

- L2: Non-null types and invariants

- L3: Non-null types and preconditions

- L4: Full specification

Results and Analysis

33

•The Mutation Score calculated per error type

Results and Analysis

34

• The highest level of specification used in our experiment failed to detect some

mutations. These mutations however do not introduce errors:

• Examples

n = category_pointer mutated to n =

+category_pointer

n-- mutated to --n

• Invariants added to some functionalities helped detect errors. Not all

functionalities had invariants, which explains the high variance in L2 results.

Validity Discussion

Internal Validity

Limited number of mutants for some

programs The calculated mutation

scores can be treated only approximately,

showing certain trends, but without

sufficient statistical power

External Validity

The Programming and Specification

Languages: C# and Spec#

35

Part IV

Summary

Summary of Contributions

The design and implementation of a reusable data model that enables the formal

representation of a data store.

A service contracting mechanism is proposed based on our data model that is used

to construct formal specification of the underlying data behavior.

Our proposed specification is represented in the form of a machine-readable

contract that can be used to automate the verification process.

We propose a methodology to verify data integrity properties in an ACID

transaction.

Empirical evidence is provided on the effect of formal methods on writing bug-free

code.

We present a comparative study using three of the leading specification languages.37

General Discussion and Future Work

Contracts for Web Services

• Challenges

• Testing and debugging

• Wide Applicability

• The promise of automatic service invocation by software agents

• Services are invoked at the provider’s end

• Integration risk due to changes in service code and data schema

• Opportunities

• Monitoring reuse

• Empowering the online community

38

Summary and General Discussion

Automatic Testing of Database Applications

39

ConstraintSolver

Extract PreconditionsInput Values + Test DB Values

Solve

Populate DB

Test CaseCollect Code

Coverage

Run and Monitor

Reset DB and Repeat

Ongoing Research: Applying to Big Data…

40

Picture Source: Mahesh Gudipati, Shanthi Rao, Naju D. Mohan, Naveen Kumar Gajja, “Big Data: Testing Approach to Overcome Quality Challenges”, 2013.

Ongoing Research: Leveraging the Crowd….

41

Questions?

Publications

Journals and Magazines

1. Iman Saleh, M.Brian Blake and Gregory Kulczycki, “Formal Methods for Data-Centric Web Services: From Model to Implementation”,

submitted to the International Journal on Software Tools for Technology Transfer.

2. Iman Saleh, M.Brian Blake and Gregory Kulczycki, “Static Detection of Implementation Errors Using Formal Code Specification”,

submitted to the Journal of Empirical Engineering.

3. Iman Saleh, M.Brian Blake and Gregory Kulczycki, "Demystifying Data-Centric Web Services", IEEE Internet Computing, vol.13, no.5,

pp.86-90, Sept.-Oct. 2009.

Conference Papers

1. Iman Saleh, Gregory Kulczycki and M.Brian Blake, "Formal Specification and Verification of Transactional Service Composition", 2011

IEEE World Congress on Services, SERVICES'11, July 2011.

2. Iman Saleh, Gregory Kulczycki and M.Brian Blake, "Formal Specification and Verification of Data-Centric Service Composition", IEEE

International Conference on Web Services, July 2010. Acceptance Rate 15.6%.

3. Iman Saleh, "Formal Specification and Verification of Data-Centric Web Services", IEEE Services' PhD Symposium, July 2010.

4. Iman Saleh, Gregory Kulczycki and M.Brian Blake, "Specification and Verification of Web Services Transactions", PhD Workshop on

Innovative Database Research, SIGMOD'10, June 2010.

5. Iman Saleh, Gregory Kulczycki and M.Brian Blake, "A Reusable Model for Data-Centric Web Services", 11th International Conference on

Software Reuse, September 2009.

6. Khaled El-Goarany, Iman Saleh and Gregory Kulczycki, "The Social Service Network - Web 2.0 Can Make Semantic Web Services

Happen," cecandeee, pp.419-423, 2008 10th IEEE Conference on E-Commerce Technology and the Fifth IEEE Conference on

Enterprise Computing, E-Commerce and E-Services, July 2008 43

Contact: iman@miami.edu