26/04/2007BIS'07 Poznan, Poland1 Evaluating Quality of Web Services: A Risk-driven Approach Natallia Kokash Vincenzo DAndrea

26/04/2007 BIS'07 Poznan, Poland 1

Evaluating Quality of Web Services:

A Risk-driven Approach

Natallia KokashVincenzo D’Andrea


Introduction□Service-centric systems□Quality of Service (QoS) Issues

□QoS-driven service selection□Risk-driven service selection

□Risk analysis□SOA risks□Failure risk

□Experimental results□Conclusions and Future Work

□Risk management for SOA□References


Service-centric systems

s2

s1

++

s3

s5

+s4+

Client

Provider

Partners

s0

Invoke

Sequential operator

s1 ; s2

Parallel operator s1 | s2

Choice operator s1 + s2

Web service si

Start state t0

End state t

si

+

|

Invoke


Quality of Service Issues□ QoS for web services:

□ Domain-independent□ Throughput, capacity, latency, response time (duration),

availability, reliability, reputation, execution cost (price)□ Domain-dependent

□ Currency converters: accuracy□ Hotel booking: prices, number of the rooms, availability rate

□ How to:1. specify QoS? 2. measure QoS?3. specify user requirements and/or preferences about QoS?4. match user requirements with existing services in terms of

QoS?5. rank services according to user preferences?6. predict QoS factors under certain environmental conditions? 7. choose web services to guarantee certain QoS level of their

composition?


QoS-driven service selection

□ Problems in quality-driven service selection:□ Lack of QoS statistics□ Volatility of QoS factors□ Multidimensionality □ Subjectivity□ Context-dependence

□ Approaches□ Multi-attribute optimization [Ardagna and Pernici 2005, Zeng et al.

2004, Yu et al. 2005 ]□ Constraints satisfaction [Martin-Diaz et al. 2005]□ Genetic algorithms [Canfora et al. 2006] □ Fuzzy [Lin et al. 2005]

□ Problems with existing approaches□ Simplified models (e.g., one service for one task)□ Dependences among QoS factors are ignored□ Context is not taken into account


Risk analysis

□ Example:□ Movie: title= Rainmaker, format=DVD,

languages=Italian, English□ Convert DVD to AVI: language=English□ SimpleDivX converter: time=2 hours, language = Italian□ Impact on time: 2 hours are lost□ Reason: Unexpected service behaviour (discrepancy

with specification)

□ Requires assessment of inherently uncertain events and circumstances

□ Two dimensions: □ how likely the uncertainty is to occur (probability)□ what the effect would be if it happened (impact)


SOA Risks

□ Threats□ Loss of service, data, users□ Unexpected service behavior, changes□ Performance problems□ Contract violation

□ Assessment□ Likelihoods and implications of threats□ Analysis of user expectations□ Service testing□ User feedback, reputation systems

□ Mitigation□ Service selection, redundancy, redesign□ Runtime monitoring□ Contracts and policies


Risk management for SOAInformation gathering

(service discovery, QoS data)

Decision making(are risks acceptable?)

Analysis of system artifacts (service testing, conformance evaluation)

System configuring(service selection, adaptation, composition,...)

Risk identification(Business risks, technical risks)

Risk analysis

Risk prioritization

Risk mitigation (policies, SLAs,...)

No

Redesignloop

Fix configuration

Define controls

Yes

Evolutionalloop

History


Risk-driven service selection

Loss function – defines the cost of service failure (money, time, resources)

Choose the composition that maximizes the expected profit:


Failure risk□probability that some fault occurs□resulting impact of this fault on the

composite service

where is the probability of the service failure.

□ Loss function includes:□ Expenses to invoke failed service (its cost and

response time)□ Service failure can cause rollback of the transaction,

therefore expenses to execute precedent services are also included

□ The provider may have to pay penalty to a user whose request was not accomplished.


Failure risk of service compositions

b-g

b-e

+

+g-t

e-t

+g-e

+

b-g

b-e

+

+g-t

e-t

+g-e

+


Failure risk: examplesSuccess rate = 0.5; execution cost = 1; penalty = 2

1.75

1.375

1.5625

1.625

1.25

s1 s2

s1 s2

s3

+ +s4

s2s1

s3

+ +

s1

s3

+ +s2

s4

++

s1s2

s3

++


Risk-driven selection algorithm

□ Select an execution path with minimum risk value□ Notation:

□ c – composition

□ q(si) – quality parameter (response time, execution cost)

□ p(si) – probability of success

□ qmax – resource limit

□ Objective function:

where


Experimental results (1)

□Goal: Compare QoS of compositions chosen by our algorithm with QoS of compositions chosen by other methods

□Zeng et al. [2004]□QoS factors: price, duration, reputation,

success rate, availability □Objective function: linear combination of

scaled QoS factors□Scaling: QoS factors range from 0 to 1□Weights reflect user preferences


Experimental results (2)□ 100 simulated service compositions□ 10 services in each composition


Conclusions and Future work

□A novel risk-based method for assessing QoS of web services is proposed

□Real world case studies□Comparative analysis of existing service

selection algorithms□Risk management framework for

automatic web service compositions

□Questions?


References1. [Ardagna and Pernici 2005] Ardagna, D., Pernici, B.: ”Global and Local QoS

Constraints Guarantee in Web Service Selection,” IEEE International Conference on Web Services, 2005, pp. 805–806.

2. [Canfora et al. 2006] Canfora, G., di Penta, M., Esposito, R., Villani, M.-L.: “QoS-Aware Replanning of Composite Web Services”, Proceedings of the International Conference on Web Services, 2005.

3. [Claro et al. 2005] Claro, D., Albers, P., Hao, J-K.: “Selecting Web Services for Optimal Composition”, Proceedings of the ICWS 2005 Second International Workshop on Semantic and Dynamic Web Processes, 2005, pp. 32-45.

4. [Gao et al. 2006] Gao, A., Yang, D., Tang, Sh., Zhang, M.: “QoS-driven Web Service Composition with Inter Service Conflicts”, APWeb: 8th Asia-Pacific Web Conference, 2006, pp. 121 – 132.

5. [Lin et al. 2005] Lin, M., Xie, J., Guo, H., Wang, H.: “Solving QoS-driven Web Service Dynamic Composition as Fuzzy Constraint Satisfaction, IEEE International Conference on e-Technology, e-Commerce and e-Service, 2005, pp. 9-14.

6. [Martin-Diaz et al. 2005] Martin-Diaz, O., Ruize-Cortes, A., Duran, A., Muller, C.: ”An Approach to Temporal-Aware Procurement of Web Services”, International Conference on Service-Oriented Computing, 2005, pp. 170–184.

7. [Zeng et al. 2004] Zeng, L., Benatallah, B., et al.: ”QoS-aware Middleware for Web Services Composition”, IEEE Transactions on Software Engineering, Vol. 30, No. 5, 2004, pp. 311–327.

8. [Yu et al. 2005] Yu, T., Lin, K.J.: ”Service Selection Algorithms for Composing Complex Services with Multiple QoS Constraints”, International Conference on Service-Oriented Computing, 2005, pp. 130–143.

Documents

26/04/2007BIS'07 Poznan, Poland1 Evaluating Quality of Web Services: A Risk-driven Approach Natallia Kokash Vincenzo DAndrea