Embed Size (px)
Citation preview
Measuring Reliability of Applications Composed of Web Services
Hangjung Zo, Derek L. Nazareth, Hemant K. Jain University of Wisconsin-Milwaukee
[email protected], [email protected], [email protected]
Abstract
Web services represent an alternative basis for the rapid development of application systems. Much of the research on composition and orchestration of web services centers around functional sufficiency and performance. Non-functional characteristics like reliability and security play an important role in the selection of web services by system architects. This paper provides a basis for measuring reliability of an application system that is assembled using web services. It outlines criteria for measuring reliability for individual tasks, specifies combining functions for differently configured tasks, and applies these techniques to a problem which contains various sizes of business processes to demonstrate the efficacy of the approach.
1. Introduction
Business environments are becoming more uncertain, inter-networked, and globally competitive [11]. To succeed in these circumstances, firms need to respond promptly and appropriately to external stimuli and market changes [3]. Information technology (IT) is considered to be one of the key enablers for building an agile and flexible business infrastructure to obtain and sustain competitive advantage. Traditional approaches for developing information systems tend to suffer from long development times and high costs, often resulting in products of uncertain quality. A number of alternative system development methodologies have been proposed to deliver applications more quickly, including rapid application development, extreme programming, agile and lightweight methodologies, and the like. These alternatives all require that software artifacts be coded, tested, debugged, and integrated into a cohesive whole. Technology can assist in the automation of part of this development effort. However, the rapid response desired in today’s environment suggests an alternative approach may be preferable.
Web services represent an alternative approach that offers some promise in this direction. Characterized as “software constructs that expose business functionalities over the Internet” [22], web services are considered to be the next technology to provide new IT architecture and strategies for enterprises. Hagel and Brown [9] declare that the age of proprietary IS has come to an end, and predict that firms will purchase or outsource their IT capabilities as services provided over the Internet in the near future. Web services enable firms to reduce the cost of conducting business, deploy solutions faster, and create new business opportunities [13]. Web services have a potential to offer solutions for building a more flexible IT architecture and integrating heterogeneous IS across enterprises in the near future.
Assembling an application using web services requires that the underlying business process be clearly specified, appropriate web services be selected for each task, and the set of selected services integrated into a cohesive application. Several factors need to be considered when selecting appropriate web services to support a specific task. At the outset, the requirements of the task must be met by the functionality provided by the web service. Satisfying this constraint may still leave an application designer with several candidate web services to choose from. Cost and performance are other criteria that have been used in the selection process [2][15][21][24]. We propose that other non-functional criteria relating to quality of service (QoS) are relevant, and should be considered when selecting a specific web service to support a business task. Prior research in the QoS area has identified several different criteria that can be considered for web service selection.
Menascé [15] characterizes QoS as a combination of several properties, including availability, security, response time, and throughput. Cardoso et al. [2] develop an ontology for the specification of QoS metrics, including task response time, task cost, and
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
1©1530-1605/07 $20.00 2007 IEEEProceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
reliability as QoS dimensions of interest. Sumra and Arulazi [24] suggest that web services QoS requirement include performance, reliability, integrity, accessibility, availability, interoperability, and security. Ran [21] proposes a new web services discovery model using QoS metrics, classifying these metrics into four distinct categories, related to execution, transaction, configuration management, and cost.
Reliability, security, cost, and performance are criteria that are frequently identified as being relevant non-functional requirements when selecting web services. To determine the relative importance of these criteria a survey was conducted among several information technology architects in the Midwest [30]. The relative weights of these criteria were inferred through the analytic hierarchy process [23]. The results were surprisingly consistent, and indicated that security and reliability were among the most important criteria that architects consider when evaluating web services. These results are depicted in Figure 1.
Figure 1. Relative importance of web service selection criteria
This paper examines the importance of reliability, and proposes an approach for its effective utilization in the selection of web services. It adopts a design science approach for addressing the problem. The rest of the paper is organized as follows. Section 2 provides a brief characterization of software reliability, and specializes it to web services. Metrics for reliability, particularly in the context of business processes supported by web services are also provided. The practical considerations of developing application systems using web services are outlined in Section 3. Section 4 describes the application of the approach to a real-world example. Implications for managers and system architects are presented in Section 5. Limitations and conclusions complete the paper.
2. Web Services Reliability
Software reliability is typically characterized as the probability that a piece of software will exhibit failure-free operation during a specified period. Xie [27] defines software reliability as “the probability that the software will be functioning without failures under a given environmental condition during a specified period of time”. Other characterizations tend to measure reliability in terms of a percentage of failure cases in a given number of attempts to compensate for variations in usage over time [7][18]. The use of multiple concepts of failure and metrics for assessment of failure gives rise to a host of definitions. In the software context, failure is generally indicative of the exposure of underlying design faults by selection of specific inputs or operating conditions. Breaking down the source and nature of failures permits more precise modeling of software artifact reliability. However, most characterizations of reliability of web services prefer to leave the sources of failure unstated [1][12][27]. Some researchers view reliability as a non-functional characteristic of web services [1]. Still others characterize it as a component in setting service level agreements with web services vendors [12]. One of the more comprehensive characterizations of web services reliability is provided by Zhang and Zhang [28] where both functional and non-functional components are considered, and reliability is defined as a composite of correctness, fault tolerance, testability, availability, performance, and interoperability. This approach is useful in modeling reliability of an individual web service, with a view towards predicting when it may fail.
A related stream of research covers reliability of component-based software. Poore et al. [19] proposed an approach comprising models for sampling, system reliability computation from component reliability, and certification to better assist component system designers in the area of reliability planning. Based on a Markov model, they conducted what-if analyses through a series of spreadsheets to study the interaction between component reliability and system reliability. Prowell and Poore [20] present a method for combining reliabilities of components using a Markov chain model. Web services can be viewed as component-based systems, making these findings useful in modeling reliability for web service applications. However, the focus in this paper is more on deriving reliability measures at the application level, rather than modeling and testing reliability.
Web Service Selection Criteria
0.07
0.12
0.360.34
0.11
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Cost Performance Reliability Security VendorReputation
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
2Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
This paper examines reliability of an entire application created from a set of web services. In that context, it places greater emphasis on combining reliability measures for different web services, than on determining the antecedents of reliability for each web service. It views reliability of web services as a measure of failure-free operation, without modeling the source and type of failure. It assumes that reliability estimates for an individual web service are available, either from vendors, or from published third party sources.
2.1. Application Development with Web Services
Application development to support a business process is viewed as selection of appropriate web services to support individual tasks in the process. It is assumed that, for each task, there are several viable alternative web services to select from. Additionally, a web service is assumed to be capable of supporting multiple tasks. Thus, it is possible that the set of services selected for supporting the process may contain some overlapping services allowing some task to have two more services capable of supporting it. These alternative services can increase the reliability of the task. The individual web services are provided by different vendors, and will involve different costs (initial and on-going), different degrees of performance (in terms of time to service the task), different levels of reliability, and will exhibit different levels of security. Application development then entails selecting the appropriate set of web services that leads to the “best” performance on these dimensions, recognizing that they likely will not all work in concert, i.e. the fastest solution, will not likely be the most secure, or the cheapest, etc. In this paper, we focus on the reliability aspects of the problem only. The use of a single criterion permits a more effective search among possible alternative solutions. It also permits the effective showcasing of how reliability of a process can be effectively estimated through the reliability estimates for individual web services.
Web services can be characterized as remote procedure call methods over the Internet. Using this characterization, it is more natural to measure reliability as a percentage of failures rather than a probability that the failure will occur during a fixed time frame. For example, a web service that exhibits 99.7% reliability suggests that the service will fail at most three times out of one thousand attempts. The reliability of individual web services can be easily
estimated using usage logs, and measured by the number of failures per unit attempt. If access to web service logs is unavailable, vendor-provided data on reliability can be used as the estimate. However, the overall reliability of the business process depends upon the structure of tasks in the process. Aggregating the individual reliability into a reliability estimate for the entire process is discussed in the next sub-section.
2.2. Aggregating Reliability Measures
Reliability for an individual web service is measured as the number of failures per unit attempts, and is presented as a number between 0 and 1. The aggregated reliability of a business process that is composed from web services depends on the structure of the business process, the degree of independence between web services, and whether there are multiple web services (in the set of selected services) capable of supporting the task. Tasks in a business process can be related in a number of ways. Cardoso et al. [2] identify the following relationships: sequential, parallel, conditional, simple loop, and dual loop. The independence assumption permits simpler combining of reliability estimates, since joint probabilities are not involved. It can be argued that this assumption does not always hold, in that multiple web services from a vendor may prove to be unavailable if the vendor’s site is down temporarily. This paper adopts the standard characterization of independence, though. Table 1 summarizes Cardoso et al. [2]’s computing method of the aggregated reliability. A similar approach is adopted by Tsai et al. [25].
As indicated earlier this paper assumes overlapping functionality of the selected web services. Thus, a task may have alternate web services (in the selected set) capable of supporting it which will have an effect on the reliability associated with the task. This paper considers reliability at the task level, rather than the web service level. Redundancy of support for a task needs to be accounted for. The paper assumes that failure at the task level constitutes failure of all alternate web services capable of supporting the task. Once again, independence between web services is assumed. The reliability of task k can then be computed as:
1 (1 )k j
j S
R R
where S represents the set of all web services that are selected to support the business process, and Rj
represents the reliability associated with web service j.
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
3Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
Table 1. Combining reliability measures Reliability Notation
Sequential
( ) ( )i jR R t R t
Parallel(AND split – AND join)
( ) ( ) ( )a b ii
R R t R t R t
Conditional (XOR split – XOR join)
( ) ( ) ( )a b i ii
R R t R t p R t
Simple-loop
)(1
)()1(
ii
ii
tRp
tRpR ,
wheren
joji pp
1
1
Dual-loop
)()(1
)()1(
jii
ii
tRtRp
tRpR ,
wheren
joji pp
1
1
3. Composing Applications with Web Services
Selecting the “best” combination of web services to support a business process represents a challenging problem. At a minimum, the web service must meet all the functional requirements for the task. In addition, non-functional requirements like cost, performance, reliability, security, vendor reputation, needs to be factored in the selection. A software architect’s preference will always lean towards an ideal
solution that exhibits the lowest cost, fastest performance, greatest reliability, highest level of security, and so on. Reality dictates that these criteria will not work in concert, and some tradeoff will need to be made when selecting the preferred combination. Preference tradeoffs among these criteria will vary with the nature of the business process, the organization, the architect’s intrinsic risk propensity, and the specific context for the business process. These tradeoffs require specific articulation from the system designer, and are likely to be cognitively taxing. This paper focuses on one of these criteria, namely reliability, and seeks to explicate the complexities that entail with this approach to application development.
The adoption and eventual success of web services as a technology for delivering applications is clearly dependent upon the availability of appropriate web services to support a given task. The current status indicates that while a number of services are available to support specific tasks like credit processing, broader support for all ranges of tasks is lacking. As a result, there may be only a limited set of web services to choose from for any given task. This situation is expected to be alleviated to some extent as more vendors enter the market, and successful adoption creates a need for more options to choose from. In addition, the overall success of the technology assumes interoperability between web services, and the push for standards will facilitate this [14]. WS-Reliability represents a standard proposed to address this [6]. However, it is directed more at the messaging between web services, and is of less consequence at the business process level.
Assuming a mature web services market, with a healthy number of alternatives to select from, the selection task becomes combinatorially explosive. Assuming a set of m tasks, with n web services available for each task, the total number of viable combinations is given by nm. For a complex process supporting 40 tasks with an average of 8 web services available to support each task, this translates to 840
solutions, or approximately 1.33x1036. Clearly an exhaustive search among these many alternatives is infeasible.
Combinatorially explosive search is not a new problem. In the context of software testing, Cohen et al. [4] have developed an automatic efficient test generator (AETG) using ideas from statistical experimental design theory to minimize the number of tests. The presence of non-linear combining functions
ti
tj
pi
po1
…
pon
ti …
pi
po1
pon
ta t2 tb
t1
tn
…
p1
p2
pn
ta t2 tb
t1
tn
…
ti tj
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
4Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
for computing reliability of web services-based applications precludes the use of this approach. To some extent, the problem can be simplified by decomposing the business process into sub-processes that are essentially independent in terms of their relative functionality. This represents a reasonable approach in that the likelihood of the tasks repeating from one sub-process to another is relatively low. If the business process can be partitioned into sub-processes involving 5 to 15 tasks apiece, the number of viable solutions translates to 32,000 to 3.53x1013. A decomposition strategy would appear to make the problem more tractable. Nonetheless, arbitrary decomposition is unacceptable, and the decomposition is likely to fall along commonality of task characteristics, e.g. tasks that are chronologically related, or performed by the same user, etc. This paper advocates a decomposition strategy that is semantically driven, using commonalities like sub-process objectives, and chronological relationship. It should be recognized that decomposition will likely produce a sub-optimal solution. The quality of that solution needs to be compared to that of the ideal to assess the viability of the technique.
A number of different selection techniques are available to address this problem. In principle, the problem can be cast in terms of a mathematical programming structure. Since only one criterion is being considered, this approach does not entail any tradeoffs among different criteria. However, the estimate for process reliability does not represent a linear function, or even a continuous non-linear function. Instead, it is characterized by discontinuities, sums, and products, making for a difficult translation into a clean function. This would appear to rule out mathematical programming as a viable approach to addressing the selection problem.
In light of the problem size, and the inherent complexity of the underlying functions, a satisficing approach may be preferable to the optimal approaches. Genetic algorithms, simulated annealing, and tabu search represent possible options. The genetic algorithm approach is an adaptive search technique based on principles of natural evolution and heredity. Since its initial development by Holland [10], they have been widely used in variety of areas including optimization, machine learning, genetics, economics, social systems, etc. [5][8][16]. Generally, genetic algorithms are considered appropriate methods for solving a problem in a space that is too large to be searched exhaustively, and is not smooth and unimodal, or not well-understood. In addition, if the
fitness or evaluation function is noisy, or if a non global optimum solution is acceptable, genetic algorithms will provide satisfactory solutions [17]. Issues of relevance in casting the selection problem as a genetic algorithm problem include problem representation, initial population generation, robustness of the evaluation function, and genetic transformation. This paper examines the effectiveness of genetic algorithms as an option for addressing web services selection problem.
4. Application to a Drop-Ship Example
In an effort to demonstrate the viability of this approach, the technique was applied to a drop-ship business process. The drop-ship business process refers to a retail business model in which a product is delivered directly to a customer in a timely and efficient manner from a direct supply vendor (DSVendor) with an online retailer on the Internet taking customer orders and providing direct customer service management [26]. This is one of the typical business processes in the online retailing area. We believe that it is an appropriate example as a composed application with web services. The process is depicted in Figure 2.
The process comprised 34 tasks, and utilized an average of 6 web services per task. This put the total number of viable solutions at 2.87x1026. Clearly the problem was too large to solve exhaustively. The business process was then decomposed into four separate process of customer identification, sales order processing, purchase order management, and shipping, involving 4, 14, 6, and 10 tasks respectively, with 24, 75, 33, and 57 candidate web services apiece. This yielded 3.32x105, 1.78x1026 1.29 x109, and 3.62x1017
possible solutions. Not all of these were feasible solutions, and a separate computation indicated 1.05 x105, 4.50 x1017, 8.20 x106, and 1.56x1012 feasible solutions. Despite the reduction from the original number, an exhaustive search for the second sub-process would still take several years of computation. These details are summarized in Table 2.
A genetic algorithm formulation of the problem was assembled. Chromosomes were created for each sub-process, with lengths determined by the number of tasks and number of web services per task. The evaluation function was based on the technique for estimating reliability as outlined in Section 2. The actual computation of reliability proved to be more challenging than initially anticipated. Reliability for
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
5Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
������������������� ���������� ��������������������� � ��� ���� ����������� ���� � ������� � ���� ������������ ���� ������������������������������������������������������������ ����������� ��������!�����"#�!��������� �����������������������������������������������#� �$���������������������� ������������������������������������������������������������������� �# � &������ � �� � � ��� � �'��� �� � �����
���������� � ����� � ����������� � ��� � �� � � �'��� � � ��� ����������������� ��*�������������� ������������������������������� ����#��!��������� ��*�����������������!�����+#� �!�������������/� ���������������������������������#
������������ �����������������������������������
��
��
��
��
�� ��
���� ���
�� ���
��
��
������������������������������
����������� ������������������!
���������������"��������#���!����
����������� "�$$��!
%
&
' (
�
�� �� �� �� ��
�%
�'
�&
�(
��
��
��
����
��
��
�%
�&
�' �( �� ��
�� ��
�� ��
)����
��*����
��
�
�
�
�
�+�!�������
,�"����������
��
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
6Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
Table 2. Characteristics of drop-ship example Sub-process Customer
IdentificationSales OrderProcessing
Purchase Order Management
Shipping
Tasks 4 14 6 10
Web services 24 75 33 57
Total solutions 3.32x105 1.78x1026 1.29 x109 3.62x1017
Feasible solutions 1.05 x105 4.50 x1017 8.20 x106 1.56x1012
Table 3. Computing of reliability for sub-processes Sub-process Reliability (R)
11A Bp p ,
1 2 3 4 2 2 1 3 4A B A Bp R R p R R R R p R p R RR
2
1C Dp p , 1E Fp p , 1G Hp p , 1I Jp p ,
5 6 7 12 138 10 9 11 14 15 16 17 18
5 6 7 12 13
1 (1 )
1 1E C D I
G HE C D I
p p R p R R p R RR R p R p R R R R R R
p p R p R R p R RR
31K Lp p ,
19 22 20 21 23 24K LR R p R R p R RR
41M Np p ,
25 27 28 29 30 31 32 33 34 26M NR p R R R R R R R R p RR
The population size was set at 1000. Crossover and mutation parameters were set at 0.8, and the algorithm was run for 300 generations. Genetic algorithm parameters are summarized in Table 4. Later, the sub-processes were combined and the analysis performed for the entire process
The results of the genetic algorithm appear in Figure 3, and are depicted for the entire process. As can be easily noted, the best case reliability for different web service combinations hovered around 1 for much of the analysis. The worst case scenario for reliability displayed some divergence, prompting an extension of the analysis to 1000 generations. These results indicate that the very good solutions assessed in terms of the aggregated reliability measure can be easily obtained through the use of genetic algorithms. Unless the reliability measure reaches the theoretical maximum of 1, no definitive claims can be made as to the optimality of the results. Nonetheless, at the end of 1000th generation, the genetic algorithm yields the set
of solutions which perform remarkably well on the aggregate reliability measure. Assuming decision makers adopt a satisficing approach, this heuristic procedure yields adequate results, and designers can select an appropriate solution from the set of best solutions to support a business process.
Table 4. Characteristics of genetic algorithm GA Property Value
Chromosome size 99 to 999 bits
Population size 1000
Evaluation function Symbolic expression of aggregate reliability
Crossover 0.8
Mutation 0.8
Number of generations 300, 1000
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
7Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
Reliability by Generation
0
0.2
0.4
0.6
0.8
1
1.2
1 39 77 115 153 191 229 267 305 343 381 419 457 495 533 571 609 647 685 723 761 799 837 875 913 951 989
Generation
Rel
iab
ility
Best Reliability Worst Reliability
Figure 3. Reliability measures during genetic algorithm analysis
5. Implications for System Designers
Web services represent an alternative approach to traditional system development that offers the opportunity of rapid deployment through the use of readily available artifacts. Development activities of design and construction are now replaced with tasks involving selection, customization, and integration – all of which can be accomplished in far less time than before. In addition, for organizations that are seeking to outsource development and operation of application systems, web services represent an attractive alternative.
Given a large set of potential web services to choose from, it is clear that the number of possible combinations to support a given business process can be extremely large, making evaluation of all solutions potentially infeasible. System designers and managers need to be reassured that good solutions can be obtained, and at relatively little search cost.
This paper tackled one aspect of this search, viz. reliability. It provided a basis for measuring reliability of applications composed of web services. Given the reliability associated with individual web services, the aggregate reliability of an entire business process was derived and then used to guide a heuristic search for good solutions. The results indicate that good solutions can be obtained relatively easily using a genetic algorithm approach, with values of best case scenarios approaching the theoretical maximum of 1.
For designers, this approach provides an opportunity to selectively analyze parts of an extremely large search space and zero in on good solutions rather quickly. Clearly, unless a theoretical maximum is achieved, or an exhaustive search is performed, there is no guarantee that the best performing solution is the optimal one. However, running the genetic algorithm for an extended session
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
8Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
provides greater confidence in the quality of the solution.
6. Limitations and Conclusions
Reliability represents one measure to select web services to support application development. Clearly, several other criteria could be used, including security, performance, cost, and the like, making this a multi-objective non-linear combinatorial optimization problem. In this case, the problem changes to finding the set of non-dominated solutions (Pareto-optimal solutions). Heuristic approaches, including multi-objective genetic algorithms can be effective in addressing multi-objective optimization problems involving [29].
This research can serve as a basis for application composition using web services considering multiple non-functional characteristics. In a service-oriented architecture, this approach can provide meaningful strategies to evaluate multiple criteria and help select appropriate solutions. When dealing with multiple criteria, it must be borne in mind that tradeoffs among different criteria will vary with designer, and a suitable multiple criteria decision making capstone may be required. In addition, it may be desirable to incorporate preferences from managers in addition to system designers. This will likely pose an additional challenge in that their preferences and value tradeoffs among criteria are likely to be different.
Further validation of the approach is necessary. While the present study demonstrates it viability, its true utility needs to be demonstrated through repeated application. Application to other processes – large and small, simple and complex – represents the next step in its assessment. In addition, assessing its robustness in situations where individual reliability varies widely among the web services represents a further area for research.
This study employed a conceptual business process (the drop-ship retail process) as an example. Although the business process is one of the typical online e-business models, application to a real-world case would strengthen its credibility.
Application development through appropriate selection of web services offers the opportunity for faster assembly of applications. In addition, it affords flexible development without the need for in-house
development. The judicious selection of web services in this application composition problem is a challenge nonetheless. This study examined one aspect of the selection, namely reliability. It provided a mechanism for deriving aggregate process reliability from individual web service reliabilities, based on the process structure. For a realistic sized process, even with a limited number of web services to select from, the number of viable candidate solutions is extremely large, rendering exhaustive search and optimization techniques inapplicable. This study adopted a genetic algorithm solution to allow system designers to explore this space selectively, and quickly zero in on promising solutions. The technique was applied to a non-trivial drop-ship retail process. Results indicate that the approach provides good solutions relatively quickly, thereby enabling the system designer to select appropriate combinations of web services that support the business process, and do so with a high degree of reliability.
7. References
[1] A. Arsanjani, B. Hailpern, J. Martin, P. Tarr, “Web Services: Promises and Compromises”, ACM Queue, 1 (1), March 2003, pp. 48-58.
[2] J. Cardoso, J. Miller, A. Sheth, and J. Arnold, “Modeling Quality of Service for Workflows and Web Service Processes,” Technical Report #02-002, LSDIS Lab, Computer Science, University of Georgia, 2002.
[3] L. Cherbakov, G. Galambos, R. Harishankar, S. Kalyana, and G. Rackham, “Impact of Service Orientation at the Business Level,” IBM Systems Journal, 44 (4), 2005, pp. 653-668.
[4] D. M. Cohen, S. R. Dalal, A. Kajla, and G. C. Patton, “The Automatic Efficient Test Generator (AETG) System,” in Proceedings of the 5th International Symposium on Software Reliability Engineering (ISSRE’94), Monterey, CA, November, 1994, pp. 303-309.
[5] Davis, L., Handbook of Genetic Algorithms, Van Nostrand Reinhold, New York, NY, 1991.
[6] C. Evans, D. Chappell, D. Bunting, G. Tharakan, H. Shimamura, J. Durand, J. Mischkinsky, K. Nihei, K. Iwasa, M. Chapman, M. Shimamura, N.. Kassem, N. Yamamoto, S. Kunisetty, T. Hashimoto, T. Rutt, and Y. Nomura, WebServices Reliability (WS-Reliability) version 1.0, Jan. 2003, accessed at http://www.hitachi.co.jp/Prod/comp/soft1/wsrm/ WSReliabilityV1.0.pdf.
[7] A.L. Goel, “Software Reliability Models: Assumptions, Limitation, and Applicability,” IEEE Transactions on
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
9Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
Software Engineering, Vol. SE-11 (12), December 1985, pp. 1411-1423.
[8] Goldberg, D. E., Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, MA, 1989.
[9] J. Hagel, and J.S. Brown, “Your Next IT Strategy,” Harvard Business Review, 79 (10), 2001, pp. 105-113.
[10] Holland, J. H., Adaptation in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, MI, 1975.
[11] S. L. Jarvenppa, and B. Ives, “The Global Network Organization of the Future: Information Management Opportunities and Challenges,” Journal of Management Information Systems, 10 (4), 1994, pp. 25-57.
[12] L.-J. Jin, V. Machiraju, and A. Sahai, “Analysis on Service Level Agreement of Web Services”, HP Labs Report HPL-2002-180, HP Laboratories, 2002.
[13] H. Kreger, Web Services Conceptual Architecture (WSCA 1.0), IBM Software Group, 2001, available at http://www-3.ibm.com/software/solutions/webservices/pdf/ WSCA.pdf.
[14] H. Kreger, “Fulfilling the Web services promise”, Communications of the ACM, 46 (6), June 2003, pp. 29-34.
[15] D. A. Menascé, “QoS Issues in Web Services,” IEEEInternet Computing, 6 (6), 2002, pp. 72-75.
[16] Michalewicz, Z., Genetic Algorithms + Data Structures = Evolution Programs, Third Ed., Springer-Verlag, New York, NY, 1996.
[17] Mitchell, M., An Introduction to Genetic Algorithms,The MIT Press, Cambridge, MA, 1996.
[18] Musa, J. D., Software Reliability Engineering, McGraw-Hill, New York, NY, 1999.
[19] J. H. Poore, H. D., Mills, and D. Mutchler, “Planning and Certifying Software System Reliability,” IEEE Software,10 (1), 1993, pp. 88-99.
[20] S. J. Prowell, and J. H. Poore, “Reliability Computation for Usage-Based Testing,” in Modern Statistical and Mathematical Methods in Reliability, Wilson A., Limnios, N., Keller-McNulty, S., and Armijo, Y. Eds., World Scientific Publishing Co., Hackensack, NJ, 2005, pp. 383-393.
[21] S. Ran, “A Model for Web Services Discovery with QoS”, ACM SIGecom Exchanges, 4 (1), Spring 2003, pp 1-10.
[22] M. Rosen, and J. Parodi, White Paper: Architecting Web Services, IONA Technologies, 2001.
[23] Saaty, T. L., The Analytic Hierarchy Process, McGraw-Hill, New York, NY, 1980. [24] R. Sumra, and D. Arulazi, “Quality of Service for Web Services – Demystification, Limitations, and Best Practices,” Developer.com, March 4, 2003, available at http://www.developer.com/services/article.php/2027911.
[25] W. T. Tsai, D. Zhang, Y. Chen, H. Huang, R. Paul, and N. Liao, “A Software Reliability Model for Web Services”, in Proceedings of the 8th IASTED International Conference on Software Engineering and Applications, Cambridge, MA, November 2004, pp. 144-149.
[26] UN/CEFACT and OASIS, Business Process Analysis Worksheets and Guidelines: Procedures for Developing Business Processes in ebXML, 2001.
[27] Xie, M., Software Reliability Modelling, World Scientific Publishing Co., Singapore, 1991.
[28] J. Zhang, and L.-J. Zhang, “Criteria Analysis and Validation of the Reliability of Web Services-Oriented Systems”, in Proceedings of the IEEE International Conference on Web Services (ICWS'05), Orlando, Florida, July 2005, pp. 621-628.
[29] Zitzler, E., Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, Ph.D. Thesis, Swiss Federal Institute of Technology Zurich, November 1999.
[30] Zo, H., Supporting Intra- and Inter-Organizational Business Processes with Web Services, Ph.D. Thesis, the University of Wisconsin-Milwaukee, August 2006.
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
10Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)0-7695-2755-8/07 $20.00 © 2007
