An Integrated Component Selection Framework for System-Level Design

Chad Calvert, Georgiana L. Hamza-Lup, Ankur Agarwal, Bassem Alhalabi College of Engineering & Computer Science

Florida Atlantic University Boca Raton, Florida

{ccalver3, ghamzal, aagarwa2, alhalabi} @fau.edu

Abstract— The increasing system design complexity is negatively impacting the overall system design productivity by increasing the cost and time of product development. One key to overcoming these challenges is exploiting Component Based Engineering practices. Components are designed for various specifications and usually variants of each component exist with different performance and quality of service parameters. Therefore it is a challenge to select the optimum components from a component library that will satisfy all functional and non-functional system requirements. If these designs and architectural decisions are delayed, they may lead to design re-spins thereby, negatively impacting the product development cost and time. In this paper we propose an integrated framework for component selection.

Keywords-component selection; system architecture; system modeling

I. INTRODUCTION With the vast growth and development of software and

hardware technologies over the past few years, systems complexity has increased dramatically and has raised several challenges in system design and implementation. One key to overcoming these challenges is exploiting Component Based Engineering practices, which are centered around the idea of developing a system from pre-existing components. Designing a system by reusing existing components can also lead to a faster time to market. However, finding the right components that satisfy given system requirements has its own challenges. First, the specification of existing components must include enough information to allow designers/architects to make informed decisions when selecting among available components. Secondly, the large number of existing components, including component variants with similar functionality but different performance or quality of service characteristics, makes component selection a difficult task for a system architect.

Several researchers have addressed these challenges, by focusing on reducing the search space of potential solutions to the component selection problem. For example, several component specification approaches have been proposed [1-3] that aim at augmenting the functional specification of components with additional non-functional attributes, such as, security specifications, compatibility information and standards used. This additional information associated with each component can help minimize the number of potential

components to be considered in the selection process, thus reducing the search space. Other research [4-9] focuses on ranking and classifying components within libraries to speed up the selection process by performing a more structured search through the solutions space. However, in all current research component selection is mainly driven by system requirements that depend directly on the attributes of the individual components. For example, system reliability, cost and delivery time are directly dependent onto the reliability, cost and delivery time of the constituent components. For example, system reliability, defined as the probability of the system to function without failure, can be computed as the product of the reliabilities of the individual components. Similarly, the cost of a system is the sum of the costs of individual components.

In contrast to existing approaches, the proposed integrated framework is driven by system requirements, whose dependencies on the components attributes cannot be expressed by exact mathematical formulas. For example, system performance cannot be expressed in terms of the performance of individual components, because the performance cannot be measured for individual components in isolation but only for the integrated system. Therefore, in order to estimate whether a combination of components satisfy given system performance requirements our framework will first use system modeling [10, 11] and statistical analysis to extract and approximate the dependencies between system performance and component attributes. This represents the first phase of our framework (illustrated in Figure 1) and is based on our previous research in [12] and [13]. The second phase is the actual component selection process, part of which a new simulated annealing algorithm was proposed and compared with two greedy-based approaches that we previously proposed in [12] and [14].

The remainder of this paper is organized as follows. In Section 2 we present our proposed component selection framework and its algorithms. In Section 3 we illustrate the application of the framework to the problem of selecting the components for a Network-on-Chip (NOC) architecture. In Section 4 we present the results obtained from the prototype that was developed to evaluate the proposed framework and to compare the performance of the newly proposed simulated-annealing algorithm with the performance of two existing greedy-based selection algorithms. Finally, we conclude and discuss our future work in Section 5.

978-1-4244-9493-4/11/$26.00 ©2011 IEEE

II. THE COMPONENT SELECTION FRAMEWORK As illustrated in Figure 1, the integrated framework

proposed in this paper is a two-phase approach that includes a system modeling and analysis phase, and a component selection phase, described in the next subsections.

A. Phase 1: System Modeling and Anaysis The purpose of system modeling and analysis phase is

twofold: (1) to annotate components with specific attributes that can then be used as selection criteria in the component selection process; specifically, the attributes of interest are performance characteristics which cannot be measured for individual components in isolation, but only in an integrated system; and (2) to estimate how well a selected combination of components satisfies given system requirements.

To be able to evaluate components, we must model them in advance in an integrated system at an abstract level that hides the detailed system functionality but allows analyzing system performance and assessing whether given requirements can be satisfied using those components. As it is unfeasible to examine all possible combinations of components in a timely manner, data provided from such model can be analyzed and used for predicting system performance for any given combination of components. In our approach, system modeling is used to measure various system performance parameters for a limited number of combinations of components. From this data, two analyses are performed: (1) regression analysis, which computes the dependency of system performance on the

attributes of individual components, and (2) probability analysis, which computes the probability that a certain component will satisfy certain performance requirements. The details of these two analyses can be found in [12] and [13]. The outputs of this phase, namely, regression equations and conditional probabilities, are used in the second phase, the actual component selection process.

B. Phase 2: Component Selection As illustrated in Figure 1, phase two begins from the system

requirements and specification. In the first step, component filtering, the search space is reduced by eliminating components that do not satisfy the constraints given in the system specification. Then a selection algorithm is chosen out of the three algorithms implemented: a greedy approach discussed in [14], a modified intelligent greedy that uses the probabilities calculated in phase 1 as discussed in [12] and a simulated annealing approach proposed in this paper and detailed in the next paragraph. Each of the three component selection algorithms produces a possible combination of components. The performance of this proposed architecture is then estimated based on the regression equations computed in phase 1 and compared against the given system performance requirements. Lastly, if the proposed architecture passes validation it is recommended to the user as a solution. Otherwise, the process loops back to select and evaluate a new combination of components. The simulated annealing approach proposed in this paper for solving the component selection problem is discussed next.

PHASE 2 - COMPONENT SELECTIONPHASE 1 - SYSTEM MODELING AND ANALYSIS

System Requirements &

Specification

High Level System Modeling

Component Ranking with Probability Analysis

System Performance Dependency Estimation with

Regression Analysis

Component Filtering

System Components Selection

System Architecture Evaluation

Simulated AnnealingGreedyIntelligent

Greedy

Component Library

Final System Architecture

System Requirements

Satisfied ?

YES

NO

Figure 1. Integrated Component Selection Framework

In order to apply the simulated annealing method to our component selection problem, the following key parameters were first defined: (1) the search space, is defined as all

possible combinations of components that have passed the filtering step (that is, they satisfy the given system constraints); (2) the energy function is defined based on the regression

equations which approximate system performance for a certain combination of components; (3) the candidate generator, which serves the purpose of perturbing the current configuration (combination of components) to find a possible better solution, is defined as a function which identifies the set of components within a certain distance from the components in the current configuration. Distance between components is calculated based on the numeric values of their attributes. The change in energy from one configuration to another is calculated by examining the difference in terms of system performances between the two configurations. If a potential configuration produces a better system performance, then it may replace the current configuration; (4) the acceptance probability function is defined such that a potential configuration will replace the current one only if it is the best among the ones generated by the candidate generator function; and (5) the cooling schedule is expressed in terms of our energy function; essentially, the search continues until no configuration generated by the candidate generator produces a better energy.

Our simulated annealing algorithm starts from a combination of components chosen at random from the search space. If that combination does not satisfy the goal (the system requirements), the algorithm continues by evaluating the surrounding combinations as produced by the candidate generator. If any of these combinations has better performance than the current configuration, it will replace it and the process continues. If any of the combinations evaluated satisfy the goal (the given system requirements) that combination is displayed as a potential solution. However, for the purpose of comparing this algorithm with the greedy approaches, once a solution is found the search does not stop but the algorithm continues to look through the search space for more solutions, until several solutions are found.

III. FRAMEWORK APPLICATION DOMAIN To illustrate and asses our framework we have chosen

Network-on-Chip (NOC) architectures as our application domain for the component selection problem. NOC architectures, discussed in detail in [15, 16], have three main functional components: resources, buffers, and switches. Each component is characterized by different attributes. Resources, which are data producers and consumers, are characterized by packet injection rate (that is, the rate at which they produce packets to be delivered through the NOC). Buffers are used for storing data packets and are characterized by size and scheduling criteria. Switches are also characterized by scheduling criteria. Buffer sizes can have different values, such as: 1, 2, 3, 4, etc, and scheduling criteria can have 4 possible values: PB (Priority Based), PBRR (Priority Based Round Robin), RR (Round Robin) and FCFS (First Come First Served). Thus, there are many possible combinations for compiling a NOC architecture and depending on the attributes of the components integrated, the resulting system will exhibit different performance. The main performance measure for NOC architectures is network latency, which is the time taken by a data packet to travel from one resource to another. We assume there are three types of data packets injected into the network: high, medium and low priority packets, and there are different latency requirements for each type of packet. Thus,

our component selection problem becomes selecting the components for a 4x4 mesh-based NOC architecture that needs to satisfy certain latency requirements and system constraints.

As shown in Figure 2, the software prototype that we developed, allows a system architect to input desired system requirements, in terms of acceptable latencies for three types of data packets injected into the network, and system constraints, in terms of injection ratio for resources and restrictions on the buffer size and scheduling criteria. Three assumptions were made: (1) all the 160 buffers of the 4×4 mesh based NOC architecture have the same size, (2) all buffers and switches have the same scheduling algorithm and (3) packet injection ratio is the same for high, mid and low priority packets. Thus, our problem becomes selecting a buffer size and a scheduling criteria for a 4x4 mesh-based NOC architecture such that to satisfy given system latency requirements and constrains.

Figure 2. Integrated Framework Prototype – User Interface

Once a component selection algorithm is chosen, the prototype automatically generates and displays combinations of components that satisfy given requirements, and the estimated performance in terms of latency for each of the combinations found. This allows the architect to make a more informed decision when selecting the final system architecture out of the potential solutions found. For example, Figure 2 shows the solutions found by the intelligent greedy algorithm given the following performance requirements: acceptable latency for high, mid and low priority is 30ns, 40ns and 50ns respectively, and the following system constraints: buffer size must be between 3 and 8, scheduling criteria must be FCFS or PB and injection ratio is 0.5. For each solution found, the number of component combinations evaluated before finding it, is also displayed and will be used to compare the performance of the three implemented component selection algorithms.

IV. RESULTS AND DISCUSSION In all tested scenarios the following system constraints were

used: (1) injection ratio was fixed to 0.5, (2) buffer size was restricted to the range 3 to 8, and (3) all scheduling criteria were acceptable. These system constraints were used by the prototype in the component filtering step, by eliminating those components that do not satisfy them. For each scenario,

different system performance requirements were given. Scenario 1, the most restricted in terms of system performance requirements, involved latency values from category 1 (10ns to 20ns) for high priority packets. Specifically, the latency values for this scenario were: 19ns, 29ns and 39ns for high, mid and low priority packets respectively. For each subsequent scenario, the system performance requirements were relaxed and the latency values for all three priorities were increased by 10, until latency values from category 5 (50ns to 60ns) were reached. At this point little to no change in the performance of the algorithms was observed.

Each scenario was executed using each of the three component selection algorithms. In the cases of both greedy and intelligent greedy, only one execution was required for each scenario, due to the fact that both algorithms are deterministic. Simulated annealing algorithm, being a non-deterministic algorithm was executed 10 times. The average performance over these 10 runs was taken.

For each scenario the first five solutions found by each algorithm were recorded and the number of combination evaluated before reaching each solution was plotted. We refer to these number of combinations evaluated as “steps” in finding a solution. The results obtained for each of the 5 scenarios (with different latency requirements) are presented in Figures 3, 5, 7, 9 and 11 respectively. Then the best five solutions (that is, the solutions with the lowest latencies) were indentified. These best five solutions and their corresponding estimated latencies are presented in Table 2. The number of combinations evaluated by each algorithm until reaching each of these five best solutions is plotted for each scenario in Figures 4, 6, 8, 10 and 12, respectively.

Table 2. Best Five Solutions with Lowest Latency Values

Solution (best to worst)

Buffer Size

Scheduling Criteria

High Latency

Mid Latency

Low Latency

Solution 1 3 PB 17 18 26 Solution 2 4 PB 18 19 37 Solution 3 5 PB 19 20 48 Solution 4 3 RR 21 21 24 Solution 5 4 RR 25 26 32

Figure 3. Quickest Solutions for Latency Category 1

Figure 4. Best Solutions for Latency Category 1

For scenario 1 (latency category 1), only two solutions were found. As illustrated in Figures 3 and 4 the greedy methods found solutions slightly quicker than simulated annealing, especially when finding the best solution: the two greedy algorithms found the best solution in 3 and 4 steps respectively, while the simulated annealing algorithm needed on average about 6 steps. However, all three algorithms found the second solution within a very similar number of steps.

Figure 5. Quickest Solutions for Latency Category 2

Figure 6. Best Solutions for Latency Category 2

For scenario 2 (latency category 2), Figure 5 illustrates that both greedy and simulated annealing can find the first three solutions equally fast, but simulated annealing takes slightly longer to find the 4th and 5th solutions. Figure 6 however,

showed that the greedy methods perform the best when searching for best solutions. This is explained by the inner nature of the greedy approaches to search for the best solutions, while simulated annealing is looking into finding any solution fast. Also, both figures illustrate that intelligent greedy is slightly faster than the other greedy approach.

Figure 7. Quickest Solutions for Latency Category 3

Figure 8. Best Solutions for Latency Category 3

In scenario 3 (latency category 3) all algorithms performed at nearly the same level, when looking at how fast each algorithm found solutions (Figure 7). Figure 8 shows that the greedy approaches find the best solution in the lowest number of steps and simulated annealing exhibits a rather constant performance amongst all five best solutions.

Figure 9 – Quickest Solutions for Latency Category 4

Figure 10. Best Solutions for Latency Category 4

Figure 11. Quickest Solutions for Latency Category 5

Figure 12. Best Solutions for Latency Category 5

In the last two scenarios, a decline in the performance of the intelligent greedy in finding the best solution was noted, going from the second best method to last in scenario 5. Intelligent greedy was able to find some solutions more quickly than others, especially in scenario 5, but constantly missed the best solutions by a considerable margin. The relative performance of the algorithms in terms of finding some solutions became almost identical in the last scenario. This is explained by the fact that, as the system performance requirements are relaxed more combinations of components become solutions and therefore, all algorithms are able to find solutions faster and the differences between the algorithms become unnoticeable.

The above figures illustrate that overall the three component selection algorithms had similar performance in terms of finding solutions fast, but the greedy approaches performed slightly better than simulated annealing in finding the best solutions. Intelligent greedy performed slightly better than the standard greedy implementation when finding the best solution, but fell short when system requirements were relaxed. The good performance of the standard greedy approach comes from the fact that it operated under the assumption that the lower the value of an attribute of a component the better that component is. While this assumption is clearly not true in general, it is a good approximation for our particular application domain, and thus it caused the greedy approach to perform unexpectedly well in our scenarios.

V. CONCLUSION In this paper we proposed an integrated framework for

component selection and applied it to the problem of selecting the components for an NOC architecture. The significance of this framework is that it is driven by system characteristics, such as performance, whose dependencies on the components attributes are unknown. For this purpose, our framework includes a modeling and analysis phase in which such dependencies are analyzed and approximated. To evaluate the framework and its algorithm a prototype was developed that allows a system architect to input desired system requirements through a graphical user interface and then generates and displays combinations of components that satisfy those requirements. The relative performance of the three component selection algorithms was quite similar in terms of finding solutions fast, but the greedy approaches performed slightly better than simulated annealing in finding the best solutions.

In our future work, we intend to expand the component selection problem to include more system requirements, such as, system power consumption, silicon size (number of gates used) and additional system constraints, such as inter-dependencies among components.

REFERENCES [1] Jing Dong, Sheng Yang, Lawrence Chung, Paulo Alencar and Donald

Cowan, “A COTS Architectural Component Specification Stencil for Selection and Reasoning”, 2nd Int’l Workshop on Models and Processes for the Evaluation of Off-the-Shelf Components, 2005, pp. 1-4.

[2] Wes J. Lloyd, “A Common Criteria Based Approach for COTS Component Selection”, 6th GPCE Young Researchers Workshop, 2004, pp. 1-7.

[3] Ed Mancebo, Anneliese Andrews, “A Strategy for Selecting Multiple Components”, ACM Symposium on Applied Computing, 2005, pp. 1505 – 1510.

[4] Jun Guo, BinZhang, Kening Gao,Hongning Zhu, Ying Liu, “A Method of Component Selection within Component Based Software Development Process”, 2004 In’l Conf. on Software Process, 2004, pp. 1-3.

[5] Andreas S. Andreou, Dimitrios G. Vogiatzis, George A. Papadopoulos, “Intelligent Classification and Retrieval of Software Components”, Computer Software and Applications, vol. 2, 2006, pp. 37-40.

[6] Valerie Maxville, Jocelyn Armarego, Chiou Peng Lam, “Intelligent Component Selection”, 28th Annual International Computer Software and Applications Conf., vol. 1, 2004, pp. 244-249.

[7] Paul Baker, Mark Harman, Kathleen Steinh¨ofel, Alexandros Skaliotis, “Search Based Approaches to Component Selection and Prioritization for the Next Release Problem”, 22nd IEEE Int’l Conf. on Software Maintenance, 2006, pp. 176-185.

[8] Olaf Hartig, Martin Kost, Johann-Christoph Freytag, “Automatic Component Selection with Semantic Technologies”, 4th Int’l Workshop on Semantic Web Enabled Software Engineering at ISWC, 2008, pp. 1-14.

[9] Maxym Sjachyn, Ljerka Beus-Dukic, “Semantic Component Selection – SemaCS”, Fifth Int’l Conf. on Commercial-off-the-Shelf (COTS)-Based Software Systems, 2006, pp. 83.

[10] Ankur Agarwal, “System Level Modeling of A Network on Chip”, Int’l Journal of Computer Science and Security”, vol. 3, issue 3, 2009, pp. 154-174.

[11] Ankur Agarwal, Ravi Shankar, Cyril Iskander, Georgiana Hamza-Lup, “System Level Modeling Environment: MLdesigner”, 2nd Annual IEEE Int’l Systems Conference, 2008.

[12] Georgiana Hamza-Lup, Ankur Agarwal, Ravi Shankar, Cyril Iskandar, “Component selection strategies based on system requirements’ dependencies on component attributes”, 2nd Annual IEEE Int’l Systems Conf., 2008, pp. 1-5.

[13] Ankur Agarwal, Varuna Gupta, Georgiana Hamza-Lup, “Statistical Analysis for Embedded System Design”, submitted to 4th Annual IEEE Int’l Systems Conf., 2010 (under review).

[14] Ankur Agarwal, Georgiana L. Hamza-Lup, Ravi Shankar, James Ansley, “An Integrated Methodology for QoS Driven Reusable Component Design and Component Selection”, 1st Annual IEEE Int’l Systems Conf., 2007, pp. 1-7.

[15] L. Benini, D. Bertozzi, “Network on chip architecture and design methods”, IEEE Computation Digital Technology, vol. 152, no. 2, 2005, pp. 261-271.

[16] Ankur Agarwal, A. S. Pandya, Y.U. Lho, ”NOC Architecture Design Methodology”, Int’l Journal KMIC, vol.1, 2005, pp. 57-64.