WDES 2014 paper: Towards the Dynamic Evolution of Context-based Systems-of-Systems

Towards the Evaluation of Systems of Systems Architecture

Daniel S. Santos Brauner R. N. Oliveira

Milena Guessi Flavio Oquendo

Marcio Delamaro Elisa Yumi Nakagawa

Schedule

• Introduction • Systems of Systems

• Architecture Evaluation Methods

• Evaluation of SoS Software Architectures • Evaluation Methods

• Quality attributes

• Challenges

• Conclusion

Towards the Evaluation of Systems of Systems Architecutre CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –

WDES 2014

2

Introduction

Systems of Systems "A set or arrangement of independent, useful systems integrated into

larger systems that deliver unique capabilities " (DoD, 2008).

Challenges: • Constituents systems are often developed and

maintained by different organizations. • These organizations can have their own stakeholders,

development teams, and processes. • Evaluation of SoS software architectures could ensure

that quality attributes are satisfied


WDES 2014

3

Introduction

Architecture Evaluation • Can be used to compare and identify strengths and weaknesses of

different architectural alternatives

• Guide the maintenance or indicate new opportunities for enhancing software architectures

• It is essential for ensuring that software architectures meet desired quality attributes


WDES 2014

4

Introduction

• Goals: To present the state of the art on SoS software architecture evaluation as well as new research perspectives.

• Based on results of a Systematic Literature Review (SLR)1. • Overall, 16 primary studies were included in this SLR.

1 Available at http://goo.gl/PU12iQ (last accessed on 07/13/2014)


WDES 2014

5

Primary studies selected


WDES 2014

6

Evaluation of SoS Software Architecture

Evaluation is performed in: • Design phase

• Comparing and identifying strengths and weaknesses of different architectural alternatives.

• Maintenance phase • Intending to analyze the flexibility and ability to evolution


WDES 2014

7

Evaluation Methods

• No convergence in using a specific type of evaluation method.

• Architecture Trade-off Analysis Method (ATAM) is the most popular.


WDES 2014

8

Frequency of use of methods and techniques

Maturity of Evaluation Methods


WDES 2014

9

31%

13% 31%

19%

6% Demonstration

Expert Opinion

Case Study

Application inIndustry

No Assessment

Maturity of the approaches

Quality Attributes

Evaluation methods can either focus on single or several quality attributes.

• Several Quality Attributes • Scenario-based methods

• No focus on specific quality attributes

• Few tangible quality attributes • simulation-based and mathematical modeling methods

• Reliability, performance, complexity, and flexibility.

• None of the studies discuss the use of quality models during architectural evaluation.

Towards the Evaluation of Systems of Systems Architecutre

CBSoft 2014 – Workshop em Desenvolvimento Distribuído de Software, Ecossistemas de Software e Sistemas de Sistemas –

WDES 2014

10

Quality Attributes

Recommendations • Evaluation methods for SoS should take into account several quality attributes.

• Evaluation methods should be able of measuring and classifying quality attributes.

• Quality models would be relevant for evaluating SoS architectures • Provide standardization for quality attributes

• Guide the identification and prioritization of quality attributes


WDES 2014

11

Conclusion

• There is still no consensus on what exactly should be considered during evaluation.

• There is still several difficulties and challenges

• Appropriate and scalable evaluation approaches still need to be developed.


WDES 2014

12

Towards the Evaluation of Systems of Systems Architecture

Daniel S. Santos Brauner R. N. Oliveira

Milena Guessi Flavio Oquendo

Marcio Delamaro Elisa Yumi Nakagawa

Education

WDES 2014 paper: Towards the Dynamic Evolution of Context-based Systems-of-Systems