Embed Size (px)
Citation preview
How do Software Architects consider Non-Functional
RequirementsAn Exploratory Study
D. Ameller, C. Ayala, J. Cabot, X. Franch
2
Outline
• Motivation and objective
• Background
• The study
• Observations
• Conclusions and related work
3
MotivationNFRs SA
“the rationale behind each architecture decision is mostly about achieving certain NFRs”
“[NFRs] play a critical role during system development, serving as selection criteria for choosing among myriads of alternative designs”
“quality attribute requirements strongly influence a system’s architecture”
Little evidence from empirical studies support these statements
4
Objective
To conduct an exploratory study to investigate the following research question:
How do software architects deal with NFRs in practice?
5
Background
RE activities
Reqs. structure
NFR types
Arch. design Analysis Companies
Prj. leades
Prd. managers
Programmers
Users
Sw. architects
Interviews 5Interviews 11Interviews 2e-Survey 25e-Survey ?Questionnaire 15Questionnaire ?Group discus. 10Interviews 12
+ SLR in [Svensson et al. 2010]
6
BackgroundObservations:• No clear view in NFR elicitation• Cost-driven NFR quantification• Role-dependent view on NFR type’s importance• Lack of knowledge about NFR management• Vagueness of NFRs• Difficulty to test NFRs
• FRs still prevale• Need for more empirical studies
7
The Study• Type:
Exploratory study using qualitative research Seven RQs
• Target population: Software architects (or practitioners with that role) We invited 21 software organizations (Spain) We recruited 12 of them
13 interviews made
8
The Study
SCC: Software Consulting Company; SH: Software House; ITD: IT Department
9
The Study• Data collection:
Semi-structured interviews Focus: one single project Duration: ~1 hour
• 7 questions about architect profile and development methodology used in the project
• 10 questions about decision-making and NFRs Audio-taped Transcribed Checked
10
The Study• Analysis:
Two authors coded data independentlytabulation
Categories generationNVivo Software
ConsolidationGroup meetings
PresentationCounting technique
11
The Study• Limitations and mitigations:
Evaluation apprehension Confidentiality Understandability Piloting (4) Influential factors Single project Bad memory Questionnaire sent in advance The best project Ask for the most familiar Omitting data Results overviewed by respondants Researcher bias Two separated interpretations Generalize the findings We do not attempt to make
universal generalizations
12
Observations• RQ1: What is the role of the software
architect? Nobody had the “software architect” position Nomination: (experience, tec. knowledge) > architect skills 12 played other roles played in the project:
3 Project
Manager
Developer 5 Both
4
13
Observations• RQ2: Are there terminological confusions on
NFRs? Inability. “availability”, “accuracy”, “sustainability” Confusion. “ergonomic” instead of “easy to use” Incorrectness. “Maintainability is very important, because
when something is working, we can’t make changes” Worsened by the Spanish translation
14
Observations• RQ3: What types of NFRs are relevant to
software architects?
Perfo
rman
ce
Usabil
ity
Secur
ity
Availa
bility
Inte
rope
rabil
ity
Main
tena
nce
Accur
acy
Fault
Tolera
nce
Reusa
bility
Scalab
ility
Mod
ularit
y
Mon
itorin
g
Porta
bility
0
1
2
3
4
5
6
7
8
9
10
49 Software qualities
DomainTacit
15
Observations• RQ3: What types of NFRs are relevant to
software architects?
Licen
sing
issue
s
Techn
ologic
al po
lices
Client
's NT re
quire
men
ts
Costs
Exter
nal r
egula
tions
Availa
bility
of s
uppo
rt
Organ
izatio
nal p
olicie
s0
1
2
3
4
5
6
7
8
9
10
33 Non-technical reqts.
16
Observations• RQ4: How are NFRs elicited?
All architects considered elicitation as a gradual process… also never-ending
10
3 Mainly architect
User and architect
OutsourcedProjects
17
Observations• RQ5: How are NFRs documented?
9
No documentation
3 Some kind of strategy
1 Plain text
“I rarely document my projects because it costs money”
VolereISO/IEC 9126-basedAd-hoc
Documentation not always evolved!
18
Observations• RQ6: How are NFRs validated?
NFRs had been met by the end of the project?
11
2 Yes Not “yes”
Very vague justifications
“We wait for the client to complain”
19
Observations• RQ6: How are NFRs validated?
What NFRs were validated?
8
Efficiency Accuracy Usability Reliability 4 Unknown
1 None
20
Observations• RQ7: What type of tool support for NFRs is
used? Architects did not use any specific tool support for
NFR management
21
Observations• RQ7: What type of tool support for NFRs is
used? What about NFR-driven MDD (cf. [Ameller RE’10])
5 Not at all
4 Not viable
2 Just support
2 Not answer
“I would not trust”
22
Conclusions• Contributions
Corroborating previous evidences
Architects…performed other activities simultaneouslydid not share a common vocabularyshowed some misunderstandings and lack of knowledgeconsidered performance and usability as most importantmainly elicited NFRs iterativelymore often than not, didn’t documentvalidated just a few typeswere not enthusiastic about advanced tool support
23
Conclusions• Contributions
Finding previously unreported observationsArchitects…
considered NTRs almost as important as quality reqts.mainly elicited NFRs by themselvesclaimed that NFRs were satisfied at the end of projectdid not use any specific tool for NFR management
24
Conclusions• Contributions
Finding some misalignments or even contradictions with previous studies
Architects…did not exist as a differentiated rolequantification of NFRs was poor
25
Conclusions• Future work
Replication; consolidation with other studies• E.g., study on OSS domain [REFSQ’12]
Changing the conditions• E.g., influence of an starting architecture (Ferrari et al, REJ10)
• More research needed on: Cost-benefit analysis (e.g., documentation) Highly customizable NFR management (e.g., existence
of architect role) Exploring other communication channels (e.g., blogs)
