A Context-sensitive User Feedback Mechanism

for Continuous Software EvolutionDora Dzvonyar (presented by Lukas Alperowitz)

Problems with User Feedback

2

communication gap too many channels

missing structure & information no clear process

[e.g. ANA14, Dam96, HL09, Kuj03, Pag13, Sch11]

Context-Aware FEedback System

Framework and Process combining approaches from 3 domains:

• Integration into development workflow

• Context-awareness

• Feedback traceability

3

Integration of Feedback into Development

• Feedback should be considered regularly [Dam96, HL09, Sch11]

• But: existing process models lack concrete integration into dev tasks

• CAFE stores user feedback in the issue tracker • Used daily by developers (acceptance)

• Less hassle to look at feedback

• Easier transformation into requirements

4

Type of feedback?

Underlying requirement?

Importance of change?

K. Schneider. Focusing spontaneous feedback to support system evolution. In RE ’11, pages 165–174. IEEE, 2011

Context-Awareness

• Compensate for the user’s lack of awareness of what is important to the development team [DSC11, Pag13]

• CAFE stores context data, e.g. • Usage environment information

• User’s location in the application

• User’s last interactions with the application

• Screenshot of problem

5

• Feedback: source or consequence of requirement

• Evolution of requirements through feedback in the same system

Message usage context

feedback type

Feedback Traceability

6

Requirement 1.1

Requirement 2Feedback 2

Feedback 1

Requirement 1

“Users don’t understand this

feature”“We introduced

this feature because users wanted it”

Overview over CAFE

7

Extract Requirement

Including e.g.physical location,current screen,

interaction steps

Where does the user see this change?

User can enrich feedback with context-based

suggestions: “Does your feedback concern…?”

Evaluation

• User experiments • Users prefer CAFE to other means of providing feedback

• Users are willing to provide additional details if they know about the feature

• Developer interviews • Usage context tracking is adequate and useful (granularity, form)

• Saving feedback in the issue tracker is convenient

• Expect better communication with the user base and increased user satisfaction

8

Conclusion & Future Work

• Demonstrated feasibility of CAFE

• Results indicate that Feedback Traceability is a valuable contribution

• Combine CAFE with AppStore—> marketing vs. product improvement

• Long-term evaluation • Do developers consider feedback more regularly?

• More efficient feedback analysis?

• Value of feedback traceability9

Literature #1

[ANA14] Malik Almaliki, Cornelius Ncube, and Raian Ali. The design of adaptive acquisition of users feedback: An empirical study. In 2014 IEEE Research Challenges in Information Science, pages 1–12. IEEE, 2014.

[ASO+ 12] Raian Ali, Carlos Solis, Inah Omoronyia, Mazeiar Salehie, and Bashar Nuseibeh. Social adaptation: when software gives users a voice. 2012.

[BD09] Bernd Bruegge and Allen H Dutoit. Object Oriented Software Engineering Using UML, Patterns, and Java. Prentice Hall, 2009.

[BKW12] Bernd Bruegge, Stephan Krusche, and Martin Wagner. Teach- ing Tornado. In Proceedings of the 8th edition of the Educators’ Symposium, ACM/IEEE 15th International Conference on Model Driven Engineering Languages and Systems, pages 5–12, 2012.

[Boe00] Barry Boehm. Requirements that handle IKIWISI, COTS, and rapid change. Computer, 33:99–102, 2000.

[BZ13] Muneera Bano and Didar Zowghi. User involvement in software development and system success: a systematic literature review. In Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering, pages 125–130. ACM, 2013.

[CHS+ 07] Sunny Consolvo, Beverly Harrison, Ian Smith, Mike Y. Chen, Katherine Everitt, Jon Froehlich, and James A. Landay. Con- ducting In Situ Evaluations for and With Ubiquitous Computing Technologies, 2007.

[Dam96] Leela Damodaran. User involvement in the systems design pro- cess - a practical guide for users. Behaviour and Information Technology, 15:363–377, 1996.

[DSC11] Marco De Sa and Luıs Carric ̧o. Designing and evaluating mobile interaction: challenges and trends. Foundations and Trends in Human-Computer Interaction, 4(3):175–243, 2011.

[Fie00] Roy Thomas Fielding. Architectural styles and the design of network-based software architectures. PhD thesis, University of California, Irvine, 2000.

[FKL13] John Fisher, D Koning, and AP Ludwigsen. Utilizing atlassian jira for large-scale software development management. In 14th International Conference on Accelerator & Large Experimental Physics Control Systems (ICALEPCS), 2013.

[GF94] Orlena C Z Gotel and Anthony C W Finkelstein. An analysis of the requirements traceability problem. In Requirements Engineering, 1994., Proceedings of the First International Conference on, pages 94–101. IEEE, 1994.

[Gol03] Nahid Golafshani. Understanding reliability and validity in qualitative research. The qualitative report, 8(4):597–606, 2003.

[HL09] Juho Heiskari and Laura Lehtola. Investigating the state of user involvement in practice. Proceedings - Asia-Pacific Software Engineering Conference, APSEC, pages 433–440, 2009.

[Hol05] Karen Holtzblatt. Designing for the mobile device: Experiences, challenges, and methods. Communications of the ACM, 48(7):32–35, 2005.

10

Literature #2

[KABW14] Stephan Krusche, Lukas Alperowitz, Bernd Bruegge, and Martin O Wagner. Rugby: an agile process model based on continuous delivery. In Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering, pages 42–50. ACM, 2014.

[KB14] Stephan Krusche and Bernd Bruegge. User feedback in mobile development. In Proceedings of the 2nd International Workshop on Mobile Development Lifecycle, pages 25–26. ACM, 2014.

[KDoD+03] Lena Karlsson, A Dahlstedt, J Natt och Dag, Björn Regnell, and Anne Persson. Challenges in market-driven requirements engineering-an industrial interview study. In Proceedings of the Eighth International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’02), pages 101–112, 2003.

[KKLK05] S. Kujala, M. Kauppinen, L. Lehtola, and T. Kojo. The role of user involvement in requirements quality and project success. 13th IEEE International Conference on Requirements Engineer- ing (RE’05), 2005.

[Kuj03] Sari Kujala. User involvement: a review of the benefits and challenges. Behaviour & information technology, 22(1):1–16, 2003.

[MBM+08] Janice M Morse, Michael Barrett, Maria Mayan, Karin Olson, and Jude Spiers. Verification strategies for establishing reliability and validity in qualitative research. International journal of qualitative methods, 1(2):13–22, 2008.

[Pag13] Dennis Pagano. Portneuf - A Framework for Continuous User Involvement. PhD thesis, Technische Universität München, 2013.

[RC08] Jeffrey Rubin and Dana Chisnell. Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons, 2008.

[Sch09] Christian Schüller. Automatisierte Feedbackgenerierung in der benutzerzentrierten Softwareentwicklung am Beispiel mobiler Anwendungen. PhD thesis, Technische Universität München, 2009.

[Sch11] Kurt Schneider. Focusing spontaneous feedback to support system evolution. In 19th IEEE International RE Conference, pages 165–174. IEEE, 2011.

[SF91] Julio Cesar Sampaio do Prado Leite and Peter A. Freeman. Requirements validation through viewpoint resolution. IEEE Transactions on Software Engineering, 17(12):1253–1269, 1991.

[SJH+ 14] Nada Sherief, Nan Jiang, Mahmood Hosseini, Keith Phalp, and Raian Ali. Crowdsourcing software evaluation. In Proceedings of the 18th International Conference on Evaluation and Assess- ment in Software Engineering - EASE ’14, pages 1–4, 2014.

[SOB11] Norbert Seyff, Gregor Ollmann, and Manfred Bortenschlager. iRequire: Gathering end-user requirements for new apps. In Proceedings of the 2011 IEEE 19th International Requirements Engineering Conference, RE 2011, pages 347–348, 2011.

11

Backup

12

Overview over CAFE

13

Extract Requirement

store usage context

“Does your feedback

concern…?”

Where does the user see this change?

Overview over CAFE

14

Integration into Dev.Extract

Requirement

Overview over CAFE

15

Context-Awareness

“Does your feedback

concern…?”

Extract Requirement

Where does the user see this change?

store usage context

Overview over CAFE

16

FB Traceability

“Does your feedback

concern…?”

Evaluation

17

• User experiments

Evaluation

18

• Semi-structured developer interviews