Ahab's Leg Dilemma

  • Published on
    22-Apr-2015

  • View
    844

  • Download
    0

Embed Size (px)

DESCRIPTION

Luca SabatucciAhab's Leg Dilemma - on the Design of a Controlled ExperimentEmpire 2011, Trento, Italy

Transcript

<ul><li> 1. Ahabs Leg Dilemma: on the Design of aControlled Experiment Luca Sabatucci Mariano Ceccato Alessandro Marchetto Angelo Susi </li> <li> 2. The Ahabs Leg dilemma When changing media (or communication style) we need to add details to a story, to keep the story engaging Umberto Eco 1956 Ray Bradbury &amp; John Huston. </li> <li> 3. The Ahabs Leg dilemma The peg leg is fundamental for the story Deciding which leg is a peg one has no bearing on it When the peg-leg is instantiated, this decision may generate of lot of consequences 1930 Warner Bros. 1956 Ray Bradbury &amp; John Huston.Directed by Lloyd Bacon. </li> <li> 4. Ahabs Leg in RE Often, narrative scenarios are used to validate requirements with stakeholders in focus groups Narrative scenarios are derived from requirements (change of communication style) Details must be added during translation to instantiate generic requirements into a concrete spatial-temporal context Stakeholders might be distracted by irrelevant details </li> <li> 5. An example from our experience Requirement: the system communicates with caregivers with low and high priority signals The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregivers PDA Maria falls on the staircase The focus group was proceeding well until a nurse commented on the PDA (Ahabs Leg) PDA is intrusive (to carry around, battery) and it would change working practices Lively discussion on less intrusive devices This was pointless, because the kind of device was not yet decided Credits: picture (c) By Chiara Leonardi </li> <li> 6. Mandatory Vs Optional Some are mandatory to make the story concrete and believable (e.g., PDA) Concreteness is important for stakeholders to envisage functionalities Some are optional and choreographic, needed just to increase the stakeholders engagement (e.g., name of the patient) Does mandatory and optional ALs affect in the same way requirement validation sessions? The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregivers PDA Maria falls on the staircase Credits: picture (c) By Chiara Leonardi </li> <li> 7. Cardinality Optional ALs can be removed Many: scenarios very concrete but with the risk of high distraction Few: scenarios very abstract, difficult to present to stakeholders Influence of personal and contextual background: Melville did not specify if all the member of the crew had two legs. But the reader assumes it based on his/her common knowledge of the real world Abstract scenarios make stakeholders mentally complete missing details The initial scenario is corrupted No possibility to control ALs by the analyst The camera PDA displays identifies Fall on that an the event staircase unknown person and sends a is fallen in the signal to staircase caregivers PDA Maria falls on the staircase </li> <li> 8. Stakeholder awareness Not realistic to compare scenario with and without ALs (the latter does not make sense) If the discussion is moderated by a facilitator, he/she could highlight irrelevant details to avoid spending time in discussing about them Risk of attracting even more attention on them Just mention that there are more and less important details (with some example) Exploiting stakeholder awareness is probably the more realistic approach and analyst would take, to limit distraction. </li> <li> 9. The plan The role of Ahabs Leg has been observer during a real project validation session We conjecture that this is not due to the specific project, but the problem is more general Test this conjecture in a controlled and repeatable in-lab experiment We control/measure all the relevant variables We change just one variable and we study the effect </li> <li> 10. Research questions Ahabs Legs are often unavoidable Not a problem, unless they divert the attention from important aspects of the story.RQ1: What is the actual impact of Ahabs Legs on thedistraction of stakeholders during a requirementvalidation sessions?RQ2: Is there a reliable way to reduce their impact onthe distraction? </li> <li> 11. Context of the experiment Subjects: Computer science master students. Background on software and requirement engineering Some actual development experience Objects: 2 software system found on the internet MyBanking: home banking application for mobile devices, designed to replace credit cards and cash. MyShopping: augmented reality application for mobile devices that display information on items pointed by the camera. </li> <li> 12. Hypothesis formulation H0: explicitly mentioning what are the over- specified details (Ahabs Legs) in application scenarios does not significantly reduce the distraction in a requirement validation session. HA: explicitly mentioning what are the over- specified details (Ahabs Legs) in application scenarios significantly reduces the distraction in a requirement validation session. </li> <li> 13. Variable selection Independent variable: explicitly telling that there are details irrelevant for the discussion (Ahabs Legs) or without such explicit mention. Dependent variable: distraction observed during the requirement validation phase. The stakeholder comment addresses a topic that is not part of the requirement (e.g., Ahabs Leg) distractioni =1 The comment requires to fix/reconsider a requirement distractioni =0 Disrtaction = distractioni </li> <li> 14. Co-factors that we measure Learning effect between the two labs System for which requirements are validated Subjects academic merit as the average of exam score Subjects background measured as they attended relevant courses Previous subjects experience On requirement validation On industrial software development </li> <li> 15. Experimental design Group1 Group2 Group3 Group4Lab1 MyBanking AL MyBanking No MyShopping AL MyShopping NoLab2 MyShopping No MyShopping AL MyBanking No MyBanking AL Fill the profiling pre-questionnaire Lab 1 Read the description of the first application For each of the 4 scenarios Read a scenario Write a comment/question for the scenario Lab 2 Read the description of the first application For each of the 4 scenarios Read a scenario Write a comment/question for the scenario Fill the feedback questionnaire. </li> <li> 16. Missing aspects? Other strategies to control the influence of Ahabs Leg dilemma? Trade-off between distraction and level of abstraction? What is the influence of subject background? </li> <li> 17. Conclusions </li> <li> 18. Questions? </li> </ul>