Ahab's Leg Dilemma: on the Design of a

Controlled Experiment

Luca SabatucciMariano Ceccato

Alessandro MarchettoAngelo Susi

1956 Ray Bradbury & John Huston.

The Ahab’s Leg dilemma

• When changing media (or communication style) we need to add details to a story, to keep the story engaging

Umberto Eco

1930 Warner Bros. Directed by Lloyd Bacon.

The Ahab’s Leg dilemma

• The peg leg is fundamental for the story– Deciding which leg is a peg one has no bearing on it– When the peg-leg is instantiated, this decision may

generate of lot of consequences

1956 Ray Bradbury & John Huston.

Ahab’s Leg in RE• Often, narrative scenarios are used to validate requirements with

stakeholders in focus groups• Narrative scenarios are derived from requirements (change of

communication style)• Details must be added during translation to instantiate generic

requirements into a concrete spatial-temporal context

• Stakeholders might be distracted by irrelevant details

An example from our experience

• The focus group was proceeding well until a nurse commented on the PDA (Ahab’s Leg)– PDA is intrusive (to carry around, battery…) and it would change working

practices– Lively discussion on less intrusive devices – This was pointless, because the kind of device was not yet decided

The camera identifies the event and sends a signal to caregiver’s PDA

Maria falls on the staircase

PDA displays that an unknown person is fallen in the staircase

Fall on staircase

Requirement:the system communicates with caregivers with low and high priority signals

Credits: picture (c) By Chiara Leonardi

Mandatory Vs Optional

• Some are mandatory to make the story concrete and believable (e.g., PDA)– Concreteness is important for stakeholders to envisage functionalities

• Some are optional and choreographic, needed just to increase the stakeholders engagement (e.g., name of the patient)

• Does mandatory and optional ALs affect in the same way requirement validation sessions?

The camera identifies the event and sends a signal to caregiver’s PDA

Maria falls on the staircase

PDA displays that an unknown person is fallen in the staircase

Fall on staircase

Credits: picture (c) By Chiara Leonardi

Cardinality• Optional ALs can be removed

– Many: scenarios very concrete but with the risk of high distraction– Few: scenarios very abstract, difficult to present to stakeholders

• Influence of personal and contextual background:– Melville did not specify if all the member of the crew had two legs. But the

reader assumes it based on his/her common knowledge of the real world– Abstract scenarios make stakeholders mentally complete missing details

• The initial scenario is corrupted• No possibility to control ALs by the analyst

The camera identifies the event and sends a signal to caregiver’s PDA

Maria falls on the staircase

PDA displays that an unknown person is fallen in the staircase

Fall on staircase

Stakeholder awareness• Not realistic to compare scenario with and without ALs

(the latter does not make sense)• If the discussion is moderated by a facilitator, he/she

could highlight irrelevant details to avoid spending time in discussing about them– Risk of attracting even more attention on them– Just mention that there are more and less important details

(with some example)• Exploiting stakeholder awareness is probably the more

realistic approach and analyst would take, to limit distraction.

The plan• The role of Ahab’s Leg has been observer during a real project

validation session• We conjecture that this is not due to the specific project, but the

problem is more general• Test this conjecture in a controlled and repeatable in-lab

experiment– We control/measure all the relevant variables– We change just one variable and we study the effect

Research questions

• Ahab’s Legs are often unavoidable • Not a problem, unless they divert the

attention from important aspects of the story.

RQ1: What is the actual impact of Ahab’s Legs on the distraction of stakeholders during a requirement validation sessions?

RQ2: Is there a reliable way to reduce their impact on the distraction?

Context of the experiment• Subjects: Computer science

master students. – Background on software and

requirement engineering– Some actual development experience

• Objects: 2 software system found on the internet– MyBanking: home banking application

for mobile devices, designed to replace credit cards and cash.

– MyShopping: augmented reality application for mobile devices that display information on items pointed by the camera.

Hypothesis formulation

• H0: explicitly mentioning what are the over-specified details (Ahab’s Legs) in application scenarios does not significantly reduce the distraction in a requirement validation session.

• HA: explicitly mentioning what are the over-specified details (Ahab’s Legs) in application scenarios significantly reduces the distraction in a requirement validation session.

Variable selection

• Independent variable: explicitly telling that there are details irrelevant for the discussion (Ahab’s Legs) or without such explicit mention.

• Dependent variable: distraction observed during the requirement validation phase.– The stakeholder comment addresses a topic that is not part of

the requirement (e.g., Ahab’s Leg) distractioni =1– The comment requires to fix/reconsider a requirement

distractioni =0

– Disrtaction = S distractioni

Co-factors that we measure

• Learning effect between the two labs• System for which requirements are validated• Subjects’ academic merit as the average of exam

score• Subjects’ background measured as they attended

relevant courses• Previous subjects’ experience

– On requirement validation– On industrial software development

Experimental designGroup1 Group2 Group3 Group4

Lab1 MyBanking AL MyBanking No MyShopping AL MyShopping No

Lab2 MyShopping No MyShopping AL MyBanking No MyBanking AL

• Fill the profiling pre-questionnaire• Lab 1

– Read the description of the first application– For each of the 4 scenarios

• Read a scenario• Write a comment/question for the scenario

• Lab 2– Read the description of the first application– For each of the 4 scenarios

• Read a scenario• Write a comment/question for the scenario

• Fill the feedback questionnaire.

Missing aspects?

• Other strategies to control the influence of Ahab’s Leg dilemma?

• Trade-off between distraction and level of abstraction?

• What is the influence of subject background?

Conclusions

Questions?