7/30/2019 Cog Metri

1/41

Cognitive Metrics for the

Management of Model-based

Software Development

Reuven Gallant, Yehuda Badihi,

Hagai SugarmanJerusalem College of Technology-Machon Lev

JCT

7/30/2019 Cog Metri

2/41

Why are managers afraid of model-based

development?It sounds good

notation

method/process tool

7/30/2019 Cog Metri

3/41

Why are managers afraid of model-based

development?

7/30/2019 Cog Metri

4/41

actor

They like pictures

Why sw engineers do/dont like

models (1)

They dont like tools

Or are lacking motivation to use them

They dont have patience for semantics

dNST: (US) (US) :(s, s') ss' + s's

They like pictures They like pictures

7/30/2019 Cog Metri

5/41

Is a picture worth 1000 words?

To set the alarm , the key must be insertedinto the lock , turned anti clockwise , and

held for at least half a second.

To reset the alarm , the key must be inserted

into the lock and turned clockwise. Thealarm will be deactivated immediately.

7/30/2019 Cog Metri

6/41

Automobile Alarm

7/30/2019 Cog Metri

7/41

Why sw engineers do/dont like models (2)

7/30/2019 Cog Metri

8/41

Key Characteristics of

Engineering Models Abstraction

Understandability

Accuracy

Predictiveness

InexpensiveBrian Selic, The Pragmatics of Model-Driven Development,IEEESoftware, Sept./Oct. 2003.

7/30/2019 Cog Metri

9/41

Where measurements can help us

manage model-based development Tool Evaluation

People Evaluation

Training Evaluation

Model Evaluation

7/30/2019 Cog Metri

10/41

Metrics must be simple and easy

to automateFenton N. and Neil M. (2000). Software

Metrics: a Roadmap. Future of Software

Engineering. Anthony Finkelstein Ed., ACM,

359-370.

7/30/2019 Cog Metri

11/41

Bottom Up Tool Evaluation

7/30/2019 Cog Metri

12/41

Gestalt Approach to Tool Evaluation

R. Gallant, M. Winokur, and J. Kudish,

"Tool and Method Reciprocity: A Case Study in Requirements Management,"

Proceedings of The Seventh I srael Conference on Computer Systems and Software

Engineer in g1996.

7/30/2019 Cog Metri

13/41

attempt toproduce the mostaesthetic, andthereforecomprehensible,

layouts.

Harels Aesthetics

An Algorithm for Blob Hierarchy Layout", Proc. Working Conf. onAdvanced Visual Interfaces (AVI2000), Palermo, Italy, ACM Press,

New York, May 1999, pp. 29-40.

7/30/2019 Cog Metri

14/41

Automobile Alarm-unaesthetic?

7/30/2019 Cog Metri

15/41

Statechart Complexity Metrics (1)

The aim of the experiment was to investigate the

relationship between the complexity of UML

statechart diagrams and their understandability(one maintainability subcharacteristic).

Defining and Validating Metrics for UML Statechart Diagrams

Marcela Genero, David Miranda and Mario Piattini, QAOOSE2002

7/30/2019 Cog Metri

16/41

Statechart Complexity Metrics (2)

7/30/2019 Cog Metri

17/41

Automobile Alarm-too many of

something?

S h C l i M i

7/30/2019 Cog Metri

18/41

Statechart Complexity Metrics:

economy vs. explicity

7/30/2019 Cog Metri

19/41

How Do We Measure Understandability? (2)Cognitive Dimensions (CDs)

Viscosity Visibility

Premature

commitment Hidden dependencies

Role expressiveness

Error-proneness

Abstraction

Secondary notation Closeness of mapping

Consistency

Diffuseness Hard mental

operations

Provisionality

Progressive evaluation

Proposed by Greene in 1989, who has developed it since

then with Petre, Blackwell, et al.

7/30/2019 Cog Metri

20/41

Automobile Alarm-CD deficiency?

7/30/2019 Cog Metri

21/41

Patterns to the rescue!

resetting

setOrNeutral

setOrSetting

set

setting neutral

[settingCondition]

[neutralCondition]

tm(setTime)[IS_IN(setting)]/setAction

[resettingCondition]/resetAction

[neutralCondition]/neutralAction

/setup()

7/30/2019 Cog Metri

22/41

Cognitive Measures

Automatization

Automatized vs. controlled tasks Types of knowledge

Declarative, Conceptual,Procedural

Expertise

From novice to expert

Learning hierarchy

Associations and chains; Discriminations Concepts; Rules ;Higher order rules

Knowledge structur ing

Surface features task specific features abstract/hierarchicalfeatures

Mental workload

single resource pool, multiple-resource

7/30/2019 Cog Metri

23/41

Automatization(1)

Please enter OMTracer Command>> go idle

OMTracer (0:00:00.200) Display[0] Sent to itself Event tm(200) at ROOT.active

OMTracer (0:00:00.200) Display[0] Received from itself Event tm(200) at ROOT.active

OMTracer (0:00:00.200) main() Invoked Display[0]->Take Event Timeout

OMTracer (0:00:00.200) Display[0] Invoked isDone()

OMTracer (0:00:00.200) Display[0]->isDone() Returned

OMTracer (0:00:00.200) Display[0] Exited State ROOT.active

OMTracer (0:00:00.200) Display[0] Invoked print(n = 1)

OMTracer (0:00:00.200) Display[0]->print(n = 1) Returned

OMTracer (0:00:00.200) Display[0] Entered State ROOT.activeOMTracer (0:00:00.200) Display[0] set tm(200) at ROOT.active

OMTracer (0:00:00.200) Display[0]->Take Event Timeout Returned

A i i (2)

7/30/2019 Cog Metri

24/41

Automatization(2)topState

A t ti ti (3)

7/30/2019 Cog Metri

25/41

Automatization(3)

off filling rinsing

evStart/

itsTank->GEN(evTankFill)

evFull

Statechart of Dishwasher

Statechart of Tank

7/30/2019 Cog Metri

26/41

Any questions?

7/30/2019 Cog Metri

27/41

actor

7/30/2019 Cog Metri

28/41

Why engineers wont use tools

7/30/2019 Cog Metri

29/41

Enter Professor Harel

7/30/2019 Cog Metri

30/41

More than one way to skin a cat!(1)

Please enter OMTracer Command>> go idle

OMTracer (0:00:00.200) Display[0] Sent to itself Event tm(200) at ROOT.active

OMTracer (0:00:00.200) Display[0] Received from itself Event tm(200) at ROOT.active

OMTracer (0:00:00.200) main() Invoked Display[0]->Take Event Timeout

OMTracer (0:00:00.200) Display[0] Invoked isDone()

OMTracer (0:00:00.200) Display[0]->isDone() Returned

OMTracer (0:00:00.200) Display[0] Exited State ROOT.active

OMTracer (0:00:00.200) Display[0] Invoked print(n = 1)

OMTracer (0:00:00.200) Display[0]->print(n = 1) Returned

OMTracer (0:00:00.200) Display[0] Entered State ROOT.activeOMTracer (0:00:00.200) Display[0] set tm(200) at ROOT.active

OMTracer (0:00:00.200) Display[0]->Take Event Timeout Returned

7/30/2019 Cog Metri

31/41

Automatization

Shiffrin et al. [9] [10] distinguish betweencontrolled and automatic processes. Whereasautomatized processes require little or nocognitive resources, controlled processes areresource intensive. Abilities are important formastering repetitive tasks, but not for performingthem once mastered. In contrast, abilities continueto be important in the performance of controlled

processes.

7/30/2019 Cog Metri

32/41

Level of Detail

Diagram Type

Application Type

7/30/2019 Cog Metri

33/41

How did the SHADOW get its

name????? 2 Theories

7/30/2019 Cog Metri

34/41

Theory I

Sikorsky Helicopter

AdvancedDemonstrator of

Operator Workload

7/30/2019 Cog Metri

35/41

Theory II

Jimmy Durantes radio program and nose!

7/30/2019 Cog Metri

36/41

The SHADOW was top top

secret!!!! Not members of the U.S. House of

Representatives

Not even SenatorsGot to see the SHADOW

7/30/2019 Cog Metri

37/41

The SHADOW was top top

secret!!!!

But the King

got to fly in it!

7/30/2019 Cog Metri

38/41

4 Helicopters were sent to fetch

the King from Boston

7/30/2019 Cog Metri

39/41

It was raining, and the King decided

that discretion was the better part of

valour

7/30/2019 Cog Metri

40/41

How Do We Measure Understandability? (2)

Cognitive Dimensions (CDs) Viscosity: resistance to change

Visibility

Greene, T. R. G. (1989). Cognitive Dimensions of Notations. InPeople andComputers V. A. Sutcliffe and L. Macaulay, (Ed.). Cambridge: CambridgeUniversity Press, 443-460.

For non-specialists (no expertise in cognitive science or HFE required)

Broad-brush

Quick to apply, no lengthy analysis

Check list approach: abstraction, hidden dependencies, premature commitment, secondary notation,

viscosity, visibility, closeness of mapping, diffuseness, error-proneness, hardmental operations, progressive evaluation, provisionality, role-expressiveness

7/30/2019 Cog Metri

41/41

Bottom Up Tool Evaluation