Software development effort reduction with Co-op

Software development effort reduction with Co-opDr.ir. Lodewijk Bergmans [lbergmans@acm.org]

About me

software composition Objects Aspects ...

Industry-as-a-laboratory ASML, Oce, Siemens, ..

European Network of Excellence on AOSD

Assistant Prof., University of Twente

Independent consultant (1994-1997, 2008-) teach & mentor SE software architecture

e.g. Philips Medical Systems, Ernst & Young MC, Ordina Panfox

Ericsson Mobile Communications (Lund)

Independent consultant,steX

Research Industry

Message of this talk

Development and evolution cost are high: for frameworks, applications & customizations

Cost can be reduced by: reducing accidental complexity improved standardization easier reuse of design solutions (idioms & patterns)

Co-op is an enabling technology

Contents

Challenges in software development Understanding the problems A solution approach Co-op Application of the Co-op solution technology Practical applicability: previous experiences Technology benefits

Challenges in software development

Some major issues in SE

1. managing complexity we build increasingly (inherently) complex systems which carry more and more accidental complexity

2. continuous change change starts on day one.. causes large maintenance costs (60%-90%)

6

Experience: complexity vs project risk

source: Wallace, Keil & Rai, “Understanding software project risk: a cluster analysis”, Information & Management 42 (2004)

7

Improvement potential

source: Walker Royce, “Improving Software Economics”: IBM white paper (2009)

cost

s (p

er p

erso

n ye

ar)

cost

s (p

er p

erso

n ye

ar) 10-35%10-35%

5-10%5-10% <5%<5%

200-1000%200-1000%

25-100%25-100%15-35%15-35% 5-25%5-25%

impa

ct (p

rodu

ctivi

ty)

impa

ct (p

rodu

ctivi

ty)

25-50%25-50%

8

Technical challenges

modularization of software avoid accidental complexity but how to keep concerns clean and well-separated

How to retain the design within the actual implementation

architectual integrity Does implementation conform to the architecture?

optimal usage of skills advanced solutions may not be effective for all developers... Can we encapsulate design solutions and make them easy

to (re-)use?

Where does accidental complexity come from?

Understanding the problem

Key technique in SE:

divide and conquer: decompose into modules that are again composed into a system

hence modules/abstractions and corresponding composition techniques: are a key element in computer science mark each paradigm shift in the history of sw.dev. continously new proposals appear...

A composition technique (‘composition operator’) combines the behavior of two or more program elements.A composition technique (‘composition operator’) combines the behavior of two or more program elements.

Result:

subroutine

co-routineaggregationinheritance (Smalltalk style)

inheritance (C++ style)

inheritance (BETA style)

multiple inheritancedynamic inheritanceassociative inherita

nce

delegation

predicate dispatchmixin inheritance

traitspoint-cut-advice compositioninter-type declarations

roles

contractscomposition filters

dependency injection

parameterization

actors

hyperspaces

layered object model

Why? Each technique involves trade-offs

e.g. Smalltalk C++ inheritance

Composition techniques in the physical world

very fewvery few fixed by languagefixed by language thus often not the best trade-off!thus often not the best trade-off!

Composition techniques in software systemsComposition techniques in software systems

Result: accidental complexity

c.f. extra glue, nails & screws: ugly and hard to attach, understand, and modify

because the composition in the problem domain cannot be expressed adequately in the implementation: extra ‘glue’ code to implement relations cannot properly separate concerns

Or need additional design patterns ->more code & complexity

much harder to maintain!

The solution approach

programming languages

should not fix composition

mechanisms,

but make them extensible

and tailorable.

“it is not the strongest of the species that will survive, or the most intelligent.It is the most adaptable to change.”

1859

Darwin (Megginson)

“We need to put tools for language growth in the hands of the users.”

1998

Steele

“... lets [the programmers] [..] express concise solutions and free the original language designer from ever having to say "I'm sorry" ”

2008

Piumarte & Warth

Co-op (‘Composition-Operator’) solution

language enhancement that: allows defining wide range of compositions

including domain-specific ones including solution patterns

compositions are composable compositions are first-class citizens

so they can be part of the solution domain to support reuse, extension and adaptation

(scalability)

or: how application developers can easily (re)use architectural strategies and tactics

Technology Application

Usage of Co-Op

Compositions are first-class abstractions provided by standard library, or custom developed by third party, or custom developed within organisation

Using them is comparable to instantiating and initializing objects.. CompOperator1 newBetween: objectA and:objectB; CompOperator2 newBetween: ClassA and:ClassB; CompOperator3 newBetween: objectA and:objectB

and:objectC; ...

Example of using Co-Op Patterns: Observer Pattern

Library provides simple Observe composition: Creates a new observer (action) that is to be

invoked whenever the methods show and hide have executed on object window:

Observe newSubject: window messages: ( ((List new) add: "show”) add: "hide”) observerAction: [ Console write: "!!! window visibility = “; Console writeln: (window isVisible) ] ;

A quick view on the definition of the operatormodule Observe { var @observerBehavior; init(subject, listOfOperations, observerBehavior) { var callsToSubject; var sendToObserver; var binding; var constraint; @observerBehavior = observerBehavior; callsToSubject = [Selector new:

{ return [[[event target] isSameObject: subject] and: [listOfOperations contains: [event selector]]]; } ];

sendToObserver = [Selector new: { return [OperationRef new: "Observe", "notify", "inner", this]; } ];

binding = [Binding new: callsToSubject, sendToObserver, { return [Dictionary

new]; }]; constraint = [PreConstraint new: defaultCallBinding, binding]; [constraint activate]; [binding activate]; } notify() { [Console writeln: "this=", this]; [@observerBehavior execute]; }}

Example of using Co-Op Patterns:State Pattern

fsm = StatePattern newWithContext: this initState: closedState;

fsm addTransitionFrom: closedState action: "openPort" to: listenState;

fsm addTransitionFrom: listenState action: "receiveSyn" to: synReceivedState;

...

Summary

This illustrates how design level solutions typically involving multiple modules

can be defined and standardized once, And can be instantiated directly by application

developers in the language that they know without needing to know the complexity behind those

solutions In other words: they become empowered to use

advanced techniques

Practical applicability

Large-scale industrial case

4 concepts ‘crosscut’ the system (-30% of code) error-prone, hard to maintain, ...

intint get_kng(KNG_struct* KNG_ptr) get_kng(KNG_struct* KNG_ptr){{ const char* func_name = "get_kng";const char* func_name = "get_kng"; int result = OK;int result = OK;

timing_handle timing_hdl = NULL;timing_handle timing_hdl = NULL; TIMING_IN;TIMING_IN;

trace_in(mod_data.tr_handle, func_name);trace_in(mod_data.tr_handle, func_name);

if (result == OK)if (result == OK) {{ /* Retrieve current KNG *//* Retrieve current KNG */ *KNG_ptr = mod_data.KNG;*KNG_ptr = mod_data.KNG; }} HE(result, "GET_KNG FAILED");HE(result, "GET_KNG FAILED");

trace_out(mod_data.tr_handle, func_name, result);trace_out(mod_data.tr_handle, func_name, result); TIMING_OUT;TIMING_OUT; return result;return result;}}

primary functionality

error handling

30

Industrial case: approach

capture the essence of the 4 concepts separate these into independent abstractions

KNG_struct KNG_struct get_kng()get_kng(){{

return ( mod_data.KNG);return ( mod_data.KNG);}}

31

Industrial case: numbers

We did a test migration for a sample module on three aspects:

code reduction of over 20% for a sample module Company estimated potential benefits (COCOMO):

7-10% effort reduction on software 3% lead time reduction on software

Later controlled experiments for the tracing aspect: initial programming effort reduction: 6% (severity of) error reduction: 77%

Technology realization

Co-op Proof-of-concept

Based on well-structured event reflection Implemented on Java

with Eclipse integration (including debugger) And two prototypes in Miranda resp. Haskell Library with sample composition operators:

single inheritance (3) BETA inheritance

multiple inheritance delegation (2) point-cut – advice (2) traits (cf. mix-ins)

tracing – abstraction memoization – abstraction subject – observer state – abstraction

IDE integration sample

Execution model perspective

Composition Model primitives

Module Event

properties: event name, sender, target, lookup type, event annotation, parameters

Module specifications (may) include: Binding:

Event Selector Action Selector Context Mapping of events

Constraints between bindings

A helicopter view

The event dispatch process

filter bindings

match bindings

apply constraints& evaluate bindings

Trends in sw. development

Upcoming technologies: dynamic languages DSLs (also: polyglot & multi-paradigm programming) Model-Driven Engineering meta-modelling & meta-programming

Common theme: separation of concerns (re-)use of tailored (application specific) abstractions

that can be parameterized avoid boilerplate code (generate)

About related work

LOTS! several frameworks that support variety of composition

operators. few that unify aspect and object composition in one mechanism.

Several approaches for extensible languages we focus on extending with new composition operators (only)

MOPs: our approach is compatible with a MOP approach different from existing MOPs:

no assumptions about (fixed) composition operators unify aspect & object composition mechanisms allow composition of composition operators

Evaluation

Benefits per stakeholder

audience benefits

application developer - straightforward application of canned solutions- write less code (=> less bugs)

software designer - designs are explicit in code, and localized- more robust to change- can choose optimal solutions (instead of a few fixed ones)

senior SE/sw. architect - more opportunities to (guarantee) use of standard solutions- more opportunities to keep code and design consistent- better tools to manage complexity

Sw. manager/project leader - better consistency (i.e. quality)- managing complexity more control over project- empower all team members to build better software easier.

end-user/ product owner - software that has fewer problems- software that is flexible: easier/cheaper to maintain and evolve

Co-op: an agile enabler

value code over documentation: design patterns & standards can be expressed in

software everything is expressed in same language

simplicity: co-op helps to keep the software simple co-op helps to reduce the amount of code

Embrace change better designs and code that matches the design

enables fluent change

Conclusion: Co-op application

Managing complexity of large software technology is a good match for agile development

Offering better means for reuse and evolution helps with customization and version issues

Common solutions can be standardized, reused and/or enforced can lead to substantial code reductions better software quality empowering application programmers

Thanks for your attention

questions? Suggestions?

assistant prof., SE groep / EEMCS

University of Twente, The Netherlands

email: lbergmans@acm.org

phone: +31-53-4894271

web: trese.cs.utwente.nl/~bergmans

software engineering specialist

steX bv, The Netherlands

email: lodewijk@stexbv.com

cell phone: +31-651100838

web: www.stexbv.com

Dr.ir. Lodewijk Bergmans