Measurement, feedback and empowerment: Critical systems theory as a basis for software

process improvement

Petter Øgland

ECIS-17 Conference, Verona, June 10, 2009

Problem

• How to select a design strategy for software process improvement (SPI) for achieving optimal payoff in an organization that is not seriously motivated to do SPI?

Two perspectives

• Brunsson et al (2000): Organizations want to be seen as complying with ISO 9000 and other SPI standards, but they do not want to do what is required

• Flood (1993): Critical social theory (Adorno, Habermas, Foucault, …) should be used as a foundation for TQM (e.g. SPI) to liberate and improve social standards while improving business processes

Hypothesis

• If we look at SPI as knowledge management, then the situation becomes political (relationship: knowledge/power), we can use Flood’s idea, and the SPI implementation strategy becomes optimal

Total Systems Intervention (Flood & Jackson, 1991)

• Creativity– Use Morgan’s metaphors for describing the

organization (problem)

• Choice (SOSM)– Viable Systems Methodology (Beer, 1972)– Soft Systems Methodology (Checkland, 1981)– Critical Systems Heuristics (Ulrich, 1983)– …

• Implementation

Triple loop learning (Flood & Romm, 1996):

2-loop learning (Argyris, 1978) + “might is right”?

1st loop2nd loop3rd loop

How?What?Why?

CST PSM OR

Design of experiment for testing hypotheses:

Design science = design QMS & evaluate

Real world problem Documented specification of solution

Real world solutionDocumented evaluation of solution

TheoryModel

Engineering design

ImplementationDecision

Formulate Solve

New knowledge

Monitoring

Theory

Case study:

• Unit within public sector organization– Approx 20 people (system designers & computer

programmers)– Average age = 40, male/female = balanced– Working according to life cycle model– There is a documented QMS– First version of information system established 1998;

system is now mature and work is concerned with annual updates and new functionality

• Generally seen as one of the better units of the organization (“role model”)

Case study: Technical results

0 20 40 60 80 100

Implementation&

documentation

Testing

Quality plans

Assessmentsand riskanalysis

Analysis &design

Updatingrequirements

Quality index - phase by phase (sorted)

0

10

20

30

40

50

60

70

80

90

100

1998

1999

2000

2001

2002

2003

2004

Annual assessm ent report

0% = Bad

Unstructured changes in format (”improvements”)register as decline in documentation quality

Evolution of QMS: social perspective

• Year 1: Distribution of SPI results sideways and upwards. Emotional stir and frustration. Complaints to head of corporation (saved by SPI owner)

• Year 2: Small improvements, people complain that “products are important, not processes”

• Year 3: Audits show that not only process is of low quality, but predictions about product development are bad too.

• Post-experiment: The methodology is rewritten to achieve better scores without achieving better quality

The effect of TSI on SPI

0

10

20

30

40

50

2000

2001

2002

2003

2004

2005

SPI index

-4

-2

0

2

4

6

8

10

12

2001

2002

2003

2004

2005

Im pr. rate AVG = 3,6UCL = 9,9 LCL = -2,6

H0: µ¹ = µ²Not rejected at significance level 0.022 using t-testwith four degrees of freedom

TSIµ¹ µ²

Discussion: A suggested TSI strategy for playing the SPI game

Create horizontal tension by benchmarking SPI results

Create vertical tension by reportingSPI results one level above internalcustomer

Make sure the QMS owner isthe winner of the political gameof SPI

Conclusion

• SPI standards can imprison organizations in “fake quality” (false beliefs)

• TSI suggests a path toward “real quality”, through critical systems theory (CST), but depends on organizational willingness to admit to problems and commit to methods

• This study presents a different way of implementing TSI, designing SPI as a “conflict machine”, minimizing the needs for admitting and committing

• Three years of data was not sufficient for statistical reasoning, but the phenomenological aspects of the study showed that the TSI-based SPI strategy was successful in the context of the case