WOPs: how to make them even better?

An approach to inform about WOPs’ performance

M.Pascual, S.Veenstra, U.Wehn, R.van Tulder, G. Alaerts 30th May 2013

What are WOPs?

• WOPs encompass a great range of collaboration approaches, each able to foster different type of results

Source: IWA & UNHABITAT., 2009

“Any form of simple or structured partnership aimed at capacity building on a not-for-profit basis” (IWA, UN-HABITAT and VEI,2009)

"(...) a structured programme of cooperation among water operators, based on mutual support and on a not-for-profit basis (UN-HABITAT, 2007).

Minimum of

P-WOPsPerformance based non-delegation driven

partnerships

Key elements for WOPs’ success

• WOP evaluation studies converge in the following recommendations for WOPs to be effective mechanisms for change– Long term collaboration

• The transformation of new knowledge into organisational change is a slow process and requires time.

– Strong engagement from local management and sector stakeholders• The change process has to be led by the local organisation

– Funds for investments • KT is not enough to enable change. Limited operational improvements are feasible if no funds

for investments are available to scale up small steps into change.– Flexibility to emerging circumstances

• WOPs as a CD intervention require flexibility to adapt to the unpredictable evolution of the change process in order to ensure relevance and effectiveness

– High relational quality between partners• Particularly in WOPs, where there is no delegation of decision making power to the external

partner, change is socially constructed by partners interaction.– Report on achievements of WOPs during implementation of the project

• In order to gain and keep confidence of partners and stakeholders to keep supporting the process

State of the art on reporting performance of WOPs

• Change in KPIs are rarely achieved by WOPs except for long term structured comprehensive WOPs associated to investment programs

– Yet, in those cases, KPI improvements take years to be achieved• Even in cases of delegated management through PPPs KPI improvements take at least 4-5 years to

be achieved. I.e. Senegal PPP ,10 years or Mali, 5.– KPIs are insensitive to improvements demonstrated in pilot areas and to nascent changes

like strengthened capacity and improved processes, typical outputs of first years of WOPs implementation

• Hence, other measures are commonly used to report WOPs achievements. However, results reported are :

– Highly heterogeneous, which hinder comparative analysis, learning and dissemination– The dominant approach is to report on the basis of

• Inputs:– No and type of trainings– No of exposure visits, etc.,

• and not so much on the basis of outputs:– No of employees that gained specific knowledge– Degree of change in working routines, etc.

• WOPs have been criticised for being an accepted mechanism based on their solidarity nature despite the lack of empirical evidence of their effectiveness (Boag & McDonald, 2010)

There is need for an innovative standardised system to report performance of WOPs

Multi-path approach to inform about WOPs’ performance*

Project inputs consolidation into change

Degree of satisfaction

KPIs changes

Partners’ relational capital

Partners’ interaction Partners’ roles Dominant governing mechanisms Conflict management

Satisfaction with the project

Selected KPIs of the targeted water operator

Most valuable aspects

Capacity changes

Specific KPI- related capacity

Plausible attribution to the partnership

-

+

Time

Other effects of the projectSustainability of the change trend

Project activities-related knowledge consolidation into change

* performance of a WOP different from performance of water operator

WOP performance is measured through the ability of the partnership to effectively support the local operator in achieving ‘progress towards impact’

Cases in which the multi-path approach to WOP performance

• 2 parallel projects• Same type of contract: 4-years service contract 2009-2013 – no delegation

– VEI and Lilongwe Water Board (LWB)– VEI and Blantyre Water Board (BWB)

• 2 main components:– (i) CD and operational support– (ii) support in the supervision of the implementation of an investment program

• VEI tasks– 1. Organisational structure and staffing – 2. Financial and commercial management – 3.Management information system – 4. Reduction of NRW– 5. Operation and maintenance plans– 6. Optimising pumping regimes – 7. Extension of water supply services to LIAs– 8. Access to water facility– 9. Public Relations works– 10. HIV/AIDS programme– 11. Implementation of investments programmes for improvement of production capacity, reduction of

NRW and extension of services to LIA

• KPIs targets– NRW, Working Ratio, No kiosks built, Production capacity (only in BWB)

• Information collected after 2 years of implementation of the project

KPI achievements in each project

• KPIs targets were not met for NRW. • Only No of kiosks built in BWB and Working Ratio in LWB were met, however, partners argued the

limited impact of VEI inputs in the achievement of those 2 KPIs.

BWB LWB

48

4231

49.1 48.1

25303540455055

Baseline Year 1 Year 2

NRW (%)

Target

Achieved

1.14 1.25

1.01 1.09 1.0611.21.4

Baseline Year 1 Year 2

Working ratio

Target Achieved

030

120

0100200300

Baseline Year 1 Year 2

No of new kiosks built

Target

Achieved

39

3430

3938.7 36.9

2530354045

Baseline Year 1 Year 2

NRW (%)

Target

Achieved

1.22 1.271.141.2

1.38

1

1.2

1.4

Baseline Year 1 Year 2

Working ratio

Target Achieved

0

90192

0 0 270100200300

Baseline Year 1 Year 2

No of new kiosks built

Target

Achieved

Change in capacity to reduce NRW and plausible attribution

BWB LWB

Highest capacity change

Organisational structure, availability of resources Strategic management, resources and capability to reduce leakages and overflows in storage tanks

Project strongest contribution

Change of organisational structure, organisational motivation, resources, capability to reduce leakages, and learning capability

Organisational motivation, resources, capability to reduce leakage and learning capability

Strategic managerial level

Responsiveness of departments

Organisational motivation

Capabiilty to reduce customer metering errors and data handling errors

Capability to reduce leakages on service

connections up to pint of customer

48%48%47%

57%50%

43%50%

41%42%

53%46%51%

44%50%

25%33%36%

27%33%

34%36%

28%29%

28%34%

32%30%

25%

28%20%18%17%17%

23%14%

31%29%

20%20%16%

26%25%

Plausible attribution of changes by BWB staff (n=15)

Internally Partnership Externally

Project inputs consolidation into change

• Szulanski (2000) defines KT as– “ an unfolding process comprised of several stages that start with the

opportunity identification for knowledge transfer and ends once the recipient unit is able to maintain satisfactory performance derived from the integration of knowledge into working routines”

Project inputs consolidation into change - Project tasks-related knowledge translation into change-

Project inputs consolidation into change - Other effects from the project -

BWB LWB

CEO A force supporting organisational change via:-Reducing internal resistance to change;- Limiting political interference;- Increasing accountability for results;

A more accurate picture of the current status of LWB

WBs’ staff -Motivation to improve and engagement of staff;-Willingness to learn and improvement of staff attitude-Sense of urgency to improve;-Commitment of the different sections with a role in NRW reduction;-Improvement of management and coordination skills

-Stronger commitment to reduce NRW throughout the organisation; -Higher sense of urgency to improve

Project inputs consolidation into change - Degree of sustainability of perceived change -

– BWB members showed mixed opinions while LWB staff argued that if the project stopped the urge to improve would not be there any more.

– The conditions identified to sustain the change trend are mostly shared by both WBs:

BWB LWB

- Strong support from BWB management to the project and increase efforts to complete implementation of improvement plans

- Capacity to retain qualified staff- Maintain joint work with VEI- Keep the momentum of change and spirit of

improvement of the local staff- Enhance management and operational skills

- Executive management engagement and higher commitment and ownership of the project, particularly in implementation of plans

- Increased ability to retain staff- Foster good relationship and understanding

with VEI to favour joint work- Stronger communication, coordination between

different sections and team spirit and good working relations within LWB

- Additional training

Partners’ relational capital- Interaction, roles, governing mechanism, conflict-

BWB LWBFormal interaction Management: high to medium

Operations: high to highManagement : high to lowOperations: high to low

Informal interaction Management: low to mediumOperations : low to high

Management: low to lowOperations: low to medium

Roles external partner Consultant -> colleague Consultant -> controller

Engagement of local partner Low to high Low to low

Dominance of governing mechanisms

Contract -> contract + trust Contract -> contract

Conflict occurrence and impact in the partnership

Yes (KPIs, role of VEI )Positive impact on the partners relationship

Yes, (KPI, role of VEI, engineering tasks take over)Negative impact in relationship

Satisfaction of partners and stakeholders

• Degree of satisfaction

• Observations from partners and stakeholders– MIWD, EIB, EU

• Appreciate the innovative approach of an external operator working together with the WBs in strengthening their capacity while there needs to be accountability for results.

• Positively valued in terms of sustainability of the achieved improvements• They call for the need to go beyond KPIs to evaluate progress of project

– WBs• BWB, appreciation of the project at all levels.• LWB, appreciation of the project at operational levels, but not at executive management.

– VEI RPM • BWB emphasised the critical role of the CEO’s support and its translation into approx. 10 members of

WB actively supporting the change process.• LWB points at the lack of support from executive management, which translates into low

commitment at operational levels (only 2 members engaged)

BWB LWB

WBs’ executive management High Low

WB’s staff at operational and management level

High High

VEI RPM (resident), VEI PD High Medium

EIB, EU, MIWD High Low- Medium

Conclusions

• Findings– KPIs achievements provided vague information on the achievements of the recipient local operator as well as

effectiveness of the partnership – The proposed complementary paths of evidence proved highly informative on the progress of each project and the

effectiveness of the partnership

• The proposed paths of evidence serve different purposes

• Data sources are highly based on perceptions. Reliability of findings requires:– An objective (external) evaluator– A standard multi-method for data collection– A standard method for data analysis– Representative sample of key informants from both partners and stakeholders– Triangulation of information

Progress towards impact Effectiveness of the WOP Learning and adaptingKPIs XCapacity changes and attribution

X X X

Knowledge consolidation into change and obstacles

X X X

Partners’ relational capital X XPartners’ satisfaction X X

Statements for future improvement of WOPs

1. P-WOPs have the potential to be a slow but firm route to public driven utility reform processes, yet they have to be accountable for results and demonstrate its effectiveness in order to become a widely legitimised approach.

2. A widely used standard system to report P-WOP performance will not only build credibility of WOPs but also enable learning towards improvement of its effectiveness.

3. P-WOP contracts should be designed according to the notion of WOP performance and not merely to performance of the local water operator – Hence, incorporate ( shared ) incentives based on the multiple evidence

paths proposed

Purpose of 5th Symposium

Thank you for your attention.

Maria PascualUNESCO-IHE

m.pascual@unesco-ihe.org