QAT Detailed Overview of QAT Process.ppt

Quality Audit Toolfor Managing Social Performance

MFC Quality Audit Tool 2

Acronyms Used

• QAT – Quality Audit Tool• ECA – region of Eastern Europe and Central Asia• MFI – microfinance institution (MF Banks, non-

banking financial institutions, non-governmental organizations)

• SP – Social Performance• SPM – Social Performance Management• SR – social responsibility


Agenda

1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation


Agenda



Rationale: ECA Region Context

• Since beginning in 1997 focus and pressure on MFIs’ self-sustainability

• Withdrawal of donors to less developed countries

• New comers – commercial investors (i.e. Deutche Bank, Raiffaisen Bank)

• Post – soviet negative connotation of ‘social’ = for free

• Commercialization of MFIs

• Mission drift observed

• Still huge interest in impact, but no resources to measure

• Difficulties in promoting Social Performance

• No commitment to social goals at the level of operations

• Interest in easy and effective tool bringing quick measurable results

Context Market trends Social Performance reality


QAT Purpose

1. Assessing status and effectiveness of systems in managing SP

2. Helping MFI realize its strengths and weaknesses in SPM

3. Helping MFI prioritize actions for improvement of their SPM

4. Ensuring institution-wide understanding and buy-in

1. To ensure MFI is on track: ‘how effective are your systems in aiming at achieving your mission?’

2. To optimize resource investment: ‘build on what you have and invest in what you really need’

3. To prevent MFI from doing everything at once and burn out shortly – ‘let’s improve step by step and look for quick wins’

4. To create enabling environment for change; to prevent SPM be one person ‘project’

QAT aims at… … in order to:


QAT Target Markets

• MFIs concerned about their Social Performance and quality of their management systems for SP

• MFIs planning social rating exercise and willing to prepare themselves

• Donors / Investors / Partners willing to support their partnering institutions in improving their SPM


SP Dimensions Assessed by QATDim Sub-dimension Content

1.1. Setting SP objectives • Mission translation• Social goals and objectives (outreach, service, change)• SR to clients, staff, communities, environment• gender aware policies

1.2. Strategy for achieving SP objectives

• Systems for developing coherent strategy

2.1. Systems for monitoring and understanding performance

• Operations and marketing: mechanism for collecting clients feedback

2.2. Information system quality and appropriateness

• Marketing, HR, Research, MIS, internal audit: data reliability

2.3. Information analysis and communication

• Marketing, operations: data analysis and reporting

3.1. Information use • Management practice: data use, performance monitoring

3.2. Decision making • Decision making at strategic and operational level

3.3. Organizational culture • Governance, leadership• Internal communication

3.4. Alignment of organizational systems

• HR: Staff Incentive System; recruitment and training• Change management capabilities

Man

ag

em

en

t syste

ms

SP

in

form

ati

on

syste

m

Inte

nt

an

d

desig

n


SPTF Dimensions in QATSPM-mission clarity How well is the mission understood by all staff?

Is mission translated into goals and SMART objectives? How well it is implemented into practice?

SPM-alignment of systems

How does internal systems support achievement of mission?

SPM-decision making Are decision (both strategic and operational) made based on SP info? What is the reliability of SP info used?

Gender approach Does MFI have gender aware policies?How effective are they implemented?

Non-financial services Covered indirectly through overall assessment of institution strategy including range of services and its ability to answer clients’ needs

Responsibility to clients

Does MFI have client SR policies?How effectively are they implemented?

Responsibility to community

Does MFI have community SR policies?How effectively are they implemented?

Responsibility to staff Does MFI have SR towards staff policies?How effectively are they implemented?

Responsibility to environment

Does MFI have environmental SR policies?How effectively are they implemented?

Social Goal--Outreach This is referred through process questions on mission clarity, strategy, objectives:

Does MFI define their target clients properly?How effective it is in reaching the target clients?

Social Goal--Services This is referred through process questions on mission clarity, strategy, objectives:

Does MFI know what are the target clients needs?How effectively are they met?


QAT Uses for MFI Manager

• To check if an MFI is at right track towards their mission achievement

• To identify strengths supporting their SPM• To identify weakness: gaps and areas for

improving their SPM• To use prioritized initiatives for developing action

plan for improvement, allowing step by step SPM institutionalization

• To buy in staff at all levels; to build understanding SPM is overarching theme rather than project or tool

• To monitor progress in SPM institutionalization (in case of repeated audit)

• To prepare for social rating


QAT Uses for MFI Manager - Examples

Horizonti QAT lessons learnt:• „This was very important for

we did the audit before implementing anything: conducting the audit we knew the gaps – where we need to pay more attention.

• One of the outcomes – we do know what we are doing, we are targeting the right people and doing a good job, but we need to work on systems to support this.

• We saw that the MIS cannot support performance, so we will have a new one improved.

• We collect a lot of information which can be used for SP indicators.”

ED, Horizonti, Macedonia

Zene za Zene QAT next steps:• Define more specific objectives

for better understanding what it means for the terms of SP?

• To experiment with MIS reports content

• To improve internal communication

• To measure drop-outs• To use client satisfaction form

across the institution

SPM Champion, Zene za Zene, B&H


QAT Uses for Donor / Investor / Partner

• To help MFI improve in SPM through diagnosing strengths and weaknesses and developing action plan

• To optimize its support through financing or helping implement the initiatives from action plan

• To better understand the institution, its mission orientation and rational behind its translation to action

• To monitor progress in SPM institutionalization (in case of repeated audit)


QAT Desired Impact: MFI Level

• Outcome - setting the stage for change processClearer operational understanding of SPMClear understanding by staff of what needs to be done to improve SPM Internal champion and sponsor identified – infrastructure for change establishedBetter communication of social orientation and required support to stakeholdersQuick wins implemented

• Short term impactProcess of SPM improvement kick-off Consistent effort of MFI to improve SPM in a manageable steps There is on-going buy-in of staff, leadership of champion and support of senior management and board for implemented changes

• Medium term impactInternal systems are improved to be better aligned with double bottom line goals

• Long term impact – expectedInstitution gets all aligned to double bottom line and results are improved


QAT Impact: Examples From MFIs

Gap: description of the target group was vague and mixed with eligibility criteria for the loan

Quick win: through a series of half day workshops defines what is meant by saying „we want to reach economically and socially challenged families” and

comes up with the indicators based on housing materials, asset base, etc.

Gap: At fast growth, new staff don’t understand and commit to mission

Quick win: re-design of training program for newly hired staff to ensure mission perspective

incorporation

Gap: collects lot of information, which is not used later on

Quick win: review the new set of MIX SP indicators to see which information they can start using

immediately

Micro Development Fund, Serbia

CARD, the Philippines

Moznosti, Macedonia

More examples of MFIs’ quick wins in MFC Newsletter Issue 1/2008

http://www.mfc.org.pl/nl7/newsletter1_2008_eng.pdf


QAT Desired Impact

• Investor / Donor / Supporter level:Increased number of partners refocus from assessing SP to improving SPM more supportive and longer term cooperation on SPM improvementMore informed partners make better decision in selecting more likeminded partnersMore efficient allocation of resources to likeminded MFIs and activities that can bring greatest value in improving SPM at MFI

• Industry levelMore MFIs aware of their SPM status and effectivenessMore MFIs improve in their SPMLess mission driftBetter public image of Microfinance


QAT Benefits for Various Stakeholders

• MFI Managers:Clear current picture of SPM systems qualityGood basis for effective implementation planEfforts oriented on mission acknowledgedStaff understands better and is bought-in through discussion about SPMAllows improving before future social rating

• MFI staffBetter understanding of how their tasks fit into social goals and strategy

• Donors / investors / supportersBetter understanding of MFI’s commitment to mission and its translation into practiceAssessment of MFI’s SPM qualityEffective identification of MFI’s needs in terms of supporting their SPM institutionalization


QAT Uses – Extracts from 5th SPM Working Group Meeting Minutes

In March 2008, the representatives of 10 organizations – members of MFC facilitated SPM Working Group, met in Warsaw to discuss lessons learnt in SPM institutionalization. Among other subjects they discussed QAT uses:

• „It can be used before the process of SPM implementation – it is nice to have gaps

• Could be used for process evaluation – a good diagnostic tool• Could be useful for taking on board those who were not directly

involved – e.g. IT people• Gives an understanding where the gaps are and what actions need to

be taken• The format is very useful – the SPM team might be biased – but

through interviews one gets more objective information• Could be used multiple times – especially before the strategic

planning• Provides awareness of different opinions and perceptions• Stimulates additional discussions at the institutions on SPM• Gives better understanding of HR needs – where there are gaps in

assumed and actual way the processes work• Remind and refresh the mission and social objectives• Reveals information flow bottlenecks


Agenda



Agenda



4 Steps in QAT

• ED• Senior management (Finance, Operations, HR, Marketing, )

• Board• Senior management• Internal auditor• MIS staff• Research staff• Regional level• Branch level• LO• Clients

• All

0,5 day 2 - 3 days1-2 days 0,5 day 1 day

Preparation

Gap analysis

Follow up interviews

Audit panel

Final report

Analysis &draft report

1 day

Sta

ff p

art

icip

ati

ng

Total time: from 6 up to 8 days


External & Internal Assessor

QAT is implemented by the team of two people: external and internal assessor:

Internal Assessor External Assessor

Neutral job position (best from HR or Marketing, with managerial perspective understanding)

Experienced in Microfinance

Internal assessor can help to ensure commitment from management & staff

Skilled in qualitative research skills ensures appropriate probing

Valuable knowledge of the organizational systems and information

Impartiality and independence ensures review of organizational systems and information to check for reliability

SPM understanding is required Skilled in analysis and synthesis of complex information


QAT Implementation Steps (1)

1. Gap analysis: presents detailed questions which are used to assess SPM; sets up a base for more in-depth follow-upSource: ED and senior management

2. In-depth follow-up: gathers more detailed information to verify and better understand gap analysis; checks status and effectiveness of systemsSource: Interviews with senior management, staff &

clients; review of documents and internal reports; and information flows.



3. Analysis and draft report: combine information and draft report highlighting strengths and weaknesses of issues in gap analysis; it captures perspectives of different stakeholders to be discussed during audit panel.

Analysis include:Qualitative data analysis using triangulation of different information sources (interviews with different staff, focus groups with clients, review of written documents)Tally sheet is used to identify common themes and divergences around strengths and weaknesses of MFI systems, notes from the interviews are reviewed for evidence and rationale behind emerging themes

Draft report is written addressing strengths, weaknesses, evidence and issues to further discuss/explore that serve as basis for discussion with representatives of different stakeholders during an audit panel.



4. Audit panel: Presentation of findings to key organizational stakeholders, including staff from all levels of organization and clients, where appropriate.

Audit panel allows for further verification of the audit results and finalize the report The initiatives for addressing gaps are identified and prioritized in order to develop action plan for SPM improvement

5. Final report: finalized based on audit panel discussion

Includes prioritized activities to be undertaken by MFI in order to improve SPM by addressing the identified gaps and by building on identified strengths


Source of Information and its Validation

Source of information Validation: what for?

In-depth interviews with senior management

Set a base, learn the strategic directions

In-depth interviews with middle management and functional staff

Confirm status, confirm evidence, cross-check data

Focus Group Discussion with Loan Officers and Clients

Confirm status, confirm evidence, cross-check data

Internal documents and reports: Board meeting minutes, codes of conduct, financial statements, portfolio reports, strategic plans, HR policies, MIS reports, clients satisfaction / exit surveys etc

Look for evidence, cross – check data

Audit panel Verifies findings, explore doubtful issues, includes more evidence


Data Collection Methodology

• Gap analysis questions: basic set of questions defining the framework of SPM assessment

i.e.: do you seek to be socially responsible organization in relation to your target clients?

• Status questions: questions validating the status of SPM components: their presence in the institution

Asked to Loan Officer: How do you communicate interest rate to clients? How do you ensure they understand?

• Effectiveness questionsAsked to clients: How do you learn about interest rate? How do you know what is it?


QAT questions examples

Sourc

e:

QA

T H

andbook


QAT Report Structure (1)

• Report is divided according to dimensions and/or sub-dimensions

• Each dimension / sub-dimension includes:[to be discussed during audit panel:]

List of institutional strengths supported by evidenceList of institutional weaknesses supported by evidencePoints for discussion: when there are different and not coherent opinions and evidence, they can be clarified and agreed during audit panel

[after audit panel:] key elements of the action plan


Dimension 1: Intent & designSub-dimension: Setting social performance objectives

MFIs strengths:• xxxx

– EVIDENCE: xxxxx

MFIs areas for improvement:• xxxx

– EVIDENCE: xxxxx

Discussion points:• Something still unclear• various opinions

Action plan:• What are the priority actions?• What are quick wins?• What are immediate next steps?

QAT Report Structure (2)


Agenda



Agenda



QAT Attributes - Time

• Preparation (documents review)• Gap analysis• Individual interviews• Focus Group Discussions with Loan

Officers and Clients• Analysis• Audit panel• Final report

• 1 – 2 days• 0,5 day• 1,5 – 2 days• 1 day

• 0,5 -1 day• 0,5 days• 1 day

• Depends on skills and experience of assessor• With external facilitator the process is more intensive and

shorter• Internal Assessor can adjust duration to the internal

dynamics

TOTAL: 6 – 8 days

Process steps Duration


QAT Attributes: Cost

• Staff timeInternal AssessorStaff in interviewsStaff in audit panel

• Travel to branches for FGDs

• Snacks for FGDs

• Includes:6 – 8 days1,5 h per person3 h per person

• Travel costs• Costs of drinks and

snacks

• External facilitator time• Lodging• Travel

• 6 days * daily rate• 6 nights * daily allowance• Cost of plane ticket

MFI

cost

sE

xte

rnal

faci

litato

r

MFC estimated cost for external assessment:6000 USD + staff time cost


QAT Attributes: Breadth of Functionality

For whom?

Implementation usage

Results usage

Staff • understanding how their tasks fit into strategy and mission• securing their buy-in

• acknowledgement of up-to-date efforts

Management and Board

• ensuring all staff is on the same page

• assessment of effectiveness in SPM• effective identification of areas and actions to improve SPM

Supporters / Investors

• understanding the organization better (it’s social mission and its translation to practice)

• managerial systems quality assessment• effective identification of areas and actions to support SPM institutionalisation


QAT Attributes: Infrastructure and Resource Requirement

• Resources required:Staff timeFunding for external facilitator budget

• Infrastructure required:Computer for assessors, recorderSeparate room for conducting interviews (preventing noise, securing proper concentration)Convenient room for 2 FGDs (with Clients and Loan Officers)Transportation to branch (if needed)Conference room for holding audit panel (including: LCD projector, flipcharts, markers)


QAT Attributes: Ease of Use

• In case of external assessment:MFI needs only to set up logistics and select internal assessorAdditional benefit: MFI is trained on how to conduct the assessment to do it independently next timeChallenge: securing the funding

• In case of self-assessment:Pre-assessment training is requiredTiming adjusted to the internal dynamics of MFIChallenge: the selection of internal assessor must be more careful


QAT Attributes: Customer Satisfaction

• Assessing both strengths and weaknesses makes the audience appreciate the objective assessment: they are motivated to improve as their up to date efforts to achieve the mission are acknowledged

• Involvement of many stakeholders into process and allowing them commenting during audit panel make them owe the findings in opposite to external findings (external assessor becomes perceived rather as facilitator)

• Developed action plan reflects the real needs and does not irrationally overburden staff

• Clear and easy to follow structure of the report makes it a handy document for further work


QAT Attributes: Customer Satisfaction in Citations

• I am pleasantly surprised how everything is now clear – the role and the goals of SPM

• External assistance in the audit is very helpful. I wonder whether to us people would say what the external auditor found out. I have a feeling they tell us what they think we want to hear.

• It was good that loan officers and branch staff were included in this audit. They gave a needed perspective for the understanding of the issues they face in their work. The fact that all staff was included means we will have full support for the changes

• I am the SPM team member. During this week it became clear that we did a lot of stuff, discussed and decided on many points. I now see what will be the result of the entire SPM effort.

• Social audit made me realize that we are already working on many aspects of SP in our organization, but have not given it the right emphasis

I am now better aware of the importance of the right loan for the client. I will pay more attention to how I do my work, how the loan is used and whether the client is advancing. This is giving me an encouragement for my future work

Partner, Bosnia and Herzegovina

In our case, this tool revealed many weak points in the systems and management processes and resulted in creating an institutional strategy with a holistic approach

AgroInvest, Montenegro, Serbia


Agenda



Agenda



QAT vs. Other Tools (1)

• Gathers perspectives of various stakeholders• Discussion on findings in held

• Assesses quantitative outputs: social accounts• Focused on outputs, not internal systems • QAT allows assessing the effectiveness of systems

Similarities with QAT

Key differences

Compliance audits: social accounting by Social Audit Network (UK)

• Assesses status and effectiveness

• Uses a scorecard to assess SPM, which is less flexible – comparison to desirable characteristics of MFI• Is not aligned with SP framework• Allows for comparison between organizations to greater extent• Does not encourage immediate management reaction and commitment to improvement

Social Performance Audit Tool (SPA) by USAID


QAT vs. Other Tools (2)

• Methodology of data collection is the same (interviews, documents)

• The greater focus is put to outputs: outreach, client service, information transparency, customer protection, • Less focused on internal management systems

Similarities with QAT

Key differences

SOCIAL Tool by ACCION

• Goal: to determine the abilities of MFI to achieve social objectives

• More quantitative questionnaire• Covers less areas: outreach to poor and excluded, adaptation of products and services to target clients, improving social and political capital, Corporate SR

Social Performance Indicators by CERISE


MFIs Implementing QAT Worldwide

• ECA regionAgroInvest and MDF, SerbiaAregak, ArmeniaFinDev, AzerCredit and Viator, AzerbaijanHorizonti and Moznosti, MacedoniaPartner and Zene za Zene, Bosnia and Herzegovina

• The PhilippinesASHI, CARD

• MENA RegionTamweelcom and DEF, JordanTill end of 2008: ARDI and AMOS, Morocco


Contact Details

For more information on QAT, please contact:Ewa Bankowska, Project CoordinatorE-mail: [email protected]: +48 22 622 34 65Skype: bankowska

Business

QAT Detailed Overview of QAT Process.ppt