Upload
kjtaylor161
View
705
Download
1
Tags:
Embed Size (px)
Citation preview
THEMOST SIGNIFICANT CHANGE
(MSC) TECHNIQUE
RARE, 2013 - Assessment Task 2
A dialogical approach to information gathering for monitoring and evaluation.
Group Presentation Member’s Profiles
Dorothy - located in Dili, East Timor.
Working in international aid within the security sector.
Karen - located in Jakarta, Indonesia. Currently working in international aid within the education sector on programs funded through Australian Aid.
Carol - located in rural Victoria, Australia. Working for an NGO within the welfare sector.
Our Stories
Presentation Content
1. Description of MSC.
2. Purpose of MSC.
3. Appropriate contexts and uses of MSC, and inappropriate contexts and uses of MSC.
4. Strengths of MSC.
5. Weaknesses of MSC.
6. Example of the MSC approach.
7. MSC activity for the class.
Introduction
• The Most Significant Change technique (MSC), is located in the monitoring form of evaluative inquiry.
• MSC can be used in a program cycle for monitoring and for program evaluation.
• MSC collects qualitative data rather than the traditional quantitative reporting in program monitoring.
1. What is MSC?
Planning
Implementation
Monitoring
Outputs
Quantitative Reporting
The Most Significant
Change technique is
a participatory form
of program
monitoring and
evaluation.
Typical Program
Cycle
…What is MSC?
• MSC contributes to program monitoring and evaluation processes through capturing participants stories, in their own words.
• The story recording and subsequent hierarchical analysis assist workers and other stakeholders to understand the impacts of the program.
• Program improvement and worker learning are thus enhanced through the MSC technique.
…The MSC Story
MSC Time Line
1994: Davies began to develop the methodIn Bangladesh
1996: Called the ‘Evolutionary Approach to Organizational Learning’ (Davies)
1998: Termed ‘monitoring without indicators’ (Guijt, Arevalo, & Salsdores)
1999: Also known as ‘The Story Approach’ (Dart)
1999: First used in Australia, called the ‘Most Significant Change’ Technique (MSC) Dart et. al.
2009: Further use in Australia – The Ambassador Project, Whitehorse, Victoria
2000: MSC approach internet discussion group established
2005: Published in the American Journal of Evaluation (Dart & Davies)
2. What does the Most Significant Change (MSC) technique do?
• Uses a concept of ‘dynamic values inquiry’.
• Searches for, and deliberates over, the value of significant program outcomes.
• Directs work toward explicitly valued directions and away from less valued directions.
…Collecting SC Stories
Formulating an open ended question to collect Significant Change (SC) stories:
1. “Looking back over the last month…
2. …what do you think was…
3. ..the most significant…
4. …change…
5. …in the quality of people’s lives…
6. …in this community?”
…The MSC Process
Participant stories collected
Stories discussed & placed in domains
SC stories chosen & reasons stated for choices
Qualitative program outcomes and impacts
Verification, quantification & report
MSC Program
Cycle
Incorporates feedback loops within the project, and/or organisational hierarchy.
…MSC Valued Directions
Voices of participant's
Workers hear effects of work and reflect on practice
Stages of participatory discussion, review and SC selection
Selection group reflects on practice
Funders & managers reflect on desirable outcomes
A picture of what participant's value, stakeholder’s value, and funder’s value.
Cycle of improvement
& learning
3. Where does MSC fit?
Introduction
• The evaluator must always choose approaches and methods which are the most appropriate for the situation.
• MSC implementation can be adapted to suit some situation specific characteristics.
• However, MSC is more useful and better aligned to certain contexts than others.
…Where does MSC fit?
Everywhere? Anywhere?
• MSC has been implemented in developing and developed countries (e.g. Australia and Solomon Islands).
• Across the range of organisations and institutions e.g. government, community groups and businesses.
Therefore, is MSC appropriate for any country, program, project or institution?
…Where does MSC fit?
Contexts
Two elements to consider for appropriateness of context for MSC for the program, project or institution are:
• Level of complexity.
• Internal characteristics.
…Where does MSC fit?
Context - Level of Complexity
Appropriate Inappropriate
Highly Complex Simple
Multilevel Single level
Multiple Stakeholders Few Stakeholders
Divergent Values Focussed Values
Conflicting Opinions Agreement
Ambiguity Clarity
…Where does MSC fit?
Context - Internal Characteristics
Characteristic Appropriate Inappropriate
Culture of Program, Project or Institution
Learning Culture Accountability Culture
Improvement Oriented Success Oriented
Open to New Approaches
Closed to New Approaches
Understanding of MSC
Accurate Inaccurate
Realistic Expectations Misguided Expectations
Initial Pilot Attempt Full Implementation
…Where does MSC fit?
Context - Internal Characteristics
Characteristic Appropriate Inappropriate
People Leadership Stable Changing Leadership
Senior Personnel Support Commitment and Involvement
No Senior Personnel Support, Commitment or Involvement
Resourcing Adequate finances Considers MSC ‘cheap’ with inadequate provision
Knowledgeable Personnel
Considers MSC ‘easy’ -lacking knowledge
Realistic allocation of time for all MSC phases
Considers MSC ‘quick’ with unrealistic time allocations
…Where does MSC fit?
For anything?
• It is possible to use MSC for planning, reporting, public relations, communication, monitoring and evaluation.
But what are the more appropriate uses for MSC?
…Where does MSC fit?
Uses
Appropriate Inappropriate
Complementary technique within a suite of M & E methods
Only technique used for M & E
Focus on extreme experience Focus on average experience
Program Improvements Retrospective evaluation for a finalised program or project
Reflective Practice
Program Outcomes and Impacts
Fluid & Emerging Outcomes Known Outcomes
Changing Focus Static Focus
…Where does MSC fit?
Uses
Appropriate Inappropriate
Capturing Unexpected Change Changes Expected
Attaining Shared Vision Vision Established
Participatory Approach Confidentiality Issues
Evaluation Capacity Development Evaluation Knowledge with Experts
Capturing ‘voice’ of beneficiaries Ignoring beneficiaries
Qualitative – Hearts & Minds Quantitative – Statistics & Numbers
4. Strengths of MSC
• Inclusive and participatory at all levels.
• Transparent.
• Reflects a strong story telling tradition: Common in many non-Western cultures.
• Contributes to organisational learning:Provides opportunities to discuss what the project was trying to achieve and how it can be improved.
Can help improve program management.
• Non-threatening:No right or wrong.Genuine input from participant’s perspectives.
…Strengths of MSC
• Highlights unexpected changes.
• Provides data on impact and outcomes.
• Highlights social benefits.
…Strengths of MSC
5. Weaknesses of MSC
• Costly:Time.Human resources.
• Captures extreme cases – not the average.
• Validity and reliability of stories.
• Language – can be a barrier/impediment to the process.
• Difficulties with domains – “fuzzy”.
…Weaknesses of MSC
6. MSC In Action
Check out this link to see a great example of MSC in action:
http://www.youtube.com/watch?v=NkuJ69zKScU
To engage you further in the MSC technique we have put together some activities for you to try.
Please refer to the activity section within the written component of our MSC presentation.
7. MSC Class Activity