Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
“Toward Social Inclusion of Vulnerable Children: Expanding
Alternative Care, Family Support and Inclusive Education
Services as Part of Child Care Reform” Project
(RFP Number: LRFP-2017-9132946)
Mid-term Evaluation
Final Report
Submitted by
UNICEF country office in Armenia
Timeframe of the evaluation September 2014 - June 2017
Geographical scope of the evaluation Yerevan, Lori, Syunik
Management of the evaluation UNICEF country office in Armenia
Independent evaluation team VISTAA Plus LLC (Armenia)
This evaluation is made possible by the support of the American people through the United
State Agency for the International Development (USAID). The contents of this evaluation are
the sole responsibility of VISTAA Plus LLC and do not necessarily reflect the views of USAID,
the United States Government or UNICEF.
December 2017
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
1
CONTENTS
CONTENTS ................................................................................................. 1
I. List of TABLES AND CHARTS ...................................................................... 2
Ii. List of Acronyms ....................................................................................... 3
Introductory Note by VISTAA Evaluation Team ........................................................................................... 5
IIi. Executive summary .................................................................................. 7
CHAPTER I. INTRODUCTION TO EVALUATION..................................................... 18
- Object of the Evaluation, Purpose, Objectives and Scope .......................................... 18
- Background of the Intervention and Context ....................................................... 19
- Logic of Interventions ................................................................................. 23
CHAPTER II. EVALUATION METHODOLOGY........................................................ 26
- Qualitative Data Collection: Approach and Implementation ....................................... 28
- Quantitative Data Collection: Approach and Implementation ..................................... 31
- Major Limitations ..................................................................................... 34
- Ethical Considerations, Human Rights and Gender ................................................. 35
CHAPTER III. FINDINGS ................................................................................. 37
CHAPTER IV: CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS ................. 82
CONCLUSIONS ............................................................................................ 82
LESSONS LEARNT ......................................................................................... 84
RECOMMENDATIONS ................................................................................... 84
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
2
I. LIST OF TABLES AND CHARTS
Table 1: Key Expected Results under the Project ......................................................................................... 23
Table 2: Stakeholder Mapping ....................................................................................................................... 27
Table 3: Distribution of In-Depth Interviews by Marz and Informant Groups .......................................... 28
Table 4: Focus Group Distribution by Marz and Type of Key Informant ................................................... 30
Table 5: Proportion of Schools by RPPSC Service Area ............................................................................... 31
Table 6: Number of Schools Included in the Sample Based on Availability of Assessed Children from
Rural and Urban Schools ................................................................................................................................ 32
Table 7: List of Schools in the Final Sample .................................................................................................. 32
Table 8: Readiness to Implement Inclusive Education (average value) ...................................................... 47
Table 9: Roles and Competencies of Project Partners .................................................................................. 50
Table 10: Transformation Status of Institutions ........................................................................................... 54
Table 11: Utility of Training Session ............................................................................................................. 61
Table 12: General Knowledge Index by Sub-region ..................................................................................... 63
Table 13: Application of knowledge (Data source-240) ............................................................................... 64
Table 14: Socializing Patterns of Children with SEN (Data source-175) .................................................... 65
Table 15: Project Budget and Spending ......................................................................................................... 68
Table 16: Assessed Children by Gender ........................................................................................................ 80
Chart 1: Rationale for Attitude ...................................................................................................................... 48
Chart 2: Average Value of Training as Assessed by Respondents (5 being highest and 1 lowest) ............. 62
Chart 3: Change in Absolute Knowledge ...................................................................................................... 63
Chart 4: Gender Distribution across Foster Families .................................................................................... 81
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
3
II. LIST OF ACRONYMS
ADP Area Development Programs
AMD Armenian Dram
BoH Bridge of Hope
CFSC Child and Family Support Center
CPD Country Program Documents
CRC Child Rights Convention
CRPD Convention on the Rights of Persons with Disabilities
CSCF Child Support Center Foundation
CwD Children with Disabilities
DFID Department for International Development
DI Deinstitutionalization
EU European Union
FAR Fund for Armenian Relief
FG Focus Group
FM Frequency Modulation
FWCPU Family Women and Child Protection Units of Regional Administrations
G2G Government to Government
GoA Government of Armenia
GTC Guardianship and Trusteeship Committees
IE Inclusive Education
IP Implementing Partner
ISS Integrated Social Services
ITT Indicator Tracking Table
LSP/TSP Local Social Planning/Territorial Social Planning
M&E Monitoring and Evaluation
MoES Ministry of Education and Science
MoLSA Ministry of Labour and Social Affairs of Armenia
MoTAD Ministry of Territorial Administration and Development
MoU Memorandum of Understanding
NGO Non-Governmental Organization
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
4
NIE National Institute of Education
NORK Social Services Technology and Awareness Center
OSI Open Society Institute
PIO Public International Organizations
PPC Psychological-Pedagogical Center
PPSC Psychological-Pedagogical-Psychological Support Centre
PSA Public Service Announcement
Q&A Question and Answer
RA Republic of Armenia
RFP Request for Proposal
RoA Republic of Armenia
RPPSC Republican Psychological-Pedagogical-Psychological Support Centre
SC Save the Children
SDG Sustainable Development Goal
SEN Special Education Need
SOAR Society for Orphaned Armenian Relief
SPSS Statistical Package for the Social Sciences
ToR Terms of Reference
ToT Training of Trainers
TSP Territorial Social Plan
UK United Kingdom
UN United Nations
UNCRC United Nations Child Rights Convention
UNDP United Nations Development Programme
UNICEF United Nations Children's Fund
USAID United States Agency for International Development
USD United States Dollar
WB World Bank
WV World Vision
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
5
Introductory Note by VISTAA Evaluation Team
To the Final Report Commissioned by UNICEF in Armenia for
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative
Care, Family Support and Inclusive Education Services as Part of Child Care Reform” Project (RFP
Number: LRFP-2017-9132946)
August 14 – November 30, 2017
In August 2017, VISTAA was awarded the contract for conducting the Mid-term Evaluation of the
‘‘Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and
Inclusive Education Services as part of Child Care Reform’’ project (LRFP-2017-9132946).
The Project itself has started in September 2014 with an end date of September 2019. In compliance with
the Request for Proposals and discussions with UNICEF, the evaluation should have focused on the first
2.5 years of the Project, more specifically on the period from the September 2014 to June 2017.
Since the contract was awarded in mid-August, 2017, the field work for quantitative and qualitative data
collection started on the 14th of September and lasted till the end of October (following finalization of the
methodology, sampling, questionnaires, etc.). Meanwhile, a number of significant developments occurred
during this period both with respect to Project management and implementation as well as Project-
induced achievements, including:
- On November 16, 2017 the Government session approved the proposed amendments to the Family
Code and related legislation and submitted those to the National Assembly of Armenia. Amendments
were adopted during the last session of the Parliament in December 2017. According to the proposed
amendments, starting from 2018, children in difficult life circumstances will be eligible to become the
beneficiaries of the foster care system. This development followed the approval of the Concept for
Foster Care in March 2016 allowing for different types of foster care, for which UNICEF and Project
partners actively advocated. The 2018 Child Protection Program has also included strong stipulations
for foster care and the state budget AMD 166,343.5 thousand is committed to fund 94 foster families,
which is around 4 time increase compared to the previous years (reaching a total of 119 foster families
starting 2018). These significant achievements are attributable both to the political will of the
Government, to Project interventions, consistent and forceful advocacy efforts of UNICEF Armenia
and Save the Children.
- With respect to the deinstitutionalization efforts as well there is considerable progress following the
Government decrees on transformation of three institutions (two special schools in Lori and one night
care institution in Syunik) and closure of one residential institution. The released State funds were
reallocated to family strengthening and alternative care, ultimately resulting in the return of more
children to families.
Specifically, the official decision on Kapan institution was passed in September 2017. It was signed in
November 2017 and became effective as of 16/11/2017, following the RA Decision N 1398-N on Renaming
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
6
the Kapan Child Care and Protection Boarding Institution SNCO into Syunik Marz Child and Family
Support Center Foundation, Approving of the Carter and Providing Property.
Draft Government decree regarding transformation of a night care institution and closure of an orphanage
in Lori region has been re-submitted to the Government for approval.
• Since September 2017, all schools in Lori and Tavush launched the full inclusion concept supported
by the teacher trainings, coaching, establishment of seven Psychological-pedagogical support centers,
four in Tavush and three in Lori.
However, in September 2017 USAID Armenia made a decision to terminate its agreement with UNICEF
(111-IO-14-00004 AID-111-IO-14-00004) thus suspending some of the Project activities. Despite the
termination, both UNICEF and USAID agreed to proceed with the evaluation based on the assumption
that its findings and lessons learnt could be useful for both organizations in their future efforts in support
of Government of Armenia’s reforms agenda in child welfare and inclusive education.
As a consequence of this situation, the assignment that was initially designed as a mid-term evaluation
evolved into more of a final review of the Project activities. However, there are certain factors and realities
to be kept in mind while reading this document:
• Some of the Project interventions were still on-going during the data collection in September- October
(depending on the contract/agreement expiry dates of specific implementing partners). As such, for
several key informants it was difficult to delineate the before June-September activities from the
ongoing ones, especially since the ground work for many of them has been done through Project
efforts. As noted above, the significant progress made with respect to the Family Code, Foster Care
and transformation of the institutions and return of significant number of children to the families was
to a great extent possible due to the Project efforts. While this report tries to maintain its focus on the
defined timeline for the evaluation, sometimes there are references to activities/events that happened
later on, since they constitute an integral part of the overall process and not mentioning those would
skew the actual achievements of the Project.
• The evaluation’s mid-term nature was somewhat affected by the overall situation since informants
were aware of the recent developments and oftentimes expressed opinions and ideas that are more
relevant to a final evaluation than to a mid-term one where the possible improvements and shifts
should be the focus.
• This said, since the Project was designed to support the Government’s recent reforms which were
taking shape and gaining momentum as the evaluation was being conducted, the evaluation team
believes that it still maintains its mid-term focus if not with respect to the Project itself, then with
respect to the reforms. All the key stakeholders involved in the Project (including UNICEF in
Armenia, USAID, implementing partners such as Save the Children, World Vision, Bridge of Hope,
FAR CSCF) are important players in advocating for these ongoing reforms and as such the evaluation
team hopes that the findings and recommendations from the assignment will be considered and
utilized to further this work.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
7
III. EXECUTIVE SUMMARY
Object and Purpose of Evaluation: UNICEF in Armenia’s mandate and operations in the country have
consistently emphasized child protection and inclusive education as priority areas. Realizing children’s
right to education alongside with inclusion of children with disabilities in mainstream education, as well
as support to ongoing child welfare reforms through deinstitutionalization and transformation of
residential care institutions, has been the focus of UNICEF Armenia’s 2012-2015 and 2016-2020 Country
Program Documents (CPD).
In early 2014, UNICEF Armenia in cooperation with World Vision (WV), Save the Children (SC) and
Bridge of Hope (BoH) submitted a project proposal to USAID to expand alternative care, family support,
and inclusive education services as a part of the child-care reforms. Following an extensive review, USAID
approved the Proposal in September 2014 as a 5-year agreement (09/09/2014 - 09/08/2019). This evaluation
focuses on the project “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as Part of Child Care Reform” (hereinafter the Project)
with its key components.
The main goal of the Mid-Term Evaluation of the “Toward Social Inclusion of Vulnerable Children:
Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child Care
Reform’’ project (the Project) is to assess the relevance, effectiveness, efficiency as well as impact and
sustainability of interventions with specific objectives defined in the ToR. The time frame covered by
evaluation is September 2014-June 2017 with geographical coverage focusing on Lori, Syunik and
Yerevan. As per the ToR, the primary users of the evaluation are expected to be UNICEF, USAID and
Government counterparts, implementing partners as well as other national and international partners.
Methodology: Based on the stakeholder mapping and the RFP requirements, a sampling and data
collection methodology was developed by the evaluation team and approved by UNICEF in Armenia.
Consequently, mixed (quantitative and qualitative) data collection methods were used by the evaluation
team for data collection from relevant groups:
• Expert interviews were used for data collection from national level key stakeholders (ministries, NIE,
etc.) as well as the Office of Human Rights Defender, legal and child protection experts, etc. In total,
25 experts were interviewed as part of this stakeholder group.
• In depth interviews (including group interviews) were used to collect data from Project
beneficiary/stakeholder groups identified through stakeholder mapping such as parents/family
members, school principals, staff of RPPSC, CFSC, regional authorities, service providers and others.
In total, in-depth interviews (including 3 small group interviews) were conducted with over 40 key
informants.
• Focus groups were conducted with teachers and parents trained under the inclusive education
component, trained foster parents, staff of institutions and services. 14 Focus groups were conducted
with a total of 104 participants.
• Desk review and observations were used as additional qualitative data collection methods.
• In compliance with the ToR for this assignment, trained school teachers under the Inclusive Education
component (implemented by the BoH) were the target respondents of the quantitative data collection.
As agreed with UNICEF during initial meetings, the evaluation prioritized Syunik for the quantitative
data collection given the lengthier experience (2 years) of the schools in this region of involvement in
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
8
inclusive education. Quantitative interviews were implemented with 312 teachers of Syunik Marz (see
Chapter II for details of the sampling approach and overall methodology and Annex 4 for survey
instrument and interview guides).
The evaluation identified some data collection limitations including non-involvement of primary
beneficiaries, i.e. children, extended timeline of the evaluation since some key results occurred beyond
June 2017, different intensity and sequencing of interventions in Lori and Syunik, and others.
The evaluation findings are presented using the evaluation criteria, i.e. relevance, effectiveness, efficiency,
sustainability and impact. As prescribed by the ToR, “Given that this is a mid-term evaluation, two of the
evaluation criteria this project will be evaluated against, namely sustainability and impact, are to be
considered to the extent possible”. This issue was discussed later with UNICEF partners as well to
reconfirm the position that sustainability and impact will be addressed only to the extent possible.
RELEVANCE
General Findings
Finding 1: Due to UNICEF’s and partners’ continuous efforts and advocacy, policy makers and decision
makers in Armenia have realized the importance and urgency to act upon their commitments on
CRC/CRPD and initiate reforms that emphasize family environment and inclusiveness in education, as
well as minimize reliance on institutions. As a result, the number of institutionalized children has
decreased significantly. However, overall acceptance and understanding of the reforms varies from high
to limited among the different stakeholder groups. In the past 25-years UNICEF in Armenia’s efforts were
heavily focused on advocating for and supporting child protection and inclusive education reforms and
policies introduced and implemented by the Government of Armenia. UNICEF also supported the
establishment of the Integrated Social Services system as an institutional mechanism to address child
protection through the social system. UNICEF’s consistent engagement with the MoLSA, MoES and other
partners resulted in increased acceptance of the need to transform institutions and set-up alternative
services, building up a momentum for the reforms in child welfare and inclusive education. The Project
itself almost identically mirrors the reforms agenda and provides support to its key directions to sustain
these achievements. However, the challenges that persist with respect to both reorganization of the
institutions, introduction of inclusive education, setting up new social services, at times unpreparedness
of biological families to deal with returning children, create some uncertainties about the timing and
sequencing of the reforms both among stakeholders and beneficiaries.
Child Welfare Reforms
Finding 2. Different Project stakeholders (including policy makers, institution staff and families) confirm
that due to UNICEF’s leadership and consistent efforts, there is overwhelming agreement about the family
being the most favorable environment to ensure a child’s best interest and wellbeing. However, since the
launch of relevant reforms (deinstitutionalization) new and flexible solutions were required from UNICEF
and partners to address the situation with the lack of a functioning system (in terms of alternative social
services, acceptable quality of life at home, social conditions, etc.). As discussed throughout this report,
the momentum for the deinstitutionalization reforms has been built over many years due to UNICEF’s
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
9
and partners’ advocacy resulting in the Government declaring deinstitutionalization a priority in 2011,
and the consequent developments. The MoLSA’s commitment to the RA Prime Minister to close all night
boarding institutions by the end of 2017 is an important evidence as to how these efforts have paid off. As
a result of these efforts the number of institutionalized children has decreased drastically in the country.
However, despite the perceived need for deinstitutionalization reforms, there is also agreement that it is
not an overnight process and needs serious preparation. Reform stakeholders believe that
deinstitutionalization will not succeed without establishing the community based alternative services in
advance, as well as providing individual solutions for each family reunited with the children returning
home. Still, this has been the case with the ongoing reforms since Child and Family support centers and
other alternative care services are either not established or not fully functional. To address the situation,
UNICEF has brought on board other partners involved in service provision to vulnerable children and
their families (Society for Orphaned Armenian Relief (SOAR) and SOS Children’s villages).
Foster Care
Finding 3: The institute of foster family is new to Armenia and is somewhat divergent with its traditional
culture. However, given the ongoing deinstitutionalization reforms, it has been considered as one of the
most appropriate alternatives for children placement and promoted as such. The Project interventions,
conducted mainly by Save the Children Armenia and FAR CSCF, have been timely and relevant with
respect to advancing foster families as a type of an alternative care in support of the deinstitutionalization
reforms. They focused on legal changes needed for introducing this form of alternative care alongside to
capacity building of foster families. However, some stereotypes about placement with foster families still
persist since despite efforts to ensure acceptance of foster care on the local level, yet there was no national
level public awareness campaign to advocate for this form of care with the larger public.
Inclusive Education
Finding 4: Introduction of the concept of inclusive education (IE) through different pilot projects and
work of both civil society organizations and policy makers has prepared the Armenian society for
embracing inclusive education in mainstream public schools. Perceptions and assessment of the surveyed
teachers on the country’s general readiness for inclusive education is rather favorable for rollout.
However, there are also those who believe that the pace of the reforms is too fast and there is more work
needed before comprehensive rollout of the system. Over the years critical steps were taken toward
integration of inclusive education into mainstream schools both on policy and practical implementation
levels. These resulted in a gradual shift of perceptions and opinions as well as general acceptance of the
notion of inclusive education. However, persisting challenges relating to the capacity of specialists,
adequacy of school infrastructure and other concerns give way to certain questions about readiness to fully
embrace IE at different levels and by different groups. In this respect, the work done under the Project is
undoubtedly contributing to changing the attitude of educators, parents, children and others toward
children with special educational needs and their place in mainstream education.
EFECTIVENESS
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
10
General Findings
Finding 5: The child welfare /inclusive education reforms are complex multifaceted undertakings. As such,
the Project structure reflects the reforms complexity through its multiple components, implementing
partners and coordination with line ministries and agencies. While the Project is intended to support the
Government’s reforms agenda, the reforms pace and project pace are at times in conflict with each other
resulting in delayed Project activities. The Project donor and implementers, the Armenian officials, the
child care institutions’ and public-school staff, and other stakeholders, all agree that the reforms, while
necessary, are complex and sensitive in addition to being resource intensive and requiring long-term
efforts. With the understanding of these complexities and to effectively implement the Project, UNICEF
in Armenia has partnered with the most competent organizations that have long-standing experience of
supporting interventions in these areas. UNICEF, as the lead organization undertook coordination efforts,
communication with government partners and USAID, policy and advocacy work, as well as some direct
interventions in such areas of its expertise as mobilization and training of ISS/case managers, local social
planning and others. While generally several results were achieved in the past 2, 5 years, there have been
certain bottlenecks that either slowed down or diverted the Project activities, including the change in the
Government resulting in delays in approving critical legislation. USAID as a donor as well as implementing
partners, while acknowledging UNICEF’s leading and mobilizing role in the Project, point out to certain
coordination challenges, both with the Government and the IPs. Other stakeholders emphasize UNICEF’s
critical guidance both for the reforms and the project, despite the existing challenges.
Child Welfare Reforms
Finding 6. While UNICEF and partners have developed and proposed to MoLSA a model for establishing
Child and Family Support Centers and supported reorganization of institutions and their transformation
into CFSCs, several factors contributed to certain inconsistencies and drawbacks in the process, mostly
due to delays and shifts in the reform priorities led by the Government. Noubarashen, Kapan and
Vanadzor have followed different timelines and patterns of transformation that affected institution/center
work and beneficiary experiences with placement/follow up and service provision. According to UNICEF
and World Vision reports, four (4) institutions have been targeted for reorganization under the Project,
including Yerevan Child Care and Protections Institute #1 (Korea Dzor), Yerevan Child Care and
Protections Institute #2 (Noubarashen), Kapan Child Care and Protection Boarding Institution and
Vanadzor Child Care and Protection Boarding Institution. However, the situation with each of these
institutions is different in terms of their legal status, Project interventions, their transformation into a
Child and Family Support Center (CFSC), provision of service, etc. As much as possible, the Project has
attempted to smooth the process by maintaining constant engagement with the Government through
supporting the legal foundations needed for moving forward (decrees, charters, etc), proposing a model
for the CFSCs, training case managers and other social service providers as well as mobilizing partner
NGOs to address the needs of children reunified with their families. By June 2017/ there has been mixed
success, including the establishment of the first CFSC and reunification of children from Noubarashen
with their families (over 50), assessment of hundreds of children and families, provision of support to
reunited families in form of material assistance and specific services. In terms of institutional mechanisms
for the functioning of the social service system, capacity building (on-the-job trainings/coaching and
mentoring) of case managers and other service providers, in-depth assessment of children (more than 400
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
11
children) from institutions and development of individual care plans (more than 300 children), as well as
development inter-agency protocols of cooperation in the area of child protection have been important
steps forward.
As to the drawbacks, absence of community based family support services in the communities of origin
where children return, has remained the paramount issue to be addressed. While a model for CFSCs has
taken shape, its full application is still to be ensured.
Foster Care
Finding 7. The Project succeeded in conducting a two-phase training for over 74 foster parents. While
improved parenting skills is noted as a training benefit by several of them, delayed placement of children
is creating disappointment among the potential foster families. 74 potential and experienced foster parents
(instead of initially planned 120) were trained by FAR’s Child Support Center Foundation (CSCF) in 5
regions, i.e. Lori, Syunik, Armavir and Ararat Marzes and Yerevan. CSCF also developed 12 training
modules and methodologies such as Guidelines for Monitoring Foster Families and Foster Care Program,
Principles and Standards for Foster Care, Public Awareness Manual. Most training participants (both FG
participants and in-depth interview informants) speak very highly about the qualifications of trainers as
well as usefulness of the trainings resulting in acquiring some critical knowledge and skills. However, for
over a year after the trainings were conducted none of the trained parents had an opportunity to practice
their foster care skills. Many of them expressed concerns that they might soon forget the knowledge and
skills acquired.
Inclusive Education
Finding 8. Implementation wise IE-related interventions have been one of the most effective components
of the Project with trainings conducted among 1670 teachers in Syunik Marz (102 schools) and 3760 in
Lori (162 schools) respectively, and successful involvement of key players and stakeholders (NIE,
coordination with the Republican Centre of Pedagogical Psychological Centre, and other involved
stakeholders including parents). As a result, several key results were fully achieved, including the
establishment of 4 Pedagogical-Psychological Support Centres (PPSC), capacity building of around 200
specialists in Syunik and Lori RPPSCs, training of over 5000 teaching staff with 70% of surveyed teachers
indicating the usefulness of trainings.
Finding 9. Knowledge of the basics of inclusive education has increased significantly as a result of teaching
staff trainings resulting in some level of practical application of learnt concepts and approaches as well as
overall acceptance of the new system (as perceived by surveyed teachers). Assessing the level of knowledge
increase among school teachers from target regions was first done by BoH and partners through pre-post
knowledge tests, according to which overall growth of knowledge among 1670 participants in Syunik is
53%. A knowledge index developed based on the current survey results (+1 to -1 interval) ranks the Syunik
teachers’ knowledge at 0.87. 84, 6 % of respondents also answered positively to the question about
knowledge application. Thus, Project interventions have increased the knowledge on IE among the
teachers in target regions resulting in certain improvements in application as well as improved attitudes
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
12
toward children with SEN. However, there are still several issues and challenges to be overcome to reach
full-fledged integration of IE in mainstream education. They include but are not limited to teachers’ ability
to assess educational needs of children with SEN, as well as develop and implement individual educational
plans and others.
EFFICIENCY
General Findings
Finding 10: From the Project perspective, the combination of USAID’s G2G support coupled with the
implementation of the soft interventions under the Project (with each partner contributing to within the
overall budget) was initially perceived as a good tool to increase Project efficiency and resource use, but
it did not work as planned. As of June 2017, around 50% of the overall Project budget has been expended
with mixed outcomes that vary from Project component to component. From state budget perspective,
unwillingness of the two line ministries to join certain aspects of the reforms agenda under one umbrella,
has raised questions among different stakeholders about efficient use of funds and resources. Given the
close interconnections between the Project interventions and the two G2G agreements (between USAID
and MoLSA, USAID and MoES), the latter two include strong conditionality language in support of the
Project implementation. However, while this approach should have resulted in favorable consequences
for advancing the reforms supported by the Project, the real progress did not fit the initial plans due to
different reasons. Lengthy negotiations between the two Governments on the different aspects of G2G
funding had certain impact on the Project progress, especially since the Project partners were not involved
in these discussions despite the fact that their respective work was affected by those.
With respect to UNICEF and IP use of Project funding and resources, as of mid-July, over 50% of the
overall Project budget has been spent, with most of UNICEF and IP contributions committed. However,
most of the tangible accomplishments as of June 30, 2017 were attributable to BoH interventions that has
completed its work in two regions (Lori and Syunik) and has spent its budget of over 500,000 USD. Some
other significant accomplishments included training of over 250 case managers and social service providers
that exceeded the initial target of 145 as well as adoption of secondary legislation resulting from UNICEF
in Armenia’s advocacy efforts and laying foundations for future progress. In agreement with all partners,
the unspent portion of SC’s budget was frozen pending the submission of the Family Code to the
Parliament and subsequent activities, while WV also faced an unexpended budget balance (31% spent)
due to delays in its planned work (mainly related to renovation and purchase of equipment and furniture
for the newly established CFSCs).
Child Welfare Reforms
Finding 11. UNICEF and MoLSA have successfully joined efforts to mobilize all ongoing interventions
related to a child’s rights to protection and social systems reforms to support the Project activities and
interventions. UNICEF and MoLSA have consistently and successfully capitalized on other critical reforms
initiatives, namely introduction of ISS and case management to support the ongoing deinstitutionalization
and IE reforms. This approach has been especially effective in the current situation when the communities
of origin face a lack of services and social workers, and case managers have been closing that gap to the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
13
extent possible. Thus, over 100 case managers and 192 social service providers were trained under the
Project, both through in classroom and on job modalities. CSCF worked with 29 case managers in Lori and
Syunik (11 and 18 respectively, as well as additional 16 case managers from Kotayk and Yerevan) providing
on-the-job guidance and mentorship to conduct needs assessment and case analysis both for special schools
and child care institutions.
Finding 12. To a certain extent, both UNICEF, WV and CSCF are all involved in the deinstitutionalization
component of the Project. As a result, there is more expertise and targeted efforts invested on the one
hand, but somewhat confusing role distribution and coordination on the other. UNICEF’s role as the lead
agency responsible for the coordination of the Project implementation was defined in the Project Proposal
and further implementation agreements. However, in addition, UNICEF also undertook several other
responsibilities with respect to assisting MoLSA in developing the strategy on deinstitutionalization and
establishment of alternative services. Though according to the initial Project setup World Vision was
responsible for establishing alternative community based services (including the CFSC model, costing,
staff training, etc.), it eventually became involved in the assessments of children, support of families
reunited with children, and other activities. CSCF also has been involved in child assessments, and
supporting case managers’ work with families. Consequently, a number of functions were replicated by
these different organizations. However, the positive aspect of this situation is that all involved
organizations have vested interest in promoting child welfare reforms and supporting families and
alternative services as part of their mission. Despite the occasional ambiguity of roles and responsibilities,
when needed, they all mobilize available resources to serve the child’s best interest.
Foster Care
Finding 13: The progress and the future of the foster care component of the Project was challenged by the
delays of the approval of the Family Code that would enable children in difficult life circumstances to
benefit from foster care services and the fact that in 2017 no funds from the state budget were committed
to increase the number of foster families, thus challenging foster family motivation. Since 2008, the
Introduction of Foster Family concept has been supported by the RA State Budget, which annually
committed funding for foster care of 25 children deprived of parental care. Currently, 25 children are
placed in 21 families. For 2017 as well, no additional funds were committed to increasing the number of
foster families, which affected Project efficiency since the foster families trainings happened in 2017
without any immediate placement of children in families. However, some positive developments are
expected in this regard in 2018. The 2018 Child Rights Protection Program approved by the Government
on September 29, 2017, plans to increase the number of children in foster families to 120. The State Budget
will fund these items of the Program by committing 166,343.5 thousand drams. It should be noted that
the approval of the Family Code, as well as the allocation of increased State funds for foster care for 2018,
are essential progress in this respect since the amendments will also allow qualifying children in difficult
life circumstances for foster care, thus increasing the potential number of children to be placed.
Inclusive Education
Finding 14: Despite significant knowledge about inclusive education, somewhat favorable perception by
schools and the entire community, lack of special conditions and resources needed has been voiced as a
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
14
critical challenge to implementing inclusive education. Additionally, the changes in the financing of the
system also affect its efficiency and effectiveness. Ensuring full efficiency of the IE component is
dependent on a good balance between soft interventions and infrastructure improvements, donor
assistance and state commitment. All these factors are also critical for further sustainability of IE
integration into the mainstream education system. While the soft interventions under the Project have
proceeded intensively and effectively, the physical improvements of the schools involved in the IE system
envisioned by the G2G agreement have not followed. In addition to physical resources, lack of specialized
professionals has also been noted as one of the impediments to implementing inclusive education. And
finally, throughout interviews and discussions concerns were raised with respect to financing mechanism
of the inclusive education system, which has affected both the motivation for its implementation and the
involvement of the specialists.
SUSTAINABIILTY
Finding 15: Sustainability of the Project interventions in support of reforms efforts depends on a number
of overarching and interlinked factors ranging from availability of alternative care mechanisms to capacity
of the professionals involved in delivery of services to public awareness and others. While the Project has
successfully established preconditions for some, others still need to be initiated and implemented.
Sustainability guarantees of these multiple and complex interventions are equally diverse and manifold.
They include laying down the foundations of the prerequisite legal framework, availability of financial
resources, availability of professional human resources to implement the reforms at local levels, such as
social services, schools, PPSCs, others, availability of alternative care services, specific guides and
regulations about service operations, public awareness and acceptance of reforms, etc. While some of these
factors are addressed by the Project, there are two aspects of the reforms that, as agreed by interviewed
experts, need significant improvement if sustainability of reforms is to be ensured. These are 1) national
level public awareness building about the reforms and 2) working with parents (both of children to be
reunited with their families and larger parents groups affected by the introduction of inclusive education
into mainstream public schools).
CROSS CUTTING ISSUES
Finding 16. While the Project has emphasized the gender aspect of its interventions requesting data
disaggregation by gender and general sensitivity to gender issues, there seems to be no visible differences
in the way deinstitutionalization or inclusive education is affecting boys and girls. However, with respect
to the specialists involved in both reforms, female professionals have a prevailing presence.
Conclusions: Over the last two decades UNICEF and development partners have successfully advocated
with the Government of Armenia for the child’s right to live in family environment and have access to
quality inclusive education as being in the child’s best interest. By 2014, when the Project was being
designed, the momentum for the reforms in both directions was irreversibly established and the Project
intended to support and sustain that momentum. The Project almost identically mirrored the key aspects
of these two reforms directions by focusing on improving the pre-requisite legal framework, supporting
transformation of existing institutions and establishment of alternative services, engaging into capacity
building of professionals and specialists required for implementation of the reforms and other relevant
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
15
efforts. Each of the Project components was tasked to a competent organization with reputation and
experience of working in the relevant areas.
By June 2017, the Project has reached some key results, including reorganization of a number of
institutions (both child care and right protection and special schools, establishment of one CFSC and 4
RPPSCs, needs assessment and placement of hundreds of children), training of thousands of teaching staff
under the IE component, as well as other professionals, development and adoption/approval of legal acts,
etc. However, there have also been pushbacks, such as the absence of a unified institution transformation
model, no visible progress with setting up of alternative community based services, delays with the
amendments to the Family Code and subsequent challenges with the placement of the children in foster
care, and others. While some of these interruptions are attributable to the Government’s uneven actions
with respect to their reforms agenda, others have been due to internal Project dynamics.
Generally, given the complex and sensitive nature of the reforms supported by the Project, more time is
needed for effective implementation of some of its aspects. However, even at this point there is confidence
about the impact of the Project interventions, especially with respect to legal foundations and strategic
priorities of the reforms. As discussed throughout the report, adoption of several decrees, concept papers,
State programs coupled with informal statements have cemented the commitment of the RA Government
to the ongoing reforms. Specific steps have been taken to close or downsize institutions (both child care
and special schools) and reunite thousands of children with their families. As such, there is confidence
among stakeholders that deinstitutionalization and inclusive education will be further implemented in
Armenia, in the best interest of the children affected. As overwhelmingly agreed by the evaluation
informants, UNICEF Armenia’s role in this process has been instrumental.
Recommendations included in this part of the report build up on the Findings and Conclusions and are
also grouped by relevant evaluation criteria.
Relevance
1. USAID as the Project donor and UNICEF as the lead Project partner should continue their efforts to
bring to fruition the Government’s reforms with respect to child welfare and inclusive education.
While currently the Project serves as the main locomotive in support of the reforms agenda, and as
such is important, UNICEF in Armenia should also prioritize its policy expertise in terms of child
rights protection and use all available channels to build on the current momentum for these reforms,
which was to a large degree achieved due to UNICEF in Armenia’s continuous efforts over the years.
Effectiveness and Efficiency
2. If possible, as a tactics moving forward, USAID should consider separating the two reform directions
(child welfare and inclusive education) into two standalone projects while UNICEF’s relevant sections
could take over responsibility for each direction (i.e. Education section taking over Inclusive
Education given its longstanding experience with work in that area). While the current structure has
been designed to ensure harmonization of the two reforms and their key directions, coordination
across Government partners, implementing partners, and different stakeholders has been challenging.
If the separation is still under the auspices of UNICEF, the harmonization could be furthered.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
16
3. In the future, while undertaking funding of a Project in support of ongoing reforms, USAID should
consider a more structured approach, especially when tying soft interventions to G2G support. One
alterative is starting small and laying the foundations of a bigger project, then expanding it and adding
budget support components once confident about the progress. In case UNICEF undertakes the role
of a Project implementer, it should prioritize pilot projects that showcase the benefits of the specific
approaches of reforms implementation and serve as a model to be replicated by the Government. To
illustrate, with the current child welfare and deinstitutionalization reforms, there seems to be a need
for a reorganization and transformation model and focusing on establishing and making functional
one CFSC could have served as such a pilot. Another alternative is piloting all possible child and family
support services in one region, including CFSCs, RPPSs, strengthening case managers, social workers
and other social services to demonstrate the interconnections between these key institutions and
services and demonstrate how effectively their cooperation could be shaped in support of children in
difficult life circumstances.
4. All Project stakeholders, including USAID as donor, MoLSA and MoES as well as UNICEF should be
aware of any duplications between the roles and responsibilities of different implementing partners
and should define the tasks of partners based on their core competencies. More involvement from the
implementing partners in policy related communication with their respective Government
stakeholders could also result in improved Project coordination.
Sustainability
5. As noted throughout the report, sustainability of the Project efforts is very much dependent on
availability of resources and such factors as availability of community based family support services,
professionally trained specialists, guidelines and manuals in support to service provision and others.
This said, all key players in the reforms, including the Government of Armenia, USAID, UNICEF and
other implementing partners should use available resources and capabilities to ensure these critical
gaps are closed. While each player may utilize different models, methodologies and mechanisms to
ensure further sustainability of the reforms, coordination of efforts will remain a critical priority.
6. In the future, USAID, UNICEF and IPs should consider including a national level public awareness
campaign in similar interventions that support large scale and sensitive reforms. Given UNICEF’s
competencies with respect to designing and implementing such campaigns, it can become one of the
key intervention areas led and managed by UNICEF.
7. Both the Government of Armenia and UNICEF should continue efforts directed to capacity building
of professionals needed for the successful implementation of this reforms both through the Project
and any other channels/initiatives available. In this respect, it is important to identify the key partner
organizations that should be strengthened to support child welfare reforms. While the IE and BoH
experience provides a good example of how the NIE and the Republican PPC became key Project
partners involved in both teacher trainings and capacity building of RPPSC specialists, the role should
be taken over by a State agency and not fall to an NGO active in that area. With respect to the child
welfare reforms, MoLSA should not take over this function given its role as a policy setter but the
National Institute of Labour and Social Research could be considered as a possible candidate for this
role.
8. MoLSA and UNICEF should continue using case managers in support of the reforms implementation
and enhancing their professional experience and resources to work further with children reunified
with their families and other children in need of support. Community social workers should also be
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
17
brought in to support both child welfare and inclusive education reforms implementation. While a
case manager’s involvement in the ongoing Project and reforms is commendable and necessary, they
are not present in the communities on daily basis and are mostly involved in referrals. Similarly,
regional/Marz administration level departments and their staff are focused on decisions in regards to
placement, documentation processing and other related activities. This creates a gap of specialists on
the community level, and social workers (as evidenced by WV model) could become that important
link in the reforms implementation.
9. Working with parents and children should be an important part of reforms that are touching their
lives and their future. As such, the Project interventions would have benefited from more intensive
engagement with both groups. While this can be partially achieved through a public awareness
campaign, targeted activities are also needed.
And finally, the Government of Armenia (both MoLSA and MoES) should undertake a more robust
leading and coordinating role with respect to all aspects of the reforms, ranging from legislative changes
to capacity building to coordination between different stakeholders including government partners,
donors and implementers. Timely and relevant delivery of all the inputs required for moving forward the
reforms process should be ensured by relevant stakeholders, including adoption of decrees and
commitment of state budget resources (e.g. for foster care or IE) and be responsible and accountable of the
overall reforms outcomes, both positive and negative.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
18
CHAPTER I. INTRODUCTION TO EVALUATION
- Object of the Evaluation, Purpose, Objectives and Scope
UNICEF in Armenia’s mandate and operations in the country have consistently emphasized child
protection and inclusive education as priority areas1. Realizing a child’s right to education alongside with
inclusion of children with disabilities in mainstream education, as well as support to ongoing child welfare
reforms through deinstitutionalization and transformation of residential care institutions has been the
focus of UNICEF in Armenia’s 2012-2015 and 2016-2020 Country Programme Documents (CPD). Thus,
throughout the last decade UNICEF has worked with the Government of Armenia to promote policy
reforms and their implementation in these areas. UNICEF has also advocated for increased cooperation
with key development partners to build up consensus around primary strategic reforms in the social sector
that either directly or indirectly support reforms and improved policies in these two areas. Consequently,
several partnerships have emerged, including:
- with the World Bank Around Integrated Social Services (ISS) reforms and capacity building of case
managers that are at the forefront of the deinstitutionalization reforms as the support base for the
children leaving the institutions;
- with the EU in promoting the social protection of families affected by migration, including children,
and finally
- with USAID in supporting deinstitutionalization and inclusive education.
In early 2014, UNICEF Armenia in cooperation with World Vision (WV), Save the Children (SC) and
Bridge of Hope (BoH) submitted a project proposal to USAID to expand alternative care, family support,
and inclusive education services as a part of child-care reforms. It should be noted that while this proposal
was not in response to USAID Armenia’s targeted solicitation, its Country Development Cooperation
Strategy FY 2013-2017 includes a focus on supporting policies that promote social inclusion and provide
services for vulnerable children and the disabled. With the heads of the two agencies (UNICEF in Armenia
and USAID Armenia) understanding the importance of joint efforts in support of Government of
Armenia’s child welfare reforms and inclusive education, a special USAID contracting instrument for
Public International Organizations (PIO) was utilized for approving the project. In compliance with PIO
procedure, UNICEF submitted a concept paper, which then underwent a rigorous review for over a year
before the project was approved. Consequently, in September 2014 the 5-year agreement (09/09/2014 -
09/08/2019) was signed between USAID Armenia and UNICEF in Armenia for implementing the “Toward
Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive
Education Services as Part of Child Care Reform” Project. UNICEF was selected as the lead project partner,
while other implementing partners were involved in the areas of their strong expertise.
Thus, the object of this evaluation is the Project with its key components including child welfare reforms
focused on deinstitutionalization/transformation of institutions and introduction of alternative care
services including foster family system, rollout of inclusive education, as well as any relevant cross-cutting
interventions such as improved case management, setting of legal frameworks and other.
1 http://www.unicef.am/en/articles/aboutunicef
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
19
The main goal of the Mid-Term Evaluation of the Toward Social Inclusion of Vulnerable Children:
Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child Care
Reform’’ project (the Project) is to assess the relevance, effectiveness, efficiency as well as impact and
sustainability of interventions with specific objectives defined in the ToR2:
- Family and community based alternative services and their role in prevention of institutionalization;
- Institutional mechanisms for the functioning of social service system;
- The process of strengthening the family substitution service system with the diversification of types
of foster care;
- The process of strengthening the inclusive education system to provide quality learning for all
children;
- Legal and regulatory framework developed to support child-care reforms;
- Institutional cooperation and coordination between implementing partners/key actors (both
government and donor community) in responding to vulnerable families and children in the frames
of the project (including financial);
- Coherence and coordination between the objectives/components of the project to meet the
overarching goal.
Gender and human rights implications of the Project and the reforms supported through it are also an
important aspect of the evaluation.
As to the scope of the assignment, the ToR defined its timeline, geographic coverage and the data
collection methods to be used the following should be noted:
• This is a mid-term evaluation, which covers the period from September 2014 to June 2017. Since the
project duration is 5 years and the closing date was initially set as September 2019, the mid-term
review of the Project intended to “assess the progress in meeting the Project goals and
outcomes….provide early lessons learnt and recommendations as well as identify discrepancies
between expected results and actual achievements” As noted above, the evaluation is intended for the
use of UNICEF, USAID, the Government of Armenia, Implementing Partners and other international
and national partners to inform “further action in the areas of child protection within the context of
deinstitutionalization”;
• Yerevan, Syunik and Lori as the target regions benefiting from Project activities, and;
• Use of mixed quantitative and qualitative methods to collect data from primary informants,
stakeholders and beneficiaries.
- Background of the Intervention and Context
In 1992, the Republic of Armenia joined the United Nations Child Rights Convention (UNCRC), which
entered into force in the RA on 22nd July 1993. Following the ratification of the Convention, Armenia
2 UNICEF Armenia’s Request for Proposals to conduct Mid-term Evaluation of the ‘‘Toward Social Inclusion of
Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child
Care Reform’’ project (LRFP-2017-9132946) (ToR).
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
20
initiated legislative changes (including the passage of the Law on the Rights of the Child in 1996) that set
the foundation for furthering a child rights protection system and the child care/welfare reforms.
Another important milestone was the ratification of CRPD by the GoA in 2010 that accelerated child care
system reforms and movement towards inclusive education systems. Several CRPD articles (Article 8,
Article 23) emphasize such requirements as human rights of children with disabilities, respect for the
Home and the Family, preventing concealment, abandonment, neglect and segregation of children with
disabilities, provision of early and comprehensive information, services and support to children with
disabilities and their families. Article 24 is specifically focused on education and the importance of IE.
UNICEF has consistently been at the forefront of these developments, advocating for the rights of children
with disabilities and their inclusion through guiding the evolution of IE practices and policies in Armenia,
followed by technical support for improvement of IE legal framework and strengthening the IE system.
In addition, UNICEF played a critical role in modelling 31 inclusive schools serving over 500 children
with special educational needs and supporting the Government-established center for early identification
of special educational needs and referral of CwD to mainstream education. Such a center was established
by the Government as part of the current action plan for implementation of a fully inclusive education
system by 2025. The plan is to establish 23 Psychological-pedagogical support centres through
transformation of special schools.
From there on the country took several steps toward fostering international best practice with respect to
a child’s right to protection and upholding the child’s best interest, including children with disabilities.
Several legal acts and programs were adopted such as the Amendments to the Law on General Education
in 2014, the 2004-2015 National Programme for the Protection of the Rights of the Child, the subsequent
2013-2016 and 2017-2021 programs. However, despite these important steps, there still exist critical gaps
in terms of enforcing the legislation and implementing an effective child rights protection system in
Armenia. Evidence to such state of affairs is provided by UNICEF commissioned 2008 analysis3 to assess
compliance of national legislation with the Convention, a 2015 Save the Children publication on Child
Rights Situation Analysis: Armenia4, statements and reports by child protection NGOs, and other sources.
While the problems are related to different aspects of children’s rights, overreliance on institutions and a
weak child right protection system has been among identified priorities.
In the past 25 years that UNICEF has been operating in Armenia, it has advocated consistently and
unwaveringly with the Government of Armenia in support of child rights protection and the child’s best
interest. Children’s rights to survival, growth, education and development, family life, protection and
participation, with focus on greater social inclusion, especially of the most vulnerable and excluded
children have been and remain the overarching goal of UNICEF Armenia’s Country Programmes both in
2012-2015 and 2016-2020. UNICEF’s focus on ensuring the child’s best interests are adhered to throughout
different policy reforms and is guided by concerns about child poverty, number of children placed in
different institutions, the weak child rights’ protection and monitoring systems and other important
considerations. More specifically:
3 Legislative reform related to the Convention on the rights of the Child. National case studies: Armenia, Barbados
and Ghana, 2008
4 https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
21
• The child poverty rate in Armenia is higher than the general poverty (29.4) according to Social
Snapshot and Poverty in Armenia, 2017 report5, ranging from 37, 2 % in the age group 15-17 (highest)
to 32.8% in the age group 10-14 (lowest). Generally, larger households with more children are exposed
to a higher poverty risk oftentimes resulting in children’s institutionalization due to social issues.
• Back in 2014, when the Project interventions were being designed, 14 child care and protection
institutions were functioning under the Ministry of Labour and Social Affairs (MoLSA), including 8
boarding schools (for children aged 6-18), 3 specialized care institutions and 3 general care institutions
serving a total of 1431 children, of which 54% were male and 46% female. With respect to
communities of origin the child distribution was the following: 25. 2 % are from Yerevan, 17.7 % from
Lori and 16.4 % from Shirak. While Lori and Shirak had higher poverty rates, there are other regions
with high poverty such as Gegharkunik or Aragatsotn from where the number of institutionalized
children was not that high. As noted in the Project Proposal document, “only approximately 10% of
children in orphanages are orphans. The majority of children in state and non-state institutions are
children from socially vulnerable families who have at least one parent alive…Yet after the placement
in an institution the majority of children spend their childhood there due to the lack of support
provided to families and children by social protection services”.
• Before the amendment of the Law on General Education in 2014, over 2,400 children with disabilities
had been accommodated by 23 special schools that were segregated educational institutions operating
under the Ministry of Education and Science (MoES). However, as noted in the above cited document,
“what brings many of the children to these schools is not only their special education needs but the
hard-socio-economic conditions or various dysfunctions of their families, since schools provide food,
care and shelter.” By 2015 Armenia had 130 inclusive schools providing education to more than 2500
children with special needs. While this has been a significant accomplishment for the country that
has a tradition of reliance on special schools and institutions, it covered less than 10% of Armenia’s
schools and was mostly implemented in urban areas. Children with disabilities are among the most
marginalized groups facing multiple deprivations. As the 2011-2012 national statistics indicate 71 per
cent do not attend preschool, 18 per cent do not attend school, 12 per cent attend special schools, 13
per cent live in institutions (special schools and orphanages), 34 per cent do not attend any community
events and 73 per cent do not use rehabilitation services.
• The current three-tier Child Protection System functioning in Armenia consists of the National
Committee on Protection of Children - a policy setting body at the highest level of the system. The
second tier is comprised of the Family, Women and Child Protection Units (FWCPU) within Marz
administrations and Yerevan municipality that are tasked with implementation and monitoring of the
policies adopted by the government of Armenia with respect to protection of children, including the
protection of children in difficult life circumstances. These units are also responsible for designing and
implementing targeted interventions for the vulnerable beneficiaries. The third tier of this system is
at the community level, where the Guardianship and Trusteeship Committees (GTC) are vested with
authority to make decisions about the care and placement of the children that have found themselves
in difficult life circumstances. There is consensus among several stakeholders of the system that the
only more or less functioning part of the system is its second tier.
5 http://armstat.am/file/article/poverty_2016a_2.pdf
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
22
• From 2010 onwards, UNICEF in Armenia has provided technical assistance to the Government of
Armenia for the identification, introduction and implementation of a model of social services that
would effectively address the needs of vulnerable families in Armenia, and ensure the best interest of
children. The Integrated Social Services model (based on Council of Europe’s Guidelines on
Transformative Social Protection) was both recommended and supported by UNICEF Armenia as one
that has proved to have a positive impact on vulnerable families as an effective system for tackling
poverty, preventing children from appearing in difficult circumstances, being separated from their
families or placed in institutions.
To summarize, while Armenia has made significant progress on child protection in recent years, the
country still faces numerous challenges in meeting its obligations under the Convention on the Rights of
the Child. These include establishing a comprehensive and coherent child care and protection system,
decreasing the number of children in institutions, expanding community based services (including
alternative family-like solutions) to preventing the inflow of children to these institutions
To address the situation, most of UNICEF Armenia’s program components and related interventions are
aimed at ensuring that different child rights are adhered to and fully exercised through its support to
relevant policies and legislation, data collection, specific projects targeting child health, education,
disability, and social protection. With respect to social protection of children UNICEF Armenia has
undertaken several efforts, including:
• Provision of technical assistance for the revised Family Code addressing foster care (2005-2010 CPD);
• In partnership with the European Union, UNICEF Armenia contributed to technical assistance to the
development of policies, plans and normative frameworks for the child protection implementation
and oversight bodies (2005-2010 CPD);
• UNICEF Armenia launched the "Every Child needs a Family" campaign in late 2011. While it is
primarily focused on advocacy for children with disabilities (who are more likely to appear in
institutions), the campaign highlighted at the same time, the issue of deinstitutionalization and the
need build up alternative child care services (2010-2015 CPD).
UNICEF Armenia supports the deinstitutionalization reforms to close residential care institutions for
children and promote community-based professional services as an alternative. In this respect, UNICEF
Armenia’s efforts focused on contributing to review and develop a number of key documents regulating
child protection issues including the National Child Protection Plan of Action 2012-2016; draft Law on
Domestic Violence (December 2017); Government’s decree on Adoption; a policy document on
deinstitutionalization and others.
UNICEF Armenia’s most significant accomplishment with respect to promoting child protection and child
care reforms have been the reduction of institutionalized children from 12,000 to 3,700 in the last two
decades as a result of its consistent efforts to promote and support relevant reforms by the government. It
has been part of these efforts that UNICEF successfully lobbied to secure USAID’s funding to support
transitional costs of the deinstitutionalization reforms of the Government of Armenia and promote
inclusive education in Armenia through the Toward Social Inclusion of Vulnerable Children: Expanding
Alternative Care, Family Support, and Inclusive Education Services as part of Child-Care Reform” Project
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
23
(the Project). It is expected that such comprehensive assistance could play an essential role in reducing
the number of children in residential care institutions even more and will establish alternative family and
community based services for vulnerable children and their families.
- Logic of Interventions
The Project’s logic of intervention is fully described in the Monitoring and Evaluation Plan including the
Indicator Tracking tables. Five (5) expected outcomes with their relevant output results are reflecting the
key Project intervention areas and contributing to the Project goal, i.e. “Children Realize their right to
live in a family environment and access community based social and educational services”. Table 1 below
summarizes key expected results under the Project.
Table 1: Key Expected Results under the Project
Level Expected Result
Goal Children Realize their right to live in a family environment and access community based social and
educational services
Outcome 1 Institutional mechanisms for the functioning of social service system are set up
Outcome 2 Alternative community-based family support services are accessible to vulnerable children and
families
Outcome 3 Family substitution service system is strengthened with the diversification of types of foster care,
establishment of monitoring and evaluation system
Outcome 4 Inclusive Education System is strengthened and expanded to provide quality learning for ALL
children in targeted regions
Outcome 5 An enabling legal and regulatory framework is established to support Childcare Reforms
While Outcomes 2-4 reflect specific targeted intervention areas under the Project that are implemented
by UNICEF and its partners, Outcomes 1 and 5 cut across the Project and are relevant for all its aspects.
As noted in the M&E Plan, “Lower level indicators (outputs) and their targets will mainly be drawn from
program joint work plans developed with Programme partners. Higher level indicators (outcome) and
their targets will be directly linked to the analysis to be conducted during the Programme implementation
to estimate the impacts of the interventions”.6 In addition to the Project-specific indicators that are
designed to track Project progress, the M&E plan includes a set of USAID’s standard indicators for unified
reporting purposes.
According to the Project Theory of Change the inputs and outputs of the interventions that contribute to
its outcome and impact level results are to be achieved through both mid-term and long-term system
changes. More specifically, it is expected that the strengthening of inclusive education system, family
6 Monitoring and Evaluation Plan: Revision1, June 2016
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
24
substitution services alongside with accessible alternative community based family services will contribute
to realizing children’s rights to live in a family environment and access community-based social and
educational services. And important aspect of this results chain is an enabling legal and regulatory
framework to support child-care reforms and improved institutional mechanisms for the functioning of
social service system.
While the Government is one of the key stakeholders for achieving such system changes through
legislative reforms, introduction of policies, setting up institutional frameworks and services, the local
communities, civil society and the public at large have an important role to play in ensuring all children’s
rights to live in a family environment, to access health, social and educational services and expanded basic
and alternative care services for most vulnerable children.
To achieve its goals and objectives the Project works through a number of inter-connected components
including:
1. Institutional strengthening of the Social System that targets empowerment of case managers in the
target regions to perform social needs assessments and follow-up the cases of children and their
families with the involvement of appropriate agents. In addition, establishing institutional cooperation
between services, formalizing information exchange and referral procedures related to vulnerable
children is emphasized;
2. Establishment of Family Support Services through region-based approaches to ensure necessary
alternative services are established in the target communities for children, in parallel to the process of
deinstitutionalization from residential care facilities;
3. A comprehensive foster care mechanism with diverse foster care types and robust monitoring
mechanism is available for vulnerable children;
4. Strengthening of the Inclusive Education System through establishing PPSCs in target regions,
training teachers and principals of mainstream school. Adjusting mainstream schools, etc.;
5. The effective implementation of all these interventions entails establishing an enabling legal and
regulatory framework in support of child-care reforms and inclusive education, which cuts across
these four components.
Geographically, the Project interventions are concentrated in Syunik and Lori regions of Armenia as well
as capital Yerevan. Key groups to directly benefit from the Project include:
- Children living in different types of residential care institutions and their families (6,000 persons
including 1,800 children (50% of children living in institutions)) who will be reunited with their
families;
- Schools’ professional personnel to be trained on inclusive education (625 schools, approx. 25,600
professional personnel);
- Children in target communities that will have access to newly established community services
(estimated 20% of total child population in the target community with access to services);
- Foster families (in all of Armenia, but preferably from the community or region) that will host 80
children from transformed residential institutions;
- 45 trained case managers and 100 trained social workers.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
25
Indirect beneficiaries of the Project include children and families from nearby communities who can
benefit from the new/expanded services (30,000 persons), schools that will receive support from the newly
established Pedagogical support centers (350 schools, approx. 14,000 professional personnel),
communities, regional and local authorities.
It should also be noted that according to Project design and timeline, interventions in the three target
areas have not been occurring simultaneously, across all components. To illustrate, formally Noubarashen
reorganization happened first, while in Kapan the works proceeded prior to Government decree (passed
in November 2017). With respect to Inclusive Education, trainings happened in Syunik almost two years
prior to Lori. Some of this sequencing was preplanned, while in some cases several factors as well as the
context contributed to the situation (see discussion across the report).
Throughout this report, achievement of planned results under the Project is discussed with respect to its
individual components and the Project generally. However, it should be noted, that given the timeframe
of the evaluation (2.5 years from the start of the Project) and the fact that over 1.5 years was committed
to preparatory work, there is not much progress on the outcome level results to report on at this stage. As
the discussion of the individual Project components indicates, some of the output level indicators have
not yet been achieved as well due to a number of reasons that are discussed throughout the Findings part
of the report.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
26
CHAPTER II. EVALUATION METHODOLOGY
In response to the goals and objectives of the assignment as well as the scope defined by the ToR, the
evaluation team proposed and implemented an evaluation methodology comprising of the following
components:
• Stakeholder Mapping and Approach to Designing the Evaluation Sample
• Data Collection Methods, including:
- Desk Review
- Qualitative Methods
- Quantitative Methods
• Data Analysis and Reporting
In compliance with UNICEF in Armenia’s requirements included in the ToR, mixed methods were used
for data collection purposes to ensure “complementary strengths and non-overlapping weaknesses… The
analysis will be built on triangulating information collected from different stakeholders and through
different methods…”7 As detailed below, in agreement with UNICEF, quantitative data collection targeted
teachers involved in IE training in Syunik region under the Project. As such, the IE related findings of the
report triangulate data from the survey, in-depth interviews, FGs as well as a desk review. For all other
components, the analyses build on the in-depth interviews, FGs as well as any quantitative or qualitative
information derived from the desk review.
Since understanding project and evaluation stakeholders is a critical part of developing an evaluation
methodology for elaborating an evaluation sample and choosing the most effective data collection methods
with respect to each group, identification and mapping of key stakeholders was conducted. The exercise
derived information from the evaluation ToR, desk review of materials provided by UNICEF, as well as
initial meeting with UNICEF and Implementing Partners. Stakeholder mapping was conducted with the
following considerations in mind:
• While the overarching goal of the Programme is to ensure that children “realize their rights to live in
a family environment and access community based social and educational services”8, it consists of a
number of components some of which are cross-cutting, while others can be also viewed
independently;
• Those aspects of the Project that relate to advancing the Child Care reforms agenda, establishing an
enabling legal and regulatory framework, institutional mechanisms and local social planning cut across
all the Project components and require commitment from the Armenian Government and major
partners including the Ministry of Labour and Social Affairs, Ministry of Education and Science,
National Institute of Education, Marz administration as well as USAID, UNICEF and other
international players;
7 ToR
8 Project Indicator Definition Table
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
27
• Each of the key focus areas of the Project such as the alternative community-based family support
services, family substitution service system and inclusive education have a specific target group as
beneficiaries and stakeholders.
With these considerations in mind key stakeholder groups and sub-groups were identified and are
summarized in Table 2 below.
Table 2: Stakeholder Mapping
Project beneficiaries
• Children in difficult life circumstances living in different types of
residential care and educational institutions
• Children in target regions/communities benefiting from newly
established services
• Families/parents of these children including foster families
• Case managers and social service providers (police, health, education,
regional and local self-governmental bodies, etc.) whose capacity is
enhanced to work with the above groups
• School professional personnel whose capacity is enhanced to work with
the above groups
National level key stakeholders • National Assembly
• Government (Prime Minister’s Office) of Armenia
• Ministry of Labour and Social Affairs (MoLSA)
• Ministry of Education and Science (MoES)
• Ministry of Territorial Administration and Development (MoTAD)
• National Institute of Education (NIE) and trainers involved
• National Institute of Labour and Social Research
• NORK Informational Center
Regional/community level key
stakeholders
• Regional administration officials involved in child care reforms (i.e.
heads and staff of relevant administration units such as Marz
departments on Family, Women and Child Protection (FWCPU), Social
Security and Health, Education, Science and Sport departments),
Guardianship and Trusteeship Commissions, local self-governmental
bodies, i.e. community leaders
• NIE regional structures in Lori and Syunik
• Institutions and newly established RPPSCs, FCSCs, residential care
facilities, special schools
• Mainstream schools involved in inclusive education reforms
Other Stakeholders • USAID, UNICEF and other international organizations such as EU,
UNDP or World Bank
• Project Implementing partners
• Office of Human Rights Defender, NGOs and experts (including legal)
involved in child protection in Armenia
Based on the above stakeholder mapping and the RFP requirements, a sampling and data collection
methodology was developed. Extensive discussions between the evaluation team and UNICEF in
Armenia’s team resulted in the following key agreements about data collection from each specific group:
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
28
• Children that are the primary beneficiaries of the child welfare and inclusive education reforms
and Project initiatives were not targeted for the purposes of this particular assignment, since it
focuses primarily on the processes and mid-term accomplishments rather than the impact of the
Project.
• Quantitative data collection followed the requirements set in the ToR and targeted teachers
involved in Bridge of Hope trainings on inclusive education with a focus on Syunik region where
the intervention has taken place over the last two years.
• Different qualitative data collection methods (detailed below) were used for collecting data from
all the other stakeholder groups, including teachers.
- Qualitative Data Collection: Approach and Implementation
Qualitative data collection: the following methods and related sampling approach were utilized for the
purposes of this evaluation:
• Expert interviews: Since expert interviews are used to gain insights from a person's special knowledge
and experiences which result from the actions, responsibilities, obligations of the specific functional
status, this method was used for data collection from national level key stakeholders (ministries, NIE,
etc.), as well as some of the stakeholders identified under “Other Stakeholders” category, i.e. Office of
Human Rights Defender, legal and child protection experts, etc. Information saturation approach was
used by the evaluation team, i.e. interviews were conducted as long as new information was being
received. In addition to the initially identified list of experts, snowballing approach was also used for
engaging additional experts referred to by those already interviewed (which resulted in dropping off
some experts from the initial list). In total, 25 experts were interviewed from this stakeholder group,
which also includes 5 UNICEF staff members (interviewed as a group). Annex 3. List of Respondents
by Data Collection Method includes the list of the experts involved in the evaluation (expert interview
is specified as the method of data collection).
In depth interviews (including group interviews) with key informants: For the purpose of this evaluation
in-depth interviews were used to collect data from Project beneficiary/stakeholder groups identified
through a stakeholder mapping (see Table 1). They were parents/family members, school principals, staff
of RPPSC, CFSC, regional authorities, service providers and others. The sampling took into account such
factors as geography (Lori, Syunik and Yerevan), and types of informant groups related to each aspect of
the project as well as cross cutting groups (e.g. parents, staff of institutions from which children were
released (care institution, special school, residential, etc.), different service providers etc.). Gender was a
key consideration, and as much as possible, a balance was ensured among the key informants. To ensure
validity of the information received from each sub-category under this stakeholder group, a minimum of
3 interviews was conducted with the representatives of each particular sub-group, including
families/parents, case managers and social service providers, school principals, staff on institutions and
newly formed services. Table 3 below summarizes the interviews conducted for this particular data
collection method.
Table 3: Distribution of In-Depth Interviews by Marz and Informant Groups
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
29
Parent/Family member
(possible gender balance
ensured)
School Principals Former/current
Institutions and
alternative
services
Marz level authorities and
social service providers
Yerevan • 2 interviews with
biological family
members/parents of a
former Noubarashen
child resident
2 school principals 1 staff members
of Yerevan night
care institution
N2 in
Noubarashen and
Day care center
1 interview with a social
service provider
2 interviews with case
managers
Lori • 1 interview with
experienced foster
family members
• 4 trained foster parents
2 school principals
(urban/rural)
2 interviews with
Vanadzor Child
Care and
Protection
Boarding
Institution staff
2 interviews with
Vanadzor, Spitak,
RPPSC
representatives
1 interview with heads of
Marz administration
departments
4 interviews with case
managers and social
service providers (e.g.
police)
Syunik • 3 interviews with
biological family
members/parents Kapan
Child Care and
Protection Boarding
Institution
• 1 interview with a
family member/mother
of a former special
school child
2 school principals
(urban/rural)
1 interview with
Kapan Child Care
and Protection
Boarding
Institution staff
3 interviews with
Kapan, Sisian and
Goris RPPSC
representatives
2 interviews with
representatives of
special schools for
Children with
Intellectual
Disabilities in
Kapan, Goris
3 interviews with heads of
Marz administration
departments
4interviews with a case
manager and a social
service provider
1 SG interview with
Kapan FWCPU
Thus, in total, in-depth interviews (including more than 4 small group interviews) were conducted with
over 40 key informants.
• Focus groups: For the purposes of this evaluation FGs were conducted with trained teachers,
parents (trained under the inclusive education component), trained foster parents, staff of
institutions and services. The participants of Focus Group discussions were selected from the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
30
databases received from the implementing partner (BoH and CSCF) and finalized at the place
depending on availability of requested participants. However, an effort was made to ensure a
balanced representation of different subject teachers as much as possible. Homogeneity was
ensured by limiting the FGs to teachers only without principals or other management staff present.
The same approach was taken in case of RPPSC staff focus groups. Most of FG participants for IE
were female reflecting the general statistics in Armenia with over 90% of teachers being women.
FGs were conducted by a moderator and an assistant, the former moderating the discussion and
the latter taking notes and providing technical support.
Table 4 summarizes Focus Group distribution by Marz and type of key informant
Table 4: Focus Group Distribution by Marz and Type of Key Informant
Foster families Teachers Parents Institution staff
Yerevan 1 FG (6 participants
from Yerevan,
Armavir and Ararat)
1 FG with
Noubarashen staff (6
participants)
Lori 3 FG (in Vanadzor
school # 16, Lernapat
school, Spitak # 5,
total 26 participants)
2 FG (in Vanadzor
school #16, Lernapat
school, total 13
parents)
1 FG with Spitak
RPPSC (6
participants)
Syunik 2 FG (Goris school #
3, Angeghakot
school, total 20
participants)
2 FG (Kapan school #
5, Goris school # 3,
total 14 parents)
1 FG with Kapan
RPPSC (6
participants)
1 FG Kapan Child
Care and Protection
Boarding Institution
(7 participants)
In total, 14 Focus groups were conducted reaching out to 104 participants.
Some additional qualitative data collection methods that were employed by the evaluation team are
described below:
• Observations were used for better understanding, as well as comparing and contrasting the work
of institutions and newly provided services focusing on the following two aspects: physical
infrastructure and its relevance, as well as service provided/methods used. Observations were
conducted in the following locations:
1. Noubarashen Family and Child Support Center
2. Goris RPPSC and Vanadzor former special school
3. 2 public schools in Lori and 2 in Syunik Marzes
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
31
And finally, a Desk Review of all the documents provided by UNICEF and IPs (such as quarterly reports,
ITTs, legal documents, assessments and research, training modules, pre-and post-knowledge tests) was
conducted as an integral part of the overall data collection (see Annex 2. List of Desk Review and
Background Materials).
Prior to the fieldwork the qualitative data collection guides were developed, pre-tested and shared with
UNICEF in Armenia (see Annex 4. Interview Guides and Survey Instrument).
- Quantitative Data Collection: Approach and Implementation
In compliance with the ToR (Annex 1) for this assignment, trained school teachers under the Inclusive
Education component (implemented by the BoH) were the target respondents of quantitative data
collection. As agreed with UNICEF during initial meetings, the evaluation prioritized Syunik for the
quantitative data collection given the lengthier experience (2 years) of the schools in this region of
involvement in inclusive education.
Quantitative interviews were implemented with 312 teachers of Syunik Marz.
The initial sample size was determined with 5% error margin and 95% confidence interval. The following
sample size formula for infinite population is used to arrive at a representative number of respondents
when population estimate is known (Godden, 2004):
𝑛 =𝑍2×𝑝(1 − 𝑝)
𝑀2
Where:
n = Sample Size for infinite population
Z = Z value (1.96 for 95% confidence level)
P = population proportion (expressed as decimal) (0.5 (50%)
M = Margin of Error at 5% (0.05)
Since the general sample (i.e. trained teachers in Syunik) is 1670, the calculated sample size is 312.
A multi-stage randomized sample was then constructed on the following levels:
1. The number of the urban and rural schools to be involved in the sampling was decided based on the
rural-urban distribution of teachers in Syunik (54% and 46% respectively), 168 of surveyed teachers
represented rural areas, while 144 came from urban areas.
2. Taking into consideration that the number of rural teachers is at least 8, and the number of the urban
teachers is at least twice more, respectively 8 and 16 interviews were conducted in each rural and
urban school selected into the sample, i.e. 21 rural and 9 urban schools that were selected. These 21
rural and 9 urban schools were allocated according to the service areas of RPPSCs in Sisian, Goris and
Kapan (which also serves the Meghri region) based on the proportion of schools in each area. The
rural/urban distribution in each area is presented in Table 5.
Table 5: Proportion of Schools by RPPSC Service Area
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
32
Proportion in percentage Interviews conducted Total
Urban Rural Urban Rural
Sisian 19% 31% 32 64 96
Goris 22% 22% 32 36 68
Kapan 59% 47% 80 68 148
Total 100% 100% 144 168 312
3. Site visits indicated that while in all urban schools of these areas there exist assessed children, the
situation is different in some rural communities with some schools still not having assessed the
children. Thus, the sample construction took into consideration this factor as well.
Table 6: Number of Schools Included in the Sample Based on Availability of Assessed Children from
Rural and Urban Schools
4. In the next stage of sampling, the schools were randomly selected from the alphabetical list and
the sample was constructed based on the above discussed factors. Table 7 below provides the list
of sampled schools.
Table 7: List of Schools in the Final Sample
Area Urban Rural
Inclusive Non-inclusive
Kapan Kapan #3 Davit Bek Artsvanik
9 21 schools were planned initially. However, 22 were selected into the sample to have substitution options in case
if the foreseen number of interviews would not be possible to conduct in the initial 21 schools.
Distribution of Schools per Assessment Factor
Urban Rural
There are assessed
children
There are no
assessed children
Sisian 2 4 4
Goris 2 2 2
Kapan 5 5 5
Total 9 11 11
229
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
33
Kapan #5 Dzagedzor Achanan
Kapan #6 Geghanush Norashenik
Kapan #10 Lernadzor Okhtar
Kapan #13 Syunik Verin Khotanan
Goris Goris #2 Karashen Khndzoresk
Goris #5 Tegh #1 Kornidzor
Sisian Sisian #1 Angeghakot Aghitu
Sisian #3 Ishkhanasar Bnunis
Shaghat Uyts
Shaki Tolors
Prior to fielding the quantitative survey, the questionnaire was pre-tested among 15 respondents (5% of
the planned interviews). For the purpose of effective resource management, the interviews were
conducted at 2 schools of Yerevan. The basis for the selection of schools for the pilot was the level of
successful implementation of inclusive education at schools assessed by study experts. Per expert estimates
three types of schools were classified: very successful (e.g. Mainstream School N 100), medium successful
(e.g. Mainstream School N 160) and less successful (e.g. Mainstream school N 125). Interviews were
conducted in school # 100 and # 160 with 8 and 7 interviews respectively.
The instrument for standardized face-to-face interviews included questions under the following main
domains:
1. Inclusive Education
• General awareness/knowledge
• Perceptions
• Practice
2. Assessment of training relevance and effectiveness
3. Availability of Supportive Resources/RPPSCs
4. Respondent profile (socio-demographic and other information)
Majority of questions included in the questionnaire are close-ended with a number of open-ended
questions that were coded based on verbatim quotations once the database was created and processed.
The questionnaire is attached to this document as Annex 4.
With respect to data analysis and reporting, VISTAA strictly followed the requirements of the RFP for
using the evaluation criteria and evaluation questions as an analysis framework, guidance for providing
good recommendations in compliance with findings, disaggregation of data, ethics and gender
considerations, and others.
Data analysis: Transcribing, tabulating and systemizing data gathered through both quantitative and
qualitative methods and cross checking with findings of the desk review and observation records was used
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
34
as a basis for analyzing and presenting the data. All data received through desk review, FGs, interviews
and surveys were analyzed and systemized to be used for the findings and recommendations of the report.
VISTAA used the SPSS statistical package/software for quantitative data entry and analyses.
Quality control of the collected data was performed by the Field Supervisor & Quality Coordinator under
the overall supervision and guidance of the Quantitative Expert responsible for all the stages and aspects
of the quantitative survey. All standardized questionnaires were checked by Field Supervisor & Quality
coordinator, by reviewing the questionnaires, checking and cleaning the database.
As to the control of qualitative data, data collection followed evaluation questions per evaluation criteria
to ensure consistency of the information received. IE related findings of the report triangulate data from
survey, in-depth interviews, FGs as well as desk review. For all other components, the analyses builds on
the in-depth interviews, FGs as well as any quantitative or qualitative information derived from the desk
review.
- Major Limitations
Some data limitations were identified both in the evaluation design and implementation phase. One of the
primary limitations was related to the challenges of reaching out to the children that are the primary
beneficiaries of project interventions. While this was not a requirement of the ToR, the evaluation team
approached the issue from different aspects, trying to assess to what extent this gap will affect the
evaluation rigor. However, as noted above, consultations with UNICEF in Armenia resulted in the mutual
agreement that at this stage the focus of the assignment was mostly on the implementation processes and
mid-term results, rather than the impact of the interventions, which should be visible primarily on
children. In addition, interviews and discussions (as well as the survey instrument) with parents/families,
institution and service staff, school teachers and principals always included inquiries about children’s
opinions and perceptions on how the reforms in question affect their wellbeing (while this was giving
secondary data on children’s opinions and perceptions, it was still considered to be valid information).
Some other limitations included:
- The timeline of the evaluation: several of the key results expected to have been accomplished as part
of the Project progress occurred beyond the timeline following the groundwork laid by UNICEF in
Armenia and its partners. However, since the field work happened in the fall of 2017, it was at times
difficult keeping the informants focused on the evaluation timeframe. To illustrate, some institutions
reorganized after June (such as special schools into RPPSCs), but since interviews were happening in
October they touched upon issues relevant to the reorganization.
- Data Collection Limitations: In very limited cases (11 of 312), surveys were not done through face to
face interviews, but distributed to teachers, explaining the instrument logic and asking them to fill in.
This was due to time limitations and short working days of teachers, who wanted to leave earlier or
in a few cases when the interviewer team was delayed due to road conditions or other unexpected
issues. A lower number than expected participants were present at parent groups FGs, than initially
planned because of teachers’ involvement as parents. While the teachers could also be interviewed in
their roles as parents, VISTAA decided removing them not to influence the expression of opinions by
other parents.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
35
- The different intensity and sequencing of interventions in Lori and Syunik also caused some
divergence in data, given the different regional experiences and cycles with respect to the Project
implementation. This was also one of the reasons why Lori was not included in the teacher survey,
assuming that the BoH trainings happened there just recently, leaving no time for the schools to
implement inclusive education integration into mainstream public schools and reflect on perceptions,
applications and other aspects included in the survey instrument.
- Generally, given the sensitivity of the reforms directions supported by the Project
(deinstitutionalization and inclusive education), their implication not only by the directly affected
groups but the larger society and public, there was significant diversity of opinions and perceptions
on many of the evaluation questions making generalizations somewhat challenging.
- Ethical Considerations, Human Rights and Gender
Ethical dimensions were taken into consideration by the evaluation team, particularly when working with
beneficiary families (vulnerable families with children). Prior to conducting interviews and focus group
discussions, the evaluators consulted them and ensured their informed consent. The evaluation team
protected the anonymity and confidentiality of the individuals and observed ethical guidelines as set out
by United Nations (UN) Evaluation Group in its standards and norms10.These standards are based on the
following principles taken into due consideration:
• Hold in confidence all information given to the evaluation team by the Client;
• Explain the rationale, objectives and the methodology used to institutions and /or individuals
taking part in the evaluation and receive informed consent from all participants;
• Listen and value all participants’ views during interviews and focus groups;
• Respect the values of the Client for whom the evaluation is undertaken;
• Reflect on own bias due to previous experience, education background, gender, ethnicity and
other factors;
• Respect the anonymity of individuals when analyzing data;
• Cross-check and triangulate all results, wherever feasible;
• Feedback on the research results to the participating institutions or individuals, where the Client
approves;
• Prohibit use of the research data for other studies without Client and participants’ permission.
Human rights based approaches to development and gender mainstreaming are considered as foundational
normative principles for UNICEF. Therefore, the extent to which these principles have been adhered to
by UNICEF Armenia in implementing the current Project, they were integrated across the evaluation
10 http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4
http://www.uneval.org/normsandstandards/index.jsp
http://www.unEvaluation.org/ethicalguidelines
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
36
focusing on human rights based approach and gender considerations that produce equitable impacts on
children’s lives and realization of their rights. A special finding is included in this report on the gender
aspects of Project interventions as required by the ToR. The team has made an effort to review the impact
of the on-going child welfare and IE reforms on both girls and boys (to the extent available that data
allowed for) as well as assessing gender balance of different stakeholders involved in the reforms and
affecting it, such as teachers, foster parents, etc.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
37
CHAPTER III. FINDINGS
Evaluation findings are presented by evaluation criteria i.e. relevance, effectiveness, efficiency,
sustainability and impact. As prescribed by the ToR “Given that this is a mid-term evaluation, two of the
evaluation criteria this project will be evaluated against, namely sustainability and impact, are to be
considered to the extent possible”. This issue was discussed later with UNICEF partners as well to
reconfirm the position that sustainability and impact will be addressed only to the extent possible. Some
of the findings discussed under each evaluation criteria are general in their nature, pertaining to the overall
Project and the reforms agenda supported through the Project. Others are specific to each individual
component of the Project, such as Inclusive Education, deinstitutionalization and transformation of
institutions, foster care, institutional mechanisms for functioning of social service systems, etc. One cross-
cutting finding discusses gender aspects and the implications on the Project.
RELEVANCE
General Findings
Finding 1: Due to UNICEF’s and partners’ continuous efforts and advocacy, policy makers and decision
makers in Armenia have realized the importance and urgency to act upon their commitments on
CRC/CRPD and initiate reforms that emphasize family environment and inclusiveness in education, as
well as minimize reliance on institutions. As a result, the number of institutionalized children has
decreased significantly. However, overall acceptance and understanding of the reforms varies from high
to limited among the different stakeholder groups.
As detailed in the background section of this report, in the past 25 years UNICEF in Armenia, with
coordinated efforts of all partners involved in this programme, including World Vision, Save the Children,
Bridge of Hope and FAR CSCF, were heavily focused on advocating for and supporting child protection
and inclusive education reforms and policies introduced and implemented by the Government of
Armenia. This consistent engagement with the MoLSA, MoES and other partners resulted in “increased
acceptance of the need to transform institutions and set-up of alternative services”11, including12:
• In 2005 the amendments to the Family Code were approved incorporating provisions on foster care;
• September 2010 ratification of the Convention on the Rights of Persons with Disabilities (CRPD) by
the Government of Armenia;
• In 2011 the Government declared de-institutionalization a priority;
• In 2012 the Minister of Education and Science, and the Minister of Labour and Social Issues signed a
joint decree to start the process of transformation of the institutions under the MoES. International
Classification of Functioning (ICF), Disability and Health was introduced as a tool for assessing and
11 Project Proposal: Prepared by UNICEF Armenia in cooperation with WV, SC and BoH
11 UNICEF In Armenia’s CPDs, annual reports and Project quarterly reports are used as sources
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
38
certifying disability and special education needs thus shifting the model for understanding disability
and planning interventions from medical-based to social-based. In 2017 3 ICF codes were
contextualized and piloted in Armenia, then adopted by the Government;
• In 2013 both ministers requested support for the development of transformation plans and the revising
of funding plans for residential care institutions. The Prime Minister approved the transformation
plans for two residential care institutions, which were developed with technical support from UNICEF
-.
• In 2014 amendment to the Law on General Education introduced elements that are critical for the
further development of an IE system in Armenia. The Project supported drafting of a number of bi-
laws that contribute to the operationalization of those amendments;
• The new State Programme for Education Development 2030 also includes provisions on ensuring full
inclusiveness of the education system by 2025.
• From 2010 onward introduction of the Integrated Social Services system emphasizing reduction of
social needs and professional case management has been supported by UNICEF as an alternative to the
much criticized three-tier child protection system;
• The Government decision of 10.03.2016 approved the Concept for Foster Care of Children in Difficult
Life Circumstances allowing for different types of foster care. As part of the process, 12 legal acts and
proposed amendments to existing acts have been drafted and once adopted, will ensure that children
in difficult life circumstances benefit from foster care services alongside with children deprived of
parental care. It is a major result in the Government’s commitment to fund 120 more foster families
from the state budget as of 2018;
• The National Strategy for Child Rights Protection 2013- 2016 and 2017-2021 contain strong provisions
further consolidating commitments to transforming the child welfare system. In addition, the
Government Programme for 2017-2022 and Human Rights Action Plan for 2017-2022 prioritize de-
institutionalization reforms.
The 2013-2016 National Strategy for Child Rights Protection declared the child’s wellbeing in a family
environment and in the society a priority through reducing the number of children in child care and
protection institutions and preventing placement of more children into these institutions. The goal and
the priorities of the current (2017-2021) National Strategy include protection of interests of children in
difficult life circumstances, improving the system of child’s right protection and ensuring equitable,
inclusive and quality education. The actions envisioned by the Strategy include reorganization of large
institutions into social centers, improving the system of trusteeship and guardianship, enhanced foster
family care, establishment of daycare centers, i.e. steps that will allow keeping the child in the biological
family. Institutions are considered only as a last resort with an emphasis on the idea of small group homes
instead of large facilities when institutionalization, in exceptional cases, cannot be avoided.
All these developments allow believing that the momentum for the reforms were established by late 2014,
when the Project interventions started. The Project itself almost identically mirrors the reforms directions
and provides support to its key interventions to sustain these achievements.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
39
The Government of Armenia’s official position states the child’s right to live in a family environment
under parental care, and targets return of children currently placed in state care to their biological family
or to alternative care. Similarly, inclusive education has evolved into a mainstream policy direction
implemented across the country after the 2014 Amendment of the Law on General Education. In 2016,
the Government of RA approved the Action Plan and Timetable for Provision of Total Inclusive Education
System. Finally, the 2017 draft State Programme for Education Development 2030 has been developed
taking into consideration the United Nations’ Transforming our world: the 2030 Agenda for Sustainable
Development and Incheon Declaration on Education 2030. Ensuring inclusive and equitable quality
education and promote lifelong learning opportunities for all.
In addition to these formal undertakings, officials at both MoLSA and MoES have articulated their
commitment to the ongoing reforms and have consistently upheld their position both in the public and
during discussions with UNICEF and other donors. All these efforts have resulted in several initiatives
that allowed decreasing the number of institutionalized children in Armenia through different pilot
projects and roll out of the reforms, which are supported by this Project.
The Government’s commitments are supported, to a certain extent, by those stakeholders that are directly
involved in the implementation of the reforms, i.e. staff of reorganized institutions and public schools,
regional authorities, case managers and social workers. However, the challenges that persist with respect
to both reorganization of institutions, introduction of inclusive education, setting up new social services,
at times unpreparedness of biological families to deal with returning children, create uncertainties about
the timing and sequencing of the ongoing reforms. Opinions like “it is too early”, “communities are not
ready”, “services are lacking” have been voiced by different key informants. With respect to families who
are directly impacted by the reforms, reactions are twofold. While emotionally and morally their majority
seems happy to have the child back, social and financial implications of the move make them question
whether the child is benefitting from reunification with the family. Similarly, with respect to inclusive
education, families voice questions about the readiness of the mainstream school system and the larger
public to accept the integration of children with disabilities.
All institutions should close down, children should be sent back to families. Expert Interview, National level
official
The idea is very good, but the families are not ready for this, yet. They should be prepared both financially and
psychologically for this change- to be able to reunite the family. In-depth interview, Regional administration
I have been lucky to have those (services) several years when the state helped me out with the boarding
kindergarten and institution (for my child). What should I have done, if not for that opportunity? Who would
have taken care of my child while I was earning the living for us? In-depth interview, Parent
It is too early to speak about the results of the introduction of the inclusive education. We are reforming and
learning in the process. Expert Interview, Independent Expert.
Child Welfare Reforms
Finding 2. Different Project stakeholders (including policy makers, institution staff and families) confirm
that due to UNICEF’s leadership and consistent efforts, there is overwhelming agreement about the family
being the most favorable environment to ensure a child’s best interest and wellbeing. However, since the
launch of relevant reforms (deinstitutionalization) new and flexible solutions were required from UNICEF
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
40
and partners to address the situation with the lack of a functioning system (in terms of alternative social
services, acceptable quality of life at home, social conditions, etc.).
The common opinion voiced by different Project stakeholders (such as staff of the transformed and to-be
transformed institutions, the heads of the Family, Women and Children’s Right Protection Departments
of the regional administration, WV as an implementer, government officials and the Human Rights
Defender’s Office, and others) confirmed that the child welfare reforms were needed and timely. Over the
years, a consensus has been reached about the family being the best natural environment for the child to
grow and develop, thus serving the child’s best interest and wellbeing. As discussed throughout this report,
the momentum for deinstitutionalization reforms has been built over many years due to UNICEF’s and
the partners’ advocacy resulting in the Government declaring deinstitutionalization a priority in 2011,
and the consequent developments. The MoLSA’s commitment made to the RA Prime Minister to close all
night boarding institutions by the end of 2017 is another important evidence as to how these efforts have
paid off.
Overall, UNICEF and the partner’s efforts resulted in the reduction of institutionalized children from
12,000 to 3,700 in the last two decades due to consistent efforts to promote and support relevant reforms
by the Government. With respect to specific Project results, by June 2017, 30413 children from child care
institutions and special schools were reunited with their families, including 140 children from Lori (3
special schools and one night care institution), 103 in Syunik (3 special schools) and 61 from both
reorganized institutions in Yerevan (Noubarashen night care and 3 1 residential care) as a direct result of
the Project interventions. It should be noted that in the period between 2014-2017 when the Project was
operational, the Government of Armenia reported reunification of around 7,000 children from the
institutions downsized across the country, which was due to direct and/or indirect advocacy by UNICEF
and partners within the Project. Some quotes below provide evidence as to how positively some of the
key informants perceived family reunification.
Of course, it is not even questioned that the children from even the poorest families would prefer going to bed at
home, and waking up with their parents (and siblings), rather than in an institution… and we are talking about
the emotional bonds within the family, which clearly does not exist in any, even the best, institution. In-depth
interview, Regional administration
It is a joy to know that my kids are at home…it was a torture to have them sleeping not at home (crying)…I will do any job to
keep them alive. In-depth interview, Parent
However, despite the perceived need for deinstitutionalization reforms and several specific steps taken to
ensure reunification of children with their families, there is also agreement that it is not an overnight
process and it needs serious preparation. Reforms stakeholders (social workers, Marz level officials, staff
of institutions and CFSC) believe that it should not be done prior to providing the community based
alternative services, as well as the individual solutions for each family whose children return home, which
is said to oftentimes be the case with the ongoing reforms. Many informants also pointed to the reasons
which caused families to place their children in institutions in the first place (supported also by UNICEF
13 The numbers have been provided by UNICEF to USAID based on the information received from each relevant IPs
and Government of Armenia.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
41
reports and the in-depth assessment implemented in 2015 by the “Center for Educational Research and
Consulting” funded by World Vision). While social and poverty related considerations prevailed in many
cases, there are others that are equally valid, including disability of a parent, alienated parents, and other
factors that at times make the child’s return to the family traumatic and in conflict with the child’s best
interest.
I myself oppose the fact of drastic change in terms of immediately sending the children home without having all
the services established and fully operating, especially with the current state of communities. In depth interview,
Stakeholder/Institution
We deal with young human beings in the process of development and self-establishment. This returning back to
the family is going to be as hard as the bringing the children to the institution. Long-time work is needed with
the children and parents to help them to reunite. The children very often see their teachers as a bigger authority
than the parents, which creates problems at home both for the children and parents. FGD, Stakeholder/Institution
It should be noted that in compliance with the internationally recognized norms and requirements with
regards to children’s rights, the country level implementation of the reforms emphases two key pre-
conditions: 1) strengthening the family’s ability to nurture their children and provide appropriate care,
and 2) establish community based social centers (Child and Family Support Centers (CFSC)) to provide
the needed support and services to children leaving institutions to return to their families. The CFSCs and
other social services are expected to support the family in a number of child-related activities, such as
“organization and provision of services of socio-psychological, socio-pedagogical and socio-legal nature”14;
social skills consultations to the children and their families; “provision of vocational orientation, art-based
therapy and occupational therapy” 15; organization and provision of primary medical aid and healthcare
services, as well as other types of services. With respect to family enhancement, while financial and in-
kind support (25,000ADM during 6 months and food/other necessity packages) is considered an important
part of preparing them to receive the children back home, improving parenting skills is a viewed as an
essential aspect of CFSC functions alongside with other relevant interventions described above. In addition
to the social services, it is expected that several institutions and professionals should extend support to
these families ranging from case managers, to Marz administration, to schools, and others.
While certain elements of this approach are implemented in Armenia, numerous challenges and concerns
still exist. Key informants prioritized some of those, including:
• Transformation of institutions and creation of CFSCs is a complex process, including aspects related
to assessment of children and families, capacity building of staff working in the centers, physical
adaptation of the centers, and others that require significant time to accomplish and are still
underdeveloped.
14 Description of Provided Services within the framework of “Provision of Social Services to Children by Social Care
Day Centers” Program, RA MoLSA, October, 2016
19\ ibid
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
42
• Readiness of the community of origin is another issue. While the institutions are transformed into
CFSCs, the children return to their own communities where the services are not available, their
integration both into family, school and the general community is an issue. Foster care as an alternative
is not fully functional and is sometimes not well perceived by the biological parents.
• Preparing and monitoring families of children reunified with their families is another key issue.
Despite provision of a meagre allowance by the state and some in-kind contributions by the Project
and other NGOs, there remain several unaddressed challenges such as housing, the physical and
mental health of the parents, even the safety of the child retuning home, which could in some cases
be questionable.
• Lack of parenting skills among many of the parents in the families that are taking children back home.
As one of the informants noticed, “there grew up a generation of parents, who are lacking parenting
skills and responsibilities and (they) think … that institution replaces the parent… this is wrong
(thinking) from the child and the parent and the state”.
With this understanding of the situation, especially at the communities of the origin, UNICEF and partners
have undertaken certain steps to support children and their families after their reunification. Given the
lack of community based alternative services, UNICEF has reached out to other partners involved in
service provision to vulnerable children and their families, such as SOAR and SOS. Thus, by late 2015
USAID, UNICEF, and MOLSA had negotiated with SOAR, the biggest diaspora organization supporting
Armenia’s orphanages, to redirect the financial and in-kind assistance from residential care institutions to
children reunited with their families. In 2016, the Project reached out to SOS Children’s villages to engage
in the process as well through providing social assistance to vulnerable families, and sharing their model
of outreach services. Case managers trained under the Project also became instrumental in the follow up
with these children and their families. Thus, NGOs alongside with the case managers trained under the
Project have been critical in filling in the gaps in services that families faced following reunification.
In addition, UNICEF in Armenia has introduced the idea of Small Group Homes to the Government of
Armenia as a temporary solution in those few cases where placement with either biological or a foster
family is impossible. The Government seems inclined to go ahead with this model as a more efficient
community based alternative service for special groups of children whose placement in families is not an
option. This includes adolescents or children with behavioral issues, children with disabilities who either
have no parents and/or whose parents are unwilling to get the child back, as well as those at risk of
violence/abuse in the family settings). While there have been some concerns raised about how
introduction of this new model will affect the existing alternatives for care (e.g. foster care) or how
appropriate it is to trade one type of institution for the other, UNICEF in Armenia emphasizes the
transitional nature of the model and its use as a temporary solution and only in those cases when placement
with a family is out of the question.
… UNICEF does not view the idea of small homes as a permanent solution… the fact that the decision on
immediately turning Dilijan and Byureghavan into family type small homes was not sent to the GoA is the result
of UNICEF’s engagement and influence. Expert Interview, International organization
Notwithstanding the Government’s public explicit commitment to the reforms, there have also been
situations when steps were taken even with respect to the legal framework which did not move the process
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
43
forward. For example, the situation with the decree # 1112-N on the Order of Provision of Care and
Conditions to Children, Elderly People, and People with Disabilities (September 2015) provides a good
illustration as to how caution and vigilance should be constantly exercised to promote the Project goals.
The decree in question did not prioritize the right of the child to live in a family environment as a first
choice, generally the sequencing of alternative care options was not in compliance with the international
best practices. Under USAID’s leadership the Project partners (UNICEF and WV) provided a legal analysis,
helping the decision makers to see that the decree was headed in the wrong direction.
To summarize, at the current stage of the reforms, several critical steps have been undertaken by the
Government of Armenia to downsize institutions and reunify children with their families. All these efforts
have been due to UNICEF and development partners’ consistent efforts and advocacy to promote the
family environment as being in child’s best interest. However, there are still concerns about community
based family support services not being established at the same pace as the closing down of institutions.
Children returning to their families from the boarding institutions do not have a support base locally if
the community of origin is different from the currently reorganized boarding institution (which happened
in Yerevan, with Noubarashen CFSC) and if there is no other relevant NGO operating in the community.
In most of the cases children face problems re-integrating back into the family, school and the community
given the circumstances discussed above, while in the worst cases they are sent to another institution.
While the solutions proposed by the Project are addressing the short-term needs of children and families,
long-term institutional mechanisms are yet to be made operational.
A few illustrations that follow provide very specific examples of how these different factors influence the
placement and the future of a child both positively and not so.
Case 1. A single mother with a boy (from Noubarashen) works double shifts at the hospital to make a
decent living for both of them. Since the family is under extreme financial challenges, SOS Kids Villages
has renovated the roof of the one room apartment (in the dormitory).
Case 2. A family of 7 (grandmother, father, mother and 4 children) “residing” in Noubarashen community,
Yerevan, with literally no housing conditions, directly dangerous for children to stay in – both in terms
of health, hygiene and psychological health. While the family is happy to have the children back and is
working hard to provide for them, their problem is the housing-one room, utilities outside, no privacy,
etc.
Case 3. In Kapan, there is a family where the mother works at the hospital as a sanitary help with night
shifts. The mother used to have a second job at a restaurant. Though she has quit the job she is officially
employed and thus she receives no welfare. Her younger child is attending the institution and stays
overnight when the mother is working. The Project provided the girl with a bed, desk, chairs and
wardrobe to facilitate her return to the family.
Case 4. Two brothers from Noubarashen: the mother and the grandmother refused having the children
back home and also didn’t agree to placement in a foster family, hence the two boys were moved from
Noubarashen to Byureghavan boarding institution “till at least one of them grows and is capable for taking
care of both”.
Foster Care
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
44
Finding 3: The institute of foster family is new to Armenia and is somewhat divergent with its traditional
culture. However, given the ongoing deinstitutionalization reforms, it has been considered as one of the
most appropriate alternatives for children placement and promoted as such.
One of the primary guarantees of the ongoing deinstitutionalization reforms is further placement of
children from institutions in families as part of meeting the best interest of children and their right to live
in a family. Continuous efforts (discussed throughout this report) by UNICEF and development partners
have ensured that policy makers and child protection actors in Armenia agree that biological families are
prioritized on the continuum of care for children. However, in those instances when the placement with
the biological family is impossible, care alternatives include guardianship care (by the kin of the child) or
foster care “where children are placed by a competent authority for the purpose of alternative care in the
domestic environment of a family other than the children’s own family, and has been selected, qualified,
approved and supervised for providing such care”16. In the past decade, several key efforts were undertaken
to introduce foster care as an alternative for care/placement of children in difficult life circumstances or
children without parental care, including:
• In 2005 UNICEF contracted CSCF to pilot foster care project in Lori and Gegharkunik regions. As a
result, 25 foster families received 30 children. Based on the results/lessons learnt from this project
technical assistance was provided to MoLSA in revising the Family Code to address foster care and the
amendments were approved in 2005.
• Starting from 2008 the Government of Armenia took the responsibility for funding and expanding
foster care in the country with 25 families enrolled in the system hosting 60 children over the past
several years and with 17 children in foster care currently.
• As a result of UNICEF’s and Save the Children’s technical support and advocacy efforts, the Concept
for Foster Care of Children in Difficult Life Circumstances was approved by the Government decision
of 10.03. 2016, allowing for different types of foster care. As part of the process, 12 legal acts and
proposed amendments to existing acts have been drafted by SC, but have not been passed yet pending
the adoption of the amendments to the Family Code (adopted in December 2017), which currently
allows foster care for only those children that are without parental care.17 Passage of these documents
will allow children in difficult life circumstances to benefit from foster care services alongside with
children deprived of parental care.
• The protocol decision N18 of the RA Government’s May 12, 2016 session on approving the Concept
of Alternative Service to Children in Difficult Life Circumstances, which makes it possible for this
group to become beneficiaries of foster care as well.
• Both 2013-2016 and 2017-2021 National Strategies for Child Rights Protection and annual
Government Programmes on Child Rights Protection include language about Foster care as an
alternative for child placement. To illustrate, the 2018 draft Government Programme defines the need
for “preventing institutionalization of children through expanding the network of alternative services,
including foster care”.18
16 Project Proposal
17 As the Introductory Note to this report specifies, the package of legal documents related to the amendments of the
Family code was submitted to the National Assembly in November 2017 and adopted in December 2017.
18 Draft Government decision on approving 2018 Program and Action Plan on Child Rights Protection
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
45
Despite these positive developments establishing a legal framework and piloting out foster care, there still
exist several challenges with respect to attitudes toward foster care and support to its implementation both
by relevant regional authorities (Guardianship and Trusteeship Committees (GTCs, Regional Social
Services, etc.) and the society at large. During the FGs and in-depth interviews several participants (around
60%) pointed to instances when the authorities did not support their enrolment in foster care.
Who told you to apply? Why to hurry? The State is not funding this program, so why are you applying? FGD,
Foster Parent
Regional Marz administration departments showed some resistance to the program. The State was playing a double
role- explicitly speaking about being for it, but showing some resistance through Marz authorities. Expert
interview, Implementing Agency
In other cases, they face negative attitude inside the community,
You have a child, a good house, so what is the use of becoming a foster family? FGD, Foster Parent
In one of the rural communities they complained the foster parents are abusing the child. Our several monitoring
visits indicated that the child was merely helping the parents in agricultural works. The attitude toward foster
care was so bad that they wanted to see it fail. In-depth interview, Implementing Agency
However, there were also cases when FG participants and informants noted (around 40%) that they faced
no issues:
Honestly speaking, we had no problems. We went to marzpetaran (regional administration), they came to assess
our house. Someone came from municipality as well. We submitted our documents and in two months got the
registration. FGD, Foster Parent
So, it is pure luck… whoever gets a good official, the process goes much easier. FGD, Foster Parent
According to FDG participants and other key informants, some of these stereotypes persist despite efforts
to increase awareness and understanding of foster care at community level and among the professional
community, such as:
• informational meetings in 38 communities (instead of planned 18);
• 496 persons reached as a result of professional forums instead of planned 300;
• 1783 persons reached as a result of community meetings instead of planned 300;
• 132 persons reached through different other meetings organized by CSCF.
In addition to these Project activities, UNICEF and CSCF undertook some extra efforts in this direction
including developing and dissemination (at the above forums) Foster Care Q&A, posters, one-pager
promoting the concept of foster care and other materials. UNICEF also separately organized a series of TV
programs and awareness through social media reaching up to 4000 persons. However, it should be noted
that the national level public awareness campaign was not fully implemented conditioned by the delay of
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
46
the adoption of the new Family Code. There were some publications in media, but the film and the Public
Service Announcements that could capture a much larger audience was delayed due to prolonged
finalization of the developed scripts for one of the PSAs (by all partners involved) in accordance with the
advocacy standards to better reflect the messaging to potential foster families, as well as due to the
agreement among all partners to broadcast the visual materials after the adoption of the Family Code
(provisions on foster care).
To summarize, the Project interventions have been timely and relevant with respect to advancing foster
care as an alternative care in support of the deinstitutionalization reforms. However, alongside with legal
changes and capacity building of foster families (discussed below), working with the key stakeholders and
larger public to ensure acceptance of this form of care is an important aspect of the Project that should be
further enhanced.
Inclusive Education
Finding 4: Introduction of the concept of inclusive education (IE) through different pilot projects and
work of both civil society organizations and policy makers has prepared the Armenian society for
embracing inclusive education in mainstream public schools. Perceptions and assessment of the surveyed
teachers on the country’s general readiness for inclusive education is rather favorable for rollout.
However, there are also those who believe that the pace of the reforms is too fast and there is more work
needed before comprehensive rollout of the system.
In 2005, the Government of Armenia adopted the concept of Inclusive Education and the Law on
Education of Persons with Special Education Needs was passed, with Inclusive Education included in the
draft Education Development National Programme for 2008-2015. However, a 2009 external evaluation
of Inclusive Education Policies and Practices in Armenia, administered by UNICEF Armenia office,
pointed out that this framework allowed for a parallel system of education (regular and special). Based on
this evaluation recommendations and as a follow up to CRPD ratification by the Government of Armenia,
the Law on Education was amended in 2014, and a new State Programme for Education Development
2030 was drafted. The Law and the new state program aim to align the education system development
with the newly adopted Sustainable Development Goals (SDGs), the Incheon Declaration, and the
adoption of the Bologna process, all of which occurred in 2015. Thus, both the Law and the Project
consider “the formation of an educational system which enables every citizen in all stages of their life to
receive quality education corresponding to their requirements, intellectual competencies and aspirations,
meets the demands of the economy and society, and fully serves the national interests”, and “The Republic
of Armenia shall declare the universal inclusive education as a guarantee of ensuring the right of each
child to education. The policy of inclusive education shall be aimed at ensuring the accessibility,
opportunity of equal participation in and quality of education of each child"19.
These achievements in the area of inclusive education have been possible due to efforts of UNICEF,
Mission East OSI, USAID and other development partners that consistently advocated for the introduction
of inclusive education in the country by exposing decision makers to international best practice,
emphasizing the human right based approach, mobilizing NGOs around the issue, organizing study tours
19 Article 4, RA Law on Education, supplemented by HO-200-N of 1 December 2014
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
47
and trainings. The first school (Yerevan School No. 27) opened its doors to children with disabilities in
1998, followed by modeling inclusive education in eight schools of Yerevan and Tavush region by UNICEF
and BoH. Since then the number of inclusive schools has been growing steadily through application of the
same model, reaching 100 by 2014 and providing education to more than 2500 children with special
educational needs. A major contribution to this development was the launch of joint the BoH and MoES
pilot project (funded by the Mission East and the Danish Government) in Tavush in 2010, aiming to create
conditions in all mainstream schools of Tavush for educating children with special educational needs and
preparing the system to transition to overall inclusiveness by 2022. As one of the interviewed NIE regional
staff members noted, “a mere hundred years ago it was difficult to imagine the existence of mixed gender
schools, which is currently the rule rather than the exception. The same can very well happen with
inclusive education.”
The feedback received through the teacher survey conducted for the purposes of this evaluation provides
additional evidence that the Armenian society has overcome its initial reservations about integrating
children with special educational needs into mainstream schools and is ready to embrace the concept of
inclusiveness.
Thus, over 80% of respondents have indicated their personal acceptance of the concept of inclusive
education, while 70% of the surveyed teachers also believe that Armenia is somewhat ready to implement
the inclusive education reforms. The teachers also have high confidence in their colleagues’ readiness to
implement inclusive education reforms. Thus, when asked to rank the readiness of different players to
introduction of the IE on a scale from 1 to 5 (1 being lowest and 5 highest), they have assessed their
colleagues’ readiness at 4.1. Interestingly, parents’ readiness is ranked as average at 3.1, while the schools
are perceived to be the least (at 2.8) ready for implementing the new system mostly due to unfit physical
infrastructure and lack of needed resources.
Table 8: Readiness to Implement Inclusive Education (average value) 20
(On a scale of 1-5 (1 being lowest and 5 highest), please assess to what extent the following groups (community
members/parents, the school, including its facilities and your colleagues with respect to their perceptions,
knowledge and skills) are ready for introduction of IE):
Community members, parents (attitude) 3.0
Their Schools (building and facilities) 2.8
Their colleges (attitude, knowledge, skills) 4.1
The respondents were asked “what role (positive or negative) the introduction of IE plays for different
stakeholder groups. The response “mostly positive” was highest (63.8%) for the official bodies (MoES, NIE)
followed by the parents of children in the inclusive education system (54.8%) and school administration
(48.7%). The response “mostly positive” was lowest for children in mainstream education and their parents
- 19.6% and 11.5% respectively. When asked to what extent they personally accept the concept of
inclusive education, 81.1% of respondents answered they accept it either fully or to some extent (see
20 Full list of output tables is included in Annex 7
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
48
Annex 7, Table 9). The respondents that provided positive and negative answers to this question, have
given justifications for their attitude. Over 54, 2 % of respondents point out to such positive implications
from the introduction of the inclusive education system as respect towards all children and an integrating
function for the overall society, while over 10% believe that it is disruptive for the entire learning/teaching
process. Chart 1 below summarizes positive and negative arguments provided by respondents as pros and
cons for inclusive education
Chart 1: Rationale for Attitude
It should be noted that while generally qualitative data also supported the need for changes to introduce
IE, more negative arguments were voiced throughout in-depth interviews and focus group discussions
with different informant groups. Both teacher and parents expressed opinions about the society not being
ready to embrace inclusive education, more time needed, parallels were drawn with how overtime there
was acceptance of people of color. Parents were more non-receptive of the idea speaking about their
reluctance to have their children attend school alongside with children from “special schools.”
Receptiveness toward inclusive education depended also on such factors as geography, lack of conditions
and resources, severity of condition of the children with special educational needs. Thus, the concept of
inclusive education is more acceptable in Syunik than in Lori, which is understandable given that more
time and effort has been invested in Syunik.
We can do nothing. If we are required to work like that, we will work like that. FGD, School /Lori
Before, I could not imagine what should be done with those children. With time, seeing their progress, I realized
that inclusion was the right thing to do. FGD, School /Syunik
54%
5%
4%
11%
11%
6%
9%Respect toward the rights of all children
IE system has an integrating function forthe society
Children feel better, psychological aspect
Lack of adequate conditions (physical,sufficient attention during the class, etc.)
Negative impact on the class, disruption ofthe classwork.
Need for individual work with SENchildren
Other
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
49
Lack of ramps, adapted restrooms, most importantly lack of special teachers, speech therapists and other
specialists, i.e. all the pre-conditions and resources needed for effective inclusive education are other areas
where unresolved issues raise skepticism about the system.
Another important factor that was voiced by informants is related to the specific condition that the
children with special educational needs have. They mostly agree that if the issue is a complicated mental
health one or mental disability, inclusion is difficult and at times, dangerous. Former special school staff
also were concerned about children with severe conditions who received special care in institutions and
were deprived of it in mainstream schools.
Most informants believed that inclusion could be more smoothly achieved in case of physical disability
and not for complicated mental disabilities.
I knew that a child in wheelchair will join our class from December. Months in advance I prepared my students,
showed a film, told stories and you should have seen the care and attention they showed this kid. FGD, School
/Syunik
To summarize, over the years, critical steps were taken toward integration of inclusive education
principles into mainstream schools both on policy and practical levels. These resulted in gradual shift of
perceptions and opinions and general acceptance of the notion of inclusive education. However, persisting
challenges related to capacity of specialists, adequacy of school infrastructure, resistance of parents and
others give way to certain questions about how ready different groups in the society are to fully embrace
IE. In this respect, the work done under the Project is undoubtedly contributing to changing the attitude
of educators, parents, children and others toward children with special educational needs and their place
in mainstream education.
EFECTIVENESS
General Findings
Finding 5: The child welfare /inclusive education reforms are complex multifaceted undertakings. As such,
the Project structure reflects the reforms complexity through its multiple components, implementing
partners and coordination with line ministries and agencies. While the Project is intended to support the
Government’s reforms agenda, the reforms pace and project pace are at times in conflict with each other
resulting in delayed Project activities.
Project donor and implementers, Armenian officials, child care institution and public school staff and
other stakeholders agree that both reform directions (child welfare/de-institutionalization and inclusive
education) while necessary, are complex, sensitive as well as resource intensive and requiring long-term
efforts. Deinstitutionalization requires reorganization of existing institutions (including legal aspects,
assessment of staff and related decisions, infrastructure improvements, etc.), assessment of children and
their return to biological families and support thereof, establishment of alternative services including
foster care and provision of relevant services, involvement of multiple players ranging from social service
providers to regional authorities to families themselves. Introduction of inclusive education into
mainstream schools is no less complex. It starts with closing down special schools and establishing
Psychological-pedagogical Support Centers, placing children with disabilities in mainstream education
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
50
and training teachers to understand the concept of inclusiveness and address the needs of the children,
working with parents (both those that have children involved in inclusive education and others), working
with children themselves. To summarize, both reforms have aspects related to reorganization, setting up
new services, infrastructure, capacity building, social support, public awareness and buy-in and others.
Consequently, they require mobilizing several stakeholders such as relevant Ministries, regional and local
authorities, social, pedagogical, psychological and health service providers, children and their families,
and the larger public.
With the understanding of these complexities and to effectively implement the Project, UNICEF in
Armenia has partnered with the most competent organizations that have long-standing experience of
supporting intervention in these areas. UNICEF’s original partners included World Vision, Save the
Children and Bridge of Hope, each responsible for a Project component matching its competencies and
experience. Later on, UNICEF also brought on board FAR’s Children’s Support Centre Foundation (CSCF),
a local NGO that initially was not included for USAID funding. Table 8 below summarizes the role and
competencies of each Project partner.
Table 9: Roles and Competencies of Project Partners
Organization and Respective
Project Component
Competence and Prior Experience
World Vision (WV): Support to
Transformation of Institutions
and set up of new services
Child protection is a cross-cutting theme throughout WVA’s 13 Area
Development Programs (ADP) in 6 Marzes and Yerevan. WV has
implemented a series of child protection projects funded by USAID and the
UK Government’s Department for International Development (DFID).
Through “Strengthened Child Protection in Armenia” $2.5-million project
funded by USAID WV has acted in support of GoA Deinstitutionalization (DI)
Strategy by enabling reunification of 172 children with their biological
families (as of June 2013).
Save the Children (SC): Foster
Care
Save the Children has been implementing child protection and child rights
governance initiatives with the focus on strengthening state systems and
mechanisms for implementation and monitoring of child’s rights. Through its
programs, Save the Children ensured training of parents and caregivers on
positive parenting approaches that need to be applied for children’s
upbringing. In 2012-2013, SC Armenia conducted research on perspectives of
foster care in Armenia.
Bridge of Hope (BoH): Inclusive
Education
Since its establishment, BoH has advocated on disability as a human rights
issue in accordance with the social model of disability. Bridge of Hope has got
branches in 4 cities of Tavush region providing services to 1200 children with
disabilities and special education needs. Since 2001 “Bridge of Hope”, as the
key partner of MoES for inclusive education, has implemented large
education programs in cooperation with the Ministry, civil society and
international agencies.
Children’s Support Centre
Foundation (CSCF): Case
CSCF is the only crisis intervention service for children and their families in
Armenia. It provides crisis intervention for children aged 3 to 18 who are in
difficult situations, including abandoned or homeless children, street
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
51
management, Foster care and
Transformation of Institutions
children, children with behavioral problems, children in conflict with family
and the law, etc. The center’s goal is to prevent these children from becoming
institutionalized.
UNICEF, as the lead organization undertook coordination efforts, communication with government
partners and USAID, policy and advocacy work, as well as some direct implementation activities in areas
of its expertise, including involvement of ISS/case managers, service providers in child protection field,
local social planning and others. Another important structure involved in Project implementation is the
operational level steering committee comprised of key government partners, USIAD and Project
implementation partners that has regular meetings around key Project developments.
It should also be added, that while the overall term of the Project is 5 years, each partner had an agreement
with UNICEF according to which the intensity and length of their involvement in the Project differed,
including:
• Initially the BoH’s contract term was until February 2017, but they requested a one-year extension to
continue their work in Armavir Marz. However, given the uncertainty of the Government’s plans for
Armavir Marz, the contract was extended only until June 2017 to wrap up activities in Lori Marz.
• The contract with Save the Children expired in May 2017 and was not extended, due to the mutual
agreement that their further work depended on the adoption of the Family Code.
• WV had a 36-months contract ending in December 2017, however as noted in UNICEF’s January-
March 2017 report, “due to slow progress in decision making regarding the transformation of
Noubarashen CFCS and the residential care institutions in Lori and Syunik regions, WV is facing
considerable underspending since significant part of their budget is dedicated to renovation and
equipment purposes”.
While overall several results were achieved in the past 2.5 years, the situation with each implementing
partner indicates that there have been a number of bottlenecks (beyond the control of the Project) which
either slowed down or diverted the Project activities.21
Project Area Key Results Challenges
Reorganization and
transformation of
institutions
• One institution closed (Yerevan child
protection and care institution #1) and one
transformed into a CFSC (Yerevan child
protection and care institution #2)
• Assessment of 150 children in 2 residential
institutions in Lori and Syunik
• Assessment of Staff in Noubarashen, Kapan
and Vanadzor.
• Slowing down of the process by
MoLSA, Noubarashen
reorganization delayed,
approach unclear
• G2G support to MoLSA still
under discussion
• Lori assessment by WV
outdated
• CFSC staff training delayed until
autumn 2017
21 UNICEF, BoH, WV quarterly reports are used as a source
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
52
Institutionalization of
Social Service system (case
management, assessment
of children, care plans and
family assistance packages
to reunified children,
territorial social planning)
• Two training packages for case managers and
service providers developed by CFSC
• Over 100 case managers trained through two
5-day training sessions
• Over 192 social service providers trained
• Different capacity level of case
managers
• Lack of community-based
alternative services for case
managers to refer cases
Foster Care • Amendments to the Family Code with
corresponding 12 bylaws regulating the
diversified foster care model were drafted
and accepted by MoLSA
• Over 74 potential foster parents trained on
foster care regulations, registered and 55
accredited
• 132 experts and stakeholders involved in
public hearings on legislative reforms
• Over 2500 persons involved in community
level awareness meetings and professional
forums
• Family Code amendments still
pending22
• Lack of state budget funding for
new foster care services23
• Broadcast of PSA and
documentary for improved
awareness not released
• Foster family database not
created
• Full system application on hold
Inclusive education • 4 special schools transformed into RPPSCs
(Syunik and Lori)
• Over 650 children assessed in Lori and
Syunik by RPPSs
• Over 90 specialists of special schools and
RPPSCs involved in a series of trainings
• Over 5000 teaching staff trained in 264
schools in Syunik and Lori Marzes, including
264 principles
• 4506 parents attended one-day seminars
• Number of children served by RPPSCs
• MOES request to postpone the
trainings of the teaching staff of
162 schools in Lori region from
August-October to November-
February resulting in slower
pace of expenditure
• Infrastructure improvements
under G2G still pending
Legal framework • Action Plan and timetable for provision of
total inclusive Education system approved
• The Concept for Foster Care of Children in
Difficult Life Circumstances approved
• Several decrees and decisions passed to
implement institutional reorganization, to
approve methodologies, etc.
• Family Code amendments still
pending24
As the above summary analyses and review of indicator tracking tables indicates, the key outputs so far
are attributed to interventions by primarily BoH, (which has completed most of its undertakings under
the Project) and CSCF with respect to exceeded number of case managers and social service providers
22 See the introductory note above.
23 See the introductory note above.
24 See the introductory note above.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
53
trained (over 250 instead of planned 145) as well as foster parent training and community level advocacy.
While SC (through CSCF) has conducted the trainings for foster parents and did significant work with
respect to legal amendments, the massive broadcasting of visual materials on foster care model is still
pending. Due to slow pace of transformation in Noubarashen, as well as child care and protection
institutions in Lori and Syunik Marzes, WV has accumulated significant underspending. Throughout the
evaluation, it was difficult to delineate the roles of WV and CSCF with respect of assessments, and SC and
CSCF with respect to trainings under foster care. The CSCF role is more complicated, since it was
contracted out by UNICEF to conduct trainings for case managers, but eventually became involved in
foster care training and assessments of children from child care and protection institutions. As noted by
UNICEF, once CSCF was on board, it was decided to combine child- focused case management trainings
with foster care trainings under direct contract with UNICEF, for efficiency purposes. As for in-depth
assessment of children, it seems that the initial assessment for Yerevan # 2 was conducted by WV, while
in Syunik both trained case managers supported by CSCF and WV social workers were involved in the
process. In Lori, CSCF and case managers conducted social assessment of families of the children from
special schools to complement the educational needs assessment by RPPSCs. Assessment of children from
Vanadzor Child Care and Protection Boarding Institution was initially conducted by WV, but was
considered outdated given the delay in the reorganization of institutions alongside concerns about its
quality. It was later tasked to CSCF, and was completed in March 2017.
Overall, the uneven curve of reforms progress is also a challenge - as some key informants noted “things
slow down for months, then there is more intense activity. The changes in the government also created
delays in the approval of critical legislation. Another evidence on divergence of reforms and Project
priorities, is MoLSA’s commitment made to the Prime Minister to close all night boarding institutions by
the end of 2017, resulting in actions toward that direction, while the Project is focused on a limited
number of institutions. Consequently, the Project interventions and expected results being directly linked
to how energetically the Government is implementing its reforms agenda, there have been several
challenges, delays and short-comings in almost all Project directions.
However, it should be noted that despite these challenges and complications, UNICEF has put significant
efforts in pushing the reforms forward through negotiations with the Government and in finding ways to
resolve implementing partner related issues (like bringing on board CSCF). While both the Government,
USAID as the donor, as well as implementing partners, acknowledge UNICEF’s leading and mobilizing
role in the Project, they also point out to certain coordination challenges, which are summarized below.
Each IP is implementing their specific component, but UNICEF is the lead partner involved in policy and advocacy
matters together with USAID. They should all sit together around negotiation table, but it is always UNICEF that
is in charge for communication with the Government. Expert Interview, Implementing Entity
Having all these components under one Project was in a way a good coordination strategy since all key players
were involved and informed, being on the same page about the overall reforms directions. Expert Interview, State
Official
Almost 1.5 years for preparation-which was acceptable given the complexity of the project and the associated
reforms-however after that USAID expected better progress. Lack of quantitative achievements is one thing, but
the qualitative aspect of UNICEF’s work is what bothered USAID, mostly lack of evidence based justifications that
could help the Government move in the correct direction. Expert Interview, Donor
The Project was badly planned and badly implemented. Expert Interview, Implementing Agency
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
54
Coordination meetings between WV-FAR-SC, also ad hoc meetings with UNICEF, they are open, we can call
them, email, but there is a lack of a systematized approach. Expert Interview, Implementing Agency
Need for better coordination of activities in Lori Marz. Expert Interview, Implementing Agency
An issue may be discussed through a chain of emails for months with no response from UNICEF, then suddenly
we learn that it was resolved. Expert interview, Implementing Agency
Child Welfare Reforms
Finding 6. While UNICEF and partners have developed and proposed to MoLSA a model for establishing
Child and Family Support Centers and supported reorganization of institutions and their transformation
into CFSCs, several factors contributed to certain inconsistencies and drawbacks in the process, mostly
due to delays and shifts in the reforms priorities led by the Government. Noubarashen, Kapan and
Vanadzor have followed different timelines and patterns of transformation that affected institution/center
work and beneficiary experiences with placement/follow up service provision.
According to UNICEF and World Vision reports, four (4) institutions have been targeted for
reorganization under the Project, including Yerevan Child Care and Protections Institute #1 (Korea Dzor),
Yerevan Child Care and Protections Institute #2 (Noubarashen), Kapan Child Care and Protection
Boarding Institution, and Vanadzor Child Care and Protection Boarding Institution. In addition, there has
been considerable ground work for transformation of Vanadzor orphanage through advocacy efforts and
planning. However, the situation with each of these institutions is different in terms of their legal status,
Project interventions, their transformation into a Child and Family Support Center (CFSC), provision of
service, etc. (see Table 10 below)
Table 10: Transformation Status of Institutions
Institution Current Status Project interventions
Yerevan Child Care
and Protections
Institute #1 (Koreai
Dzor)
Closed MoLSA closed without prior assessment and
a clear model; some children returned to
families, but most of them went to other
institutions. WV and other NGOs supported
those children that remained in Kanaker-
Zeytoun (WV ADP) and the children
became their beneficiaries.
Yerevan Child Care
and Protections
Institute #2
(Noubarashen)
Transformed into Child and
Family Support Center by GOA
decree in August 2016
Targeted by the Project, WV involved in
assessing the staff and later training,
proposed initial model that was used to
certain extent by MOLSA (65 children
leaving institutions, of which only 6 were
originally from Noubarashen district, others
returning to communities of origin),
support to children and families by ISS case
managers, employment services, SOAR,
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
55
CSCF, SOS villages (for more than 6 months
to children reunified with their families
from this institution)
Kapan Child Care and
Protection Boarding
Institution
Decree on reorganization
planned to be passed in October
2017
Staff assessment; child and family
assessment for 45 children, support to
children and families including material
support such as food, medication, firewood,
etc. (for 25,000 drams per month per family)
Vanadzor Child Care
and Protection
Boarding Institution
Decree on reorganization
planned to be effective as of
November 16, 2017
Staff assessment, child assessment and
support to children and families, including
material support such as food, medication,
fire wood, etc. (for 25,000 drams per month
per family), advocacy with regional
administration, trainings of case managers
in the region, as well as trainings on positive
parenting
Building up on its experience from the USAID-funded “Strengthened Child Protection in Armenia”
project (implemented in 2010-2013, through which over 170 children were unified with their biological
families), and the best international practice, WV developed a model for creating family support services,
which was submitted to MoLSA in April 2016. The model detailed the services to be provided by each
professional of the center, the qualification of the professionals and the set-up of the rooms, including the
description of items that should be in each room of the service center.
..we also showed that these centers don’t need to be big buildings or excessively overstaffed institutions, and can
service 3 times more children on much less budget than spent today. Expert Interview, Implementing Agency
It was expected that WV should start working with the staff of the institutions before they would be
reorganized, immediately after the staff needs assessments were implemented– to support the institutions
to hire new professional staff (psychologists, social workers, social educators, speech therapists), and to
train the staff who transitioned from the closed institutions, to develop relevant methodologies and
interventions. However, due to the changes in the state priority, decisions were made to close the Yerevan
Child Care and Protection Boarding Institute No. 1 and to reorganize the Yerevan Child Care and
Protection Boarding Institute No. 2 (Noubarashen)25, before approving the model for CFS Centers
submitted to the MoLSA by WV. Interestingly, months after the model was shared and the Noubarashen
decree passed, the Ministry requested the Project to support with Noubarashen CFSC model.
It became clear in the very beginning that the State was not ready for the reforms in terms of not having a well-
defined, clear implementation schedule and action plan. There seemed to be lack of trust in the reforms among
the public administration agencies. On the other hand – this was a new area for them, so there was most probably
25 GoA Decision N 831-N, dated as of August 11, 2016 on Reorganization of the “Yerevan Child Care and Protection
Boarding Institution N 2” into “Child and Family Support Center” SNCO.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
56
some fear, that the reforms might not be successful, which is also understandable. Expert Interview, Implementing
Agency
It took almost another year for the Government to come up with a structured process for reorganization
and transformation of the other institutions, which is formalized in the decrees for each specific one
(Kapan and Vanadzor, decrees passed in October 2017, transformation to begin in January 2018). The
overall reorganization process was structured to start with renaming the institution, drafting its charter,
legal succession and solving property rights issues, the right for implementation of aid services within the
scope of the foundation, etc. With respect to transforming the institution into a CFSC, World Vision’s
model for the Child and Family Support Center was reflected in the charters. While Noubarashen’s
transformation became a pilot for other institutions, pending issues with Noubarashen are indicative about
the differences as to how MoLSA, USAID and the Project partners perceive the model for the newly
established CFSCs. There is an ongoing controversy between the Donor (USAID), the Project partners and
MoLSA with regards to the scope and the budget for renovation activities in Noubarashen. While MoLSA
would like to establish a large-scale demo CFSC, USAID and UNICEF believe that smaller and more
flexible centers close to the communities of origin are a better solution which will serve the children and
their families better, as well as being more accessible.
... According to the model, there is no need to renovate the entire building (2 parts of it), though seismic
enhancement was done since it is important... The two floors of one of Noubarashen buildings is sufficient for the
Center we propose…but we cannot move forward before there is an agreement between the Donor and the
Ministry. Expert Interview, Implementing Agency
Some other consequences of this situation are related to delays with the training of the institution/CFSC
staff. Budget wise, it was considered inefficient to provide training to the Noubarashen staff separately but
rather train the staff of all the three institutions targeted by the Project, once Kapan and Vanadzor status
is more or less clear. As the process of reorganization moved forward (the MoLSA at least committed to
the reorganization of the three institutions within the scope of the Project26), staff assessment and
professional staff recruitment in both Kapan and Vanadzor Boarding Institutions began and a series of
joint trainings were conducted for all the three institutions (Noubarashen, Kapan and Vandazor Boarding
Institutions) in September-October 2017.
While these are positive developments that occurred beyond June 2017, the discussed issues had their
consequences since Noubarashen CFSC staff was not trained for almost a year after its establishment and
had to operate without professional guidance. Several other implications of these shifts and delays in terms
of both support to the children and their families, and the setup of the new service, i.e. the Noubarashen
CFSC, are discussed below.
26 RA Decision N 1398-N on Renaming the Kapan Child Care and Protection Boarding Institution SNCO into Syunik
Marz Child and Family Support Center Foundation, approving of the Carter and Providing Property ; draft decision
on Vanadzor Boarding Institution and Orphanage was part of the desk study, but the official decision is not made
yet.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
57
• Children from No. 1 Boarding Institution returned home in the summer and never came back to the
institution, disturbing Project plans to start the work with those children and their families in mid-
autumn, and causing “loss of some precious time to work with the children and families”.
• The mismatched schedules, delay on approving the model (by the Government), as well as the
engagement of World Vision in other activities under the Project (i.e. evaluation of seismic stability
of Noubarashen Child and Family Support Center and the related construction activities) delayed the
preparation and implementation of the training packages for the staff of the already re-organized
institution.
• The staff of the reorganized institution that stopped “accepting” (i.e. enrolling) new children in
autumn 2016, had officially been renamed into Child and Family Support Center as of January 2017
(aka Noubarashen CFSC). For an entire year it operated based on the general Job Descriptions of the
positions, their skills and intuition, as well as “finding and reading things on internet, mostly in
Russian”.
• While the CFSC is established, only 6 children originally from Noubarashen continue making use of
its services (as stated in the Charter of the Center, the children, who need the services of the Center
will continue making use of those from 6 to 12 months, with a following re-assessment), while the
Center does not track the destiny of the children that returned to other communities of origin,
especially rural areas where there are no NGOs providing services. As noted above, this issue is closely
interlinked with the ongoing conflict between MoLSA and the Project donor and partners about the
G2G investment into the center infrastructure. While MoLSA is willing to commit over 1 million USD
to creating a model first, CFSC, USAID and Project partners believe at least three such centers have to
evolve (in Erebouni and Shengavit as well) to support more children in their communities of origin.
• The destiny of those children who cannot return home remains unresolved. In Noubarashen there
were 5 children that could not be sent back home due to unacceptable situations in or of the family.
These children were placed in other institutions, since even if there are registered foster families, it is
still an issue to “place” the child in foster care, due to limiting legislation.
Despite these concerns and persisting issues, Noubarashen CFS Center is currently considered as the
pioneer and as an “experienced” center by now and it provides a basis for a few lessons learnt for the next
to-be-reorganized institutions.
• While only 5-6 former Noubarashen resident children are served by the Center, the total number of
children currently benefiting from its services is more than 100 since children in difficult life
circumstances from this community are also being served by the Center. More than 300 children have
been assessed as potential recipients of the services of the Noubarashen community as well as
Shengavit and Erebuni. This is a positive example of effective utilization of the Center and in line with
the Project proposal.
• As several informants agreed (institution staff, implementers and experts) staff training should follow
the needs assessment and recommendations for changes, happening in advance, long before the
reorganization, so that the staff is prepared to work with children and parents alike in their new roles.
In addition, placement of children should be pre-arranged, monitoring provided, and all relevant
structures mobilized in the communities of origin.
The situation of Kapan Night Boarding Institution to be reorganized into Syunik Marz Child and Family
Support Center seems to be much better. While the transformation is expected to happen following the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
58
adoption of the RA Decision N 1398-N in November 2017, assessment of the children in the institution
and their families was conducted by the case managers (with coaching by CSCF supervisors) and further
support in terms of providing family assistance packages by World Vision. Most children (35) are staying
home at night, but spending their days at the institution, including having their meals there. Since many
of them are from Kapan and its outskirts, only a handful of children that are from the villages stay
overnight. Besides, the professional team has already been enriched, two psychologists joint the team in
2017, and professional training is planned for the team for autumn 2017. The future staff of the new service
center has enough time to study the experience of the CFSC in Yerevan, visit it, discuss the “do’s and
don’ts”, which is quite helpful for the Director to start organizing its activities. Simultaneously, CSCF-
trained case managers and WV’s local social workers are involved with the families of the children and
the boarding institution on nearly daily basis. Assistance packages including food and furniture, have been
provided to families in need of those by UNICEF and WV. All the ground work that has been done by the
Project in Kapan allows for the transformation of the institution and the operations of the future service
center to be much smoother. With Vanadzor Child Care and Protection Boarding Institution, WV made
the child assessments over two years ago. However, due to delays in the Government decision to start the
reorganization and some concerns over the quality of the assessments implemented by the company hired
by WV, assessment results are outdated. The assessments were updated in January 2017 by the case
managers under the professional coaching and mentoring of CSCF supervisors. Two stage in-depth
discussions for each case were organized by CSCF and UNICEF to validate the assessments, mobilize the
partners and stakeholders, and to find common solutions to individual child’s case. During the interview
with the institution Directors (the former one and the currently acting one) it was quite clear that they
are well aware of the reorganization process, have started the process of hiring new staff for the future
service center and have also studied the lessons learnt of the Yerevan CFSC. It should be noted that with
respect to alternative services, Vanadzor (and Lori Marz in general) seem to be in an even more favorable
condition, given the presence of several NGOs in the area. World Vision itself does not have a
representation in Vanadzor, but it is present in Stepanavan and Alaverdi, which makes it possible to cover
the children from the adjacent rural communities that will return to these communities from the boarding
institution.
To summarize, the reorganization and transformation of child care institutions have understandably taken
an uneven path given its different complex aspects and involvement of different stakeholders and interests.
As much as possible, the Project has attempted to smooth the process by maintaining constant engagement
with the Government to push the legal foundations needed for moving forward (decrees, charters, etc),
proposing a model for the CFSCs, training case managers and other social service providers as well as
mobilizing partner NGOs to address the needs of children reunified with their families, to the extent
possible. By June 2017 there has been mixed success, including the establishment of the first CFSC and
reunification of children from Noubarashen with their families (over 50), assessment of hundreds of
children and families, provision of support to reunited families in form of material assistance and specific
services. As to the drawbacks, absence of community based family support services in the communities of
origin where children return, has remained the paramount issue to be addressed. As noticed above,
UNICEF and partners have found ways to overcome most of this hurdle, but not through long-term
solutions. While a model for CFSCs has taken shape, its full application is still to be ensured. To illustrate,
while the Charter of the service centers is clearly defining the general scope of activities of the staff of the
new institution, a major concern brought up during all the interviews and FG discussions, was about the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
59
lack of training and education on how to work in this new type of institution. The Job Descriptions
attached to the Order on the establishment of the service centers do not give much information on the
specific professional performance of the professional staff. While the mandate of the service center is well
understood, in general - to provide services to the family – psychological, social, legal, medical, parental
skills and household economy, the “how to do that” part is still missing. Although trainings are planned
for the general professional staff of the service centers and specific, discipline-based trainings will follow,
there is concern that one-time short-term training is not enough to build long lasting capacity of the team.
All these concerns expressed by the institution staff about effective execution of their functions are
additional evidence to the need of a clear and consistent process and model for reorganization of
institutions into CFSCs with specific guidelines, methodologies and manuals in place, as well as a quality
control mechanism/oversight to ensure that minimum standards are adhered to.
Foster Care
Finding 7. The Project succeeded in conducting a two-phase training for over 74 foster parents. While
improved parenting skills is noted as a training benefit by several of them, delayed placement of children
is creating disappointment among the potential foster families.
Trainings for 74 potential and experienced foster parents (instead of initially planned 120) was
implemented by FAR’s Child Support Center Foundation (CSCF) in 5 regions, i.e. Lori, Syunik, Armavir
and Ararat Marzes and Yerevan. CSCF specialists were also responsible for developing 12 training modules
and methodologies such as Guidelines for Monitoring Foster Families and Foster Care Program, Principles
and Standards for Foster Care, Public Awareness Manual. In addition, they supported
registration/accreditation of foster families and training of other social service providers such as the case
managers.
Trainings of potential foster parents were implemented in 2 phases through three-day sessions in each
phase. Training modules for the first phase included such topics as reforms implementation, the role of
foster families, procedures for registering as foster families, while the second phase focused on issues
related to types of families, child psychology and other related topics. As part of the training experienced
foster families shared their experiences with the potential candidates.
Some disagreements occurred between Save the Children and CSCF throughout the implementation of
this component. To illustrate, since recruitment of foster families required awareness building about this
type of care among the potential candidates, intensive ground work was required to accomplish this task
in preparation for trainings and registration. According to the CSCF head, this function was not included
in their assignment but eventually they had to visit over 200 villages to meet with people and inform them
about foster care as an alternative to biological family and institutional care.
Initially, the recruitment of foster families almost failed. I went to 268 communities using established connections
with heads of communities and schools. Public awareness was at zero level before that, no families were applying.
We also worked with media and succeeded in inspiring interest toward foster care. Expert Interview, Implementing
Agency
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
60
Focus groups and in-depth interviews with potential foster families also indicated that they learnt about
the foster care from CSCF, local TVs, and experienced foster families. As a result of these efforts 93 families
were enrolled in the program. Based on feedback of interviewed potential foster parents, the following
types of families interested in foster care were identified:
• Families with no children;
• Families with one child who want their children to have siblings;
• Families with grown up children, who wish to continue caring for younger children.
A large group of foster parents (involved in the FG discussion) are those who wanted, but did not succeed
in adopting children and they perceive foster care as a solution to this issue.
Most training participants (both FG participants and in-depth interview informants) speak very highly
about the qualifications of trainers as well as usefulness of the trainings resulting in acquiring some critical
knowledge and skills, including:
• Giving consideration to the child's point of view in different situations;
• Tolerance to unacceptable behavior displayed by children sometimes;
• Patience.
I finally made my husband attend the training with me, at least for an hour. He became so engrossed that he
stayed until the end. FGD, Parent
However, it should be noted that the majority of FG participants voiced concerns that for over a year after
the trainings none of them had an opportunity to practice their foster care skills. Many of them expressed
concerns that they might soon forget the knowledge and skills acquired. Below are a few quotations
summarizing trained foster parents’ insights about the uses of training and their concerns about delayed
applications of skills and knowledge.
If in the next few months we are not given a child, I will not be interested in this program any more. I am waiting
for that call every day. FGD, Parent
Inclusive Education
Finding 8. Implementation wise IE-related interventions have been one of the most effective components
of the Project with trainings conducted among 1670 teachers in Syunik Marz (102 schools) and 3760 in
Lori (162 schools) respectively, and successful involvement of key players and stakeholders (NIE,
coordination with the Republican Centre of Pedagogical Psychological Centre, and other involved
stakeholders including parents). Thus, several key results were fully achieved with some areas still
requiring improvements.
Bridge of Hope NGO is implementing the Inclusive Education (IE) aspect of the Project, and has worked
in several directions to ensure all pre-conditions are set for implementing the system in the Project target
regions, Syunik and Lori Marzes. This included:
• Four Pedagogical-Psychological Support Centres (PPSC) were established/supported in 2 target
regions which provide professional support to mainstream schools. With technical assistance and
policy advice from UNICEF, MoES and Bridge of Hope has already defined the functions and
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
61
operational modality of these centres. A situation analysis/resource mapping was conducted and a
transformation plan was developed for special schools. A special experience exchange between Syunik
and Lori PPSC professional staff was organized to share the lessons learned during the first phase of
the transformation.
• Based on “Amendments and additions in the RoA Law on Mainstream Education” December 01, 2014
Decree ՀՕ-200-Ն a list of documents was developed to support the implementation of the Law.
Together with MoES, BoH established 6 expert working groups to develop relevant documents and
normative acts for the Government’s approval including “Action plan and timetable for the provision
of a total Inclusive Education System”, PPSC Service Definition, Mainstream school Statute and others.
On February 18, 2016 the Government of RoA approved it.
• Around 200 specialists involved in Syunik and Lori RPPSCs and former special school staff underwent
trainings on such topics as ICF-based assessment of special educational needs of children, formative
assessment; development of individual learning plan, curricula adaptation and others.
• Inclusive education training for around 266 mainstream/regular schools in Syunik (102) and Lori (164)
Marzes reached out to over 5000 teaching staff engaging teachers, school principals, and local experts
in knowledge building and capacity enhancement. The first ToT was conducted for the principals of
these 266 schools by NIE, who then trained their staff locally.
Several other activities and interventions were implemented as part of this Project component including
children’s assessment, assessment of facility conditions (including mainstream schools) and
recommendations for improvements under G2G, and others. In designing and implementing all these
activities, BoH has identified the partners best qualified to deliver the work, including National Institute
of Education and Republican PPSC.
The intensity and rigor of all these interventions is not lost on one of the key target groups, i.e. teachers.
76.9% of surveyed teachers responded that they have attended trainings on inclusive education in the past
2 years with over 95% saying they attended the course in full. 70% of respondents stated confidently that
the trainings were useful to them (see Table 12 below).
Table 11: Utility of Training Session
Were the trainings useful for you? %
Definitely useful 70.0
Somewhat useful 27.1
Not that useful 1.7
Not useful at all 1.3
Total 100.0
With respect to how the teachers assess the different aspects off the trainings, such as effectiveness,
organization, content, etc., their overall average assessment is rather high on the scale of 1-5 (5 being the
highest). See the chart 2 below.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
62
Chart 2: Average Value of Training as Assessed by Respondents (5 being highest and 1 lowest)
Generally, during the FGs the teachers also expressed satisfaction with the trainings, they recalled the
BoH and NIE trainers and pointed out that involvement of professional organizations like NIE and RPPSCs
is a good approach. RPPSC trainings are also very highly valued and the evaluation team was able to
observe teacher assistant trainings in progress during the field work. However, despite these positive
findings both from quantitative and qualitative research, there still remain a number of issues that should
be addressed to effectively introduce inclusive education to mainstream schools. These include:
• One time and limited length of trainings with several informants indicating that a 5-day training is
not sufficient for this complex issue and requesting more;
• In some of the schools, the principals just held one general meeting to introduce the concept of IE
without taking the time to transfer the full content of the material.
• The work with parents is uneven across different schools. In some of them principals and teachers
have worked with parents (where IE is functioning longer), in others there have been zero efforts in
that direction.
And finally, monitoring of how IE is being implemented across the schools is an important aspect of the
Project that should be enhanced to achieve full and meaningful integration of IE into mainstream schools
and insure its acceptance by teachers, parents and children alike.
Finding 9. Knowledge of the basics of inclusive education has increased significantly as a result of teaching
staff trainings resulting in some level of practical application of learnt concepts and approaches as well as
overall acceptance of the new system (as perceived by surveyed teachers).
Assessing the level of knowledge increase among school teachers from target regions was first done by
BoH and partners through pre-post knowledge tests. According to BoH final report, for Goris and Sisian
area knowledge growth is 54% and for Kapan and Meghri area it is 50%, and overall growth of knowledge
4.6
4.5
4.3
4.2
4.2
4 4.1 4.2 4.3 4.4 4.5 4.6 4.7
Delivery of material (clear explanation, interactivemethods, etc.)
Content (addressing all issues, introducing newmaterials, etc.)
Logistics (time, place, refreshments, etc.)
Training effectiveness
Utility for teacher
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
63
measured among 1670 participants is 53%. With respect to Lori, the percentage of the teachers who
answered correctly to all the questions increased from 0 to 82 percent (see chart 3 below).
Chart 3: Change in Absolute Knowledge
Source: BoH Final Report
With respect to the teacher survey implemented as part of this evaluation, teacher knowledge on basic
concepts of IE was also tested, including the definition of IE, some statements about the IE as well as a
question on which children are considered as having SEN (special education need). 93.9% of respondents
were able to choose the correct IE definition. A knowledge index has been calculated on this question (+1
to -1 interval) with the general results comprising 0.87. See Table 12 below that summarizes the index per
sub-region in Syunik.
Table 12: General Knowledge Index by Sub-region
Index of knowledge
Average Kapan Goris Sisian
Definition of IE 0.87 0.91 0.85 0.85
The medical model of disability is at the basis of inclusive education -0.56 -0.69 -0.59 -0.4
Every teacher should have the skills to identify the observation and learning
abilities of the student
0.96 1.00 1.00 0.88
Subject teachers do not participate in the development and implementation
of the individual education plan. Only the parents, the class teacher, the
special pedagogue and the psychologist are involved in the process
0.39 0.41 0.47 0.29
The individual education plan is developed for one school year or based on
the short-term issue.
0.83 0.85 0.88 0.77
Children that are considered SEN 0.78 0.83 0.76 0.75
99.81%
18.22%
0.19%
81.78%
Pre Post
Answered right all thequestions
Answered wrong at leastone question
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
64
Aggregated value of Knowledge index 0.65 0.66 0.67 0.63
With respect to other statements (on medical model of disability versus social, teacher participation in
developing and implementing individual educational plans and others) as well as SEN definition, a
generalized knowledge index was also calculated and the results are summarized in Table13 below.
Table 13: Application of knowledge (Data source-240)
Do you apply knowledge gained through training Frequency %
Yes, regularly 96 40.0%
Mostly yes 107 44.6%
Mostly no 14 5.8%
No 23 9.6%
Total 240 100.0%
Qualitative research indicated that there were knowledge related differences across the two regions, i.e.
Syunik and Lori. In the first one the trainings happened back in 2016-2017, while in Lori mostly in 2017.
Thus, in Lori Marz the principals and teachers had better knowledge of theory, while in Syunik the
practical understanding of the IE system was better.
Inquiries made both through quantitative and qualitative data collection allowed assessing to what extent
teachers are applying their knowledge on IE. As table 14 below shows, 84, 6 % of respondents answered
positively to the question about knowledge application. However, it should be noted that out of 40%
responding that they regularly apply the knowledge gained, the majority were teachers of elementary
schools.
Some other questions related to application of knowledge addressed the teachers’ involvement in
assessments of children’s learning abilities. While 60.6% of respondents answered this question positively,
around 40% of trained teachers in Syunik Marz are still not involved in the process.
In addition, qualitative research allowed understanding how individual educational plans are developed
and implemented. It appears that in those schools where there is a multi-disciplinary team, the team
specialists are the ones primarily involved in the process, and the required collaboration between the
teachers (including homeroom teacher), parents, psychologist and other specialists is not widely practiced.
Some of the FG participants referred to the plans that are needed for working with children with SEN,
but did not associate those with individual education plans.
The evaluation also attempted to look at IE application through the perspective of children’s inclusion and
acceptance of the new system. Only 14.3 % of respondents said they faced no problems with children with
SEN, with 48.6% indicating some problems, and over 21% citing frequent problems. Some of the teachers
participating in FGs noted progress as a result of their work, while others claimed there is no use working
with these children. As one of the informants noted, “I work with him for two months already to teach
him the colors, no results, he does not memorize…”
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
65
One hundred seventy five (175) teachers mentioned that they have students with special educational
needs in their classrooms. 56.5 % of these teachers believed that children with SEN are socializing with
all other children in class/school, while 26.7% thinks that they mostly deal with those children that treat
them well (see Table 15).
Table 14: Socializing Patterns of Children with SEN (Data source-175)27
With whom do Children with SEN mostly socialize in
class/school
Frequency %
With all children 131 56.5%
Only with each other 5 2.2%
With neighbors’ children in the same class 16 6.9%
With children of keen in the same class 15 6.5%
With children who treat them well 62 26.7%
With no one 3 1.3%
Total 232 100.0%
FGs and in-depth interviews gave additional insights into children behaviors and attitudes, such as:
• Integration of children from special schools is easier in rural areas where the community knows the
child and the family and accepts them;
• Children with SEN have more problem in middle school when they are teased and isolated. It becomes
difficult for children with physical disabilities to access the school or the classroom;
• The children with SEN are taken out of the class for different reasons (e.g. for food or separate
classwork), which highlights their differences;
• Some of the children with more complicated issues who were dismissed from special schools become
involved in home based education. Consequently, they become more isolated, while in the past there
were integrated in at special school.
And finally, a number of different issues were identified during FGs and in-depth interviews that
challenge effective implementation of the IE in mainstream schools, including:
• In bigger classes, teachers have problems with distributing their time evenly and a lot of time and
focus goes to children with SEN. There is one teacher assistant per 100 children, which is not enough,
especially when there are a number of children with SEN in the school. It is sometimes difficult to
find individual approaches to these children.
• Knowledge assessment/testing of these children is still an issue. There are no special tests/exams for
them, just additional time allocation during exams.
27 Teachers had the option to provide more than one answer to this question
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
66
• At times children with SEN display atypical behavior, while at other time they use their limitations
as an excuse saying “I cannot do this, I do not want to do this, have mercy”.
To summarize, Project interventions have increased the knowledge on IE among the teachers in target
regions resulting in certain improvements in application and attitudes toward children with SEN.
However, there are still several issues and challenges to be overcome to reach full-fledged integration of
IE in mainstream education.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
67
EFFICIENCY
General findings
Finding 10: From the Project perspective, the combination of USAID’s G2G support coupled with the
implementation of the soft interventions under the Project (with each partner contributing to within the
overall budget) was initially perceived as a good tool to increase Project efficiency and resource use, but
it did not work as planned. As of June 2017, around 50% of the overall Project budget has been expended
with mixed outcomes that vary from Project component to component. From state budget perspective,
unwillingness of the two line ministries to join certain aspects of the reforms agenda under one umbrella,
has raised questions among different stakeholders about efficient use of funds and resources.
Under its 2013-2017 Strategy USAID approach to providing aid is based on triangular partnerships with
the host government (through Government to Government (G2G) mechanism), a technical assistance
vehicle through an international organization and a monitoring scheme through local civil society. The
approach was not different in the case of the “Toward Social Inclusion of Vulnerable Children: Expanding
Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform” Project.
While the technical assistance guided the reforms implementation through developing alternative service
models, making assessments and providing recommendations for transformation plans and advocating for
legal changes, it was envisioned that the two G2G agreements between USAID and MoLSA (2015-2018),
as well as USAID and MoE (2016-2019) would support the process through infrastructure improvements,
including:
• Infrastructure costs for the set-up of new community based alternative services guided by World
Vision Armenia should be covered by the Government of Armenia with the G2G funding from
USAID.
• While technical assistance and policy advice from UNICEF, MoES and Bridge of Hope targets defining
the functions and operational modality of RPPSCs, the infrastructure costs for service set-up should
be covered by the Government of Armenia with the G2G funding from USAID.
• School plans on providing an inclusive learning environment should include infrastructure
adjustments (wheelchair ramps, accessible bathrooms, FM systems, resource rooms to provide after-
school services and other) that should be covered by the Government of Armenia with the G2G
funding from USAID.
Given these close interconnections between the Project and the two G2G
agreements, the latter two include strong conditionality language with
respect to supporting the Project implementation. However, while this
approach should have resulted in favorable outcomes for advancing the
reforms supported by the Project, the real progress did not fit the initial
plans due to different reasons:
• Disagreement between USAID, Project partners and MoLSA about
Noubarashen reorganization model;
• Delayed reorganization of child care and protection institutions in Lori and Syunik Marzes;
• The Government request that the transitional support to families be provided from the G2G agreement
funds. This was not part of the G2G agreement and eventually state budget resources were committed
once USAID stood firm by its position;
GtG agreements have not been shared
with partners, they do not have an idea of
what is included. But pre-conditions are
good, otherwise the project will turn into
a renovation project. Expert Interview,
Implementing Agency
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
68
• Late signing of the agreement with the MoES.
Understandably, the lengthy negotiations with the Government on these different aspects of G2G funding
had to some extent a negative impact on the Project progress. It should be noted that the Project partners
were not involved in these discussions despite their respective work being affected by the G2G. UNICEF,
however, undertook an intermediary role between the MoLSA and USAID, eventually brokering an
acceptable solution around which negotiations are still ongoing.
With respect to UNICEF and IP contributions to the overall Project funding and resources, as of July 2017
they had collectively committed over 0.5 million USD to the overall Project in addition to the USAID
funding, mostly in the form of human resources and Project support through their ongoing activities (such
as UNICEF’s campaign for inclusion of children with disabilities or WV’s ADP level social work). As of
mid-July, over 50% of the overall Project budget has been spent, with most of UNICEF and IP
contributions committed (see Table 9 below for details).
Table 15: Project Budget and Spending
Total Project Budget $ 3, 481,625,00
UNICEF and IP contribution for Years 1-3 $553,769.00
Total Spent to Date (Sept 2014-July 2017) $1,804, 867.00
% Total Spent against Budget 52%
UNICEF and IP contribution to Date (Sept 2014-July 2017) $ 472,975.00
However, as noted above, most of the tangible accomplishments as of June 30, 2017 were attributable to
BoH’s interventions that were completed in two regions (Lori and Syunik) and had spent its budget of
over 500, 000 USD. Other significant accomplishments include training of over 250 case managers and
social service providers that exceeded the initial target of 145, as well as adoption of secondary legislation
resulting from UNICEF in Armenia’s advocacy efforts and laying foundations for future progress.
Budget wise, CSCF has expended over 74% of its funds by June 2017, with the amount to be almost fully
expended by December when their contract expires. A small unspent portion is due to the pending
assessment of 140 children from two night care institutions in Armavir and one in Dilijan, which did not
happen due to the termination of the Project. SC has frozen the unspent portion of its budget pending the
adoption of the Family Code and subsequent activities, while WV also faces an unexpended budget balance
due to delays in its planned work. By June 2017 UNICEF has spent only 34% of its budget, i.e. $180, 328
with 342, 994 USD still remaining unspent. With respect to this state of affairs, it is worth emphasizing
again that although the Project was designed as a 5-year intervention with very ambitious plans of
activities, the start-off phase was slow, especially with regard to recruitment of project staff, contractual
arrangements with partners, developing MOUs, assessments, getting the Government stakeholders on
board, clarifying their roles and getting their buy-in. It took over one year to bring both the Government
and the project partners on board and fully start the Project implementation.
And finally, one more aspect of Project efficiency is related to how the State budget funds are being
utilized in support of the reforms, more specifically their distribution through the two line Ministries -
MoLSA and MoES. As noted throughout this report, child care and protection institutions and respective
CFSCs fall under the MoLSA, while the special schools (and their spinoff RPPCSs) fall under the MoES.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
69
The assessment of the children from the first type of institutions is emphasizing their social needs, while
the latter is focused on educational needs. However, special schools are also boarding facilities and most
of the residing children come from families with at times complex social and poverty problems, especially
given the fact that the majority is placed there due to social and poverty related considerations. All
interviewed experts agree that there should be one unified model for children/families not to go to
different places for meeting their different needs. However, due to the conflicting interests of the two
Ministries, mostly related to the inflow of funds from the State budget, the two processes continue to be
separate, without much coordination and with use of different approaches. As one of the key informants
noted, “The funds from these sources could come to one place and serve the children in a more targeted
way”.
Child Welfare Reforms
Finding 11. UNICEF and MoLSA have successfully joined efforts to mobilize all ongoing interventions
related to a child’s rights to protection and social systems reforms to support the Project activities and
interventions.
From 2010 onward UNICEF Armenia has been heavily involved in the GoA’s social services reforms
guided by its belief that family vulnerability including issues related to child rights protection should be
addressed holistically by a single system with integrated services. With technical and financial support
from UNICEF, USAID and WB, Armenia is currently implementing comprehensive reforms in the area
of social protection, including child protection. Introduction of the Integrated Social Services (ISS) is the
main pillar of these reforms emphasizing a “one window” model for provision of cash and non-cash
benefits, co-location of different service providers (responsible for pensions, social assistance, employment
and disability certification), introduction of the concept/system of case management and project
management through local social planning efforts. These developments have important implications for
the Project since with thousands of children out of institutions and returning to their biological or foster
families, case managers’ workload is expected to increase. Thus, throughout the Project UNICEF has
emphasized enhancing case manager understanding of the deinstitutionalization processes and their
ability to support children leaving institutions in their reunification with their families and communities
of origin. UNICEF contracted CSCF to train case managers and other social service providers on foster
care and deinstitutionalization responses in general. A thirty (30)-hour training were delivered to over
100 case managers (instead of the planned 45) and 192 service providers (instead of planned 100) in Syunik,
Lori, Yerevan, Armavir and Ararat. The trainings were developed after conducting needs assessment of
case managers (through in-depth interviews and focus groups) and followed by the development and
delivery of the training modules. The insert below summarizes the topics included in the case management
modules:28
Modules Included in Case Management Manual
28 The information source include summary data provided by CSCF based on its reports and UNICEF QR 3 and 4,
2016
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
70
1. Case management in the RA within the context of current reforms;
2. Integrated provision of social services as a modern system and case management as part of it;
3. Deinstitutionalization and alternative care: Role of the case manager;
4. Child protection from violence and referral procedures in the context of the child’s best interest;
5. Specificity of work with children that need special protection and case management.
Besides, 30-hour 5-day Foster trainings to over 78 case managers and service providers (instead of planned
60) were delivered in Syunik, Lori, Yerevan, Armavir and Ararat regions. The training topics included a
more general background approach of care options: institutional care, adoption, kinship and foster care
and narrowing down to the specifics of foster care and the logical steps based on the best interest of the
child. The insert below summarized the topics included in the foster care modules:
Modules Included in Foster Care Trainings
1. Main forms of foster care for children in difficult life situations;
2. Foster Care procedure and stages of implementation;
3. Specifications of working with foster parents;
4. Specifications of care in the context of child rearing;
5. Specifications of care for children with special needs in the context of foster care.
As noted above, in total, 344 families were assessed by case managers in Lori and Syunik and an additional
52 in Kotayk. As a follow up to the needs assessment, case managers have developed individual social
action plans and worked to implement those partnering with WV whenever the involved families have
been eligible for assistance. Some targeted case managers also were trained on assessing foster families,
making decisions on placements and supervising the work of foster families.
The Project interventions allowed building on UNICEF’s other capacity building activities targeting case
managers and mobilizing them around the current deinstitutionalization and IE reforms. As agreed by
experts and stakeholders, case managers have learnt how to follow up the needs assessments and remain
continuously engaged with the families. They have also collaborated with other stakeholders such as the
police, health and education providers, and others, thus adopting the culture of institutional cooperation
for addressing children’s issues and needs. Generally, the case managers’ involvement in the Project
allowed them to apply in practice case management as a tool and understand its uses. It also had some
broader implications since the case managers and other social service providers, as key stakeholders of the
ongoing reforms, improved their understanding of their role thus also mitigating any resistance that
existed. All interviewed case managers that had attended CSCF trainings (5) assessed them as very useful
and of high quality.
Their approach is very interesting…we go to families together, conduct assessments, they guide us how to
assess…We have regular contact, call them with our questions. They are very supportive. In-depth interview,
Case manager
If there were trainings like this, I would like to attend again. Their presentation of how to manage cases was very
interesting…In-depth interview, Case manager
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
71
The competencies and techniques acquired by case managers were successfully tested during the
Noubarashen reorganization. It was an example of a good cooperation model among ISS case managers,
employment services, MOLSA, NGOs, etc. to leverage resources in support of the children being reunified
with their families. In addition to Project-led efforts, it should be emphasized that several stakeholders
noted how effectively MoLSA mobilized case managers to support children released from Noubarashen
who were returning to their communities of origin. Top level ministry officials worked closely with case
managers requiring monthly reports from them on the situation of these children. Both WV and CSCF
also worked closely with the case managers, especially in Syunik, where there is more for support. Some
quotes included below provide evidence to the effectiveness of these efforts.
Ideally, case managers should have been involved earlier, since the children were out of the institutions without
any response in place. However, MoLSA’s close supervision of the process at that point was very helpful. Expert
interview, Implementing Agency
Two-sided assessments were implemented in Lori to understand the placement options for each child, Following
this, the Marz administration decided to visit the families again, and then expanded discussions of cases were
conducted. Expert interview, Implementing Agency
… the Marz administration requested that we report on 30 cases of children that left institutions. We provide
those and know that all 30 children are enrolled in public schools. In-depth interview, Case manage
Despite these significant positive developments, both experts and case managers point to some persisting
issues that needs to be addressed, including:
• Perception of the case managers’ role as mostly that of cash benefit distributor rather than a case
manager since the same professional cannot effectively implement both functions. This issue was
especially visible in Lori, where several stakeholders noted that case managers’ involvement in
deinstitutionalization support has been suspended due to the Marz administration’s position that
those two functions are incompatible.
• The case managers’ involvement in assessing of the biological families of the children, but not
children themselves, which was done by other organizations, as well as limited cases of foster
family placements did not allow case managers to practice their skills with respect to this
alternative services.
• Lack of support services and other resources remains a constant limitation to effective
implementation of the case manager’s function.
However, it should be emphasized that the overall progress with strengthened application of case
management practice is unquestionably positive. It is largely due to UNICEF in Armenia’s support to
reforms and policies with respect to the ISS model and case management in the past decade, and testing
and enhancement of the system through projects implemented by UNICEF. This include the EU-funded
“Mitigating Social Consequences of Labour Migration and Maximizing Migrants’ Involvement in Local
Development” project and this USAID-funded Project. .
As to Territorial Social Plans (TSPs), which is part of the ISS reforms, they were embedded in the 2010 -
2014 key legislation and elaborated in the new Law on Social Assistance due to UNICEF in Armenia’s
effective advocacy efforts. Development of the LSP Concept, Methodology and Financing Models with an
eventual passage of a joint MoLSA and MTA decree on LSP introduction and reflection of related
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
72
provisions in the amended Law on Social Assistance (2014) allowed to formalize this effective vehicle for
identifying service gaps and establishing lacking social services in the communities. However, these plans
should be supported by the adequate financial resources. While the new Law on Social Assistance includes
several provisions related to institutionalization of LSP, provisions for financial resources in support of
social projects are still ambiguous and the State budget as a source of funding is not included. This may be
one of the reasons, why the TSPs were not fully utilized for the Project implementation.
To summarize, UNICEF and MoLSA have consistently and successfully capitalized on other critical
reforms initiatives, namely introduction of ISS and case management to support the ongoing
deinstitutionalization and IE reforms. This approach has been especially effective in the current situation
when the communities of origin face a lack of services and social workers, and case managers have been
closing that gap to the extent possible.
Finding 12. To a certain extent, both UNICEF, WV and CSCF are all involved in the deinstitutionalization
component of the Project. As a result, there is more expertise and targeted efforts invested on the one
hand, but somewhat confusing role distribution and coordination on the other.
UNICEF’s role as the lead agency responsible for the coordination of the Project implementation was
defined in the Project Proposal and further implementation agreements. However, in addition, UNICEF
also undertook several other responsibilities with respect to “the strengthening of the legislative and policy
framework for quality social protection of vulnerable children and their families…assisting MoLSA in
developing the strategy on deinstitutionalization and establishment of alternative services… ensure
capacity building activities for case managers and social workers in all aspects related to de-
institutionalization”29. According to the initial Project setup World Vision was responsible for establishing
alternative community based services (including the CFSC model, costing, staff training, etc.), though as
discussed throughout this report, it eventually became involved in the assessments of children, provision
of assistance, support of families reunited with children, and other activities. FAR’s CSCF that was brought
into the Project slightly later by UNICEF eventually became involved in child assessments, and supporting
case managers’ work with families. Consequently, a number of functions were replicated by these different
organizations. To illustrate:
• WV’s role in developing the model for the CFSC assumed a certain involvement in the
deinstitutionalization strategy and policy including costing of services and financial implications.
However, as the lead coordinating partner, it was UNICEF that was involved in policy level discussions
and communication with the Government partners. The ownership of the model and follow up with
the Government was at times ambiguous leading to hold up of the model for several months.
• WV was involved in the assessment of children in Noubarashen and Vanadzor. Given the delays in
reorganizing these institutions, their assessments became outdated and a need for re-assessment
emerged. While in the case of Noubarashen, WV committed additional resources to this, it is assumed
that in Vanadzor FAR’s CSCF will take over. Interestingly enough, it appeared during the interviews
that WV did not explicitly acknowledge its removal from the assessment process, but according to
29 Project Proposal
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
73
UNICEF the change was due to discontent by MoLSA about the company that WV outsourced in-
depth child assessments to. CSCF’s involvement was added by UNICEF based on the need to involve
case managers in child assessment and referral process, and CSCF’s respective capacity and experience
in this field.
• In Kapan (Syunik), UNICEF has worked directly with the institution through trained case managers
supported by CSCF, who assessed the social needs of the children and developed individual care plans.
However, throughout the interviews and FG discussions, both WV and other stakeholders indicated
their social workers’ involvement in both assessment and support to families.
You do one thing, then time passes, your work loses its relevance, and then the same thing should be done over.
Expert Interview, Implementing Agency
…. We provided case by case response by social workers, developing the individual plans and working with
families. Expert Interview, Implementing Agency
We signed a separate agreement with UNICEF for case manager training and under it we also worked on training
foster families. Expert Interview, Implementing Agency
Interestingly, while WV perceives some of its work under the Project as outside its agreed scope, in reality
the respective activities have been supported through the WV budget lines, including ADP salaries, as
noted by UNICEF in Armenia staff.
Nevertheless, duplication of efforts has definitely resulted in some inefficient use of resources, since while
World Vision has expended its entire budget for in-depth child assessments and even committed
additional resources (around 18,000 USD instead of planned 12, 000), UNICEF has also provided Project
funds to CSCF for similar activities. In addition, there were no clear distribution of roles as to who and
how supporting family integration and later monitoring of the children would take place. While the case
managers’ general role is more of a referral and consolidation of resources, the social workers have the
intimate knowledge of the beneficiaries/clients and are the resource persons available locally to address
day to day issues of children and their families. Under the Project, the case managers were trained to
expand their scope and provide services to beneficiaries under the supervision of CSCF experts, but the
process is ongoing and needs to be further enhanced, especially given that the service area under a case
manager is much larger than that covered by a community social worker. While social service NGOs like
CSCF, COAF and other have a handful of social workers, WV is the only organization that has a large base
of social workers across the country and they became the first point of contact for the reunited families in
those areas where they are present (e.g. Kanaker - Zeytoun district of Yerevan or Syunik Marz).
However, the positive aspect of this situation is that all involved organizations have vested interest in
promoting child welfare reforms and supporting families and alternative services as part of their mission.
Despite the occasional ambiguity of roles and responsibilities, they all mobilize available resources when
needed to serve the child’s best interest. Oftentimes, it is accomplished as part of their ongoing activities
that are not part of the Project, but supports the same goals and objectives (e.g. WV and CSCF activities
through their regular work).
As noted by the benefiting institutions, though the assessment of the children in the institutions was
implemented separately by three-four different actors, the results were triangulated, discussed and
decisions made jointly. To illustrate, the children residing in the institutions and their families have been
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
74
assessed by at least two groups in terms of their social situation, including home visits by the work group
of the Institution, by the staff of the FWCRP department and World Vision (first by the hired
organization, then by the WV relevant staff). In some cases, the case managers were mentioned to be of
great help during the assessments (in Yerevan especially). While assessments were implemented
separately, afterwards join discussions were held and decisions made on each individual case, jointly.
According to the Director of Yerevan CFS Center, the assessment results mostly coincided and were
discussed and as such lead to joint decisions.
Foster Care
Finding 13: The progress and the future of the foster care component of the Project was challenged by the
delays of the approval of the Family Code30 that would enable children in difficult life circumstances to
benefit from foster care services and the fact that in 2017 no funds from the state budget were committed
to increase the number of foster families, thus challenging foster family motivation.
Since 2008, the Introduction of Foster Family Institute has been supported by the RA State Budget, which
annually committed funding for foster care of 25 children deprived of parental care. As part of the Project,
it was planned that 80 children should be placed in foster care. Among those placements 10 would be
children with disabilities and 20 would be placed for a shorter term, with 80% of those children staying
in the same family for at least 6 months. However, new placements did not take place due a number of
reasons:
- According to the Project priority diversified foster placements were to be made for children in target
institutions. During project implementation phase reorganisation/transformation plan included mainly
night care institutions and special schools for children in difficult life circumstances, including those with
disabilities, whereas the acting Family Code enabled foster care placements only for children with legal
status of “deprived of parental care”.
- In 2017 as well, no additional funds were committed to increasing the number of foster families, which
affected the Project efficiency since the foster families’ trainings happened in 2017 without any immediate
placement of children in families.
However, some positive developments are expected in this regard in 2018. The 2018 Child Rights
Protection Program approved by the Government on September 29, 2017 plans to increase the number of
children in foster families to 120. The State Budget will fund these items of the Program by committing
166343.5 thousand drams. As one of the program experts noted, “we have identified and trained families,
but they are not being paid. USAID/UNICEF could have supported these families before the allocations
from the state budget were made”.
It should be noted that the approval of the Family Code is essential in this respect since the proposed
amendment will allow also qualifying children in difficult life circumstances for foster care thus increasing
the potential number of children to be placed. However, despite some progress in this direction, the
passage of legal documents is still pending31. As noted above, on March 2016 the RA Government approved
30 See the introductory note above.
31 As the Introductory Note to this report specifies, the package of legal documents related to the amendments of the
Family code was submitted to the National Assembly in November 2017.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
75
the Concept for Placing Children in Difficult Life Circumstances in Foster Care. The Concept outlined
those amendments that should be made to existing legislation, including those to the RA Family Code,
Civil Procedure Code, Administrative Code, RA Law on Protection of Child’s Rights and others. In July
2017, the RA Ministry of Justice has circulated the proposed changes on the e-draft platform. The essence
of proposed changes is summarized below:
• While according to the existing legislation only children deprived of parental care can be placed in
foster care, the amended Family Code will make children in difficult life circumstances eligible for
foster care starting from 2018. Under aged single mothers could also be placed in foster care.
• Training will become mandatory for foster families.
• The following 4 types of foster families will function- specialized (with more intense capacity building
planned for this group), crisis response, vacation, and general. In addition, age limitations are also set.
Thus, none of the foster parents should be older than 55 at the moment of placement, while the age
difference of foster parents should not be less than 18 years older than children under their care and
more than 50.
Monitoring of the foster families is also in the focus of these legislative amendments. If currently the
Guardianship and Trusteeship bodies are responsible for the selection and monitoring of foster families,
the proposed amendments transfer this authority to Marz administrations, Yerevan municipality and
Territorial Office of Social Services.
To summarize, significant efforts were invested by the Project to ensure that legal changes required for
the effective functioning of the foster care are approved and enforced. However, the delays in the passage
of the overall legislative package are currently challenging the real functioning of the system and
demotivating the potential foster families that underwent training and got accreditation as foster families,
are factors that are starting to have negative impact.
Inclusive Education
Finding 14: Despite significant knowledge about inclusive education, somewhat favorable perception by
schools and the entire community, lack of special conditions and resources needed has been voiced as a
critical challenge to implementing inclusive education. Additionally, the changes in the financing of the
system also affect its efficiency and effectiveness.
While the soft interventions under the Project have proceeded intensively and effectively, the physical
improvements of the schools involved in the IE system envisioned by the G2G agreement have not
followed. Several informants at policy making and school level note this as one of the key challenges to
inclusive education since the lack of ramps, special toilets and other aspects of physical environment
diminish accessibility of schools. In some of the schools there are no vacant classrooms to be designated
for working with children with SEN, or the needed items in such classrooms are missing (e.g. the mirror
for the speech therapist).
In addition to physical resources, lack of specialized professionals has also been noted as one of the
impediments to implementing inclusive education. There is a confusion about the multi-disciplinary team
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
76
and the teacher assistant. Some of the informants believe that the existence of the team was a good resource
for the IE, while relying only on teacher assistants is not sufficient since they are not involved in
assessment of children, which is supported by the RPPSCs and are working with a very large number of
children.
With respect to RPPSCs, their work is considered useful and supportive. In Syunik, the RPPSCs are fully
functional, involved both in the assessment of children and support to schools. In Lori, Spitak RPPSC is
more advanced than Vanadzor, which is still not fully operational. The average value of the collaboration
of schools with these centers (RPPSCS) is 3.9 (out of 5) based on the survey results. The RPPSCs are mainly
involved in needs assessment of children with SEN, supporting the schools in assessment and in working
with the children. The RPPSC staff should visit the schools to provide assistance, but they do not have
sufficient transportation means (only one car) or are under time pressure. As one of the Center heads
noted, “we did all of this in the past as well. Now our staff is to go all around the region to do the work,
but in the past, they were all in one place. That was easier and more effective”. However, it should be
noted that timetables of activities for children with SEN were available in all schools and they included
the center interventions as well (based on observations).
The RPPSC staff should also provide services in the centers, but several of them claimed that the former
special schools are not adapted for that. Some opinions were also voiced about the accessibility of the
centers since they are in the locations of the former special schools, which were intentionally put outside
the central areas. However, with current functionality, the centers should be easily reachable by the
beneficiaries, both children and professionals.
The Centers also lack specialists, such as speech therapists, special educators, and psychologists. Many of
RPPSC staff are now involved in distant learning at the Armenian State Pedagogical University or other
pedagogical universities to gain this new qualification, since in the past they used to be subject teachers at
special schools.
There is a dire need for specialists, we deal with specialists with no relevant education (psychologists, social
educators) …however, according to the Government decision they should have University education in that area.
In Goris, when they realized how serious the issue was, they opened a graduate program at the University to train
people…In-depth interview, State Agency
And finally, throughout interviews and discussions concerns were raised with respect to financing
mechanism of the inclusive education system, which has affected both the motivation for implementing
it and the involvement of the specialists. In the past, when inclusive education was the focus of the
reforms, the school was financed by the number of assessed children with SEN receiving several times
more per-child funding for these students. However, since the introduction of the concept of the overall
inclusiveness, only children with mid- to a high degree of problems receive additional funding. These
changes also affect the availability of the specialists. If under the first model the multi-disciplinary team
consisting of different specialists was required to support inclusive education, currently the institute of
teacher assistants is introduced. As discussed throughout the report, one teacher assistant is covering over
100 children without real ability to focus on their issues. Consequently, the motivation to implement
inclusive education in different regions varies. Thus, in Lori where they just introduced the system and
are using the old formula, the school principals are motivated to have more children with SEN assessed
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
77
and identified. In, Syunik, which has shifted to overall inclusiveness, the attitude is different, there is less
interest to work with the RPPSCs on initial assessment.
One of the worst things that could be done is that the multi-disciplinary team is not financed because of introducing
the teacher assistant. It is assumed that the assistants should be specialists like speech therapists or special teachers,
but the position description does not require this…in the regions the lack of specialists is overwhelming Expert
Interview. Independent Expert
To summarize, efficiency of IE interventions within the project is directly related to additional injections
into the system to support the ongoing process. The Project has successfully utilized its funds to train
thousands of professionals and establish RPPSCs as a resource base for IE implementation. As a result,
positive perceptions of inclusion of children with SEN and knowledge of basic IE concepts are becoming
integrated into the educational system. However, all these achievements are not sufficient for successful
implementation of the IE. Additional investments are needed in the physical infrastructure (through G2G)
and further capacity building of professionals involved both at schools and RPPSCs. State funding
mechanisms should also provide financial motivation for the schools to implement the system. Thus,
ensuring full efficiency of the IE component is dependent on a good balance between soft interventions
and infrastructure improvements, donor assistance and state commitment. All these factors are also critical
for further sustainability of IE integration into the mainstream education system.
SUSTAINABIILTY32
Finding 15: Sustainability of the Project interventions in support of reforms efforts depends on a number
of overarching and interlinked factors ranging from availability of alternative care mechanisms to capacity
of the professionals involved in delivery of services to public awareness and others. While the Project has
successfully established preconditions for some, others still need to be initiated and implemented.
Sustainability guarantees of these multiple and complex interventions are equally diverse and manifold.
They include laying down the foundations of the prerequisite legal framework, availability of financial
resources, availability of professional human resources to implement the reforms at local levels such as
social services, schools, PPSCs, others, availability of alternative care services, specific guides and
regulations about service operations, public awareness and acceptance of reforms, etc. Some of these
sustainability triggers have been incorporated into the initial Project design and were more or less
successfully ensured during the past 2,5 years, including:
• Advocacy with the Government of Armenia and the line ministries to adopt much the needed
programs, normative acts, decrees and decisions that will lay the foundations for pushing the reforms
forward. 2014 was a turning point in this respect since both Amendments to the Law on General
Education and the new Law on Social Assistance were adopted laying the legal foundations for
32 This finding covers both general Project related and more component-specific sustainability aspects of the Project.
Given the current state of the progress under the Project, sustainability under IE and Foster Care is combined with
findings on efficiency, while in case of deinstitutionalization, there is no standalone sustainability finding given how
its different aspects are considered within this finding.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
78
inclusive education and introducing the key elements of the Integrated Social Services (such as case
management, local social planning and institutional cooperation among social service providers).
These key developments were to a great extent attributable to UNICEF’s longstanding efforts in both
directions through its Education and Social Protection programmes.
As discussed in several parts of this report, despite some challenges with respect to Family Code and
reorganization of Lori and Syunik Marzes child care and protection institutions, several important
normative legal acts were passed by both the Government and the two line ministries in support of
the reforms. These included the 2017-2021 National Strategy on Child Protection, the Action Plan
and Timetable for Provision of Total Inclusive Education System, The Concept for Foster Care of
Children in Difficult Life Circumstances, several decrees and decisions issued to implement institution
reorganization, to approve methodologies, etc., relevant to inclusive education.
• Capacity building of different groups of professionals has also been an important aspect of almost all
Project components, including the planned training of reorganized institution staff (which happened
in September-October 2017), training of foster families, training of the staff of special schools and
RPPSCs, large scale training of teaching staff, training of social service providers, including case
managers, etc. However, despite involving these different groups and thousands (including school
teachers) of professionals in capacity building activities, these interventions were mostly one time
and/or short term in nature, whereas long term consistent efforts are required to change the attitudes
and professional behaviors of these specialists.
• The situation is less favorable with respect to availability of alternative services, especially at the
communities of origin of the children leaving the institutions. This aspect of the Project, as discussed
throughout the report, has been one of the most problematic, especially with respect to social services.
While Noubarashen institution was reorganized into a CFSC (2016), the transformation of Syunik and
Lori institutions is still pending. With respect to Noubarashen, several of the children that left the
institution returned to their communities of origin, where service provision is almost nonexistent. As
the WV representative put it, “If WV does not work there, then no support is provided to them. WV
does some work in Lori and Syunik Marzes, but children return to different places and in this case,
they should integrate both into the family and the education system.” To address the above mentioned
situations and to overcome the existing service gaps, UNICEF mobilized its local and international
partners (such as SOAR, SOS-villages, CSCF) and the ISS case managers to provide the needed support
to children reunified with their families.
• Well-developed and approved regulations, guides, manuals and other tools are also an essential part
of rolling out new systems and services that provide consistency and direction to reforms
implementation by relevant institutions/players. While the reorganization decrees set out the
functions of new service centers or RPPCs, there is still a lack of specific guidelines and methodologies
as to how these new functions should be implemented. To illustrate, while several teachers raised the
issue of not knowing how to work with individual cases/children, NIE experts pointed out that they
had sufficient understanding of the general approach to tackle each child specifically. In case of foster
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
79
care, comprehensive guides and manuals have been developed, but their application is still pending
on the adoption of the amendments to Family Code33.
With respect to financial resources, as noted above, the approach that combined the Project and G2G
resources has been perceived as an effective mechanism for ensuring that the Government reforms have
sufficient donor support. Once these funds are provided and are matched with Project interventions, the
reforms progress should be more visible. In addition, UNICEF has provided technical assistance to the
Government in costing the reforms and redirecting funds from reorganization of institutions toward
supporting families and establishing alternative care services.
And finally, there are two aspects of the reforms that, as agreed by interviewed experts, need significant
improvement if sustainability of reforms is to be ensured. These are 1) public awareness building about
the reforms and 2) working with parents (both of the children to be reunified with their families and
larger parents’ groups affected by the introduction of inclusive education into mainstream public schools).
There is a shared belief among donors, implementers and experts that the Project has not emphasized
these aspects of the reforms despite their importance for the success of the Project and the reforms. Some
quotations and reflections on these issues are presented below.
The Education component of the current project hasn’t foreseen awareness rising, though it is a very important
aspect to remove the invisible barriers at schools for children with disabilities. BoH Final Report
On the 6th day of the in-service inclusive education training School the principal/teacher will conduct an expanded
seminar for the parent community at the school and introduce the overview of inclusive education. BoH Final
Report
…there should be daily engagement with the community, awareness building, but no one is doing that. Smaller
efforts, but not for the bigger reforms. Neither the staff, nor the families know what is happening. Expert Interview,
Independent Expert
… We identified potential foster parents, introduced them to regional authorities, but they did not
support these parents. Expert Interview, Implementing Agency
One of the weaknesses of the Project is that public awareness is not included in the Project. Expert Interview,
Implementing Agency
CROSS CUTTING ISSUES
Finding 16. While the Project has emphasized the gender aspect of its interventions requesting data
disaggregation by gender and general sensitivity to gender issues, there seems to be no visible differences
in the way deinstitutionalization or inclusive education is affecting boys and girls. However, with respect
to the specialists involved in both reforms, female professionals have a prevailing presence.
At the stage of the Project Proposal, UNICEF in Armenia and partners’ joint analysis indicated that” the
institutionalization affects both boys and girls and their number in institutions is generally equal. The only
exception is that the proportion of girls with disabilities to boys in specialized orphanages (55% boys with
33 As noted in the introductory note to this report, the amendments to the Family Code were submitted to the
National Assembly in November 2017 and adopted in December 2017.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
80
disabilities vs 45% girls) is higher than the same proportion in the families (68% boys with disabilities vs
32% girls).”34 Table 16 below confirms the initial assumption that the number of boys who have been
certified for SEN is slightly higher. However, throughout the evaluation no obvious differences in how
deinstitutionalization and inclusive education reforms is affecting boys and girls were observed.
Table 16: Assessed Children by Gender
Number of
assessed
children
Total No.
of children
No of
Certified
children
according
to 2017
First Level assessment Size of Support
No SNE Have SNE
Mild Modera
te Severe
Compoun
d
Girl
Bo
y Girl
Bo
y Girl
Bo
y
Gir
l
Bo
y Little
Modera
te
High
Freque
nt
Total or
Regular
Number of
First level
assessed
children
640
67
0 75
12
1 11 12 64 106
1310 196 23 170 41 34 34 57
Number of II
level assessed
children
Total No.
of children No SNE Have SNE Mild
Modera
te Severe
Compoun
d
38 57 0 2 37 56 5 8 13 17 14
2
9 4 3
95 2 93 13 30 43 7
Source: BoH Final Report
With respect to inclusive education, as both quantitative and qualitative research showed, adaptation of
girls and boys involved in the inclusive education system is conditioned mostly by the issue they are facing
regarding SEN and not primarily by their gender. There were some opinions voiced during the qualitative
interviews that girls with issues had more complexes and found it difficult to overcome their situation or
that sometimes parents of girls are more inclined to avoid assessment supposedly not wanting to stigmatize
their children. However, as already noted, the informants always emphasized the fact that that seriousness
of the condition faced by the child is the main factor affecting adaptation rather than gender.
One gender-specific attitude toward children was expressed during FGs with foster parents who indicated
that it is preferable to adopt girls since there is the perception that girls are easier to deal with.
However, the situation is quite different in case of professionals working with children, the predominant
majority of whom are women with few exceptions where men are heads of RPPSCs or public schools. This
situation is indicative of the general trend in Armenia’s education sector. Thus, 92.3 % of all surveyed
teaching staff members are female with only 7.7 % being male (mostly PT teachers).
34 Project Proposal.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
81
Similarly, the staff of child care and rights protection institutions are also mostly female. It has also been
observed during the visits to these institutions that it is mostly female family members (mothers and
grandmothers) that are accompanying children to the CFSCs, while fathers are present when the child
with disability needs physical support.
An interesting gender trend is observed in case of the foster care generally. Since it is not required that a
foster family comprise of both a father and a mother, in those cases when the foster care giver is not a full
family, it is almost always a woman and these are the majority of cases. However, there are cases when
two women are registered as a foster pair, and very few cases when a male is foster parent without a
woman pair. Chart 4 below summarizes the gender dynamics under foster care.
Chart 4: Gender Distribution across Foster Families
Source: List of trained foster families provided by SC/CSCF
However, it should also be noted that since only one of foster parent’s presence at trainings is sufficient
for qualifying the family for foster care, most of trained foster family care givers were women.
30
1
19
2
Female Male M/F F/F
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
82
CHAPTER IV: CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS
CONCLUSIONS
Over the last two decades UNICEF and development partners have successfully advocated with the
Government of Armenia for the child’s right to live in family environment as being in the child’s best
interest. Several study visits, expert and situational analysis/assessments, costing analysis with respect to
the financial burden that institutions present for the state budget and other efforts have resulted in the
Government of Armenia’s declaring deinstitutionalization reforms a priority back in 2011. From there on,
momentum for the reforms was irreversibly established as evidenced throughout this report by
referencing several key policy and legal documents (including a number of decrees and decisions for
reorganizing specific institutions). Most importantly, several key national programs and strategies such as
the National Strategy for Child Rights Protection 2013- 2016 and 2017-2021, the Government Program
for 2017-2022 and Human Rights Action Plan for 2017-2022 further consolidated commitments to
transforming the child welfare system and prioritizing de-institutionalization reforms.
Similarly, following the amendment of the Law on General Education in 2014 and the approval of the
most recent State program for Education Development 2030, inclusive education has become one of the
key priorities of the Armenian Government.
Both reform directions have been supported through several pilot initiatives helping the Government to
observe and assess the practical implications and implementation modalities of the reforms.
Thus, by 2014 when the Project was being designed, the momentum for the reforms in both directions
was irreversibly established and the Project intended to support and sustain that momentum.
As emphasized in this report, the Project almost identically mirrored the key aspects of these two reforms
directions by focusing on improving the pre-requisite legal framework, supporting transformation of
existing institutions and establishment of alternative services, engaging into capacity building of
professionals and specialists required for implementation of the reforms and other relevant efforts. Each
of Project components was tasked to a competent organization with reputation and experience of working
in the relevant areas (World Vision, Bridge of Hope, Save the Children, FAR’s Child Support Foundation)
with UNICEF acting as the lead partner and coordinating all Project efforts. However, due to this initial
distribution of these roles and responsibilities, coordination between partners and UNICEF as well as
UNICEF and USAID, and the Project and Government partners has been at times challenging and
ineffective.
Two key considerations are:
• Inclusion of two major reforms directions (with their different aspects) into one Project complicated
coordination efforts. While both areas of reforms deal with child’s rights and transformation of
institutions involved in child care and protection, they have different stakeholders and beneficiaries
both at the Government, regional and local level (MoLSA and MoES, child care institutions and special
schools, alternative services and public schools, different Marz departments, etc.).
• UNICEF’s role is to be an advocate for policy making in the best interest of children. However, due to
the complexities of the Project, there were at times some confusion around the role of UNICEF as the
coordinator and implementer of the Project. This situation is most visible in the relations between the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
83
Government, USAID and UNICEF, where the Project implementation was in parallel with the
interests of G2G support between USAID and the Government, but perceived as the same process and
not two different relationships between the three parties, which at times led to misunderstandings
about who was accountable for what.
The Project has reached some key results including reorganization of a number of institutions (both child
care, child rights protection and special schools, establishment of one CFSC and 4 RPPSCs, needs
assessment and placement of hundreds of children), training of thousands of teaching staff under the IE
component, as well as other professionals, development and endorsement of legal acts, etc. However, there
have also been pushbacks such as the delays with the transformation model in Noubarashen, no visible
progress with setting up of alternative community based services, delays with the amendments to the
Family Code and subsequent challenges with the placement of the children in foster care, and others.
While some of these interruptions are attributable to the Government’s uneven actions with respect to
their reforms agenda (i.e. situation with the Family Code)35, others have been due to the internal Project
dynamics (i.e. non-clarity with child assessment and follow up family support. PSAs and documentary on
foster care were produced but not publicly released yet, upon the decision by all parties to go public once
the Family Code is adopted).
The Project has also been at disadvantage given the non-inclusion of a rigorous national-level public
awareness campaign in all its directions, i.e. deinstitutionalization and foster care as well as inclusive
education. The public at large is mostly uninformed about its general goal, objectives and directions,
though efforts have been made to inform and engage key stakeholders and at times beneficiaries through
local meetings and forums.
The future of the Project interventions is very much dependent on the availability of a professionally
trained cadre of specialists to implement the reforms locally as part of newly established services (CFSCs,
RPPSCS) and those supporting inclusive education in mainstream schools. While the Project has designed
and implemented capacity building interventions, they are mostly one time and short term in nature
which is a concern expressed by different Project stakeholder groups in terms of continuity and quality.
Overall, given the complex and sensitive nature of the reforms supported by the Project, more time is
needed for effective implementation of some or its aspects (such as establishing CFSCs and making them
operational, putting in place professionals, etc.). As the situation with inclusive education component
indicates, the fact that Bridge of Hope has been involved in similar interventions for almost over a decade
and is replicating its pilot project at a larger scale through the current Project, the implementation of this
component turned out to be more comprehensive and effective.
And finally, while assessment of the Project impact was not directly targeted by this evaluation given its
focus on mid-term results, it should be noted that even at this point there is confidence about the impact
of the Project interventions, especially with respect to legal foundations and strategic priorities of the
reforms. As discussed throughout the report, adoption of several decrees, concept papers, State programs
coupled with informal statements have cemented the commitment of the RA Government to the ongoing
reforms. Specific steps have been taken to close or downsize institutions (both child care and special
schools) and reunite thousands of children with their families. As such, there is confidence among
35 See the introductory note above.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
84
stakeholders that deinstitutionalization and inclusive education will be further implemented in Armenia,
in the best interest of the children affected. As overwhelmingly agreed by the evaluation informants,
UNICEF Armenia’s role in this process has been instrumental.
LESSONS LEARNT
Some lessons learnt that can be derived from the result of this evaluation include the following:
• Putting two reforms and directions under on one Project umbrella has complicated the Project
structure and coordination creating pressures with respect to communication with partners and
stakeholders, involving them in advocacy in different areas, undertaking responsibility for ensuring
results in too many directions.
• UNICEF in Armenia’s role as a major international organization involved in policy advocacy on child
rights protection is very important and as much as possible should not be mixed up with the role of
implementer, thus challenging its involvement with national policy makers as a policy leader and a
watchdog.
• A national level public awareness campaign should have been incorporated in the design of the Project
and its standalone components especially given its focus on sensitive reforms touching upon thousands
of lives, including children and their families, as well as requiring involvement and coordination of
different stakeholders from different levels in society.
• Acceptance of a model for reorganization of the institutions should have been more rigorously
pressured and promoted to the Government of Armenia allowing for a specific approach to closing
down the institutions and establishing alternative services across the Project-targeted institutions.
Similarly, community based alternative services should have been established in the communities of
origin of returning children. UNICEF in Armenia could have worked more intensively with the NGOs
present in the regions to involve them in the follow up of the reunified children and their families (as
was the case with SOS villages, which was mobilized at later stages of the Project to support its
implementation throughout its ongoing activities).
RECOMMENDATIONS
Recommendations included in this part of the report build up on the Findings and Conclusions resulting
from this evaluation and are also grouped by relevant evaluation criteria.
Relevance
1. USAID as the Project donor and UNICEF as the lead Project partner should continue their efforts to
bring to fruition the Government’s reforms with respect to child welfare and inclusive education.
While currently the Project serves as the main locomotive in support of the reforms agenda, and as
such is important, UNICEF in Armenia should also prioritize its policy expertise in terms of child
rights protection and use all available channels to build on the current momentum for these reforms,
which was to a large degree achieved due to UNICEF in Armenia’s continuous efforts over the years.
Effectiveness and Efficiency
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
85
2. If possible, as a tactics moving forward, USAID should consider separating the two reform directions
(child welfare and inclusive education) into two standalone projects while UNICEF’s relevant sections
could take over responsibility for each direction (i.e. Education section taking over Inclusive
Education given its longstanding experience with work in that area)..While the current structure has
been designed to ensure harmonization of the two reforms and their key directions, coordination
across Government partners, implementing partners, and different stakeholders has been challenging.
If the separation is still under the auspices of UNICEF, the harmonization could be furthered.
3. In the future, while undertaking funding of a Project in support of ongoing reforms, USAID should
consider a more structured approach, especially when tying soft interventions to G2G support. One
alterative is starting small and laying the foundations of a bigger project, then expanding it and adding
budget support components once confident about the progress. In case UNICEF undertakes the role
of a Project implementer, it should prioritize pilot projects that showcase the benefits of the specific
approaches of reforms implementation and serve as a model to be replicated by the Government. To
illustrate, with the current child welfare and deinstitutionalization reforms, there seems to be a need
for a reorganization and transformation model and focusing on establishing and making functional
one CFSC could have served as such a pilot. Another alternative is piloting all possible child and family
support services in one region, including CFSCs, RPPSs, strengthening case managers, social workers
and other social services to demonstrate the interconnections between these key institutions and
services and demonstrate how effectively their cooperation could be shaped in support of children in
difficult life circumstances.
4. All Project stakeholders, including USAID as donor, MoLSA and MoES as well as UNICEF should be
aware of any duplications between the roles and responsibilities of different implementing partners
and should define the tasks of partners based on their core competencies. More involvement from the
implementing partners in policy related communication with their respective Government
stakeholders could also result in improved Project coordination.
Sustainability
5. As noted throughout the report, sustainability of the Project efforts is very much dependent on
availability of resources and such factors as availability of community based family support services,
professionally trained specialists, guidelines and manuals in support to service provision and others.
This said, all key players in the reforms, including the Government of Armenia, USAID, UNICEF and
other implementing partners should use available resources and capabilities to ensure these critical
gaps are closed. While each player may utilize different models, methodologies and mechanisms to
ensure further sustainability of the reforms, coordination of efforts will remain a critical priority.
6. In the future, USAID, UNICEF and IPs should consider including a national level public awareness
campaign in similar interventions that support large scale and sensitive reforms. Given UNICEF’s
competencies with respect to designing and implementing such campaigns, it can become one of the
key intervention areas led and managed by UNICEF.
7. Both the Government of Armenia and UNICEF should continue efforts directed to capacity building
of professionals needed for the successful implementation of this reforms both through the Project
and any other channels/initiatives available. In this respect, it is important to identify the key partner
organizations that should be strengthened to support child welfare reforms. While the IE and BoH
experience provides a good example of how the NIE and the Republican PPC became key Project
partners involved in both teacher trainings and capacity building of RPPSC specialists, the role should
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support
and Inclusive Education Services as part of Child Care Reform” project
86
be taken over by a State agency and not fall to an NGO active in that area. With respect to the child
welfare reforms, MoLSA should not take over this function given its role as a policy setter but the
National Institute of Labour and Social Research could be considered as a possible candidate for this
role.
8. MoLSA and UNICEF should continue using case managers in support of the reforms implementation
and enhancing their professional experience and resources to work further with children reunified
with their families and other children in need of support. Community social workers should also be
brought in to support both child welfare and inclusive education reforms implementation. While a
case manager’s involvement in the ongoing Project and reforms is commendable and necessary, they
are not present in the communities on daily basis and are mostly involved in referrals. Similarly,
regional/Marz administration level departments and their staff are focused on decisions in regards to
placement, documentation processing and other related activities. This creates a gap of specialists on
the community level, and social workers (as evidenced by WV model) could become that important
link in the reforms implementation.
9. Working with parents and children should be an important part of reforms that are touching their
lives and their future. As such, the Project interventions would have benefited from more intensive
engagement with both groups. While this can be partially achieved through a public awareness
campaign, targeted activities are also needed.
And finally, the Government of Armenia (both MoLSA and MoES) should undertake a more robust
leading and coordinating role with respect to all aspects of the reforms, ranging from legislative changes
to capacity building to coordination between different stakeholders including government partners,
donors and implementers. Timely and relevant delivery of all the inputs required for moving forward the
reforms process should be ensured by relevant stakeholders, including adoption of decrees and
commitment of state budget resources (e.g. for foster care or IE) and be responsible and accountable of the
overall reforms outcomes, both positive and negative.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 1 TERMS OF REFERENCE
1. Program Information:
Program (Outcome WBS & Name): 0260/AO/05/882. Outcome 2. Disadvantaged families
Project (Output WBS & Name): 0260/AO/05/882/005 Output 2.5. Family Environment
Activity (Activity WBS & Name): 0260/AO/05/882/005/001 Activity: 001 – Child Care System Reform
UNICEF is mandated by the United Nations General Assembly to advocate for the protection of children's
rights, to help meet their basic needs and to expand their opportunities to reach their full potential. The UNICEF
staff and experts/consultants should act in accordance with the UN Code of Conduct and UNICEF Mission.
2. Background, Object and Rationale of the Evaluation:
While Armenia ratified the CRC in 1992, which stipulates the right of the child to live in a family environment,
the country remains significantly reliant on institutional care for children in difficult life circumstances. In 2009,
the Guidelines for Alternative Care of Children welcomed by the UN General Assembly were adopted to
implement care provisions as per the CRC that strongly recommended for children to be brought up in family
environment, and clearly stated that socio-economic status of the family should not be a reason for separating
the child from his/her family.
The overarching goal of the Child Welfare Reform is to ensure that the child care system in Armenia
significantly reduces reliance on large-scale institutions through establishment of a network of community and
family based services responding to the needs of children and their families. UNICEF Armenia, with USAID
support is implementing ”Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family
Support and Inclusive Education Services as part of Child Care Reform” program in partnership with Bridge of
Hope NGO, Children’s Support Center Foundation, World Vision Armenia office and Save the Children. It
supports the Government of Armenia to achieve this goal through the following objectives:
Institutional mechanisms for the functioning of social service system are set up;
Alternative community-based family support services are accessible to vulnerable children and
families;
Family substitution service system is strengthened with the diversification of types of foster care,
establishment of monitoring and evaluation system;
Inclusive education system is strengthened to provide quality learning for all children through set-
up of pedagogical-psychological support centers and mainstream school trainings;
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
An enabling legal and regulatory framework is established to support child-care reform;
Social norms regarding children with disabilities have changed favorably towards inclusion.
The project is subject to mid-term evaluation in accordance with the AID-111-IO-14-00004 agreement with
USAID. The mid-term evaluation is to be conducted for the period of October 2014 until June 2017. Provided
that the duration of the project is 5 years and the closing date for the Grant Agreement is 09/08/2019, the mid-
term review of the project is due.
Mid-term evaluation will assess the progress in meeting the project goals and outcomes (as defined in the M&E
Plan). They will provide early lessons learnt and recommendations as well as identify significant discrepancies
between expected results and actual achievements, including an analysis behind the reasons for discrepancies
between actual and projected indicators, if any.
The primary users of the evaluation are UNICEF (Country Office, Regional Office, Headquarter divisions),
USAID and Government counterparts (primarily Ministry of Labour and Social Affairs, Ministry of Education
and Science, Ministry of Territorial Administration and Development, regional administrations in target areas)
as well as Implementing partners (“Bridge of Hope” NGO, “Children's Support Centre” Foundation, Fund for
Armenian Relief, Save the Children, World Vision Armenia). Other national and international partners,
primarily those involved in the field of child and social protection in Armenia are also seen as part of the
audience of the report and key stakeholders. Evaluation findings, conclusions and recommendations will be
made available for the use of above-mentioned audience for further consideration of better responding to the
needs of vulnerable children and their families in Armenia. In addition, other UN agencies working in Armenia
may use evaluation findings for their strategic planning and/or review processes.
In sum, the evaluation will inform both the UNICEF and USAID, and other stakeholders on further actions in
the areas of child protection within the context of deinstitutionalization reform.
3. Purpose and Objective
The overall purpose of the Mid-Term Evaluation of the project is to assess the relevance, effectiveness,
efficiency as well as impact and sustainability of interventions under the “Toward Social Inclusion of
Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of
Child Care Reform” project.
The specific objectives of the mid-term evaluation are to assess the relevance, effectiveness, efficiency, impact
and sustainability of project interventions in the following directions:
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Family and community based alternative services and their role in prevention of institutionalization;
Institutional mechanisms for the functioning of social service system;
The process of strengthening the family substitution service system with the diversification of types of
foster care;
The process of strengthening the inclusive education system to provide quality learning for all children;
Legal and regulatory framework developed to support child-care reform;
Institutional cooperation and coordination between implementing partners/key actors (both government
and donor community) in responding to vulnerable families and children in the frames of the project
(including financial);
Coherence and ordination and compatibility of the congruence between six objectives/components of
the project to meet the overarching goal.
4. The Scope of the Evaluation and Limitations
Given that this is a mid-term evaluation, two of the evaluation criteria this project will be evaluated against,
namely impact and sustainability, are to be considered to the extent possible.
The geographical coverage of the evaluation includes Lori and Syunik Marzes (regions) and capital city
Yerevan. The evaluation will primarily focus on the beneficiaries of various project interventions that will
include beneficiary families, teachers, case managers, local, regional and national authorities.
The main guiding document for the evaluation will be the Project Monitoring and Evaluation Plan, which states
the logic of interventions with clearly set objectives, results, activities and corresponding indicators on output
and outcome levels, means of verification, risks and assumptions. These will be thoroughly reviewed (in terms
of validity and limitations) by the independent evaluation team during the inception phase.
The limitations of the indicators are conditioned by the fact that some of them need to be viewed together with
the identified risks and external conditions, as stipulated in the M&E plan. The M&E plan also includes data
limitations such as absence of common definition of the parameter of “a reform”, program self-reporting being
a subject to reliability and accuracy, etc.
Taking into account that the evaluation is a mid-term and will be followed by the summative evaluation at the
end of the Project, this evaluation needs to ensure the appropriate quality and comparability of baseline data,
including disaggregation by gender, age, income quintile, disability status, etc.
The evaluation will be conducted by a team of independent evaluators (an organization) in close cooperation
and coordination with UNICEF in Armenia, USAID, as well as relevant Ministries and Implementing partners.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
5. Evaluation Questions
Core evaluation criteria such as relevance, effectiveness, efficiency, sustainability and impact (the latter - to the
extent possible) will be analyzed. Key evaluation questions will include but are not limited to the following:
Relevance
Are the project activities/components relevant to the actual/defined needs of the vulnerable families
with children? Were the objectives clear and feasible? Do the main components of the project contribute
to the planned objectives and logically interlinked? How well are they linked to each other?
Is the project in line with the current priorities of the country? Is the Government committed (both in
terms of timing and financially) to the project? How the project is aligned with and supports the national
development plans, strategies and national plans of action?
Has the project involved relevant stakeholders through consultative processes or information-sharing
during its preparation phase? Was the needs assessment/analysis carried out at the beginning of the
project reflecting the various needs of different stakeholders? Are these needs still relevant? Have there
any new, more relevant needs emerged that the project should address?
To what extent were the reform initiatives informed by the needs and interests of diverse groups of
stakeholders (also from gender equality perspective)?
What are the beneficiaries’ views about the services provided? What are their views about improving
services (if there is a need to improve)?
Effectiveness
How effective has the project been in establishing ownership by the stakeholders? Can the project
management and implementation considered as participatory. If yes, does this approach contribute to
achievement of the project objectives? Has the project been appropriately responsive to political, legal,
economic, institutional (and other) changes?
Is the project making sufficient progress towards its planned objectives? What are the key achievements,
challenges and implementation lessons? Will the project likely achieve its planned objectives upon
completion and demonstrate success?
Are the strategies appropriate and effective to achieve the planned objectives? What, if any, alternative
strategies would have been more effective in achieving its objectives?
How effectively the family support services are provided in regional (and community) level and how
they are in line with actual needs?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Efficiency
To what extent has UNICEF and implementing partners made good use of the human, financial and
technical resources, and has used an appropriate combination of tools and approaches to pursue the
achievement of project results in a cost-effective manner?
Was there a clear distribution of roles and responsibilities of key actors involved, including staff,
implementing partners and governmental partners towards the achievement of project objectives?
To what extent budgetary allocations by the donor between the partners were efficient? To what extent
did UNICEF and implementing partners capitalize on other complementary initiatives to the project to
reinforce the results of the project?
Has the project done appropriate financial planning and reporting? Has the management of funds (and
reporting) been done in an efficient manner? Has the co-funding/contribution from different project
partners been made at the expected level? Are there established procedures for checks and controls in
the day-to-day financial management, procurement of good and services and decisions regarding
resource allocation for the project?
Have project funds and activities been delivered in a timely manner? If not, what were the bottlenecks
encountered? Were the procedures put in place for staffing, systems (administration, accounting, other)
and reporting efficient?
Sustainability
To what extent have the UNICEF country office and project partners been able to support the
government and beneficiaries in developing capacities and establishing mechanisms to ensure
ownership and the durability of effects under the reform initiatives?
Is there a need to scale down the project (i.e. if the project duration is shorter than planned)? If so, do
project objectives and strategies have to be adjusted?
Has the project successfully built or strengthened an enabling environment (laws, policies, people's
attitudes etc.)?
How does the government intend to maintain the provision of family support services in future? Are
there any specific/envisioned plans/projects?
What are possible sustainability mechanisms for each of the project component? Can the project
approach or results be replicated or scaled up by national partners or other actors? Is this likely to
happen? Can any unintended or unexpected positive or negative effects be observed as a consequence
of the project's interventions?
Impact
Has the project contributed to or is likely to contribute long-term social changes for vulnerable families
and children? What difference has it made, including policy changes? What are the realistic long-term
effects of the project on de-institutionalization of children?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Has the project increased the satisfaction with family and community based alternative services?
The evaluation questions will be refined with the independent evaluation team and finalized during the inception
phase with a goal to explore other aspects, such as coordination between the partners and cooperation.
6. Duty station: Yerevan, Armenia
7. Supervisor:
The evaluation team will be supervised and report to the UNICEF Child Protection (CP) Specialist with regular
de-briefing with UNICEF Deputy Representative, UNICEF Representative and UNICEF M&E and Child
Rights Systems Monitoring Specialist about the progress of the evaluation. The independent evaluation team
will work on daily basis with UNICEF Country Office Child Protection (CP) Unit. The CP team will support
the selected evaluation team in contacting with the government partners and stakeholders during the fieldwork
(if need be).
8. Evaluation Methodology
In this evaluation mixed method approach will be applied by combining qualitative and quantitative components
to ensure complementary strengths and non-overlapping weaknesses. The independent evaluation team will
collect data from desk review and verify them with soft data from field visits, in-depth interviews and focus
groups. The analysis will be built on triangulating information collected from different stakeholders (project
staff, project partners, stakeholders and beneficiaries) through different methods including secondary data and
documentation review and primary data. It should critically examine the information gathered from the various
sources, and synthesize the information in an objective manner. If contradictory information is obtained from
different stakeholders, an effort should be made to understand the reasons for such information, including any
gender-based differences.
The evaluation team will review the following documents before conducting any interviews or field trips: project
documentation, progress reports, work plans, mission reports, monitoring data, workshop reports, minutes,
country data, policies, legal documents, etc.
Quantitative methods will include the conduct of a survey among teachers benefited from the project. In-depth
qualitative interviews will be conducted with implementing government and non-government partners,
including international/donor organisations.
During the report writing phase, the evaluation team should pay close attention to formulating good
recommendations, clearly prioritizing 5-10 recommendations for concrete action. The following are most likely
to lead to good recommendations:
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
- Key stakeholders are consulted during the development of recommendations, which heightens
programmatic and technical relevance.
- An appropriate sequencing in the implementation of recommendations is noted, especially when one
part of the response is contingent upon a prior action being completed.
- The recommendations clarify where change is needed to solve problems and also where positive aspects
should be continued or enlarged.
- Recommendations are referenced at the point where the finding evidence and analysis is made, to show
the logical connection. The full recommendations can be fully presented in a concluding chapter, but
referencing them within the document is helpful.
- The specific organizations that the recommendation is directed to should be noted, so there is no mistake
about who should respond.
The evaluation approach and data collection and analysis methods should also focus on gender and rights
aspects, be responsive and appropriate for analyzing the gender equality, human rights issues including child
rights identified in the scope. Gender, equity and human rights considerations need to be further elaborated by
the evaluation team during the inception phase and addressed across the final report.
To the extent possible data should be disaggregated by gender, age, minority and socially excluded groups, as
well as encompass urban/rural divide.
8.1 Evaluation approach, data collection methods and instruments
The independent evaluation team will identify key stakeholders/informants (including but not limited to project
implementers, decision makers, direct and indirect beneficiaries, etc.), and appropriate data collection methods
for each informant category (such as semi-structured or in-depth interviews, expert interviews, focus groups).
A combination of these methods should be proposed by the independent evaluation team in the Implementation
Plan and Methodology (Inception Report), and revisited, if necessary, during the fieldwork preparation.
In close cooperation with UNICEF project team, the independent evaluator will also be responsible for the
development of appropriate instruments, including questionnaires, interview and focus group guides,
observation check-lists for each of the methods selected, in Armenian and English. All materials should be
gender-competent in language and presentation, as well as take into consideration human rights and equity
angles, if possible. While there may be overlaps in the topics and items covered for the different informant
categories, the guides/protocols should be customized appropriate to each informant category to be able to
extract the relevant information from each group, and address the key process questions listed earlier.
Interviewers/facilitators involved in this project by the independent evaluator must have relevant qualifications
and be adequately trained/consulted prior to fieldwork (including gender-competency knowledge). Prior to
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
fieldwork the independent evaluator will be responsible for an interviewer training and pre-testing of the
instruments to be utilized.
8.2 Fieldwork Approach
UNICEF Armenia requires the independent evaluator to propose a fieldwork schedule and approach that will
ensure the accuracy and reliability of information gathered through the effective use of methods, staff, funds
and time. The independent evaluator will develop a preliminary fieldwork plan for coordinating the evaluation
efforts. This plan should focus on the following:
- Draft Schedule of fieldwork activities;
- Draft Schedule and approach to conducting the survey among teachers benefitted from the project;
- Number of interviews/qualitative and quantitative/ in each target group/region/community, criteria for
selecting interview respondents;
- Number of focus group (FG) discussions in each target group/region/community, criteria for selecting
FG participants;
- Approach to the site-visit/interview/focus group protocols and transcripts.
It is envisioned that evaluation fieldwork will cover different stakeholder groups, including decision makers,
those who are implementing and overseeing the implementation and those who are supposed to benefit from it.
9. Major tasks to be accomplished:
The evaluation process will consist of 4 key phases.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Major tasks RESPONS
IBLE
COORDINA
TION
DEADLIN
E
Des
ign
ph
ase
Desk review of reference material
- UNICEF Armenia CP Unit will support the
evaluation team to compile the list of most
important background material, documents, and
reports for review.
- The project M&E plan will be reviewed by the
evaluation team along with other key documents.
Stakeholder mapping
- The evaluation team will prepare a mapping of
stakeholders relevant to the evaluation. The
mapping will include ministries, regional and local
authorities, implementing partners, service
providers, direct and indirect beneficiaries,
development partners.
Implementation plan and methodology
- Evaluation matrix will be developed based on
mixed method design.
- Questionnaires (qualitative and quantitative) will
be developed.
- The set of evaluation questions will be finalized.
- Field work schedule and approach will be
drafted.
Inception Report
- Inception report will be developed and presented
to UNICEF Armenia with all the document/tasks
listed above for this phase embedded.
Independen
t
Evaluation
team
UNICEF
Child
Protection
Section; M&E
Specialist,
MoLSA,
MoES
July/August,
2017
Fie
ld p
hase
Data collection and analysis
- Collection of evaluation data (primary and
secondary) will be carried out through different
techniques, including survey, in-depth, informal
and semi-structured interviews, focus group
discussions and observations.
Independen
t
Evaluation
team
UNICEF
Child
Protection
Section; M&E
Specialist,
August/Sept
ember, 2017
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
- Survey instrument will be pretested based on the
methodology developed by the evaluation team.
- Data collection report will be submitted covering
the following pieces: documentation of survey
implementation, quality control, survey findings,
survey final instrument, dataset, users’ manual for
dataset, evaluation guides and summary of
protocols.
- The analysis will be based on detailed
protocols/transcripts of interviews, focus groups
and survey results, comparative analysis.
Debriefing
- Debriefing meeting will be organized for
UNICEF Armenia on the preliminary findings,
testing elements of conclusions and tentative
recommendations.
MoLSA,
MoES
Rep
ort
ing p
hase
Evaluation report
- The mid-term evaluation report will be drafted.
- Comments provided by UNICEF Armenia and
other key stakeholders (government, donor and
other development partners) will be addressed.
- Validation meeting will be organized with
partners and stakeholders to present evaluation
findings.
- The Final Mid-Term Evaluation Report will be
submitted.
Independent
Evaluation
team
UNICEF
Child
Protection
Section; M&E
Specialist,
USAID,
MoLSA,
MoES
September/
October,
2017
Dis
sem
ina
tio
n a
nd
Foll
ow
-up
Presentation and Dissemination
- The Evaluation Report presentation will be
organized for government partners and key
stakeholders.
- The mid-term evaluation report will be
disseminated (to stakeholders, development
partners, and Regional Office).
Follow-up
- Preparation of the management response.
Independent
Evaluation
team
UNICEF
UNICEF
Child
Protection
Section;
Management
October,
2017
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
The evaluation will follow the principles of the UN Evaluation Group’s norms and standards in particular with
regard to independence, objectiveness, impartiality and inclusiveness and will be guided by the UN ethics
guidance as guiding principle to ensure quality of evaluation process, especially apropos conflict of interest,
confidentiality of individual informants, sensitive to beliefs, manners and customs, discrimination and gender
equality, to address issues of vulnerable population, particularly families with children that are disadvantaged
and excluded.1
10. Deliverables:
The evaluation team is expected to produce and submit the following deliverables with detailed description what
each of the deliverable should cover:
- Inception Report (August)
Implementation plan and methodology including but not limited to: a) work plan; b) a stakeholder map; c)
the evaluation matrix/guides (including the final set of evaluation questions listed by domains and
indicators), d) survey instrument (questionnaire) with pre-test methodology, e) the overall evaluation design
with a detailed description of the data collection plan for the field phase;
- Draft Mid-Term Evaluation Report (first week of October)
a. Data collection report, including documentation of survey implementation, quality control, survey
findings (variable aggregation and descriptive report), survey final instrument, dataset, users’ manual for
dataset, evaluation protocols/transcripts.
b. Debriefing document (two-three pages overview) synthesizing the main preliminary findings, conclusions
and recommendations of the evaluation, to be presented and discussed with UNICEF Armenia management
during the debriefing meeting planned at the end of the field work phase.
- Validation workshop with government partners and other stakeholders (mid-October)
Validation workshop brief report to be included into the Final Mid-Term Evaluation report as an annex.
- Final Mid-Term Evaluation Report (end of October)
a. Final report with all the comments addressed. The report with the Executive Summary should be
maximum 50 pages, excluding annexes.
b. Presentation of the mid-term evaluation findings for key stakeholders (Power Point presentation).
The Evaluation Report is proposed to have the following structure, to be reviewed once the contractor is
selected:
1 http://www.uneval.org/document/detail/102
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
LIST OF ACRONYMS
ACKNOWLEDGEMENTS
EXECUTIVE SUMMARY
CHAPTER I: INTRODUCTION TO THE EVALUATION, OBJECT AND METHODOLOGY
1.1 Introduction
1.2 Object of the Evaluation
1.3 Background of the Action and Context
1.4 Logic of Intervention
CHAPTER II. EVALUATION PURPOSE, OBJECTIVES AND SCOPE
CHAPTER III. EVALUATION METHODOLOGY
3.1. Qualitative Data Collection: Approach and Implementation
3.2. Quantitative Data Collection: Approach and Implementation
3.3. Major Limitations
3.4. Ethical Considerations, Human Rights and Gender
CHAPTER IV. ANALYSES AND FINDINGS
4.1. Relevance
4.2. Effectiveness
4.3. Efficiency
4.4. Sustainability
4.5. Impact
4.6. Cross-cutting topics
CHAPTER V. CONCLUSIONS, LESSONS LEARNT AND RECOMMENDATIONS
5.1. Conclusions
5.2. Lessons Learnt
5.3. Recommendations
ANNEXES
1. Terms of Reference
2. Desk Review and Background Documents
3. List of Respondents by Data Collection Method
4. Detailed Methodology
5. Interview Guides and Survey Instrument
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
6. Information on Evaluation team
7. Evaluation matrix
8. Results framework
9. Output tables
10. Other documents, if relevant.
All deliverables will be drafted in English. All reports should follow the structure and detailed outlines discussed
and agreed with UNICEF in Armenia. The final report will be translated into Armenian.
11. Time-Frame:
The evaluation is expected to take place during the period of July 2017 – October 2017 with overall 50 days of
level of effort.
The selected institution (independent evaluation team) will work for the period of 50 work days within 3 months.
With consideration of reconstructing suitable basis for evaluation and quantitative survey, the level of effort of
some of the experts, as well as interviewers/field supervisors may well exceed 60 days.
It is envisaged that the evaluation will cover the capital city of Yerevan and Lori, Syunik regions including
number of communities. The exact schedule of the activities will be agreed with the institution (independent
evaluation team) based on the consultancy implementation progress. The deadline for submission of final
deliverables to UNICEF in Armenia is October 31, 2017.
12. Qualifications or specialized knowledge/experience required:
The team will consist of a team leader and three experts.
The team leader will be responsible for managing the evaluation and delivering the final report. The experts/team
members will be responsible, inter alia, for the evaluation design, data collection, analysis of some sections of
the report, implementation and logistics.
Team Leader to act as an evaluation coordinator and to work with UNICEF Armenia:
Required Qualifications:
Advanced university degree and/or academic background in Sociology, Economics, Public Policy
or a related field;
At least 10 years of proven record in managing project/program evaluations in child protection
and/or other social sectors;
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Extensive experience in designing evaluations and household surveys, conducting qualitative
analysis and surveys, data analysis and report writing;
Proven very good knowledge of child protection reform processes in Armenia;
Ability to work within the international and multicultural environment;
Very good communication and presentation skills with government and community members;
High analytical and conceptual skills and ability;
Good knowledge of computer applications;
Fluency in written and spoken English.
Experts/team members (up to 3 experts proficient in quantitative and qualitative methods).
Required Qualifications:
Advanced university degree and/or academic background in Sociology, Economics or a related
field;
At least 7 years of experience in program evaluation, particularly in conducting qualitative analysis
in child rights/protection and/or other related fields;
Good knowledge of child protection reforms processes in Armenia.
Experience in data collection and conducting surveys (including household surveys);
Demonstrated ability to prepare interview protocols and working with databases;
Previous experience in evaluation report writing;
Demonstrated ability to work in multicultural teams;
Demonstrated gender competency;
Excellent communication skills;
Fluency in written and spoken Armenian and English.
To facilitate gender sensitive focus group discussions, the independent evaluation team’s gender balance is an
advantage. The team leader’s experience with the United Nations or other development agencies is an asset.
13. Procedures and logistics:
UNICEF does not provide or arrange health insurance coverage for the members of evaluation team.
UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if
work/outputs is incomplete, not delivered or for failure to meet deadlines (fees reduced due to late submission:
1 month - 20%; 2 months - 50%; more than 2 months – payment withhold). All materials developed will
remain the copyright of UNICEF and that UNICEF will be free to adapt and modify them in the future. This
ToR is an integral part of the contract (SSA) signed with the contractor (independent evaluation company).
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
14. Terms of the application
Interested organizations are requested to express their interest to UNICEF in Armenia within 2 weeks from
the date of the publication of this announcement, specifying:
Implementation Plan and Methodology for the completion of the assigned tasks;
A detailed Work-plan with activities and timeframe;
Detailed Budget for completing the assignment (including travel);
CVs of proposed experts;
Past experience in fulfilling similar tasks and the outcome of at least one presented as an
attachment.
The proposal with all necessary documents should be submitted in a sealed envelope, either in MS Word or
Adobe PDF format in 3 (three) hard copies in English, to the following address: 14 Petros Adamyan str.,
Yerevan, Armenia, UNICEF in Armenia with the following subject: Mid-Term Review of “Toward Social
Inclusion of Vulnerable Children in Armenia” project.
The final selection of the evaluation institution will be conducted by UNICEF in Armenia based on previous
experience in implementing similar assignments; competence in conducting similar activities; evaluation
institution’s overall access to expertise and qualifications required for performing this task and fees proposed.
The overall score for each of the proposals is calculated based on a ratio of 70% - 30% between the technical
and financial proposals.
Maximum Points
Technical Proposal Score 70 points
Financial Proposal Score 30 points
Overall Proposal Score 100 points
Technical proposals received will be evaluated against the set evaluation criteria (total 70 points). The
maximum number of points (30) will be allotted to the lowest financial proposal that is opened and compared
among those invited institutions which obtain the threshold points in the evaluation of the technical
component (49 points). All other financial proposals will receive points in inverse proportion to the lowest
price.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
15. Prepared by:
Hayk Khemchyan
Child Protection Specialist Signature and Date:
Lusine Yeremyan
Monitoring and Evaluation, Child Rights Systems Monitoring
Specialist Signature and Date:
16. Reviewed by:
For Operations related
UNICEF Operations Manager Signature and Date:
Gayane Avanesyan
17. Approved by:
OIC for UNICEF Representative in Armenia Signature and Date:
Liv Elin Indreiten
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 2 LIST OF DESK REVIEW AND BACKGROUND MATERIALS
ENGLISH SOURCES
1. Amendment number four to assistance agreement between the Government of the United States of America and
Government of the Republic of Armenia for improving the Effectiveness of Health and Social Protection Services
(September 27, 2013)
2. BoH Budget3. BoH Project Final Proposal4. BoH revised Budget to UNICEF for reprogramming (June 05, 2015)5. BoH: Program Proposal Outline for Programme Cooperation Agreement6. BUDGET: SAVE THE CHILDREN ARMENIA. "Expanding Alternative Care, Family Support and Inclusive
Education Services as part of Child Care Reform” project (Revised 26/06/2015)7. Budget_NILSR 2016-20178. CPD 2016-20209. Draft Government decision on approving 2018 Program and Action Plan on Child Rights Protection10. GoA Decision N 831-N, dated as of August 11, 2016 on Reorganization of the “Yerevan Child Care and
Protection Boarding Institution N 2” into “Child and Family Support Center” SNCO11. Handbook on legal regulations12. Joint Workplan UNICEF Armenia and World Vision Armenia13. Legislative reform related to the Convention on the rights of the Child. National case studies: Armenia, Barbados
and Ghana, 200814. Monitoring and Evaluation Plan: Revision1, June 201615. Program Proposal Outline for Programme Cooperation Agreement with Save the Children International NGO16. Project Indicator Definition Table 17. Project Logframe. Project title - ”Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform”18. Project Proposal: Prepared by UNICEF Armenia in cooperation with WV, SC and BoH 19. Rapid Assessment of Child Care and Protection Institutions under the Ministry of Labor and Social Affairs,
Marina Galstyan, commissioned by UNICEF under USAID funding, Yerevan 201420. Report on local best practices of case management
21. "Toward Social Inclusion of Vulnerable Children in Armenia" Program Quarterly Progress Updates QR 1-11
22. "Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and
Inclusive Education Services as Part of Child Care Reform" Monitoring & Evaluation Plan (Revision 1, June
2016)23. "Toward social inclusion of vulnerable children: Expanding alternative care, family support, and inclusive
education services as a part of child-care reform"
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
24. UNICEF Armenia’s Request for Proposals to conduct Mid-term Evaluation of the ‘‘Toward Social Inclusion of
Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of
Child Care Reform’’ project (LRFP-2017-9132946).
25. UNICEF Child Protection Revised budget 0209
26. UNICEF Child Protection WVA Budget Allocation (Revised 09/10/14)
27. UNICEF In Armenia’s CPDs, annual reports and Project quarterly reports are used as sources
28. UNICEF, BoH, WV quarterly reports are used as a source
29. WV Project Proposal Outline
ARMENIAN SOURCES
1. Դեպք վարողների համար մոնիթորինգի մեթոդական ուղեցույց2. .Դեպքի վարում Մեթոդական ձեռնարկ դեպք վարողների համար3. 2018 Երեխայի իրավունքների պաշտպանության թվականի տարեկան ծրագիր և
, միջոցառումների ցանկ 2017 28- ՀՀ կառավարության թվականի սեպտեմբերի ի նիստի
որոշմամբ
4. Երեխաների պաշտպանության պրակտիկային վերաբերող չափորոշիչները
5. Իրազեկման նյութ՝ խաչվող կետեր․ Ապաինստիտուցիոնալացում և ինտեգրված
ծառայություններ
6. « »Խնամատար ընտանիք ․ Ձեռնարկ դեպք վարողների համար
7. -Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիտորինգի ձեռնարկ
ուղեցույց
8. « »Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիթորինգ ․
-Մեթոդական ձեռնարկ ուղեցույց
9. « »Խնամատար ընտանիք ․ ,Մեթոդական ուղեցույցներ ծնողների համար
, մասնագետների համար Հանրային իրազեկման համար)
10. « ».Խնամատար ընտանիք Մեթոդական ուղեցույց վերապատրաստվողների համար
( Մոդուլներ ծնողների համար
11. « »Խնամատար ընտանիք ․Մ . եթոդական ուղեցույց Հանրային իրազեկման արշավի
կազմակերպման առանձնահատկությունները
12. « ». Խնամատար ընտանիք Մեթոդական ուղեցույց վերապատրաստվողների համար
( )Մոդուլներ ։ « . ,Դեպի երեխաների սոցիալական ներառում այլընտրանքային խնամքի
ընտանեկան աջակցության և ներառական կրթության ծառայությունների
» զարգացումը՝ որպես երեխաների խնամքի բարեփոխում ծրագիր
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
13. « »Խնամատար ընտանիք ․ Մեթոդական ձեռնարկ վերապատրաստողների համար։
« . , Դեպի երեխաների սոցիալական ներառում այլընտրանքային խնամքի ընտանեկան
աջակցության և ներառական կրթության ծառայությունների զարգացումը՝ որպես
» երեխաների խնամքի բարեփոխում ծրագիր14. « »Խնամատար ընտանիք ․ Մեթոդական ձեռնարկ պոտենցիալ խնամատար ծնողներին
վերապատրաստելու համար։ « .Դեպի երեխաների սոցիալական ներառում
, այլընտրանքային խնամքի ընտանեկան աջակցության և ներառական կրթության
» ծառայությունների զարգացումը՝ որպես երեխաների խնամքի բարեփոխում ծրագիր
15. « »Խնամատար խնամքի սկզբունքներ և չափորոշիչներ ․ -Մեթոդական ձեռնարկ
ուղեցույց
16. -Խնամատար ընտանիքների և խնամատարության ծրագրի մոնիտորինգի ձեռնարկ
ուղեցույց17. Կյանքի դժվարին իրավիճակներում հայտնված երեխաներին խնամատարության
,հանձնելու ընթացակարգի բարելավման հայեցակարգ 2016 .ՀՀ կառավարության թ
10- 9 մարտի ի թիվ արձանագրային որոշմամբ18. (9 , 2004 .)Հայաստանի Հանրապետության Ընտանեկան օրենսգիրք նոյեմբեր թ19. « Հայաստանի Հանրապետության ընտանեկան օրենսգրքում փոփոխություններ և
», « լրացումներ կատարելու մասին Վարչական իրավախախտումների վերաբերյալ
»,Հայաստանի Հանրապետության օրենսգրքում լրացումներ կատարելու մասին
« Հայաստանի Հանրապետության քաղաքացիական դատավարության օրենսգրքում
» « » փոփոխություն կատարելու մասին և Երեխայի իրավունքների մասին Հայաստանի
»Հանրապետության օրենքում փոփոխություններ և լրացում կատարելու մասին
, Հայաստանի Հանրապետության օրենքների նախագծեր https://www.e-draft.am/projects/313
20. Հայաստանի Հանրապետության կառավարության որոշում « Գյումրու Երեխաների
խնամքի և Պաշտպանության N1 » Գիշերօթիկ Հաստատություն Պետական Ոչ
« Առևտրային Կազմակերպությունը Շիրակի Մարզի Երեխայի և Ընտանիքի
» Աջակցության Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու
մասին
21. Հայաստանի Հանրապետության կառավարության որոշում « Դիլիջանի Երեխաների
»,խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն «Բյուրեղավանի
» Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն Պետական Ոչ
Առևտրային Կազմակերպությունները լուծարելու և Հայաստանի Հանրապետության
2007 26- Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ փոփոխություններ
կատարելու մասին
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
22. Հայաստանի Հանրապետության կառավարության որոշում « Դիլիջանի Երեխաների
»,խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն «Բյուրեղավանի
Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ « » Հաստատություն Պետական
Ոչ Առևտրային Կազմակերպությունները լուծարելու և Հայաստանի
2007 26- Հանրապետության Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ
» փոփոխություններ կատարելու մասին Հայաստանի Հանրապետության
Կառավարության որոշման նախագծի ընդունման վերաբերյալ
23. Հայաստանի Հանրապետության կառավարության որոշում « Դիլիջանի Երեխաների
»,խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն «Բյուրեղավանի
Երեխաների խնամքի և Պաշտպանության Գիշերօթիկ « » Հաստատություն Պետական
Ոչ Առևտրային Կազմակերպությունները լուծարելու և «Հայաստանի
2007 26- Հանրապետության Կառավարության թվականի հուլիսի ի N 890- Ն որոշման մեջ
» փոփոխություններ կատարելու մասին Հայաստանի Հանրապետության
կառավարության որոշման նախագծի վերաբերյալ ստացված առաջարկությունների
ամփոփաթերթ
24. Հայաստանի Հանրապետության կառավարության որոշում « Կապանի Երեխաների
» խնամքի և Պաշտպանության Գիշերօթիկ Հաստատություն Պետական Ոչ Առևտրային
« Կազմակերպությունը Սյունիքի Մարզի Երեխայի և Ընտանիքի Աջակցության
» Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու մասին
25. Հայաստանի Հանրապետության կառավարության որոշում « Վանաձորի Երեխաների
խնամքի և Պաշտպանության N 1 » Գիշերօթիկ Հաստատություն Պետական Ոչ
« Առևտրային Կազմակերպությունը Լոռու Մարզի Երեխայի և Ընտանիքի
» ,Աջակցության Կենտրոն Հիմնադրամի վերակազմավորելու և գույք տրամադրելու
« » Վանաձորի Մանկատուն Պետական Ոչ Առևտրային Կազմակերպությունը լուծարելու
և 2002 28- NՀայաստանի Հանրապետության կառավարության թվականի նոյեմբերի ի
1906-Ն որոշումն ուժը կորցրած ճանաչելու մասին
26. . . Մոդուլ Սուպերվիզիան սոցիալական պաշտպանության ոլորտում նպատակներ և
գործառույթներ
27. ( 2016)Շտեմարան Երևան
28. Սոցիալական դեպքի վարում․ Մեթոդական ձեռնարկ մասնագետների համար
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
29. Սոցիալական Ծառայությունների Արձանագրային նախագիծ Հայաստանի
Հանրապետության Լոռու Մարզի Երեխաների Շուրջօրյա Հաստատությունների
վերակազմավորման վերաբերյալ ծրագրին և ծրագրի իրականացման միջոցառումների
ժամանակացույցին հավանություն տալու մասին։
ONLINE SOURCES
1. http://www.unicef.am/en/articles/aboutunicef 2. https://armenia.savethechildren.net/sites/armenia.savethechildren.net/files/library/book-eng.pdf 3. http://armstat.am/file/article/poverty_2016a_2.pdf 4. http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=45. http://www.uneval.org/normsandstandards/index.jsp 6. http://www.unEvaluation.org/ethicalguidelines
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 3. LIST OF REPONDENTS BY DATA COLLECTION METHOD
List of Experts InterviewedName Position Method
1. Robert Stepanyan Head of Development Programs and Monitoring Department of the Ministry of Education and Science of the RA
Expert interview
2. Anahit Muradyan Chief Specialist of the General Education Department of the Ministry of Education and Science of the RA
Expert interview
3. Anna Safaryan Assistant to the Minister, Ministry of Labor and Social Affairs RA
Expert interview
4. Lena Hayrapetyan Head of Child Rights Protection Unit, Ministry of Labor and Social Affairs RA
Expert interview
5. Armen Abrahamyan Director of Nork Center, Ministry of Labor and Social Affairs RA
Expert interview
6. Varduhi Katinyan Open Government Partnership Expert/Center for Pedagogical Psychological- Assessment ofYerevan/ Republican Pedagogical and Psychological Support Center
Expert interview
7. Anahit Bakhshyan Former Deputy Director of NIE MOES of the RA CJSC, inclusive education specialist
Expert interview
8. Eduard Israyelyan Head of Child Rights Protection Division at Human Rights Defender's Office of the Republic of Armenia
Expert interview
9. Marine Aghajanyan Director, Yerevan Child and Family Support Center Foundation, under the RA Ministry of Labour and Social Affairs
Expert interview
10. David Tumasyan Legal Expert, Head of Legal and Statistics Department, Investigative Committee of RA
Expert interview
11. Armen Hovsepyan Head of General Education Management and Content of Norms Development Division of NIE MOES of the RA
Expert interview
12. Gnel Tigranyan NIE regional structures in Lori Director, Ministry of Education and Science, National Institute for Education
Expert interview
13. Magda Gevorgyan, Marine Mirzoyan
NIE regional structures in Kapan Director, Coordinator, Ministry of Education and Science, National Institute for Education
Expert interview
14. Meline Grigoryan Inclusive Education Expert Expert interview
15. Susanna Tadevosyan, Hripsime Nazaretyan
BoH DirectorProject coordinator, Bridge of Hope
Expert interview
16. Grigori Grigoryants Child Protection Expert, Save the Children Expert interview17. Mira Antonyan Director of FAR Child Support Center Expert interview18. Voskan Ghazaryan Project Coordinator, World Vision Armenia Expert interview19. Hayk Khemchyan Child Protection Officer at UNICEF. Expert interview20. Hasmik Arakelyan Program Officer at U.S. Fund for UNICEF Expert interview
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Armenia21. Artak Shakaryan Child Protection Officer at UNICEF Armenia Expert interview22. Lusine Yeremyan M&E Officer at UNICEF Armenia Expert interview23. Liv Elin Indreiten Deputy Representative, UNICEF Armenia Expert interview24. Ani Manukyan USAID/Armenia Project Management
SpecialistExpert interview
25. Aida Muradyan World Vision, Child Protective Specialist Expert interview
List of Key Informants InterviewedName Position Method
1. Maga Ter- Hovhannisyan Social worker, Trainer, FAR Child Support Center
Individual interview
2. Anahit Sahakyan Lecturer, Trainer, Yerevan State University Individual Interview3. Anahit Galstyan Yerevan, No 160 School Principal Individual interview4. Anush Abelyan Yerevan, No 100 School Principal Individual Interview5. 2 biological family parents, Yerevan Parents of a former Noubarashen child
resident Individual interview
6. Erjanik Sargsyan, Hayk Harutyunyan
Department of Education, Culture and Sport of the Syunik region of the RA
Individual Interview
7. Anna Voskanyan Head of the Family, Women and Children’s Division ,Syunik Marzpetaran
Individual interview/Small group interview
8. Rouzanna Hovhannisyan Director ,Kapan Child Care and Protection Boarding Institution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Syunik Child and Family Support Center Foundation)
Individual Interview
9. Artour Hakobyan Director of the Kapan Territorial EmploymentCenter of State Employment Agency under the Ministry of Labour and Social Affairs
Individual Interview/Small group interview
10. Araik Hayrapetyan Director of the Kapan Territorial EmploymentCenter of State Employment Agency under the Ministry of Labour and Social Affairs
Individual Interview/Small Group Interview
11. Hermine Shalunc Director of Goris Pedagogical and Psychological Support Center
Individual interview
12. Hermine Davtyan Director of Sisian Pedagogical and Psychological Support Center
Individual Interview
13. Liana Grigoryan Ofelya Hovakimyan
Director of Kapan Pedagogical and Psychological Support Center, Deputy Director
Individual interview
14. Karmen Karapetyan Syunik City School, Goris School No.3 School Principal
Individual Interview
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
15. Anahit Petrosyan Syunik Rural School, Angeghakot School Principal
Individual interview
16. Anna Parsadanyan Mother of Arustamyan Mariam, Kapan, Syunik Marz
Individual Interview
17. Susanna Hovhannisyan Mother of Hovhannisyan Alvina, Kapan, Syunik Marz
Individual Interview
18. Gayane Abelyan Mother of Abelyan Gor, Kapan, Syunik Marz Individual Interview
19. Shaghat School Mother of Hayk Stepanyan, Shaghat, Syunik Marz /an inclusive child/
Individual interview
20. Narek Sargsyan Head of Staff of Lori Marzpetaran Individual Interview
21. Sasun Harutyunyan Director of Former Special school of Vanadzor, Vanadzor Pedagogical and Psychological Support Center
Individual interview
22. Hasmik Khachatryan Director of Former Special school of Spitak, Spitak Pedagogical and Psychological Support Center
Individual Interview
23. Martirosyan Lori, City School, Deputy Director of School No. 16 in Vanadzor
Individual interview
24. Abazyan Lori, Rural School Lernapat Secondary School Principal
Individual Interview
25. Marine Yarmaloyan Lori, City School, Spitak No 5 School Principal
Individual interview
26. Hasmik Nerkararyan Former Director,Vanadzor Child Care and Protection BoardingInstitution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Lori Child and Family Support Center Foundation)
Individual Interviews/Small GroupInterview
27. Mariam Aperyan Acting Director, Vanadzor Child Care and Protection BoardingInstitution SNCO, under the RA Ministry of Labour and Social Affairs (to be reorganized into Lori Child and Family Support Center Foundation)
Individual interviews/Small group interview
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
28. Lernuhi Gevorgyan Secretary of Vanadzor Guardianship and Trusteeship Commission, Vandzor Mayor’s Office
Individual interviews
29. Anahit Poghosyan Qnqush Harutyunyan
Vanadzor Social Support Regional Service Individual interviews
30. 4 Candidate Foster Families and 1 experienced foster family
Vanadzor, Lori Marz Individual interviews
31. Ruzanna Yazeryan Case manager, Yerevan, Shengavit SSTD
Individual interviews
32.Ruzanna Karapetyan Case manager, Yerevan, Shengavit SSTD
Individual interviews
33.Garik Galstyan Case manager, Lori, Vanadzor SSTD Individual interviews
34.Qristine Alaberqyan Case manager, Lori, Vanadzor SSTD Individual interviews
35.Ruzanna Khachartyan Case manager, Syunik, Kapan SSTD Individual interviews
36.Gohar Zaqaryan Case manager, Syunik, Meghri SSTD Individual interviews
List of Focus Group Discussions Conducted Institution Participants Method
1. Yerevan Child and Family Support Center Foundation, Yerevan
Social Workers – 6 respondents all female FGD
2. Yerevan, Ararat and Armavir Marzes 6 Candidate Foster Families FGD3. Kapan No 5 School Parents -10 FGD4. Kapan Local Psychological
Pedagogical Support CenterStaff – 6 FGD
5. Goris No 3 School Teachers – 9 FGD6. Goris No 3 School Parents –4 FGD7. Angeghakot School Teachers – 11 FGD8. Kapan Child Care and Protection
Boarding Institution SNCO, under theRA Ministry of Labour and Social Affairs
Social workers, social educators and psychologists – 7 respondents 6 female, 1 male
FGD
9. Vanadzor No 16 School Parents-9 FGD
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
10. Vanadzor No 16 School Teachers– 6 FGD11. Spitak former special school, current
Local Psychological Pedagogical Support Center
Staff – 6 FGD
12. Lernapat School Teachers – 13 FGD13. Lernapat School Parents – 4 FGD14. Spitak No 5 School Teachers – 7 FGD
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 4: SURVEY INSTRUMENT & INTERVIEW GUIDES
Quantitative questionnaire for interviews with teachers
Questionnaire N _________
Dear teacher,VISTAA Expert Center is conducting a survey commissioned by UNICEF in Armenia for the purposes ofevaluating introduction of inclusive education in Armenia, particularly in the Syunik region.Your participation in the survey is voluntary. However, with your candid answers you can greatly contribute notonly to the implementation of the survey but also the implementation of system reforms in the country in thecorrect direction. Your name will not be mentioned anywhere, your answers will not be presented individuallybut will be analysed aggregately along with the responses given by all participants. Thank you for your cooperation
Identify the answers to the questions below in advance and fill out the following for all interviews carried out at thegiven school:
Number of children in inclusive education __________________________
Number of children from special schools registered at the given school _______________________________
Number of children from special schools enrolled in the given school _______________________________
Section 1. General education
A. Overall awareness
1. Are there children enrolled in the inclusive education system at your school?
1. Yes
2. No
98. Difficult to say
2. What is inclusive education? It is a system of organizing education, according to which:
1. …children with special education needs study in a separate institution
2. Children with special education needs study at general education schools together with their peers while receiving additional services
3. Children with special education needs study at general education schools in separate classrooms
98. Difficult to say
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
3. I will read out some options for you to select if these children are considered children with special educationneeds.
3.1. Who do you think should be included? 3. Considered
as such3.1. Should be included
1. Children of socially insecure families 1. Yes 2. No 1. Yes 2. No2. Children whose parents work and are unable to tend to the education of their child/children
1. Yes 2. No 1. Yes 2. No
3. Children with physical impairments 1. Yes 2. No 1. Yes 2. No4. Children with visual impairments 1. Yes 2. No 1. Yes 2. No5. Children with speech and language impairments 1. Yes 2. No 1. Yes 2. No6. Children with hearing impairments 1. Yes 2. No 1. Yes 2. No7. Children with mental health issues 1. Yes 2. No 1. Yes 2. No8. Children without parents 1. Yes 2. No 1. Yes 2. No9. Children from large families 1. Yes 2. No 1. Yes 2. No10. Children with mental disorders 1. Yes 2. No 1. Yes 2. No11. Children who are deaf and speech-impaired 1. Yes 2. No 1. Yes 2. No
12. Blind children
13. Other /please note/ 1. Yes 2. No 1. Yes 2. No98. Difficult to say
4. Are the following statements accurate?
Yes No Difficult to say
The medical model of disability is at the basis of inclusive education
1 2 3
Every teacher must have the skills for child observation andassessing learning capabilities of the student.
1 2 3
Teachers of respective subjects do not take part in thedevelopment and implementation of individual education plans.Only the parents, the class teacher, the special teacher and thepsychologist are involved in the process.
1 2 3
The individual education plan is developed for 1 school year orin response to a short-term issue.
1 2 3
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
B. Attitude
5. Do you think Armenia is ready for the introduction of the inclusive education system?
1. Yes, absolutely
2. To some extent yes
3. To a certain extent, no
4. Not ready at all
98. Difficult to say
6.On a scale of 1 to 5 (where 1 is the lowest score and 5 is the highest) please rate thereadiness for the introduction of inclusive education of:
1 2 3 4 5
The members of your community, parents (attitude) 1 2 3 4 5
Your school (the building and other amenities) 1 2 3 4 5
Your colleagues (attitude, knowledge, skills) 1 2 3 4 5
7. To what extent do you personally accept the idea of introducing the inclusive education system?1. I absolutely accept it2. I accept it to a certain degree 3. I do not accept it to a certain degree4. I do not accept it at all98. Difficult to say
7.1. Why do you think so?
__________________________________________________________________________________________________________________________________________________________
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
8. Please note whether the introduction of the inclusive education system has a positive or a negative role forthe following groups/institutions.
Absolutely negative
Som
ewhat negative
Neither
positive,
nor
negative
Som
ewhat positive
Absolutely positive
Difficult to say
1. Children with physical impairments 1 2 3 4 5 98
2. Children with mild mental impairments 1 2 3 4 5 983. Children with severe mental impairments 1 2 3 4 5 984. Children of socially insecure families 1 2 3 4 5 98
5. Children excluded from the inclusive educationsystem
1 2 3 4 5 98
6. Parents of children excluded from the inclusiveeducation system
1 2 3 4 5 98
7. Parents of children enrolled in the inclusiveeducation system
1 2 3 4 5 98
8. Teachers 1 2 3 4 5 989. School administration 1 2 3 4 5 9810. Community 1 2 3 4 5 9811.Society/country 1 2 3 4 5 9812. Stakeholders interested in the introduction of
the system /ministry of education, NationalInstitute of Education, other…/
1 2 3 4 5 98
9. What do your colleagues think of the children in inclusive education? They say that … (do not read out theoptions. Only up to 3 options can be accepted)
1. They are no different from other children2. They disturb the class3. Children in inclusive education pay more attention to classes4. It is impossible to control them5. Working with them is a pleasure
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
6. Working with them is very difficult7. It would be better to have relevant specialists working with them8. They make significant progress9. It would be better if they attended relevant educational institutions10. It is very good that they study together with all other children11. Other __________________________________
10. What does a teacher need for working more effectively in the inclusive education system? (Do not read outoptions but note only 1).
1. Extra payment, salary
2. Didactic materials
3. Verbal encouragement
4. Promotion
5. Other _____________________________________
98. Difficult to say
C. Practice
11.Does your school have children from special schools?
1. Yes, if yes, how many? ___________________________
2. No
98. Difficult to say
12. Are there children enrolled in the inclusive education system in the classes you teach? 1. Yes 2. No /Move on to Section 2/98. Difficult to say /Move on to Section 2/
13. I will read out a number of observations. Please state which one you agree with and which one you do not.
Absolutel
y agree
Som
ewha
t agree
Som
ewha
t disagree
Do
not
agree
atall
Difficult
to say
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
I spend most of my time in the classroom working with thechild/children in the inclusive education system than with others
1 2 3 4 98
I work with children in the inclusive education system outside themain school hours as well
1 2 3 4 98
I prepare for the work with the children in the inclusiveeducation system for much longer than for working with theother children.
1 2 3 4 98
14. What is the attitude of the classmates towards children in the inclusive education system?
1. Very good
2. Good
3. Neutral
4. Not so good
5. Poor
98. Difficult to say
15. How often have you had problems/difficulties in dealing with children in the inclusive education system?1. Very often2. Often3. Sometimes4. Rarely5. Never 98. Difficult to say
16. How do you decide where the child from the inclusive education system sits in the classroom?1. The needs of the child are taken into account
2. The needs of the class are taken into account3. The child sits randomly. Nothing special is taken into account98. Difficult to say
17. Whom do the children in the inclusive education system mostly interact with in the classroom/school? /Noteup to 3 answers/
1. All children in classroom2. Only with one another3. With children of neighbours’ attending the same school4. With relatives attending the same school5. With children who are nice to them6. With no one7. _______________________8. _______________________
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
18. Do you cooperate with your colleagues while working with children in inclusive education?1. Yes, always2. Sometimes yes3. Mostly no /Move on to Q 20/4. No, never /Move on to Q 20/
19. How often do you cooperate with regards to the issues of children in the inclusive education system?V
eryoften
Often
Som
etim
es
Rarely
Never
With the teacher assistant or members of the multidisciplinary team 1 2 3 4 5
With employees of the pedagogical and psychological support center 1 2 3 4 5
With the representatives of the National Institute of Education 1 2 3 4 5
With the parents of children in the inclusive education system 1 2 3 4 5
With the parents of other children 1 2 3 4 5
20. Do you think your work has affected the following…?
Yes Somewhat yes
Somewhat no
No Difficultto say
The attitude of children towards those in the inclusive educationsystem has changed
1 2 3 4 98
The attitude of children in the inclusive education system haschanged towards other children
1 2 3 4 98
The behaviour of children in the inclusive education system haschanged
1 2 3 4 98
The performance of children in the inclusive education systemhas changed
1 2 3 4 98
Section 2. Relevance and efficiency of classes
21. Have you participated in a training course on inclusive education in the last 2 years?1. Yes2. No /Move on to Section 4/
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
22. When and where was the course? How long was it and who was it organized by?
N When/year Where1. At the school2. At a differentschool in theprovince3. Local NIEoffice4. Local NIEoffice
Duration/howmany days
Organized by1. Administration of the given school2. Members of the multidisciplinary team3. Staff of NIE Yerevan office4. NIE regional office staff5. Bridge of Hope NGO6. Staff of national pedagogical and psychologicalsupport center7. Staff of local pedagogical and psychologicalsupport center8. Other
1
2
3
23. Did you receive a certificate upon completion of the training course implemented by your schoolrepresentative and the NIE staff?
1. Yes
2. No
24. Did you find the trainings useful? 1. Absolutely 2. Somewhat useful 3. Not so useful 4. Not useful at all
25. How was your attendance in the trainings? 1. I did not miss a training2. I attended more than half 3. I attended half4. I attended less than half of the trainings 5. I barely attended the trainings
26. How many of your school’s teachers do you think still need trainings on inclusive education?1. All teachers2. More than half of teachers3. Half of teachers4. Fewer than half of teachers5. Several teachers6. No one /Move on to Q28/7. Difficult to say
27. /If in answer to Q 26 it was mentioned that there were teachers who need additional training, clarify what type ofschool teachers are they?
ALL following questions are about the training courses organized by Bridge of Hope NGO, whichwere implemented by the school director and the NIE employee.
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
1. Elementary school teachers 2. Middle school teachers3. High school teachers4. Other /please note / ____________________________
28. Do you personally apply the knowledge you gained during trainings in class? 1. Yes, always2. Mostly yes3. Mostly no4. No, never
29. One a scale of 1 to 5 (with 1 being the lowest and 5 being the highest score) please rate the following for theimplemented trainings:
1 2 3 4 5… effectiveness 1 2 3 4 5…usefulness to you 1 2 3 4 5…adequate organization (venue, period, duration of training, hospitality, etc.) 1 2 3 4 5
…quality of provision of materials (was it done in a comprehensible fashion in clearand understandable language? Were interactive techniques used, etc.?)
1 2 3 4 5
…contents of materials (were all questions discussed? Did the material includenovelty, etc.?)
1 2 3 4 5
Section 3. Availability of auxiliary resources
30. Do you conduct a needs assessment of students?1. Yes2. No /Move on to Q32/
31. If yes, then whom do you cooperate with for this purpose?1. Relevant specialists from the school2. Local NIE staff3. Staff of local pedagogical and psychological support center4. Other _______________________________________
32. Do representatives of the pedagogical and psychological support center work with your school?1. Yes2. No /Move on to Section 5/98. Difficult to say /Move on to Section 5/
33. If yes, then please rate how effective the cooperation was on a scale from 1 to 5.
1 2 3 4 5 98
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
34. What issues do they cooperate with the school on /note all possible answers/?1. They carry out a needs assessment for the child2. They provide consultations on issues of children in the inclusive education system3. They develop individual education plans4. They develop a support plan for the child5. They work with children in the inclusive education system6. They don’t do anything7. Other /please note/_____________________________________________98. Difficult to say
35. Does the introduced assessment system allow the objective assessment and identification of children withissues?
1. Yes, absolutely2. Mostly yes3. Mostly no4. Not at all98. Difficult to say
36. How the efficiency of cooperation between the center and the school be improved?
__________________________________________________________________________________________________________________________________________________________________________
Section 4. Social, demographic, and other characteristics of respondent
37. Age _________________________
38. Gender1. Male2. Female
39. You are a …1. Teacher
2. Teacher’s assistant3. Social pedagogue4. Speech therapist 5. Psychologist6. Other ________________________________
40. What subject do you teach? ____________________
1. I do not teach
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
41. How long have you been working in the education sector? ______________________________
42. How long have you been working at the given school? ________________
43. Have you ever worked at a special education school? 1. Yes 2. No
THANK YOU
Telephone: _____________________________________________________________
(When asking for the telephone number, notify them that it is only for the verification of the interviewer’s work and formaking further clarifications later, which does not contradict with the requirements of anonymity.)
Is there anything you would like to add? _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Fill out after finishing the interview
Area of residence 1. Urban 2. Rural
Community ____________________________________________
School number ________________________________________________
Interviewer __________________________
Date entry by_______________________
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questionnaire for in-depth interviews with school principals
Establishing contact Presenting survey objectives, principles of data summary and anonymity
Relevance
A1. Perceptions on the necessity for the introduction of the inclusive education system
1. How long has your school been involved in the process of introducing inclusiveeducation?
2. How do you generally view the idea of inclusive education? Why do you thinkso?
3. How effective is the introduction process of that system? Why do you think so?Do you think the Armenian society, the parents, the teachers and the children areready for inclusive education?
4. Is the introduction of inclusive education in the best interests of the children, thesociety, the parents, the teachers, the government and the implementers? Why doyou think so?
5. Do you think inclusive education reflects the needs of such children?6. Overall, how would you describe children enrolled in inclusive education? How
are they different from other students?7. Based on the assessment system, children with different needs have been
enrolled. How are they different from one another? How different are the needsof girls and boys?
8. How many children of each group are there at your school? How many girls andhow many boys? What are their peculiarities /needs and so on/?
9. How would you describe the process of introduction of inclusive education inArmenia? What are the obstacles? What are contributing factors?
A2. Participation in trainings on inclusive education
10. Have you participated in trainings on inclusive education? When and where andhow long? How would you rate those trainings overall?
11. Rate the trainings according to:a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality
12. What changes have occurred as a result of those trainings?13. Have you applied the knowledge you gained from those trainings? How?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
14. Have you passed the knowledge gained from the training on to your colleagues?How was the training of other teachers done?
15. What was the time ratio for training and for being trained? Do you think that theassigned time /30 hours/ is enough for passing on the knowledge gain in thesame time period?
16. What would you suggest to improve the effectiveness of the process oftransferring the knowledge /changing the way of organizing the courses,assigning higher roles to pedagogical and psychological centers …/.
Effectiveness
B1. Transfer of knowledge on inclusive education to teachers
Organization of training courses on inclusive education17. How did you get ready for the training of other teachers? Did you prepare
beforehand or you just passed on the knowledge you had gained from thecourse?
18. How long was the training that you had organized? What were the objectives andthe outcomes?
19. What sorts of problems did you have to face in your work with teachers aimed atthe introduction of inclusive education?
20. What sorts of changes have you noticed as a result of your work in the attitudesand work styles of teachers?
21. How do you as a principal take part in and control the process of introducinginclusive education at your school?
22. How well did the teachers of your school comprehend the idea of inclusiveness?To what extent do they use or display an inclusive approach?
23. How do other individuals, organizations, institutions participate in theintroduction of the inclusive education system? How willing are they? Howinterested are they to do so?
Availability of supporting resources; establishment of pedagogical andpsychological centers
24. Are there special education schools in your community? (If it did exist and wasturned into a pedagogical and psychological center, then) do you thinkpedagogical and psychological schools successfully replace special schools?Why do you think so?
25. How do you collaborate with the center? Please describe the procedure. Whatactivities do they do and how frequently do you meet?
26. What should be done for increasing the effectiveness of cooperation between thecenter and the school?
B2. Perceptions of effectiveness of work with children in the inclusive education system
27. How does inclusive education help children enrolled in the system? Does it havea positive or a negative impact on them? Is there a difference for boys and girls?Why do you think so?
28. What kind of changes have you noticed since the school has become inclusive?Is there a difference in terms of girls and boys? Why?
29. What group of children enrolled in the inclusive education system is the lattermore effective? Why do you think so? Are the boys more easily adapted or thegirls?
30. What would help the introduction of inclusive education be more effective?Efficiency
C1. Assessment of conditions
31. Has the school been adapted for all children /ramp, toilets, classrooms/? Istransportation provided? Does the food provided at school help enhanceinclusiveness?
32. Does the school have the required professional staff?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
required for working withchildren in the inclusive
education system
33. What do you need to have the children in the inclusive education system fullyengaged? Is there a difference between the needs of boys and girls?
34. What do the children need that is not provided and the lack of which hinderstheir inclusion?
35. What sort of support is needed to increase the efficiency of the inclusiveeducation system?
Sustainability
D1. Perceptions of sustainability of the project
36. In your opinion, how effective is the introduction of inclusive education deemedby parents of schoolchildren and parents of children in inclusive education?What kind of feedback do you get?
37. Are the children ready for inclusiveness? What about the parents? Why do youthink so? Are the boys more easily adapted or the girls?
38. How do you personally rate the impact of activities you carried out? How arethey perceived by the parents?
39. How do you think the introduction of inclusive education generally impactscommunities?
40. Does inclusive education replace special education schools whilst filling theirnegative gaps? What sorts of problems does it solve?
41. How would you describe the impact of inclusive education in your owncommunity? What visible changes are there?
42. What should be changed in the inclusive education system?43. Do you think the current Armenian schools and the society are ready for the
introduction of inclusive education? Why?44. What should be done to have a smooth introduction of inclusive education
accepted by all?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questionnaire for the focus group discussion with teachers
Establishing contact Introduction of the objectives if the survey, principles of anonymity and data summary by the moderatorSelf-introduction of participants (profession, subject taught and position, number ofyears worked at the school, previous work experience with children in the inclusive education system)
Relevance
A1. Perceptions on the need for the introduction of the inclusive education system
How do you generally view the idea of inclusive education? Why do you believe so?How efficient is the process of introduction of that system in Armenia? Why do you think so? Do you think the Armenian society, the parents, the teachers and the children are ready for inclusive education?Is the introduction of inclusive education in the best interests of the children, the society, the parents, the teachers, the government and the implementers? Why do you think so?How would you describe children enrolled in inclusive education? How are they different from other students? How do the needs of boys and girls differ?Based on the assessment system, children with different needs have been enrolled. How are they different from one another? How different are the needs of girls and boys?How many children of each group are there at your school? How many girls and how many boys? What are their peculiarities /needs and so on/?
A2. Participation in trainings on inclusive education
Have you participated in trainings on inclusive education? When and where and how long? How would you rate those trainings overall?
Rate the trainings according to: a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality
Did the training course meet your needs? Are there teachers who, according to you, needed the training more than others did? If yes, then who? Why do you think so?What changes have occurred as a result of those trainings?Have you applied the knowledge you gained from those trainings?Have you passed the knowledge gained from the training on to your colleagues? How?Do you think the knowledge and skills you have now are enough for applying an efficient inclusive approach? If no, then why? What kind of additional knowledge and skills do you think you still need?Were monitoring visits paid to these trainings? If yes, then by whom? How did that go? What would you recommend to increase the efficiency of the training courses?
Effectiveness
Observation of children in the inclusive education system and identification of their learning abilitiesDo you have individual education plans for working with the children? How realistic are those plans?When you first walk into a classroom, do you give or receive more information /observing, meeting/?How do you identify the children that need to be enrolled in the inclusive education system?Describe how you conduct a needs assessment. How do you decide which method you should use for working with children?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
B1. The process of working with children enrolled in the inclusive education system
Availability of supporting resources; establishment of pedagogical and psychological centersAre there special education schools in your community? (If it did exist and was turned into a pedagogical and psychological center, then) do you think pedagogical and psychological schools successfully replace special schools? Why do you think so?How do you collaborate with the center? Please describe the procedure. What activities do they do and how frequently do you meet?Does the center work with children with inclusive education, their families and teachers?How do the children and their families see the role of the center? What is the attitude of the community towards the activities of the center?What should be done for increasing the efficiency of cooperation between the school andthe center?
Development and implementation of individual plans for the studentsHow do you develop the individual education plan? Who participates in the process? How is that carried out?Is the plan accepted by the parents? How is the implementation of the individual student plan carried out?How do other people, organizations and institutions contribute to the process of providing support to the child? How willing are they to do so? And how motivated are they for doing so?
B2. Perceptions of effectiveness of work withchildren in the inclusive education system
To what extent does inclusive education help children? Does it have a positive or a negative impact on them? Why do you think so?When you started working in the class that had a child from the inclusive education system, what kind of problems were you faced with? Are the issues of girls different from those of the boys? How do you deal with various problems? What are the problems that remain without solutions?What kind of changes have you noticed in your work?What group of children in inclusive education benefits the most from your work? Is therea difference in terms of boys and girls? Why do you think so? What would help to make the “education for all” and “education for everyone” approaches more effective?
Efficiency
C1. Assessment of conditions required for working with children in the inclusive education system
Do you think you have the knowledge needed to work with children in the inclusive education system?Do you have relevant the relevant technical means for working with the children?Has the school been adapted for all children /ramp, toilets, classrooms/? Is transportation provided? Does the food served at school contribute to the enhancement of inclusiveness?Does the school have the relevant professional staff?What do you see is lacking for the full engagement of children in the inclusive education system? Is there a difference between the needs of boys and girls?What sort of support is needed to have the performance of the inclusive education systembe more effective?
Sustainability
In your opinion, how effective is the introduction of inclusive education deemed by parents of schoolchildren and parents of children in inclusive education? What kind of feedback do you get?Are the children ready for inclusiveness? What about the parents? Why do you think so?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
D1. Perceptions of sustainability of the project
Are the boys more easily adapted or the girls?How do you personally rate the impact of activities you carried out? How are they perceived by the parents?How do you think the introduction of inclusive education generally impacts communities?Does inclusive education replace special education schools whilst filling their negative gaps? What sorts of problems does it solve?How would you describe the impact of inclusive education in your own community? What visible changes are there?What should be changed in the inclusive education system?Do you think the current Armenian schools and the society are ready for the introduction of inclusive education? Why?What should be done to have a smooth introduction of inclusive education accepted by all?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questions for the expert interviews (generalized questionnaire)
Deinstitutionalization, introduction of inclusive education and all related functions
1. Please tell us a little about the ongoing reforms
2. How are they conducted?
3. How does your organization take part in the processes? What international organizations do you cooperate withduring the implementation of those activities? How do you see your cooperation with other organizations?
4. How effective are the ongoing reforms? What are their strengths and weaknesses?
5. How are the ongoing reforms related to state policy, priorities and the interests of children?
6. Are the actions undertaken by UNICEF coordinated and brought in line with the reforms carried out by thegovernment? Why? Why not?
7. What are the main directions and stages of reforms since the launch of the project? What activities are foreseenfor the coming years?
8. What are the challenges during the implementation of the reforms?
Legal
Financial
Organizational
Content
9. What are the contributing and hindering factors for the reforms?
10. How do various organizations i.e. schools, governor’s office, special education schools and others cooperate withone another?
11. Two types of integration is done: with the school and with the family. How are these processes going? How doesthe government respond to challenges arising on the way?
12. How do you see the future of the reforms without the financial, methodological and other forms of supportprovided by UNICEF and other donors? What will the government do? How do you see the sustainability of thereforms?
13. What are the lessons learned?
14. What would you change in the implementation of the project?
15. Do you know about the feedback from beneficiaries? How is it?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questionnaire for in-depth interviews with family members
/Biological family, foster family – with and without experience/
Adapted for each case
Establishing contact Introduction of survey objectives, principles of anonymity and data summary
Self-introduction/ introducing the child’s story
Please tell us about your child a little – age, gender, which institution he/she attends or used to attend, causes for attending the institution
When did he/she stop going? What was the reason (a question for the biological family)?
How and why did you get involved in the foster family system? What kind of foster experience do you have (a question for the family with foster care experience)?
How and why did you want to get involved in foster services (a question for the family with no foster care experience)?
Relevance
A1. Perceptions of the need for the implementation of the reforms
1. What do you think of the new approach?2. Are you ready for the new approach? What about your child? 3. How ready are the child’s friends, teachers, social workers or other supporting
organizations for the new approach?4. How has your situation (social, psychological, domestic, financial) changed /or will
change/ following the reform? What about the child?5. Does the new approach reflect your child’s actual needs? Is it good or bad for you
personally?A2. Participation in training courses related tothe reforms
6. Did you take part in training courses within the frameworks of the reform? When,where and how long was the training? How would you assess the courses in general?
7. Rate the training courses according to: a) Relevance/necessity of topic b) Contentsc) Level of preparedness of training specialistsd) Duration (long enough or not)
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
e) usefulness and applicability, correspondence to Armenian reality8. Did this training course meet your needs?9. What kind of changes occurred due to the training? 10. What other types of knowledge/skills do you feel you still need to acquire?11. What would you recommend to make the training courses more effective?
Effectiveness
B1. Process of working with children
Observation of children and identification of needs12. Are there institutions or people who work with your family/child? Who are they?
Please tell us a bit about how they support or help you. How do you see caring foryour child without such help?
13. Does your child attend any other institution?14. Do you receive support from the state? What kind? How has the support provided by
the state changed following the reform? Does the state support more now thanbefore? If no such support existed, how would you be able to care for your child?
15. How is the cooperation done in the following chain: parent-child-social worker-casemanager-institution-state agency?
16. What should be done to make the cooperation between various organizations andstakeholders more effective?
B2. Perceptions of effectiveness of work withchildren involved in the reforms
17. What do you think is the best place for the child? The family, the institution, theschool …?
18. Is the child better off now? What is better and what is worse now (If there are nochanges yet, ask what will happen if there are changes)?
19. Are you personally better off now? What is better and what is worse? What changedin your family for the better and what changed for the worse (if no changes havehappened, ask what will happen if there are changes).
20. What is better for the future of the child?21. What difficulties do you face/will you face after the change? How do you overcome
difficulties? 22. What would help you overcome difficulties more easily?
Efficiency
C1. Assessment of conditions required for the reforms
23. Do you think the organizations or individuals working with children have adequateknowledge and skills? What is their attitude? Have you noticed attitude changestowards the child? If yes, then what do you think that has to do with?
24. Do you think the institutions your child attends have adequate conditions to meet theneeds of the child? What do these institutions need to meet the needs of your child ina better way?
25. Do you have adequate conditions (time, knowledge/skills, finances, domesticconditions, resources, etc.) for meeting the needs of the child and ensuring a normallife? What would you need to better be able to meet the needs of the children?
26. What kind of assistance is needed to make the system more effective?Sustainability
D1. Perceptions of sustainability of the project
27. Do you think the current approach is better than the old one? Why do you think so? 28. Are people around you (other children, specialists, institutions, etc.) ready for the
shift to the new system?29. What has changed? How? 30. What should be changed in the process of implementation of the reforms?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
A question for the foster family
31. What kind of cultural and legal obstacles are there?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questionnaire blocks for the focus group discussion with staffs of institutions (boarding school, special education school, care center, pedagogical and psychological center)
Adapted for each case
Establishing contact Introduction of the objectives if the survey, principles of anonymity and data summary by the moderator
Self-introduction of participants (profession, occupation and position, work experience with children)
Relevance
A1. Perceptions on the need for the introduction of the inclusive education system
1. How do you generally view the idea of reforms in the sector (for each case thename of the corresponding reform is read out, i.e. deinstitutionalization,introduction of inclusive education and other related activities)? Why do youthink so?
2. How efficient is the process of introduction of that system in Armenia? Why do youthink so? Do you think the Armenian society, the parents, the teachers and thechildren are ready for inclusive education?
3. Is the introduction of inclusive education in the best interests of the children, thesociety, the parents, the teachers, the government and the implementers? Why do youthink so?
4. How would you overall describe the process of implementation of the reforms interms of the situation of children before and after the reform? How has the situationof children changed?
5. What happened to your organization as a result of the reforms?6. In your opinion, how are the implemented reforms related to state policy, priorities
and the interests of children?7. Are the actions undertaken in the frameworks of the reforms coordinated? How do
various parties cooperate? A2. Participation in trainings related to the reforms
8. Have you participated in trainings carried out in the frameworks of the reform?When, where and how long was the training? How would you rate those trainingsoverall?
9. Rate the trainings according to: a) The need/relevance of the topic b) Contentsc) Preparedness of training specialistsd) Duration (appropriate or not)e) efficiency and applicability, correspondence to Armenian reality
10. Did the training course meet your needs? Are there specialists who, according toyou, needed the training more than others did? If yes, then who? Why do you thinkso?
11. What changes have occurred as a result of those trainings?12. Have you applied the knowledge you gained from those trainings?13. Do you think the knowledge and skills you have now are enough for working
efficiently? If not, then why? What kind of additional knowledge and skills do youthink you still need to acquire?
14. Were monitoring visits paid to these trainings? If yes, then by whom? How did thatgo?
15. What would you recommend to increase the efficiency of the training courses?Effectiveness
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
B1. The process of working with children
Observation of children and identification of needs16. Do you have mechanisms for the observation of children and the identification of
their needs? What are those and are they effective or not? Why?17. How has your work changed before and after the reform in terms of different
procedures, styles, etc.?18. What organizations do you cooperate with in your work? Please describe main
directions of cooperation. What sorts of challenges do you face during thatcooperation? How do other people, organization and institutions contribute to theprocess of providing support to the child? How willing and motivated are they?
19. Two types of integration is done: with the school and with the family. How are theseprocesses going? How does the government respond to challenges arising on theway?
20. How is the cooperation maintained in the parent-child-institution-state agency chain?21. What should be done to increase the effectiveness between different organizations
and stakeholders?
B2. Perceptions of effectiveness of work withchildren involved in the reforms
22. How have the implemented reforms supported children in that system? Have theyhad a positive or negative impact on them? In terms of the impact, does it matter ifthe child is a girl or a boy? Why do you think so?
23. Where do the unfavorable conditions for the child lie in the institution-foster family-biological family-school chain?
24. What problems did you have to face as a result of the implementation of thereforms? How do you deal with the issues that come up? What are the problems thatstill require solutions?
25. What changes have you noticed in your work?26. What group of children involved in the reforms benefits the most from your work?
Do the girls adapt more easily than the boys?27. What would help to make your work more effective?
Efficiency
C1. Assessment ofconditions required for
the reforms
28. How well do you think you possess the knowledge required for working withchildren in the new system?
29. Do you have the corresponding technical means to work with the children?30. Has the institution been adapted for all children /ramp, toilets, classrooms/? Is
transportation provided?31. Does your institution have relevant professional staff?32. What do the children need that is not provided and the lack of which hampers their
efficient wellbeing? Is there a difference between girls and boys in terms of thoseneeds?
33. What kind of support is need to increase the level of efficiency of the reformssystem?
Sustainability
D1. Perceptions of sustainability of the project
34. How effective do you think the society /community, parents, children, staff workingin the system/ deems the introduction of the new system? What sorts of feedback doyou get?
35. Are the children ready to switch to the new system? What about the parents, thespecialists and the institutions? Why do you think so?
36. How do you personally rate the impact of the activities you carried out? How do themembers of the community perceive that?
37. Overall, how do you think the implementation of the reforms impacts the
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
communities? What sorts of visible changes are there in your own community oramong the children?
38. What should be changed in the process of implementation of the reforms?39. Do you think the current Armenian society is ready for the undertaken reforms?
Why?40. What should be done to have the reforms implemented smoothly and in a manner
accepted by all?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Questionnaire for in-depth interviews with case managers and social service providers
Adapted for every case
Establishing contact Introduction of survey objectives, principles of anonymity and data summary
Relevance
A1. Perceptions of the need for the implementation of the reforms
1. What do you think of the reforms in the sector in general (for each case the title ofthe corresponding reform is given, i.e. deinstitutionalization, introduction ofinclusive education and other related activities) Why do you think so?
2. How effective is the introduction of that system in Armenia? Why do you think so?Do you think the Armenian society, the parents, the teachers and the children areready for that change?
3. Are the social service providers /case managers ready for those reforms?4. How would you describe the process of implementation of the reforms and the
situation of children before and after the reforms? How has the situation of thechildren changed?
5. What changed in your activities due to the reforms? What functions did you stopcarrying out and what new functions did you receive?
6. How do you think the reforms in implementation are related to state policy, prioritiesand the interests of children?
7. Are the actions undertaken in the frameworks of the reforms coordinated well? Howdo the various parties cooperate?
A2. Participation in training courses related tothe reforms
8. Did you participate in training courses in the frameworks of the reforms? When,where and how long were the courses? How would you rate those trainings ingeneral?
9. Rate trainings according to: a) Relevance/necessity of topic b) Contentsc) Level of preparedness of training specialistsd) Duration (long enough or not)
e) usefulness and applicability, correspondence to Armenian reality
10. Did that training course meet your needs?11. What changes occurred due to the training?12. Have you applied the knowledge gained from the training?13. Do you think your current knowledge and skills are adequate for working
effectively? If not, then why? What sorts of additional knowledge/skills do you thinkyou still need?
14. What would you recommend to increase the effectiveness of training courses?
Effectiveness
Observation of children and identification of needs15. Do you have monthly or annual plans of case management with families? How
realistic are those plans?16. Do you have mechanisms for the observation of children and identification of needs?
What are those? Do they work? If not, then why?17. What organizations do you cooperate with in your work? Please describe the main
directions of cooperation. What kind of problems do you face in the course of thesecollaborations? How do other people, organizations and institutions get involved in
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
B1. Process of working with children
the process of providing support to the child? How willing and motivated are they?18. Two types of integration is done: with the school and with the family. How are these
processes going? How does the government respond to challenges arising in theprocess?
19. How is the cooperation done in the following chain: parent-child-social worker-casemanager-institution-state agency?
20. What should be done to increase the effectiveness between various organizations andstakeholders?
B2. Perceptions of effectiveness of work withchildren involved in the reforms
21. How do the implemented reforms help children enrolled in the system? Do they havepositive or negative impacts on the children? Is there a difference between theimpacts on boys and on girls? Why do you think so?
22. Where are the most favorable conditions for the child in the following chain:institution-foster family-biological family?
23. What issues did you have to face in relation to your work as a result of the reforms?How do you deal with these issues? What type of issues still need solutions?
24. What kind of changes have you noticed resulting from your work? 25. What group of children in the system benefits the most from your work? Why do you
think so? Do girls adapt more easily than boys?26. What would help you to make your work more effective?
Efficiency
C1. Assessment ofconditions required for
the reforms
27. Do you think you have the adequate knowledge required for working with childrenunder the new system?
28. What types of technical resources do you need for the work with children/transportation, etc./?
29. What do the children need that is not provided to them and the lack of which hinderstheir effective wellbeing? Is there a difference between boys and girls in terms ofneeds?
30. What kind of assistance is needed to increase the efficiency of the system ofreforms?
Sustainability
D1. Perceptions of sustainability of the project
31. How effective is the introduction of the new system deemed by the society/community, parents, children, system staff/? What kind of feedback do you get?
32. Are the children ready for the shift to the new system? What about the parents, thespecialists and the institutions? Why do you think so?
33. How do you personally rate the impact of your activities? How are they perceived bythe members of the community?
34. How do you think the implementation of reforms affects the communities? Whatsorts of visible changes have you noticed among the children in your community?
35. What should be changed in the process of implementation of the reforms?36. Do you think the current Armenian society is ready for these reforms? Why? 37. What should be done to have a smooth implementation of reforms carried out in a
fashion acceptable by all?
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 5: EVALUATION MATRIX
Rel
evan
ce
# Are the project activities/components relevant to the actual/defined needs of the
vulnerable families with children? Were the objectives clear and feasible? Do the
main components of the project contribute to the planned objectives and logically
interlinked? How well are they linked to each other?
FG: With parents of vulnerable childrenIn depth Interviews with:
1. Families with vulnerable children 2. Foster families 3. Case managers 4. Staff and representatives of institutions, including special schools 5. USAID, UNICEF, Implementing Partners 6. RA MoE, MoLSA
# the Government committed (both in terms of timing and financially) to the
project? How the project is aligned with and supports the national development
plans, strategies and national plans of action? Is the project in line with the current
priorities of the country?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners
# Has the project involved relevant stakeholders through consultative processes or
information-sharing during its preparation phase? Was the needs
assessment/analysis carried out at the beginning of the project reflecting the various
needs of different stakeholders? Are these needs still relevant? Have there any new,
more relevant needs emerged that the project should address?
Expert Interviews and/or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners3. Foster families4. Case managers
Document review# To what extent were the reform initiatives informed by the needs and interests of
diverse groups of stakeholders (also from gender equality perspective)?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners In depth Interviews
3. Families with vulnerable children 4. Case managers 5. Trained teachers
# What are the beneficiaries’ views about the services provided? What are their FG
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
views about improving services (if there is a need to improve) effectiveness? 1. Families with vulnerable children 2. Teachers
Quantitative Survey 1. Teachers
In depth Interviews or a specially utilized method (such as essays ordiscussions)
1. Children
Eff
ecti
ven
ess # How effective has the project been in establishing ownership by the stakeholders?
Can the project management and implementation considered as participatory. If yes,
does this approach contributes to achievement of the project objectives? Has the
project been appropriately responsive to political, legal, economic, institutional (and
other) changes?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners
Document review
# Is the project making sufficient progress towards its planned objectives? What are
the key achievements, challenges and implementation lessons? Will the project
likely achieve its planned objectives upon completion and demonstrate success?
# Are the strategies appropriate and effective to achieve the planned objectives?
What, if any, alternative strategies would have been more effective in achieving its
objectives?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners FG
1. Families with vulnerable children 2. Teachers
Quantitative Survey 1. Teachers
In depth Interviews or a specially utilized method (such as essays ordiscussions)
1. Children# Are the strategies appropriate and effective to achieve the planned objectives? What, if any, alternative strategies would have been more effective in achieving its objectives?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners
# How effectively the family support services are provided in regional (and
community) level and how they are in line with actual needs?
FG1. Teachers 2. Parents of vulnerable children
In depth interviews
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
1. Families of vulnerable children 2. Case managers 3. Children of institutions 4. Family and community based alternative services
Eff
icie
ncy
# To what extent has UNICEF and implementing partners made good use of the
human, financial and technical resources, and has used an appropriate combination
of tools and approaches to pursue the achievement of project results in a cost-
effective manner?
Expert interviews or in depth interviews 1. MoE, MoLSA2. UNICEF, USAID, Implementing partners
Document review
# Was there a clear distribution of roles and responsibilities of key factors involved,
including staff, implementing partners and governmental partners towards the
achievement of project objectives?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document review
# To what extent budgetary allocations by the donor between the partners were
efficient? To what extent did UNICEF and implementing partners capitalize on
other complementary initiatives to the project to reinforce the results of the project?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document review
# Has the project done appropriate financial planning and reporting?
Has the management of funds (and reporting) been done in an efficient manner?
Has the co-funding/contribution from different project partners been made at the
expected level? Are there established procedures for checks and controls in the day-
to-day financial management, procurement of good and services and decisions
regarding resource allocation for the project?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document review
# Have project funds and activities been delivered in a timely manner?
If not, what were the bottlenecks encountered? Were the procedures put in place for
staffing,
systems (administration, accounting, other) and reporting efficient?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document review
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
Su
stai
nab
ilit
y # To what extent have the UNICEF country office and project partners been able to
support the government and beneficiaries in developing capacities and establishing
mechanisms to ensure ownership and the durability of effects under the reform
initiatives?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document reviewsFG
1. Teachers 2. Families of vulnerable children
In depth interviews 1. Families of vulnerable children 2. Case managers 3. Staff and representatives of institutions 4. Family and community based alternative services
# Is there a need to scale down the project (i.e. if the project duration is shorter than
planned)? If so, do project objectives and strategies have to be adjusted?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document reviews FG
1. Teachers 2. Families with vulnerable children
Quantitative Survey 1. Teachers
In depth interviews 1. Families of vulnerable children 2. Case managers 3. Staff and representatives of institutions 4. Family and community based alternative services
# Has the project successfully built or strengthened an enabling environment (laws,
policies, people's attitudes etc.)?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document reviews FG
3. Teachers
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
4. Families with vulnerable children Quantitative Survey
2. Teachers In depth interviews
5. Families of vulnerable children 6. Case managers 7. Staff and representatives of institutions
1. Family and community based alternative services# How does the government intend to maintain the provision of family support
services in future? Are there any specific/envisioned plans/projects?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document review
# What are possible sustainability mechanisms for each of the project component?
Can the project approach or results be replicated or scaled up by national partners or
other actors? Is this likely to happen? Can any unintended or unexpected positive or
negative effects be observed as a consequence of the project's interventions?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners Document reviewFG
5. Teachers 6. Families with vulnerable children
Quantitative Survey 3. Teachers
In depth interviews 8. Families of vulnerable children 9. Case managers 10. Staff and representatives of institutions
1. Family and community based alternative services
Imp
act # Has the project contributed to or is likely to contribute long-term social changes
for vulnerable families and children? What difference has it made, including policy
changes? What are the realistic long-term effects of the project on de-
institutionalization of children?
Expert Interviews or in-depth interviews with key informant 1. GoAm, RA MoE, MoLSA, NIE, NIE regional branches, regional
administrations, regional psychological and pedagogical supportcenters, child and family support centers
2. USAID, UNICEF, Implementing Partners
Document reviewFG
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services aspart of Child Care Reform” project UNITED NATIONS CHILDREN'S FUND in Armenia
1. Teachers 2. Families with vulnerable children
Quantitative Survey 1. Teachers
In depth interviews 2. Families of vulnerable children 3. Case managers 4. Staff and representatives of institutions 5. Family and community based alternative services
# Has the project increased the satisfaction with family and community based
alternative services?
FG1. Teachers 2. Families with vulnerable children
Quantitative Survey 1. Teachers
In depth interviews 2. Families of vulnerable children 3. Case managers 4. Staff and representatives of institutions 5. Family and community based alternative services
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care,
Family Support and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 6 RESULTS FRAMEWORK: M&E PLAN
Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and Inclusive Education
Services as Part of Child Care Reform
Monitoring & Evaluation Plan
Revision 1June 2016
2Monitoring & Evaluation Plan, Revision 1
Table of Contents
Toward Social Inclusion of Vulnerable Children in Armenia: Expanding Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform 1List of Acronyms 31. Overview 42. Summary of the Program and Objectives 4
Program Goal and Objectives 63. Monitoring Component 6
Indicators 7Baselines and Targets for Performance 7Data Disaggregation 7Data Quality Reviews (DQR) 7
4. Progress, Annual and Final Reports 85. Evaluation Component: Mid Term and Final Evaluations 96. Implementation and Management of M&E 10
Responsibilities 107. Review and Revision of the M&E Plan 118. Coordination of M&E Data Gathering 11
Annex 1. Theory of Change (ToC) 13
Annex 2. Performance/Indicator Tracking Table 15
Annex 3 Indicator Definition Table 26
3Monitoring & Evaluation Plan, Revision 1
List of Acronyms
AOR Agreement Officer’s Representative
CRC Convention on the Rights of the Child
DPO Disabled People’s Organization
DQA Data Quality Assessment
DQR Data Quality Review
HACT Harmonized Approach to Cash Transfer
ICF International Classification of Functioning, Disability and Health
IDT Indicator Definition Table
ITT Indicator Tracking Table
M&E Monitoring and Evaluation
NGO Nongovernmental Organization
NSS National Statistical Service
PCR Program Completion Report
QA Quality Assurance
ToC Theory of Change
UNICEF United Nations Children’s Fund
USAID United States Agency for International Development
USG United States Government
4Monitoring & Evaluation Plan, Revision 1
1. Overview
Monitoring and Evaluation (M&E) is essential for UNICEF’s resultsbased approach to program management towards realization of child rights, particularly the most disadvantaged, vulnerable, and excluded. UNICEF’s continuum for results is designed to foster learning and accountability and serve as a model for dialogue about development results among government and development partners and other stakeholders. This is a key component in UNICEF in Armenia’s work Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform (hereinafter “the Program”). It remains incorporated into all facets of the Program cycle, from design through program closeout.
The M&E Plan/Framework serves the following functions:
Explains in detail how UNICEF in Armenia and USAID will monitor the Program to determine whether they are achieving their intended results and to measure larger impacts over time through evaluations.
Outlines any M&E requirements that UNICEF in Armenia and implementing partners, namely Save the Children, World Vision, and Bridge of Hope, must meet during the implementation of the Program.
Serves as a guide for the Program implementation and management, so that UNICEF, beneficiaries, and other stakeholders understand the objectives and targets they are responsible for achieving under this Program and are aware of their progress towards those objectives and targets during implementation.
Establishes a process to alert implementers, stakeholders, and UNICEF to any problems in Program implementation and provides the basis for any needed program adjustments.
This M&E plan is considered a binding document for UNICEF in Armenia and implementing partners Save the Children, World Vision, and Bridge of Hope. The listed organizations are required to provide respective data under components they are responsible for including the indicator tracking table which needs to be completed on a quarterly basis.
Any significant changes to the approved M&E plan will require approval of USAID.
2. Summary of the Program and Objectives
Although Armenia ratified the Convention on the Rights of the Child (CRC) in 1992, which stipulates the right of the child to live in a family environment, the country remains reliant on institutional care for children in difficult life circumstances. In 2009, the UN Guidelines for alternative care were established to implement care provisions per the CRC that strongly recommend for children to be brought up in family environments and clearly state that socioeconomic status of the family should not be a reason for separating a child from his/her family.
5Monitoring & Evaluation Plan, Revision 1
Developing a system for the protection of children’s rights ranks high on the Government of Armenia’s agenda. Deinstitutionalization of children from residential care institutions in 2007 was an important achievement in the reform of child protection services: it led to restructuring of 17 boarding schools and reintegration of approximately 4,000 boarding school students into their biological families. In 2011, the government reaffirmed its dedication to the deinstitutionalization of children and declared it a government priority. Despite this progress, Armenia currently has 40 residential institutions, including 8 childcare institutions (6 state and 2 nonstate), 9 care and protection boarding institutions (8 state and 1 nonstate), and 23 special educational institutions. This translates into 4,000 institutionalized children, approximately 1,500 of whom are in state institutions, 300 in private care institutions, and 2,200 in special educational institutions. The majority of those institutionalized children have at least one living parent. Children in special schools largely lack access to inclusive education and diverse social and rehabilitation services in their communities of origin and, therefore, are enrolled in special schools as a last resort.
Primary reasons identified reasons for child institutionalization are the lack of communitybased services to assist families and children in need, especially children with disabilities from the families with poor economic conditions; the lack of educational programs able to accommodate varied developmental capacities; an absence of community services for children with special needs; the lack of information for parents on the conditions of children in special schools; and limited coverage for the recently introduced system of diversified family assistance.
The 2013 Concluding Observations of the Committee of the Rights of the Child recommend that the state ensures institutional care is used only as a last resort and that adequate safeguards and clear needsbased and best interests of the child criteria.1 Furthermore, it recommends mobilizing efforts and resources to strengthen families, promote family reunification, and limit further separation of children from families to a necessary minimum. The strategy ensures that the most vulnerable families have access to basic universal services and social assistance programmes2 through reallocation of funds from residential care to community and familybased services3 and proper case management work for vulnerable children and their families. In parallel, it is imperative to strengthen the intersectoral cooperation on child issues where the child and his/her rights are viewed as a holistic matter for policy making.
Ensuring the rights of children with disabilities to family life and community, social, and educational inclusion should be a priority. According to a recent survey conducted by UNICEF in Armenia4, 13% of children with disabilities live in residential institutions (either orphanages or special boarding schools5). There are few communities where residential institutions offer rehabilitation services, education support, daytime care, development services, and social support for the families of disabled children. In this regard, the Committee of the Rights of the Child recommends that the state takes measures to deinstitutionalize children with disabilities and
1 Concluding observations of the Committee of the Rights of the Child, combined third and fourth periodic reports of Armenia, 27 May 14 June 20132 This is one of the recommendations of the Committee of the Rights of the Child (2013)3 Elena Andreeva (2010), Towards Alternative Child Care Services in Armenia: Costing Residential Care Institutions and Community Based Services. UNICEF4 It’s about Inclusion, UNICEF, 2012, p.405 It’s about Inclusion, UNICEF, 2012, p.24
6Monitoring & Evaluation Plan, Revision 1
provide them alternative familybased care options and education in mainstream schools as well as allocate adequate human, technical, and financial resources to ensure early detection and rehabilitation services.
According to UNICEF’s analysis, although institutionalization affects both boys and girls equally, gender discrimination is apparent, as 68% of boys with disabilities stay with families as opposed to only 32% of girls. Additionally, the fact that boys are preferred over girls when it comes to adoption further leads to an undervaluing of girls in society. To ameliorate this situation, gender sensitive approaches need to be applied for girls with disabilities. Finally, along with several reported cases of misuse of financial resources at residential institutions, it is also a costly and inefficient service for a country with a very limited social budget and high rates of poverty.
Program Goal and Objectives
The overall goal of the program is to support the Government of Armenia to ensure that children realize their rights to live in a family environment and access communitybased social and educational services. The project will ensure that the child care system in Armenia significantly reduces reliance on largescale institutions through the establishment of a network of communitybased services responding to the needs of the children, families, and community. The Program will achieve the overall goal through the following objectives (summarized in the Program Theory of Change, See below):
Institutional mechanisms for the functioning of social service system are improved. Alternative communitybased family support services are accessible to vulnerable
children and families. Family substitution service system is strengthened with the diversification of types of
foster care establishment of monitoring and evaluation system. Inclusive Education System is strengthened to provide quality learning for all children
through set up of pedagogical psychological support centers and mainstream school trainings.
An enabling legal and regulatory framework is established to support childcare reform. Legal norms and social perceptions regarding children with disabilities have changed
favorably towards inclusion.
7Monitoring & Evaluation Plan, Revision 1
Figure 1. Theory of Change (ToC)
8Monitoring & Evaluation Plan, Revision 1
General public supports childcare reform to expand and adapt basic services for
most vulnerable children and adolescents
Community heads, town hall representatives, and city councils support vulnerable children to
have access to communitybased social and educational services
Accessible alternative community‐based family support services to vulnerable children and families
Strengthened family substitution service system with the diversification of types of foster care, establishment of monitoring and evaluation system
UNICEF Core Roles System Change Mediumterm changes Long term changes
fa
Inputs and Outputs Outcomes Impact
Voice
Leveraging
National Dialogue
M&E
Knowledge Exchange
Policy Advice and TA
Modelling
Children’s rights to live in a family environment and access communitybased social and
educational services are realized
Change in legal and social norms towards foster care and inclusion
Established national framework for institutional cooperation between services with sample protocols of
cooperation developed Established or expanded family
support services
Developed and approved foster care policy
Accessible Foster Care services for children in diverse, difficult life
circumstances
Approved by the Government and by the National Assembly recommendations for improving legal and regulatory framework
Established pedagogical support centers to support inclusive schools in
their geographical area Inclusion principles accepted by
teachers and directors for mainstream schools to provide education to all
children
Children claim and exercise their rights to access basic health, education and social/child protection services in their communities
An enabling legal and regulatory framework to support child‐care reform and improved institutional mechanisms for the functioning of social service system
NGOs, think tanks, civil society leaders, and media advocate and
influence public opinion to fulfil child rights to
live in a family environment and
strengthen inclusion
Strengthened inclusive education system to provide quality learning for all children
Legal norms and social perceptions regarding CWD changed favorably towards inclusion
9Monitoring & Evaluation Plan, Revision 1
3. Monitoring Component
Program and Activity performance will be monitored systematically and on an ongoing basis through the regular indicator tracking system, in line with UNICEF HACT requirements; programmatic visits; spot checks; and audits. Analysis of the information derived from these monitoring mechanisms will inform needed programmatic adjustments with the view towards improving the overall impact of this Program.
Indicators Lower level indicators (outputs) and their targets will mainly be drawn from program joint work plans developed with Program partners. Higher level indicators (outcome) and their targets will be directly linked to the analysis to be conducted during the Program implementation to estimate the impacts of the interventions (Indicator Tracking Table (ITT), Annex 2). The data for the analysis will be collected through surveys/evaluations.
The Indicator Definition Table (IDT, Annex 3) included in the M&E Plan will provide a detailed definition of each indicator, unit of measurement, source of data, method of data collection, frequency of data collection, and the entity responsible for collecting the data. Indicator Tracking Table that is part of the M&E Plan will ensure tracking of interim progress towards goals. All baselines and targets relevant to specific activity will be analyzed and specified.
Baselines and Targets for PerformanceThe baselines and targets for each indicator are shown in the Performance Tracking Tables. Baselines for some of the indicators will be collected by program partners. Outputs and their targets are mainly derived from Program joint work plans. Outcomes and their targets are linked to the analysis to be conducted during the program implementation to estimate the impacts of the interventions. The data for the analysis will be also collected through ad hoc studies, surveys, and evaluations.
Data DisaggregationProgram performance indicators will be disaggregated by gender where appropriate and feasible. This way, the Program implementers will assess program effectiveness at supporting gender balance in its activities. Disaggregation by age, disability status, ethnicity, and income quintile can be performed on need basis during the implementation.
Data Quality Reviews (DQR) – by UNICEF
The goal of the DQR is to conduct an independent review of the quality of indicators and data under the Program through verification of the quality and the consistency of performance data over time, across different implementers/partner organizations and reporting institutions (i.e. administrative data, if relevant). Such data quality reviews will also identify cases in which the highest degree of data quality is not possible given the realities of the data collection circumstances. These assessments will cover data reported from Program partners
10Monitoring & Evaluation Plan, Revision 1
and other data sources, such as administrative (i.e. data provided by Ministries, NSS official data, etc.), as necessary.
The particular objectives for the data quality reviews will be identification of the following parameters: i) what proportion of the data collected during the Program implementation has quality problems (completeness, conformity, consistency, accuracy, duplication, integrity); ii) which of the records in the dataset are of unacceptably low quality; iii) what are the most predominant data quality problems within each field provided by Program partners.
The DQR should be conducted once during the life of the Program (the date to be agreed with USAID), focusing on verifying the universally accepted aspects of data quality (such as accuracy, reliability, timeliness, objectivity, and relevance) through review of all components of data collection, processing, analyses, and reporting processes. UNICEF together with Program partners will contract the data quality reviewers to perform the task. UNICEF M&E specialists, the Child Rights Systems Monitoring Specialist, and the Child Protection Specialist will also regularly check the quality of data collected and provided by partners. USAID AOR will be present at the DQR and review the data systems firsthand.
Data Quality Assessments (DQA) – by USAID
USAID requires that a DQA must occur for all externally reported indicators sometime within three years of data collection and before being reported. The USAID AOR is responsible for conducting the DQAs for indicators in the M&E Plan that will be reported to USAID/Washington. Implementing Partners and third party M&E contractors will often be involved and engaged in conducting DQAs. If the AOR does not conduct the DQA, he/she is responsible for certifying the DQA, once done, and for addressing findings with the implementer and ensuring that corrective actions are taken.
Data should reasonably meet these five standards of data quality: validity, integrity, precision, reliability, and timeliness.
4. Progress, Annual and Final Reports
Program partners will produce quarterly reports (Progress Reports) starting from the second quarter (Feb/Mar 2015). Progress Reports will track the progress of the Program by including data on the indicators described in the Monitoring Component, methodology of data collection, and analysis of the data alongside with other information related to program success, any challenges and lessons learnt. These documents will also discuss field visits performed by implementing partners to observe the performance and identify the cases of special interest (if any). UNICEF/Armenia’s Child Protection Specialist will review and approve the reports.
Based on progress report data submitted by program partners, the UNICEF Child Protection Specialist will compile quarterly progress updates and share with USAID/Armenia. The Quarterly Progress Updates shall collectively discuss the work of the program highlighting:
Achievements during the quarter;
11Monitoring & Evaluation Plan, Revision 1
Update indicators; Timeline of events; Summary of obstacles and issues and how they are being addressed; Program activities planned for the next quarter.
The quarterly updates shall cover US Government fiscal quarters and be due within 15 days after each fiscal quarter.
The Annual Supplementary Reports will be prepared by Program partners by end of quarter four and provide additional information on accomplishments and developments of Program implementation as well as lessons learnt to complement UNICEF annual country office reporting. Some of the highlights of Annual reports can be made available to the public through UNICEF website and other means of communication as they become available.
UNICEF will also prepare a final report Program Completion Report (PCR) to be submitted to USAID within 90 days of the agreement end date. All three partners (Save the Children, World Vision, Bridge of Hope) will contribute to the development of this report. The PCR should provide: A concise description of the Program from proposal to completion including
objectives, strategies and activities undertaken within the program implementation and corresponding results;
A preliminary assessment of the Program’s outcomes; Identification of beneficiaries including relevant characteristics, such as gender, age,
and income level, and degree of participation; Identification of stakeholders supporting the Program; A summary of problems/obstacles encountered during the implementation and how
the obstacles were addressed; An executive summary of the accomplishments and results achieved; A preliminary assessment of the Program’s sustainability that is, its likelihood to
reach the future monitoring targets established as a measure of the programs’ sustainability;
Lessons learned, including changes that might have been made in M&E criteria, policies, procedures and practices related to the program.
1. Evaluation Component: Mid Term and Final Evaluations
Evaluation is an essential element for UNICEF in Armenia. Midterm and/or final evaluations will be conducted during the life of the Program to improve program management and provide lessons for future program implementation as well as assess the effectiveness, efficiency, relevance, impact, and longterm sustainability of the Program. UNICEF will engage independent evaluators to conduct evaluations.
Midterm evaluation planned in 2017 will assess progress in meeting the Program goals and outcomes. They will provide early lessons learned and identify significant discrepancies between expected results and actual achievements, including an analysis behind the reasons for discrepancies between actual and projected indicators.
In 2019/2020, UNICEF, jointly with the Government of Armenia, will arrange for final evaluation (endofproject evaluation) to address the relevance, effectiveness, efficiency,
12Monitoring & Evaluation Plan, Revision 1
impact and sustainability of family and community based alternative services and their role in prevention of institutionalization. The evaluation findings will inform decisionmakers on alternative care, family support mechanisms and inclusive education as well as contribute to learning and knowledge sharing on social inclusion of vulnerable families with children.
In addition, the following issues will be covered under the evaluation:
Effectiveness of the Program at meeting goals; Reasons behind the success or failure to achieve goals, objectives and targets; Unintended results of the program (positive and negative); Longterm sustainability of results; Lessons learned and recommendations.
Both midterm and end of project evaluations will be conducted with the involvement of external evaluation experts selected through an open tender process. The entity selected to conduct the evaluation will decide the overall methodology to be used in the evaluation based on a prior, agreedupon statement of work developed by UNICEF.
A sound quality assurance (QA) system for evaluation is critical to UNICEF’s mandate and to enhance the organization’s reputation as a knowledge leader on children, ensure scarce resources are not wasted, and to protect the children, women, and others who participate in the research, and whose lives are affected by its outcomes. Therefore, all evaluations commissioned and/or conducted by UNICEF will go through the quality assurance system and be subject to assessment by the UNICEF Office of Research.
2. Implementation and Management of M&E
Prior to starting implementation of the program activities, UNICEF will orient Program implementers on how program performance is to be measured and will provide necessary support to comply with the M&E Plan. UNICEF may make adjustments to the M&E Plan as needed, provided any modification or amendment of the M&E Plan (including ITT) and communicate the changes to the implementing partners.
Data StorageUNICEF will share the M&E plan including Indicator Performance Table, Indicator Definition Table and Indicator Tracking Table with the Implementing Partners. Each partner will collect the information they are responsible for according to Indicator Performance and Tracking tables. Partners will use the same approach while collecting the data to ensure the reliability, validity and integrity of data. UNICEF will consolidate the data provided by each IP in a database.
Implementing partners will share with UNICEF the reports, policy documents, program implementation documents the copies of which will be stored at Child Protection section.Filing and record keeping systems for both hard copies and digitized information will be set up.
Responsibilities
UNICEF in Armenia assumes general M&E responsibilities, including the following:
13Monitoring & Evaluation Plan, Revision 1
Guide the establishment of the M&E system, including datacollection, dataanalysis and reporting systems;
Ensure that the M&E Plan is modified and updated as improved information becomes available;
Ensure that findings are disaggregated by gender, age, and income, as applicable; Support to design midterm and final evaluations and surveys; Participate in monitoring through site visits, review of program reports and secondary
data; Facilitate learning exchanges and information dissemination; Organize and oversee regular independent data quality reviews.
UNICEF Armenia M&E specialist and Child Protection Specialist, in cooperation with USAID Agreement Officer’s Representative, will ensure realization of the referenced tasks.
3. Review and Revision of the M&E Plan
The M&E Plan is designed to evolve over time, adjusting to changes in program activities and improvements in performance monitoring and measurement. In the fourth quarter of every year or as necessary, the M&E, Child Rights Systems Monitoring Specialist, and the Child Protection Specialist will review how well the objectives set in the M&E Plan are accomplished. The review is intended to ensure that the M&E Plan measures program performance accurately and provides critical information on the need for changes in project design. The review will ensure that the M&E plan developed for this Program:
Show whether the logical sequence of intervention outcomes are occurring; Check whether indicator definitions are precise and timely; Check whether M&E indicators accurately reflect program performance; Update indicator targets, as allowed; Add indicators, as needed, to track hitherto unmeasured results.
The M&E Plan will be revised by UNICEF, in agreement with USAID, when the need for change has been identified in the review. Any significant changes to the approved M&E plan will require additional approval of USAID. UNICEF will communicate the revised M&E Plan to the Program partners.
4. Coordination of M&E Data Gathering
UNICEF will receive data disaggregated by gender (further disaggregation, such as age, disability status, ethnicity, and income quintile can be taken into consideration at later stages when need be) from Program implementers (Save the Children, World Vision, and Bridge of Hope) responsible for tracking respective indicators on quarterly/annual basis as well as the National Statistical Service (NSS) and respective lineministries. The Indicator Tracking Table of the M&E Plan will be updated by UNICEF each quarter, once the data is compiled and reported by partner organizations. The following diagram6 (See Diagram 1) displays the flow of information from these organizations to UNICEF in Armenia7.
6 The diagram is meant to depict the flow of information, not the supervisory relationship of the organizations.
14Monitoring & Evaluation Plan, Revision 1
Diagram 1. The flow of information from different agencies to UNICEF in Armenia
National Statistical Service
Ministry of Labor and Social
Affairs
Ministry of Education
Ministry of Territorial
Administration
7 The Indicator Definition Table provides the definition of each indicator, unit of measurement, methodology and frequency of data collection as well as outline the information that will be collected and reported by each institution, methodology and frequency of data collection.
Save the Children
World Vision
USAID in Armenia
Bridge of Hope
UNICEF in Armenia
15Monitoring & Evaluation Plan, Revision 1
16Monitoring & Evaluation Plan, Revision 1
Annex 2.Performance/Indicator Tracking Table
Program Goal Targets
Objectives Indicators Units Baseline(2014)
Year 1 Year 2 Year 3 Year 4 Year 5
Goal: Children realize their rights to live in a family environment and access communitybased social and educational services.
Percent decrease of children among children in targeted residential care facilities
Percentage 0 0 10% 20% 35% 50%
Number of children from target institutions reunified with families and remaining in families for at least six months in target communities
NumberCumulative
40(state
deinstitutionalization program)
40 250 450 800 1100
Number of children from targeted institutions placed in alternative family environment in target communities remaining in alternative families for at least six months
NumberCumulative
64 64 120 250 400 700
Number of children removed from residential care and from target communities using communitybased social services
NumberCumulative
2400 2400 2800 3200 3900 4500
Number of children and families using family support services at least 3 times over a year
Number cumulative
150 300 450 600
Number of children with special educational needs benefitting from communitybased inclusive education at least one year
NumberCumulative
3000 3000 3600 4500 5000 5500
Objectives Indicator Units Baseline Year 1 Year 2 Year 3 Year 4 Year 5 Implementing
17Monitoring & Evaluation Plan, Revision 1
(2014) 2015 2016 2017 2018 2019 partnerOUTCOME #1: Institutional mechanisms for the functioning of social service system are set up
Child protection is a part of integrated social services
Annual number of referrals of child cases to social services
NumberCumulative
0 0 370 700 1200 1800 UNICEF in Armenia
Output 1.1: Case managers are fully functional in the target regions, equipped to perform social needs assessments and followup the cases of children
1.1.1 Training package for the case managers is developed
Training module developed Date Aug 2015
UNICEF in Armenia
1.1.2 Respective bodies identified and trained, including regional level child protection units, case managers involved in foster care processes
Number of bodies trained NumberCumulative
5 25 35
Output 1.2: A national framework for institutional cooperation between services is established and a guideline on its implementation is developed and approved
1.2.1 A national framework for institutional cooperation between services on child care and protection is established
A government decree regulating institutional cooperation among services adopted
Date Feb 2016
UNICEF in Armenia1.2.2 Guideline on implementing
institutional cooperation is developed and approved
Developed guideline is submitted to MOLSA
Date Sep 2016
Developed guideline is approved by MOLSA
Date Nov 2016
Output 1.3: Territorial social plans addressing the needs of the socially vulnerable populations are in place
1.3.1 Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved of regional administration
Amendments in territorial social plans approved
Date Date 2015 (Lori)
TBD TBD TBD TBD
Objectives Indicator Units Baseline (2014)
Year 12015
Year 22016
Year 32017
Year 42018
Year 52019
Implementing partner
18Monitoring & Evaluation Plan, Revision 1
OUTCOME #2: Alternative communitybased family support services are accessible to vulnerable children and families
2.1 Increased access to community based alternative services in targeted communities
Number of communitybased services accessible to children and their families in difficult life circumstances
NumberCumulative
0 0 4 8 10 12 World Vision
2.2 Enhanced knowledge of staff on newly introduced services
Percent increase of the knowledge of the staff on newly introduced/expanded services
Percentage8 0 N/A 15% 15% 15% N/A
Output 2.1: Family support services (medical and social rehabilitation, afterschool care, psychosocial support, etc.) are established or expanded
2.1.1 Residential institutions closed based on closure plan of each institution
Number of institutions closed Number Cumulative
0 1 2 4 6
World Vision2.1.2 Residential institutions transformed based on the transformation plan of each institution
Number of residential institutions transformed into alternative services based on transformation plan
NumberCumulative
0 4 8 12 16
2.1.3 Availability of trained staff with the capacity to provide services based on newly introduced and/or expanded service/s
Number of staff trained to deliver new services
NumberCumulative
0 0 20 40 60
2.1.4 Social needs of the child and the family assessed
Number of institutionalized childcases assessed on social needs
NumberCumulative
0 450 1200 2600
2.1.5 Individual care plans developed and followedup
Number of social cases conducted by case managers
NumberCumulative
0 450 1200 1800
Number of social cases with demonstrated progress
Number Cumulative
0 200 600 900
8 Note that the presented percentages are not cumulative as the participants in each training group will be different. The percentage increase is going to be calculated compared to its baseline which will be estimated via pretest.
19Monitoring & Evaluation Plan, Revision 1
Percent of social cases solved by case managers
Percentage 0 10% 35% 60%
2.1.6 Family support services, including medical and social rehabilitation, afterschool care, psychosocial support, and other services established/expanded in selected regions
Number of community based services established classified by type
NumberCumulative
24 24 26 28
Objectives Indicator Units Baseline (2014)
Year 12015
Year 22016
Year 32017
Year 42018
Year 52019
Implementing partner
OUTCOME #3: Family substitution service system is strengthened with the diversification of types of foster care, establishment of monitoring and evaluation system3.1 Expansion and diversification of foster care policy in Armenia to be approved by the Government
Foster care policies revised and approved
Date Oct 2015 Save the Children
3.2 Enhanced knowledge of families on being a foster family
Percent increase of the knowledge of trained families to be a foster family
Percentage9 0 15% 15% 15% N/A N/A Save the Children
Output 3.1: A comprehensive foster care policy is developed and approved by the government
3.1.1 A comprehensive foster care policy is developed and approved by the government
Approved policy on foster care
Date
3.1.2 Existing foster care system/framework is analyzed and compared to international standards
Foster care system in Armenia analyzed and compared to international standards
Date Feb 2015
Save the Children
3.1.3 Full foster care model in line Full foster care Date Sep
9 Note that the presented percentages are not cumulative as the participants in each training group will be different. The percentage increase is going to be calculated compared to its baseline which will be estimated via pretest.
20Monitoring & Evaluation Plan, Revision 1
with UNICEF pilot/modelling guidelines developed and customized to the Armenian context
model/framework developed 2015
3.1.4 Foster care policy package is drafted
Foster care policy drafted Date Jun 2015
3.1.5 System for monitoring and supervision of foster care placements is introduced
Foster care placement monitoring and supervision document adopted
Date Jan 2017
3.1.6 System for monitoring and supervision of foster care placements is operational
Foster care placement monitoring and supervision established
Date Mar 2017
Output 3.2: Foster Care services are available for children in diverse, difficult life circumstances
3.2.1 Database for foster families developed
Database of foster families developed
Date Jan 2017
Save the Children
Number of foster families registered in the database
Number Cumulative
30 45 80 95
3.2.2.Database for foster families linked to staterun database and maintained
Database of foster families linked to NORK database
Date Oct 2017
3.2.3 Awareness raising activities to promote foster care are organized
Number of TV programs broadcasted on local TV stations
NumberCumulative
0 0 6 9 9 9
3.2.4 Families for different types of foster care are identified and trained
Number of potential foster families trained
Number Cumulative
30 45 80 95
3.2.5 Children from residential care facilities in targeted communities removed and placed in foster
Number of children placed in foster care
NumberCumulative
15 30 50 80
21Monitoring & Evaluation Plan, Revision 1
families in cooperation with the Government of Armenia
Percent of children remaining in foster care after 6 months of being placed.
Percentage 0 80% 80% 80%
Objectives Indicator Units Baseline (2014)
Year 12015
Year 22016
Year 32017
Year 42018
Year 52019
Implementing partner
OUTCOME #4: Inclusive Education System is strengthened and expanded to provide quality learning for ALL children in targeted regions
4.2 Increased access to communitybased inclusive schools in selected regions
Number of children from 6 to18 transferred from special educational institutions to community based inclusive schools
NumberCumulative
0 0 200 500 800 1100
Percent of transferred children successfully completing at least one academic year
Percentage 0 0 40% 50% 55% 70%
Number of out of school and home schooled children in target regions decreased
NumberCumulative
1000 900 800 600 500 300
4.3 Enhanced capacity of the staff of Pedagogical support centers in conducting ICF based assessments
Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive
Percentage10 0 15% 15% 15% 15% 15%
10 Note that the presented percentages are not cumulative as the participants in each training group will be different. The percentage increase is going to be calculated compared to its baseline which will be estimated via pretest.
22Monitoring & Evaluation Plan, Revision 1
education and conducting ICF based assessments
Output 4.1: Pedagogical support centers are established to support inclusive schools in their geographical area
4.1.1 Enhanced capacity of communitybased inclusive schools in selected regions
Percentage of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers
Percentage 0 5% 10% 25% 35% 50% Bridge of Hope
4.1.2 The capacity of newly established pedagogical support centers is enhanced to support inclusive schools in their geographical area
Number of staff of Pedagogical Support Centers trained in supporting inclusive education and conducting ICF based assessments
NumberCumulative
0 40 80 160 200 240
Number of trained Pedagogical Support Centers staff conducting ICF based assessments
Number Cumulative
0 0 40 80 160 200
Output 4.2: Teachers and directors for mainstream schools (in the programme areas) understand and accept the principles of inclusion and have the capacity to provide education to ALL children4.2.1 Mainstream schools in targeted regions have the capacity to provide education to children with disabilities
Number of mainstream schools in targeted regions trained on inclusive education practices
Number Cumulative
139 209 299 469 590 764 Bridge of Hope
Objectives Indicator Units Baseline (2014)
Year 12015
Year 22016
Year 32017
Year 42018
Year 52019
Implementing partner
OUTCOME #5: An enabling legal and regulatory framework is established to support Childcare Reform
5.1. Local legislation is adjusted to respond to childcare reform directions
Number of legal recommendations submitted to the Government of Armenia
Number Cumulative
2 6 7 9 0 0
UNICEF in
23Monitoring & Evaluation Plan, Revision 1
Armenia5.2 Legal and policy framework is set up to allow outsourcing of social services to nonstate providers
Number of Social Assistance legal/policy amendments adopted
Number Cumulative
2 2 4 0 0 0
5.3. Master Plan for transformation of residential care institutions is developed
Rapid assessment of residential care institutions conducted
Date 0 Jan 2015
Output 5.1: Recommendations for improving legal and regulatory framework are approved by the Government and by the National Assembly, as appropriate
5.1.1 Family code and bylaws are adjusted to provide framework for transparent adoption process
Number of legal recommendations on adoption approved by the government of RA
Number 0 5 15
UNICEF in Armenia
5.1.2 Law on Social Assistance and by laws are amended to allow outsourcing of state social services to nonstate providers
Number of communitybased NGOs outsourced by state for providing service to vulnerable children and their families
NumberCumulative
2 5 10 15 18 22
5.1.3 Policy document on deinstitutionalization, transformation of residential institutions and types of alternative services developed
Types of alternative services defined
Date Mar2015
OUTCOME #6: Social norms regarding children with disabilities have changed favorably toward inclusion
6.1 People in Armenia have favorable attitude towards children with physical and intellectual disabilities
Percent increase in the positive attitude toward children with physical disabilities
Percentage 0 TBD UNICEF in Armenia
24Monitoring & Evaluation Plan, Revision 1
Percent increase in the positive attitude toward children with intellectual disabilities
Percentage 0 20%
25Monitoring & Evaluation Plan, Revision 1
USAID Standard indicators M&E Plan
Indicator Description Units Baseline
Year 1
Year 2
Year 3
Year 4
Year 5
Indicator #1
Number of social protection policy reforms drafted, adopted or implemented with USG support
Number 4 1 2 TBD TBD
Indicator #2
Number of vulnerable people benefiting from USGsupported social services
Total Number 1246 2200 2654 2046 TBD
Number of men Number 250 400 465 446 TBD
Number of women Number 373 801 697 670 TBD
Vulnerable children Number 623 999 1312 930 TBD
Indicator #3
Number of service providers trained who serve vulnerable persons
Total Number 3330 4712 5265 5265 TBD
Number of men Number 999 1885 1580 1580 TBD
Number of women Number 2331 2872 3685 3685 TBD
Indicator #4
Number of USG assisted organizations and/or service delivery systems
TotalNumberCumulative
112 176 130 130 TBD
Local NGO Number 2 5 4 4 TBD
26Monitoring & Evaluation Plan, Revision 1
strengthened who serve vulnerable populations
(nonDPO)
Disabled People's Organizations (DPO)
Number 1 1 1 1 TBD
Schools Number 102 162 125 125 111
Other Number 7 8 TBD TBD TBD
27Monitoring & Evaluation Plan, Revision 1
28Monitoring & Evaluation Plan, Revision 1
Annex 3Indicator Definition Table
Goal Indicators
Program Goal Indicator Definition of Indicator Units Source/ Responsible Entity
Methodology Frequency of Data
CollectionChildren realize their rights to live in a family environment and access communitybased social and educational services.
Percent decrease of children among children in targeted residential care facilities
Percent decrease of children (disaggregated by sex and age) among children in targeted residential care facilities (by type).
Percentage Line ministries, NSS, UNICEF in Armenia
Administrative data, program implementation documents, Final evaluation
Annual
Number of children from target institutions reunified with families and remaining in families for at least six months in target communities
Number of children reunified with their families in target communities (disaggregated by age, sex and type of disability, if relevant)
Number Cumulative
Line ministries, NSS, UNICEF in Armenia
Administrative data, program implementation documents, Final evaluation
Annual
Number of children from targeted institutions placed in alternative family environment in target communities remaining in alternative families for at least six months
Number of children placed in alternative families (disaggregated by type – kinship, foster, adoption) in target communities.
Number Cumulative
Official registrar, National Statistical Service, UNICEF in Armenia
Administrative data, program implementation documents, Final evaluation
Annual
Number of children removed from residential care and from target communities using communitybased social services
Number children and their families in difficult life circumstances accessing communitybased services
Number Cumulative
Line ministries, Service delivery units, NSS
Administrative data, program implementation documents, Final evaluation
Annual
Number of children and families using family support services at least 3 times over a
Number of children and families benefitting from family support services on
NumberCumulative
Line ministries, service delivery units, UNICEF in
Administrative data, program implementation
Annual
29Monitoring & Evaluation Plan, Revision 1
year average annually Armenia documents, Final evaluation
Number of children with special educational needs benefitting from communitybased inclusive education at least one year
Number of children (disaggregated by age and sex) with special educational needs benefitting from communitybased inclusive education
Number Cumulative
Institutions providing inclusiveeducation, UNICEF in Armenia
Administrative data, program implementation documents, Final evaluation
Quarterly
Program outcome and output level indicators
Purpose Indicator Definition of Indicator Units Source/ Responsible Entity
Methodology Frequency of Data
CollectionOUTCOME #1: Institutional mechanisms for the functioning of social service system are set up
Child protection is a part of integrated social services
Annual number of referrals of child cases to social services
Number of cases referred to case managers
Number cumulative
Regional administration, UNICEF in Armenia
Administrative data, program implementation documents, Final evaluation
Annual
Output 1.1: Case managers are fully functional in the target regions, equipped to perform social needs assessments and followup the cases of children
1.1.1 Training package for the case managers is developed
Training module developed
Training modules are developed for case managers and foster families
Date UNICEF in Armenia Training module package
Once
1.1.2 Respective bodies identified and trained, including regional level child protection units, case managers involved in foster care processes
Number of bodies trained
Number of regional level child protection units, case managers that are involved in foster care processes trained.
Number Cumulative
UNICEF in Armenia Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents
Twice a year
Output 1.2: A national framework for institutional cooperation between services is established and sample protocols of cooperation are developed.
30Monitoring & Evaluation Plan, Revision 1
1.2.1 A national framework for institutional cooperation between services is established
A government decree on institutional cooperation among services adopted
A government decree regulating institutional cooperation among services is developed and adopted
Date UNICEF in Armenia Government decree, program implementation documents
Once
1.2.2 Guideline on implementing institutional cooperation is developed and approved
Developed guideline is submitted to MOLSA
Guideline developed and submitted to MOLSA for review
Date UNICEF in Armenia Copy of guideline, program implementation documents
Once
Developed guideline is approved by MOLSA
Developed guideline is approved by the government decree
Date UNICEF in Armenia Government decree, program implementation documents
Once
Output 1.3: Territorial social plans addressing the needs of the socially vulnerable populations are in place1.3.1 Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved by the regional administration
Amendments in territorial social plans approved
Amendments in territorial social plans addressing the needs of the socially vulnerable populations are approved
Date UNICEF in Armenia Copy of approval, Administrative data, program implementation documents
Once
Program Goal Indicator Definition of Indicator Units Source/ Responsible Entity
Methodology Frequency of Data Collection
OUTCOME #2: Alternative communitybased family support services are accessible to vulnerable children and families
2.1 Increased access to communitybased alternative services in targeted communities
Number of communitybased services accessible to children and their families in difficult life circumstances
Number of communitybased social services utilized by the target communities and by children in residential care
Number Cumulative
Program partner/World vision
Administrative data, program implementation documents, Final evaluation
Annual
2.2 Enhanced knowledge of staff on newly introduced services
Percent increase of the knowledge of the trained staff on newly
Percent increase of the knowledge of the trained staff on newly
Percentage World Vision Pre/post test Annual
31Monitoring & Evaluation Plan, Revision 1
introduced/expanded services
introduced/expanded services
Output 2.1: Family support services (medical and social rehabilitation, afterschool care, psychosocial support, etc.) are established or expanded
2.1.1 Residential institutions closed based on the closure plan of each institution
Number of institutions closed
Residential institutions that are closed according to their closure plan
Number Cumulative
Program partner/ World vision
Administrative data, program implementation documents
Annual
2.1.2 Residential institutions transformed based on the transformation plan of each institution
Number of residential institutions transformed into alternative services based on transformation plan
Residential institutions that are transformed into alternative services based on transformation plan
Number Cumulative
Program partner/World vision
Administrative data, program implementation documents
Annual
2.1.3 Availability of trained staff with the capacity to provide services based on newly introduced and/or expanded service/s
Number of staff trained to deliver new services
Number of newly hired and/or old staff that are trained to deliver services
Number Cumulative
Program partner/World vision
Prepost training evaluation, Service contracts with trainers, training modules, Program implementation documents
Annual
2.1.4 Social needs of the child and the family assessed
Number of institutionalized childcases assessed on social needs
Number of institutionalized childcases assessed on social needs
Number Cumulative
Program partner/World Vision
Program implementation documents, Copy of assessed cases
Annual
2.1.5 Individual care plans developed and followedup
Number of social cases conducted by case managers
Number of social cases managed by case managers
Number Cumulative
Regional administration, World Vision
Administrative data, Program implementation documents, case reports
Annual
Number of social cases with demonstrated
Number of cases which registered advancement
Number Cumulativ
Regional administration,
Administrative data, Program
Annual
32Monitoring & Evaluation Plan, Revision 1
progress e World Vision implementation documents, case reports
Percent of social cases solved by case managers
Number of cases completed Percent Regional administration, World Vision
Administrative data, Program implementation documents, case reports
Annual
2.1.6 Family support services, including medical and social rehabilitation, afterschool care, psychosocial support, and other services established/expanded in selected regions
Number of community based services established classified by type
Number of community based services established that are classified by type (at least one of each type)
Number Cumulative
Regional administration, World Vision
Administrative data, Program implementation documents, reports
Annual
Program Goal Indicator Definition of Indicator Units Source/ Responsible Entity
Methodology Frequency of Data Collection
OUTCOME #3: Family substitution service system is strengthened with the diversification of types of foster care, establishment of monitoring and evaluation system
3.1 Expansion and diversification of foster care policy in Armenia to be approved by the Government
Foster care policy revised and approved
Foster care policy revised and approved
Date Program partner/Save the Children
Policy document; administrative data, program implementation documents, Final evaluation
Once
3.2 Enhanced knowledge of families on being a foster family
Percent increase of the knowledge of trained families to be a foster family
Percent increase of the knowledge of trained families to be a foster family
Percentage Program partner/Save the Children
Pre/post test Annual 11
11 The responsible organization will conduct data collection before and after each training through pre and post training evaluations.
33Monitoring & Evaluation Plan, Revision 1
Output 3.1: A comprehensive foster care policy is developed and approved by the government
3.1.1 A comprehensive foster care policy is developed and approved by the government
Approved policy on foster care
Developed and approved policy on foster care which includes status clarification of children eligible for foster care, criteria for foster parents and monitoring and supervision of foster care
Date Program partner/Save the Children
Policy document, Program implémentation documents
Once
3.1.2 Existing foster care system/framework is analyzed and compared to international standards
Foster care system in Armenia analyzed and compared to international standards
Foster care system analyzed and compared to international standards
Date Program partner/Save the Children
Copy of the report Once
3.1.3 Full foster care model in line with UNICEF pilot/modeling developed and customized to the Armenian context
Full foster care model/framework developed
Foster care model/framework developed in line with UNICEF pilot/modeling guidelines and customized to the Armenian context
Date Program partner/Save the Children
Program implémentation documents, report
Once
3.1.4 Foster care policy package is drafted Foster care policy drafted Foster care policy is drafted Date Program
partner/Save the Children
Admin data Once
3.1.5 System for monitoring and supervision of foster care placements is introduced
Foster care placement monitoring and supervision document adopted
Foster care placement monitoring and supervision document/plan is developed and introduced
Date Program partner/Save the Children
Monitoring and supervision plan, Program implementation documents
Once
3.1.6 System for monitoring and supervision of foster care placements is operational
Foster care placement monitoring and supervision established
Foster care monitoring and supervision is established and operational
Date Program partner/Save the Children
Monitoring and supervision plan, Program implementation documents
Output 3.2: Foster Care services are available for children in diverse, difficult life circumstances
34Monitoring & Evaluation Plan, Revision 1
3.2.1 Database for foster families developed
Database of foster families developed
Database of foster families developed
Date Program partner/Save the Children
Report extracted from database, Program implementation document
Once
Number of foster families registered in database
Number of foster families who are registered in database
Number Cumulative
Program partner/Save the Children
Report extracted from database, program implementation document
Annual
3.2.2 Database for foster families linked to staterun database and maintained
Database of foster families linked to Nork database
Linkage of foster families’ database to staterun Nork database
Date Program partner/Save the Children
Once
3.2.3 Awareness raising activities to promote foster care are organized
Number of TV programs broadcasted on local TV stations Number of userfriendly brochures for wider public developed on amended family code and diversified family care
Number of TV programs broadcasted on local TV stations and number of userfriendly brochures for wider public developed on amended family code and diversified family care
Number Cumulative
Program partner/Save the Children
Copy of brochures and broadcasted programs (copied in a udrive)
Annual
3.2.4 Families for different types of foster care are identified and trained
Number of potential foster families trained
Number of families trained to be foster
Number Cumulative
Program partner/Save the Children
Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents
Quarterly
3.2.5 Children from residential care facilities in targeted communities removed and placed in foster families in cooperation with the
Number of children placed in foster care Number of children
(disaggregated by gender and age) placed in foster care
Number Cumulative
Program partner/Save the Children
Administrative data, Program implementation documents
Twice a year
35Monitoring & Evaluation Plan, Revision 1
Government of Armenia
Percent of children remaining in foster care after 6 months of being placed
Number of children (disaggregated by gender and age) remaining in foster care after 6 months of placement
Percentage Program partner/Save the Children
Administrative data, Program implementation documents
Program Goal Indicator Definition of Indicator Units Source/ Responsible Entity
Methodology Frequency of Data Collection
OUTCOME #4: Inclusive Education System is strengthened to provide quality learning for ALL children
4.1 Increased access to communitybased inclusive schools in selected regions
Number of children from 6 to18 transferred from special educational institutions to community based inclusive schools
Number of children (girls and boys) from 6 to18 transferred from special educational institutions to community based inclusive schools in target regions
Number Cumulative
Bridge of Hope Administrative data, Prepost training assessment, Program implementation documents, Final evaluation
Annual
Percent of transferred children successfully completing at least one academic year
Number of transferred children who completed at least one school year
Percentage Bridge of Hope Administrative data of line ministries, Program implementation documents, Final evaluation
Annual
Number of out of school and home schooled children in target regions decreased
Number of out of school and home schooled children in target regions decreased
Number Bridge of Hope Administrative data of line ministries, Program implementation documents, Final evaluation
Annual
4.2 Enhanced capacity of the staff of Pedagogical support centers in conducting ICF based assessments
Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive
Percent increase in the knowledge of the staff of Pedagogical Support Centers trained in supporting inclusive education and
Percentage Bridge of Hope Pre/post test Annual
36Monitoring & Evaluation Plan, Revision 1
education and conducting ICF based assessments
conducting ICF based assessments
Output 4.1: Pedagogical support centers are established to support inclusive schools in their geographical area
4.1.1 Enhanced capacity of communitybased inclusive schools in selected regions
Percent of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers
Percent of inclusive schools in targeted regions receiving methodological and practical support from Pedagogical Support Centers over the total number of inclusive schools at that point of time
Percentage Bridge of Hope Administrative data, Program implementation documents
Annual
4.1.2 The capacity of newly established pedagogical support centers are enhanced to support inclusive schools in their geographical area
Number of staff of Pedagogical Support Centers trained in supporting inclusive education and conducting ICF based assessments
Number of staff of Pedagogical Support Centers trained in supporting inclusive education and conducting ICF based assessments
Number Cumulative
Bridge of Hope Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents
Quarterly
Number of trained Pedagogical Support Centers staff conducting ICF based assessments.
Number of Pedagogical Support Centers’ staff who are trained and conduct ICF based assessments
Number Cumulative
Bridge of Hope Program implementation documents
Annual
Output 4.2: Teachers and directors for mainstream schools (in the programme areas) understand and accept the principles of inclusion and have the capacity to provide education to ALL children4.2.1 Mainstream schools in targeted regions have the capacity to provide education to children with disabilities
Number of mainstream schools in targeted regions trained on inclusive education practices
Number of mainstream schools in targeted regions trained on inclusive education practices of the total number of mainstream schools in targeted regions at that point of time
Number Cumulative
Bridge of Hope Prepost training evaluation, Service contracts with trainers, training package, Program implementation documents
Quarterly
Program Goal Indicator Definition of Indicator Units Source/ Methodology Frequency of
37Monitoring & Evaluation Plan, Revision 1
Responsible Entity
Data Collection
OUTCOME #5: An enabling legal and regulatory framework is established to support Childcare Reform
5.1. Local legislation is adjusted to respond to childcare reform directions
Number of legal recommendations submitted to the Government of Armenia
Number of legal recommendations submitted to the Government of Armenia
Number Cumulative
UNICEF in Armenia Final report with legal recommendations attached
Annual
5.2 Legal and policy framework is set up to allow outsourcing of social services to nonstate providers
Number of Social Assistance legal/policy amendments adopted
Number of Social Assistance legal/policy amendments adopted
Number Cumulative
UNICEF in Armenia Report on adopted social assistance legal/policy amendments
Annual
5.3. Master Plan for transformation of residential care institutions is developed
Rapid assessment of residential care institutions conducted
Rapid assessment of residential care institutions conducted
Date UNICEF in Armenia Assessment report Once
Output 5.1: Recommendations for improving Legal and regulatory framework are approved by the Government, and by the National Assembly, as appropriate
5.1.1 Family code and bylaws are adjusted to provide framework for transparent adoption process
Number of legal recommendations on adoption approved by the Government of RA
Number of legal recommendations on adoption approved by the Government of RA
Number UNICEF in Armenia Program implementation documents/reports
Annual
5.1.2 Law on Social Assistance and by laws are amended to allow outsourcing of state social services to nonstate providers
Number of communitybased NGOs outsourced by state for providing service to vulnerable children and their families
Number of communitybased NGOs outsourced by state for providing service to vulnerable children and their families
Number UNICEF in Armenia Administrative data, program implementation documents
Annual
5.1.3 Policy document on deinstitutionalization, transformation of residential
Types of alternative services defined
Policy document on types of alternative services defined
Date UNICEF in Armenia Policy document, program implementation
Once
38Monitoring & Evaluation Plan, Revision 1
institutions and types of alternative services developed
reports
OUTCOME #6: Social norms regarding children with disabilities have changed favorably toward inclusion
6.1 People in Armenia have favorable attitude toward children with physical or intellectual disabilities
Percent increase in the positive attitude towards children with physical disabilities
Percent increase in the positive attitude towards children with physical disabilities (in comparison with the attitudes in 2013)
Percentage UNICEF in Armenia Survey data Once
Percent increase in the positive attitude towards children with intellectual disabilities
Percent increase in the positive attitude towards children with mental disabilities (in comparison with the attitudes in 2013)
Percentage UNICEF in Armenia Survey data Once
USAID Standard indicators
Indicator Definition Linkage to LongTerm Outcome or Impact
Indicator Type
Unit of Measure
Use of Indicator
Data Source and Reporting Frequency
Known Data Limitations
Baseline Timeframe
Disaggregate(s)
#1 Number of social protection policy reforms drafted, adopted or implemented with USG support
Number of social protection policy reform policies drafted, adopted, and/or implemented as a result of
This output provides the necessary foundation work for outcome level changes in social services
Output Number of policy reforms drafted, adopted or implemented
Can be used to monitor progress
Review of Project reports/Program review results Documents/Unit count of policiesAnnual reporting
A “reform” is not a defined unit of measure that permits one to meaningfully conclude that implementing twelve “reforms” is four times better than
Baseline timeframe needs to be established by the Operating Unit
None
39Monitoring & Evaluation Plan, Revision 1
(3.3.113) USG technical support/input. (Higher = Better) Provides information about progress in the development of a country’s social protection policies (policies drafted). It also provides proxy measures for social protection regulations (policies adopted) and for systems (policies implemented).
policies. implementing three “reforms”—or even better at all. Moreover, there is no common definition of how to define the parameter of “a reform”. Missions should include a narrative when reporting on this indicator explaining the relevance and impact of any reform.
#2 Number of vulnerable people benefitting
Number of vulnerable people receiving services from
Provides an essential link to higher order outcome
Output Number of people
Monitor Progress
Review of Project reports/Program review results Documents/Reviews of service records
Program self reporting is always subject to variability of inter rater
Baseline timeframe needs to be established by the
• Sex• Vulnerable children • Other targeted vulnerable
40Monitoring & Evaluation Plan, Revision 1
from USGsupported social services (3.3.28)
programs funded in whole or in part by the USG. (Higher = Better)Simple output measure to enable the roll up of vulnerable people assisted through USG resources.
measure Annual reporting reliability Operating Unit
people• Victims of torture• War victims• Persons with disabilities• Males with disabilities• Females with disabilities
#3 Number of service providers trained who serve vulnerable persons (3.3.213)
Number of service providers trained who serve vulnerable persons. (Higher = Better) Tracks improvement in the capacity to provide social assistance and social service
This measure provides an initial basis for getting to higher order outcomes.
Output Number of service providers
Tracks capacity improvement
Review of Project reports/Program review results Documents/Unit count from training recordsAnnual reporting
Program self reporting is subject to error
Baseline timeframe needs to be established by the Operating Unit
Sex
41Monitoring & Evaluation Plan, Revision 1
programs. Also serves as an indicator of a government’s commitment and capacity to protect its vulnerable populations.
42Monitoring & Evaluation Plan, Revision 1
#4 Number of USGassisted organizations and/or service delivery systems strengthened who serve vulnerable populations (3.3.215)
Number of organizations or service delivery systems strengthened (providing better or more efficient services) with USG resources. (Higher = Better) Tracks improvement in the capacity to provide social assistance and social service programs. Also serves as an indicator of a government’s commitment and capacity to protect its vulnerable populations.
Improvement in systems delivery is essential to long term outcome achievement
Output Number of organizations and/or delivery systems
Tracks progress towards outcomes
Review of Project reports/Program review results Documents/Unit count from project recordsAnnual reporting
Program self reporting is subject to error
Baseline timeframe needs to be established by the Operating Unit
• Local NGO (nonDPO); • DPO;• Schools;• Other.
43Monitoring & Evaluation Plan, Revision 1
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
ANNEX 7 OUTPUT TABLES GENERATED FROM QUANTITATIVE SURVEY
General Awareness on Inclusive Education
Table 1. Knowledge of the definition of inclusive education according to type of community (village,town) Community type Total
Urban RuralWrong answer 7 12 19Correct answer 137 156 293Total 144 168 312
Awareness coefficient 0.90 0.86 0.87
Table 2. Knowledge of the definition of inclusive education according to regions (Kapan, Goris,Sisian) Region Total
Kapan Goris SisianWrong answer 7 5 7 19Correct answer 141 63 89 293Total 148 68 96 312Awareness coefficient 0.91 0.85 0.85 0.87
Table 3. Knowledge of teachers on which children are considered as having special education needs
Correctanswer
Wronganswer
Total Index
1. Children of socially insecure families 86.90% 13.10% 100%
Kapan 139 9 148 0.88Goris 50 18 68 0.47Sisian 82 14 96 0.712. Children whose parents work and cannotengage with the child(ren)’s education
88.50% 11.50% 100%
Kapan 134 14 148 0.81Goris 57 11 68 0.68
Sisian 85 11 96 0.773. Children with physical disabilities 90.10% 9.90% 100%
Kapan 135 13 148 0.82Goris 57 11 68 0.68Sisian 89 7 96 0.854. Children with visual impairments 76.90% 23.10% 100%
Kapan 105 43 148 0.42Goris 57 11 68 0.68Sisian 78 18 96 0.63
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
5. Children with speech impairments 89.40% 10.60% 100%
Kapan 126 22 148 0.70Goris 65 3 68 0.91Sisian 88 8 96 0.836. Children with hearing impairments 92.00% 8.00% 100%
Kapan 133 15 148 0.80Goris 63 5 68 0.85Sisian 91 5 96 0.907. Children with psychological disorders 92.00% 8.00% 100%
Kapan 142 6 148 0.92Goris 64 4 68 0.88Sisian 81 15 96 0.698. Children without parents (orphans) 86.50% 13.50% 100%
Kapan 135 13 148 0.82Goris 56 12 68 0.65
Sisian 79 17 96 0.659. Children from large families 91.70% 8.30% 100%
Kapan 143 5 148 0.93Goris 60 8 68 0.76Sisian 83 13 96 0.7310. Children with mental disorders 95.50% 4.50% 100%
Kapan 146 2 148 0.97Goris 64 4 68 0.88
Sisian 88 8 96 0.8311. Deaf and hard of hearing children 93.30% 6.70% 100%
Kapan 143 5 148 0.93Goris 62 6 68 0.82
Sisian 86 10 96 0.7912. Blind children 91.30% 8.70% 100%
Kapan 142 6 148 0.92Goris 63 5 68 0.85Sisian 80 16 96 0.67Average index 0.78Kapan 0.83Goris 0.76Sisian 0.75
Table 4: Correct or incorrect answers to presented statement
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Statements Correctanswer
Wronganswer
Difficult to say
Total
The medical model of disability is at the basis of inclusiveeducation
19.60% 75.00%
5.40% 100%
Every teacher should have the skills to identify theobservation and learning abilities of the student
98.10% 1.60% 0.30% 100%
Subject teachers do not participate in the developmentand implementation of the individual education plan.Only the parents, the class teacher, the special pedagogueand the psychologist are involved in the process
69.20% 29.80%
1.00% 100%
The individual education plan is developed for one schoolyear or based on the short-term issue.
91.70% 5.10% 3.20% 100%
Table 5: Calculation of awareness indexes according to regions
Index of knowledge
Average Kapan Goris Sisian
The medical model of disability is at the basis of inclusive education
-0.56 -0.69 -0.59 -0.4
Every teacher should have the skills to identify theobservation and learning abilities of the student
0.96 1 1 0.88
Subject teachers do not participate in thedevelopment and implementation of the individualeducation plan. Only the parents, the class teacher,the special pedagogue and the psychologist areinvolved in the process
0.39 0.41 0.47 0.29
The individual education plan is developed for oneschool year or based on the short-term issue.
0.83 0.85 0.88 0.77
Table 6: Index of general knowledge according to regions
Kapan Goris Sisian0.66 0.67 0.63
Attitude Table 7: Is Armenia ready for the introduction of the inclusive education system?
Answer %
Yes, absolutely 19 6.1%To some extent yes 202 64.7%To some extent no 51 16.3%Not ready at all 35 11.2%Difficult to say 5 1.6%Total 312 100.0%Table 8: Average score given by teachers (on a scale of 1 to 5) about the level of readiness toimplement inclusive education by the following entities/groups:
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Community members, parents (attitude) 3.0Their Schools (building and amenities) 2.8Their colleges (attitude, knowledge, skills) 4.1
Table 9: How do you personally accept the idea of introducing the inclusive education system?
Answer %
I absolutely accept it 102 32.7I somewhat accept it 151 48.4I somewhat do not accept it 32 10.3I do not accept it at all 18 5.8Difficult to say 9 2.9Total 312 100.0
Table 10: Why do you think so? Answer %
Positive arguments
Respect for the rights of all children 169 54.2
The inclusive system contributes to the integration of the society, has aneducational function
17 5.4
Children feel good; psychological aspect 12 3.8Negative arguments
Negative impact on classroom; the learning process (the lesson) is disrupted 34 10.9
No corresponding conditions (physical, adequate attention during thelesson, mentality of other children’s parents)
35 11.2
Those children require individual approach 18 5.8In some cases special schools convey more useful knowledge thanmainstream schools
7 2.2
They stand out anyway and their issue is emphasized 5 1.6Inefficient spending of resources (finances, time) 5 1.6
Difficult to say 10 3.2Total 312 100.0
Table 11. Teachers’ perception about which children should be considered as having special educationneeds
1. Yes 2. No Total
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
9. Children from large families 16.0% 84.0%
100%
2. Children whose parents work and cannot engage with theireducation
17.3% 82.7%
100%
1. Children of socially insecure families 21.5% 78.5%
100%
8. Children without parents (orphans) 23.4% 76.6%
100%
4. Children with visual impairments 79.8% 20.2%
100%
7. Children with psychological disorders 81.1% 18.9%
100%
12. Blind children 81.1% 18.9%
100%
11. Deaf and hard of hearing children 84.0% 16.0%
100%
10. Children with mental disorders 84.3% 15.7%
100%
6. Children with hearing impairments 88.5% 11.5% 100%
5. Children with speech impairments 89.7% 10.3%
100%
3. Children with physical disabilities 90.4% 9.6% 100%
Table 12: Impacts of the introduction of the inclusive education system on the following groups/structures:
Ab
solu
tely
n
egat
ive
Som
ewh
at n
egat
ive
Nei
ther
pos
itiv
e n
or n
egat
ive
Som
ewh
at p
osit
ive
Ab
solu
tely
pos
itiv
e
Dif
ficu
lt t
o sa
y
Tot
al
1. Children with physicalimpairments
1.6% 5.8% 6.7% 43.6% 41% 1.3% 100%
2. Children with mild mentalimpairments
3.5% 7.1% 11.5% 42.6% 34.3% 1% 100%
3. Children with severe mentaldisorders
17.3% 10.9% 22.1% 22.8% 23.4% 3.5% 100%
4. Children of socially insecurefamilies
4.8% 3.6% 37.8% 26.9% 24% 2.9% 100%
5. Children excluded from theinclusive education system
14.1% 20.9% 22.4% 21.5% 19.6% 1.5% 100%
6. Parents of children excludedfrom the inclusive educationsystem
14.7% 22.4% 24% 24% 11.5% 3.4% 100%
7. Parents of children in theinclusive education system
2.6% 2.6% 7.9% 31.1% 54.8% 1% 100%
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
8. Teachers 3.5% 11.6% 19.2% 29.5% 34.9% 1.3% 100%
9. School administration 0.3% 4.8% 15.4% 26.3% 48.7% 4.5% 100%
10. Community 0.6% 2.9% 20.5% 32.1% 39.7% 4.2% 100%
11. Society/country 1.3% 3.2% 19.9% 33% 37.5% 5.1% 100%
12. Stakeholders interested in theintroduction of the system/Ministry of Education, NationalInstitute of Education, other …/
0.3% 1.6% 8.3% 18.6% 63.8% 7.4% 100%
Table 13: What do your colleagues think of the children in inclusiveeducation? (up to 3 options for answers were available)
Answer %
It is very good that they study with all other children 117 19.6%Working with them is very difficult 98 16.4%It would be better if relevant specialists worked with them 95 15.9%
They disrupt the lesson 80 13.4%They make significant progress 67 11.2%They are in no way different from other children 55 9.2%It would be better if they attended corresponding educational institutions 27 4.5%
Working with them is a pleasure 18 3.0%They pay more attention in the classroom 11 1.8%It’s impossible to supervise them 11 1.8%The human value of the child increases 5 .8%We all have to help them 4 .7%Special plans are needed 3 .5%Difficult to say 3 .5%There are no relevant conditions 2 .3%The student exploits the fact of having an impairment 1 .2%Total 597 100.0%
Practice
Table 14: What does a teacher need to work more effectively withinthe inclusive education system?
Answer %
Didactic materials 87 27.9Corresponding knowledge (professional education), skills andtrainings
61 19.6
Personal qualities: patience, strong will power 33 10.6Additional payment, salary 32 10.3Permanent availability of a corresponding specialist 28 9.0
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Time 14 4.5Methodological guideline 13 4.2Experience 11 3.5Material and technical base, facilities, additional classroom 10 3.2
Verbal rewards 4 1.3Fewer number of students in class 3 1.0Promotion 2 .6Difficult to say 14 4.5Total 312 100.0
56.1% of respondents have noted they had a child from the inclusive education system in their class, while 43.9% saidthey did not. The rest of the questions were answered by those who taught children with special education needs
Table 16: Which of the following do youagree with?
Completelyagree
Somewhatagree
Somewhatdisagree
Do notagree atall
Difficult tosay
Total
I spend a bigger part of the lessonworking with the child(ren) with specialeducation needs, others
10.9%
14.8% 29.1% 44.6% 0.6% 100%
I work with children with specialeducation needs also outside of the mainschool hours
14.3%
20.6% 10.3% 53.1% 1.7% 100%
I spend more time preparing for the workwith the children with special educationneeds, than for work with other children
33.1%
32.6% 13.1% 20.6% 0.6% 100%
Table 17: In the classroom the seat of the child enrolled in the inclusive education system isdecided according to:
%
The child’s needs 76.0The child’s free will 17.7The needs of the class 5.1Difficult to say 1.1Total 100.0
Table 18: Attitude of peers towards children in the inclusive education system %
Very good 22.9Good 58.3Poor 9.7Very poor 8.6Difficult to say .6
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Total 100.0
Table 19: Frequency of issues with children enrolled in the inclusive education system %
Very often 4.6Often 17.1Sometimes 48.6Rarely 14.3Never 15.4Total 100.0
Table 20: Who do children enrolled in the inclusive education system mostly interact with inclass/school?
%
All children in class 56.5%Only among themselves 2.2%With neighbor’s children attending the same school 6.9%With relatives attending the same school 6.5%With children who treat them nicely 26.7%With no one 1.3% Total 100.0%
Table 21: Do you collaborate with your colleagues when working with children in the inclusive education system?
%
Yes, always 74.9
Sometimes yes 23.4
Mostly no 1.7
Total 100.0
Table 22: How frequently do you collaborate in relation to issues of children in the inclusive educationsystem?
Veryfrequently
Frequently
Sometimes
Rarely Never Total
With the Teacher’s Assistant or members ofthe multidisciplinary group
29.7% 33.1% 25.0% 3.5% 8.7% 100%
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
With staff of pedagogical and psychologicalsupport center
7.6% 22.7% 36.0% 13.4% 20.3% 100%
With representatives of the NationalInstitute of Education
4.1% 10.5% 25.6% 16.9% 43.0% 100%
With parents of children enrolled in theinclusive education system
21.5% 41.3% 27.3% 6.4% 3.5% 100%
With the parents of other children 10.5% 22.7% 28.5% 15.7% 22.7% 100%
Table 23: Changes due to introduction of inclusive educationYes Somewha
t yesSomewhat no
No Difficult to say
Total
The attitude of children towards theirpeers included in the inclusive educationsystem has changed
64.0%
30.8% 0.6% 4.0%
0.6% 100%
The attitude of children enrolled in theinclusive education system towards otherchildren has changed
50.9%
37.7% 1.1% 6.9%
3.4% 100%
The behavior of children in inclusiveeducation has changed
42.3%
49.7% 2.9% 5.1%
0.0% 100%
The performance of children in inclusiveeducation has changed
35.4%
50.3% 7.4% 6.3%
0.6% 100%
Relevance and Efficiency of Trainings
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Table 24: Have you participated in trainings on inclusive education during the last 2 years?
%
Yes 76.9No 23.1Total 100.0
Respondents who said they participated in trainings have answered the rest of the questions (total number n=240):
Table 25: Date of the training (year)
Total
Trainings of teachers Training of teachingassistants
Difficult to say
Difficult to say 4 0 1 5
2015 139 9 2 1342016 93 15 0 842017 50 26 1 47Total 233 33 4 2701
Respondents who said they took part in trainings organized by BoH (240 persons) noted whether they received certificates
Table 26: Received a certificate %
Yes 80.4
No 19.6
Total 100.0
Table 27: Were the trainings useful for you? %
Very useful 70.0Somewhat useful 27.1Not so useful 1.7Not useful at all 1.3Total 100.0
Table 28: Which part(s) of the training did you attend? %
All of it 96.7
More than half 2.9
Half .4
1 Respondents could give more than 1 answer, that is why the total quantity is 270
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Total 100.0
Table 29: Which part of the teachers still need trainings on inclusive education? %
All of them 12.5
More than half 7.5Half 5.0Fewer than half 14.6Some of them 20.8No one 29.2Difficult to say 10.4Total 100.0
Table 30: Do you utilize the knowledge gained from the trainings? %
Yes, always 40.0Mostly yes 44.6
Mostly no 5.8
No 9.6
Total 100.0
Table 31: Average score (on a scale of 1 to 5, where 5 is the highest and 1 is the lowest mark) to thefollowing:Efficiency of the training 4.2Usefulness of the training for teacher 4.2Level of organization of training (venue, time period, duration of training, friendliness, etc) 4.3Quality of teaching (whether things were explained in a proper, simple and comprehensiblemanner, use of interactive techniques, etc.)
4.6
Training contents (whether all topics were discussed, if training material containednovelties, etc.)
4.5
Availability of Supporting Resources
Table 32: Do you conduct students’ needs assessments? Answer %
Yes 189 60.6No 123 39.4Total 312 100.0
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Table 33: If yes, then who do you collaborate with for that purpose (opportunity for severaloptions)?
%
With corresponding specialists of school 76.7%
With local NIE staff 5.3%With staff of local pedagogical and psychological support center 29.1%
Total 111.1%
Table 34: Do representatives of Pedagogical and Psychological Support Center work withthe school?
%
Yes 72.1No 21.8Difficult to say 6.1Total 100.0
Table 35: What issues do you collaborate on? %
Conducting a needs assessment for the child 23.4%Working with children enrolled in the inclusive education system 23.1%
Providing consultancy to the school on issues related to children enrolled in the inclusiveeducation system
22.1%
Developing individual education plans 15.1%Developing support plan for the child 14.0%Difficult to say 2.3%They don’t do anything .2%Total 100.0%
Table 36: Does the assessment system that has been introduced allow for the objectiveidentification and assessment of children with issues?
%
Yes, absolutely 23.6Mostly yes 56.4Mostly no 12.9Not at all 3.6Difficult to say 3.6Total 100.0
Table 37: Recommendations on improving cooperation between theschool and the Center
Answer %
Mid-term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, FamilySupport and Inclusive Education Services as part of Child Care Reform” project
UNITED NATIONS CHILDREN'S FUND in Armenia
Frequent and regular meetings, consultancy 86 27.6
Organization of joint seminars 44 14.1Allocation of work, mutual arrangements 12 3.8Availability of relevant conditions at school /specialists, classroom,didactic materials/
9 2.9
Working with parents and other students 9 2.9Engagement of all teachers and not just teaching assistants 7 2.2
Further cooperation on working with children 7 2.2
Longer and more detailed assessment of children 7 2.2
There is already a high level of cooperation 6 1.9A bigger number of professional specialists at the center 5 1.6Implementing supervision 1 .3Difficult to say 119 38.1Total 312 100.0
Table 38: Gender of respondent %
Male 7.7Female 92.3Total 100
Table 39: What subject do you teach? %
I don’t teach 8.3
Elementary school teacher 21.5
Languages 30.4
Humanities/social science, religion, history, geography, military science 12.8
Physics and mathematics/ computer science, physics, mathematics, geometry 16.7
Science/chemistry, biology, environment 5.4
Physical education, labor, technology, music, chess 4.8
Total 100