Independent Implementation Evaluation for the ... · IMPAQ International, LLC Page ii ILAB Colombia Final Implementation Evaluation . ACKNOWLEDGMENTS . This report describes in detail

Bureau of International Labor Affairs (ILAB) Department of Labor (DOL)

Independent Implementation Evaluation for the

Strengthening Protections of Internationally Recognized Labor Rights in Colombia Project

Final Implementation Evaluation Report

FINAL

Contract Number: DOLQ129633247

August 4, 2016

Authors:

Sarah Pedersen Melissa Paredes William Prieto Daniela Zapata

Submitted to: Theresa de Silva

Bureau of International Labor Affairs (ILAB) U.S. Department of Labor 200 Constitution Ave. NW

Washington, DC 20210

Submitted by: IMPAQ International, LLC

10420 Little Patuxent Parkway Suite 300

Columbia, MD 21044 www.impaqint.com

http://www.impaqint.com/

IMPAQ International, LLC Page ii ILAB Colombia Final Implementation Evaluation

ACKNOWLEDGMENTS This report describes in detail the final implementation evaluation of the Strengthening Protections of Internationally Recognized Labor Rights in Colombia project. IMPAQ International, LLC, conducted an independent evaluation in collaboration with the project team and stakeholders and prepared the evaluation report according to the terms specified in its contract with the U.S. Department of Labor Bureau of International Labor Affairs. IMPAQ would like to express sincere thanks to all the parties involved for their support and valuable contributions.

Funding for this evaluation was provided by the United States Department of Labor. This material does not necessarily reflect the views or policies of the United States Department of Labor, nor does the mention of trade names, commercial products, or organizations imply endorsement by the United States Government.

IMPAQ International, LLC Page iii ILAB Colombia Final Implementation Evaluation

TABLE OF CONTENTS

TABLE OF EXHIBITS .................................................................................................................. v

LIST OF ACRONYMS ................................................................................................................ vi

EXECUTIVE SUMMARY ............................................................................................................. 1

1. PROJECT BACKGROUND AND DESCRIPTION ...................................................................... 10

1.1 Background: Labor Rights in Colombia ........................................................................ 10

1.2 Escuela Nacional Sindical and the Workers’ Rights Centers (CALs) .............................. 12

1.2.1 CAL Services ................................................................................................................. 14

1.2.2 Program Logic .............................................................................................................. 15

1.2.3 Project Assumptions .................................................................................................... 18

1.2.4 Critical Factors for the CALs’ Success .......................................................................... 19

1.3 Operational Context .................................................................................................... 20

2. EVALUATION PURPOSE AND METHODOLOGY ................................................................... 25

2.1 Evaluation Purpose ..................................................................................................... 25

2.2 Methodology .............................................................................................................. 25

2.2.1 Evaluation Schedule .................................................................................................... 26

2.2.2 Data Collection ............................................................................................................ 27

2.2.3 Data Analysis ................................................................................................................ 29

2.2.4 Limitations ................................................................................................................... 30

3. FINDINGS .......................................................................................................................... 31

3.1 Validity of the Project Design ...................................................................................... 32

3.1.1 Logic of the Project Design .......................................................................................... 32

3.1.2 Project Assumptions .................................................................................................... 35

3.1.3 Lessons Learned and Promising Practices ................................................................... 39

3.2 Relevance and Strategic Fit ......................................................................................... 40

3.2.1 Relevance within Country Context .............................................................................. 40

3.2.2 Targeting Vulnerable Economic Sectors and Groups .................................................. 40


3.3 Project Progress and Effectiveness .............................................................................. 44

3.3.1 Status in Achieving Target Goals ................................................................................. 44

3.3.2 Follow-up of CAL Workers ........................................................................................... 56

IMPAQ International, LLC Page iv ILAB Colombia Final Implementation Evaluation


3.4 Efficiency in the Use of Project Resources ................................................................... 59

3.4.1 Adequacy of Time, Budget, and Human Resources .................................................... 59


3.5 Effectiveness of Management Arrangements .............................................................. 63

3.5.1 Support Structure and Project Communication .......................................................... 63

3.5.2 Centralized vs. Decentralized Model ........................................................................... 64

3.5.3 Performance Monitoring System ................................................................................ 64


3.6 Impact Orientation and Sustainability ......................................................................... 66

3.6.1 Transfer of Project Management to the CUT and CTC Unions.................................... 66

3.6.2 Strategies to Engage Unions and Other Stakeholders ................................................ 67

3.6.3 Fee for Services and Other Pricing Options ................................................................. 67


4. CONCLUSIONS AND RECOMMENDATIONS ........................................................................ 69

4.1 Conclusions by Area of Analysis .................................................................................. 70

4.2 Key Recommendations for ENS ................................................................................... 73

4.3 Final Considerations for ILAB ....................................................................................... 74

APPENDIX A: Schedule of Evaluation Activities ..................................................................... A-1

APPENDIX B: Interview and Focus Group Guides .................................................................. B-1

APPENDIX C: Summary of Activity Progress by CAL ............................................................... C-1

IMPAQ International, LLC Page v ILAB Colombia Final Implementation Evaluation

TABLE OF EXHIBITS Exhibit 1.1: Map of CAL Offices ............................................................................................................ 13

Exhibit 1.2: Types of Legal Assistance Offered by CAL Offices ............................................................. 14

Exhibit 1.3: Mobile CAL Coverage ......................................................................................................... 15

Exhibit 1.4: Program Logic .................................................................................................................... 16

Exhibit 1.5: Operational Context for Receiving and Providing Legal Support in Colombia .................. 22

Exhibit 2.1: Methodology by Area of Analysis ...................................................................................... 26

Exhibit 2.2: Semi-Structured Interview Participants, March 2016 ....................................................... 28

Exhibit 2.3: Number of Focus Groups, March 2016 ............................................................................. 28

Exhibit 2.4: Documents Reviewed by Category .................................................................................... 29

Exhibit 3.1: Profile of CALs .................................................................................................................... 31

Exhibit 3.2: Logic of the Project Design ................................................................................................ 33

Exhibit 3.3: Status of Project Assumptions ........................................................................................... 36

Exhibit 3.4: Status of Internal Critical Factors ...................................................................................... 37

Exhibit 3.5: Main Reporting Issues in Technical Progress Reports ....................................................... 38

Exhibit 3.6: Status of External Factors .................................................................................................. 39

Exhibit 3.7: Distribution of CAL Actions by Socioeconomic Strata, January–April 2016 ...................... 42

Exhibit 3.8: Distribution of CAL Cases by Economic Sector, January–April 2016 ................................. 43

Exhibit 3.9: Workshops with the Worker’s Rights Handbook, April - June 2016 ................................. 45

Exhibit 3.10: Sources of Referrals by CAL, 2015 ................................................................................... 50

Exhibit 3.11: Number of CAL Users, 2014-2016 ................................................................................... 51

Exhibit 3.12: Number of CAL Users by Quarter, 2014-2016 ................................................................. 52

Exhibit 3.13: Number of Legal Actions Taken across CALs, 2014–2015 ............................................... 53

Exhibit 3.14: Mobile CALs by Quarter, 2015 ........................................................................................ 55

Exhibit 3.15: Verified Actions by CAL, 2015 .......................................................................................... 57

Exhibit 3.16: Results of Follow-up by CAL, 2015* ................................................................................ 57

Exhibit 3.17: Allocation of Resources by Category ............................................................................... 60

Exhibit 3.18: Allocation of Resources for Personnel by Year ............................................................... 61

Exhibit 3.19: Total Budget and Percentage of Total Budget Spent ...................................................... 62

IMPAQ International, LLC Page vi ILAB Colombia Final Implementation Evaluation

LIST OF ACRONYMS CAL: Centro de Atención Laboral [Workers’ Rights Center]

CEJ: Corporación Excelencia en la Justicia [Corporation for Excellence in Justice]

COLabora: Centro de Orientación y Atención Laboral [Center for Orientation and Assistance on Labor Issues]

CGT: Confederación General de Trabajo [General Labor Confederation]

CTC: Confederación de Trabajadores de Colombia [Confederation of Workers in Colombia]

CUT: Central Unitaria de Trabajadores [Central Union of Workers]

ENS: Escuela Nacional Sindical [National Union School]

ILAB: Bureau of International Labor Affairs

ILO: International Labor Organization

MPG: Management Procedures Guidelines

MSP: Ministry of Social Protection

PMP: Performance Monitoring Plan

TPA: United States–Colombia Trade Promotion Agreement

TPR: Technical Progress Report

USDOL: United States Department of Labor

IMPAQ International, LLC Page 1 ILAB Colombia Final Implementation Evaluation

EXECUTIVE SUMMARY In December 2012, the United States Department of Labor (USDOL) Bureau of International Labor Affairs (ILAB) awarded a $1.5 million grant to Colombia’s Escuela Nacional Sindical (ENS) to implement the Strengthening Protections of Internationally Recognized Labor Rights in Colombia (Workers’ Rights Centers1) project over 3 years. In 2015, ENS received a 1-year extension with an additional $600,000 in funding for a total of $2.1 million. The project focused on increasing the awareness of workers’ rights and sustaining the Colombian government’s efforts to strengthen labor rights. As part of the United States–Colombia Trade Promotion Agreement (TPA), which went into effect on May 15, 2012, the Colombian government agreed to develop and implement an Action Plan Related to Labor Rights (Action Plan) as a precondition for the TPA to take effect. Under the Action Plan, the Colombian government committed to take specific steps to bring Colombian labor laws and practices into greater conformity with internationally recognized labor rights.2 ENS believes that the best way to realize the full potential of reforms initiated by the Colombian government is to engage workers and assist them in presenting well-supported workplace violations claims to the proper administrative or legal authorities. To achieve this goal, ENS established three Centros de Atención Laboral (CALs) [Workers’ Rights Centers] in Bogotá, Cartagena, and Bucaramanga and expanded an existing CAL in Medellín. The CALs provide free legal assistance services to facilitate the effective protection of workers’ rights and help achieve the project’s two main outcomes:

Outcome 1: Workers, with the assistance of the CALs, will improve their knowledge of and ability to defend their labor rights.

Outcome 2: Workers will make more frequent and effective use of the administrative and legal mechanisms to protect their labor rights.

In September 2013, ILAB contracted with IMPAQ International, LLC (IMPAQ) to conduct an independent evaluation of the project under the contract titled ILAB Impact and Implementation Evaluation for Colombia. The current report presents the final implementation evaluation of the ILAB Colombia Workers’ Rights Centers project. The report provides an assessment of ENS’s progress toward achieving its objectives, identifies lessons learned from the program strategy, provides feedback to ENS with regard to its achievements, and makes recommendations to ENS and ILAB for future consideration of similar projects. This implementation evaluation focused on six areas that ILAB identified in the project’s Terms of Reference:

1. Validity of the project’s design

1 http://www.dol.gov/ilab/projects/summaries/Colombia_Worker_Rights_Centers.pdf. 2 https://ustr.gov/trade-agreements/free-trade-agreements/colombia-fta.

http://www.dol.gov/ilab/projects/summaries/Colombia_Worker_Rights_Centers.pdf

https://ustr.gov/trade-agreements/free-trade-agreements/colombia-fta/final-text


2. Relevance and strategic fit

3. Project’s progress and effectiveness

4. Efficiency of resource use

5. Effectiveness of management arrangements

6. Impact orientation3 and sustainability To assess these six areas, the team conducted observations, semi-structured interviews, and focus groups during two site visits, one in March 2015 and another in March 2016. In addition to the gathering information during site visits, the team reviewed project documents as they became available. This report augments the findings presented in the Midterm Implementation Evaluation Report finalized on September 8, 2015. Specifically, it describes and evaluates the project’s progress and the changes that have been made since the midterm report. Below, the evaluation team presents the key findings by area of analysis, recommendations for ENS as it continues to implement the project, and final considerations for ILAB as it considers similar projects in other localities. Overall, the evaluation team believes this is a valuable project that has contributed to the immediate goal of assisting workers in their ability to defend their labor rights (Outcome 1) and assisting workers in making more frequent and effective use of the administrative and legal mechanisms to protect their labor rights (Outcome 2). Implemented by a non-governmental organization that specializes in workers’ labor rights, the project was designed to assist both independent workers and unionized workers by focusing on three main mechanisms: providing basic legal assistance to workers, documenting and monitoring CAL trends and cases, and using these data to pursue “emblematic cases” (i.e., strategic cases that have the potential to establish new legal precedents or impact a large group of workers). Although the project undertakes significant efforts to collect information in these areas, it did not set measureable targets to evaluate the project’s overall success in reaching its goals and outcomes, thus limiting the extent to which the evaluation team can make specific conclusions in this report. Both Outcomes, however, are evaluated more rigorously in IMPAQ’s companion impact evaluation report. Despite this limitation, the evaluation team would like to highlight some noteworthy observations in the project’s efforts to reach workers, particularly those in the priority sectors (i.e., special sectors of interest and need), all of which are located in rural areas. Targeted priority sectors were assigned to each of the CAL cities based on their proximity, so each CAL has both an urban and a rural target population. While the urban CAL offices serve all workers who seek services, each office also coordinates periodic mobile CALs to specifically target the harder-to-reach rural workers in the priority sectors. Despite the mobile CALs’ continued efforts, the project appears to have been more effective at reaching urban populations than rural populations, which

3 Impact orientation refers to the steps taken by ENS to achieve the project’s objectives in the long term and ensure that the project’s activities will continue after the project’s completion.


is an unsurprising observation, given that the CAL offices are centrally located in urban areas and open to the general public for most of the year. The level of workers reached in rural areas, however, varies by city. More definitive research would be needed to understand how the mobile CALs can be made more effective, but two points should be noted:

Rural workers are harder to reach than urban ones not only due to proximity (transportation) but to various factors, such as lack of privacy in approaching a mobile CAL and challenges in developing trust with a new service provider of any kind.

The variations in reaching rural workers via mobile CALs appears to be largely due to: 1) the relationships between a given CAL and key stakeholders, particularly local unions and 2) the level of union organization of workers within a priority sector. Reaching workers in rural areas has proven more effective in cases in which these workers are already organized, as unions can provide much needed logistical support to the mobile CALs.

For these reasons--and considering that union workers in general may be more likely to contribute to the sustainability of the project in the long-term (e.g., logistical support and help with following up on CAL client cases)--during the design phase of the project, more consideration should be given to the mobile CAL component of the strategy and to union (or other stakeholder) involvement so as to maximize the project’s effects on priority rural sectors. Each CAL has unique strengths and challenges. Exhibit 3.1 in Section 3 presents a summary of key characteristics across CALs. Because each CAL has distinct characteristics and circumstances, one CAL’s performance or “success” should be determined not by measuring it against another but according to its own merits. Key Findings by Area of Analysis 1. Validity of the Project Design

The project design was rooted in evidence regarding the legal needs and obstacles facing workers. ENS appropriately identified union federations (CUT and CTC) as key project partners; however, ENS did not sufficiently consider the political will and inner workings of union federations when designing the original sustainability strategy that transferred management and financing of the project to the federations. This has caused partnership building and sustainability efforts with the federations to stall. As a result, ENS and the CALs have turned to other national and local unions to demonstrate the need for the CALs and secure resources. However, ENS and the CALs have not developed indicators to reflect their work with other national and local unions. Established indicators and targets could help ENS and the CALs better understand the full spectrum of work that they do and the effort required for such work. This project was built on the premise that offering legal services to individual workers at the CALs would broaden and expand the knowledge and expertise of ENS and the unions. An increase in knowledge and expertise was expected to help develop a legal strategy that would improve the legal context for workers in Colombia by pursuing “emblematic cases”.


According to the project design, both of these strategies (i.e., CAL legal services and emblematic cases) were necessary to help workers “make more frequent and more effective use of the administrative and legal mechanisms to protect their labor rights” (Outcome 2). In particular, emblematic cases, such as the ones presented to the Ministry of Labor, would serve as a concentrated effort to demonstrate how organized workers can and do file claims at the Ministry, increasing the effectiveness of this mechanism to protect labor rights. However, while ENS has publicized labor rights through media sources when filing these emblematic cases, this strategy, as it stands now, does not seem to be sufficient to achieve this outcome as the cases presented to the Ministry of Labor remain unresolved. Establishing new CALs exposed a tension between the two main mechanisms designed to reach the project’s goals. On-the-ground CAL staff and clients believe that assisting individual workers at the CALs provides quick results and is a unique and important service for which there is much demand. However, ENS believes that CALs are not the main mechanism that will bring the greatest benefit to the majority of workers and that pursuing emblematic cases could bring about more impactful changes over time. Unfortunately, the advancement of emblematic cases has been slow due to what ENS perceives as insufficient effort from the Ministry of Labor to investigate these cases. The slower-than-expected progress of emblematic cases is a source of frustration and has affected the progression of the project’s goal to improve the overall labor rights justice system for workers. Yet, as an ILO representative explained during the midterm evaluation, the emblematic cases that the CALs have presented at the Ministry of Labor have started to give visibility to the broader issue of labor investigations. This visibility is important because all major stakeholders as well as external experts and informants identified the need to assist workers in filing collective claims as a strength of the project. Also, ENS can more easily and effectively assess the progress of emblematic cases, as opposed to individual cases for which monitoring outcomes is resource intensive. Gathering evidence of the slow progress of these investigations could help the project bring attention to the performance of public institutions and therefore make headway on Outcome 2. ENS is already drafting an “access to justice” report for this purpose and intends to use this information to start a dialogue with the newly-appointed Minister of Labor and find a resolution to these cases. 2. Relevance and Strategic Fit

The CAL project is aligned with the objectives of the U.S. and Colombian governments to incorporate high labor standards into their trade agreement. The project directly addresses the commitment in the Action Plan to protect internationally recognized labor rights through ENS’s research on emblematic cases. In addition to the CAL offices in four cities, mobile CALs were created to reach workers in the priority sectors identified in the Action Plan (i.e., palm oil, sugar, mines, ports, and flowers); however, only a small proportion of all workers assisted at the CALs in 2015 (approximately 11 percent) were assisted through a mobile CAL, indicating that the proportion of CAL users who belong to priority sectors has been relatively low. 3. Project Progress and Effectiveness


As noted during the midterm evaluation, the performance monitoring plan (PMP) lacks detail on process and outcome indicators and is not a suitable tool to measure progress. This finding is still true for the final evaluation. While the CALs and ENS report data for most project activities, this information is dispersed across many different documents rather than being reported in a consistent, standardized way. Despite these limitations, the evaluation team observed substantial progress in two key activities: (1) the rollout of the CAL database and (2) the publication of the Workers’ Rights Handbook. Roll-out of the new CAL database started in August 2015 and was completed by the end of 2015. The Workers’ Rights Handbook has been improved and now uses simpler language and depictions that are more suitable for workers. A strategy to disseminate and promote the handbook has been designed and will be implemented this year. The project’s outreach campaign, another key activity, is no longer suitable for its original purpose. The initial outreach strategy, to increase demand for CAL services, is no longer feasible. Most CALs do not have the physical capacity to assist a larger number of workers given their current office sizes. Instead, ENS has revised the outreach strategy to help disseminate the CALs’ work, share practical information and news on labor rights, and provide a communication platform to unions. To do this, the CALs are increasing their online presence through a new website (www.calcolombia.co), various social media platforms (Facebook, Twitter, YouTube), and electronic newsletters. Leveraging resources from other funding sources, ENS hopes to provide unions with a more comprehensive strategy that combines legal assistance, trainings, and communications. The CALs rely significantly on law student interns to provide services to workers. Students’ knowledge of the CALs’ legal assistance protocols varies within and across the CALs. Variation also exists across the CALs in recording information in the CAL database, particularly data related to user follow-up. Without consistent follow-up data, the CALs are unable to comprehensively monitor and track the outcome of workers’ cases. ENS leaders have learned from experience over the past 2 years that labor inspectors or prosecutors do not necessarily abide by favorable rulings from the high courts. ENS leaders believe that the effectiveness of emblematic cases depends on the political will of Colombian institutions to follow the rule of law and ensure compliance at all levels with the rulings of the high courts. 4. Efficiency in the Use of Project Resources

Overall, ENS and the CALs have used project resources efficiently. However, three main categories seem to have been under-budgeted in the original 3-year project:

Equipment, like office furniture and computers

Activities related to completion of Outcome 1: Workers’ improved knowledge of and ability to defend their labor rights

http://www.calcolombia.co/


Activities related to completion of Outcome 2: Workers’ use of the administrative and legal mechanisms to protect their labor rights

ENS compensated for this budget shortage in the extension year. About one-third of the additional funds received in the extension year were allocated to these categories. 5. Effectiveness of Management Arrangements

CAL leadership demonstrated diverse strengths in managing the CALs’ various activities (e.g., day-to-day operations, legal assistance, and strategic alliances). This diversity of strengths does not appear to have affected the quality of services but has had a direct effect on progress of the CALs’ sustainability efforts. ENS´s decentralized management model has given the CALs the freedom to develop their own student training, mobile CAL coverage, and outreach efforts. This approach has worked best when the CAL director has either strong institutional support from ENS (Medellín) or from highly organized unions (Bucaramanga). 6. Impact Orientation and Sustainability Partnership building and collaboration with the national federations, a cornerstone of the initial project strategy, has been slower than anticipated. Some evidence exists to support an incremental change in attitude about partnering with government institutions to further the CALs’ objectives. For example, Bucaramanga is currently pursuing a strategy to engage mayors from three municipalities to include the CAL initiative in their development plans. Further, the CALs have focused their attention on more established unions as an option to sustain the project post-USDOL funding. For example, the Cartagena CAL office moved to a union-owned building, and students in Bucaramanga and Bogotá provide legal assistance at union offices at least once a week. The CALs are considering other partners and mechanisms to sustain themselves post-ILAB funding, including payment for services. Given the socio-economic background of the overwhelming majority of CAL users, it is unlikely that a standard fee-for-services approach would be feasible to sustain services for this population. However, other alternatives to this model could be tested in certain CALs. For example, some users who are already familiar with the CAL and have demonstrated a willingness to pay for additional CAL services could be offered these extra services for a subsidized fee. As another option, fees could be charged on a sliding scale to help cover some of the CAL costs.


Key Recommendations for ENS The following recommendations are based on the evaluation team’s findings and conclusions. 1. CAL Database and Workers’ Rights Handbook

a. Follow up on usage of the new CAL database, particularly during the initial months after rollout. Facilitating tutorials for users is a good first step to train users in the new database. This guidance could be strengthened with Skype calls with users to identify and respond to their main issues.

b. Develop a CAL database oversight protocol to ensure that all students are adequately and consistently recording all of the necessary user and case information. The current video tutorials are a first step in this process, but the oversight protocol should describe the responsibilities of each staff member, explain the database fields that students find the most challenging, and include a standard methodology to follow up on workers’ cases. In addition, CALs should continue to strongly emphasize the importance of recording accurate data in the CAL database and the value of user follow-up.

c. Collect workers’ and CAL directors’ feedback during the implementation of Worker’s Rights Handbook workshops. Since this will be the first time workers will receive and use the handbook, keeping track of their impressions and level of comfort in using it will be useful so that the tool can be further refined to meet their needs. In addition, CAL directors and ENS staff should keep track of relevant legal changes that may require revisions in the handbook.

2. Outreach and Communications Strategy

Continue adapting the outreach strategy to the changing circumstances and needs of the project. The initial outreach strategy, to increase the demand for CAL services, is no longer feasible. Instead, the CALs’ outreach strategy should be targeted to decision-makers who may provide sustainability to the initiative. For example, municipalities may contribute free office space (through comodato contracts) for CAL functioning. ENS could also partner with other labor rights-focused organizations to advocate with their political representatives and pass legislation beneficial to workers.

3. Internal Protocols

Ensure that protocols and training materials are distributed and readily available to all students and volunteers. As the project matures, a strong foundational structure is important in case of expansion, high student turnover, or management issues.

4. Sustainability

a. Continue strategic partnerships with established and new unions to embed the CALs’ objectives into the labor rights strategy of local unions.


b. Reach out to stakeholders who have alternative financing ideas and evaluate the most promising designs. Actively testing different approaches is important, particularly when the approaches involve charging users, to avoid an undue burden on the workers.

c. Develop a plan to leverage data gathered from the CAL database to engage other institutions. For example, data on results could be used to demonstrate the efficacy of the project to potential donors.

d. Actively engage government institutions to demonstrate the benefits of the CAL initiative. The example of the Bucaramanga CAL engaging mayors in neighboring communities demonstrates the potential of this approach.

Final Considerations for ILAB Based on the findings of the implementation evaluation, the team identified key factors that ILAB should consider when implementing similar workers’ rights projects.

a. Start-up time: ENS underestimated the level of time and effort necessary to open offices in new cities. The most significant challenges were obtaining office space and setting up agreements with the labor federations and universities. Future projects should take into account potential start-up delays and challenges in project work plans and budgets and consider alternative options to maintain momentum during project design.

b. Reporting and evaluation requirements: Though experienced with international donors, ENS was largely unaware of the level of effort required for the regular reporting and evaluation activities associated with implementation of the CAL project and the project evaluation. Further, the project coordinator was unaware of ILAB’s Management Procedures Guidelines (MPG). In the future, ILAB should consider emphasizing technical assistance to its grantees to ensure that grantees are aware of and understand their responsibilities and the time and resources necessary for reporting and evaluation requirements. Even though these requirements are described in the MPG, the information provided is heavily focused on child labor projects, and parsing out what is relevant to these kinds of projects could be difficult. Further, ILAB should consider requiring at least one full-time person dedicated to monitoring and evaluation, data collection and reporting activities across the initiative to ensure consistency of data and reporting and to allow on-the-ground staff more time for service delivery.

c. Clearly defined and documented indicators: This project’s PMP lacked detail on process and outcome indicators, though such indicators are essential to monitoring progress and assessing the impact of a project. Future projects should finalize the outcome indicators before project implementation or refine them early in the implementation phase if necessary. A greater effort should be made at the project design stage to refine the kinds of outcome indicators the project can track without putting undue burden on implementers’ systems and resources. Further, indicators should be realistic and reflect the full spectrum of project activities.


d. Assessment of the full political landscape: The project design should include a full analysis and understanding of the political landscape of all major stakeholders in the project. For example, since the union federations were thought to be a critical partner in the implementation of the project, project assumptions should have also included how the labor unions’ political environment could affect the success of the project.

e. Strong relationships with partner organizations: As key beneficiaries of the initiative, all partners should be actively engaged (including local and national unions) and should guarantee support and commitment to the initiative even before the CALs begin operating.

f. Active institutional support: The CALs benefit from the institutional and management support provided by ENS, such as support for the CAL database and research on priority sectors. This support has allowed the CAL directors to focus more intently on sustainability efforts and the day-to-day operations of the CAL offices and mobile CALs.

g. Engagement of other stakeholders: Assessing all potential partnerships is important to sustainability. For example, the CALs should actively seek out opportunities to engage other potential partners who could contribute to the sustainability of certain activities, such as national and local governments. Ideally, the identification and engagement of other stakeholders should be carried out even before the CALs begin operation, and the nature of these partnerships should be detailed in the project’s sustainability plan.

h. Stable workforce: Since law student interns are an integral part of the initiative, the CALs should initiate university agreements that allow students to intern for longer periods of time (at least 20 hours a week). Greater stability of the CAL workforce would allow for more consistency and better integration of the students into the CALs’ operations.

i. Detailed and effectively executed transition strategy: Implementing a well-defined and realistic sustainability strategy from the start of the initiative and engaging stakeholders throughout the project is necessary to effectively establish and achieve milestones.

j. Target priority sectors: While the Project Document identified priority sectors that the CALs were meant to target through mobile CALs, it did not establish concrete goals to measure the level of success in reaching these sectors. Given that there is demand for CAL services among non-priority sectors, more discussion is needed within ILAB and with the CALs to reach an agreement on what is expected of this approach and to establish realistic targets. Recognizing the need for flexibility when identifying priority sectors and setting CAL-specific targets is important, since factors inherent to the sector (i.e., level of union organization) and to the CAL itself (e.g., other priorities, CAL characteristics) can influence how effective CALs are in reaching them.


1. PROJECT BACKGROUND AND DESCRIPTION The United States Department of Labor (USDOL) Bureau of International Labor Affairs (ILAB) contracted with IMPAQ International, LLC (IMPAQ) to conduct an independent evaluation of the Strengthening Protections of Internationally Recognized Labor Rights in Colombia project4 under the contract titled ILAB Impact and Implementation Evaluation for Colombia. This report presents the final results of the implementation evaluation. The purpose of this evaluation was to:

1. Assess the progress of Colombia’s Escuela Nacional Sindical (ENS) in achieving its objectives

2. Identify lessons learned from its program strategy and key services implemented

3. Provide feedback to ENS and ILAB with regard to the project’s achievements and recommendations for future projects

This report augments the information presented in the Midterm Implementation Evaluation Report. Specifically, this report assesses the changes and progress made since that time in the following six areas of analysis:






6. Impact orientation5 and sustainability The report places special emphasis on program sustainability. 1.1 Background: Labor Rights in Colombia Workers in Colombia have minimal access to social protections and labor rights. Participation levels in the informal sector of the economy are high, at 74 percent. The great majority of workers in the informal sector earn less than the minimum wage (approximately $225 per month)6,7 and lack access to comprehensive social protections, including health insurance, retirement savings,

4 The ILAB-ENS Cooperative Agreement states the agreement period as December 27, 2012 to December 26, 2015. USDOL ILAB granted ENS a 1-year extension of the project from December 27, 2015 to December 26, 2016. 5 Impact orientation refers to the steps taken by ENS to achieve the project’s objectives in the long term and ensure that the project’s activities will continue after the project’s completion. 6 World Bank. (2010). Informality in Colombia: Implications for Worker Welfare and Firm Productivity. https://openknowledge.worldbank.org/handle/10986/2889. License: Creative Commons Attribution CC BY 3.0. 7 In 2016, the approved minimum wage was increased by 7 percent to COP 689,455. Due to currency depreciation, however, the minimum wage decreased in value from $270 per month as of January 2015 to $225 per month as of May 2016.

https://openknowledge.worldbank.org/handle/10986/2889


and occupational accident insurance. In addition, Colombia has a high level of structural unemployment with more than 2.3 million people unemployed. Nearly half the people who are unemployed are youths under the age of 25.8 As part of the United States–Colombia Trade Promotion Agreement (TPA), which went into effect on May 15, 2012, the Colombian government agreed to develop and implement an Action Plan Related to Labor Rights (Action Plan), which was signed by Presidents Barack Obama and Juan Manuel Santos on April 7, 2011. Successful implementation of key elements of the Action Plan was stipulated as a precondition for the TPA to take effect. Under the Action Plan, the Colombian government committed to take specific steps to bring Colombian labor laws and practices into greater conformity with internationally recognized labor rights. Specifically, the Action Plan is geared toward meeting the following five objectives:

1. Promoting the formalization of labor by eliminating illegal labor intermediation

2. Protecting the right to organize unions

3. Protecting the right to collective bargaining

4. Strengthening state institutions charged with labor administration

5. Overcoming violence against unions and impunity for its perpetrators In Colombia, during President Álvaro Uribe’s administration (2002-2006), the Ministry of Labor was merged with the Ministry of Social Security to form the Ministry of Social Protection (MSP).9 At that time, concerns emerged about the capacity of the MSP to enforce labor rights. For this reason, the current president of Colombia separated these institutions again in 2011. The newly re-established Colombian Ministry of Labor houses a labor inspectorate, which is charged with enforcing the provisions of the labor code among other duties. Historically, labor inspectors have been unable to enforce the labor code efficiently and effectively because of a variety of logistical and regulatory impediments and a lack of training.10 Even with an increase in the number of labor inspectors and investigations conducted in priority sectors, several challenges still remain—for example, the minimal collection of fines imposed by Colombian authorities,11 which may be sending the wrong message to society. As part of its continued efforts to inform workers about their rights, the Ministry of Labor recently established a program called COLabora (Centro de Orientación y Atención Laboral) [Center for Orientation and Assistance on Labor Issues]. COLabora does not provide legal assistance services to individuals. Rather, it gives general guidance and responds to frequently asked questions about such topics as the amount of the minimum monthly wage or the maximum number of

8 Departamento Administrativo Nacional de Estadística. (2014). http://www.dane.gov.co/index.php/estadisticas-por-tema/mercado-laboral. 9 USDOL ILAB. (2011). Colombia Labor Rights. http://www.dol.gov/ilab/reports/pdf/colombia_LRR.pdf. 10 Ibid, p. 15. 11 Office of the United States Trade Representative and U.S. Department of Labor. (2015). Standing Up for Workers: Promoting Labor Rights Through Trade, p. 23. http://www.dol.gov/ilab/reports/pdf/USTR%20DOL%20Trade%20-%20Labor%20Report%20-%20Final.pdf.

http://www.dane.gov.co/index.php/estadisticas-por-tema/mercado-laboral

http://www.dane.gov.co/index.php/estadisticas-por-tema/mercado-laboral

http://www.dol.gov/ilab/reports/pdf/colombia_LRR.pdf

http://www.dol.gov/ilab/reports/pdf/USTR%20DOL%20Trade%20-%20Labor%20Report%20-%20Final.pdf


hours a person can work before overtime compensation is required. COLabora operates two offices in Bogotá, but the majority of workers access COLabora services through a website and a toll-free telephone number.12 1.2 Escuela Nacional Sindical and the Workers’ Rights Centers (CALs)13 Building on the momentum from the Action Plan, in December 2012, ILAB awarded a $1.5 million grant to Colombia’s Escuela Nacional Sindical (ENS) to implement the Strengthening Protection of Internationally Recognized Labor Rights in Colombia project. Originally a 3-year project, it received a 1-year extension with an additional $600,000 in funding for a total of $2.1 million. The project focuses on increasing the awareness of workers’ rights and supporting the Colombian government’s efforts to strengthen labor rights through the strengthening and creation of four Centros de Atención Laboral (CALs) [Workers’ Rights Centers] operated by ENS in Medellín, Bogotá, Bucaramanga, and Cartagena. The project was designed to reach local workers in six priority sectors: palm oil, sugar, mines, ports, flowers, and the public sector. The CALs provide free legal assistance services to facilitate the effective protection of workers’ rights and help achieve the project’s two main outcomes:

Outcome 1: Workers, with the assistance of the CALs, will improve their knowledge of and ability to defend their labor rights.

Outcome 2: Workers will make more frequent and more effective use of the administrative and legal mechanisms to protect their labor rights.

In September 2005, ENS partnered with the Central Unitaria de Trabajadores (CUT) [Central Union of Workers], Colombia’s largest trade union, to pilot the CAL project in Medellín. Because of the high worker demand for CAL services, ENS and CUT opened a new office in Bogotá 4 years later.14 In 2011, CUT partnered with another trade union, Confederación de Trabajadores de Colombia (CTC) [Confederation of Workers of Colombia] to further the CAL initiative and strengthen the institutional and political foundation of the project. ENS has also established a strategic alliance with the American Center for International Labor Solidarity (Solidarity Center) to promote research and projects protecting vulnerable groups. Both organizations work closely to support the Afro-Colombian Labor Council to advance racial inclusion in the labor movement and in society. With funding from the USDOL grant, ENS reopened the CAL in Bogotá and established new offices in Cartagena and Bucaramanga in March 2014. Exhibit 1.1 shows a map of the four CAL offices.

12 Colombian Ministry of Labor, COLabora. http://www.mintrabajo.gov.co/colabora. 13 For clarity, this document will follow the same pattern as the Project Document and differentiate between project staff working directly in the CALs in the four cities (“CAL staff”) and those working at ENS headquarters in Medellín (“ENS staff”). 14 This CAL closed in 2012.

http://www.mintrabajo.gov.co/colabora


Exhibit 1.1: Map of CAL Offices

Administratively, the four CALs are governed by a national executive board composed of the directors of the CUT and CTC union federations, with ENS acting as the technical advisor to the initiative. The executive board provides strategic guidelines for CAL services and oversees four regional boards, one for each CAL. Each regional board is charged with monitoring the operationalization of the guidelines established by the national executive board. The regional boards are composed of the presidents of the unions’ departmental sub-directorates. In its technical advisory role, ENS is actively involved in developing management tools, reviewing and improving the quality of the CALs’ services, and managing donor and partner relationships. According to an ENS document on the CALs’ sustainability strategy, ENS was supposed to transfer management of the CAL project to the CUT and CTC union federations through the regional boards.15 As will be discussed later in the report, this transfer is unlikely to take place in the near future, and ENS is currently working on a new sustainability strategy. 15 ENS. CAL’s Sustainability Strategy [Estrategias de Sostenabilidad de los CAL]. Shared with IMPAQ on November 2014.


1.2.1 CAL Services CAL offices offer assistance to victims of labor rights violations by providing free and immediate legal advice to workers.16 Whenever possible, the CALs have been located close to the regional office of the Ministry of Labor to increase the number of workers who seek legal assistance. In addition, ENS staff collaborate with the CALs and the Solidarity Center to pursue strategic or emblematic cases, which have the potential to set legal precedents in favor of workers’ rights. The main services provided by the CALs are legal assistance services [asesorias jurídicas]. Two types of services are offered: (1) actionable services17 and (2) informational services (see Exhibit 1.2).

Exhibit 1.2: Types of Legal Assistance Offered by CAL Offices

Nature of Legal Service Type of Legal Action/Service

Actionable Legal Services

Constitutional challenge [tutela]18 Formal requests [derechos de petición] Payment claims [reclamaciones de pago] Complaints [quejas] Criminal complaints [querellas] Research inquiries [solicitudes de investigación] Enforcement actions [desacato] Appeal of a constitutional challenge [impugnación de

tutela] Involuntary resignation [renuncia motivada]

Informational Legal Services

Labor settlements [liquidaciones laborales] Labor rights information requests [conceptos

juridicos] Insistence requests [solicitudes de insistencia]

The CAL program targets workers in six priority economic sectors identified in the Project Document submitted by ENS to ILAB.19 These priority sectors are the public sector, ports, sugar cane, mining, flowers, and African palm. The primary mechanism that the CALs use to reach workers in these sectors is the mobile caravans (or mobile CALs). In principle, the mobile CALs offer legal assistance services to rural workers who are unable to travel to any of the four urban CAL offices. Each CAL determines how to operate the mobile efforts based on estimates of the number of workers that the mobile units will assist. CAL staff undertake trips to the catchment areas (geographical areas served by each CAL) linked to their office. The mobile CALs also aim to 16 The fundamental rights are defined by the Colombian labor code, the National Constitution of Colombia, and the ILO agreements on forced labor, as equal remuneration for male and female workers, minimum age of entrance into the labor market, and collective bargaining. 17 ENS identifies these legal activities as “acciones jurídicas que se verifican.” The services referred to as actionable are legal assistance services that are provided to workers so that they can file or otherwise initiate a legal claim. 18 A legal instrument that allows individuals and organizations that have experienced a violation of a constitutional right to present a legal claim before a judge. 19 ENS. (Sept. 2014). Project Document, p. 2-3.


increase workers’ awareness of labor rights violations in the priority sectors by providing labor rights forums (talleres sobre derechos laborales). Exhibit 1.3 presents the mobile CALs’ coverage by sector of interest and geographic region.

Exhibit 1.3: Mobile CAL Coverage

Sector of Interest CAL Office Regional Influence

Ports Medellín Municipality of Turbo y

Buenaventura Cartagena Barranquilla and Santa Marta

Sugar cane Medellín Municipality of Palmira y Cali

Mining (oil refinery) Cartagena Barranquilla and Santa Marta Bucaramanga Barrancabermeja

Flowers Bogotá Gran Sabana region African palm Bucaramanga Puerto Wilches

Note: Since public sector workers are located primarily in cities, the CAL offices are the primary resource for this sector.

1.2.2 Program Logic The program logic model that Exhibit 1.4 presents is based on the project’s purpose, outcomes, outputs, and activities as described in the Project Document.20 The blocks on the left side of Exhibit 1.4 list the activities that are implemented; the blocks in the middle list the outputs; the blocks on the right side identify the outcomes and impacts that will be achieved as a result of the interventions. As described in the Project Document21 and in Exhibit 1.4, the project will achieve its outputs by performing the following activities:

A. Prepare a Workers’ Rights Handbook.

B. Build CAL infrastructure.

C. Formulate and sign agreements with universities.

D. Train CAL staff.

E. Collect information on CAL cases.

F. Launch an outreach campaign.

G. Provide legal assistance services.

H. Initiate administrative or legal actions to protect labor rights.

I. Undertake structural investigations of the specific sector-based manifestation of labor rights violations in order to initiate constitutional processes.

J. Undertake legal processes to benefit a large group of workers.

K. Initiate penal and labor processes before Colombian judges. 20 Ibid., p. 11-16.


Exhibit 1.4: Program Logic

Source: The logic model depicted was drawn from the Project Document. Arrows describe the logical relationships among activities, outputs, and outcomes stated in the Project Document (p. 11–16).


As indicated in Exhibit 1.4, activities A through F (left column) are expected to lead to the following outputs:

(1.1) Strengthen CAL offices in Bogotá and Medellín (these offices existed before USDOL funding and are being expanded) and establish two new CAL offices in Bucaramanga and Cartagena to provide permanent technical and legal assistance to workers seeking redress for workers’ rights violations. The four CALs will conduct bimonthly mobile CAL legal assistance caravans for the entire project’s duration.

(1.2) Produce a manual to help workers prepare administrative and criminal complaints. The handbook will contain the legal aspects (substantive and procedural) that are necessary for workers to prepare complaints and charges.

(1.3) Develop CAL databases that include information on clients who have obtained legal assistance, legal and administrative actions taken, and outcomes of actions.

(1.4) Launch a regionally targeted public information campaign that will consist of distributing flyers and organizing and attending academic conferences on labor law issues. The flyers will contain contact information and a brief description of CAL services. The academic conferences will take place at universities, and will aim to increase awareness of labor right violations and disseminate recent changes in labor law.

Activities A through D and their associated outputs focus on building the CALs’ capacity—for example, staffing and training--and creating resources that will be readily available both to CAL staff and workers. The CALs will prepare periodic reports that provide examples of how workers can ensure the protection of their labor rights. These reports will inform the content of the Workers’ Rights Handbook. The CALs will also organize workshops that will explain how workers can use the Workers’ Rights Handbook effectively and follow its procedures to make demands and initiate complaints to protect their labor rights. In addition, the systematic information collected at the CALs via the CAL databases (Activity F) will enable staff to gather evidence to support cases, design pathways for worker protection, and support the use of institutional mechanisms of worker protection. Thus, by seeking assistance from the CALs (Activity G), workers will gain access to resources about how to file labor claims. They will also gain broader knowledge about new and existing labor laws and a better understanding of the legal system (Outcome 1). These benefits will be reinforced by the implementation of the public information campaign (Activity F). The rest of the CALs’ activities (H through K) focus on the provision of legal assistance services, which can take several forms (e.g., asesorias jurídicas (legal counseling) or, in some cases, the initiation of penal and labor processes before Colombian judges). These activities will lead to the following outputs:

(2.1) Research reports analyzing repetitive violations

(2.2) Legal and administrative processes presented to the competent public authority

(2.3) Legal decisions dictated by the corresponding authority


For example, through Activity I (ENS’s analysis of repetitive types of cases brought to the CAL), the CALs are meant to furnish the information needed for ENS’s research reports analyzing structural investigations (Output 2.1), which can guide the design of appropriate pathways for worker protection. Overall, by receiving direct assistance from CAL services and guidance through the processes of the legal system, workers will learn how to file grievances, criminal complaints, and tutela actions and be equipped to choose the most appropriate legal mechanisms to vindicate their labor rights. For example, one of the potential benefits of CAL services is that workers will be able to identify the correct legal instrument or mediation effort that they should use in each case. Use of the correct legal instrument may increase the likelihood that workers will have a positive result. Moreover, the CALs are able to facilitate claims that would otherwise be difficult for an individual worker to pursue on his or her own. Accordingly, increasing the use of CAL services will result in increasing the frequency with which workers use administrative and legal mechanisms to protect their labor rights and the effectiveness of these mechanisms (Outcome 2). 1.2.3 Project Assumptions In the Project Document, ENS identified a set of project assumptions.22 These assumptions are critical to the performance of the project because, if the assumptions do not hold, the project’s performance may be affected. The following are the project assumptions. Section 3.1 provides an assessment of the assumptions.

Colombia maintains relative political stability to make room for workers to freely exercise their rights.

There is an environment of tranquility without threats or violations of human and labor rights against the working population.

The tools created within the Action Plan are maintained with no detrimental changes to rights.

The Colombian government shows the political will to improve compliance with labor rights.

The Colombian government adopts labor measures that have a positive impact on the working conditions of workers.

Government institutions provide the information necessary to ensure compliance via administrative and judicial actions.

The U.S. and Colombian governments maintain their interest and political will to implement the Action Plan Related to Labor Rights.

Workers commit to new processes to increase participation in democracy, overcome employers’ impunity, and defend labor rights.

22 ENS. Project Document, p. 16-19.


Workers who are in a precarious situation are interested in improving their working conditions and trust the ENS.

Workers are willing to work collaboratively within their sectors and with strategic actors to strengthen advocacy efforts.

The universities in the departments of Bolívar, Santander, Cundinamarca, and Antioquia are willing to establish agreements with the ENS to select the final-year law students.

Law students from the universities in the departments of Bolívar, Santander, Cundinamarca, and Antioquia are interested in participating in the training processes and in the CALs’ work.

1.2.4 Critical Factors for the CALs’ Success After reviewing ILAB reports from similar projects, 23 the evaluation team identified a set of critical internal and external factors that may affect the CALs’ success. These factors, which are different from the project assumptions described in the Project Document, denote internal and external aspects of the project that affect its success. These critical factors were discussed with ENS during the evaluation team’s first visit to Colombia in March 2014. In general, ENS recognizes these critical factors. Section 3.1 provides an assessment of the critical factors. Internal Critical Factors The internal factors refer to aspects that are influenced by the project’s performance. The team identified the following internal critical factors and reviewed them with ENS:

A Performance Monitoring Plan (PMP) is in place: The PMP contains the performance indicators for the project’s outcomes and outputs24 and is a critical tool in any performance-based management approach. It should also include a detailed plan for action over time and across the CALs, allowing the project to collect comparable data even when key personnel change. Without a PMP, no baselines or targets against which to evaluate the project’s achievements exist.

The outreach strategy drives enough demand for legal assistance: Without sufficient demand for the legal services offered by the CALs, the project will fail to achieve its expected outcomes. The project’s outreach campaign, which consists of distributing flyers and organizing and attending academic conferences, is intended to attract workers who have grievances.

The curriculum of the training program is relevant and based on the current legal framework: The materials included in the training program are adequate to guide law students who offer legal assistance services. If the volunteer attorneys and law students do not have specific or relevant training in labor law (in particular, training on legal issues

23 Independent Mid-term Evaluation of the Union Capacity Development Component of the “Better Work Vietnam Program.” (June 2013). U.S. Department of Labor, Bureau of International Labor Affairs. (Dec. 2008). Final Evaluation of the Labor Law Compliance Program in Central America and the Dominican Republic. Final Report. 24 ENS. Project Document, p. 25.


that would help workers), the quality of the legal assistance services will drop, thus reducing the project’s capacity to achieve its expected outcomes.

Technological capacity exists: CAL offices have a technological platform in place to manage and analyze data from workers. Without the proper technological capacity, CAL staff will not have the data to plan, monitor, and evaluate the project and conduct research.

All offices are able to hire lawyers and recruit law students to assist workers in need: The signed agreements with universities are designed so law students can practice law at the CALs. Without lawyers and law students, the CALs cannot offer legal assistance services to workers.

Legal claims are supported by complete and appropriate documentation:25 Without solid documentation and evidence, the labor rights claims presented to the administrative and judicial authorities will be of substandard quality and not persuasive.

External Critical Factors The external factors are those beyond the project’s control that affect the project’s performance. The following are the external critical factors that the team identified based on contextual research on the Colombian labor market:

Legal regulation: If the country’s legal framework for workers changes because of the enactment of new laws or new institutions protecting workers’ rights, labor conditions likely will improve correspondingly or workers will have more successful claims independent of the project’s direct intervention. On the other hand, if the government’s administrative or legal authorities do not have the capacity, knowledge, or willingness to respond to workers’ claims, then the project will have limited effect regardless of the intervention.

Economic performance: If the country’s economic performance causes an increase or a reduction in informal work arrangements and poverty, the number of workers who use legal services may be affected independently of the project’s direct intervention.

Cultural practices: If workers’ attitudes toward legal assistance services improve as a result of external factors, such as a new policy aimed at protecting workers’ rights in the palm sector, then the number of workers who access the CAL offices and use legal services may increase independently of the project’s direct intervention.

1.3 Operational Context The operational environment of the CALs consists of several institutions and organizations that seek to support and improve the labor rights of workers, which include government offices, workers’ unions, university legal clinics, non-governmental organizations, and international 25 A legal claim is an action filed by a worker whose rights have been violated, as outlined in the 1991 Colombian Constitution, international treaties, and the Colombian Labor Code. Some examples of legal claims are denuncias penal, prevaricato, quejas, querellas, and tutela.


organizations. Exhibit 1.5 presents a diagram of the context in which the CALs operate and the course for workers to receive legal support for labor rights violations in Colombia.


Exhibit 1.5: Operational Context for Receiving and Providing Legal Support in Colombia


As indicated in the top row of Exhibit 1.5, in Colombia, the worker–employer relationship should be governed by a contract between the two parties; however, employers often outsource services through intermediaries to avoid paying the benefits and compensation mandated by law. In cases in which a labor dispute arises between workers and employers, workers often go to the Ministry of Labor for initial assistance (middle row of Exhibit 1.5). Ministry staff may provide basic legal information, but the law prohibits them from offering legal assistance. Workers are then referred to other organizations, such as university legal aid clinics and the nearest CAL. At the CAL, law student interns help workers to file complaints with the Ministry of Labor or file legal claims in the court system, both of which have jurisdiction over employers. In addition to legal assistance to individual workers, the CALs also offer legal training to their student interns and organized workers and contribute to the legal advocacy work of ENS and the Solidarity Center. The key stakeholders are described below. Ministry of Labor. A major stakeholder in labor rights advancement in Colombia is the Ministry of Labor. As Section 1.1 mentioned, the Ministry of Labor supports two efforts to assist workers: the COLabora program and the labor inspectorate. In May 2013, the Ministry created the COLabora program (Centro de Orientación y Atención Laboral), intended as a national program with the purpose to give workers “better tools to guide, inform, and answer concerns” raised by both workers and employers and “answer updated information of duties and labor rights of all Colombians.” In 2015, the program assisted approximately 1.5 million people via four assistance channels: telephone (69 percent), onsite (20 percent), mail (7 percent), and Internet (4 percent).26 Although established as a national program, the only two COLabora offices are located in Bogotá, but the Ministry of Labor has territorial branches in all departments, which are available to the public. In addition, the Ministry of Labor employs labor inspectors who investigate labor complaints to ensure employers’ compliance with labor regulations. Workers visiting a CAL are often referred by a Ministry of Labor office. According to the project’s annual progress report for 2015, 78 percent of the clients of the Bogotá CAL were referred by the ministry. As part of their legal services, the CALs routinely help workers file complaints or petitions to labor inspectors, who also have the power to serve as arbitrators between employers and workers. Court System. If a worker and an employer consent, labor disputes may be submitted to an arbitration panel. Cases can also be initiated in the lower courts, either in a labor circuit court or a civil circuit court if there is no labor circuit court in the area.27 In special circumstances, the CALs assist workers in filing criminal complaints (querellas) through the legal system. If workers’ constitutional rights are being violated by an action or an omission of a public authority or if there is an imminent threat to their wellbeing, workers can file a tutela, or constitutional claim, with

26 Ministry of Labor. (2016). COLabora Management Report, 2015. 27 USDOL ILAB. (2008). Colombia Labor Rights Report. http://www.dol.gov/ilab/reports/pdf/ColombiaLaborRights.pdf.

http://www.dol.gov/ilab/reports/pdf/ColombiaLaborRights.pdf


any competent judge. The CALs assist workers in collecting evidence and documentation to present a tutela in the labor circuit court or, if necessary, appeal a ruling. Workers’ Unions and Federations. The main union partners are the CUT and the CTC, which are two of Colombia’s three trade union federations. The third federation, which is not involved in this initiative, is the Confederación General de Trabajo (CGT) [General Labor Confederation]. The CUT and the CTC have regional representatives on the CAL executive board. Depending on the city, the unions may also provide operational support for the mobile CALs and help convene workers. Universities. Partnerships with universities are essential to the CALs because they depend on law students to offer legal services. The CALs, in turn, offer students additional training in labor law and other relevant topics. International Labor Organization (ILO). The ILO in Colombia promotes international labor standards. The organization recently signed a memorandum of understanding with the Colombian government to collaborate and create decent working conditions for rural workers. ENS and ILO work jointly in the program to strengthen labor inspection in the Ministry of Labor. Solidarity Center. The Solidarity Center in Colombia works with unions and community groups to help them protect their right to association, to help end labor subcontracting, and to promote inclusion of women and Afro-descendant workers. Like the CALs, they also assist workers in vulnerable sectors such as palm, sugar, ports, and the public sector. ENS has entered into an agreement with the Solidarity Center to assist in the project’s design and cooperate during project implementation. The director of the Solidarity Center is a member of the CAL steering committee, and the center also has a representative on the management committee. Non-Governmental Organizations (NGOs). The CALs cooperate with local NGOs to provide additional services to workers and reach workers in priority sectors.


2. EVALUATION PURPOSE AND METHODOLOGY 2.1 Evaluation Purpose The purpose of the final implementation evaluation was to assess ENS’s progress toward achieving its objectives, identify lessons learned from its program strategy, provide feedback about the project’s achievements, and make recommendations for future projects. The implementation evaluation focused on the following six areas of analysis, which ILAB identified in the project’s Terms of Reference:






6. Impact orientation and sustainability Since this is a final evaluation report, more attention is given to impact orientation and the sustainability of project activities. 2.2 Methodology For this implementation evaluation, the methodology for data collection and analysis was primarily qualitative in nature. Quantitative data were obtained from project documents and reports and incorporated into the analysis when appropriate. Data collection methods and stakeholder perspectives were triangulated to bolster the credibility and validity of the results. The evaluation team collected and analyzed data from the following sources:

Key documents from the project

Site visit to the CALs, including focus groups with direct beneficiaries and CAL student lawyers, and interviews with project implementers and key stakeholders

Observations of the CALs Exhibit 2.1 lists each area of analysis, its main features, and the methodology that the team used to conduct the data analysis. The column labeled “Main Features” presents the significant concepts that the team examined. For example, in the validity analysis, the team looked at the logic of the project’s design and the consistency between the project’s goals and the legal needs of workers who might be interested in using CAL services. Under impact orientation and sustainability, the team evaluated the stakeholders’ commitment to continue supporting the services implemented in the CAL project.


Exhibit 2.1: Methodology by Area of Analysis

Area of Analysis Main Features Data Source

Validity of the project design

Logic of the project design Consistency between the project’s goals and the

needs of workers/clients

Interviews Focus groups Document review Observations

Relevance and strategic fit

Assessment of workers’ legal needs Relevance within the country context Strategy to address illegal indirect hiring

practices


Project’s progress and effectiveness

Status of the project’s outputs Progress in each project activity Effectiveness of the project’s capacity-building

strategies to work with clients and effectiveness of its offered services


Efficiency in the use of project resources

Management and use of project resources Collaborative efforts to reduce project costs and

increase outputs

Interviews Document review

Effectiveness of management Arrangements

Management plan to promote a positive and success-oriented work environment in responding to the challenges of offering legal assistance

Support structure and team communications


Impact orientation and sustainability

Stakeholder commitment Prospects for continued funding Orientation toward a labor rights advocacy

network

Interviews Focus groups Document review

2.2.1 Evaluation Schedule As part of the implementation evaluation, the team completed two site visits to each of the four CALs to collect information through semi-structured interviews and focus groups. The first site visit took place during the midterm of the project, in March 2015, and the second took place toward the end of the project, in March 2016. The evaluation team developed interview and focus group guides prior to the first site visit and modified them for the final visit. The second visit focused more intently on sustainability and identified changes that had been made following the midterm site visit in March 2015. The evaluation team also completed a project document review before each site visit. In addition to focus groups and interviews, the team conducted two sets of observations at the Bogotá and Bucaramanga CALs. The first observations took place in December 2014 and January 2015, to inform the midterm evaluation. The second round of observations took place during the March 2016 site visit as part of the final evaluation.


After completion of the site visits, the evaluation team held separate debriefings with ENS and ILAB to discuss key findings and recommendations. Appendix A shows the complete schedule of evaluation activities. 2.2.2 Data Collection The evaluation team collected data from four sources: (1) semi-structured interviews, (2) focus groups, (3) document reviews, and (4) observations at the CALs. The team used the data from these sources to answer the evaluation questions proposed for each analytic area. Semi-structured Interviews. During the midterm and final site visits, the team conducted semi-structured interviews to obtain stakeholders’ perspectives on the project’s implementation and progress. During both rounds of visits, the team interviewed members of the steering and management committees,28 project experts at ENS,29 and union federation representatives. The team also interviewed local experts outside ENS who had political and historical knowledge of labor rights issues in Colombia. External experts were identified through a convenience sampling approach and included individuals referred to the evaluation team by ENS, CAL, and ILAB staff. Whenever possible, the team interviewed the same person during both rounds of visits. Exhibit 2.2 presents the number and types of interviews conducted in each city in March 2016. Appendix B includes the interview guides. The Midterm Implementation Evaluation Report presents the number and types of interviews conducted in each city in March 2015 and the guides used.30 In Bogotá, the team interviewed the CAL director, the director of the Solidarity Center, and three external experts, including an ILO representative, three government representatives, and a university professor. In Bucaramanga and Cartagena, the evaluation team interviewed the CAL directors. In Medellín, the team interviewed the CAL director; members of the steering committee (the ENS project coordinator, the ENS general director, and the ENS academic director); the ENS expert on project planning, monitoring, and evaluation; and the ENS expert on research and labor rights documentation.

28 The steering committee consists of the ENS general director, the academic director, the director of the defense of rights and communications units, the ENS project director, and the director of the Solidarity Center in Colombia. The members of the management committee include the CAL directors; the ENS expert on project planning, monitoring, and evaluation; and a representative of the Solidarity Center. 29 The project experts at ENS include an expert on labor and constitutional law, an expert on research and labor rights documentation, and an expert on the investigation and documentation of violence and legal proceedings. 30 The number and types of interview participants were similar during the first and second site visits.


Exhibit 2.2: Semi-Structured Interview Participants, March 2016

City Steering Committee

Management Committee

CUT/CTC Representatives

Project Experts

External Experts Total

Bogotá 1 1 3 0 4 7 Bucaramanga 0 1 2 0 0 3 Cartagena 0 1 3 0 0 4 Medellín 2 1 2 2 0 7

Total 3 4 10 2 4 23 Focus Groups. During the two site visits, the evaluation team conducted focus groups in each city to assess the institutional capacity of the CALs. The two target groups were CAL clients and law student interns practicing in CAL offices. Each focus group addressed different types of questions. Appendix B includes the focus group guides. As discussed in Section 1.2.1, the CALs offer two types of legal assistance services: actionable and informational services. The team conducted focus groups with CAL clients who had sought legal assistance from the CALs for violations that were actionable, to understand the perspectives of workers who had experienced labor rights violations and were familiar with the CALs’ services, their effectiveness, and their effects on intended outcomes. The team conducted two focus groups with CAL clients at each of the four locations, one for male clients and one for female clients. The team also conducted a focus group at each CAL with law student interns, whose main responsibility is to offer direct legal assistance to clients. Exhibit 2.3 shows the number and type of focus groups conducted in each city. The Midterm Implementation Evaluation Report presents the number and types of focus groups conducted in each city in March 2015.31

Exhibit 2.3: Number of Focus Groups, March 2016

City CAL Clients* CAL Lawyers Total Bogotá 2 1 3 Bucaramanga 2 1 3 Cartagena 2 1** 3 Medellín 2 1 3

Total 8 4 12 *The evaluation team conducted one focus group with male CAL clients and one with female CAL clients at each site. **Due to a flight delay, the evaluation team was unable to conduct a focus group with student lawyers in Cartagena. Instead, the team interviewed one student who was available.

31 The team conducted the same number of focus groups with CAL clients and CAL lawyers during the first and second site visits.


Document Review. Prior to both site visits, the evaluation team conducted a document review to inform the semi-structured interviews and focus groups and supplement findings from other data collection efforts. In addition to addressing the six areas of analysis, the document review addressed overarching questions about the project, such as the cohesiveness of the project’s training and educational activities, operations, and outreach efforts. The team identified relevant documents for the review based on the research questions and the document’s main purpose. The three categories of documents reviewed were (1) training and education, (2) operation and reporting, and (3) outreach efforts. Exhibit 2.4 lists each document by category.

Exhibit 2.4: Documents Reviewed by Category

Training and Education Operation and Reporting Outreach Efforts 1. Workers´ Rights

Handbook 2. Workers’ Rights

Handbook workshop protocol

3. Workers’ Rights Handbook workshop curriculum

4. Defense of Labor Rights workshop protocol

5. Defense of Labor Rights workshop curriculum

6. CAL staff training protocol

7. CAL staff training curriculum

1. Project Document 2. Document on institutional design 3. Performance monitoring plan 4. Work plans 5. Quarterly and monthly progress

reports 6. Technical progress reports for ILAB 7. CAL database protocol 8. Special populations protocols 9. Follow-up on case reports protocol 10. CAL survey protocol 11. Reports on labor cases at the CALs 12. Research reports analyzing repeated

violations 13. Document on workers’ legal needs and

illegal indirect hiring 14. Strategic plan for sustainability

1. Mobile CALs strategy protocol

2. Outreach campaign strategy

3. Document on communication strategy

4. Cooperative agreements

Observation at the CALs. The team conducted observations at two CALs, one established (Bogotá) and one new (Bucaramanga), at two points in time. The first observations for the midterm evaluation took place in December 2014 and January 2015. The second round of observations took place during the March 2016 site visit as part of the final evaluation. The main objective of these observations was to view the daily operations of the CALs. Visiting one new and one established CAL enabled the team to obtain a comprehensive picture of how the CALs function at various stages of operation, including their policies, procedures, and activities. During these visits, the team examined the CALs’ infrastructure, including electronic and paper-based tracking databases, schedules, and records; monitoring and evaluation practices; and management approaches. 2.2.3 Data Analysis The document review, stakeholder interviews, focus groups, and observations generated a substantial amount of raw qualitative data. The team used standard qualitative data analysis methods, including matrix analysis, to categorize, compare, and summarize the raw data


captured from the interview and focus group transcripts, internal document review report, and internal observation report. The data analysis process was driven by the six areas of analysis. 2.2.4 Limitations The evaluation team collected information through interviews with various stakeholders. A limitation of data collected in this way is that, as in all qualitative research, the relevance of the findings depends on the accuracy of the information provided by the informants. The team made an effort to elicit other important perspectives by interviewing outside experts over the course of the midterm and final evaluations, including three Ministry of Labor representatives and an academic leader in Colombian labor rights. To assess the project assumption regarding the Colombian government’s political will, the evaluation team made an effort to interview government representatives recommended by ENS staff who are familiar with the project. However, the information provided was inconclusive. Whenever possible, the report includes references to publications that support stakeholders’ views. Because the team did not interview other government officials or employer/private sector representatives, the team was not able to corroborate all the information provided by workers. The team had to rely on the implementing agency to convene workers to participate in the focus groups. Colombia’s labor context made it highly unlikely that workers would respond to calls from an unfamiliar organization. At the request of the evaluation team, the implementing agency tried to recruit a balanced mixed of participants (i.e., unionized vs. non-unionized workers), but such balanced recruitment was not always possible.


3. FINDINGS This section presents the evaluation findings based on data collected from (1) interviews and focus groups conducted with project stakeholders in Colombia, (2) observations at two CALs, and (3) a review of project documents and reports. The key findings are presented according to the six analysis areas:

1. Validity of the project design


3. Project progress and effectiveness



6. Impact orientation and sustainability. Each section concludes with a summary of lessons learned and promising practices based on the findings. As a general guide, Exhibit 3.1 includes a brief profile of the four CALs.

Exhibit 3.1: Profile of CALs

Characteristics Medellín Bogotá Cartagena Bucaramanga

Average staff size (2015)

30 student interns and 4 volunteers; 1 social work intern

49 student interns and 20 volunteers; no social work intern



Target sectors Ports and sugar cane Flowers Ports and mining

(oil refinery)

Mining (oil refinery) and African palm

# of CAL users (2015) 3,618 9,157 1,652 1,872

% of CAL users who are female (2015)

42% 52% 50% 30%

% of CAL users served by mobile CAL (2015)

21% 3% 13% 29%

Notable characteristics Oldest CAL Largest demand

of users Fewest CAL users to date

Stable student workforce

Key Strengths

Has strong institutional support from ENS due to close proximity to ENS’s office

Has additional support from an administrative coordinator who manages CAL logistics

Secured office space from union partnership

Has strong union support with mobile CAL effort


Characteristics Medellín Bogotá Cartagena Bucaramanga

Key Challenges

Longer distances to priority sectors means mobile CALs are more expensive

Difficulty in organizing staff trainings due to size and schedule of student interns

Underwent several changes in CAL management causing disruptions in leadership

Small office space means crowding and no privacy

3.1 Validity of the Project Design 3.1.1 Logic of the Project Design The project design was based on the experience of the CAL in Medellín. A joint initiative between ENS and the CUT and CTC union federations, the Medellín CAL celebrated its 10th anniversary in September 2015. While the union federations provide the political backing and direction to the project, ENS is responsible for providing the technical and management expertise required for the day-to-day functioning of the project. At the start of the project, ENS submitted a draft Project Document to ILAB that laid out the project’s design. Specifically, it detailed the project’s purpose, logical framework and theory of change, management approach, and sustainability strategy. The final version of the Project Document approved by ILAB still contained internal inconsistencies regarding outputs and activities as well as the issues regarding the PMP found during the midterm evaluation that Section 3.3.1 below describes; the evaluation team therefore relied on additional project documentation and site visit interviews to better understand the project’s vision and objectives. A representative from the Ministry of Labor in Bogotá explained that workers often lack access to quality legal assistance to determine whether they have a valid labor complaint and, if a violation does exist, the resources to pursue redress through institutional channels. This lack of access and resources is precisely in line with the project justification provided in the draft Project Document. A national survey of unmet legal needs in Colombia (2011–2013) found that the first barrier to formal systems of conflict resolution was a simple one—the lack of a basic understanding of the mechanisms available.32 In addition, people in extreme poverty are often the ones with the least knowledge of institutions that offer free legal assistance.33 The representative further noted that although he was not particularly knowledgeable about the CAL in Bogotá or about ENS as the implementer of the project, he stressed the importance of initiatives of this nature to assist workers in defending their labor rights. This project was built on the premise that offering legal services to individual workers would broaden and expand the knowledge and expertise of ENS and the unions, further allowing them to develop a legal strategy that would improve the legal context for workers in Colombia. 32 Bogotá Chamber of Commerce. National Survey of Legal Needs in Colombia 2011-2013: An evaluation of the impact of alternative methods of conflict resolutions in meeting the legal needs of Colombians. 33 Ibid.


Exhibit 3.2 summarizes the logic behind the project design.34 By offering high quality legal services to workers to defend their individual or collective rights, CALs are able to collect and store user information in a unified database. These data can then be used to analyze trends in violations, economic sectors, and other variables of interest, increasing unions’ and ENS’s knowledge of the current legal environment. At this second level, the CALs—sometimes in collaboration with ENS—engage other unions or affected groups to provide them with the technical legal assistance they need to submit their complaints to the relevant authorities. The third level, referred to as acciones jurídicas emblemáticas (emblematic legal cases) are highly visible strategic cases based on the information ENS has learned through the CALs and mainly driven by ENS’s legal team. These are cases that have the potential to set favorable legal precedents or impact a larger number of workers. The first initiative ENS established as part of the strategic cases included a list of norms in the Colombian Labor Code that ENS considers to be unconstitutional. ENS staff present constitutional challenges regarding these norms. Sometimes CAL directors are also involved in these cases, such as the filing of more than 150 querellas (administrative complaints) to the Ministry of Labor against collective pacts.

Exhibit 3.2: Logic of the Project Design

According to the project design, both of these strategies (i.e., CAL legal services and emblematic cases) were necessary to help workers “make more frequent and more effective use of the administrative and legal mechanisms to protect their labor rights” (outcome 2). In particular, emblematic cases, such as the ones presented to the Ministry of Labor, would serve as a concentrated effort to demonstrate how organized workers can and do file claims at the Ministry, increasing the effectiveness of this mechanism to protect labor rights. Further, ENS can more

34 Based on an interview with the project coordinator and on project work plans.


effectively and efficiently monitor and assess progress of emblematic cases, compared to individual cases where monitoring outcomes is resource intensive. Though not explicit outcomes in the Project Document, there are additional advantages of the project design that could potentially contribute to the overall long-term success of this project and the advancement of workers’ legal rights:35

1. Improving the public perception of unions: By offering free, high quality legal services to the general public, not just union workers, the union federations hope that the CALs will contribute to improvements in workers’ perceptions of unions. Their hope is that workers who have positive experiences receiving services at the CALs will share their experiences with other workers and, ideally, join a union.

2. Shape and expand the number of future labor lawyers: Project designers were also concerned with what they perceived as a trend among law faculties to de-emphasize the labor law specialization and focus on other more lucrative concentrations. To offset this trend, the designers believe that meaningful on-the-job training will motivate and encourage students to pursue degrees in labor law and dedicate their professional careers to defending workers’ rights.

The expansion of CALs to other cities across the country has made evident the importance of strong management capabilities and a common vision to sustain activities in the future. For example, although ENS articulates the vision and direction of the project, CAL directors may implement a particular activity with more or less emphasis depending on their context. This became relevant in the Bogotá CAL, for instance, where the previous director did not prioritize the work with unions to organize mobile CALs or other efforts as envisioned by ENS, in part due to the large demand from individual users. The Bogotá CAL has therefore reached few organized workers in the flower sector. The close proximity of the Medellín CAL to the ENS office has allowed ENS to take on some of the administrative burden of the Medellín CAL. Directors at other CALs noted that administrative tasks and responsibilities require a significant amount of their time, leaving less time for strategic activities that promote sustainability and partnerships. When redesigning the CAL initiative, ENS did not fully take into account the value and benefit of ENS’s daily institutional support to the Medellín CAL. Expansion to other cities has also created a tension between the two main mechanisms designed to reach the project’s goals (i.e., CAL level vs. strategic case level). The majority of CAL staff36 and direct beneficiaries interviewed (i.e., those directly working in and receiving services at the CALs) believe that the CALs’ main strength is in the provision of legal services to workers in the CAL offices and through the mobile CALs but that the CALs are expensive to maintain (see Section 3.4, Efficiency in the Use of Project Resources). The project designers, however, consider emblematic 35 It is important to note that ENS has not collected data on these outcomes, since they are not explicit outcomes for the purposes of this project. For this reason, the evaluation team is unable to determine whether these have had or could have a significant impact on the project’s overall goals. Instead, ENS considers these positive by-products of the project design. 36 “CAL staff” refers to CAL directors, administrative assistants, student lawyers, and volunteer lawyers.


cases to be the most effective mechanism to achieve the ultimate goals of the project. This tension has been intensified by the slow progress in obtaining a positive resolution to the emblematic cases, particularly the more than 150 administrative querellas filed at the Ministry of Labor since 2014 due to what ENS perceives as insufficient effort from the Ministry of Labor to investigate these cases. In contrast, the free legal and advisory services provided by the CALs bring almost immediate relief to their users and tend to be highly regarded by both individual workers and union members. Yet, as an ILO representative explained during the midterm evaluation, the emblematic cases presented at the Ministry of Labor by the CALs have started to give visibility to the broader issue of labor investigations. This visibility is important because all major stakeholders as well as external experts and informants identified the need to assist workers in filing collective claims as a strength of the project. Gathering evidence of the slow progress of these investigations could help the project bring attention to the performance of public institutions and therefore make headway on Outcome 2. ENS is already drafting an “access to justice” report for this purpose and intends to use this information to start a dialogue with the newly-appointed Minister of Labor and find a resolution to these cases. 3.1.2 Project Assumptions Overall, the project assumptions provided at the time of project design remain valid and relevant to the project. These assumptions can be grouped into three categories:

1. An enabling institutional environment and governmental political will

2. Interest from university administrators and law students

3. Interest and commitment from workers to defend their labor rights Exhibit 3.3 includes a summary of the status of the project assumptions. During the midterm evaluation, the evaluation team found that the initial expectations of institutional change had not materialized and that ENS leadership held a general perception of a lack of political will,37 particularly on the part of the Ministry of Labor, to ensure compliance with the administrative actions presented by the CALs. This perception has remained largely unchanged over the last year, although ENS reported that the Ministry recently started assigning case numbers to some of the querellas to initiate the administrative procedure. Unfortunately, when the evaluation team followed up with Ministry representatives in Bogotá regarding the project, they were unable to provide specific information on the status of these cases. In addition to the political will of the Colombian and U.S. governments, the political will and inner workings of the union federations themselves may also affect the outcomes of the project. The national and various regional representatives of the CUT and the CTC expressed different

37 See section 2.2.4 for limitations. The evaluation team was not able to independently evaluate the assumption regarding the Colombian government’s political will.


priorities and at times different perceptions of the project’s strengths. As an external expert remarked, the three main union federations in Colombia differ in their approach to negotiations, have different relations with other stakeholders, and have different internal functioning. These differences have resulted in fragmented collaboration between ENS, the CALs, the CUT, and the CTC and a nonviable expectation that the federations will take over the management of the CAL initiative. Better articulation by ENS on the role that the union federations are willing to assume in the CAL project would largely benefit the sustainability of the project. To address this issue, the CALs introduced a new initiative to collaborate with unions at all levels (and not just the union federations at the national level with political inner workings that may prove challenging) that are willing to commit resources to guarantee the continuation of certain project activities. However, ENS and the CALs have not developed standard indicators to track this important work across all CALs. Similarly, the status of the internal and external factors has also remained largely unchanged.38 Since the midterm evaluation, ILAB and ENS agreed that modifications to the PMP would not be feasible given that the ENS project ends in late 2016. However, some staff at ENS expressed interest in learning how to develop a PMP more aligned with ILAB standards and requirements.

Exhibit 3.3: Status of Project Assumptions

Project Assumptions Final Evaluation Findings

There is political will by the Colombian and U.S. governments to implement the Action Plan; the Colombian institutional environment upholds and protects labor rights.

There is relative political stability.

Initial expectations of institutional change have not materialized. The Ministry of Labor has not ensured compliance with administrative actions that the CALs presented. Access to the judicial system is still seen as a major obstacle for workers and the CALs.

There is a tranquil environment. U.S. and Colombian governments maintain interest in implementing the Action Plan. Tools created within Action Plan are maintained. The Colombian government has political will to improve compliance. The Colombian government adopts labor measures that have a positive impact on the working conditions of workers. Government institutions provide the information necessary to ensure compliance via administrative and judicial actions.

There is interest from university administrators and law students.

Universities are willing to establish agreements.

The CALs have reached agreements with relevant universities in the cities in which they are located. University students have shown strong interest in the program, and applicants to the

Law students are interested in participating.

38 See section 2.2.4 for limitations. The evaluation team was not able to independently evaluate the assumption regarding the Colombian government’s political will.


Project Assumptions Final Evaluation Findings program will soon surpass the CALs’ capacity to accept them.

Workers are interested and committed to defend their rights through advocacy efforts.

Workers are committed to defending their rights.

Workers are interested in and committed to defending their rights. Workers who visit the CAL trust the staff and collaborate with students to gather the information needed to present a claim.

Workers who are in a precarious situation are interested in improving their working conditions and trust the ENS. Workers are willing to work collaboratively within their sectors and with strategic actors to strengthen advocacy efforts.

A significant obstacle to measuring the success of the project continues to be the lack of agreed-upon target goals for the main outcomes. For example, the indicator for Outcome 2 is the “percentage of legal or administrative actions submitted by the surveyed users who obtain a positive result”39 but does not state the target percentage. Following up with CAL users is also substantially difficult. Estimates from the Medellín CAL, which is the CAL with the most experience tracking users, puts the percentage of cases in which students are able to follow up with workers at around 40 percent in a given year. Some staff also believe there is an inherent difficulty in ensuring that quantifiable indicators reflect what success looks like for a project of this nature. For example, while it is not uncommon for right to petition claims to be denied in the first place by employers, it is a necessary step in the legal strategy before workers can follow up with conciliation or file a tutela if needed. Although tallying right to petition claims and other types of actions would not be an appropriate outcome indicator, keeping track of their frequency does help to give a sense of the level of effort involved, thus making it a relevant output indicator.

Exhibit 3.4: Status of Internal Critical Factors

Internal Critical Factors Final Evaluation Findings

A performance monitoring plan (PMP) is in place.

The PMP, as described in the Project Document, was not in place. The evaluation team relied on the work plans and TPRs to measure progress for outputs.

The outreach strategy must drive enough demand for legal assistance.

The current strategy is appropriate given the CALs’ limited resources.

The curriculum of the training program is relevant and based on the current legal framework.

Trainings are relevant and customized depending on law students’ needs.

39 The indicator defines “positive result” as any of the following: economic payment (severance payment in cases of dismissal); granting of tutela action; response to a right of information request; investigation by the Ministry of Labor.


Internal Critical Factors Final Evaluation Findings Technological capacity exists. Rollout of the new CAL database was completed.

All offices are able to hire lawyers and recruit law students to help workers in need of assistance.

All the CALs have signed agreements with universities and have been able to recruit students.

Legal claims should be supported by complete and appropriate documentation.

According to interviewees and focus group participants, legal claims are complete and supported by appropriate documentation.

The Technical Progress Reports (TPRs) submitted to ILAB in 2015 showed important improvements in addressing identified consistency and clarity issues with reporting the progress on output indicators. A few discrepancies still remained, particularly when comparing results that should be the same across outputs. For example, one would expect that the number of tutelas listed under Output 1 would be the same as that reported under Output 2 or that an explanation would be included of why the numbers are different. Another important improvement has been a more detailed account of the actions that the mobile CALs undertook, as well as the number of workers assisted during the quarter. However, some important annexes required as part of the semi-annual reports were still missing, particularly the data tracking table template described in Annex E of the Management Procedures and Guidelines.40 Exhibit 3.5 presents the main issues in the TPRs submitted to ILAB.

Exhibit 3.5: Main Reporting Issues in Technical Progress Reports

Indicators in Technical Progress Report Main Issues

Number of workers assisted No issues (all CALs report consistent and complete data) Referrals Inconsistent reporting; missing for last quarter of 2015

Legal actions, administrative, and other Classification inconsistent across CALs and reporting periods

Mobile CALs Some quarterly information missing for Cartagena Student lawyers No issues (all CALs report consistent and complete data)

CAL staff trainings Report numbers of student trainings but do not measure “acquisition of knowledge” as suggested in the PMP

University agreements No issues (all CALs report consistent and complete data) Legal actions Improved (mostly consistent and complete data) Administrative actions Improved (mostly consistent and complete data) Verification process Improved (mostly consistent and complete data)

External factors that could impact the project’s success have remained largely the same since the start of the project and midterm evaluation (Exhibit 3.6). The most relevant external critical factor during project implementation has been the country’s economic performance. Strong positive growth could potentially lead to a reduction in informal work arrangements and poverty, and the

40 http://www.dol.gov/ilab/grants/OTLA_2015_09_10_FY2015_MPG.pdf.

http://www.dol.gov/ilab/grants/OTLA_2015_09_10_FY2015_MPG.pdf


number of workers who use legal services might be affected independent of the project’s direct intervention. An initial analysis indicates that the overall economic situation has not had this effect on the project. In fact, ENS staff were concerned that the country’s current economic slowdown may have the opposite effect. According to the latest International Monetary Fund (IMF) information, the country has maintained a strong policy framework even though the prices of oil (the country’s biggest export) have declined significantly.41 Economic growth slowed from 4.9 percent in 2013 to 3.1 percent in 2015. The rate of growth in 2016 is expected to continue to decline, but the medium-term economic outlook remains positive, thanks in part to the government’s infrastructure and development agenda. While there has been a slight upturn in unemployment during 2016, the annual unemployment rate during project implementation has hovered around 9 percent (9.1 percent in 2014 and 8.9 percent in 2015).

Exhibit 3.6: Status of External Factors

External Critical Factors Final Evaluation Findings Legal regulation The country’s legal framework remains the same. Economic performance The country’s economic performance remains the same. Cultural practices No significant changes in cultural practices have occurred.

3.1.3 Lessons Learned and Promising Practices

ENS’s effort to collect and document standardized and universal data at the CALs is an important asset for this type of initiative. Standardized data on workers’ background, types of labor rights violations, and legal actions undertaken is a first step in showing how the output activities can lead to desired outcomes. Additional work is needed to define and track realistic outcome indicators for the project. For example, CALs spend considerable time helping workers negotiate with employers; however, this work is not currently documented in a standardized way.

National labor union federations, key partners in this initiative, are paramount to the success of the project; however, the political will and inner workings of the union federations were not sufficiently considered in the initial project assumptions. Better articulation by ENS of the role that union federations are expected to play in the CAL project would largely benefit the sustainability of the project. Indicators to track work and partnerships with local, regional, and national unions is important for sustainability efforts.

Several stakeholders, including government representatives, ENS staff, and CAL leaders, agreed that the justification for the project is sound and relevant—workers generally do not know their labor rights and lack access to high quality legal assistance. The CALs are seen as a fitting initiative to begin to fill this gap.

ENS and the CALs have not yet defined target goals for the project’s main outcomes. Such indicators are important to measuring and monitoring the success of the project.

41 IMF Executive Board concludes 2016 Article IV consultation with Colombia. (May 2016). Press Release No. 16/199. https://www.imf.org/external/np/sec/pr/2016/pr16199.htm.

https://www.imf.org/external/np/sec/pr/2016/pr16199.htm


3.2 Relevance and Strategic Fit 3.2.1 Relevance within Country Context The project is aligned with the U.S. and Colombian governments’ objectives to incorporate high labor standards into their trade agreement. It directly addresses the Labor Action Plan’s commitment to protect internationally recognized labor rights42 through ENS’s research of emblematic cases. For instance, the yearly work plans include emblematic cases on the misuse of temporary service agencies and collective pacts. Throughout the project, ENS has gathered evidence that, even when workers present claims to public institutions like the Ministry of Labor, the claims are not been processed in a timely fashion—although a source of frustration, ENS is currently gathering this evidence in an “access to justice” report to give visibility to the problem. Further demonstrating the relevance of this project within the country context, the Colombian government recently approved Decree 583 to regulate general aspects of the Ministry of Labor’s administrative investigations for labor outsourcing. This decree has been controversial, with different labor organizations criticizing it for what they see as legalizing previously illegal outsourcing. During one of the project interviews, a government representative defended this decree as a positive development for workers’ rights. He maintained that it would help the CALs and similar initiatives to more clearly distinguish between which types of outsourcing violate workers’ rights and which are legal.

3.2.2 Targeting Vulnerable Economic Sectors and Groups While the Project Document identified priority sectors that the CALs were meant to target through mobile CALs, it did not establish concrete goals to measure the level of success in reaching these sectors. However, ENS and CAL staff maintain that the sectors targeted by the Action Plan43 (palm, sugar, mines, ports, and flowers) are a priority in their work. As part of its research on emblematic cases, ENS provides updates on the implementation of the Labor Action Plan, which includes a chapter dedicated to developments in the 5 priority sectors.44 For instance, ENS has undertaken “diagnostic investigations” in the CAL’s priority sectors, in which incidence of illegal outsourcing was thought to be particularly high. The main output is a mapping of the sector’s production chain and activities that tracks where workers should be hired directly by the employer and not through outsourcing contracts. ENS researchers also gather information on which companies are part of the sector, their regional location, and the number of workers hired directly and indirectly. An initial look at these reports indicates that they are well developed and provide substantial information about the sectors of interest.45

42 https://ustr.gov/trade-agreements/free-trade-agreements/colombia-fta. 43 Although the public sector is also a target for this project, it is not part of the five Action Plan sectors. 44 ENS. (2015). Report on the First Four Years of Implementation of the Labor Action Plan 2011-2015. 45 The evaluation team had access to reports on three sectors: palm, sugar, and mining. A fourth report on the flower and coffee sectors is being financed by another institution and has not yet been published.

https://ustr.gov/trade-agreements/free-trade-agreements/colombia-fta/final-text


The new CAL database has the ability to track the gender, ethnic background, displacement situation, socio-economic stratum,46 economic sector, and social organization of each of the workers who receive services from the CALs. Most of these variables are recorded at the time of worker intake and registration. Exhibit 3.7 shows the type of information that can be generated using data from the CAL database. Because a given user may have more than one legal case, the analysis shows the distribution of CAL actions by user characteristics and therefore also captures the underlying user frequency. From January to April 2016,47 the majority of actions that CALs undertook (64 percent) were for users belonging to the lowest socioeconomic strata (45 percent in stratum 1 and 19 percent in stratum 2). Important variations in this proportion exists among the CALs—the Bogotá and Bucaramanga CALs were below the overall average of 64 percent (56 and 61 percent, respectively) while the Medellín and Cartagena CALs were above (68 and 79 percent, respectively). In addition, the proportion of users belonging to stratum 1 in Cartagena was greater than in Bogotá and Bucaramanga. Across the four CALs, slightly less than half of CAL users were female (47 percent), but differences exist among the CALs. The Bucaramanga CAL served the lowest proportion of females (32 percent), while the Bogotá CAL assisted the highest proportion of females (52 percent).

46 Colombia’s socio-economic stratification is based on household characteristics and is categorized as follows: (1) low-low; (2) low; (3) middle-low; (4) middle; (5) middle-high; (6) high. Categories 1–3 are beneficiaries of government subsidies in municipal public services, while categories 5 and 6 pay an extra amount for these services. Category 4 pays the exact cost for the services received. http://www.dane.gov.co/files/geoestadistica/Preguntas_frecuentes_estratificacion.pdf. 47 CAL data were available only for the first 4 months of 2016. The reason is that the CALs transitioned to the new database system at the end of 2015, and data from previous years had not yet been transferred to the new system.

http://www.dane.gov.co/files/geoestadistica/Preguntas_frecuentes_estratificacion.pdf


Exhibit 3.7: Distribution of CAL Actions by Socioeconomic Strata, January–April 2016

Note: In the CAL database, actions include cases (psychosocial and legal assistance) and consultations (counsel and orientation). Disaggregating the number of CAL cases by economic sector can help show the degree to which the project has reached the project’s priority sectors (palm oil, sugar, mines, ports, flowers, and the public sector). Although the CALs did collect this kind of data in the previous databases, these data were not reported in the documentation provided to the evaluation team.48 The figures in Exhibit 3.8 are provided, therefore, only as an example of the type of data that the CALs are capable of generating from their new database. For example, CAL staff categorize cases into economic categories by the type of company or legal entity (employer) in which the violation occurred. However, as Exhibit 3.8 shows, a large percentage of cases (45 percent) in early 2016 did not provide specific information about the economic sector. The largest proportion of these cases (24 percent) was classified as “activities not specified,” and 21 percent of the cases had no information because these violations were committed by an individual employer and not a company (sector information is collected at the company level, not the individual level).49 Of the 18 economic sectors that are represented, “social services and health” and “transportation, storage and communications” had the highest concentrations of cases with adequate information, 11 percent and 7 percent, respectively. “Agriculture and livestock,” which incorporates three of the six priority sectors, represents less than 2 percent of the cases, and “mining and quarrying exploitation” represents less than 1 percent. Section 3.3.1 will show that this distribution is likely due to the fact that the rural sectors were meant to be reached by the mobile CALs, which varies widely by CAL.50

48 The evaluation team did not have access to the databases used by each CAL prior to 2016 to compile this information in the way presented in this section. 49 A potential limitation of only collecting data on cases that are associated with a company is that, in theory, rural workers who are employed by an individual employer would not be captured in this system. For this reason, CALs are possibly reaching more rural populations than is currently shown in the data. More research is needed to estimate the extent of this possibility. 50 Please note that the new database only provides overall sector information. It does not disaggregate the “agriculture” sector by sub-categories like palm oil, sugar or flowers.


Exhibit 3.8: Distribution of CAL Cases by Economic Sector, January–April 2016

Note: Consultations (counsel and orientation) were excluded from the analysis because the economic sector variable is not required at the time of registration for this type of service. Note 2: N/A is not applicable because these violations were committed by an individual employer and sector information is collected at the employer level. Since overall economic sector information was not reported in any of the TPRs either for total workers or for those reached by mobile CALs51, the evaluation team used a proxy to estimate the extent the project reached the priority sectors. If the number of workers reached by a mobile CAL during 2015 serves as a proxy for workers reached in the priority sectors during the same year, then approximately 11 percent of all CAL users in 2015 belonged to one of the priority sectors. 3.2.3 Lessons Learned and Promising Practices

The CAL directors are able to rely on the substantial body of research that ENS provides through its “diagnostic investigations” in the 5 Action Plan priority sectors. This information helps the CALs to better understand the labor rights context within these sectors and develop a strategy before reaching out to workers to offer legal

51 Starting in 2015, the TPRs included more information on mobile CALs, such as the date and location it took place, but not which priority sector.


services. ENS is able to provide this information because it has staff dedicated to conducting and disseminating this type of research.

Moving forward, the CAL database will help the CALs and project managers to better track and visualize the worker characteristics important to their work, such as gender, ethnic background, displacement situation, socio-economic strata, and economic sector. These data will only be as good as the effort that the interns make when registering CAL users. In addition, the technical assistance and video tutorials provided by ENS’s IT team is essential to the maintenance and usefulness of the database.

3.3 Project Progress and Effectiveness 3.3.1 Status in Achieving Target Goals Although the draft Project Document included a matrix outlining the PMP, the evaluators found a number of issues (Section 3.5.2) that made it difficult to use the PMP to measure progress. The work plans were the only available documents that included targets for most of the project activities (A through H). The work plans included one-time activities (i.e., completion of protocols, publication of the Workers’ Right Handbook, development of a new CAL database) and activities with periodic outputs and targets. To better understand the progress made in each of these activities, the evaluators created an Excel document similar to the one provided in ILAB’s Management Procedures and Guidelines (Annex II), which indicates the targets and actual numbers for each TPR period. Most of the information needed to create the table was dispersed among several documents. The team compiled the various targets found in the work plans and compared these against the information provided in the quarterly TPRs submitted to ILAB. When information was missing, the evaluation team reached out to ENS’s project coordinator to either supply the missing information or provide additional documentation (see Appendix C). The following sections summarize the progress for each activity. Activity A: Prepare a Workers’ Rights Handbook At the time of the Midterm Implementation Evaluation Report, the project had developed a version of the Workers’ Rights Handbook that, while rich in legal content, was not appropriate as a teaching tool for an audience without a legal background. ENS staff indicated that workers’ workshops on the topics presented in the handbook (e.g., labor formalization and illegal outsourcing) were being held despite this delay in finalizing a more suitable version. To address this issue, the project contracted a university in Medellín to modify the content for the target audience. The evaluation team has seen an online version of this new handbook, but, at the time of the final site visit, printed copies were not available. Starting April 2016, ENS printed 5,000 copies of the handbook and have distributed 400 copies in Medellín, 900 in Bogotá, 200 in Cartagena, and 400 in Bucaramanga.


At the time of the March 2016 site visit, ENS and CAL staff stated that a strategy was in place to disseminate and train workers in using the handbook. The strategy includes sharing the handbook in CALs’ waiting rooms and union offices, as well as distributing the handbook during the workers’ workshops. Exhibit 3.9 shows the number of Workers’ Rights Handbook trainings held and participants attending at the CALs between April and June 2016. To implement the workshops, some CAL directors have discussed collaborations with unions and NGOs to help with resources such as meals and transportation for workers and office space for the meetings, while the CALs would pay for their own staff costs and educational materials.

Exhibit 3.9: Workshops with the Worker’s Rights Handbook, April - June 2016

City # of Participants Training

Bogotá 44 Curso Básico Sindical Bogotá 180 Tips Asamblea Sindefonahorro Bogotá 12 Píldoras informativas usuarios CAL Bogotá Bogotá 8 Socialización Manual de derechos Colaboradores CAL Bogotá 20 Formación debido proceso y negociación colectiva Sintraosi Bogotá 120 Proceso de formación subdirectivas Nacionales de SNTT Medellín 16 Formación a usuarios Medellín 53 Curso Básico Sindical Bucaramanga 18 Formación Sintrainagro Bucaramanga 49 Formación Sintrapalmas Bucaramanga 52 Capacitación Organizaciones sindicales Bucaramanga 19 Formación sintraproaceites Cartagena 21 Formación a Sindicatos Cartagena 20 Reuniones focales sindicales Cartagena 10 Socialización Manual de derechos Colaboradores CAL Mobile CAL 30 Caravana Puerto López TOTAL 672

The handbook has six sections: (1) employers’ legal obligations to employees, (2) workplace harassment, (3) rights related to social security, (4) special protection status, and (5 and 6) due process in labor relationships. Each section begins with an illustrated story in which the characters discuss a typical labor situation involving the topic discussed in that section. The rest of the section explains the applicable laws in more detail, using bullet points wherever possible, and includes a brief Q & A segment. Since workers will be receiving and using the handbook for the first time, ENS and the CALS should keep track of workers’ impressions and level of comfort in using it.


Activity B: Build CAL infrastructure52 This activity comprises the establishment of new CALs in Cartagena and Bucaramanga, the strengthening of the CALs in Bogotá and Medellín, and the internal service protocols that all CALs are expected to adopt. Each of these activities had been completed by the time of the midterm report. Since the midterm evaluation, the Cartagena CAL moved to a new office space thanks to a partnership with a local union, helping them save money in office rent. In addition, as Activity E explains, ENS has finalized the rollout of the new CAL database. The main issue with the internal service protocols for the law student interns who provide the legal assistance was that the protocols were not readily available across all CALs, which remained true at the time of the final site visit. One strategy the Bogotá CAL uses to disseminate and remind the interns of the information in the protocols is to display posters with excerpts from the legal assistance protocol throughout the office. Activity C: Formulate and sign agreements with universities Each CAL director is in charge of formalizing agreements with university law schools so that their students can carry out legal internships at the CALs. Partnerships with law schools are vitally important to the project, because the CALs rely almost solely on law student interns to provide legal services and support to their users. At a minimum, the project work plan indicated that each director should each agreements with at least three universities to maintain an adequate number of interns at each CAL. All CALs, except Bucaramanga, have been able to sign agreements with at least three universities. As of December 2015, the Bogotá CAL had signed agreements with six universities for legal interns, but had been unable to secure one for the social work intern. All the other CALs were able to sign agreements by the date indicated in the first work plan (April 2014)53. The focus groups with students clarified that the agreements are not standard across universities due to differences in the requirements of the various law schools. Most law schools require their students to complete a professional practicum before graduating, and some even run their own legal clinics where students can complete the required credit hours. While some universities require that their students complete 2 hours of legal work per week, others, such as the Universidad Industrial de Santander in Bucaramanga, structure their courses so that the students are able to work full time for a semester to complete this requirement. Even though the Bucaramanga CAL only has agreements with two universities, this internship structure has allowed the CAL to sustain a stable cadre of well-trained interns over time, while also allowing the students to better embed themselves in the daily operations of the CAL. Another difficulty with the university agreements surfaced when the CALs looked for social work student interns with a psychosocial background. To approve a student’s internship, all

52 In the Midterm Implementation Evaluation Report, the evaluation team included information on the mobile CALs under this activity, even though it was not included in the original work plans, to facilitate reporting on progress. In this report, mobile CALs infrastructure is treated separately. 53 This was part of the first set of documents ENS shared with the evaluation team during 2014.


universities require evidence of a “suitable match” between the student’s specialization and the place of internship. If the university does not consider the site to be a suitable match for the area in which the student is specializing, it will not sign an agreement. Though the Botogá CAL has attempted to obtain a social work student intern, it has not been able to find a university with which to partner due to university requirements. This lack of a social work intern has been frustrating for CAL staff and current student interns. In contrast, Bucaramanga’s CAL director is a trained pedagogue as well as a lawyer; thus the university considered her a suitable mentor for the social work students. However, these partnerships have advantages and disadvantages. A clear benefit is that, once the university accepts the social work internship, it also provides the student with a professor/mentor who coordinates the internship. According to some CAL directors, these mentors have been significant assets for the psychosocial component of the CALs’ work because each mentor is a professional who helps manage the social work program at the CALs in conjunction with the student intern. The main disadvantage is that the CAL director is asked to act as a thesis coordinator for the social work students, which involves considerable additional work and meetings at the university. Activity D: Train CAL staff CAL directors, who are experienced lawyers specializing in labor rights, are in charge of organizing regular trainings for their students to ensure the delivery of high quality legal services to CAL users. The directors are responsible for determining students’ training needs, which they tailor according to the group’s knowledge level or weaknesses. In general, the two types of trainings are induction trainings, which occur when a student intern begins work at the CAL, and monthly trainings. As during the midterm evaluation, student interns expressed a great deal of satisfaction with the trainings as well as the professional experience they were gaining at the CALs. The induction trainings include trainings on how to use the CAL database, among other tools. To ensure students are recording worker information in the database, CAL staff check the physical sign-in sheet signed by workers when they first arrive against the information in the database. In addition, as part of the agreements with the universities, CAL directors include in their grading criteria the recording of worker information in the database. According to the ENS project coordinator, each CAL director decides how to conduct worker follow-up based on the CAL’s own internal dynamics. Although all CAL directors are expected to organize monthly trainings for student interns, the number of trainings provided varied across the CAL offices. The Bogotá CAL continues to conduct the smallest number of staff trainings, mainly due to the large number of law student interns and volunteers that the CAL coordinates. The amount of interns and volunteers has made scheduling difficult and requires the director to prioritize supervision over frequent trainings. On average, in 2015, the Bogotá CAL had 49 student interns and 20 volunteers, most of them available for only a few hours a week. That year, the Bogotá CAL organized 7 trainings, not counting induction trainings, compared to 15 trainings organized in Medellín (30 interns and 4 volunteers), 17 in Cartagena (3 interns and 5 volunteers), and over 20 in Bucaramanga (17 interns and 4 volunteers).


During focus groups with the law students,54 the evaluation team noted a difference across CALs in the level of knowledge and the ease with which students talked about topics other than their daily tasks. While most students focused only on their personal experiences working at the CAL, students in the Bucaramanga CAL, for example, stood out for their ability and willingness to discuss topics such as the strategy, importance, and mission of the CALs. This enthusiasm seemed to be a result of their training but also reflected the fact that these students are at the CAL for a longer period of time. During the entire period of project operation, the Bogotá CAL has trained 45 students, the Cartagena CAL 25 students, and the Bucaramanga CAL 36 students. In addition, 11 interns have trained at ENS in the rights advocacy unit.55 In addition, each CAL was required to organize a special labor rights seminar (diplomado) in partnership with a local university. To date, all the CALs except the one in Medellín have organized such a seminar, reaching their target audiences of unions, local organizations, lawyers, and others interested in labor rights. The CAL in Bucaramanga has been able to maintain its alliance with the university and organized a second seminar for the first quarter of 2016. Activity E: Collect information on CAL cases An integral part of the CAL initiative is the daily collection of worker information and cases. The new CALs have been able to gather worker information since first opening to the public because they borrowed the database system already in use at the CAL in Medellín. Over time, the project managers recognized that a major disadvantage of decentralizing the database system was that this model made data collection for the mobile CALs and other efforts in the field less efficient. Interns and volunteers would have to wait until the mobile CALs returned to the office to input the user information into the system. Since the initial rollout of the database to the new CALs, ENS and the CALs have devised a plan to integrate the system into a web-based database. This allows users to connect to the database via the Internet so that information can be shared across CAL offices. Rollout of the new integrated database started in August 2015 and continued until the end of 2015. ENS’s IT team helped coordinate this effort and prepared instructional videos to explain how to enter information in the database for the transition period. At the time of the site visit, the web-based database was operational at each of the four CALs. The evaluation team and ENS’s IT team conducted an initial analysis of the database, which revealed that all the data generated prior to the rollout was unreliable with the current search mechanism. For example, it was not possible to generate a report that would give the correct number of CAL users by individual CALs for 2014 or 2015. However, ENS’s IT team stores the information from the old databases in their servers. The team heard several complaints about system interruption while interns were providing assistance. Periodically, the system failed to save data, and data had to be reentered. Some of

54 Student focus groups were conducted in all the CALs except for Cartagena. 55 The evaluation team did not receive data on the Medellín CAL. These numbers already account for students who have stayed for more than one period and only counts them once.


the issues raised could have probably been resolved quickly by someone with more knowledge of the new database. Students at the Medellín CAL seemed to be the only ones who did not report any major problems, which is likely because the Medellín CAL director is the main architect behind the CAL database, and both he and the CAL assistant showed great knowledge of the old and new CAL databases. As with the original database, ENS’s IT team is responsible for the maintenance of the new database, although their plan is still to turn over the database management to the unions. Activity F: Launch an outreach campaign The main purpose of this activity is to raise workers’ awareness of their rights and publicize the services the CALs offer through several media outlets (radio, flyers, union networks), as well as through the Ministry of Labor. The sub-activities described in the work plans include the design and implementation of a communications strategy and the hiring of a communications intern. While the TPRs submitted to ILAB the type and number of dissemination efforts during the quarter, reporting on CAL user referrals has been inconsistent since CALs have not always reported this information in the quarterly TPRs. As Exhibit 3.10 shows, during most of 2015,56 the main sources of referrals were the Ministry of Labor (45 percent), other CAL users (28 percent), publicity (10 percent), and other, unspecified sources (10 percent). The CUT-CTC-ENS referrals made up less than 6 percent of the referrals for the period. However, looking at these percentages by CAL is important because the Bogotá CAL skews these percentages. During the first three quarters of 2015, the main sources of referrals for the CALs in Bogotá and Medellín were the Ministry of Labor (76 and 38 percent, respectively), CAL users (14 and 38 percent), and other sources (5 and 18 percent). The Medellín CAL shows a more even distribution of sources, while the Bogotá CAL is heavily reliant on Ministry referrals. Referrals from unions and through publicity were more important for the smaller CALs. The top three sources for Bucaramanga were CAL users (58 percent), other sources (14 percent), and unions (15 percent). In Cartagena, the top three sources were publicity (44 percent), unions (21 percent), and CAL users (20 percent).

56 This information was compiled for the first three quarters of 2015. Referral information by CAL was not included in the TPR for the last quarter of 2015.


Exhibit 3.10: Sources of Referrals by CAL, 2015

In the focus groups conducted during the final site visit, CAL users in all four cities commented that most people are still largely unaware of the CALs’ services. Even though they were eager to recommend the CAL to other people, they nevertheless thought more publicity was needed to reach more potential users. The majority of CAL users who belonged to a union also recognized that most people were not aware of the CAL, but these users said they were active in distributing flyers and reaching out to members and non-members. In one city, a few of the union members who had recently used the CAL services were surprised to learn that the CALs had already been operating for 2 years. Despite this clear request for greater publicity from users, the CALs would not likely be able to cope with a substantial increase in users given their current physical locations and still maintain the same level of services. The CAL offices are generally at maximum capacity in terms of their daily workload.

Other activities, such as the reports on strategic cases—another dissemination strategy identified by ENS—are not currently captured in the TPRs, but the evaluation team has seen a sample of these cases. Some of them have already been uploaded to the new CAL website (www.calcolombia.co). While a new website was not originally contemplated in the project design, the maintenance of a web presence separate from ENS and the union federations is an



important development that will help consolidate the project’s message. ENS has developed the project’s outreach strategy to help disseminate the CAL’s work, share practical information and news on labor rights, and provide a communication platform to unions. To do this, they are increasing their online presence through a new website (www.calcolombia.co), various social media platforms (Facebook, Twitter, YouTube), and electronic newsletters. Leveraging resources from other funding sources, ENS hopes to provide unions with a more comprehensive strategy that combines legal assistance, trainings, and communications. Activity G: Provide legal assistance services Each CAL’s work plan includes a target number of users to be assisted by the CAL. Except for Cartagena in 2014, all CALs have met or surpassed their target numbers for each year. During 2015, the Bogotá CAL continued to attract the highest volume of users (9,157), followed by Medellín (3,618), Bucaramanga (1,872), and Cartagena (1,652). Exhibit 3.11 compares the number of users assisted during 2014 and 2015 for each CAL. After meeting their 2014 targets, Bucaramanga, Bogotá and Medellín increased their number of users in 2015 by 47, 42, and 19 percent, respectively. Even the Cartagena CAL was able to recover from underperformance in its first year and surpassed the 2015 target goal of 1,000 users by 65 percent.

Exhibit 3.11: Number of CAL Users, 2014-2016

* Number is for the first quarter of 2016 only. Exhibit 3.12 shows the number of CAL users reported in the quarterly TPRs. The peak of demand for most CALs seems to occur either in the second quarter (Medellín and Bucaramanga) or the third quarter of the year (Bogotá and Cartagena). This pattern is likely due to the fact that the CALs are usually closed for several weeks in December and January,57 when demand for certain services, such as counsel or orientation, could be just as high as or even higher than at other times. The breakdown between male and female users also shows the particularity of the

57 The CALs were closed from December 5, 2015 to January 18, 2016.



Bucaramanga CAL and how it consistently serves more males than females. In 2015, only 30 percent of CAL users in Bucaramanga were women.

Exhibit 3.12: Number of CAL Users by Quarter, 2014-2016

Activity H: Initiate administrative or legal actions to protect labor rights The work plans include a target number for legal and administrative actions taken per year by each CAL. Compared to the first work plan in 2014, the subsequent work plans have substantially reduced the target numbers of actions. For example, the target number of administrative actions per year for the Medellín CAL decreased from 400 in 2014 to 100 in 2015, and the number of legal actions from 1,400 to 500. This decrease is likely due to the reclassification of administrative and legal actions during the past year as compared to the 2014 TPRs. Before the reclassification, certain frequently used actions like right to petition were often treated as administrative actions, greatly skewing the numbers. Exhibit 3.13 shows a list of actions reported using the CAL database and the current classification. Based on this reclassification, none of the CALs met the 2014 targets for administrative and legal actions. During 2015, only Cartagena struggled to meet the new targets. Medellín and Bucaramanga met the new targets, while Bogotá surpassed them considerably. The 2016 work plan was not adjusted to reflect the new ability of the CALs to meet the targeted numbers. Appendix C includes the progress each CAL made in the three main categories against the yearly targets.


Exhibit 3.13: Number of Legal Actions Taken across CALs, 2014–2015

2014 2015 Administrative Actions Solicitudes administrativas [Administrative investigation request] 24 209 Defensa ante el Ministerio [Defense before the Ministry] 0 5 Impugnación [Objection] 33 365 Queja por acoso laboral [Workplace harassment complaint] 31 100 Querella ante MinTrabajo* [Criminal complaint] 0 0 Judicial/Legal Actions Acción de tutela [Writs of Protection of Fundamental Rights (FR)] 241 2,248 Cumplimiento fallo de tutela [Compliance to writs of protection of FR] 0 215 Impugnación fallo de tutela [Deposition to writs of protection of FR] 0 171 Incidente de desacato [Contempt] 11 41 Insistencia ante la Corte [Persistence before the court] 7 17 Other Actions Asesoría [Counseling] 4,068 4,929 Impugnación [Depositions] 14 21 Concepto Legal [Legal concepts] 189 269 Memoriales [Memorials] 182 211 Carta de renuncia motivada [Involuntary resignation letters] 115 100 Orientación [Orientation] 33 212 Otro [Other] 66 7 Reclamación de pago [Payment claim] 4,273 3,719 Trámite ante el empleador [Proceedings before the employer] 26 30 Derecho de petición [Right to petition] 1,372 2,932 Liquidaciones [Severance payments] 2,417 2,625 Total 13,102 18,426

* Although querellas are included in the database as one of the legal action options, these have not been reported in the TPRs under this matrix. Instead, they have been reported as part of the emblematic cases. The recent work plans include new sub-indicators for the following administrative and legal actions: percentage of administrative actions in formalization per year and percentage of freedom of association legal actions per year. However, these new indicators are only partially reported in the TPRs, in the sub-section on actions taken during mobile CALs, but not for the actions taken for the entire period, which would include both the CAL offices and the mobile CALs (TPR section output 2.1). During the final site visit, the evaluation team found no evidence that CAL directors are tracking this new sub-set of indicators; thus the mechanisms being used to monitor their progress are unclear. Activities I, J, and K: Strategic or emblematic cases During the midterm evaluation, ENS leaders defended the importance of taking on collective and emblematic cases as a critical part of the CALs’ work. The strategy behind these activities is to “establish in-depth investigations into systematic and structural patterns of labor violations” and,


with the help of ENS staff, present legal cases in courts that will set a legal precedent in defense of labor rights.58 While ENS still believes that these cases can help clarify legal norms and close loopholes in business practices, over the last year, the staff have also viewed such cases as part of a renewed strategy to engage with unions. The current focus is to offer a more comprehensive package, which includes helping the unions to develop their legal strategy, supporting their legal efforts, and assisting them with communication and training. To date, the project has helped various unions submit over 150 querellas to the Ministry of Labor as part of an overall strategy to bring attention to the issues of the illegality of collective pacts and to pressure the ministry for labor formalization agreements. At the time of the final site visit, the overwhelming majority of the querellas had not even been provided with an order to initiate investigation (auto de apertura). In an interview, the Ministry of Labor representative was unable to provide any information on the status of these querellas. The representative explained that he knew of the CALs’ existence but had not been able to accept an invitation to visit them and was unaware that ENS was the project’s implementer. ENS leaders have learned from experience over the past 2 years that labor inspectors or prosecutors do not necessarily abide favorable rulings from the high courts. ENS leaders believe that the effectiveness of emblematic cases depends on the political will of Colombian institutions to follow the rule of law and to ensure compliance at all levels with the rulings of the high courts. Mobile CALs Although mobile CALs were included in the Project Document, project managers never established any targets for the number of workers to be reached. Initially, it was expected that the CALs would conduct one mobile CAL per month to reach the Action Plan priority sectors, but this target was not achieved in 2014. Starting in 2015, however, more mobile CALs were organized to make up for the previous year’s shortcoming, although their frequency varied by quarter. The ENS project coordinator included more information in the TPRs on the number of workers reached and the legal actions undertaken during these efforts. As Exhibit 3.14 shows, Bucaramanga has organized the most mobile CALs (32), reached the second highest number of workers (534), and undertaken the highest number of legal actions (118) and the second largest number of total actions (552). These numbers mean that, on average, Bucaramanga has reached 17 workers per mobile CAL, which is similar to Cartagena’s record, with 18 workers per mobile CAL. The mobile CALs at Medellín and Bogotá have been more efficient at reaching workers, averaging 42 and 31 workers per mobile CAL, respectively. Medellín has reached the highest number of workers (763) and undertaken the highest number of total action (839). In 2015, approximately 11 percent of workers were assisted through a mobile CAL, which represents 10 percent of total actions undertaken in the year. Comparing the percentage of workers reached through a mobile CAL within each city shows that Bucaramanga has been able

58 ENS. Project Document, p. 6.


to assist almost a third of its 2015 users through a mobile CAL, followed by Medellín (21 percent), Cartagena (13 percent), and Bogotá (3 percent).

Exhibit 3.14: Mobile CALs by Quarter, 2015

No. Mobile CALs

No. Workers

Administrative Actions

Judicial/ Legal Actions

Other Actions

Total Actions

Bogotá 10 308 23 118 171 312 2015-I 0 – – – – – 2015-II 2 43 15 1 27 43 2015-III 7 220 4 90 126 220 2015-IV 1 45 4 27 18 49 Bucaramanga 32 534 26 155 371 552 2015-I 7 222 11 11 177 199 2015-II 7 128 6 21 108 135 2015-III 13 162 6 120 53 179 2015-IV 5 22 3 3 33 39 Cartagena 12 220 2 2 131 135 2015-I 0 – – – – – 2015–II 7 135 2 2 131 135 2015–III 1 17 – – – – 2015–IV 4 68 – – – – Medellín 18 763 10 43 786 839 2015-I 3 129 5 29 133 167 2015-II 6 299 3 6 384 393 2015-III 7 283 1 6 269 276 2015-IV 2 52 1 2 0 3 Total 72 1,825 61 318 1,459 1,838

Note: Dashes indicate that the information was not available for that time period. The variation across cities in the number of workers reached through mobile CALs is not surprising given that the CAL offices are centrally located in urban areas and open to the general public for most of the year. More definitive research would be needed to understand how the mobile CALs can be made more effective, but two points should be noted:

Rural workers are harder to reach than urban ones due to not only proximity (transportation) but various factors, such as lack of privacy in approaching a mobile CAL and challenges in developing trust with a new service provider of any kind. The variation in reaching rural workers via mobile CALs appears to be largely due to: 1) the relationships between a given CAL and key stakeholders, particularly local unions, and 2) the level of union organization of workers within a priority sector. Reaching workers in rural areas has proven more effective in cases in which these workers are already organized, as unions can provide much needed logistical support to the mobile CALs.


3.3.2 Follow-up of CAL Workers An important mechanism to monitor progress and assess the immediate impact of legal assistance is to follow up with workers on legal actions. After workers have obtained legal assistance at the CALs, the law student interns are supposed to follow up with workers who had an “actionable legal service” (Exhibit 1.2) by calling them. The intern asks the worker the results of the action taken and, if necessary, instructs the worker to return to the CAL for further assistance. In the focus group discussions with students, the team learned that this process is not standard across CALs. For instance, interns at the Bogotá CAL only follow up with workers who have filed tutelas, because of the volume of workers with actionable items. In the Bucaramanga CAL, interns have a similar type of ranking for follow-up based on the urgency of the action; tutelas are also given priority. Reaching workers is a major challenge across all CALs at this time because many workers do not answer their phone. One CAL director estimated that the success rate for obtaining a response from workers has been 30 to 40 percent in the last 2 years. The results of this follow-up on administrative and legal actions were also reported in the TPRs during 2015.59 While some inconsistencies remained, the data presented were clearer and the majority of issues mentioned in the midterm report had been addressed. As Exhibit 3.15 shows, 52 percent of the administrative and tutela actions undertaken by the CALs were verified, meaning that an intern was able to contact the worker to inquire about the action. Exhibit 3.16 provides more detail on the results obtained for these actions (i.e., granted, not granted, action to be presented, no information, not filed, pending inquiry after ruling). When assessing relative success across CALs, however, considering the differences in follow-up rates (e.g., 29 percent in Medellín vs. 67 percent in Bucaramanga), the fact that students in Bogotá only follow up on tutelas due to workload, and that staff size varies by CAL (e.g., average of 3 student interns in Cartagena vs. 49 in Bogotá) is important.60 In addition, ENS staff believe that courts from different cities handle labor rights cases differently: courts in Bucaramanga are perceived as more receptive to collective labor rights cases, as opposed to Bogotá, which is perceived as receptive only to individual labor rights cases.61 Thus, the data provide insight at the individual CAL level (minimum outcomes) and perhaps some preliminary insight at the overall project level (when combining all CAL data) but should not be used to compare CALs to each other. In general, the higher the percentage of verified actions, the more reliable the outcome as an indicator of a specific CAL’s performance. 59 The evaluation team was unable to analyze the results in the 2014 TPRs due to considerable inconsistencies in the reporting across and within CALs. 60 Exhibit 3.1 presents staff size and other characteristics for each CAL. 61 Individual labor rights refers to individual employment terms like minimum wage, health and benefits, discrimination, etc.; whereas collective labor rights cases refer to actions like collective bargaining or other union-related activities.


Key findings from Exhibits 3.15 and 3.16 are as follows:

Of the 52 percent of actions for which follow-up information was obtained across CALs, approximately 42 percent of the verified actions were granted or considered favorable.

Although Bucaramanga is one of the smaller CALs, it reported the highest number of actions of all the CALs and has the highest percentage of verified actions. In addition, Bucaramanga has a significant percentage of actions with favorable results (43 percent were granted), although slightly more cases did not have any information (44 percent), meaning that the person contacted could not provide more information at the time.

Although the Medellín CAL has the largest percentage of favorable actions (69 percent were granted), it also has the smallest percentage of verified actions (29 percent).

The Bogotá CAL has been able to verify 60 percent of its total actions but only obtained favorable results in 23 percent of them.

The Cartagena CAL has significantly fewer actions than the other CALs, accounting for less than 3 percent of all actions across CALs. This CAL was able to verify 55 percent of its actions and obtained favorable results in 45 percent of them.

Exhibit 3.15: Verified Actions via Follow-up Calls by CAL, 2015

CAL Total Actions Follow-up Percentage Follow-up

Medellín 1,257 369 29% Bogotá 964 579 60% Cartagena 97 53 55% Bucaramanga 1,302 876 67% Total 3,620 1,877 52%

Exhibit 3.16: Results of Follow-up by CAL, 2015*

Follow-up Outcomes by CAL**

Administrative Actions % Legal/Judicial

Actions % Total Actions %

Medellín 16 100% 353 100% 369 100% Granted*** 11 69% 243 69% 254 69% Not granted 3 19% 71 20% 74 20% No information 0 0% 29 8% 29 8% Not filed 2 13% 10 3% 12 3%

Bogotá 16 100% 563 100% 579 100% Granted 0 0% 134 24% 134 23% Not granted 0 0% 138 25% 138 24% To be presented/ Not filed 0 0% 107 19% 107 18% No information 16 100% 130 23% 146 25% Pending inquiry after ruling 0% 54 10% 54 9%

Bucaramanga 630 100% 246 100% 876 100%


Follow-up Outcomes by CAL**

Administrative Actions % Legal/Judicial

Actions % Total Actions %

Granted 276 44% 100 41% 376 43% Not granted 47 7% 45 18% 92 11% No information 290 46% 96 39% 386 44% Pending inquiry after Ruling 17 3% 5 2% 22 3%

Cartagena 7 100% 46 100% 53 100% Granted 1 14% 23 50% 24 45% Not granted 3 43% 19 41% 22 42% Lack of interest 2 29% 4 9% 6 11% No information 1 14% 0 0% 1 2%

Total 669 1,208 1,877 *These results are based on the percentage of workers (52 percent overall) that CALs were able to contact for follow-up. **The evaluation team advises caution in comparing across CALs because the rate of follow-up varies significantly as shown in Exhibit 3.15. *** “Granted” means the result was favorable to the worker. 3.3.3 Lessons Learned and Promising Practices

Although an initial version of the Worker’s Rights Handbook was completed by the time of the midterm evaluation, the technical language in this version was not suitable for workers. Project staff contracted a university to resolve this issue and adapt the handbook to its intended audience. The CALs have a strategy in place to disseminate the handbook through workshops and other channels. Once again, similar to the assistance provided with mobile CALs, the collaboration with other partners such as unions and NGOs is important in the organization of these workshops.

Ensuring that the internal protocols for legal assistance are readily available to all law student interns is important. A good practice observed at the Bogotá CAL was the display of posters throughout the office (which were also clearly visible to workers) reminding students of certain aspects of the assistance protocol.

The main goal of the CALs’ original outreach strategy was to publicize CAL services to increase demand. However, based on the significant volume of workers that the CALs currently assist, ENS has adjusted its outreach strategy. Mainly through email and social media (such as Facebook), the CALs disseminate their work, share practical information and news on labor rights, and provide a communication platform to unions.

University agreements to obtain student interns are key to the CALs’ work, but can also represent substantial additional work for the CAL directors. Differences in university internship requirements mean that some CALs benefit from students who can intern full time for a semester, while others have to coordinate multiple schedules. These schedule differences can also make organizing regular trainings more difficult.


Although emblematic cases may be important for establishing favorable legal precedents for workers, the effectiveness of these cases also depends on the decisions and effectiveness of other institutions such as the Ministry of Labor and the legal system.

Currently, more work is needed to develop a detailed standard methodology to follow up on workers’ cases, since follow-up is an important mechanism to monitor progress and assess the immediate impact of legal assistance. The standard methodology should include clear definitions and example scenarios to guide students in how to proceed with recording accurate and complete data.

3.4 Efficiency in the Use of Project Resources 3.4.1 Adequacy of Time, Budget, and Human Resources The estimated 4-year project budget (initial 3 years plus the extension year) is divided into seven main categories: (1) personnel costs, (2) equipment, (3) office expenses, (4) activities related to Outcome 1, (5) activities related to Outcome 2, (6) monitoring and evaluation, and (7) other (contingency and indirect costs). This estimate allocates 52 percent of resources to personnel costs, 21 percent to activities related to Outcomes 1 and 2, and 27 percent to the other categories (Exhibit 3.17).62 Personnel costs are the most important category given that the main service provided is personal legal assistance. In the grant extension, a greater proportion of the funds was allocated to outcome activities and a reduced proportion to personnel costs. The proportion of the budget for Outcome 2 activities increased from 11 percent in the main budget to 25 percent in the extension budget, while the proportion for personnel costs decreased from 56 percent to 40 percent.

62 Before the 1 year extension, these allocations were 56, 18, and 26 percent, respectively.


Exhibit 3.17: Allocation of Resources by Category

As Exhibit 3.18 shows, approximately 44 percent of the personnel costs are allocated to the CALs, 28 percent to ENS researchers and other experts, and 23 percent to ENS management personnel. There were three new personnel line items in the extension budget: a mobile CAL director, a social communicator, and an expert on union organizing. These new items represent about 5 percent of the total (4-year) personnel budget and about 22 percent of the extension budget. The other ENS experts were assigned the largest share, with 29 percent of the extension funds. Only three of the four CALs received additional allocations for personnel costs (11 percent each) since the Medellín CAL receives funding from other sources. Although the TPRs did not include information regarding the justification for the new line items, the project coordinator explained that it is part of ENS’s efforts to make the mobile CALs more efficient. Instead of mobilizing a small team from the CAL offices to other municipalities, the new mobile CAL director will be in charge of laying the groundwork and undertaking the legal actions personally. The CAL offices are expected to support the mobile CAL director when necessary. However, none of the other CAL directors provided substantial information about this new collaboration. One of the directors stated that the CALs were still expected to conduct their mobile CALs as they have in the past.


Exhibit 3.18: Allocation of Resources for Personnel by Year

Disbursements during the first year were slower than anticipated. ENS underestimated the time and effort necessary to open offices in the new cities even though they had had previous experience with a similar initiative in Bogotá. The most significant challenges in establishing the new offices were obtaining office space and setting up agreements with the union federations and universities. The budgetary resources that had been allocated to the first year but not spent were added to subsequent years of the project. In addition, interviews with some project staff indicate that ENS management was unaware of the time and effort necessary to complete the documentation required by the donor, such as the Project Document, before the funds could be disbursed. Some ENS staff said that, although they have had extensive experience with international donors, this was their first time collaborating with an external, independent evaluation. While conducting the midterm implementation evaluation, the evaluation team reported the limitations of the Project Document (e.g., indicators lacked detail, no outcome indicators). During preparation of the present report, ILAB informed the evaluation team that ENS would not be required to produce a new version of this document with an updated PMP with target indicators. Despite these delays, ENS was prepared to start implementing some of the project activities, drawing on its own matching funds. ENS spent approximately 20 percent of these resources in


the first quarter to hire a project coordinator and other management support staff.63 During the site visit, several staff members emphasized the need to ensure collaboration with other local actors before starting any of the project activities. For example, some staff believed that they could have obtained computers and other equipment from donations and even office space from some of the local unions. They recommended that more time and emphasis be given to these activities before opening CAL offices to the public. According to the financial information provided in the TPRs, the project exceeded its original budget by 2 percent in December 2015. ENS staff explained that the budget was surpassed for two main reasons: 1) higher number of student interns hired to meet CAL demand and 2) payment made to a local survey firm for the impact evaluation.64 The evaluation team did not have access to this level of detail in the budget, so it could not independently confirm this assertion.65 Even with the money allocated in the extension, the project is projected to go over budget in two categories: equipment and activities related to Outcome 1 (Exhibit 3.19).

Exhibit 3.19: Total Budget and Percentage of Total Budget Spent

Regarding human resources, all CALs have been able to maintain an adequate number of student interns to offer legal assistance. However, the Bucaramanga CAL has continued to grow over the past year, and the structure of just one lawyer and one administrative assistant may not be sustainable. Further, the Bogotá CAL has been unable to resolve its difficulty in securing a social work intern. At the CALs where social work interns were available, law student interns spoke highly of their work and its importance when offering legal assistance. Social work interns are

63 First Quarter TPR 2013. 64 Email shared with evaluation team on July 1st, 2016. 65 The evaluation team was unable to carry out a proper analysis of cost efficiency due to lack of an output-based budget, which was required in the terms of the grant agreement with USDOL.

Description Total Budget (4 Yrs) Accrued Expenses to Dec 2015Office Expenses 221,885 150,768 68%Equipment 33,700 43,573 129%Other 243,466 127,266 52%Direct Personnel 1,086,359 736,828 68%Monitoring and Evaluation 66,590 47,575 71%Outcome 1 137,500 160,965 117%Outcome 2 310,500 258,794 83%Total 2,100,000 1,525,769 73%

Percent of Total Budget Spent

Description Original Budget (3 Yrs) Accrued Expenses to Dec 2015Office Expenses 157885 150,768 95%Equipment 22500 43,573 194%Other 172567 127,266 74%Direct Personnel 842459 736,828 87%Monitoring and Evaluation 44590 47,575 107%Outcome 1 101000 160,965 159%Outcome 2 159000 258,794 163%Total 1,500,001 1,525,769 102%

Percent of Total Budget Spent


able to see CAL clients when needed, but also offer trainings to the law students at least once every quarter on topics such as “psychological first aid,” “psychosocial awareness,” and “social and gender inclusion.” Only a small minority of participants in the focus groups of CAL clients reported meeting the social worker intern, but those who did spoke favorably about the attention they received. 3.4.2 Lessons Learned and Promising Practices

Including the project extension, approximately half of the project’s total budget is dedicated to CAL operations, 19 percent to research activities led by ENS, and 12 percent to project management.

There are three new personnel line items (a mobile CAL Director, a social communicator, and an expert on union organizing), which seem to align with ENS’s strategy to focus its sustainability efforts on engaging and supporting local unions. However, during interviews, none of the CAL directors mentioned the new mobile CAL director. The CAL directors did not seem to think the new position would affect or enhance their own mobile CAL efforts.

ENS and CAL staff acknowledged that opening CALs in new cities takes considerable time and effort, particularly to develop the strong partnerships needed to sustain project activities. This initial groundwork should be taken into account when planning new offices.

The CALs that have a social worker intern are better equipped to offer assistance to workers who have experienced harassment or mistreatment at their workplace. It is also important for the students offering legal assistance to have someone available to talk to as needed.

3.5 Effectiveness of Management Arrangements 3.5.1 Support Structure and Project Communication As designed, each CAL is intended to operate with a CAL director and an administrative assistant to provide support. This structure has worked well for all the CALs except the Bogotá CAL, primarily due to its large number of student interns, which places a considerable administrative and logistical burden on CAL staff. Bogotá is the only CAL that sees users by appointment only, and, during periods of high demand, scheduling an appointment can take up to 2 weeks. The other CALs receive mainly walk-ins but also set up appointments with workers if appropriate. Two CALs have started assisting workers via Skype calls. During the midterm evaluation, some CAL directors mentioned that communication with ENS could be improved. When the evaluation team followed up on this issue during the final site visit,


communication with ENS no longer seemed to be an issue for the present CAL directors.66 In the case of Bogotá, this shift may be due to the fact that ENS’s project coordinator temporarily took over as the Bogotá CAL director after the previous director resigned, which may have strengthened communication between the CAL and ENS. During the final site visit, the CAL directors demonstrated diverse strengths in managing the CALs’ various activities (e.g., day-to-day operations, legal assistance, and strategic alliances). The CALs have had the flexibility to adjust to different circumstances, and this diversity of strengths does not seem to have affected the quality of services. As an example, although the Bogotá interim CAL director is not a lawyer by training, she has been able to leverage the expertise of CAL lawyers and ENS staff to keep the CAL running smoothly. Standard procedures for offering legal services exist, but they were not equally available in the CALs. During the focus groups, most students were able to communicate what was expected of them in great detail, but were not able to say where to access this information at the office. As Section 3.3 mentions, making internal documents available to all staff is considered good practice, although the evaluators do not believe this practice has impacted the quality of service provided by the law student interns. 3.5.2 Centralized vs. Decentralized Model Although ENS provides institutional support to all the CALs, the directors have had a considerable amount of independence to implement the initiative in their own cities. CAL directors have had the freedom to organize their student trainings, plan mobile CALs, and engage union members and other stakeholders. This approach has worked well in some cities, such as Medellín, which has strong support from ENS, and Bucaramanga, where the director has established networks within the labor movement. In the case of Bogotá, where union organizations are not as strong as in other cities, the CAL has been less successful in securing partnerships to guarantee its sustainability even though demand for CAL services has been the greatest in Bogotá. In the case of Cartagena, having three different project directors and one interim director in less than 4 years has impacted its overall effectiveness, although securing office space in collaboration with a union is a promising step. 3.5.3 Performance Monitoring System As described in the Midterm Implementation Evaluation Report, the PMP does not provide targets against which to measure project success or lack of success (e.g., what percentage of favorable outcomes is required for the project to be considered successful). Even though these requirements are described in ILAB’s Management Procedures Guidelines, the ample information provided is heavily focused on child labor projects, and parsing out what is relevant to these kinds of projects could be confusing. In fact, the CALs have not tracked and measured

66 The evaluation team was unable to follow up directly with the people who mentioned this issue during the midterm report, because they were no longer working at the CAL.


all of the outcome indicators described in the Project Document related to knowledge acquisition either for workers assisted by the CAL or for CAL personnel (e.g., improvement in knowledge following training activities). This does not seem to be the main tool used by ENS and the CALs to monitor project performance. Instead, ENS staff use a different document called a “management matrix” (matriz de gestión) for management oversight, and each CAL maintains its own Excel files with indicators that it finds relevant. According to the ENS project manager, CAL directors undergo a yearly performance review. The main indicators used in this evaluation are contained in the management matrix, which groups indicators under 15 different topics, including administration, training, networking, outreach, and follow-up.67 The evaluation team had access to a sample of these evaluations for three of the four CALs for the 2014 performance review.68 Each indicator has a proposed goal and a verification method. The project manager evaluates whether the goal has been completed, partially completed, or not completed, adding observations where appropriate. There are three indicators under quality: case supervision and support, final case revision, and random case quality checks. The CAL director is responsible for these quality indicators, and the ENS project manager conducts field visits to verify and assess the quality of the CAL’s work related to these indicators. According to the verification report, only the Medellín CAL director had a clear idea of what was expected in the random quality checks. In the case of the Bogotá CAL, this procedure was not followed because the project manager reviewed all cases without exception. Moreover, this tool does not include indicators for the number of workers and the number of cases filed or resolved as described in Project Document. As an outcome of the 2014 performance review, the project manager decided not to renew the contract of the Cartagena CAL director, because the target number of users during the first year was not met.69 However, a yearly review may not be the most appropriate tool to keep track of progress and potential issues or to use as a basis for taking timely corrective actions. 3.5.4 Lessons Learned and Promising Practices

Circumstances in two CALs have caused ENS’s project manager to temporarily assume the role of CAL director for these CALs. Becoming more involved in daily operations at the CAL improved the communication between ENS and these CALs, and this improvement can hopefully be maintained as the new CAL directors settle into their roles.

Information on project progress is dispersed among several documents, such as work plans and TPRs, which makes monitoring and comparing performance across the CALs difficult. The CAL directors maintain their own Excel files with information they find

67 The 15 areas are administration, training, networking, psychosocial, outreach, data base, follow-up/verification process, quality, university agreements, sustainability, ENS activities, collective reparations, priority sectors, and tools. 68 Information for Medellín was missing from the file received. 69 2015 TPRs, first and second quarters.


useful, but these different tools should be standardized to make them more useful to ENS management.

Frequent performance monitoring and feedback is important to help mitigate and address potential challenges and take timely corrective action. ENS’s reliance only on annual performance reviews has meant that CAL staff are not receiving real-time feedback about their performance or strategies they can employ to better serve and support workers.

3.6 Impact Orientation and Sustainability 3.6.1 Transfer of Project Management to the CUT and CTC Unions Collaborating with the CUT and CTC union federations continues to be a central part of the sustainability strategy for the ENS leadership. As part of the organizational structure, regional representatives of the CUT and CTC form part of each CAL’s management committee and are meant to provide the political support and strategic guidance for the CAL. The ultimate goal of the project design is that these organizations will take over the management and financial support of the CAL project. However, the evaluation team found little evidence that these organizations are prepared to assume these responsibilities in the near future. Taking on full management responsibilities for the project is likely the most challenging aspect for the union federations. The decentralized nature of the CAL project means that it is important to maintain a cohesive strategy, particularly as the project seeks to expand to other regions. Although the federations work together on certain initiatives, they are still separate entities with different priorities and capabilities at the regional level as well as the national level. During interviews with the regional representatives, some had diverging views regarding the project’s strengths and objectives. Some union representatives explained that they had initially been skeptical about the merits and potential of the initiative but, as they have become more involved in the CALs’ activities, have slowly become convinced of its importance. They seemed to be deeply suspicious of unfamiliar collaborations, for example, with government institutions, and skeptical of schemes they thought “too good to be true.” In addition, the highly technical nature of the project requires a level of legal expertise that unions may not be in a position to provide. Project managers at ENS and the union federation representatives were also uncertain about the federations’ ability to take financial ownership of the project. When discussing the likelihood of taking this on, almost all the interviewees referenced the same statistic—less than 5 percent of Colombian workers belong to a union—as evidence that the federations do not have the financial resources to support the initiative. However, some representatives seemed more willing to consider other sources of financing to sustain their individual CALs after the end of the current financing. Section 3.6.3 discusses these examples.


3.6.2 Strategies to Engage Unions and Other Stakeholders During the second year of operation, the CALs placed an even greater emphasis on their collaboration with individual unions. CAL directors have reached agreements with different unions, and student interns are now sent to work at union offices to offer CAL services at least once or twice a week depending on the CAL. Union members were pleased with this arrangement and have started to respond accordingly. The Bucaramanga CAL is currently functioning at an office provided by a local union and is in conversations to move to a different location belonging to a major union. The Cartagena CAL reached an agreement to move to a local union’s headquarters in the historic district. The evaluation team has seen a change in the union federation representatives’ initial attitude toward partnering with government representatives to further the CALs’ objectives. Whereas during the midterm evaluation some representatives were deeply suspicious of potential collaborations, in interviews during the final visit these individuals seemed more open to the idea. At the time of the final site visit, the Bucaramanga CAL director was actively pursuing a strategy to engage mayors from three nearby municipalities to include the CAL initiative in their development plans. The preliminary agreement at the time of the site visit in March 2016 entailed the municipalities contributing office space and resources for daily operations and the union federations directing the overall political guidelines of the project to safeguard workers’ interests. Union representatives from one of the CALs referred to the data gathered by the CALs as one of the most valuable assets of the project. One of the representatives had several years of experience offering free legal services through the union, but little had been documented, which he thought was one of the greatest weaknesses of that effort. Another representative in a different CAL spoke positively of the “statistical reports” provided by the CAL director to update them on the CAL’s progress. 3.6.3 Fee for Services and Other Pricing Options During the final site visit interviews and focus groups, the evaluation team probed stakeholders about the viability of financing the CALs through their own resources. As expected, responses were highly varied, but some interesting patterns emerged. The majority of CAL users, both men and women, were enthusiastic about the possibility of contributing to the CAL to have a CAL lawyer see their individual cases through the legal process once the student interns were not legally able to do so. In sharp contrast, the student interns were not only skeptical about users’ ability to pay for services but in some cases felt that charging a fee would go against their values and the CAL’s mission. The responses from the CAL directors and union representatives were more mixed. While the majority of respondents were uncertain about users’ ability to pay for the CAL services, only a few seemed to have given some thought to other options. For example, in Bucaramanga, small businesses exist to help teachers submit various documents to the Secretary of Education. The


union representatives were considering a proposal to have the CAL provide this service for a small fee to subsidize their core legal services. In contrast, a CAL director described a first-hand experience of what happened when a similar free initiative started charging users for services: the project saw a 70 percent reduction in the number of users. A small minority of interviewees believed that users could afford to pay for certain services at a subsidized price, but stressed that it would not be enough to make the CALs fully self-sustainable. Rather, fees would have to be part of an overall strategy to mobilize various resources. 3.6.4 Lessons Learned and Promising Practices

Although the union federations are an important partner and may offer operational support, the CALs are also seeking individual unions to sustain various project activities. In some cases, these unions have contributed infrastructure and logistical support for the mobile CALs.

An important aspect of the role of the CAL directors is the continuous development of potential partners. Partners may not be possible in every circumstance, but reaching out to local governments, as in the case of the Bucaramanga CAL, is a promising practice for the sustainability of the initiative.

In an effort to mobilize resources for the CALs, some stakeholders have started to consider other options for self-financing. More research is needed to determine the viability of this approach and the form it could take.


4. CONCLUSIONS AND RECOMMENDATIONS This section presents the evaluation team’s conclusions about the project’s overall implementation and progress by area of analysis based on the final evaluation findings. It also provides a list of recommendations to ENS as it looks to sustain the project and to ILAB as it considers similar initiatives. Overall, the evaluation team believes this is a valuable project that has contributed to the immediate goal of assisting workers in their ability to defend their labor rights (Outcome 1) and assisting workers in making more frequent and effective use of the administrative and legal mechanisms to protect their labor rights (Outcome 2). Implemented by a non-governmental organization that specializes in workers’ labor rights, the project was designed to assist both independent workers and unionized workers by focusing on three main mechanisms: providing basic legal assistance to workers, documenting and monitoring CAL trends and cases, and using these data to pursue “emblematic cases” (i.e., strategic cases that have the potential to establish new legal precedents or impact a large group of workers). Although the project undertakes significant efforts to collect information in these areas, it did not set measureable targets to evaluate the project’s overall success in reaching its goals and outcomes, thus limiting the extent to which the evaluation team can make specific conclusions in this report. Both Outcomes, however, are evaluated more rigorously in IMPAQ’s companion impact evaluation report. Despite this limitation, the evaluation team would like to highlight some noteworthy observations in the project’s efforts to reach workers, particularly those in the priority sectors (i.e., special sectors of interest and need), all of which are located in rural areas. Targeted priority sectors were assigned to each of the CAL cities based on their proximity, so each CAL has both an urban and a rural target population. While the urban CAL offices serve all workers who seek services, each office also coordinates periodic mobile CALs to specifically target the harder-to-reach rural workers in the priority sectors. Despite the mobile CALs’ continued efforts, the project appears to have been more effective at reaching urban populations than rural populations, which is an unsurprising observation, given that the CAL offices are centrally located in urban areas and open to the general public for most of the year. The level of workers reached in rural areas, however, varies by city. More definitive research would be needed to understand how the mobile CALs can be made more effective, but two points should be noted:

Rural workers are harder to reach than urban ones not only due to proximity (transportation) but to various factors, such as lack of privacy in approaching a mobile CAL and challenges in developing trust with a new service provider of any kind.

The variations in reaching rural workers via mobile CALs appears to be largely due to: 1) the relationships between a given CAL and key stakeholders, particularly local unions and 2) the level of union organization of workers within a priority sector. Reaching workers in rural areas has proven more effective in cases in which these workers are already organized, as unions can provide much needed logistical support to the mobile CALs.


For these reasons--and considering that union workers in general may be more likely to contribute to the sustainability of the project in the long-term (e.g., logistical support and help with following up on CAL client cases)--during the design phase of the project, more consideration should be given to the mobile CAL component of the strategy and to union (or other stakeholder) involvement so as to maximize the project’s effects on priority rural sectors. Each CAL has unique strengths and challenges. Exhibit 3.1 in Section 3 presents a summary of key characteristics across CALs. Because each CAL has distinct characteristics and circumstances, one CAL’s performance or “success” should be determined not by measuring it against another but according to its own merits. 4.1 Conclusions by Area of Analysis 4.1.1 Validity of the Project Design

The project design was rooted in evidence regarding the legal needs and obstacles facing workers. ENS appropriately identified union federations (CUT and CTC) as key project partners; however, ENS did not sufficiently consider the political will and inner workings of union federations when designing the original sustainability strategy that transferred management and financing of the project to the federations. This has caused partnership building and sustainability efforts with the federations to stall. As a result, ENS and the CALs have turned to other national and local unions to demonstrate the need for the CALs and secure resources. However, ENS and the CALs have not developed indicators to reflect their work with other national and local unions. Established indicators and targets could help ENS and the CALs better understand the full spectrum of work that they do and the effort required for such work. This project was built on the premise that offering legal services to individual workers at the CALs would broaden and expand the knowledge and expertise of ENS and the unions. An increase in knowledge and expertise was expected to help develop a legal strategy that would improve the legal context for workers in Colombia by pursuing “emblematic cases”. According to the project design, both of these strategies (i.e., CAL legal services and emblematic cases) were necessary to help workers “make more frequent and more effective use of the administrative and legal mechanisms to protect their labor rights” (Outcome 2). In particular, emblematic cases, such as the ones presented to the Ministry of Labor, would serve as a concentrated effort to demonstrate how organized workers can and do file claims at the Ministry, increasing the effectiveness of this mechanism to protect labor rights. However, while ENS has publicized labor rights through media sources when filing these emblematic cases, this strategy, as it stands now, does not seem to be sufficient to achieve this outcome as the cases presented to the Ministry of Labor remain unresolved. Establishing new CALs exposed a tension between the two main mechanisms designed to reach the project’s goals. On-the-ground CAL staff and clients believe that assisting individual workers at the CALs provides quick results and is a unique and important service for which there is much


demand. However, ENS believes that CALs are not the main mechanism that will bring the greatest benefit to the majority of workers and that pursuing emblematic cases could bring about more impactful changes over time. Unfortunately, the advancement of emblematic cases has been slow due to what ENS perceives as insufficient effort from the Ministry of Labor to investigate these cases. The slower-than-expected progress of emblematic cases is a source of frustration and has affected the progression of the project’s goal to improve the overall labor rights justice system for workers. Yet, as an ILO representative explained during the midterm evaluation, the emblematic cases that the CALs have presented at the Ministry of Labor have started to give visibility to the broader issue of labor investigations. This visibility is important because all major stakeholders as well as external experts and informants identified the need to assist workers in filing collective claims as a strength of the project. Also, ENS can more easily and effectively assess the progress of emblematic cases, as opposed to individual cases for which monitoring outcomes is resource intensive. Gathering evidence of the slow progress of these investigations could help the project bring attention to the performance of public institutions and therefore make headway on Outcome 2. ENS is already drafting an “access to justice” report for this purpose and intends to use this information to start a dialogue with the newly-appointed Minister of Labor and find a resolution to these cases. 4.1.2 Relevance and Strategic Fit

The CAL project is aligned with the objectives of the U.S. and Colombian governments to incorporate high labor standards into their trade agreement. The project directly addresses the commitment in the Action Plan to protect internationally recognized labor rights through ENS’s research on emblematic cases. In addition to the CAL offices in four cities, mobile CALs were created to reach workers in the priority sectors identified in the Action Plan (i.e., palm oil, sugar, mines, ports, and flowers); however, only a small proportion of all workers assisted at the CALs in 2015 (approximately 11 percent) were assisted through a mobile CAL, indicating that the proportion of CAL users who belong to priority sectors has been relatively low. 4.1.3 Project Progress and Effectiveness

As noted during the midterm evaluation, the performance monitoring plan (PMP) lacks detail on process and outcome indicators and is not a suitable tool to measure progress. This finding is still true for the final evaluation. While the CALs and ENS report data for most project activities, this information is dispersed across many different documents rather than being reported in a consistent, standardized way. Despite these limitations, the evaluation team observed substantial progress in two key activities: (1) the rollout of the CAL database and (2) the publication of the Workers’ Rights Handbook. Roll-out of the new CAL database started in August 2015 and was completed by the end of 2015. The Workers’ Rights Handbook has been improved and now uses simpler language and depictions


that are more suitable for workers. A strategy to disseminate and promote the handbook has been designed and will be implemented this year. The project’s outreach campaign, another key activity, is no longer suitable for its original purpose. The initial outreach strategy, to increase demand for CAL services, is no longer feasible. Most CALs do not have the physical capacity to assist a larger number of workers given their current office sizes. Instead, ENS has revised the outreach strategy to help disseminate the CALs’ work, share practical information and news on labor rights, and provide a communication platform to unions. To do this, the CALs are increasing their online presence through a new website (www.calcolombia.co), various social media platforms (Facebook, Twitter, YouTube), and electronic newsletters. Leveraging resources from other funding sources, ENS hopes to provide unions with a more comprehensive strategy that combines legal assistance, trainings, and communications. The CALs rely significantly on law student interns to provide services to workers. Students’ knowledge of the CALs’ legal assistance protocols varies within and across the CALs. Variation also exists across the CALs in recording information in the CAL database, particularly data related to user follow-up. Without consistent follow-up data, the CALs are unable to comprehensively monitor and track the outcome of workers’ cases. ENS leaders have learned from experience over the past 2 years that labor inspectors or prosecutors do not necessarily abide by favorable rulings from the high courts. ENS leaders believe that the effectiveness of emblematic cases depends on the political will of Colombian institutions to follow the rule of law and ensure compliance at all levels with the rulings of the high courts. 4.1.4 Efficiency in the Use of Project Resources

Overall, ENS and the CALs have used project resources efficiently. However, three main categories seem to have been under-budgeted in the original 3-year project:

Equipment, like office furniture and computers

Activities related to completion of Outcome 1: Workers’ improved knowledge of and ability to defend their labor rights

Activities related to completion of Outcome 2: Workers’ use of the administrative and legal mechanisms to protect their labor rights

ENS compensated for this budget shortage in the extension year. About one-third of the additional funds received in the extension year were allocated to these categories. 4.1.5 Effectiveness of Management Arrangements CAL leadership demonstrated diverse strengths in managing the CALs’ various activities (e.g., day-to-day operations, legal assistance, and strategic alliances). This diversity of strengths does not



appear to have affected the quality of services but has had a direct effect on progress of the CALs’ sustainability efforts. ENS´s decentralized management model has given the CALs the freedom to develop their own student training, mobile CAL coverage, and outreach efforts. This approach has worked best when the CAL director has either strong institutional support from ENS (Medellín) or from highly organized unions (Bucaramanga). 4.1.6 Impact Orientation and Sustainability Partnership building and collaboration with the national federations, a cornerstone of the initial project strategy, has been slower than anticipated. Some evidence exists to support an incremental change in attitude about partnering with government institutions to further the CALs’ objectives. For example, Bucaramanga is currently pursuing a strategy to engage mayors from three municipalities to include the CAL initiative in their development plans. Further, the CALs have focused their attention on more established unions as an option to sustain the project post-USDOL funding. For example, the Cartagena CAL office moved to a union-owned building, and students in Bucaramanga and Bogotá provide legal assistance at union offices at least once a week. The CALs are considering other partners and mechanisms to sustain themselves post-ILAB funding, including payment for services. Given the socio-economic background of the overwhelming majority of CAL users, it is unlikely that a standard fee-for-services approach would be feasible to sustain services for this population. However, other alternatives to this model could be tested in certain CALs. For example, some users who are already familiar with the CAL and have demonstrated a willingness to pay for additional CAL services could be offered these extra services for a subsidized fee. As another option, fees could be charged on a sliding scale to help cover some of the CAL costs. 4.2 Key Recommendations for ENS The following recommendations are based on the evaluation team’s findings and conclusions. 4.2.1 CAL Database and Workers’ Rights Handbook

a. Follow up on usage of the new CAL database, particularly during the initial months after rollout. Facilitating tutorials for users is a good first step to train users in the new database. This guidance could be strengthened with Skype calls with users to identify and respond to their issues that they are facing.

b. Develop a CAL database oversight protocol to ensure that all students are adequately and consistently recording all of the necessary user and case information. The current video tutorials are a first step in this process, but the oversight protocol should describe the responsibilities of each staff member, explain the database fields that students find the most challenging, and include a standard methodology to follow up on workers’ cases. In


addition, CALs should continue to strongly emphasize the importance of recording accurate data in the CAL database and the value of user follow-up.

c. Collect workers’ and CAL directors’ feedback during the implementation of Worker’s Rights Handbook workshops. Since this will be the first time workers will receive and use the handbook, keeping track of their impressions and level of comfort in using it will be useful so that the tool can be further refined to meet their needs. In addition, CAL directors and ENS staff should keep track of relevant legal changes that may require revisions in the handbook.

4.2.2 Outreach and Communications Strategy

Continue adapting the outreach strategy to the changing circumstances and needs of the project. The initial outreach strategy, to increase the demand for CAL services, is no longer feasible. Instead, the CALs’ outreach strategy should be targeted to decision-makers who may provide sustainability to the initiative. For example, municipalities may contribute free office space (through comodato contracts) for CAL functioning. ENS could also partner with other labor rights-focused organizations to advocate with their political representatives and pass legislation beneficial to workers.

4.2.3 Internal Protocols

Ensure that protocols and training materials are distributed and readily available to all students and volunteers. As the project matures, a strong foundational structure is important in case of expansion, high student turnover, or management issues.

4.2.4 Sustainability

a. Continue strategic partnerships with established and new unions to embed the CALs’ objectives into the labor rights strategy of local unions.

b. Reach out to stakeholders who have alternative financing ideas and evaluate the most promising designs. Actively testing different approaches is important, particularly when the approaches involve charging users, to avoid an undue burden on the workers.

c. Develop a plan to leverage data gathered from the CAL database to engage other institutions. For example, data on results could be used to demonstrate the efficacy of the project to potential donors.

d. Actively engage government institutions to demonstrate the benefits of the CAL initiative. The example of the Bucaramanga CAL engaging mayors in neighboring communities demonstrates the potential of this approach.

4.3 Final Considerations for ILAB Based on the findings of the implementation evaluation, the team identified key factors that ILAB should consider when implementing similar workers’ rights projects.


a. Start-up time: ENS underestimated the level of time and effort necessary to open offices in new cities. The most significant challenges were obtaining office space and setting up agreements with the labor federations and universities. Future projects should take into account potential start-up delays and challenges in project work plans and budgets and consider alternative options to maintain momentum during project design.

b. Reporting and evaluation requirements: Though experienced with international donors, ENS was largely unaware of the level of effort required for the regular reporting and evaluation activities associated with implementation of the CAL project and the project evaluation. Further, the project coordinator was unaware of ILAB’s Management Procedures Guidelines (MPG). In the future, ILAB should consider emphasizing technical assistance to its grantees to ensure that grantees are aware of and understand their responsibilities and the time and resources necessary for reporting and evaluation requirements. Even though these requirements are described in the MPG, the information provided is heavily focused on child labor projects, and parsing out what is relevant to these kinds of projects could be difficult. Further, ILAB should consider requiring at least one full-time person dedicated to monitoring and evaluation, data collection and reporting activities across the initiative to ensure consistency of data and reporting and to allow on-the-ground staff more time for service delivery.

c. Clearly defined and documented indicators: This project’s PMP lacked detail on process and outcome indicators, though such indicators are essential to monitoring progress and assessing the impact of a project. Future projects should finalize the outcome indicators before project implementation or refine them early in the implementation phase if necessary. A greater effort should be made at the project design stage to refine the kinds of outcome indicators the project can track without putting undue burden on implementers’ systems and resources. Further, indicators should be realistic and reflect the full spectrum of project activities.

d. Assessment of the full political landscape: The project design should include a full analysis and understanding of the political landscape of all major stakeholders in the project. For example, since the union federations were thought to be a critical partner in the implementation of the project, project assumptions should have also included how the labor unions’ political environment could affect the success of the project.

e. Strong relationships with partner organizations: As key beneficiaries of the initiative, all partners should be actively engaged (including local and national unions) and should guarantee support and commitment to the initiative even before the CALs begin operating.

f. Active institutional support: The CALs benefit from the institutional and management support provided by ENS, such as support for the CAL database and research on priority sectors. This support has allowed the CAL directors to focus more intently on sustainability efforts and the day-to-day operations of the CAL offices and mobile CALs.


g. Engagement of other stakeholders: Assessing all potential partnerships is important to sustainability. For example, the CALs should actively seek out opportunities to engage other potential partners who could contribute to the sustainability of certain activities, such as national and local governments. Ideally, the identification and engagement of other stakeholders should be carried out even before the CALs begin operation and the nature of these partnerships should be detailed in the project’s sustainability plan.

h. Stable workforce: Since law student interns are an integral part of the initiative, the CALs should initiate university agreements that allow students to intern for longer periods of time (at least 20 hours a week). Greater stability of the CAL workforce would allow for more consistency and better integration of the students into the CALs’ operations.

i. Detailed and effectively executed transition strategy: Implementing a well-defined and realistic sustainability strategy from the start of the initiative and engaging stakeholders throughout the project is necessary to effectively establish and achieve milestones.

j. Target priority sectors: While the Project Document identified priority sectors that the CALs were meant to target through mobile CALs, it did not establish concrete goals to measure the level of success in reaching these sectors. Given that there is demand for CAL services among non-priority sectors, more discussion is needed within ILAB and with the CALs to reach an agreement on what is expected of this approach and to establish realistic targets. Recognizing the need for flexibility when identifying priority sectors and setting CAL-specific targets is important, since factors inherent to the sector (i.e., level of union organization) and to the CAL itself (e.g., other priorities, CAL characteristics) can influence how effective CALs are in reaching them.

IMPAQ International, LLC Appendix A-1 ILAB Colombia Final Implementation Evaluation

APPENDIX A: SCHEDULE OF EVALUATION ACTIVITIES

Evaluation Activity Site Date Document review Desk review October–December 2015 Observation of CAL Bogotá 7 March 2016 Interview with CUT and CTC representatives Bogotá 7 March 2016 Focus group with CAL clients (female and male clients separately)

Bogotá 7 March 2016

Interview with CAL Director Bogotá 8 March 2016 Interview with Solidarity Center Director Bogotá 8 March 2016 Focus group with CAL students Bogotá 8 March 2016 Observation of CAL Bucaramanga 9 March 2016 Interview with CUT and CTC representatives Bucaramanga 9 March 2016 Focus group with CAL students Bucaramanga 9 March 2016 Interview with CAL Director Bucaramanga 10 March 2016 Focus group with CAL clients (female and male clients separately)

Bucaramanga 10 March 2016

Interview with CAL student Cartagena 11 March 2016 Focus group with CAL clients (female and male clients separately)

Cartagena 11 March 2016

Interview with CAL Director Cartagena 12 March 2016 Interview with CUT and CTC representatives Cartagena 12 March 2016 Focus group with CAL students Medellín 14 March 2016 Focus group with CAL clients (female and male clients separately)

Medellín 14 March 2016

Interviews with ENS Academic Director, Subdirector, Expert on Project Planning, Expert on Research and Project Coordinator

Medellín 15 March 2016

Interview with CAL Director Medellín 16 March 2016 Interview with CUT and CTC representatives Medellín 16 March 2016

IMPAQ International, LLC Appendix B-1 ILAB Colombia Final Implementation Evaluation

APPENDIX B: INTERVIEW AND FOCUS GROUP GUIDES


Interview Guide: CAL Director

Informed Consent

Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of the form to keep for his or her records. Outline with Timings

Time in each section (in minutes) Topic Elapse time at the end of

section (in minutes) 3 Introduction 3 2 Interviewee Background 5

15 Progress and Effectiveness of Program Strategy 20

5 Validity of the Project Design 25 3 Relevance and Strategic Fit 28

6 Efficiency in the Use of Project Resources 34

6 Effectiveness of Management Arrangements 40

15 Impact Orientation and Sustainability 55 3 Recommendations and Lessons Learned 58 2 Conclusion 60

Introduction (3 minutes)

The purpose of this interview is to discuss your ongoing activities related the Escuela Nacional Sindical (ENS) with regards to the Centros de Atencion Laboral (CALs), including challenges, successful strategies, perceived outcomes, and sustainability plans. This interview will last approximately 60 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. This interview will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.


Interviewee Background (2 minutes)

1. What is your title?

2. How long have you been with the CAL?

3. What are your main responsibilities in your position?

4. For new directors: Please describe any training received for this new position. (When was it offered, how long did it take, content, materials, etc.)

Progress and Effectiveness of Project Strategy (15 minutes)

5. What are the CAL’s main objectives and activities for the remainder of the year? Are the project’s planned activities proceeding on schedule? Please explain.

a. What has helped stay on schedule and what has made it difficult?

6. How does the CAL measure and monitor project effectiveness?

a. Are there sufficient project indicators to measure outputs and outcomes?

b. Has the process for measuring effectiveness changed over time?

c. How does the CAL use the collected data?

d. What are the strengths and weaknesses of the CAL’s use of data?

7. What outputs and/or outcomes has the project achieved so far?

a. Are expected results occurring as planned?

b. What has helped achieve desired outcomes and what has made it difficult?

8. Have the project’s activities reached the target worker population in the sectors of interest as planned, both in terms of type and number of workers?

a. What have been the obstacles and facilitators?

9. What outreach efforts has the CAL implemented?

a. What have been the objectives of the outreach efforts? Have these strategies changed over time?

b. Do you think these efforts have achieved their desired objectives?

c. How can these efforts be improved? Validity of the Project Design (5 minutes)

10. From your perspective, what external factors have influenced the CAL’s success? These may include labor law regulation, economic policy for job creation, civil displacement by internal conflict, and the efficiency of the justice system. How have these external factors impacted the CAL’s performance?

11. Are the initial assumptions of the CAL project still valid? Have they changed over time? For example, assumptions like: workers are interested in improving their working conditions


and trust ENS; the country maintains relative political stability and the government demonstrates political will to improve labor rights.

12. What are the strengths of the project’s design? What are its weaknesses? Relevance and Strategic Fit (3 minutes)

13. Have there been changes in labor regulations or laws that affect how it provides legal services? If yes, how well has the project reacted and accommodated to those changes? Please explain.

14. In the last year, have workers’ legal needs changed over time in a way that has affected the project?

a. If so, how has the project responded to changing needs? Efficiency in the Use of Project Resources (6 minutes)

15. To your knowledge, to what extent are the planned activities being implemented according to the budget?

a. What obstacles in allocating budget resources have arose? How were they overcome and at what cost?

16. Has the financial management model, whereby ENS manages and allocates financial resources, been effective for CAL performance at the local level? Why or why not?

a. What improvements could be made to the financial management model to increase effectiveness at the local level?

17. How well do you think the project has used its financial and human resources?

a. What improvements could be made?

18. How does the CAL manage resource constraints (e.g., computers, printers, space), if any?

Effectiveness of Management Arrangements (6 minutes)

19. How are policies and other information communicated from ENS to your CAL? Among the other CALs?

a. Is this effective? How can it be improved?

20. What is the role of the Steering Committee and what is your relationship with the other members? How does this impact the success of the project?

21. What challenges, if any, has the CAL faced in operating with a primarily student (and therefore transient) workforce?

a. How has the CAL overcome these challenges?

22. What challenges, if any, has the CAL faced in recruiting and obtaining high quality student lawyers to provide legal services for workers?

a. How has the CAL overcome these challenges?


23. What indicators are used to evaluate practices and policies within the CAL offices?

a. Have there been any obstacles to improving practices? How have they been resolved?

24. Over the last year, has the CAL developed standard operating procedures and tools? If so, please explain.

a. Are the tools and procedures consistent across all CALs? Impact Orientation and Sustainability (15 minutes)

25. Key sustainability goals of the CAL project are to strengthen workers’ knowledge of their rights and institutions’ ability to support workers’ rights. What strategies have been used to accomplish this?

26. Are the project results, achievements, and benefits likely to be durable over time? Please explain.

27. How has the CAL engaged other stakeholders (e.g., workers, unions, government, and local organizations) to sustain the CAL services after the project funding ends?

a. Do you anticipate that referrals to the CAL will be sustainable beyond grant funding?

b. Can the project approach or results be replicated or scaled up by national partners or other actors? Is this likely to happen? What would support their replication and scaling up?

28. Moving forward, what strategies will the CAL pursue in order to ensure long-term sustainability?

29. Have you observed any unintended or unexpected positive or negative effects of the project? If so, what’s been the impact of these effects? How have the CALs or ENS reacted to these effects? For example, have the positive effects been integrated in to the project strategy or has the strategy been adjusted to minimize negative effects?

Recommendations and Lessons Learned (3 minutes)

30. What lessons have been learned and how can they be taken into account for future projects?

31. Knowing what you know now, is there anything you would do differently in order to have a more successful outcome?

Conclusion (2 minutes)

32. Is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today?


Interview Guide: Steering Committee Member

Informed Consent



section (in minutes) 3 Introduction 3 2 Interviewee Background 5 5 Validity of the Project Design 10 2 Relevance and Strategic Fit 12


6 Efficiency in the Use of Project Resources 28




The purpose of this interview is to discuss the ongoing activities related to the Escuela Nacional Sindical Centros de Atencion Laboral (ENS CALs), including challenges, successful strategies, perceived outcomes, and sustainability plans. This interview will last approximately 60 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. This interview will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.




2. How long have you been with your current organization?

3. What are your main responsibilities in your position, particularly related to the CAL project?

a. Have your responsibilities changed or evolved over the course of the CAL project? Validity of the Project Design (5 minutes)

4. Do you think the planning and organization of the CALs was well-planned and realistic in terms of its objectives, desired outcomes, targets, and timeframe? Why or why not?


6. Are the initial assumptions of the CAL project still valid? Have they changed over time? For example, assumptions like: workers are interested in improving their working conditions and trust ENS; the country maintains relative political stability and the government demonstrates political will to improve labor rights.


8. Have there been changes in labor regulations or laws that may have affected the CALs? If yes, from you knowledge, how has the project reacted to and accommodated those changes?

9. Have workers’ legal needs changed over time in a way that has affected the project?

a. If so, how has the project responded to changing needs? Progress and Effectiveness of Project Strategy (10 minutes)

10. Are the project’s planned activities proceeding on schedule? Please explain.


11. In the remaining year, what are the main outcomes this project hopes to achieve?

a. Are expected results occurring as planned?

b. What have been the obstacles and facilitators?

12. How do the CALs measure and monitor project effectiveness?

a. Are there sufficient project indicators to measure outputs and outcomes?


c. How do the CALs use the collected data? How could it be used better?


13. Have the project’s activities reached the target worker population in the sectors of interest as planned, both in terms of type and number of workers?

a. What have been the obstacles and facilitators? Efficiency in the Use of Project Resources (6 minutes)

14. To what extent are the planned activities being implemented according to the budget?

a. What obstacles in allocating budget resources have arose? How were they overcome and at what cost?

15. Approximately, what percentage of the project funds have been used for each activity? For example, what percentage of the funds has been allocated for CAL offices and what percentage has been allocated for mobile efforts?

16. How well do you think the project has used its financial and human resources?

a. What improvements could be made?


17. What kind of support does the steering committee provide to the CALs?

a. Has this relationship been effective? How can it be improved?

18. What challenges, if any, have the CALs faced in operating with a primarily student (and therefore transient) workforce?

a. How have the CALs overcome these challenges?

19. What challenges, if any, have the CALs faced in recruiting and obtaining high quality student lawyers to provide legal services for workers?

a. How have the CALs overcome these challenges?

20. How do the CALs manage resource constraints (e.g., computers, printers, space), if any?

21. What indicators are used to evaluate practices and policies within the CAL offices?

a. Have there been any obstacles to improving practices? How have they been resolved?

Impact Orientation and Sustainability (15 minutes)

22. In what ways, if any, will the CALs’ activities be sustained beyond grant funding?

a. If sustaining activities, what resources or investments will be necessary to sustain them? Where will those resources/investments come from?

b. If not sustaining activities, what makes the project unsustainable?

28. Are there financial commitments from other sources to ensure the continuation of project?

a. Which elements (activities and products) of the project are self-sufficient and which rely on external financial support for their continued operation?


29. How has the CAL engaged other stakeholders (e.g., workers, unions, government, and local organizations) to sustain the CAL services after the project funding ends?

a. Do you anticipate that referrals to the CAL will be sustainable beyond grant funding?


31. Have you observed any unintended or unexpected positive or negative effects of the project? If so, what’s been the impact of these effects? How have the CALs or ENS reacted to these effects? For example, have the positive effects been integrated in to the project strategy or has the strategy been adjusted to minimize negative effects?


32. What lessons have been learned and how they can be taken into account for future projects?





Interview Guide: ENS Project Expert Informed Consent



section (in minutes) 3 Introduction 3 3 Interviewee Background 6 5 Validity of the Project Design 11

12 Relevance and Strategic Fit 23




The purpose of this interview is to discuss the ongoing activities related to the Escuela Nacional Sindical Centros de Atencion Laboral (ENS CALs), including challenges, successful strategies, perceived outcomes, and sustainability plans. This interview will last approximately 60 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. This interview will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.




2. How long have you been with ENS?

3. What are your main responsibilities in your position?

4. Describe your involvement with the CAL project, if any.

Validity of the Project Design (5 minutes)

5. Do you think the planning and organization of the CALs was well-planned and realistic in terms of its objectives, desired outcomes, targets, and timeframe? Why or why not?

6. From your perspective, what critical factors have influenced the CALs’ success? These may include labor law regulation, economic policy for job creation, civil displacement by internal conflict, and the efficiency of the justice system. How have these external factors impacted the CAL’s performance?


8. Have there been changes in labor regulations or laws that affect how the CALs provide legal services? If yes, how well has the project reacted to and accommodated those changes? Please explain.

9. In your opinion, how has the CAL project affected the landscape of labor rights advocacy in Colombia?

a. How can the project improve to better complement or advance labor rights advocacy in the country?

10. Do you think the project is effectively assisting or supporting workers who experience indirect illegal hiring? Why or why not?

a. How can the project improve to better address this issue?

11. Do you think the project is effectively engaging and serving workers who have experienced violence? Why or why not?



12. Do you think the project’s activities have reached workers in sectors of interest, including ports, sugar cane, mining, flowers, and African palm? Why or why not?

a. How can the project improve to better serve these worker populations?

13. If expert on emblematic cases: Last year you mentioned a new approach to engage the Ministry of Labor that relied heavily on “tutelas por vía de hecho”.


a. What have been the developments of using that approach? Did you obtain the expected results?

b. Could you explain a bit more how the tutelas originate and were subsequently filed? Was it from cases of workers across all CALs or just from the Medellin CAL?

14. If monitoring and evaluation expert: How does the CAL project monitor and evaluation progress?

a. What are the main outputs and outcomes the project has achieved so far? What does it hope to achieve in the remaining of the year?


c. How do the CALs use the collected data?

d. What are some of the strengths and weaknesses of the CAL’s use of data?

15. What are the strengths of the project’s progress in terms of targeting and implementation? What are its weaknesses?


16. From your perspective, which CAL activities and processes will be sustainable beyond grant funding and which will not be sustainable?

17. In your opinion, how effective has ENS and the CALs been at engaging stakeholders (e.g., workers, unions, government, and local organizations) to sustain the CAL services after the project funding ends?

a. How can ENS and the CALs better engage and coordinate with labor rights stakeholders to sustain services?

18. Moving forward, what strategies should the CALs pursue in order to ensure long-term sustainability?

19. If expert on emblematic cases (director area defensa): Last year you mentioned that you would like to restore the “political bridge” between the government and workers. Have there been any developments in the political dialogue to sustain CAL activities?







Interview Guide: Local Expert Informed Consent








The purpose of this interview is to discuss the Escuela Nacional Sindical Centros de Atencion Laboral (ENS CALs), particularly your opinion about the project’s relevance to labor rights in Colombia, challenges, successful strategies, perceived outcomes, and sustainability plans. This interview will last approximately 60 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. This interview will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.



1. What is your title and place of employment?

2. Briefly describe your involvement with labor rights in Colombia.

3. Describe your involvement with the CAL project, if any.

Validity of the Project Design (10 minutes)

4. From your perspective, do you think the planning and organization of the CALs was well-planned and realistic in terms of its objectives, desired outcomes, targets, and timeframe? Why or why not?

5. Do you think critical factors that may influence the CALs’ success were effectively taken into account when designing the project? These include labor law regulation, economic policy for job creation, civil displacement by internal conflict, and the efficiency of the justice system. If so, how?


7. Do you think the project meets workers’ legal needs? Why or why not?

a. How can the project improve to better meet the needs of workers?

8. Have there been changes in labor regulations or laws that affect how it provides legal services? If yes, how well has the project reacted to and accommodated those changes? Please explain.

9. In your opinion, how has the CAL project affected the landscape of labor rights advocacy in Colombia?

a. How can the project improve to better complement or advance labor rights advocacy in the country?

b. If government representative: How would you describe the CALs’ relationship with the Colabora project? How can it be improved?

10. Do you think the project is effectively assisting or supporting workers who experience indirect illegal hiring? Why or why not?


11. Do you think the project is effectively engaging and serving workers who have experienced violence? Why or why not?


12. Do you think the project is effectively engaging and serving underserved or special populations, such as women, youth, union members, workers with disabilities, or ethnic/racial minorities?

a. How can the project improve to better serve these populations?



13. From your perspective, what outputs or outcomes has the project achieved so far?

14. Do you think the project’s activities have reached workers in sectors of interest, including ports, sugar cane, mining, flowers, and African palm? Why or why not?

a. How can the project improve to better serve these worker populations?

15. What are the strengths of the project’s progress in terms of targeting and implementation? What are its weaknesses?


16. In your opinion, how effective has ENS and the CALs been at engaging stakeholders (e.g., workers, unions, government, and local organizations) to sustain the CAL services after the project funding ends?

a. How can ENS and the CALs better engage and coordinate with labor rights stakeholders to sustain services?

17. Moving forward, what strategies should the CALs pursue in order to ensure long-term sustainability?


18. What lessons can be learned from the project and how they can be taken into account for future projects?

19. Knowing what you know about the project, is there anything you would do differently in order to have a more successful outcome?




Interview Guide: Union Leaders CUT-CTC

Informed Consent

Have participants read and sign the informed consent form. Give participants an unsigned copy of the form to keep for his or her records. Outline with Timings








The purpose of this focus group is to discuss your ongoing activities related to the Centros de Atención Laboral (CALs), including challenges, successful strategies, perceived outcomes, and sustainability plans. This focus group will last approximately 60 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. This interview will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation in this interview is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data gathered through these interviews will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.


Participant Background (2 minutes)


2. What are your main responsibilities in your position, particularly related to the CAL project? Validity of the Project Design (10 minutes)


4. From your knowledge, what are the target goals for the project? Is the timeframe to achieve these goals realistic?

Relevance and Strategic Fit (2 minutes)

5. Have there been changes in labor regulations or laws that may have affected the CALs? If yes, from you knowledge, how has the project reacted to and accommodated those changes?


6. How do the CALs and ENS measure and monitor CAL’s effectiveness?

a. Has the process for measuring effectiveness changed over time?

b. How do the CALs and ENS use the collected data? How could it be used better?

7. Have the project’s activities reached the target worker population in the sectors of interest, both in terms of type and number of workers?


8. How are the unions involved in the CALs’ outreach activities?

a. What have been the objectives of the outreach efforts?

b. Do you think these efforts have achieved their desired objectives?

c. How can these efforts be improved?

9. Are there any other CAL activities where your presence (or the union’s) is expected? Please explain.


10. Please describe the purpose of the Steering Committee meetings and your (or the union’s) role on this committee.

a. Where and how often does the Steering Committee meet?

11. What indicators are used to evaluate practices within the CAL offices?


b. Have there been any obstacles to improving practices? How have they been resolved?


12. It is our understanding that the CALs have undergone a process to redesign their strategy to sustain the project after funding from the US Department of Labor ends. Could you briefly explain the new strategy and the reasons behind the redesign?

13. Are the project results, achievements, and benefits likely to be durable? Are results anchored in national institutions and can partners maintain them financially at end of project?

14. What level of funding from the unions is currently being provided to the CALs?

a. How are these levels going to progress in the future?

15. How has ENS or the CALs engaged other stakeholders (e.g., workers, unions, government, and local organizations) to sustain the CAL services after the project funding ends?

16. Is the CAL strategy currently integrated into the CUT-CTC’s strategic plans? Please explain.

a. Is CUT-CTC funding from the national government earmarked for the management of the CALs?

b. Are CALs performance reports presented to the CUT-CTC’s executive committees at the departmental and national level? If so, how often?

17. Could you explain what activities have taken place to transition the management of the project from ENS to the unions?

18. From your perspective, which CAL activities and processes will be sustainable beyond grant funding and which will not be sustainable?


20. Can any unintended or unexpected positive or negative effects be observed as a consequence of the project’s intervention? If so, how has the project strategy been adjusted? Have the positive effects been integrated in to the project strategy? Has the strategy been adjusted to minimize negative effects?


21. What lessons have been learned and how can they be taken into account for future projects?





Focus Group Guide: CAL Clients

Informed Consent



section (in minutes) 5 Introduction 5 5 Participant Background 10

25 Accessing the CAL 35 30 Received Services from the CAL 60 15 Satisfaction with the CAL 80 10 Conclusion (including distribution of

incentives) 90


The purpose of this focus group is to discuss your experiences accessing and receiving services from the Escuela Nacional Sindical Centros de Atención Laboral (ENS CALs). This focus group will last approximately 90 minutes. With you permission, we will audio record the discussion so we can fill anything we miss in our notes. No one outside the evaluation team will have access to this recording. The focus group will work best if you do most of the talking. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. Your participation is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. We will be doing groups like this in different cities. The information we collect in these group conversations will be used to write a report. The report will put together the information from all the groups, highlighting informational points from specific CALs but not from particular individuals. You will not be identified by name. To thank you for your participation, we will provide COP 30,000 at the end of the discussion. Please make sure to see me after the focus group to receive your thank you gift. Do you have any questions for me before we begin? Okay, let’s get started.



1. What is your first name or nickname?

2. In which labor sector do you work?

3. How did you first hear about the CAL?

4. Are you a member of or affiliated with a labor union? Accessing the CAL (25 minutes)

5. What brought you to visit or contact the CAL for the first time?

a. When (month and year) was the first time you visited the CAL?

b. Did you attempt to talk to your employer before visiting the CAL? What was the result?

6. Do you remember what you thought of the CAL before visiting for the first time? What did you expect before you visited?

7. How many times did you visit the CAL before your problem was resolved?.

a. If you contacted the CAL more than once, was it for the same purpose each time or for different purposes? Please explain.

b. Did the CAL meet your expectations?

8. Do you think some workers are hesitant to contact the CAL for legal assistance? Why or why not?

a. From your point of view, what is the biggest deterrent for workers to file a complaint?

9. For women: Do you think some women workers are hesitant to contact the CAL for legal assistance? Why or why not?

a. What is the biggest deterrent for women workers to file a complaint?

10. What challenges, if any, did you experience when accessing CAL services? Receiving Services from the CAL (30 minutes)

11. What services have you received from the CAL?

12. For those who filed a legal claim (tutela, querella, etc.):

a. How long after visiting the CAL did you file the claim?

b. How did the CAL help you with your legal claim?

c. Which actions did you take in order to file your legal claim?

d. Did you also try talking to your employer? Why or why not?

13. If you could not (or chose not to) file a legal claim, why?

a. Did you try another strategy like talking to employer? Why or why not?


14. What has been the outcome of the legal claim that you filed?

a. Are you satisfied with the outcome?

b. If your outcome was not resolved, do you know why? Do you receive any feedback from the authorities?

15. Approximately, how long did it take to receive an outcome on your legal claim?

a. Did you experience any delays or obstacles?

16. Have you received services through the mobile CAL? If so, what services did you receive?

17. Have you heard of trainings provided by the CAL for workers? If so, where did you hear about them?

18. Have you attended any training provided by the CAL? Please explain.

a. What did you expect from the training? Did it meet your expectations?

b. For women: Did the training include any information for domestic workers specifically? Pregnant or lactating workers? What about on sexual harassment?

19. Did you suffer any retaliation in the workplace following the pursuit of a legal claim and/or trying to seek CAL’s assistance?

Satisfaction with the CAL (10 minutes)

20. Overall, did you have a positive or negative experience with the CAL? Please explain.

21. Would you recommend CAL services to a family member or friend if they had a problem at work?

22. How can the CAL improve to better meet your legal assistance needs?

23. How can the CAL improve to better meet the legal assistance needs of other workers?

a. For women: How can the CAL improve to better meet the legal needs of women workers?




Focus Group Guide: CAL Lawyers

Informed Consent



section (in minutes) 5 Introduction 5

10 Participant Background 15 15 Background of the CAL 30 25 Service Provision to CAL Clients 55 25 Project Progress and Management

Arrangements 80 5 Recommendations and Lessons Learned 85 5 Conclusion 90


The purpose of this focus group is to discuss your ongoing activities related to the Centros de Atención Laboral (CALs), including challenges, successful strategies, and perceived outcomes. This focus group will last approximately 90 minutes. With you permission, we will audio record the discussion to assist with note-taking. No one outside the evaluation team will have access to this recording. The focus group will work best if each of you participates actively. Feel free to speak openly and candidly about your experiences and perspectives regarding this project. We are not looking for agreement; it's fine if you don't agree with someone else in the group, just tell us how you see things. Your participation is voluntary. If, at any time, you wish to discontinue participation, you may do so without penalty. The data will be reported in an aggregate manner, highlighting informational points from specific CALs and not from particular individuals. You will not be identified by name. Do you have any questions for me before we begin? Okay, let’s get started.



1. To begin, please, I'd like to hear a little bit about each of you. Let's go one by one and please tell us:

a. What is your first name or nickname?

b. How long have you been with the CAL?

c. Are you a student? If so, at which university are you studying?

d. What are your main responsibilities in your position?

Background of the CAL (15 minutes)

The first few questions are about the CAL’s purpose and its ability to respond to changes in labor rights law.

2. Please explain in your own words the objectives and desired outcomes of the CAL.

a. What is the CAL doing to meet these objectives and desired outcomes?

b. Do you think the CAL has achieved or will achieve its objectives? How could the CAL improve its services?

3. Have you experienced any changes in labor regulations or laws that affect how you or the CAL provide legal services?

a. If so, how has the CAL managed these changes?

4. Have workers’ legal needs changed over time in a way that has affected the project?

a. If so, how has the CAL responded to changing needs? Service Provision to CAL Clients (25 minutes)

Next, I’d like to learn more about the type of services you provide and legal cases that you see.

5. What are the most common workers’ rights violations that workers report at the CAL?

6. Do you know approximately what percentage of workers file a complaint after visiting the CAL?

7. How much time does it take for a typical claim to be resolved? If there is not a typical case, what would you say is the minimum and maximum time it takes for the claim to be resolved?

8. From your experience, what factors contribute to positive outcomes in legal cases? What factors contribute to negative outcomes?

9. Describe a particularly challenging case that you have worked on. Why was it challenging? What was the result?

10. How do you assist workers who have experienced illegal indirect hiring practices?

a. What obstacles have you experienced in improving the knowledge and understanding of illegal hiring practices?

11. How do you engage and serve workers who have experienced violence?


a. What obstacles have you experienced in engaging and serving workers who have experienced violence?

12. How effective is the CAL in serving underserved or target populations, such as women, youth, union members, workers with disabilities, or ethnic/racial minorities? Are there protocols in place?

a. What obstacles have you experienced in engaging and serving these populations?

13. What outreach efforts has the CAL implemented?

a. Do you think these efforts have achieved their desired objectives?

b. How can these efforts be improved? Project Progress and Management Arrangements (25 minutes)

The next set of questions are about the CAL’s progress in terms of meeting its objectives and the strategies to support CAL lawyers.

14. How does the CAL measure and monitor effectiveness of legal services?

a. Is there a system to track outcomes of legal cases? (Probe for use of database)

15. Does the CAL use standardized tools and protocols to determine the type of legal instruments to pursue? Please explain.

16. What kind of technical and administrative support do you receive as a CAL lawyer?

a. Is the support you receive effective and sufficient? How can it be improved?

17. What is the most frustrating part of your job? What is the most rewarding part?

18. Are there any resources that the CAL does not have that would be beneficial to you?

19. What training have you received from the CAL? How frequently do you receive training?

a. Do you think the training has been effective and adequate? How can they be improved?

b. How does the CAL determine the training needs of its lawyers?

c. How does training address the needs and process for accurate and complete documentation of legal claims?

d. What resources are available to lawyers to learn about previous cases and best practices? For example, is there an online library or database of previous cases that student lawyers can consult?

20. What strategies are used to manage periods of high demand for legal services?

a. What improvements could be made to avoid management challenges during periods of high demand?


We are now on the last section, which is about recommendations and lessons learned.




22. Lastly, is there anything that I did not ask about that you would like to share with me, or do you have any additional thoughts about what we have discussed today?

IMPAQ International, LLC Appendix C-1 ILAB Colombia Final Implementation Evaluation

APPENDIX C: SUMMARY OF ACTIVITY PROGRESS BY CAL


Activity AENS Number of printed manuals Target

Progress

Digital version uploaded to website Target

Progress

Jul-14 Sep-14 Nov-14 TOTAL Mar-15 May-15 Jul-15 Sep-15 TOTAL Mar-16 May-16Numer of workshops Target 1 1 1 3 1 1 1 1 4 1 1

Progress 0 0Number of worker's (union leaders) Target* 100 100

Progress 0 0Number of users in priority sectors Target NS 0 0

Progress 0 0Numer of workshops Target 1 1 1 3 1 1 1 1 4 1 1









Progress 0 0

Target*= This target was not specified by period, only in the yearly total

NS= Not specified

CAL Cartagena

CAL Bucaramanga

2015 2016

5,000 Trainigs have started as of April 2016, but number of manuals distributed is

unkown

No date given/specified

As of March site visit this version of the Handbook had not been uploaded

to website.

Worker's Rights Handbook

(Printed version and digital version

uploaded to website)

Workshops to train workers in

labor rights/Worker's Rights Manual

Workshops

2014

CAL Medellin

CAL Bogota


Activity BEstablishment of 3 CALs Completed in 2014CAL redesign document Completed in 2014Service Protocol (emphasis on vulnerable populations)

Completed in 2014

Include perspective on labor inclusion and victims of anti-union violence

Completed in 2014

Activity C

CAL MedellinNumber of university agreements (at least 3) Target University 1 University 2 University 3 University 4 University 5 University 6

Progress Autonoma LatioamMedellin Antioquia N/A N/A N/A

CAL BogotaNumber of university agreements (at least 3) Target University 1 University 2 University 3 University 4 University 5 University 6

Progress Nacional Santo Tomas Autonoma Militar Cooperativa Politecnico grancolombiano

CAL CartagenaNumber of university agreements (at least 3) Target University 1 University 2 University 3 University 4 University 5 University 6

Progress Fundación Univers Cartagena Rafael Nuñez N/A N/A

CAL BucaramangaNumber of university agreements (at least 3) Target University 1 University 2 University 3 University 4 University 5 University 6

Progress Industrial de SantaSanto Tomas N/A N/A N/A N/A

Agreements with universities

Activity DHiring of CAL directors, assistants and law interns

1 director and 1 assistant per CAL; 7 to 15 interns depending on CAL size

Onboarding training by roles and functions for CAL personnel

1 director and 1 assistant per CAL; 7 to 15 interns depending on CAL size

Medellin CAL lawyers trainingOther CAL personnel training

10 lawyers trained; 1 training per month

Bogota CAL lawyers trainingOther CAL personnel training


Cartagena CAL lawyers trainingOther CAL personnel training


Bucaramanga CAL lawyers trainingOther CAL personnel training


CAL Medellin Quarter/year of completion:Number of participants (students, CAL users and others) Target

30

Progress

CAL Bogota Quarter/year of completion:Number of participants (students, CAL users and others) Target

30

Fourth quarter 2015 Progress 30

CAL Cartagena Date of completion:Number of participants (students, CAL users and others) Target

30

Fourth quarter 2014 Progress 29

CAL Bucaramanga Quarter/year of completion:Number of participants (students, CAL users and others) Target

30

First quarter 2015 Progress 34

Diplomados


Activity E

Roll-out of new CAL databaseRoll-out completed from Aug to Dec 2015

Ongoing collection of user information in CAL Database

Ongoing

Mar-14 Jun-14 Sep-14 Dec-14 Mar-15 Jun-15 Sep-15 Dec-15All CALs (work dsitributed among CALs) 1 report/quarter Target

CAL 1 CAL 2 CAL 3 CAL 4 CAL 1 CAL 2 CAL 3 CAL 4

Progress (Please state CAL name)

All CALs (work submitted by each CAL and collected by Project Manager) 1 report/quarter Target

1 1 1 1 1 1 1 1

Progress 1 1 1 1 1 1 1 1

Quarterly statistical reports (Reportes informes estadísticos trimestrales)

CAL database trend reports (Reportes de tendencia bases de datos CAL)

Activity F2014 2015 2016* 2016*= Only from January until March 31, 2016

CAL MedellinNumber of media outlets/year

Target (minimum)

3 3 3

Progress 4 11 1

CAL BogotaNumber of media outlets/year

Target (minimum)

3 3 3

Progress 4 12 2

CAL CartagenaNumber of media outlets/year

Target (minimum)

3 3 3

Progress 11 28 2

CAL BucaramangaNumber of media outlets/year

Target (minimum)

3 3 3

Progress 10 15 2

May-14 Jun-14 Jul-14 Aug-14 Mar-15 May-15 Jul-15 Sep-15 Nov-15 Mar-16

All CALs (work dsitributed among CALs)

Target (minimum)

CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota CAL Bogota

Progress 1 1 1 1 1 1 1 1

Topic

Indi ferencias en la rea l idad labora l de Bogotá

¿Discriminación hacía la mujer?

Economía del cuidado, pol ítica , fi sca l y género

Periodis ta a lemana en el Centro de Atención Labora l Bogotá

La his toria de Jhon y José, víctimas de RAMO

Adriana, víctima de enfermedad y discriminación labora l

Nuevo caso de violencia labora l hacía los trabajadores del servicio doméstico

Bogotá, en busca de un sueño labora l


Target (minimum)

CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin CAL Medellin

Progress 1 1 1 1 1 1

TopicLa mujer entre lo privado y lo públ ico

Una vida asediada por la inseguridad labora l

Un oficio no recompensado: madres comunitarias

Paula López Moreno: La digni ficación del servicio doméstico

Don Hugo, un conductor de bus que es obl igado a renunciar a su trabajo

Maricela , una mujer embarazada que logró el pago de sus prestaciones socia les

Sara , una madre joven que interpuso tutela para reclamar sus derechos labora les

10 años del CAL Medel l ín, a l servicio de todos los trabajadores pa isas

El drama de los pa leros de la Minoris ta , una his toria s in fin


Target (minimum)

CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena CAL Cartagena

Progress 1 1 1 1 1

TopicSNTT: un s indicato con mucha fuerza

de está la justicia? Trabajo indecente

Sin seguridad, s in trabajo y s in dignidad.

Una excelente labor del CAL a l servicio de los trabajadores

La atunera Seatech Internacional cierra su planta y echa a sus trabajadoras cada vez que le da la gana


Target (minimum)

CAL B/manga CAL B/manga CAL B/manga CAL B/manga CAL B/manga CAL B/manga CAL B/manga CAL B/manga CAL B/manga

Progress 1 1 1 1

Topic

BUCARELIA: en huelga imputable a l empleador

Jul ieta , madre comunitaria que lucha por sus derechos

En paro de 24 horas 35 mi l trabajadores tercerizados de Ecopetrol

El enfoque de género en el fenómeno de la violencia antis indica l en Colombia

Number of chronicles or emblematic cases (each CAL is in charge of at least 1/year)





Activity G2014 2015 2016*

CAL Medellin Number of users/year Target 2,000 2,000 3,000 Progress 3,030 3,618 760

CAL Bogota Number of users/year Target 5,000 5,000 6,000 Progress 6,462 9,157 1,392

CAL Cartagena Number of users/year Target 1,000 1,000 1,200 Progress 578 1,652 251

CAL Bucaramanga Number of users/year Target 1,000 1,000 1,200 Progress 1,270 1,872 301

Activity H2014 2015 2016*

CAL Medellin Number of administrative actions/year Target 400 100 100Progress 179 206 694

Percent of admin. actions in formalization/year Target N/A 20% 20%Progress 0%

Number of judicial actions/year Target 1400 500 500Progress 766 1290 385

Percent of judicial actions in freedom of association/year Target N/A 20% 20%Progress 1%

CAL Bogota Number of administrative actions/year Target 600 100 120Progress 42 115 285




CAL Cartagena Number of administrative actions/year Target 200 50 50Progress 20 21 57




CAL Bucaramanga Number of administrative actions/year Target 200 50 50Progress 26 337 83




CAL Medellin Number of victims of union violence/year Target N/A 20 10Progress

CAL Bogota Number of victims of union violence/year Target N/A 20 10Progress

CAL Cartagena Number of victims of union violence/year Target N/A 20 10Progress

CAL Bucaramanga Number of victims of union violence/year Target N/A 20 10Progress

CAL Medellin Number of actions/year Target N/A 10 20Progress

CAL Bogota Number of actions/year Target N/A 10 20Progress

CAL Cartagena Number of actions/year Target N/A 10 20Progress

CAL Bucaramanga Number of actions/year Target N/A 10 20Progress

CAL Medellin Number of users/year Target N/A 20 30Progress 17 27 2

CAL Bogota Number of users/year Target N/A 20 30Progress 0 0 0

CAL Cartagena Number of users/year Target N/A 20 30Progress

CAL Bucaramanga Number of users/year Target N/A 20 30Progress 50 43 5

2016*= Only from January until March 31, 2016N/A= Not applicable because this indicator was not included in the 2014 Workplan.

Actions presented for labor discrimination to vulnerable groups (disability, race, LGBTI, etc.)

Users assited through the psychosocial component

Number of users assisted by each CAL

Administrative and judicial actions undertaken by each CAL

Assistance to victims of union violence