Terms of Reference for Mid Term Evaluation Somali Program
Project Title: Youth Development and Livelihoods
Project Period: October 2021 – September 2024.
Background
American Friends Service Committee is a Quaker organization that includes people of various faiths who are committed to social justice, peace, and humanitarian service. Guided by the Quaker belief in the divine light of each person, AFSC works with communities and partners worldwide to challenge unjust systems and promote lasting peace. AFSC has more than 100 years of experience building peace in communities worldwide and works in the following ways: bridging and convening, community organizing and civil society strengthening, advocacy, research, and analysis. It seeks to change situations and systems that lead to violence. Much of AFSC’s work includes support to opportunities for dialogue and capacity building led by the affected communities.
About the Program
AFSC is implementing a youth empowerment program aimed at promoting peaceful co-existence within Somalia and Northern Kenya communities by working directly with two local partners in Kismayu and Dadaab Refugee Complex for a period of three years. The program aims to support youths in peacebuilding initiatives while enabling earning of sustainable livelihoods, access to psychosocial support for IDP’s, refugees, returnees, and host communities as well as build capacities of youth leaders in both Dadaab and Kismayu to advocate for migrants’ rights. A series of activities have been implemented to ensure success of the program.
Somalia Context
Somalia has long been considered a fragile state due to a combination of political, security, social and economic challenges. These factors have contributed to the overall fragility and unstable ecosystem leading to politization of clannism within the country a factor that has contributed to conflicts within the borders and across. With an estimated population of 18.1 million, shared across five clans, the in roots of clannism continue to take shape in various aspects of life including politics, governance, and resources distribution (UNFPA, 2023). The power dynamics continue to be shaped, influenced, and distributed among different clans with alliances contributing to fragmentation and subsequent instability of the country.
Despite these challenges, efforts have been made to address clannism in Somalia leading to promotion of a more inclusive and unified nation through the formation of transnational federal government and government. These efforts notwithstanding, Somalia continues to witness intermittent conflicts that have continued to put pressure on the country’s social fabric which has equally exacerbated other triggers brought about by ongoing droughts that have led to food insecurity, and pre-existing clan tensions among communities, economic disparities, regionalism, and religious afflictions.
Climatic shocks that have remained prolonged such as droughts, floods and cyclones have increased Somalis vulnerability as it has led to water scarcity, reduced crop production and loss of livestock’s leading to livelihoods of millions of people affected and exacerbating the already fragile humanitarian situation. According to Food Security and Nutrition Analysis Unit (FSNAU) and the Famine Early Warning Systems Network (FEWS NET), an estimated 5.9 million people in 2022 needed direct humanitarian assistance due to the persistence levels of food insecurity.
Over the years, Somali’s economy has rebounded with a GDP growth of 2.9% percent in 2021 up from a contraction of 0.3% in 2020 with a higher percentage of the current year forecast of 2.4%. this is despite the significant shocks and factors that have muted economic recovery including political disruptions, prolonged drought, increased insecurity, and economic bottlenecks. In 2023 Somalia GDP growth stands at 3.6% with a projection of 3.7% in 2024 which is pegged on human capital, integrated investments, Somali diaspora support and national social safety net system (World Bank, 2022).
Evaluation Objective:
The overall objective of the mid-term evaluation is to assess the achievement of the project objective against indicators.
The project goal is; Somali youths assume responsibilities and leading roles in their communities and actively participate in non-violent and innovative community-based initiatives, thus contributing towards building peaceful and safe communities in Somalia. The specific objective and indicators are as shown below:
Objective no. 1: Young Somalis in the target regions are actively involved in peace and reconciliation work and earn sustainable livelihoods through income-generating measures.
Indicators:
- By the end of the funding period, at least 80% (40% of them female) of the 1,190 trained young Somalis are actively involved in peace and reconciliation work within their communities.
- By the end of the funding period, 70% (30% of them female) of 450 targeted young Somalis who have acquired vocational skills within the framework of this project, are able to earn a living as groups or individuals.
Objective no. 2: Young refugees in Dadaab camp, returnees, and internally displaced persons (IDPs) in Kismayo actively participate in social development activities and have access to psycho-social support.
Indicator:
- By the end of the funding period, at least 80% of targeted young refugees (50% of them female) in Dadaab camp as well as returnees and IDPs (2,510 in total) and the host communities are actively involved in trauma and reconciliation work.
Objective no. 3: The capacities of young refugees in Dadaab camp and Kismayo to advocate for migrants and refugees are strengthened.
Indicator:
- By the end of the funding period, at least 80% of the 150 trained youths have acquired skills and knowledge in advocacy and actively advocate for the rights of refugees and migrants.
Scope
The evaluation will cover the period from October 2021 to May 2023, to create an accurate and comprehensive picture of the project implementation, generating findings on evaluation criteria and documenting good practices and lessons learnt. Project sites in this evaluation will be Kismayu, Somalia and Dadaab Refugee Complex (Ifo), Kenya.
The mid-term evaluation will specifically:
a) Assess Program relevance, performance, effectiveness, efficiency, impact, and sustainability.
b) Qualitatively and quantitively, capture the key lessons and progress towards achieving the indicator by assessing the categories below:
- Young Somalis (IDP’s, refugees, returnees, and host community members) reached through peacebuilding efforts, psychosocial support, and vocational training.
- Somali youths’ leaders reached through advocacy efforts.
- Implementing partners and their staff.
- Other stakeholders such as, local government administration, religious leaders, clan elders, women groups, camp protection and peace committees.
- Former project alumni’s
c) Highlight key success stories, best practices, challenges, and recommendations for the remaining implementation period and future projects.
AFSC has used various methods such as partner accompaniment, mentorship, continuous monitoring, and partner capacity building to ensure acquisition of knowledge on various aspects of peacebuilding approaches, psychosocial support, and advocacy to achieve desired project goal. The evaluation findings will be utilized internally and be made available to AFSC staff, partners, and the donor Misereor.
AFSC encourages evaluators to develop Key Evaluation Questions based on the Quality Standards of the Development Assistance Committee (DAC) of the Organization of Economic Co-Operation and Development (OECD) which give emphasis on Relevance, Coherence, Effectiveness, Efficiency, Impact and Sustainability.
Key Evaluation Questions based on DAC Criteria.
a) Are the implemented activities relevant, appropriate, effective, and implemented in an efficient manner?
b) What are the key program outputs and outcomes and how are the program activities contributing to the outcomes?
c) What capacity building activities have been undertaken to strengthen implementing partners and how did they contribute to achieving program goal?
d) To what extent is the program leading to peaceful coexistence, psychosocial wellbeing and contributing to migrant rights advocacy in project areas?
e) How were the different clusters of youths (women, IDP’s, returnees, refugees, host communities and vulnerable groups) involved in the program?
f) What are the factors contributing to achievement or non-achievement of the program objective?
Impact:
a) What real difference has the program brought about for the participants? (What would have happened without the activity?
b) How many people have been affected since the program started to date? Planned target group vis à vis really addressed?
c) What are the short and medium term (intended and unintended) outcomes of the program?
d) To what extent was the selected target groups reached?
*The evaluator should feel to suggest additional questions that emphasize on the impact of the action/project.
Sustainability of the program:
a) To what extent will the positive impacts or changes of the program (are likely to) continue?
b) Which measures are implemented to support continuity?
c) What are the major factors influencing the achievement or non-achievement of sustainability of the program?
d) To what extent is the exit strategy relevant?
Evaluation Methodology, process, and reporting
The evaluation will involve participatory methodologies and tools. The evaluator will creatively employ a mix of techniques for data collection and will among others hold meetings and discussions with Somali youth/program participants, key informants, interviews with the implementing partners, 5 participating partners’ staff and AFSC staff. AFSC will make accessible program documents for review and secondary data collection. The documents will include the program proposal and budget, cooperation agreement, partnership documents, activity reports, AFSC internal and donor reports.
Evaluation process and timeline
The evaluation will be carried out in the month of August 2023. The assignment is proposed to be completed in 23 days ensuring that the final report is submitted to AFSC by latest Mid-September 2023.
Proposed Timelines
Tasks No. of days
Analysis of relevant documents, development of evaluation design 1
Initial meeting with Program Team 1
Finalize inception report 1
Field work 9
Compilation and preparation of preliminary findings 3
Review and compilation of implementing organizations and staff feedback 2
Preparation of draft report and submission for feedback 2
Finalize report factoring in AFSC and Misereor feedback 4
Total 23
Proposal Submission Requirements
All proposals must be received by 5:00 pm Kenyan time, by COB on 1st August 2023. A cover letter and proposal with budget and timelines should be sent to AFSC via InfoAfrica@afsc.org
Proposals should include the complete scope of work and deliverables including the following sections:
a) Organization/Evaluator Background Peacebuilding, psychosocial, migration, advocacy and development experience, education, skills, and languages. Proposals must include three examples of related work completed and contact information for the organizations served.
b) Statement of Proposed Work clearly indicating the methodologies proposed to complete the evaluation and a final report including a time frame for completion of specific tasks, the personnel needed to complete tasks, and expectations for support and assistance from AFSC. Provide a clear work plan for the assignment.
Key Deliverables
The outputs expected from the consultant are:
a) Inception report upon signing a contract; submit an inception report detailing the evaluation design, methodology and data collection tools to be discussed and agreed upon with AFSC.
b) Produce a draft report; The mid-term evaluation report including stories of change and lessons learnt.
c) Final Midterm evaluation report incorporatingsuccess stories and comments from AFSC
1. Information Page:
Basic organizational data, duration of the project to be evaluated, title of the evaluation, principal of the evaluation (who commissioned the evaluation), contractor of the evaluation and date of the report.
2. Executive summary: tightly drafted, to-the-point, free-standing document including the key issues of the evaluation, main analytical points, conclusions, lessons learnt and recommendations.
3. Introduction: purpose of the evaluation, scope of the evaluation and key questions. Short description of the project to be evaluated and relevant frame conditions.
4. Evaluation design/methodology.
5. Key results/findings: about the questions pointed out in the ToR and the projects’ specific intervention components.
6. Conclusions: summary based on evidence and analysis.
7. Recommendations: on the findings leading to suggestions to be used for the way forward.
8. Lessons learnt: all relevant information beneficial to the partnership between AFSC and implementing partners.
9. Annexes (TOR, instruments used, list of persons/organizations consulted, literature and documentation consulted, copy of any relevant documentation used for the assessment, and CV of the evaluation team).
The evaluator should include the most significant change stories. While the evaluators are expected to work independently, MEAL officer, Somalia Program officer and Country Representative will assist in facilitating access to program participants, documents and other logistical support required to facilitate the evaluation.
c) Budget and Deliverables
Provide a detailed budget as well as a description of the specific deliverables that will be submitted and expected schedule of compensation.
d) Evaluation Schedule
The proposal should include a workplan showing how the evaluation will be carried out.
Proposal Evaluation Procedure.
After having received the responses from the applicants, a consultant will be selected based on an appropriate selection process in consultation with the Country Representative Somalia and South Sudan, Regional Director for Africa, and the international program support staff as needed. A signed agreement should be processed with the selected consultant and in accordance with AFSC vendor practices.
Qualifications of Evaluation Team
The consultant/team should have experience of working in crisis response, conflict resolution or community development be fluent in English, spoken and written, have strong analytical skills, good listening and discernment skills, and proven experience in evaluating peacebuilding and migration programs. Somalia socio-cultural awareness will be an added advantage.
Other requirements:
- Advanced university degree in relevant field.
- At least 5 -7 years of experience conducting similar assignments.
- Extensive experience in field research in Somali or other conflict zones in Horn of Africa.
- Excellent research, report writing and analytical skills.
- Well conversant with qualitative and quantitative methods of research.
- Proven capacity to write analytically, understandable, and simple reports.
- Experience of working with secondary data analysis/desk reviews.
- Experience of developing research tools and carrying out research.
- Advanced knowledge of the new trends and developments in Somali.
- Ability to provide clear guidance to field research teams.
- Proven ability to deliver against targets and meeting deadlines within short timeframe.
- Relevant computer skills: Word, Excel, internet, Power point.
Inquiries
Questions that help clarify the work to be completed may be submitted to Zaina Kisongoa, Somalia Country Representative, at ZKisongoa@afsc.org.
b) Statement of Proposed Work clearly indicating the methodologies proposed to complete the evaluation and a final report including a time frame for completion of specific tasks, the personnel needed to complete tasks, and expectations for support and assistance from AFSC. Provide a clear work plan for the assignment.
