Terms of Reference for End Line Evaluation of Pastoralist Communities Resilience to Climate Change

About Christian Aid

Christian Aid is an international development organization working globally for profound change that eradicates the causes of poverty, and striving to achieve equality, dignity, and freedom for all, regardless of faith or nationality. We work with partners to implement integrated and innovative programs to end poverty by addressing the underlying causes. In Kenya, we focus on strengthening and empowering citizens in Kenya to demand accountable and delivering institutions, access quality health services and realize resilient and thriving livelihoods.

1. Introduction

Christian Aid through partners PACIDA and CARITAS Maralal is concluding the implementation of a project entitled ‘Enhance the climate resilience and adaptive capacity of communities in Marsabit and Samburu Counties’. The project aimed at contributing to achievement of this goal by achieving the following outcomes: 1. Increased capacity of communities to meaningfully participate in County Planning and Budgeting for Climate Change Adaptation 2. Integrated climate change knowledge and learning into programme management 3. Improved capacity of community-based institutions in sustainable rangeland management and livelihoods diversification 4. Improved policy environment for sustainable rangeland management, livestock production and access to climate information services at the county level.

The project was implemented in partnership with CARITAS Maralal in Samburu County and PACIDA in Marsabit County. Key project implementation approaches included capacity building for community and County Government, system strengthening and policy advocacy, community led bottom-up policy advocacy and participatory approaches (Participatory Vulnerability Capacity Assessment (PVCA), community budgeting and planning) and women and youth empowerment. Learning, adaptation, and continuous review of programme approaches is integrated into programme management through Operations, Formative and Action Learning Research. In order to inform the Monitoring, Evaluation and Learning (MEL) component of this project, Christian Aid Kenya seeks to engage the services of a qualified and experienced consultant (s)/consultancy firm to undertake project end line evaluation in Marsabit and Samburu Counties.

2. Overall Objective

The objective of the end line evaluation is to assess the progress made by the project in fulfilling its agreed objectives through the planned activities and to evaluate the relevance of the project to the implementation areas; and to assess the efficiency and effectiveness with which resources have been used to generate results and achieve project objectives.

3. Specific Objectives

• To assess the extent to which the project outputs have been achieved taking into account the indicators given in the project document.
• To evaluate the level of contribution of the outputs towards the achievement of the expected project objectives, goal, and impact.
• To assess the relevance, effectiveness, and efficiency of interventions as well as, the sustainability of the results, and the degree of satisfaction of the beneficiaries.
• To evaluate the appropriateness of the strategies and approaches used for implementing the project.
• To identify major external factors that influenced or impacted on the implementation of the project and evaluate their implication on future interventions.
• To highlight lessons learned from the project and make recommendations for future strategies. **

4. Methodology and Technical Approach

A cross-sectional study will be conducted in the project implementation areas of Marsabit and Samburu Counties. The consultant is expected to conduct a participatory evaluation providing for meaningful involvement by project partners, beneficiaries, and other interested parties. Primary and secondary data is expected to be gathered using a mixed method to evaluate programme implementation. Thus, the evaluation will involve collecting the following data:

Quantitative data: It will be used to measure indicators of the project expressed in terms of numbers/percentages (quantitative indicators). Information will be collected using questionnaire survey which will be administered to a sample of households. The consultant is expected to determine the sampling technique and size which would be representative, and adequate to detect the changes observed and do basic descriptive statistics.

Qualitative data: Qualitative information will be collected through Focus Group Discussions (FGDs), Key Informant Interviews (KIIs) and In-depth Interviews (IDIs) with key stakeholders including beneficiaries, community leaders, community/traditional leaders, local government offices, civil society organisations, donors, local implementing partners, project staff and others to complement the information collected through quantitative approaches. Secondary documentation: Review of related documents will also be important to establish key insights relevant to the effective implementation of the project.

It is expected that the assessment and data collection will use a mixed methodology approach, combining use of secondary data, primary quantitative data collected through electronic/digital data systems and primary qualitative data. The evaluation must collect detailed demographic data on target beneficiary groups (both direct and indirect beneficiaries) in the project locations. All data, both qualitative and quantitative must also be disaggregated by location, age, sex, disability etc. All data and analysis developed must be handed over to Christian Aid at the end of the evaluation and must be in English.

Triangulation of information gathered during the quantitative and qualitative research will be a critical aspect of this evaluation. The analysis must provide a clear explanation of how the different data sets relate to one another and how they relate to the findings in the secondary documentation. The complete project log-frame detailing the process, output and outcome indicators will be provided to the selected consultant prior to project initiation.

5. Scope of Work

The Project Evaluation will be conducted in the project implementation sites including Laisamis, North Horr, Moyale and Saku Sub-Counties of Marsabit County and Samburu East, Samburu West, and Samburu North Sub-Counties of Samburu Counties.

6. Specific Tasks

Specifically, the consultant will work closely with Christian Aid, PACIDA and CARITAS Maralal project staff and MERL Team to:

• Participate in briefing and consultation meetings to discuss the assignment, become familiar with the Resilience to Climate Change Project and to collaborate with the CA project and partner staff and other project implementing partners on the development of the end line evaluation design and execution plan.
• Develop in collaboration with CA and partners a detailed research and analysis plan, including plans for pre-testing tools, recruitment and training of research assistants, meaningful participation of partners and community, sampling methodology, data collection in the field, qualitative and quantitative data analysis and reporting.
• Develop in collaboration with CA and partners, rigorous and robust, qualitative, and quantitative data collection tools that can gather reliable and replicable data on each of the project indicators (sample tools to be developed, shared, and approved by CA prior to data collection). Provide detailed indicator protocols describing the data collection methodology for each indicator.
• Lead data collection in the field including designing and leading key informant interviews and focus group discussions; designing and leading surveys; training research assistants/enumerators; monitoring and ensuring data quality and research ethics of research assistants.
• Analyse data in a rigorous, robust manner using appropriate statistical and qualitative analysis techniques.
• Develop an up-to-date database for the customary and emerging institutions such as, County Climate Change Planning Committee (CCCPCs), Ward Climate Change Planning Committees (WCCPCs), Water Resource User Associations, Thematic Working Groups (TWGs), youth, and women groups in the project implementation areas.
• Draft a high-quality end line evaluation report. The report must be written in good quality English and must include robust and detailed analysis of data findings.
• Review the project MEL Framework, to identify challenges and shortcomings, and present strong recommendations for improved programming in future.

7. Outputs and Deliverables

1. Develop an inception report that should include data collection tools to guide field interviews and final schedule of field work and serve as an agreement between parties on how the evaluation will be, conducted. Items to address:
2. Develop and submit an acceptable draft report after the analysis of data for CA staff feedback following an agreed upon format. The report should be comprehensive and provide detailed specific findings within each result area and considerations/recommendations. The draft report should be no more than 35 pages (excluding references and annexes) following an agreed upon format.
3. Lead the analysis and synthesis of evidence to develop key findings; Discuss findings with Christian Aid and implementing partners and incorporate feedback to generate the final report.
4. Develop and submit to Christian Aid offices in Nairobi 3 hard copies and a soft copy of an acceptable final report of no more than 35 pages (excluding references and annexes) following an agreed upon format. The report should be in the format indicated below, to be submitted to, Christian Aid Office. It is preferable to illustrate the results by appropriate graphs, visuals, tables and/or a dashboard with an accompanied explanatory text. The report should consist of;

• Executive Summary in bullets
• Introduction
• Methodology, including sampling and limitations
• Analysis and findings of the evaluation. The analysis should be done according to the objectives: including an implementation strategy for the recommendations
• Address concerns, lessons learned and comments from Christian Aid. Stories of change and quotes from respondents
• Conclusions for each of the end line evaluation objectives
• Recommendations for future projects
• Annexes:
• Relevant maps and photographs of the evaluation areas where necessary
• Bibliography of consulted secondary sources
•  Summary of key findings in the form of Data Infographics format to be used as a quick and engaging way of sharing the key findings.
• Summary of the findings in (MS PowerPoint) to be used for subsequent dissemination.
• The final cleaned data set used for analysis and production of the final report in an acceptable electronic format (Excel, SPSS, STATA). All raw data (quantitative data files, transcripts from In-depth Interviews, FGDs etc. should also be submitted to Christian Aid offices in Nairobi for reference and future use.
• Proposal for a learning event/validation of evaluation findings

8. Time frame

The data collection phase in the field is to be, confirmed between the consultant and Christian Aid, but ideally would start in the second week of April 2022, with the final report deadline by the 29^th of April 2022.

Inception phase: Draft inception report

Tools development phase: Deliverable 1: final inception report including budget, methodology and qualitative research tools, approved by SCD, and South Sudan Project team. 50 % of total budget- 5 working days

Data collection phase: Desk review, interviews in and field visits in Marsabit and Samburu- 11 working days

Data analyses phase: Presentation of initial findings in Christian Aid and draft of the report- 3 working days

Evaluation report phase; Draft Evaluation Report, for comment by project team in South Sudan and SCD. 3 working days. Deliverable 2: Learning session in Nairobi, 1 working day. Deliverable 3: Final Evaluation Report, 50 % of total budget- 2 working days

Total: 100%

25 Days

Timetable depends on the weather, not all areas accessible in case of rain or insecurity. The consultant is expected to carry out the exercise within 25 working days with the exercise starting from 28^th March 2022 and concluding not later than 30^th April 2022.

9. Consultant Profile

The consultant (s)/consultancy firm must have the following qualifications and experience to be eligible for this assignment:

• The lead consultant must have a technical background in Climate Change (Adaptation and Financing), Disaster Risk Reduction, or related field, (at a minimum of master’s degree level).
• The lead consultant (s) must have demonstrated experience and expertise in designing and managing end line studies for similar projects and in delivering agreed outputs on time and within budget.
• Demonstrated experience and expertise in implementing rigorous project evaluation, analyzing Quantitative and Qualitative data and reporting.
• Understanding and experience working on Climate Change governance in Kenya is a key requirement for this assignment. Knowledge on devolved climate financing will be added advantage.
• Should demonstrate broad knowledge on policy advocacy and institutional capacity development.
• Demonstration of good knowledge of the context in Marsabit and Samburu County is an added advantage.

10. Ethics and Child Protection

It is the responsibility of the consultant to demonstrate commitment to strictly adhere to Christian Aid’s Child Protection Policy. The consultant has the responsibility to ensure that any persons hired, used, or consulted during the process are made familiar with the policy and commit to abide by it during execution of this work. The consultant and associates must sign the policy prior to engagement in the assignment.