Post-Project Evaluation of the JPF-funded WASH interventions in the Kalobeyei Integrated Settlement, Kenya

1. Background:

The Japan Platform (hereinafter referred to as “JPF”) is an international emergency humanitarian aid organization which offers the most effective and prompt emergency aid in response to humanitarian needs, focusing on issues of refugees and natural disasters. JPF conducts such aid through a tripartite cooperation system where NGOs, business communities, and the government of Japan work in close cooperation, based on equal partnership, and making the most of the respective sectors’ characteristics and resources.

JPF serves as an intermediary support organization providing various types of assistance to member NGOs in Japan to deliver quick and comprehensive aid on their own. JPF has supported aid activities of 45 member NGOs, each with its own set of diverse strengths. It has delivered humanitarian assistance to 55 nations and regions about 1,500 projects, with a total financial contribution of 60 billion yen. JPF has built a strong reputation based on trust by promoting cooperation among private sectors and NGOs and by accurately reporting all its activities. Established in 2000, JPF has been providing emergency relief in conflict affected countries as well as disaster-stricken regions including South-Sudan, Syria, Iraq, Afghanistan, Yemen, Bangladesh, Ukraine, and others. To ensure the quality of the assistance and promote accountability, JPF is seeking qualified Monitoring and Evaluation specialists. More information on JPF can be found at https://www.japanplatform.org/index.html.

Since 2006, JPF has been funding Japanese NGOs responding to the chronic humanitarian crisis in South Sudan, predating South Sudan’s independence in 2011. As of 31 December 2022, South Sudan was the largest displacement crisis in Africa, with 4 million forcibly displaced. Given fragile peace and security in the region, compounded by multiple challenges, there has been pendular movements of South Sudanese between countries of origin and asylum in the past. In line with the international pledges under the Global Compact of Refugees, the asylum countries are making progress towards integration of South Sudanese refugees into the national economic, education and health systems. JPF repeatedly renewed its annual funding commitment to the South Sudan refugee programme in South Sudan and its neighboring countries. In FY 2021, JPF funded 8 projects in WASH, protection and education implemented by 6 agencies in South Sudan, Ethiopia, Uganda, Sudan and Kenya.

JPF annually conducts evaluations on selected projects under the South Sudan refugee assistance program. Projects in Kenya was last subjected to a third-party monitoring in late Feb 2020. This evaluation will specifically focus on WASH interventions in the two latest projects funded by JPF in Kalobeyei and Kakuma implemented between 2020-2022.

2. Purpose of the evaluation:

JPF seeks to engage an independent consultant to evaluate PWJ’s WASH interventions in Kalobeyei in 2020-2022. Building on the pre-pandemic 2020 evaluation, this post-project evaluation aims to deepen learning on sustainability, localization and an effective transition from one source of funding to another, in a way that will guide JPF’s focused, multi-year financial support for the South Sudan refugees assistance programme and support PWJ’s WASH program currently scaled up across the 3 refugee hosting areas. The evaluation will inform JPF’s and its member agencies’ ongoing work in South Sudan and Ethiopia, another CRRF pilot country hosting South Sudanese refugees, and critically assess alignment of JPF’s funding scheme with the GCR and the localization agenda. While this study is a post-project evaluation for JPF, it is also a baseline for PWJ’s work with UNHCR. It is expected that this evaluation will illustrate the current state in Kalobeyei in a way that is comparable to the other two PWJ project areas, and lay the foundation for subsequent outcome monitoring PWJ wishes to introduce in the UNHCR-funded program.

The primary audience of the evaluation are JPF, PWJ and other implementing agencies in JPF’s South Sudan Working Group. The evaluation report will be also communicated to the wider audience in Kenya and in Japan to share relevant lessons learnt for the broader sector and as a means to hold JPF accountable to its supporters and stakeholders.

3. Scope of the evaluation:

With an aim to drive the realization of Kalobeyei Integrated Socio-Economic Development Programme (KISEDP)’s goal of sustainable and integrated WASH service delivery in the protracted refugee situation, PWJ jointly developed an outcome framework for Kalobeyei and Kakuma with the former WASH sector lead, Norwegian Refugee Council (NRC). This outcome framework is meant to be used in the UNHCR funded WASH project, and encompasses nearly all the areas of change sought by the JPF-funded WASH component, except for menstrual hygiene management (MHM) and sexual and reproductive health (SRH) aspects. The evaluation will primarily draw on this framework, and incorporate these missing gender aspects in a study design to illustrate the current state of WASH in Kalobeyei.

The evaluation will analyze if and how the JPF-funded projects of 2020-2022 have or have not contributed to the outcomes sought as per the framework under the WASH program beyond JPF funded projects. The evaluation may complement the existing output & outcome data and project reports on the JPF-funded projects with additional qualitative and quantitative data to do so. The evaluation may also refer to relevant reports and data on other completed WASH projects by PWJ where necessary. The evaluation will also draw on the pre-existing evaluations and learning on CRRF applications in practice, such as the 2019 “joint evaluation of the integrated solutions model in and around Kalobeyei, Turkana, Kenya”, the 2019 “Comprehensive Refugee Response Framework: Progress in Kenya” and the KISEDP Phase II 2023-2027 which includes a review of the phase I achievement, challenges and lessons, among others.

Based on the above data and analysis, the post-project evaluation will respond to the following issues of interest to JPF and PWJ, and make relevant recommendations for PWJ, JPF and other JPF member agencies part of its South Sudan refugee assistance programme.

• Sustainability and localization
  1. To what extent are the outcomes achieved by the JPF funded projects likely to be sustained beyond the life of the projects? Why so? What are the enabling and constraining factors that influenced sustainability of outcomes achieved during the lifetime of the project?
  2. To what extent have local knowledge, capacities, ownership and systems been harnessed by the JPF funded projects to achieve and sustain the outcomes assessed? In what ways? Have there been any missed opportunities to involve local actors or build on their knowledge, capacities, ownership and systems? Why so?
• Alignment with CRRF in terms of social integration of refugees and host populations
  1. How equitably have the JPF-funded projects involved and benefited refugees and host populations? How do the target refugees and host populations respectively assess relevance of the JPF-funded projects to their current and long-term needs, interests and aspirations? In what ways, if any, have the projects influenced perceptions of the target refugees and host populations of each other?
  2. In what ways have the JPF-funded projects taken advantage of the policy and institutional environment created under the CRRF in the benefit of the target refugees and host populations? What is it that a “whole of society” approach has enabled the projects to do that would have not been possible under the encampment approach? Have there been any missed opportunities to maximize the benefits of CRRF for the refugees and host populations?
  3. What have been specific contributions of the JPF-funded projects to KISEDP, both intended and unintended? How effectively have the projects linked humanitarian assistance to development in enabling self-reliance of refugees, progressively reducing their dependency on humanitarian support?
• Alignment of JPF’s funding mechanism with GCR and the localization agenda
  1. How has JPF’s annual funding scheme and financing modalities facilitated or hindered sustainability and localization, as assessed under 1.1-1.2, and alignment with CRRF, as assessed under 2.1-2.3?
• A process of phasing out JPF funding
  1. How are overall funding needs for the WASH sector currently met in KISEDP and in PWJ’s WASH program portfolio? What are the characteristics of JPF funding, including strength and weakness from the perspective of implementing agencies, relative to other donor funds? How are different funds coordinated to ensure continuity and coherence of WASH interventions?
  2. What does an “ideal” process look like for a donor to phase out from the perspectives of PWJ and local stakeholders? How have PWJ and local stakeholders, including the county water, health and environment departments, the target refugees and host populations, experienced an end of JPF funded projects in reality?
  3. How seamlessly has PWJ managed a transition process from one project to another? In what ways has JPF’s decision making process enabled or constrained PWJ’s capacity to manage the transition process? What kind of inter-agency coordination, at what level, has been required for the project closure and how well has it been managed?

Lastly, the consultant will review what worked and what didn’t in collecting the baseline data as per the outcome framework, and make recommendations for the methodology and approach for outcome monitoring to be undertaken by PWJ under the UNHCR funded program.

4. Deliverables

• An inception report, describing the methodology and plan for the evaluation and annexing the evaluation tools.
• A debriefing meeting with PWJ’s field teams
• A final report, responding to the Terms of Reference
• Baseline data set, in a way that is usable for future evaluations of the WASH program, while ensuring legal and ethical compliance with personal data protection.

5. Timeline:

An evaluation is expected to take 4 months from Aug to early December 2023.