Lead the overall WASH response evaluation process
Multisectoral evaluation do not focus on a specific sector, except in specific cases (evaluation of cholera outbreak might focus more on WASH; evaluation of drought response might focus on FSL etc.). In some context, the WASH coordination platform and UNICEF might find relevant to organize a specific evaluation of the WASH sector response. This has to be differentiated from usual WASH program evaluation, as it is about evaluating the WASH response as a whole, often including an evaluation of the WASH coordination system as well. Depending on the objective and the scope of the evaluation several methodologies can be considered:
Key informants interview & documentary review
This is the most common type of sector level evaluation. It consists in reviewing main partner's projects and evaluation, perform key informant interview at capital and field level, review data from other cluster such as health and nutrition, review recent coordination monitoring process and finally draw conclusion on the WASH response following certain agreed criteria. The DAC/OECD evaluation criteria, generally used by partners for their project evaluation, can be use as well at the level for the sector: Relevance, Effectiveness, Efficiency, Impact, Sustainability (which can be replaced in emergency by localization and connection with development/emergency sector).
Systematic review of partner’s evaluation (meta-evaluation)
WASH partners often implement internal or external evaluation of their projects. A systematic review will gather all partners' evaluation, compare their result and try to extrapolate the results to the whole response. This exercise will be possible if:
- Significant number of partners have evaluated their projects
- Partners have used comparable evaluation criteria
- Evaluation are geographically complementary
- Program evaluated follows HRP/SOF strategy
Consequently, it is critical for the WASH coordination platform to agree with the SAG on an harmonized evaluation protocol, setting up some minimum criteria partners should respect when carrying their evaluation (same evaluation criteria, robust methodology etc.).
Response-wide field evaluation
This is a complete evaluation of the whole WASH response, involving interviews and site visit of a representative sample of beneficiaries and WASH infrastructure in the whole country/region where the response has been taken place. Unless the crisis and the response is very localized, this type of sector specific country wide evaluation is rare and challenging, involving considerable resources.
Evaluation of WASH coordination
This is usually included in WASH response evaluation exercise, considering WASH coordination as one of the response inputs. The objective is to evaluate whether coordination has been efficient, and which effect it had on the response. Coordination performance monitoring questionnaires can be used, as well as other indicators such as time for recruitment/deployment of coordination staff, value for money, achievement of GWC minimum requirement for coordination, and achievement of coordination core functions.
Impact measurement and research
Although not a priority in emergency, evaluate the impact of the response can be critical to orientate and improve future responses, and obtain evidence to improve fundraising. Emergency WASH response's overall objective is usually to decrease morbidity & mortality rate, and in some extend, protect people’s livelihood, protection and improve child education. Measure the impact of WASH on these outcomes is possible but complex, as many confounding factors will influence them (seasonality of disease, level of education, healthcare environment, food security...). Accurate data are also difficult to obtain in emergency context. Impact measurement should involve setting up of complex research protocols, in partnership with universities and research institutes. It often involves the use of control groups (group of people similar to the one studied but who did not benefit from the intervention), which has some ethical implication, especially in emergency.