WASH sector evaluation

Lead the WASH sector response evaluation process

Multisectoral evaluations do not focus on a specific sector, except in specific cases (evaluation of cholera outbreak have a greater focus on WASH and Health; evaluation of drought response focuses on FSL etc.). In some contexts, the WASH coordination platform and UNICEF may find relevant to organize a specific evaluation of the WASH sector response. This has to be differentiated from usual WASH project evaluation, as it is about evaluating the WASH response as a whole, often including an evaluation of the WASH coordination system as well. In the key guidance and tools on top of this page can be found a general methodology to carry out evaluation of humanitarian action (2016 ALNAP Evaluation Humanitarian Action Guide). It is important to involve the GWC when organizing a WASH response evaluation, as it can lead to the production of GWC lesson learned or technical guidance documents (see Technical guidance chapter). Depending on the objective and the scope of the evaluation, several specific methodologies can be considered:

Evaluate response through key informants interview & documentary review

This is a common type of sector level evaluation. It consists in reviewing cluster's and main partners' projects documents, perform key informant interviews at capital and field level, review data from other clusters related to WASH such as Health and Nutrition, review recent coordination monitoring results and finally draw conclusion on the efficiency, quality and relevancy of the WASH response following certain agreed criteria. The DAC/OECD evaluation criteria, generally used by partners for their project evaluation, can also be used for sector evaluation: Relevance, Effectiveness, Efficiency, Impact, Sustainability (the later can be replaced in emergency by Localization and Humanitarian/Development Nexus).  

Lead a systematic review of WASH partner’s projects evaluation (meta-evaluation)

WASH partners often implement internal or external evaluation of their projects. A systematic review will gather all partners' evaluation and extrapolate their results to the whole response. This exercise is feasible if:

• A significant number of partners have evaluated formally their projects
• Partners have used comparable evaluation criteria
• Evaluation are geographically complementary
• Evaluated projects followed HRP/SOF strategy

Consequently, it is critical for the WASH coordination platform to agree with the SAG on an harmonized evaluation protocol, setting up some minimum criteria partners should respect when carrying their evaluation (similar evaluation criteria; robust methodology etc.).

Lead a response-wide field evaluation

This is a complete evaluation of the whole WASH sector response, involving interviews and site visit of a representative sample of beneficiaries and WASH infrastructures in the whole country or region where the response took place. Unless the crisis and the response is very localized, this type of sector specific country wide evaluation is rare and challenging, involving considerable resources. 

Evaluate WASH coordination system

This is usually included in WASH response evaluation exercise, considering WASH coordination as one of the response inputs. The objective is to evaluate whether coordination has been efficient, and which effect it had on the response. Cluster Coordination Performance Monitoring guidance can be used, as well as other indicators such as "time for recruitment/deployment of coordination staff", value for money analysis, achievement of GWC minimum requirement for coordination, and achievement of coordination core functions.

Coordinate WASH response impact measurement and research

Although not a priority in emergency, evaluate the impact of the WASH response can be critical to orientate and improve future responses. Obtain evidences of strong impact will also improve fundraising. Emergency WASH response's overall objective is usually to decrease morbidity & mortality rate, and in lesser extend, protect people’s livelihood, improve protection and child education. Measure the impact of WASH on these outcomes is possible but complex, as many confounding factors will influence them (seasonality of disease, level of education, healthcare environment, food security...). Accurate data are also difficult to obtain in emergency context. Impact measurement should involve setting up of complex research protocols, in partnership with universities and research institutes. It often involves the use of control groups (group of people similar to the one studied but who did not benefit from the intervention), which has some ethical implication, especially in emergency.