Lead the overall WASH response evaluation process
Contribute to overall multisectoral response evaluation
In L3 emergency crisis there are two main requirement imposed by the regarding evaluation as per IASC guidelines:
- Operational Peer Review. It is an evaluation carried out internally by UN officials 3 months after the begining of the response, aiming at supporting or reorienting the multisectoral strategy if needed
- Inter-Agency Humanitarian Evaluation (IAHE). IAHE are independent assessment of results of the collective humanitarian response by member organizations of the IASC to a specific crisis.
The WASH cooridnation platform is not directly involved in this process, but will be requested to contribute and provide information, as the other sectors'coordination platform
Set up WASH response evaluation framework
Multisector evaluation usually do not focus on a specific sector, apart from specific case (evaluation of cholera outbreak might focus more on WASH; evaluation of drought response might focus on FSL etc.). The HWCP and the CLA (UNICEF) might find relevant to organise a specific evaluation of the WASH sector response. This has to be differentiated from usual WASH program evaluation, as it is about evaluating the WASH response as a whole, often including the WASH coordination system
Lead specific overall WASH response evaluation
Multisectoral evaluation do not focus on a specific sector, except in specific cases (evaluation of cholera outbreak might focus more on WASH; evaluation of drought response might focus on FSL etc.). In some context, the WASH coordination platform and UNICEF might find relevant to organize a specific evaluation of the WASH sector response. This has to be differentiated from usual WASH program evaluation, as it is about evaluating the WASH response as a whole, often including an evaluation of the WASH coordination system as well. Depending on the objective and the scope of the evaluation several methodologies can be considered:
Country wide field evaluation
Make participant reflect whether such evaluation is possible. It would involve selecting a representative sample of benef over the whole country, and visit a representative sample of communities and infrastructures. It would be very challenging, considering the various geographical areas of the country, unless the crisis and the response is very localized.
Systematic review of partner’s project
This can be done, although it requires a lot of resources, as there are probably hundreds of different project to be implemented, and tens of partners to interview. It can probably be more reasonable to focus it on CBPF/main donor funded project.
Systematic review of partner’s evaluation
A systematic review will gather all partner’s evaluation, compare their result and extrapolate the results to the whole response. This exercise will be easy if:
- Partner’s have evaluated their projects
- Partners have used same evaluation criteria
- Evaluation are geographically complementary
- Program evaluated follows HRP/SOF strategy
Consequently, it can be interesting for the WASH coordination platform to anticipate and try to harmonize partner’s evaluation, providing them with some parameters to be respected when carrying their evaluation (same evaluation criteria, evaluation coordinated between partner’s etc…)
Key informants interview & documentary review. This is probably the most common type of evaluation carried out at the level of the sector.
Ask participant to browse as an example in their in handouts the Yemen cholera response
- Example in handouts: impact of WASH on nutrition
Evaluation of WASH coordination
This is usually done at the same time as the response evaluation. The objective is to evaluate whether coordination has been efficient, and which impact it had on the response. The coordination performance monitoring questionnaire can be used (see after in this session), as well as time for recruitment/deployment of coordination staff, value for money, achievement of GWC minimum requirement for coordination, coordination platform outputs…
Impact measurement and research
First, have a discussion with participants on what are impact evaluations, giving some examples: Impact of WASH activities on nutrition, on morbidity & mortality, on people’s livelihood, on attendance to school.
Then let participants come up with ways of doing such evaluation:
- Internal or external evaluation
- HH based survey, with representative sampling
- Epidemiology study (especially done for outbreak such as cholera, hepatitis E…)
- Research project: several research designs exist, the main one being “randomized control trials” (blinded or not). Mention ethic issues of using control group.
- Systematic review of partner’s impact evaluation = evaluation of evaluation, to be able to generalize results to the whole population of a country or the planet
Although impact measurement in emergency are not a priority, coordination platforms can take some initiatives in recovery or preparedness phase to evaluate impact of the sector response, to strengthen impact evidences and improve future strategy. As for quality monitoring, one of the difficulty of the process is to be able to obtain representative data from the whole response, while impact evaluation from partners project will tend to be disconnected between them. To address this issue, the coordination platform can set up an impact evaluation framework for the response, to be used by partners when they will carry out evaluation activities. The framework can provide priority response aspects and zone to be evaluated, and propose standard evaluation methodology so meta-evaluation can be done.
In some cases, overall WASH response evaluation can also be organized (usually by UNICEF, with the close collaboration of the WASH coordination platform). They are usually qualitative, but can also include interviews of a representative cross partner sample of WASH beneficiaries.