...
| Expand | ||
|---|---|---|
| ||
| Expand | ||
|---|---|---|
| ||
| Expand | ||
|---|---|---|
| ||
...
Lead the WASH sector response evaluation process
Multisectoral
...
evaluations do not focus on a specific sector, except in specific cases (evaluation of cholera outbreak
...
have a greater focus
...
on WASH and Health; evaluation of drought response
...
focuses on FSL etc.). In some
...
contexts, the WASH coordination platform and UNICEF
...
may find relevant to organize a specific evaluation of the WASH sector response. This has to be differentiated from usual WASH
...
project evaluation, as it is about evaluating the WASH response as a whole, often including an evaluation of the WASH coordination system as well. In the key guidance and tools on top of this page can be found a general methodology to carry out evaluation of humanitarian action (2016 ALNAP Evaluation Humanitarian Action Guide). It is important to involve the GWC when organizing a WASH response evaluation, as it can lead to the production of GWC lesson learned or technical guidance documents (see Technical guidance chapter). Depending on the objective and the scope of the evaluation, several specific methodologies can be considered:
...
Evaluate response through key informants interview & documentary review
This is a complete evaluation of the whole WASH response at crisis/country level. It involves interviews and site visit of a representative sample of beneficiaries and WASH infrastructure for the whole WASH response. Unless the crisis and the response is very localized, this type of sector specific country wide evaluation is rare and challenging. It would involve considerable resources.
Systematic review of partner’s project
This can requires significant resources and time, as there are usually hundreds of different project implemented by partners, and tens of partners to interview. It is probably reasonable to focus this exercise only on CBPF/main donor funded projects.
...
common type of sector level evaluation. It consists in reviewing cluster's and main partners' projects documents, perform key informant interviews at capital and field level, review data from other clusters related to WASH such as Health and Nutrition, review recent coordination monitoring results and finally draw conclusion on the efficiency, quality and relevancy of the WASH response following certain agreed criteria. The DAC/OECD evaluation criteria, generally used by partners for their project evaluation, can also be used for sector evaluation: Relevance, Effectiveness, Efficiency, Impact, Sustainability (the later can be replaced in emergency by Localization and Humanitarian/Development Nexus).
Lead a systematic review of WASH partner’s projects evaluation (meta-evaluation)
WASH partners often implement internal or external evaluation of their
...
projects. A systematic review will gather all
...
partners' evaluation and extrapolate
...
their results to the whole response. This exercise
...
is feasible if:
...
- A significant number of partners have evaluated formally their projects
- Partners have used
...
- comparable evaluation criteria
- Evaluation are geographically complementary
...
- Evaluated projects followed HRP/SOF strategy
Consequently, it
...
is critical for the WASH coordination platform to
...
agree with the SAG on an harmonized evaluation protocol, setting up some minimum criteria partners should respect when carrying their evaluation (
...
similar evaluation criteria
...
Key informants interview & documentary review. This is probably the most common type of evaluation carried out at the level of the sector.
Ask participant to browse as an example in their in handouts the Yemen cholera response
- Example in handouts: impact of WASH on nutrition
Evaluation of WASH coordination
...
; robust methodology etc.).
Lead a response-wide field evaluation
This is a complete evaluation of the whole WASH sector response, involving interviews and site visit of a representative sample of beneficiaries and WASH infrastructures in the whole country or region where the response took place. Unless the crisis and the response is very localized, this type of sector specific country wide evaluation is rare and challenging, involving considerable resources.
Evaluate WASH coordination system
This is usually included in WASH response evaluation exercise, considering WASH coordination as one of the response inputs. The objective is to evaluate whether coordination has been efficient, and which
...
effect it had on the response.
...
Cluster Coordination Performance Monitoring guidance can be used, as well as other indicators such as "time for recruitment/deployment of coordination staff", value for money analysis, achievement of GWC minimum requirement for coordination,
...
...
and achievement of coordination core functions.
Coordinate WASH response impact measurement and research
First, have a discussion with participants on what are impact evaluations, giving some examples: Impact of WASH activities on nutrition, on morbidity & mortality, on people’s livelihood, on attendance to school.
Then let participants come up with ways of doing such evaluation:
- Internal or external evaluation
- HH based survey, with representative sampling
- Epidemiology study (especially done for outbreak such as cholera, hepatitis E…)
- Research project: several research designs exist, the main one being “randomized control trials” (blinded or not). Mention ethic issues of using control group.
- Systematic review of partner’s impact evaluation = evaluation of evaluation, to be able to generalize results to the whole population of a country or the planet
Although impact measurement in emergency are not a priority, coordination platforms can take some initiatives in recovery or preparedness phase to evaluate impact of the sector response, to strengthen impact evidences and improve future strategy. As for quality monitoring, one of the difficulty of the process is to be able to obtain representative data from the whole response, while impact evaluation from partners project will tend to be disconnected between them. To address this issue, the coordination platform can set up an impact evaluation framework for the response, to be used by partners when they will carry out evaluation activities. The framework can provide priority response aspects and zone to be evaluated, and propose standard evaluation methodology so meta-evaluation can be done.
...
Although not a priority in emergency, evaluate the impact of the WASH response can be critical to orientate and improve future responses. Obtain evidences of strong impact will also improve fundraising. Emergency WASH response's overall objective is usually to decrease morbidity & mortality rate, and in lesser extend, protect people’s livelihood, improve protection and child education. Measure the impact of WASH on these outcomes is possible but complex, as many confounding factors will influence them (seasonality of disease, level of education, healthcare environment, food security...). Accurate data are also difficult to obtain in emergency context. Impact measurement should involve setting up of complex research protocols, in partnership with universities and research institutes. It often involves the use of control groups (group of people similar to the one studied but who did not benefit from the intervention), which has some ethical implication, especially in emergency.