Lessons learned from evaluations of the impact of WFP programmes on moderate acute malnutrition in the Sahel
By Jonas Heirman, Mica Jenkins and Jennifer Rosenzweig
Jonas Heirman leads impact evaluation activities for the World Food Programme (WFP). Jonas has previously worked in research and evaluation capacities for the UK Department for International Development (DFID), the Institute of Development Studies (IDS) and European Centre for Development Policy Management (ECDPM).
Mica Jenkins is the Research and Evidence Officer for the Nutrition Division at WFP headquarters. She holds an MSc in Health and Human Development with a focus on nutrition and sustainable food systems. Prior to joining WFP, Mica implemented rural development and research projects in Mozambique and in the Dominican Republic.
Jennifer Rosenzweig is a senior development professional with 20 years’ experience working on nutrition and health issues for United Nations agencies and non-governmental organisations (NGOs). She has worked in the WFP Nutrition Division since 2013.
Location: Chad, Mali, Niger and Sudan
What we know: There is insufficient understanding of the relationship between moderate acute malnutrition treatment and prevention programmes.
What this article adds: World Food Programme (WFP) commissioned a four-country evaluation series in the Sahel, conducted by International Initiative for Impact Evaluation, to assess the relationship between treatment and prevention in emergency and post-emergency contexts. Key lessons included the importance of timing, sequencing and the roll-out schedule of an intervention package to enhance effectiveness; the benefit of close partnerships and coordination; understanding of barriers to coverage and access, including infrastructure and quality of services; and awareness of challenges related to monitoring and evaluation data. WFP has acted on practical recommendations on strategic relationships, communication with beneficiaries and communities, contextualised intervention packages, data sharing, and monitoring and evaluation systems as part of an ongoing process of learning and improvement.
Recent estimates show that more than one quarter of wasted children under five years old live in Africa (UNICEF, 2019). Among these 18.2 million children, 14 million suffer from moderate wasting. While there is evidence of the effectiveness of interventions to treat moderate acute malnutrition (MAM) (Lazzerini et al, 2013; Gera et al, 2017; Fabiansen et al, 2017), there remains insufficient understanding of the relationship between MAM treatment and prevention programmes. In response to this knowledge gap, World Food Programme (WFP) commissioned a four-country evaluation series in the Sahel to assess the relationship between treatment and prevention in emergency and post-emergency contexts (WFP, 2018).
The findings from this evaluation are relevant to the ongoing global effort to improve the continuum of care for acute malnutrition. In particular, the findings related to prevention programmes, through blanket supplementary feeding alone or in combination with food assistance for assets and/or targeted supplementary programmes, support a better understanding of specific mechanisms to reduce the prevalence and incidence of acute malnutrition.
Summary of the four evaluations
Although considerable evidence exists on the effectiveness of MAM interventions in controlled environments, more evidence is needed on the effectiveness of these interventions in the contexts in which WFP operates. To address these evidence needs, the MAM impact evaluation series was commissioned by WFP’s Office of Evaluation and managed by International Initiative for Impact Evaluation (3ie).
High rates of poverty and insecurity in the Sahel pose particularly significant challenges for nutrition interventions. Chad, Mali, Niger and Sudan were selected by WFP and ongoing programmes were paired with researchers. This approach created limitations for the feasibility of certain impact evaluation methods and no study could use a randomised controlled trial. Despite this, each study provided useful evidence for WFP’s nutrition programming. The following is a summary of each evaluation.
Targeted and blanket supplementary feeding programmes in Chad (Saboya et al, 2018)
The evaluation in Chad focused on the impact of WFP’s targeted supplementary feeding programme (TSFP) and blanket supplementary feeding programme (BSFP) in a region with high levels of MAM. BSFPs are a preventative intervention, whereas TSFPs are a MAM treatment intervention. The primary evaluation question was:
What is the impact of interventions for preventing moderate acute malnutrition on the incidence and prevalence of moderate acute malnutrition in children under two years with varying levels of access to moderate acute malnutrition treatment?
The evaluation used propensity-score matching to compare beneficiaries of the BSFP with non-recipients. Distance to nearest health centres was used to estimate the additional effect of access to the TSFP.
The evaluation found raw correlation between BSFP and MAM of 5 percentage point reduction (from 17% to 12%). Testing for significance, the evaluation found a 4.7 to 8 percentage point (comparing OLS, PROBIT, IPW and IPWRA models) reduction in propensity to become MAM in households benefitting from the BSFP, which is significant at the 5% level when accounting for false discoveries. It also found that the incidence of MAM is 3.6 percentage points lower (significant only in PROBIT model) when there is good access to the TSFP. Lack of reliable census data and maps meant that the identification of areas with good or poor access to TSFP had to be done directly in the field.
Food assistance for food-insecure populations during conflict in Mali (Gelli et al, 2018)
Mali faces significant development and security challenges. The impact evaluation examined the effects of general food distribution and school feeding on populations in conflict-affected areas. The primary evaluation questions were:
- What are the impacts of conflict and food assistance on child malnutrition and other development outcomes?
- What are the effects of food assistance on conflict-affected populations?
- What coping strategies were most effective in mitigating the effects of conflict?
- How did humanitarian aid influence the effectiveness of coping strategies at household and community level?
The evaluation relied on a longitudinal study of a population in the Mopti region. The population was subdivided according to: i) extent of humanitarian assistance received since conflict; and ii) extent to which areas were affected by conflict. Data was collected through two household surveys (2012 and 2017). WFP food-assistance activities in northern Mali during the period evaluated included BSFP, TSFP, general food distribution (GFD) and school feeding. The effect of food assistance on populations was analysed by combining propensity-score matching and difference-in-difference estimations.
The evaluation found no significant effects of food assistance on the prevalence of MAM in the population surveyed. However, all forms of assistance were found to increase the consumption of vitamin A and monthly food and non-food expenditures.
Treatment and prevention of moderate acute malnutrition in Niger (Brück et al, 2018)
In Niger WFP’s Protracted Relief and Recovery Operation (PRRO) employed a wide range of interventions, including targeted food assistance (TFA), blanket supplementary feeding (BSF), food assistance for assets (FFA), and targeted supplementary feeding (TSF). The impact evaluation assessed to what extent FFA in combination with other interventions is more effective at addressing MAM than prevention or treatment alone. The primary evaluation question was:
What are the impacts on nutrition outcomes of various combinations of programme components in WFP’s protracted relief and recovery operation?
The evaluation relied on quantitative and qualitative methods of analysis. Quantitative analyses relied on panel survey data collected in 2014 and 2016. To analyse impact, the evaluation used difference-in-difference analysis to compare three groups: i) people receiving FFA at baseline but no assistance by end-line; ii) people receiving FFA at baseline who stopped receiving FFA and instead received TFA, BSF and/or TSF at end-line; and iii) people receiving FFA at both baseline and end-line and receiving another intervention at end-line.
The evaluation found that children who received both FFA and treatment and/or prevention assistance at the end-line of the evaluation (group three) were 20% less likely to suffer MAM than children receiving no assistance at end-line (group one). Additionally, children in group three were 15.5% less likely to suffer from MAM than those in group two. These findings suggest the FFA is potentially a valuable addition to MAM interventions. However, data availability and the non-random assignment of individuals made the evaluators cautious not to overstate the validity of findings.
MAM treatment and prevention programmes in Sudan (Guevarra, 2017)
In Sudan WFP delivers both MAM treatment and prevention interventions. In 2015 WFP expanded the use of food-based prevention of MAM (FBMAM) in areas already receiving TSFP interventions. WFP has also explored the relative effectiveness of integrating food-based interventions with social and behaviour change communications (SBCC) and water, sanitation and hygiene (WASH) interventions. The primary evaluation question was:
What is the impact of different MAM treatment and prevention interventions on the incidence and prevalence of moderate and severe acute malnutrition in children under 5 years and pregnant and lactating women?
The evaluation relied on a stepped-wedge, cluster-controlled trial design that followed the expansion of FBMAM interventions in areas receiving TSFP. However, the assignment of clusters to individual expansion rounds was not random and expansion also faced implementation delays. In addition, the evaluation nested a two-arm, parallel cluster-controlled design to assess the prevalence of malnutrition in areas with treatment compared to those with both treatment and prevention.
The evaluation was unable to detect any significant difference in the prevalence of global acute malnutrition (GAM), MAM or severe acute malnutrition (SAM) in populations receiving prevention interventions. The importance of ensuring quality and delivery of services was highlighted as an area for future WFP attention. However, the evaluation did find up to a 12% reduction in the number of at-risk individuals after the fourth round of FBMAM interventions. The evaluators offer three interpretations for these findings: i) there is a time-lag between at-risk reduction and prevalence rates; ii) prevalence rates remained high in areas with prevention interventions because cases shifted from SAM back to MAM; or iii) prevention coverage was not high enough to reduce population-level prevalence rates. Also, no significant effects were found for SBCC interventions due to implementation challenges and low participation.
Lessons and recommendations from the MAM series (WFP, 2018)
The MAM series was particularly useful for WFP because it provided significant new evidence related to operational challenges and the impact of treatment and prevention interventions in the Sahel. A synthesis of the four evaluations identified the following lessons:
- Greater attention to the timing, sequencing and roll-out schedule of a package of interventions is likely to result in enhanced effectiveness.
- Closer partnerships and coordination can support more effective and efficient delivery.
- Barriers to achieving better coverage and access include, but are not limited to, infrastructure deficits. Awareness of the availability of quality services is also important.
- The quality, availability and lack of monitoring and evaluation (M&E) data are problematic.
These lessons informed the following recommendations for WFP:
- WFP should invest in the strategic deepening of its relationships with partners and stakeholders, providing capacity-strengthening where relevant;
- WFP interventions should pay greater attention to communication with target groups, effectiveness of case-finding, and community sensitisation, all of which may improve targeting, efficiencies and effectiveness of MAM treatment and prevention programmes;
- MAM programme components (e.g., packages) should be better tailored to context and may need to be integrated into longer-term, multi-sector solutions;
- Greater effort should be made to share data among agencies working towards Sustainable Development Goal 2 and mainstream compatibility of tools and systems;
- Greater support and attention are needed to improve the collection and use of monitoring and cost data.
These recommendations point to the importance of considering MAM treatment and prevention interventions along the continuum of care. The layering of treatment, prevention and other interventions such as FFA were found to have significant benefits in terms of effectiveness, depending on the combination and context. In addition, all studies highlighted the importance of improving the collection of comparable cost data to enable more strategic decision-making. After the synthesis in 2018 many of these lessons and recommendations were immediately taken onboard by WFP.
Acting on recommendations
The MAM series provided WFP with evidence and recommendations to support improvements in the planning and delivery of malnutrition treatment and prevention interventions. WFP Nutrition accepted all five recommendations made by the synthesis report and immediately began incorporating them into partnership, policies, guidance and support. The following are examples of how MAM series recommendations influenced WFP’s nutrition programming.
Recommendation 1: Deepen strategic relationships and support capacity-strengthening of partners
At the corporate level, the WFP Nutrition Policy (2017-2021) made partnerships a central priority and recognised the need for nutrition actors to work jointly to identify and improve upon delivery platforms to ensure that all vulnerable groups have access to healthy diets. WFP’s partners in nutrition are diverse and include national governments, other UN agencies, non-government organisations, academic institutions and the private sector, all of which bring important strengths to WFP’s nutrition programmes.
In 2018 WFP piloted its external digital learning platform (nutx), which is designed to share the organisation’s experiences and expertise in nutrition with external partners, including governments, UN agencies, national and international NGOs, academic institutions and private sector actors1. This platform offers a broad range of e-learning courses and activities aligned with WFP’s nutrition priorities, with the aim of promoting capacity-strengthening of partners and fostering the achievement of SDG2. nutx will be officially launched in quarter four of 2019.
Recommendation 2: Enhance communication with target groups and community sensitisation
The MAM series identified weaknesses in WFP’s communications, particularly in the implementation of SBCC. To address these and other weaknesses in SBCC across contexts, WFP is investing in SBCC in multiple ways. Since finalising and launching its comprehensive, field-oriented SBCC Guidance Manual for Nutrition programmes in 20182, WFP Nutrition has focused on country-level actions to improve the quality, reach and impact of its SBCC programmes, including training for over 350 staff and partners in five regions. As a result of the evaluation recommendation to review SBCC interventions, actions and delivery platforms in Sudan, WFP expanded its efforts to include a nutrition-oriented TV show, radio programme and the sending of bulk SMS messages to the target population. These innovative media-based approaches build on the traditional approaches of community mobilisation, nutrition or health days, care groups and targeted SBCC training to strengthen staff and partner capacity.
In Chad WFP is piloting a peer-to-peer model in remote areas through which ‘role mothers’ share good practices on hygiene and nutrition with a group of 12-15 women over 12 days. The model includes the use of ‘nutricards’ used by role mothers to teach other mothers how to prepare nutritious meals using local foods, as well as interactive games with their children to stimulate cognitive development. The approach builds community ownership and aims to improve the coverage and efficiency of the MAM treatment programme.
Recommendation 3: Tailor responses to context
WFP’s new country strategic plan (CSP) approach has encouraged every country office to ensure its programmes are tailored to country context and needs. In West Africa the formulation of the CSP process was used as an opportunity to embed nutrition-sensitive thinking into country portfolios. WFP’s Sahel resilience scale-up initiative embedded nutrition-sensitive thinking and actions into the selection of all activities. Findings from Niger on the benefits of layering FFA with other interventions also encouraged the integration of activities in the Sahel.
Nutrition analysis is being aided by WFP’s Fill the Nutrient Gap3 assessment, which reveals context-specific barriers to the consumption of healthy and nutritious foods and supports multi-sector decision-making on the prevention of malnutrition. The assessment focuses on gaps in nutrient intake to inform country-level policies and actions that can be taken to improve nutrition among the population, with a focus on the most vulnerable.
Recommendation 4: Improve inter-agency data collection methods and enhance data sharing
WFP is making significant efforts to improve the quality of nutrition data collected for monitoring and analysis. SCOPE CODA is WFP’s cloud-based innovation, with the potential to transform data management in malnutrition treatment programmes. The application gives a digital identity to patients and tracks healthcare services, replacing paper-and-pen records, ration and referral cards and reports in healthcare centres with a personalised smartcard linked to an electronic database. It reduces human error, streamlines data and improves the effectiveness of treatment of acute malnutrition. SCOPE CODA strengthens partner coordination by providing a unified platform and linking to existing systems for all stakeholders in community-based management of acute malnutrition (CMAM), including governments, UNICEF, international organisations and implementing partners, to access and share programme information. It is being developed as a single data system across treatment programmes for SAM (managed by UNICEF) and MAM (managed by WFP) and thus will result in better partner coordination, integrated data management and reduced loss at referral.
Recommendation 5: Improve use of monitoring and cost data
Although an effort was made to collect and analyse cost data in this evaluation series, the findings were inconclusive and point to a need for more rigorous cost studies. WFP is improving the quality and usefulness of nutrition intervention monitoring and cost data. To improve cost analysis, WFP recently launched a new costing tool that supports countries to collect comparable data on interventions. The tool uses both monitoring and financial data from within WFP to generate estimates for MAM treatment programming, using a globally standardised methodology. This ensures consistency in both data sourcing and calculations across countries to enable global comparison. Furthermore, the tool analyses additional information, such as cost drivers, to support programmers in the field as they work to improve cost-effectiveness over time.
The tool will be used to estimate the cost to WFP to treat a child for MAM. With partner collaboration it can also be used to determine the full cost to treat a child for MAM. The tool is internal as it is built around WFP data systems in order to minimise burden as well as encourage consistency between countries. However, the methodology and results of the tool will be available to partners.
The findings from the MAM impact evaluation series are supporting a process of learning and improvement in WFP. The synthesis of these evaluations provided practical recommendations that are currently guiding WFP’s efforts to improve acute malnutrition treatment and prevention interventions. These improvements also provide new opportunities to test theories and generate new evidence to answer questions raised by the MAM evaluation series.
For more information please contact Jonas Heirman.
1nutx is accessible at: https://cdn.wfp.org/nutrition/nutx/
2Social and Behaviour Change Communication: Guidance Manual for WFP Nutrition. Available at: https://docs.wfp.org/api/documents/WFP-0000102103/download/
Brück T, Ferguson NTN, Ouédraogo J and Ziegelhöfer Z, 2018. Impacts of the World Food Programme’s interventions to treat malnutrition in Niger. 3ie Impact Evaluation Report 80. New Delhi: International Initiative for Impact Evaluation (3ie). Available at: doi: https://doi.org/10.23846/tw6IE80
Fabiansen C, Yaméogo CW, Iuel-Brockdorf A-S, Cichon B, Rytter MJH, Kurpad A et al, 2017. Effectiveness of food supplements in increasing fat-free tissue accretion in children with moderate acute malnutrition: A randomised 2 × 2 × 3 factorial trial in Burkina Faso. PLoS Med 14(9): e1002387. https://doi.org/10.1371/journal.pmed.1002387
Gelli A, Tranchant J-P, Bliznashka L, Diallo AS, Sacko M, Siegel E, Aurino E and Masset E, 2018. The impact of food assistance on food insecure populations during conflict in Mali. 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie).
Gera T, Pena-Rosas JP, Boy-Mena E, Sachdev HS (2017) Lipid based nutrient supplements (LNS) for treatment of children (6 months to 59 months) with moderate acute malnutrition (MAM): A systematic review. PLoS ONE 12(9): e0182096. https://doi.org/10.1371/journal.pone.0182096
Guevarra E. Mandalazi S. Belagamire K. Albrekstan K. Sadler K. Abdelsalam K, Urrea G and Alawad S, 2018. Impact evaluation of the World Food Programme’s moderate acute malnutrition treatment and prevention programmes in Sudan. 3ie Impact Evaluation Report 79. New Delhi: International Initiative for Impact Evaluation (3ie). Available at: doi.org/10.23846/tw6IE79
Lazzerini M, Rubert L, Pani P, 2013. Specially formulated foods for treating children with moderate acute malnutrition in low- and middle-income countries. Cochrane Database of Systematic Reviews 2013, Issue 6. Art. No.: CD009584. DOI: 10.1002/14651858.CD009584.pub2.
Saboya M et al, 2018. Impact evaluation of WFP’s programs targeting moderate acute malnutrition in humanitarian situations in Chad. Impact Evaluation Report 78. 3ie: https://assets.publishing.service.gov.uk/media/5b73fb1440f0b6139e05a458/ie78-mam-chad.pdf
UNICEF, WHO and World Bank (2019) Levels and trends in child malnutrition: key findings of the 2019 Edition of the Joint Child Malnutrition Estimates. Geneva: World Health Organization; 2019 Licence: CC BY-NC-SA 3.0 IGO.
WFP, 2018. Synthesis report on four evaluations of the impact of WFP programmes on nutrition in humanitarian contexts in the Sahel. https://docs.wfp.org/api/documents/80ec7f0c-bd3b-4689-ad2e-c5254e98066b/download/?_ga=2.171021408.1395824924.1552663230-595083097.152941453
More like this
FEX: MAM and SAM cases reduced through a stunting prevention programme in Malawi and the associated costs averted
View this article as a pdf Lisez cet article en français ici By Jackson Mason-Mackay, Arghanoon Farhikhtah, Mica Jenkins, Trust Takudzwa Mlambo, Emma Lefu and Nancy...
FEX: Addressing acute malnutrition in Cameroon during an emergency: Results and benefits of an integrated prevention programme
View this article as a pdf Lisez cet article en français ici By Eveline Ngwenyi, Mica Jenkins, Nicolas Joannic and Cécile Patricia Eveline Ngwenyi is a...
View this article as a pdf Lisez cet article en français ici By Jeremy Shoham and Marie McGrath, Field Exchange Co-Editors Rationale for FEX special edition We are...
View this article as a pdf In this issue of Field Exchange we are delighted to feature, for the second year running, a special section that shares key outputs of Action...
FEX: Enhancing the effectiveness of a community-based management of acute malnutrition programme in Zambia
View this article as a pdf Lisez cet article en français ici By Stefania Moramarco with technical input from Giulia Amerio, Elisabetta Garuti, Gloria Gozza, Ersilia...
Terms of Reference: Nutrition Advisor/World Food Prorgamme/Kinshasa, DRC Background Despite large investment and government renewed committed to combat malnutrition, the...
FEX: SAM and MAM programming in East and West Africa: An insight into continuum of service provision for acute malnutrition treatment
View this article as a pdf By Rebecca Brown, Kate Sadler, Tanya Khara, Marie McGrath and Jeremy Shoham Rebecca Brown is an experienced public health nutritionist engaged as...
FEX: UNHCR experiences of enabling continuity of acute malnutrition care in the East, Horn of Africa and Great Lakes region
View this article as a pdf Lisez cet article en français ici By Naser Mohmand Naser Mohmand is currently working with UNHCR as Senior Regional Nutrition and Food...
Summary of published1 and unpublished research By Beatrice Lorge Rogers (Principal Investigator), Patrick Webb (Principal Investigator), Jocelyn Boiteau (Project...
Resource: A mapping exercise and online survey to investigate continuity of care in acute malnutrition treatment in East Africa
Between January and June 2019, Emergency Nutrition Network (ENN) undertook a mapping exercise and a review of current practice to gain an insight into the extent to which...
FEX: SCOPE CODA: World Food Programme innovation to improve data management in malnutrition treatment
View this article as a pdf The World Food Programme (WFP) has introduced a cloud-based innovation named 'SCOPE CODA' to improve data management in malnutrition treatment...
FEX: Food aid for nutrition: A landscape review of current research and implications for future studies
View this article as a pdf By Maria Wrabel, Kristine Caiafa, Beatrice Lorge Rogers and Patrick Webb All authors are affiliated with the Food Aid Quality Review (FAQR) project...
The WHO, in collaboration with UNICEF, WFP and UNHCR, hosted a second consultation to discuss the programmatic aspects of the management of moderate malnutrition in children...
Resource: A mapping exercise and online survey to investigate continuum of care for acute malnutrition in West and Central Africa
Between January and June 2019, Emergency Nutrition Network (ENN) undertook a mapping exercise and a review of current practice to gain an insight into the extent to which...
View this article as a pdf By Tracy Dube, Mary Chelang'at Koech, Piyali Mustaphi, Sandra Harlass, Jessica Bourdaire and Karanveer Singh Tracy Chiridza Dube is a Public...
FEX: Scaling-up of care for children with acute malnutrition during emergency nutrition response in South Sudan between 2014 and 2018
View this article as a pdf Lisez cet article en français ici By Dina Aburmishan, Vilma Tyler, Lucas Alamprese, Priscilla Bayo Nicholas, Marie Darline Raphael, Joseph...
By Jutta Neitzel TSF distribution in Degehabur, Somali Region Jutta Neitzel is the Head of the Nutrition and Education Section at WFP Ethiopia. She has worked for the...
View this article as a pdf Lisez cet article en français ici By Mahmoud Bozo, Hala Khudari and Mutasem Mohammad Mahmoud Bozo is a World Health Organization (WHO)...
FEX: Implementation of the Expanded Admission Criteria (EAC) for acute malnutrition in Somalia: interim lessons learned
View this article as a pdf Lisez cet article en français ici By John Ntambi, Pramila Ghimire, Ciara Hogan, Madina Ali Abdirahman, Dorothy Nabiwemba, Abdiwali Mohamed...
FEX: Leçons apprises grâce aux évaluations d’impact des programmes du PAM pour lutter contre la malnutrition aiguë modérée dans le Sahel
Read an English version of this article here Par Jonas Heirman, Mica Jenkins et Jennifer Rosenzweig Jonas Heirman est responsable des activités d'évaluation...
Reference this page
Jonas Heirman, Mica Jenkins and Jennifer Rosenzweig (2019). Lessons learned from evaluations of the impact of WFP programmes on moderate acute malnutrition in the Sahel. Field Exchange issue 60, July 2019. p107. www.ennonline.net/fex/60/impactwfpprogrammessahel