Independent Evaluation of the CMAM Model Surge Pilot
Author: Peter Hailey (Director Centre for Humanitarian Change) for Concern Worldwide
Year: 2015
Resource type: Other
Executive Summary
This report is the result of an Independent Evaluation of the Pilot CMAM Surge Model project conducted in two subcounties of Marsabit between May 2012 and October 2014.
1.1 BACKGROUND
In May 2011, the president of Kenya declared the drought of 2010/2011 a national disaster1 . It is estimated that the 2011 drought in Kenya resulted in an estimated 3.75 million Kenyans and 500,000 refugees requiring food aid, while over 300,000 children were affected by acute malnutrition2 . The worst affected were the Arid and Semi- Arid Lands (ASALs) of north and north-eastern Kenya, where rates of global acute malnutrition in some areas vastly exceeded emergency thresholds. Concern Worldwide (Concern) was part of the humanitarian response in Marsabit County, one of the areas that was affected by the drought. A post analysis of the response by Concern and Sub County Health Management Teams (SCHMTs) revealed that there was a lack of pre-emergency planning (despite slow onset of the emergency and early warning); limited use of available data and contextual analysis; and, limited understanding of how and when to scale up interventions3. These lessons as well as the release of the “Suggested New Design Framework for CMAM programming”4 , prompted Concern and SCHMT to develop the CMAM surge model.
1.2 THE CMAM SURGE MODEL PILOT
The aim of the CMAM surge model is to strengthen the capacity of government health systems to effectively manage increased caseloads of severe acute malnutrition (SAM) and moderate acute malnutrition (MAM), during predictable emergencies without undermining ongoing health and nutrition systems strengthening efforts. It is based on one of the fundamental principles of CMAM; that early detection of malnutrition leads to improved treatment outcomes and fewer cases of SAM, as children are treated before their malnutrition becomes severe.
The pilot project was initiated by Concern in collaboration with the SCHMT as well as health facility staff in May 2012, in 14 health facilities drawn from Moyale, Chalbi and Sololo (Moyale and North Horr Sub-Counties) in Marsabit County. This pilot project was part of a larger ECHO funded project the ‘Marsabit County Emergency Recovery Project (March 2012 to February 2013)’ whose aim was to assist the two SCHMTs in Moyale and North Horr to strengthen their contingency planning capacity by February 2013.
Concern designed the Integrated Management of Acute Malnutrition (IMAM) Surge Model to enable a health system to cope with spikes in cases of acute malnutrition. The pilot has been conducted in 14 health facilities and the pilot programme was initiated in May 2012. Operational feedback has shown that the model is technically feasible generating interest from the Ministry of Health and other stakeholders, with regard to rolling out the approach in a wider area of Kenya, with a view to making the Model part of the health system in the fragile areas of Kenya.
1.3 EVALUATION OBJECTIVES
Therefore, it was agreed that an evaluation of the model be carried out prior to any scale up of the model. The evaluation aims to
• Examine if the model works in the way that it had been conceived,
• Share lessons learnt as others implement the model.
Should the evaluation recommend further scale up as part of the process to prepare the scale up it is envisaged that a manual and other tools including a costed budget for scale up will be developed.
The principal evaluation question is:
Can the IMAM Surge Model strengthen the health system to manage increased caseloads of acute malnutrition during predictable emergencies without undermining ongoing health systems strengthening efforts?
The evaluation is based around Concern’s ongoing programme in Chalbi, Moyale and Sololo in Marsabit County, where the model has been implemented for 29 months in 14 pilot health facilities. These facilities provide an essential package of health and nutrition services including IMAM.
The objectives of the evaluation are as follows:
•. To determine whether the model is effective in setting realistic threshold levels and whether the interventions proposed take place and are appropriate when thresholds are reached
• To determine whether the model positively or negatively influences other health system activities (facility and district level)
•. To determine the acceptability of the model to the various stakeholders
•. To determine whether the model is more cost-effective than previous standard practice of external non-integrated support
•. To determine the sustainability of the model
• To share lessons learned with involved stakeholders
Concern has defined the IMAM Surge Model as “an innovation that enables the health system to predict and cope with surges in cases of acute malnutrition through the setting of caseload thresholds and a set of phased actions to respond flexibly to a threshold being met”.
This definition and the ideas framed in the main evaluation question indicate that there are two main objectives of the IMAM Surge Model;
• Strengthening the health system to manage periodic surges in caseloads of acute malnutrition,
• Support the health system to predict, and plan to respond to periodic and predictable surges in caseloads of acute malnutrition.
I.e. a planning and preparedness objective and a response objective.
The evaluation has reviewed both aspects of the model.
1.4 METHODOLOGY
The review used a mixed methods design. Methods included key informant interviews and focus group discussions at health facility, sub-county and county level. Selected key informant interviews were also conducted at National Level. A desk review of relevant internal and external data and documents was also conducted. Nine Health Facilities were visited, three in Chalbi, two in Moyale, three in Sololo (1 an outreach site) and one in Marsabit Central.
Description of Data.
Four admissions for morbidities were monitored throughout 2012 and 2013;
• Severe Acute Malnutrition (SAM) admissions to Out-patient Therapeutic Programmes (OTPs),
• Moderate Acute Malnutrition (MAM) admissions to Targeted Supplementary Feeding Programmes (TSFPs),
• Diarrhoea admissions, and
• Pneumonia admissions5.
Overall, admissions of all four morbidities were higher in 2014 than 2013 and the three year average. Nutrition admissions showed no seasonal pattern and surges in numbers of children admitted seemed to be mostly related to programming issues such as mass screenings or local conflict causing rapid in-flows and out-flows of malnourished children. On the other hand diarrhoea and pneumonia admission do show pronounced seasonal patterns with diarrhoea in particular being related to both rainy seasons and pneumonia increases related to the long rains. Significant changes in admissions of these two morbidities do not appear to be related to programme issues, local conflict, changes in malnutrition admissions or other issues such as transhumance.
1.5 EVALUATION RESULTS MATRIX AND FINDINGS
The findings of the evaluation and an overall ranking have been summarized in the tables below. The ranking system used is as follows:
1. Poor- Highly non satisfactory
2. Fair- Non satisfactory
3. Good - Moderately satisfactory
4. Very good - Satisfactory
5. Excellent - Highly satisfactory
The principal evaluation question was posed as follows:
Can the IMAM Surge Model strengthen the health system to manage increased caseloads of acute malnutrition during predictable emergencies without undermining ongoing health systems strengthening efforts?
Overall the evaluation rated the Surge Model Pilot to be 4. VERY GOOD – SATISFACTORY. The pilot was able to show that it has contributed to strengthening the health system to increased caseloads of acute malnutrition during predictable AND un-predictable emergencies without undermining ongoing health system strengthening efforts.
Therefore, the evaluation recommends further scale up within the pilot sub-counties and at a wider scale in Kenya and elsewhere.
1.6 RECOMMENDATIONS
The report has reviewed each aspect of the pilot project in detail and has made several recommendations that may be taken into account in the next phases of the scale up of the Surge Model. The recommendations can be found throughout the report. The most notable recommendations for the next phase are as follows:
• As the pilot is scaled up to full sub-counties and counties in Kenya, specific attention should be given to establishing the lead and ownership of the County Health Management Team. This will be of particular importance for issues of human resources and supply responses to triggers, especially for the higher thresholds.
• The sustainability of the financing of the Surge Model should be given particular attention. It appears that the specific surge costs of responding to all but the largest surges are low, as many of the health systems strengthening activities contribute to surge responses. It is suggested that the Surge Model can support Health System Resilience in areas where shocks and stresses are common. In this case, the use of the Surge Model approach to develop Health System costed contingency plans based on internal capacity assessment could represent a low cost and adapted approach to dealing with the Health Systems need to be resilient in the face of constant shocks and stresses experienced in these areas. The data collected during the pilot period also suggest that the approach could be extended to other morbidities such as diarrhoea that put a strain on the capacity of the health system when stresses and shocks occur.
• In the shorter term it is important that the next phase of the Surge Model better demonstrates its efficiency (especially financial) both as an alternative to episodic injections of emergency aid and as a system that develops the capacity of the health system to respond better to and cope with emergencies. For fund raising and support from Government’s Health Systems and International donor’s a clear demonstration of this new approaches’ value for money and relevance is urgently required so that it can replace the more traditional approach to nutrition and health emergencies.
• The noted significant improvements in the Health Facilities use of data for planning and management of nutrition programmes and effective communications between levels of the health system could be duplicated at the SCHMT and CHMT levels. Adaptation of the tools and threshold approaches to monitoring challenges to the health systems capacity would allow the County Management Team to adapt and focus their response to shocks based on a real time analysis of where the needs for what support are and when.
• The use of capacity based thresholds to clarify when, where and what external health system support is needed can be extended to clarify the linkage between the Health Systems response to shocks and when there is a need for further support from NDMA, and other external emergency resources, in response to an extraordinary and rare surge in needs.
• The study noted that despite improved coverage there remains a delink between the numbers predicated by nutrition surveys and the actual numbers of children admitted to nutrition treatment programmes in the health system. Thus fewer acutely malnourished children are managed by the health system than would be forecast by the survey. It is suggested that a distinction is made between a Health System emergency and an emergency indicated by a nutrition survey or early warning. Therefore, it is suggested that the Surge Model approach is used to plan, predict and provide additional resources to the health system to mitigate the possibilities of health system
emergencies. Nutrition surveys and related early warning would then be used to identify the very rare extraordinary emergencies that require significant external resources. The border between the two would be established by the Health Systems regular analysis of its capacity to cope with surges in need. Thus the threshold for investment of external resources would be set based on each counties health systems own analysis of its ability to cope. The threshold would change over time, hopefully upwards, as the County Health System increases its capacity through support to Health System Strengthening.
Original link to resource here.
1 Food Assistance Integrity Study - Analysis of the 2011 drought response in Kenya; Transparency International 2012
2 The Africa Portal Backgrounder series, No.33 > July 2012
3 Regine Kopplow, Yacob Yishak, Gabrielle Appleford and Wendy Erasmus (2014). Meeting demand peaks for CMAM in government health services in Kenya. Field Exchange 47, April 2014.
p3. www.ennonline.net/fex/47/meeting
4 Peter Hailey and Daniel Tewoldeberha (2010). Suggested New Design Framework for CMAM Programming. Field Exchange 39, September 2010. p41. www.ennonline.net/fex/39/suggested
5 Throughout the report the term pneumonia has been used as a synonym for respiratory infections rather than referring only to the official definition of pneumonia.
Download
evaluation_of_the_cmam_model_surge_pilot_2015.pdf (PDF, 2.5mb)
More like this
FEX: Meeting demand peaks for CMAM in government health services in Kenya
By Regine Kopplow, Yacob Yishak, Gabrielle Appleford and Wendy Erasmus Regine Kopplow is a Global Nutrition Advisor with Concern Worldwide. She previously worked in various...
FEX: Implementing the IMAM Surge approach - experiences from Kenya
View this article as a pdf By Weldon Ngetich, Grace Gichohi, Francis Wambua, Tewoldeberhan Daniel, Yacob Yishak and Patrick Codjia Weldon Ngetich is a Nutrition Specialist...
en-net: Consultancy to carry out CMAM Surge Approach Cost Effectiveness Analysis
Background Concern Worldwide developed the CMAM Surge Approach to support health systems to become more resilient to the shocks and stresses that regular surges in the demand...
en-net: Consultancy for CMAM Surge Approach Learning Review and Evaluation
Background Concern Worldwide developed the CMAM Surge Approach to support health systems to become more resilient to the shocks and stresses that regular surges in the demand...
NEX: Meeting demand peaks for CMAM in government health services in Kenya
Regine Kopplow et al. Field Exchange 47, p3 The humanitarian community and national governments increasingly recognise that early warning and response are more effective and...
en-net: Appel aux Conseillers pour l'adaptation de l'approche CMAM Surge (augmentation de la prise en charge communautaire de la malnutrition aiguë sévère) au contexte Éthiopien.
Background Concern Worldwide developed the CMAM Surge Approach to support health systems to become more resilient to the shocks and stresses that regular surges in the demand...
FEX: CMAM Surge: lessons learned on the journey so far
View this article as a pdf By Erin McCloskey, Amanda Yourchuck and Peter Hailey Erin McCloskey is a Public Health Nutritionist and independent consultant currently working...
en-net: Health and Nutrition EU Aid Volunteer Niger
• Job Title: Health and Nutrition EU Aid Volunteer • Location: Niamey • Based in Country: Niger • Closing Date: 19th April 2019 The EU Aid Volunteers...
FEX: Expanding CMAM Surge beyond nutrition – towards a broader Health Surge approach
View this article as a pdf By Erin McCloskey, Kate Golden and Amanda Yourchuck Erin McCloskey is a Public Health Nutritionist and independent consultant currently working for...
FEX: CMAM Surge: the way forward
View this article as a pdf By Kate Golden and Sophie Whitney Kate Golden is Senior Nutrition Adviser at Concern Worldwide. Her career started in Ethiopia in 2003 in one of...
FEX: The ‘CMAM Surge’ approach: setting the scene
View this article as a pdf By Amanda Yourchuck and Kate Golden Amanda Yourchuck is an Emergency Nutrition Advisor for the United States Agency for International Development...
FEX: Evaluation of mobile application to support the treatment of acutely malnourished children in Wajir county, Kenya
Summary of presentation1 View this article as a pdf By Emily Keane, Natalie Roschnik, Joanne Chui, Ibrahim Ahmed Osman and Hassan Mohamed Osman Emily Keane is a Nutrition...
FEX: Transitioning nutrition programming from humanitarian aid to health system strengthening in Kenya
View this article as a pdf Lisez cet article en français ici By Peter Hailey, Brenda Akwanyi, Emiliana Mbelenga and Tamsin Walters Peter Hailey is the Director of the...
FEX: Scale up of high-impact nutrition interventions in the informal settlements of Nairobi, Kenya
View this article as a pdf Lisez cet article en français ici By Kassim Lupao and Esther Mogusu Kassim Lupao is a senior manager for the Nairobi urban health and...
FEX: CMAM Surge: understanding costs and potential contribution to CMAM’s cost-effectiveness
View this article as a pdf By Kate Golden and Simone van Dijk Kate Golden is Senior Nutrition Adviser at Concern Worldwide. Her career started in Ethiopia in 2003 in one of...
FEX: Timely expansion of nutrition development activities in repose to an acute flooding emergency in Malawi
By Natasha Lelijveld, Elizabeth Molloy, Jennifer Weiss, Gwyneth Hogley Cotes Natasha Lelijveld has recently completed a nutrition research project in Blantyre, Malawi and is...
FEX: The role of coordination in CMAM Surge scale-up in West and Central Francophone Africa
View this article as a pdf Lisez cet article en français ici By Diane Moyer, Amanda Yourchuck and Patricia Hoorelbeke Diane Moyer is the current Regional Humanitarian...
FEX: Developing a mobile health app to manage acute malnutrition: a five-country experience
By Tine Frank, Emily Keane, Natalie Roschnik, Colleen Emary, Melani O'Leary and Laura Snyder View this article as a pdf Tine Frank is a consultant engaged by Save the...
FEX: Report on innovations in CMAM
By Anne Marie Kueter, Claudine Prudhon, Emily Keane and Megan Gayford The implementation of community-based management of acute malnutrition (CMAM) as the standard model of...
FEX: Humanitarian-development nexus: nutrition policy and programming in Kenya
By Carmel Dolan and Jeremy Shoham View this article as a pdf Carmel Dolan is an ENN Technical Director, co-editor of the ENN publication Nutrition Exchange, and co-lead on...
Reference this page
Peter Hailey (Director Centre for Humanitarian Change) for Concern Worldwide (2015). Independent Evaluation of the CMAM Model Surge Pilot. www.ennonline.net/cmammodelsurgepilotevaluation
(ENN_6910)