• Facebook
• Twitter

Independent Evaluation of the CMAM Model Surge Pilot

Author: Peter Hailey (Director Centre for Humanitarian Change) for Concern Worldwide
Year: 2015
Resource type: Other

Executive Summary

This report is the result of an Independent Evaluation of the Pilot CMAM Surge Model project conducted in two subcounties of Marsabit between May 2012 and October 2014.

1.1 BACKGROUND

In May 2011, the president of Kenya declared the drought of 2010/2011 a national disaster¹ . It is estimated that the 2011 drought in Kenya resulted in an estimated 3.75 million Kenyans and 500,000 refugees requiring food aid, while over 300,000 children were affected by acute malnutrition² . The worst affected were the Arid and Semi- Arid Lands (ASALs) of north and north-eastern Kenya, where rates of global acute malnutrition in some areas vastly exceeded emergency thresholds. Concern Worldwide (Concern) was part of the humanitarian response in Marsabit County, one of the areas that was affected by the drought. A post analysis of the response by Concern and Sub County Health Management Teams (SCHMTs) revealed that there was a lack of pre-emergency planning (despite slow onset of the emergency and early warning); limited use of available data and contextual analysis; and, limited understanding of how and when to scale up interventions³. These lessons as well as the release of the “Suggested New Design Framework for CMAM programming”⁴ , prompted Concern and SCHMT to develop the CMAM surge model.

1.2 THE CMAM SURGE MODEL PILOT

The aim of the CMAM surge model is to strengthen the capacity of government health systems to effectively manage increased caseloads of severe acute malnutrition (SAM) and moderate acute malnutrition (MAM), during predictable emergencies without undermining ongoing health and nutrition systems strengthening efforts. It is based on one of the fundamental principles of CMAM; that early detection of malnutrition leads to improved treatment outcomes and fewer cases of SAM, as children are treated before their malnutrition becomes severe.

The pilot project was initiated by Concern in collaboration with the SCHMT as well as health facility staff in May 2012, in 14 health facilities drawn from Moyale, Chalbi and Sololo (Moyale and North Horr Sub-Counties) in Marsabit County. This pilot project was part of a larger ECHO funded project the ‘Marsabit County Emergency Recovery Project (March 2012 to February 2013)’ whose aim was to assist the two SCHMTs in Moyale and North Horr to strengthen their contingency planning capacity by February 2013.

Concern designed the Integrated Management of Acute Malnutrition (IMAM) Surge Model to enable a health system to cope with spikes in cases of acute malnutrition. The pilot has been conducted in 14 health facilities and the pilot programme was initiated in May 2012. Operational feedback has shown that the model is technically feasible generating interest from the Ministry of Health and other stakeholders, with regard to rolling out the approach in a wider area of Kenya, with a view to making the Model part of the health system in the fragile areas of Kenya.

1.3 EVALUATION OBJECTIVES

Therefore, it was agreed that an evaluation of the model be carried out prior to any scale up of the model. The evaluation aims to
• Examine if the model works in the way that it had been conceived,
• Share lessons learnt as others implement the model.

Should the evaluation recommend further scale up as part of the process to prepare the scale up it is envisaged that a manual and other tools including a costed budget for scale up will be developed.

The principal evaluation question is:
Can the IMAM Surge Model strengthen the health system to manage increased caseloads of acute malnutrition during predictable emergencies without undermining ongoing health systems strengthening efforts?

The evaluation is based around Concern’s ongoing programme in Chalbi, Moyale and Sololo in Marsabit County, where the model has been implemented for 29 months in 14 pilot health facilities. These facilities provide an essential package of health and nutrition services including IMAM.

The objectives of the evaluation are as follows:
•. To determine whether the model is effective in setting realistic threshold levels and whether the interventions proposed take place and are appropriate when thresholds are reached
• To determine whether the model positively or negatively influences other health system activities (facility and district level)
•. To determine the acceptability of the model to the various stakeholders
•. To determine whether the model is more cost-effective than previous standard practice of external non-integrated support
•. To determine the sustainability of the model
• To share lessons learned with involved stakeholders

Concern has defined the IMAM Surge Model as “an innovation that enables the health system to predict and cope with surges in cases of acute malnutrition through the setting of caseload thresholds and a set of phased actions to respond flexibly to a threshold being met”.

This definition and the ideas framed in the main evaluation question indicate that there are two main objectives of the IMAM Surge Model;

• Strengthening the health system to manage periodic surges in caseloads of acute malnutrition,
• Support the health system to predict, and plan to respond to periodic and predictable surges in caseloads of acute malnutrition.

I.e. a planning and preparedness objective and a response objective.

The evaluation has reviewed both aspects of the model.

1.4 METHODOLOGY

The review used a mixed methods design. Methods included key informant interviews and focus group discussions at health facility, sub-county and county level. Selected key informant interviews were also conducted at National Level. A desk review of relevant internal and external data and documents was also conducted. Nine Health Facilities were visited, three in Chalbi, two in Moyale, three in Sololo (1 an outreach site) and one in Marsabit Central.

Description of Data.

Four admissions for morbidities were monitored throughout 2012 and 2013;
• Severe Acute Malnutrition (SAM) admissions to Out-patient Therapeutic Programmes (OTPs),
• Moderate Acute Malnutrition (MAM) admissions to Targeted Supplementary Feeding Programmes (TSFPs),
• Diarrhoea admissions, and
• Pneumonia admissions⁵.

Overall, admissions of all four morbidities were higher in 2014 than 2013 and the three year average. Nutrition admissions showed no seasonal pattern and surges in numbers of children admitted seemed to be mostly related to programming issues such as mass screenings or local conflict causing rapid in-flows and out-flows of malnourished children. On the other hand diarrhoea and pneumonia admission do show pronounced seasonal patterns with diarrhoea in particular being related to both rainy seasons and pneumonia increases related to the long rains. Significant changes in admissions of these two morbidities do not appear to be related to programme issues, local conflict, changes in malnutrition admissions or other issues such as transhumance.

1.5 EVALUATION RESULTS MATRIX AND FINDINGS

The findings of the evaluation and an overall ranking have been summarized in the tables below. The ranking system used is as follows:

1. Poor- Highly non satisfactory
2. Fair- Non satisfactory
3. Good - Moderately satisfactory
4. Very good - Satisfactory
5. Excellent - Highly satisfactory

The principal evaluation question was posed as follows:

Can the IMAM Surge Model strengthen the health system to manage increased caseloads of acute malnutrition during predictable emergencies without undermining ongoing health systems strengthening efforts?

Overall the evaluation rated the Surge Model Pilot to be 4. VERY GOOD – SATISFACTORY. The pilot was able to show that it has contributed to strengthening the health system to increased caseloads of acute malnutrition during predictable AND un-predictable emergencies without undermining ongoing health system strengthening efforts.

Therefore, the evaluation recommends further scale up within the pilot sub-counties and at a wider scale in Kenya and elsewhere.

1.6 RECOMMENDATIONS

The report has reviewed each aspect of the pilot project in detail and has made several recommendations that may be taken into account in the next phases of the scale up of the Surge Model. The recommendations can be found throughout the report. The most notable recommendations for the next phase are as follows:

• As the pilot is scaled up to full sub-counties and counties in Kenya, specific attention should be given to establishing the lead and ownership of the County Health Management Team. This will be of particular importance for issues of human resources and supply responses to triggers, especially for the higher thresholds.

• The sustainability of the financing of the Surge Model should be given particular attention. It appears that the specific surge costs of responding to all but the largest surges are low, as many of the health systems strengthening activities contribute to surge responses. It is suggested that the Surge Model can support Health System Resilience in areas where shocks and stresses are common. In this case, the use of the Surge Model approach to develop Health System costed contingency plans based on internal capacity assessment could represent a low cost and adapted approach to dealing with the Health Systems need to be resilient in the face of constant shocks and stresses experienced in these areas. The data collected during the pilot period also suggest that the approach could be extended to other morbidities such as diarrhoea that put a strain on the capacity of the health system when stresses and shocks occur.

• In the shorter term it is important that the next phase of the Surge Model better demonstrates its efficiency (especially financial) both as an alternative to episodic injections of emergency aid and as a system that develops the capacity of the health system to respond better to and cope with emergencies. For fund raising and support from Government’s Health Systems and International donor’s a clear demonstration of this new approaches’ value for money and relevance is urgently required so that it can replace the more traditional approach to nutrition and health emergencies.

• The noted significant improvements in the Health Facilities use of data for planning and management of nutrition programmes and effective communications between levels of the health system could be duplicated at the SCHMT and CHMT levels. Adaptation of the tools and threshold approaches to monitoring challenges to the health systems capacity would allow the County Management Team to adapt and focus their response to shocks based on a real time analysis of where the needs for what support are and when.

• The use of capacity based thresholds to clarify when, where and what external health system support is needed can be extended to clarify the linkage between the Health Systems response to shocks and when there is a need for further support from NDMA, and other external emergency resources, in response to an extraordinary and rare surge in needs.

• The study noted that despite improved coverage there remains a delink between the numbers predicated by nutrition surveys and the actual numbers of children admitted to nutrition treatment programmes in the health system. Thus fewer acutely malnourished children are managed by the health system than would be forecast by the survey. It is suggested that a distinction is made between a Health System emergency and an emergency indicated by a nutrition survey or early warning. Therefore, it is suggested that the Surge Model approach is used to plan, predict and provide additional resources to the health system to mitigate the possibilities of health system
emergencies. Nutrition surveys and related early warning would then be used to identify the very rare extraordinary emergencies that require significant external resources. The border between the two would be established by the Health Systems regular analysis of its capacity to cope with surges in need. Thus the threshold for investment of external resources would be set based on each counties health systems own analysis of its ability to cope. The threshold would change over time, hopefully upwards, as the County Health System increases its capacity through support to Health System Strengthening.

Original link to resource here.

Read more...

Download

evaluation_of_the_cmam_model_surge_pilot_2015.pdf (PDF, 2.5mb)

Close

Reference this page

Peter Hailey (Director Centre for Humanitarian Change) for Concern Worldwide (2015). Independent Evaluation of the CMAM Model Surge Pilot. www.ennonline.net/cmammodelsurgepilotevaluation

(ENN_6910)

Close