CMAM Report: development of a global online reporting system for CMAM programming
The authors would like to express their gratitude to all individuals that have contributed to the development of the CMAM Report since 2008: Carlos Navarro-Colorado, Jeremy Shoham, Vicky Sibson, Mark Yarmoshuk, Jennifer Martin, Lourdes Vazquez, Shafat Sharif, Emily Mates, Emily Keane, Geraldine Le Cuziat, Andrew Crossman, Jaime Hargreaves, Terry Reed, Vasantha Mountford, Stewart Gordon, Elena Pullara, the MSM software developer’s team around Graham Douglass and Simon Gibbs and all consultants. Further acknowledgements go the steering committee of agencies that contributed to the development of the standardised reporting categories and indicators used in CMAM Report - CDC, Concern Worldwide, FANTA2, IRC, Nutrition Information in Crisis Situations (NICS), Save the Children UK, UNHCR, UNICEF, Valid International, WFP, WHO and World Vision – and all agencies that improved the software by giving continuous feedback to the development team – Concern Worldwide, GOAL, International Medical Corps (IMC), ACF-USA, Eastern Deanery Aids Relief Programme (EDARP) and Save the Children.
The MRP/CMAM Report was funded by OFDA 2008-11, ECHO 2011-12 and HIF 2013-15. The CMAM Report online software was developed by Management Systems Modelling (MSM) Ltd, UK
Since the publication of the Humanitarian Practice Network (HPN) Network Paper Measuring the effectiveness of Supplementary Feeding Programmes (SFPs) in emergencies in 20081 highlighted inconsistencies, inadequacies and bias associated with reporting on Supplementary Feeding Programmes (SFP), the idea of standardising community based management of acute malnutrition (CMAM) reporting categories and indicators across implementing agencies and countries has gained traction. This has culminated in the development of a comprehensive reporting system for all aspects of CMAM, which features a web-based central database called the ‘CMAM Report’. This article describes certain experiences and lessons learnt from the development process of this reporting system and its use as a management tool in CMAM programming.
Background
The CMAM Report started out in 2008 as the Minimum Reporting Package (MRP), with the formulation of standardised reporting categories and indicators for SFPs coordinated by a steering committee of twelve international agencies, led by the ENN and Save the Children. These were piloted in four countries. In 2011/12 and in response to the feedback from the pilot, Outpatient Therapeutic Programme (OTP) and Stabilisation Centre (SC) reporting categories and indicators were added through a similar consultative process and a Microsoft AccessTM based reporting software was developed. Both reporting standards and software were subsequently rolled out by seven non-governmental organisations (NGOs) in 15 countries; Save the Children, Concern Worldwide, International Medical Corps (IMC) and GOAL were the main users.
A number of challenges were identified following this first phase in relation to the indicators and the software. Despite guidance on standardised indicators, these were still not fully adopted, even by agencies committed to the MRP principles. The software appeared not to be user friendly enough. Evaluations and feedback on the software also emphasised the focus on NGO implementation, the lack of capacity for coordination and the potential for parallel systems. For example, there were concerns that the MRP added another layer of reporting on top of government reporting systems (this was not the intention but rather to be a complementary initiative, whereby the standardised reporting categories and indicators of the MRP could positively influence government systems in the long run and where systems needed strengthening).
With these issues in mind, a new phase of software development and reorientation took place 2013-15. The software was redesigned into a global web-based database with a specific function for United Nations (UN) agencies and the Ministry of Health (MoH). Mid upper arm circumference (MUAC) screening and blanket supplementary feeding programme (BSFP) components were added. The system was finally renamed as CMAM Report.
Developing an online software
In order to address all the challenges identified with the initial software, Save the Children decided to modify the software to a web-based tool with offline capability. The aim was to develop a system that would not require installation and that gave access to data from several levels via synchronisation with a central server. The actual development of the software was given to a software developer but additional funding was needed to meet the cost of development, which proved several times higher than expected. When development started, it was through an iterative process between the developer, the Save the Children IT team (who reviewed system structure and general functionality) and the Save the Children Nutrition team (who reviewed content and usability). For each iteration delivered by the software developer, testing took place by Save the Children IT and nutrition experts and was reported back to the developer. The main phase of software development was completed in June 2014.
In parallel to the software development, the team at Save the Children facilitated consultations with partners and users, which resulted in a number of changes to the software and the addition of some new components:
- The structure of the software was changed from being focused around programmes funded through specific grants to the unit of focus being sites, i.e. programme feeding sites, health centres or hospitals are the main level for data entry and analysis. Grants and contextual information can be attached to this to support 1-click reporting if desired.
- INS3has been renamed within the software to ‘OTP discharges in TSFP.'3
- A ‘Country Administrator’ user access level was added. This is a view-only function that allows the MoH and UN agencies to view and analyse data from all agencies in a given country.
- Stock tracking, MUAC screening and BSFP components have been added.
- The software and all supportive material are now available in English and French.
With the wider launch to the nutrition community mid-2015, CMAM Report can now be used on computers, laptops and tablets with a common browser4 and is freely available to all agencies/countries that wish to use it. The software supports secure data collection through controlled access from the field via country to headquarters levels (five adaptable user access levels). It enables the use of standardised reporting categories and indicators (enables comparable data and unbiased reporting) and gender and age disaggregated reporting through all reporting categories. Users can analyse data for trends and create summary tables and graphs by feeding site, group of feeding sites and geographical location up to country and global levels for any required time period in one month steps. The system features export of tables and graphs to PDF, Excel and Word. Raw data can be selectively exported to Excel and in turn to statistical software. Data entry mistakes are reduced by real-time validation checks.
Lessons learned in software development
The main challenges in establishing this system have been due to the lengthy time taken to develop complex new software, the unexpected costs of this process and lack of a framework for defining what constitutes a change to the original specification for the new software.
The following lessons can be taken away from the development process:
- Software development projects require flexibility in delivery time and financial resources in order to meet needs which evolve over time.
- Clear definition on what constitutes change in specification should be defined with the software developer early on.
- Wider learning is essential as use of Information & Communication Technology (ICT) increases in our programmes. This kind of development requires a specific skill sets and level of shared understanding and common language between both IT and nutrition expertsto build a software that is fit for purpose for the end users in the field
- The potential for parallel systems and duplication of efforts remains; there are a number of ongoing related initiatives (e.g. other agencies who are developing national and global reporting systems with similar objectives). Transparency and coordination around such initiatives is essential in order to avoid duplication and maximise reporting effectiveness. Lessons should be shared between developers of different systems.
- Systems should be flexible in terms of indicators to be collected, but definitions should be standardised.
- There is much sensitivity around data storage, sharing and ownership. There may be some governments who do not wish for their data to be stored outside their country. There are questions as to whether NGO collected data belongs to the NGO or to the government (since it concerns their citizens).
- Advanced systems are not a solution to poor quality data; the system is only as good as the data.
- Software development projects should be adequately resourced, both technically and financially.
- In the world of information systems, there is no one solution that would fit everywhere; platforms which can link different systems are likely to play a vital role in facilitating more efficient ways of working in the future.
Examples of good practice and added value
Several examples of good practice were found by a recent evaluation.
Using the CMAM Report software to inform programme management
In Yemen, high defaulter rates were revealed and resulted in the management team adapting their programme delivery model. In Asia, a large number of defaulters were revealed due to a supply chain break. The CMAM Report software made this visible and results could be traced back to specific sites (Figure 1). In West Africa, the CMAM Report clearly showed a number of programming issues and therefore where to focus technical support. Stock management is one of the main challenges for CMAM programming and the CMAM Report is particularly effective and useful for estimating caseloads and supply needs, e.g. in Puntland, Somalia.
Box 1 - Yemen experience
The CMAM Report software replaced Excel sheet reporting; this was prone to errors and had limited use. The Yemen team cited the following as ways in which the CMAM Report has added value to reporting:
“With the CMAM Report, we can see indicators directly when entering data – a major advantage over the old system. We can take actions if indicators do not reach Sphere standards in single feeding sites. Data in the CMAM Report format is shared with the Ministry of Public Health and UNICEF. The CMAM Report helps to improve the quality of the programme. Before starting the CMAM Report, the defaulter rate of the programme was very high. With the CMAM Report it has been easier to monitor the data and to take corrective actions.”
Highlighting issues with data quality
CMAM Report has been useful in highlighting issues with data quality across programmes, partly through the standardisation to calculate performance indicators. Using the CMAM Report software, indicators can also be easily compared across programmes and compliance and validation reports have highlighted missing, late and erroneous data. One organisation conducted a global assessment of missing, late and erroneous data during the initial months of CMAM Report implementation and it was observed that data quality improved during that time.
Using standardised indicators to inform national reporting systems
In some contexts, agencies have also entered into discussions with partners, government and UN agencies on using the standardised indicators to inform national reporting systems:
Somalia: Save the Children Somalia was invited to present the CMAM Report as a potential reporting tool for a nutrition consortium in April 2014 that includes the CMAM implementers IMC, ACF and Oxfam. The additional utility of the tool was recognised and it was decided to use the CMAM Report as a consortium reporting tool.
Yemen: Save the Children and IMC presented the CMAM Report at a cluster meeting and it was well received. In addition, a joint CMAM Report training was conducted in Yemen between IMC and SCUK staff.
West Africa: Agencies are using their own internal databases due to the lack of a strong national system in many countries. There is therefore a need for a regionally standardised CMAM reporting tool which could be met by the CMAM Report software.
The added value of the software reported by users is listed in Box 2.
Box 2 - Added value of the CMAM Report software
- Data are easy to manage and share
- Reduces reporting time compared to Excel
- Error proof as false data can’t be entered
- Highlights missing data, can be used to enforce better reporting
- Compliance reports improve timeliness of reporting
- Facilitates trend monitoring to feed into donor reports/stock checks
- Easy to extract indicators for donor reporting
- Useful for programme review
- Field staff using analysis functions to produce reports and graphs
- Analysis by feeding site possible, so easier to target support in programme management Able to verify/compare with national reporting system to do a data quality assessment and inform monthly data discussions with cluster
- Graphs highlight problems assisting management
- Support from head office is easier as programme details can easily be accessed
- Good information source
- Encouraged monthly instead of quarterly HQ reporting and increasing transparency
- Remote supervision and technical support is easier
- Transparent - easier to trace back results to original sites.
The software, user manuals, standardised indicator guidelines, e-learning, demo version and more information can all be found at www.cmamreport.com.
For more information and to receive your personal login details, contact: cmamreport@savethechildren.org.uk
1http://www.ennonline.net/sfpeffectiveness
2Individual Nutrition Support: A term used for beneficiaries that move from OTP to TSFP and that are kept separate from MAM cases that enter TFSP. This allows unbiased calculation of TSFP performance.
3Targeted supplementary feeding programme (TSFP)
4 Not running on Apple devices
More like this
FEX: Update on Minimum Reporting Package (MRP) trainings in London and Nairobi
By Emily Mates, Nutrition Advisor, MRP, Save the Children UK The ‘Minimum Reporting Package’ (MRP) has been developed to support standardised data collection for emergency...
FEX: Minimum Reporting Package (MRP) on Supplementary Feeding Programmes
In order to evaluate the quality and impact of nutrition interventions, agencies need effective monitoring and reporting systems in place for programmes. A retrospective...
Resource: CMAM Report
CMAM Report is a comprehensive monitoring and reporting package for global reporting on all CMAM components - TSFP, OTP, SC - as well as MUAC screening and BSFP. Visit the CMAM...
en-net: Output Monitoring System for Integrated Management of Acute Malnutrition (IMAM)
Dear colleagues, The IMAM team in Viet Nam will establish an output monitoring system this year for SAM and MAM management and ideally integrate other key interventions in the...
en-net: Minimum Reporting Package
MRP is a tool that was developed for Supplementary Feeding programmes,I understand its under study,does anyone have experience based on some study on its usage ? Is it user...
en-net: SCUK and ENN seeking 3 project officers for SFP Minimum Reporting Package project
SCUK and ENN are looking to recruit up to three Project Officers for an ECHO funded project to roll out a Minimum Reporting Package for the treatment of MAM. The MRP is the...
FEX: Developing a mobile health app to manage acute malnutrition: a five-country experience
By Tine Frank, Emily Keane, Natalie Roschnik, Colleen Emary, Melani O'Leary and Laura Snyder View this article as a pdf Tine Frank is a consultant engaged by Save the...
en-net: Vacancies for Save the Children UK / ENN project supporting the introduction of the Minimum Reporting Package for SFPs
Save the Children UK and the ENN are excited to have four positions open (for nutritionists/similiar background with familiarity with SFPs) to lead and run this 18 month ECHO...
FEX: Global CMAM mapping in UNICEF supported countries
Summary of review1 A recent review commissioned by UNICEF set out to develop a global map on the status of Communitybased Management of Acute Malnutrition (CMAM) with a focus...
FEX: UNICEF Global reporting update: SAM treatment in UNICEF supported countries
A child being screened for malnutrition in a UNICEF supported programme By UNICEF Nutrition in Emergencies Unit and Valid International Following the CMAM mapping exercises...
FEX: Improving nutrition information systems: lessons from Kenya
By Lucy Maina-Gathigi, Louise Mwirigi, Veronica Imelda, Dr Oleg Bilukha, Eva Leidman, Lucy Kinyua and Kibet Chirchir View this article as a pdf Lucy Maina-Gathigi is a...
FEX: ENN secures funding for SFP Research Projects
The ENN has just received a grant from USAIDs Office of U.S. Foreign Disaster Assistance (OFDA) to carry out two projects related to emergency supplementary feeding programmes...
en-net: News on the Minimum Reporting Package?
Hello, Any news regarding the Minimum Reporting Package under development by Carlos Navarro ? Someone knows when it will be published? Thank you Hi Elisa Thanks for your...
en-net: CMAM Report and CMAM reporting
Hello friends, can anyone tell me what has happened since the Save the Children led CMAM report project finished in 2015? Is it still used?? Thanks Vicky Thanks very much...
FEX: UNHCR standardised nutrition survey guidelines and training
By Melody Tondeur and Andrew Seal Melody Tondeur is an ENN Consultant and one of the team members implementing the Anaemia Control, Prevention and Reduction Project, a...
FEX: Improving the quality of nutritional survey data worldwide: Putting Child Kwashiorkor on the Map Initiative
By Lauren Browne Lauren Browne is the Data Manager for the Kwashiorkor Mapping project and joined the team after interning with ACF-UK and Save the Children UK. She completed...
FEX: Implementation of nutrition surveys using SMART methodology in sub-Saharan Africa
By Jana Daher, Fanny Cassard, Sara Gari, Victoria Sauveplane-Stirling, Mara Nyawo and Patrick Codjia View this article as a pdf Lisez cet article en français ici Jana...
FEX: Streamlining SMART survey processes in support of a high quality, nationally owned nutrition information system in Kenya
View this article as a pdf Lisez cet article en français ici By Lydia Ndung'u, Lucy Maina-Gathigi, Lucy Kinyua, Hassan Ali Ahmed, Kibet Chirchir and Samuel Mahinda...
FEX: Philippines Nutrition Cluster: lessons learnt from the response to Typhoon Haiyan (Yolanda)
By Aashima Garg, Andrew Rene Bucu and Rene Gerald Garela Aashima Garg is Nutrition Specialist based in UNICEF New York. At the time of writing, she was Nutrition Manager with...
FEX: Improving the assessment and attribution of effects of development assistance for health
Summary of research1 Location: Global What we know: It is difficult to relate Overseas Development Assistance for health (DAH) and health outcomes; there are multiple funding...
Reference this page
Rachel Evans, Christoph Andert and Susan Fuller (). CMAM Report: development of a global online reporting system for CMAM programming. Field Exchange 50, August 2015. p57. www.ennonline.net/fex/50cmamreportonlinereportingsystem
(ENN_4988)