How to overcome data management challenges in research in crisis contexts
Summary of panel discussion1
This session was moderated by Eva Leidman, of Centers for Disease Control and Prevention (CDC). Presentations were given by Mary Hodges of Helen Keller International (HKI), Karine Le Roch of Action Against Hunger (AAH) and Oleg Bilukha of CDC. Each presenter shared research they had undertaken, challenges experienced and results achieved.
Karine Le Roch (AAH) spoke about the follow-up of severely acutely malnourished children project (FUSAM), a psychosocial intervention in children aged 6-24 months in Saptari, Nepal.
Action Against Hunger conducted a randomised controlled trial (RCT) to assess the effectiveness of the addition of psychosocial support in an outpatient therapeutic programme (OTP). OTP sites were randomly assigned as either intervention sites (n=6) or control sites (n=6). The study was conducted in partnership with the Government of Nepal Ministry of Health (MoH), Child Health Division, Saptari District Public Health Office and the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr’b) and supported by UK Aid, PATH, R2HC, Elrha and Wellcome Trust.
The study experienced significant challenges due to the 2015 earthquake, insecurity in 2015 and 2016 and floods in 2016. Although the original plan had been to monitor the study closely with real-time data capture and analysis, staff evacuations led to reliance on remote project management and loss of good visibility and quality control. Significant difficulties ensued in supervising record-keeping and physically extracting the data booklets from each site for analysis, as well as accessing homes for data collection due to floods and curfews.
In addition to these unexpected events, there was slow integration of psychosocial services into the healthcare system due to unfamiliarity with the approach and lack of recognition of the support required. Study sites were spread geographically, which created challenges in reaching beneficiaries and exacerbated pre-existing low treatment compliance, largely attributed to conflicting beliefs within the community.
Significant differences were found in child development scores between the group that received both psychosocial and nutritional support and the control group that had received only nutritional support. However, these differences were no longer significant at 11 months post-intervention and they never reached the level of children in the non-SAM group.
Mary Hodges (HKI) reported on a project to implement Essential Nutrition Actions (ENA) at the six-month point of contact for children at clinics before and during the Ebola crisis in Sierra Leone. The project was initiated prior to the Ebola outbreak in response to a government request to adapt the mass vitamin A supplementation (VAS) campaign conducted twice a year to a routine service at the six-month contact point (6MCP).
HKI established three groups to test three different approaches:
- VAS integrated within expanded programme on immunisation (EPI) (at six months).
- VAS integrated within EPI plus preparation of complementary food with the mother’s participation and then feeding of her infant by spoon.
- As group two plus routine ‘quality’ confidential one-to-one counselling on family planning (FP) (provided in a private room with a dedicated health worker) and provision of short-term FP commodities as appropriate or referral for long-term provision.
The results showed that routine VAS at 6-8 months was 60 per cent, 72 per cent and 75 per cent in the three groups respectively; 96 per cent of children achieved full vaccination status at six months in all three groups. In Group 3, 62 per cent of mothers received routine counselling on family planning at the 6MCP. Of these, 70 per cent accepted family planning commodities, helping to fill the ‘contraceptive gap’. It was evident that the 6MCP for routine VAS enabled mothers to access routine family planning and boosted VAS coverage.
In response, HKI scaled up this integrated approach and trained health workers from 11 out of 14 districts before Ebola broke out. Of 74 people trained in Kailahun district at the beginning of the Ebola outbreak, 16 died of the disease within six weeks.
HKI continued to support and monitor services throughout the emergency. It found that, of the clinics that had fully integrated the 6MCP, attendance fell significantly less during the Ebola crisis in integrated clinics (39 per cent) than in non-integrated clinics (59 per cent). The main reasons underlying this were the benefits perceived by mothers in preparing a complementary food to feed their infants, and, in terms of family planning, preventing pregnancy at a time when maternity services were scarce and pregnancy was regarded as risky.
Since then HKI has been scaling up the 6MCP with the Ministry of Health to 340 of 1,281 health facilities nationwide and has plans and funding to complete the scale-up and expand to a 6MCP within four years.
Oleg Bilukha from CDC talked about conducting nutrition surveys in difficult places. UNICEF often coordinates surveys but has a focus on children and women. He posed several questions, including what happens when the elderly are the most vulnerable or when wasting prevalence is low? He also observed how infant and young child feeding (IYCF) activities are often the ‘knee-jerk’ reaction in response.
Recently in the Ukraine, the Nutrition Cluster carried out three investigations that required modification of cluster survey methods:
1. An IYCF survey among displaced women living outside the conflict zone
This survey focused on women with children under two years of age. Participants were difficult to find. Lists of women were sought that included addresses or telephone numbers, and methods to update the lists explored, to ensure that they were representative for selection for the study. Some women were registered with the government, but the government was not willing to share data; others were registered with non-governmental organisations (NGOs)/United Nations (UN) agency programmes. There was an anticipated challenge of potential high non-response. During the survey, 20 per cent non-response/telephone number not working was recorded; some women had left the area, while others were afraid to meet. The surveyors adopted a protocol of calling three times before moving on to the next name on the list and used a quota sample, continuing down the list until the sample size was achieved.
2. A representative survey of older people (over 60 years of age) in the conflict zone
The team recorded anthropometry (weight, MUAC, arm demi-span), diet diversity and food frequency, as well as chronic disease, access to medicines, disability and mental health. The team members followed cluster survey methodology as the available population data were reliable and there was little displacement at that time. Electoral precinct data enabled specification of boundaries and households. Three random start points were identified in each cluster and interviewers went from these points to select houses randomly. Approximately 40 to 50 per cent of households included older people; however around 50 per cent of households did not answer the door, while others refused to participate.
3. A survey of pregnant women living in areas around the conflict frontline, to include anaemia assessment
The main challenge with a survey of pregnant women is that they represent a small percentage of the population. It would require 15 to 20 household visits to find one pregnant woman, so a household visit methodology is not conducive for this type of study. The study was therefore designed around antenatal clinics. In Ukraine a woman is obliged to register as early as possible in pregnancy and attend for regular, compulsory check-ups, IYCF interventions, nutritional status assessment and access to humanitarian assistance. The survey team randomly selected clinics and days and surveyed all women who attended to achieve a semi-representative sample. Working through clinics also enabled them to use the last haemoglobin test result from the clinic data to assess anaemia, rather than collecting samples first-hand with a haemocue.
What would you do differently?
The panel was asked to reflect further on their research, what they might have done differently and what advice they would give to others when approaching research in crisis environments.
Karine acknowledged that they probably wouldn’t have conducted an RCT if they had known all the challenges in advance. This is a difficult study design in any context, but extremely so in insecure situations. Anticipating some of the potential obstacles and preparing a ‘plan B’ for the study design might have enabled them to adapt the design according to the contextual factors and constraints.
Mary noted that if they had increased the sample size of their study, they could have compared the three different groups in terms of growth of children and other outcomes. Their study was narrowly focused on one outcome, but with additional funds it could have furnished further interesting data on nutrition.
Oleg noted that when conducting surveys, the concern is often non-response and achieving sample size. It is important to try to anticipate these things from the start. For example, for the challenge of absenteeism, it is useful to consider which days of the week are market days, which hours people are working in the field and when are the peak agricultural working seasons to establish when people will be at home. Refusals are high in Ukraine, so this factor could be anticipated in that context and the survey design adapted accordingly. One way to assess the situation prior to embarking on a survey is to conduct a rapid test in advance; for example, call 50 to 100 people on the list and work out the non-response rate, or pre-test the survey instruments in the field to find out how many people refuse to participate. There are two types of bias of concern in surveys: measurement and selection. Measurement bias is often a manifestation of team quality and can be addressed early through supervision if one team is identifying more malnutrition than others. Where there is selection bias, it should be recognised and qualified in the text of the report. However, care should be taken not to exaggerate the relevance of bias. For example, wealth quintiles may not display huge differences in wasting; the data of the ‘middle-ground’ (e.g. clinic visitors) are useful and often reasonably representative.
All three presenters spoke about challenges faced when conducting research in unpredictable environments. A commonality between these reflections is the necessity of thorough preparedness, including scenario-building in advance to try to anticipate as many challenges and risks as possible. While some of these can be predicted (such as the high non-response rate/refusal to participate characteristic of populations in Ukraine and the absenteeism related to seasonal events or labour schedules), not all challenges can be foreseen, as was the case in the Ebola outbreak in Sierra Leone and staff evacuation due to insecurity in Nepal. Study designs therefore require flexibility to adapt where necessary and feasible, a ‘plan B’ if possible and researchers need to maintain regular communication with donors/funders and stakeholders of the study, including the communities in which they are operating. This will facilitate anticipation and adaptation to ensure that useful and quality results can be obtained.
1Panel discussion at the ACF Research for Nutrition Conference, Pavillon de L’Eau, 13th November, 2017. A video of the panel discussion can be found here.
More like this
FEX: Child Survival Week as a platform for promoting vitamin A supplementation in Niger
By Doudou Halidou Maïmouna, Banda Ndiaye and Aissa Diatta Lisez cet article en français ici Dr Doudou Halidou Maïmouna is a paediatrician and holds a...
en-net: Helen Keller International recruits a consultant for a comparative cost effectiveness study in 3 countries
Background Vitamin A supplementation (VAS) for children under five years of age has been a cornerstone of global child survival programs for decades, and VAS has been shown...
en-net: consultancy for assessment on Vitamin A Supplementation in 4 African countries
Scope of Position When delivered twice a year to at least 80% of children aged 6 to 59 months in countries with high under five mortality and high rates of vitamin A...
FEX: Enhanced Outreach Strategy/ Targeted Supplementary Feeding for Child Survival in Ethiopia (EOS/ TSF)
By Selamawit Negash, Nutrition Specialist, UNICEF Ethiopia Selamawit Negash (MPH) has been working as a Nutrition Specialist with UNICEF Ethiopia since April 2007. From...
en-net: Consultancy for evaluation of HKI Vitamin A supplementation regional project
Consultancy for evaluation of HKI Vitamin A supplementation regional project Helen Keller international is looking for a consultant, under the supervision of a Team Leader...
en-net: Ukraine -- Recent Information on IYCF-E for Ukraine IDPs -- Non-breastfed children
I am working on issues surrounding how to best deal with issues related to the non-breastfed child in emergency situations. One of the situations where I am focusing on is the...
en-net: Team leader for the evaluation of HKI Vitamin A supplementation regional project
Team leader for the evaluation of HKI Vitamin A supplementation regional project Helen Keller international is looking for a team leader to be responsible for the management...
FEX: Acceptability of peanut-based RUTF to malnourished pregnant and lactating women in Bangladesh
Summary of research1 Location: Bangladesh What we know: Malnutrition prevalence is high among pregnant and lactating women in Bangladesh. Nutrition rehabilitation programmes...
FEX: Tech RRT IYCF-E/CMAM adviser: experiences from Niger, Haiti and Nigeria
By Michele Goergen View this article as a pdf Lisez cet article en français ici Michele Goergen has been an IYCF-E/CMAM adviser with the Tech RRT for one and a half...
FEX: Consumption of iron-rich foods among adolescent girls in Nepal: Identifying behavioural determinants
By Ajay Acharya, Pooja Pandey Rana, Bhim Kumari Pun and Basant Thapa View this article as a pdf Ajay Acharya is a Family Planning Specialist for the USAID-funded Suaahara II...
FEX: Diarrhoea risk associated with not breastfeeding in Botswana
Summary of report and presentation1 Between November 2005 and February 2006, there were unusually heavy rains and flooding in Botswana, and by January 2006, there was an...
FEX: Evaluation of mobile application to support the treatment of acutely malnourished children in Wajir county, Kenya
Summary of presentation1 View this article as a pdf By Emily Keane, Natalie Roschnik, Joanne Chui, Ibrahim Ahmed Osman and Hassan Mohamed Osman Emily Keane is a Nutrition...
NEX: Mobile clinics as a strategy to identify and treat children with acute malnutrition in difficult-to-reach areas in Chad: A case study of the Wadi Fira Region
Dahab Manoufi is an economist and manages the NGO Bureau d'Appui Santé et Environnement (Health and Environment Support Bureau), Chad. Dr Hervé Oufalba Mounone...
FEX: Relapse after treatment for moderate acute malnutrition: Risk factors and interventions to prevent it
Summary of presentation1 based on published research2 View this article as a pdf By Heather Stobaugh and Mark Manary Dr Heather Stobaugh has a PhD in Food Policy and Applied...
FEX: One hundred years of famine – a pause for reflection
Dublin '99 This piece was written by Fiona Watson, Institute of Child Health - NutritionWorks with contributions from the editors As the millennium draws to a close, memories...
en-net: Consultancy: 5 weeks consultancy to update a survey database and conduct initial analysis
Helen Keller International Job Announcement 5 weeks consultancy to update a survey database and conduct initial analysis Location (Nairobi, home based with regular office...
FEX: Review of food security and nutrition amongst urban poor
Summary of review1 Location: Kenya, Niger, Bangladesh What we know: A significant and increasing proportion of the world population resides in urban slums. Achieving food...
en-net: Vacancy - 5 weeks home based consultancy for Helen Keller International
5 weeks consultancy to update a survey database and conduct initial analysis Location (Nairobi, home based) Established in 1915 with Helen Keller as a founding trustee, Helen...
FEX: Follow-up status of children with SAM treated with RUTF in peri-urban and rural Northern Bangladesh
By Dr. Charulatha Banerjee, Monsurul Hoq and Dr. Ehsanul Matin Charulatha Banerjee is Regional Advisor on Maternal and Child Health & Nutrition, South Asia with the Terre des...
FEX: Managing severe acute malnutrition in India: prospects and Challenges
By Biraj Patnaik Biraj Patnaik is the Principal Adviser to the Commissioners of the Supreme Court of India in the right to food case. He is also associated with the Right to...
Reference this page
How to overcome data management challenges in research in crisis contexts. Field Exchange 57, March 2018. p75. www.ennonline.net/fex/57/managementchallengesinresearch