The REFANI Project in Pakistan: adapting research to a multi-sectoral programme for impact measurement
By Zvia Shwirtz, Bridget Fenn, Riccardo Mioli, Ghulam Murtaza Sangrasi and Maureen Gallagher
Zvia Shwirtz is currently the REFANI Communications and Research Uptake Officer, based in the Action Against Hunger office in New York.
Bridget Fenn is an experienced epidemiologist and longstanding ENN consultant. She is ENN’s principal investigator for the REFANI Pakistan study.
Riccardo Mioli is the Field Coordinator of Action Against Hunger in Dadu (Sindh, Pakistan), and manages district projects operations. Riccardo is an anthropologist who has been working in the humanitarian sector for the past four years in Ethiopia, Indonesia and Pakistan.
Ghulam Murtaza Sangrasi is the Study Manager of the REFANI Pakistan research. He has been working with ACF since December 2014 in Dadu district and has worked in Sindh province since 2010. He is leading the REFANI study in Pakistan.
Maureen Gallagher is the Senior Nutrition & Health Advisor Action Against Hunger US based in New York. She is an experienced public health specialist with an MSc in Social Policy and Planning specialising in health policy.
The authors would like to thank key stakeholders and field teams in Pakistan for their hard work and commitment to the programme and Julien Morel, ACF France, for his contributions to this article. The authors would like to also acknowledge the contribution of the European Union for their support to nutrition programming in Sindh Province, and ECHO and UKAid/DFID for their support to the related REFANI (Research on Food Assistance for Nutritional Impact) research project shared in this article
What we know: Undernutrition is prevalent in Pakistan, associated with systemic poverty and shortcomings in feeding practices and maternal care that are compounded by food crisis and climatic challenges which impact on food security, such as flooding.
What this article adds: The European Union-funded Women and children/infants Improved Nutrition in Sindh (EU-WINS) project is a multi-sector approach to address undernutrition in the province, involving access to maternal nutrition and health services, SAM treatment, use of nutritious foods by women and children, and evidence-based learning. A randomised controlled trial, as part of the DFID-funded REFANI Research Consortium, is investigating the nutritional impact and cost-effectiveness of cash and voucher-based food assistance in the context of the WINS programme. Four research arms are being explored, led by ENN in collaboration with ACF. Results will be available in early 2017 that will include cost-effectiveness. Valuable learning regarding operational research in challenging contexts is emerging.
Action Against Hunger (AAH) has been working in Pakistan since 1979 during recurrent humanitarian situations. Programming is now governed by a full nutrition security-oriented strategy, strengthening the links between Nutrition, Food Security and Livelihoods (FSL), Water, Sanitation and Hygiene (WASH) and Disaster Risk Management, to achieve a higher level of impact for the population at risk of malnutrition. Since 2011, Action Against Hunger has been successfully implementing intensive, community-based management of acute malnutrition (CMAM) and nutrition-sensitive programming in Sindh (Thatta, Tando Muhammad Khan, Badin, Dadu and Jacobabad), addressing the nutrition emergency affecting the entire province. In 2013, AAH expanded its presence in Dadu through a four-year, nutrition security-focused development project funded by the EU – the WINS project – partnering with Merlin in Thatta and Sujawal districts and Save the Children in Shikarpur district.
Pakistan has some of the most serious undernutrition problems in South Asia, reflected in Global Acute Malnutrition (GAM) (17.5%), Severe Acute Malnutrition (SAM) (6.6%), underweight (40.5%) and stunting (49.8%) prevalence rates in children under five years old in Sindh province (NNS, 2011). Key attributable factors are reflected in data from Sindh province that include poor infant and young child feeding (IYCF) practices (40.5% breastfeeding initiation, 64.7% exclusive breastfeeding); low dietary diversity for children 6-23 months (4.6% minimum acceptable diet for 6-23 month olds); women’s micronutrient deficiency (60% anaemic, 49.5% zinc deficient); low birthweight (11.2% and 10.1% in north and south Pakistan respectively); poor access to safe drinking water, sanitation and health services (50% prevalence of diarrhoea in under fives, 21% of boreholes not protected (AAH, 2011)); and high food insecurity (72% of Sindh population in state of severe and moderate hunger) (NNS, 2011 unless otherwise stated).
Systemic poverty coupled with the lack of alternative income sources affects the capacity of households to cope with shocks and climate trends in a sustainable manner (Nutrition Causal Analysis (NCA), ACF, 2012). This is compounded by high incidence of illnesses due to poor access to water and sanitation infrastructures, and poor knowledge and practices regarding child feeding, hygiene and care. The already limited access to nutritious food among chronically poor households in Pakistan has further deteriorated in recent years due to the 2007-2008 global food crisis and continued high volatility in food prices. This impacts intra-household food security, determining dietary patterns in pregnancy and early childhood. As reflected earlier, the scarcity of iron, zinc, iodine and vitamin A in the diet and poor IYCF practices contribute significantly to child undernutrition in the country.
An integrated, multi-sector approach was recommended by the 2012 ACF NCA to tackle the identified basic causes of the high prevalence of SAM in Sindh. The NCA was conducted in Thatta and Dadu districts in Sindh Province. In response and in line with the Pakistan Integrated Nutrition Strategy and the Poverty Reduction Strategy Paper, in-country partners developed the EU-WINS project. The programme’s objective is to strengthen the capacity of district and provincial governments to address high rates of undernutrition in Sindh Province. The four key results and specific activities/modalities of the multi-sector approach are summarised in Table 1. EU-WINS targets, projected and reached by Year 3, are included in Table 2.
Nutrition-sensitive FSL and WASH activities were designed to integrate the nutritional treatment and prevention components of the programme. For example, complementary feeding food vouchers, livestock vaccination interventions, construction and rehabilitation of water points often happen in common geographical areas and will always include the same BCC/community mobilisation component. Since the start of the programme in 2011, strong synergies have developed at community level, as all components also involve community-based organisations.
The long-term vision is that local communities, healthcare workers and nutrition and food security stakeholders at the district and provincial levels will benefit from the project as key activities become accessible through government mechanisms. Within the scope of EU-WINS, all health and nutrition-related activities are undertaken in close collaboration with the Department of Health and the Provincial Nutrition Cell (Sindh), as well as the People’s Primary Healthcare Initiative. Cash-based interventions are linked with the BISP in order to support improvement in policy and delivery of social safety net services. The close synergy of all aspects of the project with government policies and processes is the foundation for achieving the longer-term vision.
The focus of this article is on the operational research, which has been designed to look more closely at the nutritional impact of this type of intervention.
Challenges of measuring nutritional impact of interventions
ACF’s International Nutrition Security Policy, published in 2014, specifically calls for a multi-sector approach in order to achieve a long-term and sustainable impact on nutrition (through 12 guiding principles specifically; see separate article in this edition of Field Exchange). The Policy also promotes mixing curative and preventative measures for undernutrition and targeting its underlying causes, as well as other structural factors. However, there are challenges in circumventing the barriers which exist between nutrition-related sectors.
Overall monitoring and evaluation mechanisms were put in place for the various WINS activities (including baseline/endline and external evaluations), yet there was still need for robust impact measurement. The DFID-funded REFANI project (see Box 1), further supported in its intervention component with one additional research arm by ECHO, proved a unique opportunity to scientifically measure impact of the nutrition-sensitive aspect of EU-WINS in relation to the social safety net cash-based interventions.
Box 1: REFANI Project
The REFANI Pakistan study is part of a larger DFID-funded REFANI programme of study currently also being implemented in Niger and Somalia. The REFANI programme aims to assess the impact of cash transfer programming on nutrition in humanitarian contexts. There is evidence that shows that CTPs increase income and protect family assets from being sold, which in turn favours behaviours that could protect children from undernutrition by increasing the number of calories consumed and improving dietary diversity (Manley, Gitter & Slavchevska, 2012; DFID, 2011). However, more evidence is required to understand how and whether CTPs with and without complementary interventions are effective in preventing or reducing the risk of developing malnutrition. (ACF 2015 ).
The REFANI project aims to provide more data on the nutritional impact and cost-effectiveness of cash and voucher-based food assistance programmes, as well as identify the mechanisms through which this is achieved. The study is designed to understand the factors that determine the ways in which households use the different transfers. It explores the role of the different processes influencing the intervention outcomes and how they interact with the context using both quantitative and qualitative data. A separate qualitative study will help to understand any associations found (or not) within the quantitative analysis, as well as explore further differences in undernutrition between the study arms.
REFANI partners (Action Against Hunger, Concern Worldwide, University College London (UCL) and ENN are running a series of three complementary country studies in Niger (Concern/UCL), Pakistan (ACF/ENN) and Somalia (Concern/UCL). This will inform the evidence base regarding timing of CTPs, duration, amount, frequency, cash vs. vouchers, targeting criteria, etc. It will also inform on the sustainability of CTPs and their cost-effectiveness. Results from all three countries will be available by early 2017.
For more information, visit: www.actionagainsthunger.org/refani
The REFANI Project in Pakistan
The REFANI-Pakistan study is a collaboration between ENN, as the lead research partner, and Action Against Hunger, as the lead implementing partner. The study specifically looks at the effectiveness of different cash transfer programmes (CTPs) in reducing the risk of undernutrition in children under five years of age from households who also have access to the same elements of the programme covered by the EU-WINS activities, which in this case are nutrition and BCC programming. The specific research arms are detailed in Box 2.
Box 2: The four REFANI-Pakistan research arms
The study specifically compares four different arms of intervention:
1) Households receiving the standard EU-WINS intervention (control group);
2) Households receiving the EU-WINS care and 1,500 rupees per month (funded by the EU);
3) Households receiving the EU-WINS care and 3,000 rupees per month (separately funded by ECHO); and
4) Households receiving the EU-WINS care and a monthly fresh food voucher worth 1,500 rupees per month to be exchanged for fresh foods at specified traders (funded by the EU).
The standard EU-WINS intervention includes SAM treatment at an outpatient therapeutic programme (OTP) site for all research sites, micronutrient supplementation, complementary food vouchers for OTP discharged children, IYCF services and integrated sensitisation sessions on six key behaviours for prevention and treatment of malnutrition.
The additional cash-based support is provided for six months; thus totalling 9,000 rupees for arm 2 and 4 and 18,000 rupees for arm 3 beneficiaries.
Primary and secondary outcome data being collected to compare levels of risk of undernutrition are*:
- Primary outcomes – prevalence of wasting (as measured by weight-for-height Z-score (WHZ) < −2) and mean WHZ score in children;
- Secondary outcome in children – cumulative incidence of wasting, prevalence of SAM (<−3 WHZ), low mid-upper arm circumference (MUAC) (<125 mm & < 115 mm), stunting (<−2 height-for-age Z-score) and morbidity (mainly diarrhoea, difficulty breathing/cough, malaria);
- Secondary outcome in mothers and children – mean haemoglobin concentration (Hb g/dL) and prevalence of anaemia (children: Hb <11.0 g/dL; severe anaemia as Hb < 7.0 g/dL; non-pregnant women Hb <12.0 g/dL; pregnant women Hb <13.0 g/dL); and
- Secondary outcome in mothers – prevalence of low body mass index (<16, <17, <18.5 kg/m2), low height (<145 cm) and low MUAC (<230 mm & <210 mm).
* Measurements are collected on all study households (household questionnaire) and participants (mother and child questionnaire) at month 0, 1, 2, 3, 4, 5, and 12 and are collected during the week following the receipt of each transfer.
The process of developing the REFANI project in Pakistan followed a number of steps and will provide useful ways of working and lessons learned for partnerships engaging in developing research as part of an operational programme already in place. It was agreed by partners that a cluster randomised controlled trial (cRCT) of CTPs, focusing on nutritional outcomes in Dadu, was possible. The CTP is designed to address a seasonal hunger gap, where ideally the transfer transitions from an emergency setting to a national/regional safety net approach, which would offer the research teams an opportunity to address common research questions related to CTPs, their methodology and sustainability.
The overall research structure includes six rounds of quantitative data collection, including baseline and endline surveys, one more data collection round after one year, a mixed-methods process evaluation, a qualitative research component, and a cost-effectiveness analysis study. The total planned sample size is 2,496 households from poor and very poor households located in 17 union councils from different livelihood zones (barrage and riverine). The particular study design allows for probability analysis of the impact of different cash transfer interventions on undernutrition, comparing three different interventions within the same population and region to a control group.
A cRCT study, especially in humanitarian settings, brings a number of challenges. The difficult context of the study (including conflict, security issues, religious holidays, weather conditions, and risk of emergency) can easily impact on implementation. The potential for flooding in the area was a major concern, so a contingency plan covering three levels of flooding (mild to severe) was built into the study. The REFANI project has successfully recruited a qualified study manager, deputy study manager and qualitative researcher under Action against Hunger protocols. The national team has been successful in ensuring high standards of data collection, with remote supervision from the ENN team and good collaboration between ENN and ACF.
The REFANI-Pakistan study has been approved both by the National Bioethics Committee in Pakistan and the Western International Review Board. The study has been registered as an International Standard Randomised Controlled Trial (ISRCTN10761532).
The study is mid-way completed; baseline and five consecutive months of data have been collected (see Table 2). Qualitative data collection in ongoing, adapted as results from the quantitative research comes in. First results will be available later in 2016. The aim is to publish at least three peer-reviewed publications, as well as disseminate findings at relevant conferences, stakeholder and other meetings.
The EU-WINS programme encountered a number of challenges in the operationalisation of a multi-sector approach, mainly as a result of vertical sector-based government systems and dramatically variable climate in the intervention area. The REFANI-Pakistan study has been able to effectively and quickly address some challenges that arose within the first year of implementation. These included:
Recruitment: There was some difficulty around recruiting and retaining female data collectors, which is essential for accessing female household members. However, the Action Against Hunger office in Pakistan managed to locate the necessary female enumerators. Action Against Hunger also focused on capacity-building of staff on research during the implementation in order to counter the difficulty in hiring female staff.
Timing: The study overlapped with Ramadan (June-July 2015) as well as extremely high temperatures (>50 degrees Celsius), which resulted in more time needed to cover all eligible households, since office hours are reduced during this period and the capacity of staff was lower. This resulted in the baseline data collection taking longer than expected; as a result of this delay, the ENN principal investigator decided to forgo the first month of the monthly data collection as this would have impacted on the timing of subsequent programme implementation. This is not expected to impact negatively on the study.
Access: Floods and tribal conflicts occurred in August 2015, which led to the replacement of some study areas and participants (in case of security risk).
Loss of study participants: This was noted during the second month of data collection, mainly within the control group, and reflected lack of incentive to participate. Efforts to reduce this were made through regular sensitisation carried out with the help of community mobilisers.
Opportunities for operation research development
There is a growing trend among organisations to use cash transfers in their food assistance programming to combat growing rates of undernutrition, especially during emergencies.
The process and results of the EU-WINS and REFANI interventions will be of use to many stakeholders, both in Pakistan and globally. There are lessons to be learned on the optimal design of operational research and on the nutritional impact of multi-sector interventions. These can be used to inform and provide evidence to support scale-up and change.
For more information, contact: Maureen Gallagher
ACF (2015). EU-WINS Midterm Report, August 2015.
ACF (2015 ). Research on Food Assistance for Nutritional Impact (REFANI): Literature Review. March 2015.
ACF (2012). Nutrition Causal Analysis Thatta and Dadu district, Sindh Province. www.linknca.org
ACF (2011) WASH baseline survey; Dadu district; Sindh; Pakistan.
DFID (2011). Cash Transfers Literature Review.
EU WINS (2011). Project proposal.
Manley, J., Gitter, S. & Slavchevska, V. (2012). How Effective are Cash Transfers at Improving Nutritional Status? A Rapid Evidence Assessment of Programmes’ Effects on Anthropometric Outcomes. World Development. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.
National Nutrition Survey (2011). Pakistan Aga Khan University.
More like this
Bridget Fenn, the REFANI Pakistan Principal Investigator, talks about the “hot off the press” result from the study at the Research for Nutrion Conference. What is...
FEX: A cluster RCT to measure the effectiveness and of cash-based interventions on nutrition status, Sindh Province, Pakistan
By Bridget Fenn Bridget Fenn is an epidemiologist with a background in nutrition. She is currently a consultant for the Emergency Nutrition Network (ENN) and is Principle...
REFANI is a 3-year research project funded by the Department of International Development (DFID/UKAID) of the United Kingdom and implemented by ACF, Concern Worldwide,...
New paper out by the REFANI project in PLOS Med ,"Impact evaluation of different cash-based intervention modalities on child and maternal nutritional status in Sindh...
Academic and Humanitarian Aid Organizations Launch Research Project, REFANI, to Measure Impact of Cash Transfers on Child Undernutrition. Click here to go to the REFANI webpage...
en-net: REFANI Research Coordinator
ACF has received a three years consortium research grant from the British Government (UKAid/DFID) for the Research for Food Assistance Nutritional Impact (REFANI). The REFANI...
Blog post: Food Assistance and Nutrition- a recent presentation of our research findings in Pakistan
Hello, I am Bridget Fenn, ENN's long standing lead research investigator. I recently I gave a virtual presentation about the REFANI to participants at a research conference in...
Blog post: Research on multi-sectoral programming: reflections on a cash and WASH, nutrition integrated approach
Cliquez ici pour lire en français Some time ago I had the opportunity to attend a regional event for sharing multi-sectoral nutrition approaches, organized by ACF and...
By Ali Dino Kunbher, Shafqat Ullah and Dr Mazhar Alam Ali Dino Kunbher is a Programme Manager for Welthungerhilfe (WHH) in Sindh, Pakistan, covering food and nutrition...
By Bridget Fenn and Ellyn Yakowenko Bridget Fenn is an epidemiologist with a background in nutrition. She is currently a consultant for the Emergency Nutrition Network (ENN)...
Brief overview of the recent visit As part of ENN's role as an academic lead in the DFID funded 'Research on Food Assistance for Nutritional Impact (REFANI) Consortium...
The post is for a Research Associate at the Institute for Global Health at University College London (UCL). It is funded by the Department for International Development via...
Research on Food Assistance for Nutritional Impact (REFANI) consortium releases a brief about its upcoming research activities. REFANI is a DFID funded joint research project...
REFANI (Research on Food Assistance for Nutritional Impact) is a three-year research project aiming to strengthen the evidence base on the nutritional impact and...
Summary of research1 Location: Global What we know: Cash transfer programmes (CTPs) have a proven impact on poverty reduction and social protection but their overall impact...
Summary of Research1 Save the Children. Cost-benefit analysis of the kitchen garden intervention: Women and children/infants improved nutrition in Sindh, Pakistan Report, July...
Summary of research1 Location: Global What we know: Cash transfers are an important element of national social protection systems to alleviate poverty and...
By Stephanie Stern and Melchior de Roquemaurel Stephanie Stern leads the Action Against Hunger LAB project which aims to reinforce the impact and uptake of knowledge. Before...
Concern Worldwide is a Dublin-based non-governmental, international, humanitarian organization dedicated to reducing suffering and eliminating extreme poverty in the world's...
ENN has released a new discussion paper on 'Stunting in Protracted Emergencies'. Funded by Irish Aid, this paper outlines the difficulties of gaining a clear...
Reference this page
Zvia Shwirtz, Bridget Fenn, Riccardo Mioli, Ghulam Murtaza Sangrasi and Maureen Gallagher (2016). The REFANI Project in Pakistan: adapting research to a multi-sectoral programme for impact measurement. Field Exchange 51, January 2016. p100. www.ennonline.net/fex/51/refanipakistan