Operational research in low-income countries: what, why, and how?
Summary of research1
A Lancet published article puts forward a definition of operational research, articulates its relevance to infectious-disease-control programmes, and describe some of the enabling factors and challenges for its integration into programme settings and into changing policy and practice.
From a health programme perspective, a pragmatic definition of operational research is the search for knowledge on interventions, strategies, or tools that can enhance the quality, effectiveness, or coverage of programmes in which the research is being done.
What is operational research?
Operational research involves three main types of method: descriptive (cross-sectional, if a strong analytic component is also present), case–control, and retrospective or prospective cohort analysis. Basic science research and randomised controlled trials should not be included as operational research. The randomised controlled trial determines efficacy of an intervention in a strictly controlled environment with inclusion and exclusion criteria, whereas operational research should assess effectiveness within routine settings. Both types of research play an important part in the generation of new knowledge: the randomised trial provides clear-cut data on the efficacy of an intervention in defined groups of patients, whereas operational research determines how such interventions are translated into benefit in the heterogeneous setting of routine care.
The key elements of operational research are that the research questions are generated by identifying the constraints and challenges encountered during the implementation of programme activities (prevention, care, or treatment), and the answers provided to these questions should have direct, practical relevance to solving problems and improving health-care delivery. A strong connection exists between good monitoring and evaluation of infectious-disease programmes and operational research.
Good quality data on cases and treatment outcomes can be used to do operational research, which in turn can help to improve the routine data collected in the field. Nothing is more encouraging to healthcare workers than to see their work in recording and monitoring data on treatment cards and registers being used to answer important questions, provided that this performance is recognised and applauded.
There are at least three reasons why operational research is relevant to health. To improve programme outcomes in relation to medical care or prevention, to assess the feasibility of new strategies or interventions in specific settings or populations, and to advocate for policy change.
The study question must be of importance to programme implementation for ‘buy in’. Much of the internationally published research done in Africa has been generated by academic institutions and researchers, predominately reflecting their interests or based around basic science or questions of intervention efficacy. Although useful, this type of research needs to be balanced by increasing the work done by operational organisations (e.g. non-governmental organisations (NGOs)) that will have different perspectives. Different actors will naturally have comparative advantages for particular kinds of research that can benefit programmes. For example, an academic institution might be best placed to design and implement a randomised clinical trial or a vaccine study, whereas an implementing organisation might be best suited to take the lead in feasibility and acceptability studies. If research is disconnected from health-service delivery with little or no input from programme staff in the research design and process it may be being resented as an additional and often unwanted extra on existing busy and often overburdened services.
Enabling factors and challenges
Factors that enable operational research and its translation into policy and practice include:
- Research questions are generated from within programmes
- Research planning, agenda setting, objectives, targets, and budgeting are included within programme plans and as agenda items in programme management meetings
- Research projects use simple designs and are focused to answer implementers’ questions
- Close collaboration and partnership has been established between researchers and programme managers
- Research is done within existing systems and not done in parallel
- A competent research officer works alongside the programme manager
- Training, mentorship, and on-the-job supervision is sustained over time
- Sufficient programme capacity exists to host workshops, present, and discuss research findings, and ensure their translation into policy and practice
- Programme staff have access to scientific literature through subscribed journals or the internet
- Sufficient numbers of programme staff are available with the capacity to do operational research, write up manuscripts, and publish relevant research
- Funding for applied research is available and individuals develop a desire to participate in research and are mentored
- NGOs and other stakeholders are recognised and have a contributory role in operational research
- Good quality, appropriate, and relevant research gets translated into policy and practice and thereby has a spin-off effect to stimulate more research
One challenge is that foreign academic institutions often have the funding, time, and mandate for research and thus the associated power in decisions about what gets done. Local institutions should also be supported with money and staff for operational research, thus allowing them the necessary independence to make decisions, take responsibility, and establish partnerships that are more equal in resources and decision-making power.
The authors suggest building a research agenda into district and national programmes, based on local needs, but primarily reflecting the research priorities of the country. Within a country, it is important to have a coordination mechanism to provide a clear strategy of who sets research priorities and how choices are made at national level.
A bibliographic analysis of tuberculosis research done globally between 1997 and 2006 showed that Africa, which has the highest tuberculosis case rate burden in the world, contributed only 7% of global research output.
Local programmes have a tendency to outsource research to academic institutions, which then set up parallel research systems or affiliated sites. Although this might be a highly efficient means to produce quality research and scientific publications, if there is no satisfactory mechanism for integration, collaboration, and communication with the programme, this approach might hinder the development of operational-research capacity by drawing national researchers away from national programmes. Furthermore, because research institutions and technical agencies (either international or national) normally have no mandate or responsibility for implementing research findings after their studies are completed, the results often end up being sent or presented to busy programme managers, who have no ownership of the research and who are therefore unlikely to direct the effort needed to translate the research into policy and practice.
A change to a partnership model would enable greater involvement, co-ownership, and responsibility of programme staff along with researchers and policy makers. For example, the research question should be developed by the entire team, including those involved with questionnaire development, collection and analysis of data, and dissemination of the results. Planning at this stage also requires a clear engagement with the people who make decisions on policy so that they are aware of what is being asked, supportive of the research, and interested to learn about the findings.
Capacity and time for research activities, such as writing study protocols or dealing with peer review are often lacking within most programme settings but are essential to see research to completion. However, if they are planned as an essential part of the programme, they can be accomplished (see Box 1 for an example).
Box 1: Malawi National Tuberculosis Control programme
The experience of integrating operational research within the Malawi National Tuberculosis Control (NTP) programme is an example of how research elements can be successfully built into a national programme, and be of great value in shaping policy and practice.
Between 1996 and 2004, many studies were designed, planned, and budgeted within the NTP. The programme invested in a full-time operationalresearch officer and a data-management officer who worked alongside the programme manager and provided on-the-job training and supervision of research. Planned and continuing programmatic research was always an agenda item at the regular 6- weekly programme management group meetings. The NTP held an annual scientific review and dissemination meeting, and operational-research findings were presented by local and international investigators to all stakeholders. A medical editor was recruited to give an annual workshop on paper writing skills and on how to get the research published.
All publications in national and international journals were collated each year, and the resulting booklet was distributed to health-care workers around the country at national meetings and during supervision so that everyone had potential access to the results of locally generated research. All these components were built into the NTP plans with explicit budget streams, and these helped to develop the capacity to ask pertinent questions and to carry them through to publication for dissemination.
The failure to publish research is not just confined to the low-income countries: a recent report found that only 53% of 79 research studies reported in conference abstracts were published in peer-reviewed journals after nine years. Common reasons for so-called research ‘waste’ include the wrong choice of research question, poorly designed studies, and failure to publish relevant research promptly or at all.
There is a perceived need to create a socalled critical mass of trained researchers within health programmes to ensure that sufficient numbers of researchers continue in post to sustain future research. The Japanese Research Institute of Tuberculosis, the Japanese Foundation for AIDS Prevention, the International Union Against Tuberculosis and Lung Disease, and the US Centres for Disease Control and Prevention are among some of the institutions that support international training on operational research.
There is also a need to assess whether current capacity-building initiatives are having an effect, such as tracking personnel after training (e.g. through databases) to document the outcomes, explore any barriers, and capture suggestions for improving the situation. Many researchers from low-income countries, even after obtaining PhDs, do not take up research when working back in disease-control programmes. This might be because they end up in senior-level management posts, the infrastructure to plan or do research is lacking, or there are simply no opportunities
There are various ways in which these opportunities could be created. First, small grants could be offered to pursue locally applied research. Second, junior and senior operational research fellowships could be created for colleagues in low-income countries with active mentoring by international researchers, institutions, or NGOs.
Bureaucracy should be kept to a minimum with the main focus on deliverable outcomes that would include publications with specific benefits to programmes and communities. Targets for research output should be set (e.g. one or two research papers each year submitted to a peer-reviewed journal), with financial and technical support continued when targets have been met and termination of support if targets have been missed. Young national researchers should have the opportunity to present scientific abstracts and participate at regional and international conferences, and more attention should be paid to teaching the principles of how to write scientific papers and to mentoring.
Attention must also be paid to the problem of poor access to up-to-date scientific literature, and despite laudable initiatives (such as the Health InterNetwork Access to Research Initiative), this remains a barrier in low-income countries. Free and open access for all articles of interest to low-income countries is urgently needed. For example, Médecins Sans Frontières (MSF) has negotiated with publishers to allow free access to all articles written by its staff.
The distinct role for NGOs in operational research should be recognised for two main reasons. First, NGOs such as MSF often work in conflict settings, with marginalised and vulnerable populations, or with neglected diseases. Academics rarely have access to such settings, and national programmes might decide they do not have sufficient resources to study them. Research in these areas is, nevertheless, needed to better understand how to manage questions such as mental health issues in war zones, treatment and diagnosis of neglected diseases, or offering of HIV/AIDS care in slum settings. Secondly, NGOs are, by mandate, implementers and can thus be involved in the translation of research findings into policy and practice. If they have skills in research and advocacy as well as sufficient financial and human resources, then they probably have the potential to actively engage in operational research and help change practice.
However, NGOs are sometimes not the appropriate entities for designing or implementing research. They might lack the institutional support, culture, and skills for interacting with national programmes and decision makers. NGO focus might be on solving localised, short-term problems, they might have had little exposure to systems thinking .and they might lack the training and capacity to do rigorous research. They might also have a rapid turnover of staff, which hinders the sustainability of research and the ability to build up trust and understanding with country partners. These points might explain why NGOs rarely undertake research, are rarely asked by country programmes to do so, and why the research they do undertake is sometimes badly done, with little or no programme impact.
The authors conclude that they have made the case for the importance of operational research as a necessary component of health programming in low-income countries. What is needed now is further development of operational- research capacity, allocation of specific resources, and the need for different participants such as international and national academic institutions, national programme managers, and NGOs to work together in promoting operational research.
1Basinga. P et al (2011). Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet 2011, 377: 1421-28
1Zachariah et al (2009). Operational research in low-income countries: what, why, and how? The Lancet Infectious Diseases, Volume 9, Issue 11, Pages 711 - 717, November 2009
More like this
By Stephanie Stern and Melchior de Roquemaurel Stephanie Stern leads the Action Against Hunger LAB project which aims to reinforce the impact and uptake of knowledge. Before...
Summary of published paper1 A recent article in the Lancet argues that very little attention has been devoted to the relationship between external human rights contexts and the...
This overview of the EEFSG and report on the recent Dublin meeting was prepared by Niall Cassidy, Concern Worldwide, in consultation with the meeting participants. Concord is...
View this article as a pdf Lisez cet article en français ici By Helen Young, Abdalmonim Osman, Dr. Marshak, Anne Radday, Emmanuella Olesambu, Nola Jenkins, Darana...
Summary of letter1 and responses2,3 A typical example of a shebeen in an informal settlement, Namibia This article raises a number of issues about corporate sponsorship in the...
By Stephanie Stern Stephanie Stern leads the Action Against Hunger LAB project which aims to reinforce the impact and uptake of knowledge. Before joining Action Against...
FEX: Accessing quality research to enable evidence-informed national policy: experiences from Zimbabwe
By Willie Ganda View this article as a pdf Willie Ganda is Director for Research Development and Innovation, Ministry of Higher and Tertiary Education, Science & Technology...
Summary of panel discussion1 Alice Obrecht of ALNAP moderated this session. Panellists were Gwen Luc of Action Against Hunger (AAH) Link Nutrition Causal Analysis (NCA)...
By Jeremy Shoham and Marie McGrath, Field Exchange co-editors ENN was invited by Evidence Aid to share our perspective regarding evidence in emergencies in their online blog....
View this article as a pdf In this issue of Field Exchange we are delighted to feature, for the second year running, a special section that shares key outputs of Action...
Tesfaye Hailu has an MSc in Human Nutrition. He has worked for the Ethiopian Public Health Institute (EPHI)1 for the last eight years as a researcher. He is also a member of...
View this article as a pdf Barbara Baille is a Nutrition Advisor at Capacity for Nutrition-National Information Platforms for Nutrition (C4N-NIPN) Global Coordination team, a...
On 27th January 2016, ENN, London School of Hygiene and Tropical Medicine (LSHTM) and Save the Children co-hosted a one day meeting of a Management of Acute Malnutrition in...
By Andrew Mellen, Emmanuel International Andrew Mellen has been the relief programme manager with EI Malawi since 2003. With a background in agriculture, he previously worked...
By Mieke Moens, MSF Mieke Moens is a paediatric nurse, and is responsible for the PMTCT and nutrition programme for MSF Thyolo, Malawi The author would like to acknowledge...
FEX: Capacity support to states in fragile and conflict affected situations: an analytical framework
Summary of research1 Location: Global What we know: Strengthening state capacity to deliver services is a common aid initiative in fragile and conflict affected contexts....
FEX: Health systems strengthening in fragile contexts: A partnership model in South West State, Somalia
By Kevin Paul Mackey and Sirya Ezekiel Kiptum View this article as a pdf Lisez cet article en français ici Kevin Mackey is the Programme Development and Quality...
FEX: From the editor
Ethiopia is a diverse country where a significant proportion of the population live on or below the poverty line, where food insecurity is widespread and rates of acute...
View this article as a pdf Dear Field Exchange readers, Welcome to the 64th edition of Field Exchange. We are delighted to kick start 2021 featuring a special section on...
Summary of review1 A recent review commissioned by UNICEF set out to develop a global map on the status of Communitybased Management of Acute Malnutrition (CMAM) with a focus...
Reference this page
Operational research in low-income countries: what, why, and how?. Field Exchange 42, January 2012. p18. www.ennonline.net/fex/42/operational