Menu ENN Search

Improving the assessment and attribution of effects of development assistance for health

Summary of research1

Location: Global

What we know: It is difficult to relate Overseas Development Assistance for health (DAH) and health outcomes; there are multiple funding resources, the contribution of health systems is typically underestimated and assumptions regarding the impact of external donor investment are overgenerous. Channelling funds through national delivery systems makes attribution to a specific funder difficult.

What this article adds: A set of agreements between donors on the principles underpinning assessment and reporting of the effect of DAH is suggested by the authors of a recent review. There are a number of promising initiatives on impact assessment (NOINE, 3ie, UN interagency groups, HIAs IHP+) to draw upon. Approaches should respond to country needs with appraisal of potential biased reporting; identify contextual factors affecting observed results; take into account confounding factors, such as economic growth or shocks; and adapt to different settings and needs. Lack of definition on what constitutes health systems strengthening activities and expenditures and methodological challenges reinforces the large knowledge gap in assessment of DAH effect. 


The authors of a recent paper in the Lancet suggest a set of agreements between donors on the principles underpinning assessment and reporting of the effect of Overseas Development Assistance for health (DAH).  Their suggestions stems from the methodological challenges in establishing a relation between Overseas Development Assistance for health (DAH) and health outcomes. The validity of the results presented by major funding agencies has been questioned because of weaknesses in models used to estimate outcomes, inappropriate counterfactuals and overgenerous assumptions of investment effects. Attribution of health outcomes to external investments by specific agencies has also proved to be challenging because of the multiplicity of funding sources. Furthermore, methods that attribute health effects to single interventions or specific funds do not reflect the vital contribution of health systems to health outcomes.

Most donors have committed to aid effectiveness principles underpinned by the Paris Declaration (2005) and the Accra Agenda for Action (2008), which stress national ownership and leadership, as well as harmonisation and alignment by donor agencies to national systems. Increasingly DAH is channelled through national delivery systems, making it difficult to attribute results to a specific funder. Yet most donor agencies continue to report their results.

The most commonly used approaches to assess and report the effect of DAH on health outcomes are tracking of trends in mortality, morbidity and coverage indicators; and estimation of lives saved and deaths averted. Methodological challenges include insufficient reliable data and poor disaggregation by age, sex, socioeconomic groups and geographical location. Assessment of the health effect of health systems strengthening (HSS) interventions with use of these measures is particularly challenging, because of the complex causal pathways through which these interventions are likely to have an effect. Furthermore, gaps in tracking input (e.g. HSS expenditures) and output indicators (e.g. measures of service availability, accessibility, and quality) limit the ability of studies to fully describe the sequence of inputs, activities, outputs and outcomes leading to an effect.

Donor agencies also use lives saved or deaths averted metrics to quantify the effect of DAH targeting disease-specific interventions, by comparison of health outcomes after the implementation of an intervention with the outcomes in similar geographical areas that did not receive the intervention during the same period; with outcomes in the same area(s) before the intervention was implemented; and with a hypothetical counterfactual scenario for the same area during the same period. Peer-reviewed models are used to estimate outcomes with these measures on the basis of estimated population coverage of interventions with well documented health outcomes. For example, the Global Fund has estimated lives saved by disease-specific interventions such as anti-retroviral therapy (ART) for AIDS and by insecticide-treated nets that reduce mortality.

Six points of interaction of DAH with national health systems have been identified: governance, finance, health workforce, health information systems, supply management systems and delivery of health services. However, there is no single framework to classify these interactions and research has identified several challenges, e.g. insufficient data, including for contextual factors that might have affected health effects and both the complexity of health system interventions and of the causal pathways leading to health effects.

Two main approaches used by donor agencies when attributing health effects to their funding are determination of plausible counterfactuals and establishment of exclusive attribution. When determining plausible counterfactuals, no intervention has frequently been used to attribute a health effect that could be linked to a specific source of funding, but determination of counterfactuals has been challenging, especially since an ideal control group was difficult to identify. The result attributed to a funder is sometimes assumed to be a proportion commensurate with the donor agency’s contribution to the overall investment for the intervention or country in question, although funding is often pooled at country level.

Promising initiatives on impact assessment       

There is as yet no consensus between key stakeholders on the strategies to improve and harmonise measurement approaches, but they could benefit from refinement and wider application of several promising approaches used by donors outlined below.

Beyond health, the Network of Networks for Impact Evaluation (NONIE) and the International Initiative for Impact Evaluation (3ie) support the use of study designs that enable analysis of interventions across the whole causal chain, from input to effects, and advocate the use of mixed methods for impact assessment to provide a more complete picture of effect.

In an attempt to strengthen country monitoring and evaluation platforms, the UN interagency groups (e.g. UN Evaluation Group, Inter-agency Group for Child Mortality Estimation, and Maternal Mortality Estimate Inter-agency Group) aim to develop new indicators, enhance capacity at country level to improve monitoring systems and strengthen data use, and harmonise work across partners on assessment and reporting of progress towards the Millennium Development Goals (MDGs).

Additionally, the Development Impact Evaluation Initiative by the World Bank provides technical assistance to projects and partners throughout the project cycle to ensure high-quality monitoring and effect evaluation.

One proposed approach, Health Impact Accounts (HIAs), aims to identify crucial value chains (such as infrastructure, transportation, procurement systems, etc) required for effective implementation of interventions (whether disease-specific or HSS interventions ) at country-level, and then apportions credit for effect to all investors, according to their relative contribution to different components of the value chain. 

Recent efforts, including those by International Health Partnership Plus (IHP+), have tried to improve and harmonise approaches used by donor agencies and partner countries to assess the effectiveness of DAH to reduce demands on weak health systems for donor-specific information; strengthen transparency and enhance comparability across different donor agencies; show the contribution of health systems to improvement of health; increase the efficiency of resource use; and provide reliable estimates of benefits of DAH. IHP+ encourages the documentation of results and of health aid effectiveness through in-depth individual country case studies. It also advocates joint annual reviews of country health strategies, including monitoring, evaluation, and review mechanisms, based on country monitoring platforms. However, the extent to which the IHP+ support to improved monitoring of national health strategies produces improved output information that could serve as a basis for further health impact assessment remains to be established. Joint annual reviews by several donors would improve harmonisation and mutual accountability by aligning donor assessment and reporting with national assessment and reporting mechanisms, thereby avoiding parallel donor-specific systems.

Justine Greening (UK International Development Secretary at the time) and The UK Prince of Wales at a WFP supermarket in the Zaatari refugee camp, Jordan. This visit marked the announcement of a further £100 million of assistance by the UK government for Proposed agreements between donors

A set of agreements between donors on the principles underpinning assessment and reporting of the effect of DAH are suggested:

First, approaches to assessment and reporting should mainly respond to country needs (including national governments and end beneficiaries) and emphasis mutual accountability. Country-owned monitoring and evaluation platforms need to be strengthened by external and domestic investments, and include robust additional studies. For example, a national evaluation platform that uses the district as the unit of design for natural, quasi-experimental or experimental studies and is based on continuous monitoring of core indicators can provide a rigorous comparison of effectiveness of interventions and different scale-up approaches.

Strengthening of monitoring and evaluation platforms should include a careful appraisal of potential incentives towards biased reporting, which exist in the relationship between partner countries and donors, especially in performance oriented initiatives such as GAVI’s (vaccine alliance) immunisation services support. Such perverse incentives might affect the accuracy of the data used to assess the effect of DAH, which in turn misinforms donors and their constituencies about their progress and might counteract efforts to strengthen country monitoring and evaluation platforms. Validated and standardised instruments and sampling frames, establishing systems for quality control and periodic surveys on a range of health topics that benchmark country-reported data, could help to address biases arising from perverse incentives.

Second, approaches to assess and report the health effects of DAH should identify the contextual factors both within and outside the health system that contribute to the observed results and might affect their wider applicability.

Photo 1

Third, there should be greater consistency and transparency in the selection of counterfactual scenarios, taking into account potential confounding by factors such as economic growth or conversely, economic shocks that can affect the determinants of health.

Fourth, approaches to assess and report the health effects of DAH should be adapted to different settings and needs. They should draw on a range of methods and designs dependent on what inference is likely to be made from them. They should be tested in both stable and fragile states, with coordinated approaches to monitoring sector performance that foster country ownership and reduce uncoordinated multiple donor programmes.

Substantial knowledge gaps in assessment of the effect of DAH exist, especially for investments in health systems. These knowledge gaps are accentuated by the absence of consensus about the definition of what constitutes HSS activities and expenditures.

Methodological challenges include double counting, underestimation of the contribution of health systems to health effect, and overlooking of system-wide effects of DAH. These challenges could undermine public confidence on the fidelity of the figures presented on the effectiveness of DAH. Exclusively attributing effect to disease-specific interventions supported by donor agencies could lead to channelling of greater financial support to disease-focused interventions than to HSS interventions, which could be particularly damaging in view of the plateauing in DAH since 2010. Some studies have suggested that disease-specific interventions have potential adverse effects on the health system, e.g. distorting national health priorities and diverting health workers from other responsibilities. However, there is paucity of empirical evidence to convincingly conclude whether the effects of disease-specific interventions on health systems have been negative or positive. Rigorous methods are needed to assess whether negative system-wide effects of DAH, including the potential skewing of expenditure (fungibility) are a substantial problem. Donor agencies need to develop their own skills and competencies and work with the research community to further address the methodological challenges inherent in assessment approaches.  

Improved measurement, enhanced comparability across different agencies and strengthened transparency in assessment of the effect of DAH should give credence to the results reported by funding agencies. To advance this agenda and appraise these new approaches in more depth is crucial, in view of the large amounts of funds spent, the growing recognition of the importance of HSS interventions to achieve global health goals and the political pressures on aid budgets at a time of economic austerity. 



1Ataya. N et al (2014). Improving the assessment and attribution of effects of development assistance for health., vol 384, pp 2256-2259, Dec 20/27, 2014

More like this

FEX: SC UK Experiences of Food Security Information Systems

Summary of review1 SC UK recently conducted a review of their experiences of supporting Food Security Information Systems (FSIS) over the past 15-20 years. The review drew on...

FEX: Public financing of health in developing countries

Summary of published research1 Increasing amounts of international aid have been given to health sectors in developing countries. Development assistance for health (DAH) has...

FEX: Transitioning nutrition programming from humanitarian aid to health system strengthening in Kenya

View this article as a pdf Lisez cet article en français ici By Peter Hailey, Brenda Akwanyi, Emiliana Mbelenga and Tamsin Walters Peter Hailey is the Director of the...

FEX: A Review of the advances and challenges in nutrition in conflicts and crises over the last 20 years

Abbreviated version of unpublished paper Food distribution at the ICRC kitchen in Tonj. By Frances Mason and Anna Taylor This paper is a shortened version of the complete...


BACKGROUND The new WFP Corporate Strategic Plan, which reconfirms the shift made by WFP from food aid to food assistance, explicitly puts an increased emphasis on measuring...

FEX: Review of integrated nutrition programming by ACF International

A pond to breed fish as a part of an income generation scheme in West Timor, Indonesia Summary of review1 Action Against Hunger International Network (ACF) recently...

FEX: Special Section: Research for Nutrition Conference - Editorial

Myriam Ait Aissa leads the Action Against Hunger Research and Analyses Department, which overall aim is to produce evidence for better action. Before joining Action Against...

en-net: Health & Nutrition Coord (Maternity), Malawi, Concern Worldwide

Description: The Health and Nutrition Programme in Malawi currently consists of a portfolio of large and small projects, across three districts, encompassing sexual and...

en-net: Nutrition and Health Manager / ACF / Nigeria / HSS profile expert

Nutrition and Health Manager Maiduguri, Borno, Nigeria * Nutrition and Health DESCRIPTION About Us Action Against Hunger-USA is a global humanitarian organization that...

en-net: 15-581: Country Director

Essential Job Duties/Scope of Work: International Medical Corps is a global, humanitarian, nonprofit organization dedicated to saving lives and relieving suffering through...

FEX: Meeting on approaches to address moderate malnutrition in emergencies

The Emergency Nutrition Network (ENN) and Save the Children UK (SC UK) recently held a meeting on approaches to address moderate malnutrition in emergencies (29th-30th May...

en-net: 15-536: Monitoring, Evaluation, Accountability and Learning (MEAL) Manager

Country:Turkey Department:International Operations Essential Job Duties/Scope of Work: PROGRAM BACKGROUND The conflict in Syria is entering its fourth year and shows no...

en-net: GOAL seeks Nutrition Coordinator in Niger

Since the food crises of 2005, GOAL has continuously supported the population of Mirriah district in Zinder Region through the implementation of community-based programmes...

en-net: UNHCR: Senior Nutrition Officer, P4, Geneva

Organizational Setting and Work Relationships The Senior Nutrition Officer is ultimately responsible for providing technical support, coordination, policy and planning,...

en-net: 15-402: Country Director - Afghanistan

Country:Afghanistan Department:International Operations Essential Job Duties/Scope of Work: International Medical Corps is a global, humanitarian, nonprofit organization...

FEX: Transitioning nutrition programming from humanitarian aid to health system strengthening in Kenya

This is a summary of a Field Exchange field article that was included in issue 64. The original article was authored by Peter Hailey, Brenda Akwanyi, Emiliana Mbelenga and...

en-net: JOB OPENING: Health and Nutrition Advisor, Medair

Role & Responsibilities Provide technical advice and support to the field management and the health, nutrition, and protection staff in Syria and to interact with Medair's...

FEX: Using flow charts and health systems strengthening to improve antenatal nutrition services in India

Vani Sethi is a Nutrition Specialist at UNICEF Regional Office of South Asia. Archana Mishra is a Deputy Director at the Maternal Health Division, Government of Madhya...

en-net: Sr. Technical Advisor, Nutrition - Team Lead Open Postion at CRS

To Apply About CRS Catholic Relief Services (CRS) is the official international humanitarian agency of the Catholic community in the United States. CRS works to save,...

en-net: Nutrition Coordinator -- Pakistan

Description General Objectives: This is a Senior Management position expected to develop national NUT programming and co-ordinate the effective technical implementation and...


Reference this page

Improving the assessment and attribution of effects of development assistance for health. Field Exchange 50, August 2015. p27.



Download to a citation manager

The below files can be imported into your preferred reference management tool, most tools will allow you to manually import the RIS file. Endnote may required a specific filter file to be used.