Menu ENN Search

The use of evidence in humanitarian decision making

Summary of study1

Location: Ethiopia, DRC and Philippines

What we know: Decision making in humanitarian response requires timely information and analysis and there are ongoing efforts to coordinate and improve needs assessment to inform decision making. How decisions are made in practice is influenced by many factors.

What this adds: The influence of evidence on programmatic response is limited by previously decided strategic priorities, is considered selectively by decision makers, is influenced by personal experiences and is typically used to justify rather than determine interventions. An ‘automatic’ response is common in chronic situations; limited response flexibility within agencies and by donors means there is little incentive or capacity to innovate according to need. The decision-making process lacks transparency and is poorly documented.

A recent paper reports the results of a study undertaken during 2012 by Tufts University to address the question of how assessments and other sources of information and analysis are used by humanitarian decision makers. The study is based on a combination of literature review, case studies (Ethiopia, Democratic Republic of the Congo (DRC) and Philippines), and key informant interviews.

The study asks three main questions. First, how do decision makers in the humanitarian sector currently use information and analysis? Second, what factors, other than information and analysis, are influential in making decisions? Third, what would enable better-informed response decisions?

In order to address these questions, the study looks first at some of the main processes of decision-making in the humanitarian sector and the factors that appear to have most influence on decisions of different kinds. It goes on to look at the way information and analysis is currently generated in the humanitarian sector—both through formal and informal means—and related questions of relevance and credibility. These two topics are then brought together in addressing the question of the use of information by decision makers and what might enable more informed and evidence-based response decisions.

In reviewing the way decisions are made in practice, the study considers the ways in which such information is used (or not) at different points in the process, which varies across different kinds of decisions in different contexts.

Findings

Overall, the study revealed high levels of diversity in the contexts for decision making, as well as in the use of information and analysis. Some patterns emerge, however. In those contexts where strong governmental systems exist, the generation and use of information is either highly controlled by government (Ethiopia) or else is dominated by government-led systems (Philippines), with international actors playing only an auxiliary role. Most of the key decisions regarding resource allocation are in effect made by local and national government officials in these cases, on the basis of national or regional plans. Domestic political factors represent a significant potential bias which risks distorting the data available. That said, there are checks in most systems. In Ethiopia for example, international actors partner with central and local government in both the assessment of need and the provision of relief. While political bias may affect which areas are prioritised for relief, major discrepancies between assessed and “stated” need are hard to disguise, and the larger international donors have a substantial influence over the recipient government in this regard. Thus, although the validity of the published data may be questionable (Ethiopia), the process of micro-resource allocation and programme design is able to a substantial degree to iron out some of the more obvious anomalies at the local level. In contexts where government is relatively absent from humanitarian decision making, a different set of factors are at play. In the most extreme cases (Somalia, Eastern DRC, parts of Afghanistan), government systems are almost completely absent or bypassed by the international system. Here the dominant political narrative is an international one and it provides the backdrop for macro-resource allocation decisions. The biases in these cases come as much from pre-determined international strategic priorities as from domestic factors; aid is provided as much in proportion to an area’s strategic significance as it is based on assessed need. This is evident in the ebb and flow of funding in response to annual appeals (Consolidated Appeals (CAP), etc.), which fluctuates more according to foreign policy agenda, like counterterrorism and stabilisation, than it does according to apparent need. This factor also affects those countries like Central Africa Republic whose international profile and related strategic priority is low. The threshold for response is correspondingly higher in such contexts.

In this second category of contexts, the data available come mainly from international agencies. In most crisis contexts, however, there is a mix of government-generated (e.g., National Disaster Management Authorities (NDMA)) and external agency-generated data and analysis. Increasingly the mainly UN-led Clusters or else government- led coordination bodies are attempting to bridge the gap between the two. Joint assessment processes are one feature of this, an attempt to forge consensus and buy-in, as well as to streamline and harmonise data collection. This has potential strengths and weaknesses from the point of view of evidence-based responses. The main strengths come from comparability of data and ‘buy-in’ for the process and its results. The main weaknesses relate partly to the often cumbersome and slow nature of these joint processes, and partly to the potential for ‘group think’ to dominate the related analysis. In this regard, independent assessment and monitoring processes (e.g. by individual agencies) continue to be an essential part of the evidential picture, often acting as early warning or corrective to the wider system, whose processes may not be responsive to significant changes at either the micro or macro level.

Despite the diversity of contexts for decision making, it is possible to draw a number of conclusions from the study.

Conclusions

Decision making

In most decision making processes in the sector, the range of options is limited by previously decided strategic priorities, resource allocation and other biases. In some cases, these are parameters set by host government authorities, in other cases, they are set more by donors and by implementing agencies. This significantly limits the extent to which decisions are open to influence by evidence, particularly where organisational incentives to generate and respond to new evidence are limited.

Children during the floods in the Philippines in 2009

The extent to which decisions are ‘predetermined’ varies according to the type of decision. In some of the cases reviewed, the dominant political narrative and relative strategic priority given to the country/crisis in question was the factor that had by far the most significant bearing on strategic decisions about crisis response (approach, level of funding, etc.). In some cases of protracted crisis like DRC and Ethiopia, it is more about programmatic inertia: programmes ‘roll’ from year to year without fundamental reassessment of approach. Where programmes are more responsive to context, this tends to be at the lower levels of decision making and at the more local level of programming.

Decision makers may be highly selective in their uptake and interpretation of evidence. Personal biases, rules of thumb, and mental models – as well as a variety of (dis) incentives – may prevent individuals and organizations from responding to a situation in the way that evidence appears to demand. It is common for experienced staff to base decisions mainly on past experiences, instinct, and assumptions – even in the face of contradicting evidence. In institutional terms, this in turn leads to building agency capacity around established interventions types, which continue to be the ‘preferred response’ with each new crisis, irrespective of available evidence.

The use of standard predefined response packages is now being challenged, particularly in the area of food security and livelihoods. In evidential terms, this should involve combining evidence about context with historic knowledge about “what works.” Yet it remains the case that very few agencies conduct a formal or structured analysis of the various options and base decisions on the evidence that points to the most appropriate response. Even when assessment is viewed as a priority for programme planning, agencies often disregard field-validated assessments as a precursor to intervention. Ultimately, the choice of response does not always involve an evidence-based, analytical process.

Current processes of decision making tend to be undocumented and untransparent. It is therefore hard to judge whether or how information and evidence have been used to inform them. In particular, key assumptions are often unstated and therefore hard to test.

Generation of evidence

Relatively few documented needs assessments are available beyond the confines of the agencies that conduct them. There has been a rise in the number of joint (multi-agency, multisector) needs assessments, and increased focus on the use of the Multi-Cluster Initial Rapid Assessment (MIRA) tool in rapidonset crises. This is a significant advance, although there remains a lack of genuine multi-sectoral analysis with the result that responses remain largely “siloed.” To date, there has been less progress on joint assessment in protracted crises.

Even where documented assessments exist, the link between assessment and decision making appears weak. Assessments are still largely front-loaded and used to justify proposals or appeals (Flash Appeal, CAP, etc.). It remains the case that most assessments are conducted in order to substantiate a case made for funding by a particular agency to do a particular thing. Inevitable biases result in a lack of credibility – both of the analysis and of proposed interventions based on that analysis.

Clearly, there is still room for improvement regarding the processes of data collection and needs assessment, but this is not a silver bullet for achieving improved decision making. First, larger, systemic changes must occur whereby there are better incentives for generating and using evidence in decision making. Second, ongoing assessment and situational monitoring must be more widely adopted. However, in order for this to be effective in improving humanitarian response, the wider humanitarian system must allow flexibility for agencies to adapt programmes to meet the changing needs throughout the duration of a crisis. Third, quality analysis and the use of evidence must be highly valued through increased investments in diagnostics. Fourth, the evidence base proving which humanitarian responses are most effective is extremely lacking. Investments must be made in the consolidation of evidence about what works in response to different kinds of needs in different contexts. Fifth, the way evidence is presented is often crucial to its uptake. Knowing how to present it, to whom, and in what form may be essential to informed decision making.

Show footnotes

1Darcy J et al (2013). The use of evidence in humanitarian decision making Assessment Capacities Project (ACAPS). Operational learning paper. Feinstein International Centre January 2013. http://www.acaps.org/img/documents/t-tufts_1306_acaps_3_online.pdf

More like this

FEX: Needs Assessment and Decision Making in Emergencies

Summary of published report1 Sorghhum plant hit by drought Researchers from the Humanitarian Policy Group (HPG) at the Overseas Development Institute (ODI) recently undertook...

FEX: Links between needs assessment and decisions in food crisis responses

Summary of published review1 Workers wade into the Sea to collect WFP food at the southern Somali port of Merka A recently completed study under the WFP Strengthening...

FEX: Shock-responsive social protection systems research

Shock-responsive social protection systems research Summary of research1 Location: Global What we know: Cash is increasingly used in humanitarian response; there...

FEX: Strengthening nutrition humanitarian action: Supporting humanitarian cluster/sector coordination transition

By Peter Hailey and Brenda Akwanyi Peter Hailey is founding Director of the Centre for Humanitarian Change (CHC), a humanitarian think tank based in East Africa. He has over...

FEX: Livelihoods analysis and identifying appropriate interventions (Special Supplement 3)

3.1 Livelihoods assessment and analysis in emergencies The livelihoods framework provides a tool for analysing people's livelihoods and the impact of specific threats or shocks...

FEX: Experiences of the ‘Whole of Syria’ coordination for nutrition

By Saja Farooq Abdullah and Lindsay Spainhour Baker Dr Saja Abdullah is currently working as the 'Whole of Syria' (WoS) coordinator with UNICEF MENARO office. Saja is a...

FEX: Emergency Market Mapping and Analysis (EMMA) tool

By Lili Mohiddin and Mike Albu Lili Mohiddin has been an Emergency Food Security and Livelihoods Advisor with OXFAM GB since September 2005, based in the UK. Mike Albu is an...

FEX: The Syria Needs Assessment Project

By Yves Kim Créac'h and Lynn Yoshikawa Yves Kim Créac'h is currently the Project Lead for the SNAP Project. He is a seasoned humanitarian worker, with 15...

en-net: National Nutrition Cluster Co-coordinator South Sudan: Concern Worldwide

National Nutrition Cluster Co-Coordinator, South Sudan

About the role: This is a 12 month, role with unaccompanied terms based in Juba with a salary of Grade 6...

FEX: Robust evidence for an evidence-based approach to humanitarian action

Research By Mike Clarke, Jeroen Jansen and Claire Allen Mike Clarke is one of the founders of Evidence Aid and is now voluntary Research Director and Chair of the Board of...

en-net: Hiring: Emergency Response Personnel (ERP) – Nutrition Adviser, Save the Children UK

JOB TITLE : [b]Emergency Response Personnel (ERP) – Nutrition Adviser[/b] TEAM/PROGRAMME : [b]Humanitarian Department[/b] GRADE: [b]3[/b] POST TYPE:...

FEX: Real time evaluation of Pakistan Flood Response

Summary of evaluation1 The Pakistan floods of 2007 devastated large swathes of rural Sindh and Balochistan Provinces in southern Pakistan, destroying homes, crops and roads,...

FEX: From the editor

This issue of Field Exchange gives extended coverage to a briefing paper just released by Oxfam and SC UK on the 2011 response to the Horn of Africa crisis. This paper argues...

FEX: Designing an inter-agency multipurpose cash transfer programme in Lebanon

By Isabelle Pelly Isabelle Pelly was Save the Children's Food Security & Livelihoods Adviser in Lebanon until September 2014, and co-chair of the Lebanon Cash Working Group....

en-net: FEWS NET VII - Food Security/Early Warning Communications Advisor (Washingon, DC) Needed

FEWS NET is a complex activity designed to deliver evidence-based analysis and early warning of hazards, food insecurity, vulnerability to food insecurity, and famine to the...

FEX: Climate Changeas a driver of humanitarian crisis and response

Summary of published research1 Thousands of Somalis have been displaced by what is described as the worst floods in the country in 10 years. Tufts University has recently...

FEX: Aid effectiveness and Vulnerability Assessment Framework: determining vulnerability among Syrian refugees in Jordan

By Hisham Khogali, Lynette Larsen, Kate Washington and Yara Romariz Maasri Hisham Khogali is an independent consultant with 19 years of experience in a range of humanitarian...

FEX: Who’s Listening? Accountability to affected people in the Haiyan Response

Summary of case study report1 Report front cover image Thanks to Margie Buchanan-Smith and Sarah Routley, researchers and co-authors of the original report, for sharing this...

FEX: Development of multi-cluster rapid and in-depth assessment methodologies in Afghanistan

By Anna Ziolkovska, Hassan Ali and Baseer Qureshi Anna Ziolkovska is currently the Nutrition Cluster Coordinator in Yemen. Anna has a PhD and over ten years' experience in...

FEX: Can the Nutrition Information System be ‘trusted’ to build on available data sources?

By Patrizia Fracassi Patrizia holds an M.Sc. in Development Management. Over the past two years, she has consulted in Ethiopia for UNICEF and the World Bank. She previously...

Close

Reference this page

The use of evidence in humanitarian decision making. Field Exchange 45, May 2013. p8. www.ennonline.net/fex/45/decisionmaking