Evaluation of MSF Holland mission in Afghanistan

Kandahar, Afghanistan, 2002
Summary of Evaluation1
Last year, MSFH Holland (MSFH) commissioned an evaluation of their activities in Afghanistan between May 2000 and May 2002. The evaluation set out to:
- understand MSFH's role in Afghanistan and how MSFH adapted nutritional and health programmes in a rapidly changing political and operational context in the cultural setting of Afghanistan
- examine how MSFH used its proximity in the region to witness violations of humanitarian principles and law and how MSFH positioned itself in its advocacy work.
This summary of the evaluation findings concentrates of the first of these objectives.
Methods
Two external consultants with backgrounds in nutrition and health undertook the evaluation. Field work took place in Kabul, Herat and Kandahar where interviews were carried out with MSFH project staff , personnel from INGOs, local NGOs, UN agencies, and MoPH staff. The consultants also attended a regional workshop in Ashgabad and a medical coordination day in Amsterdam. Phone interviews were conducted with staff in Quetta, Pakistan while a number of MSFH staff no longer involved in the programme were interviewed in London and Amsterdam.
Limitations of the evaluation
Limitations included lack of a central repository for reports and information on the programme. Much information was lost or misplaced during the evacuation. The high staff turnover meant that most MSFH international staff connected to the programme over the time period in question were no longer in Afghanistan at the time of the field visit nor were many key personnel from other INGOs and UN agencies present during the period under question.
Findings
Within a context of the appalling health status of women and children and poor access to basic health facilities in Afghanistan, the focus of MSFH's operational activities was affected by 'what the Taliban would allow', security , a developing drought induced food crisis, lack of reliable information and how MSFH perceived its role (i.e. whether it should focus on needs arising out of crisis, human rights monitoring, or health needs of the most vulnerable). Over the time period in question the focus changed from improving health care access in rural areas to focusing on health needs in cities and eventually to safe motherhood in remote areas.
MSFH's primary analysis of the impact of the drought was in keeping with many other agencies, i.e. that there was a developing food crisis, which required emergency food and nutrition interventions. Although MSFH's perception was that a General Food Distribution (GFD) was the most appropriate response, staff were conflicted about whether MSFH should take on an implementing role. Ultimately the decision was taken (based on many factors and previous experience) that MSFH would not 'do' GFDs. MSFH therefore focused on advocating for improved GFD and later implemented Blanket Food Distributions (BFDs) as stop gap measures. Although much of the advocacy work was important, MSFH were largely unable to influence or affect the mobilisation of GFDs in areas where they were implementing BFDs.
MSFH implemented traditional emergency feeding programmes in a number of open settings partially to collect data which would be useful in advocacy but also to prevent severe malnutrition. However, there was limited evidence of 'alarming' levels of malnutrition and traditional feeding programmes were found to be inappropriate to the cultural setting so that eventually these were either discontinued or adapted (e.g. mobile SFP).
Thorough investigation of outcomes in terms of malnutrition and mortality was not rigorously undertaken by MSFH. However where other agencies managed to undertake thorough representative investigation mortality was found to be high and related to disease while levels of malnutrition were lower than expected in the context of a food crisis.
There was a considerable need within Afghanistan for credible information on how the drought was impacting the health and nutrition of the population. As a health agency with historical capacity in nutrition the evaluators suggested that MSFH could have capitalised more on their comparative advantage in the health sector by analysing and highlighting the health component of the crisis and by focussing their interventions more fully on the health sector, i.e. investing more resources in further strengthening and expanding health programmes and integrating nutrition components (where necessary) into these health programmes. Also MSFH was well placed in the regional paediatric ward to strengthen and support health information systems and analysis and strengthen links in referring clinics, however this was not done. This may have been a better option than focussing on a sector (food aid) where MSFH were dependent upon the commitment and response capacity of other agencies.
MSFH placed too much confidence in rapid MUAC assessments which it saw as a viable alternative to weight for height surveys. Rapid MUAC assessments found alarming rates of Global Acute Malnutrition (GAM) which were not the case where weight for height surveys were undertaken. Similar prevalences of GAM were not identified in a limited number of surveys reviewed where both indicators were measured. The correlation between weight for height measurements and MUAC requires further analysis in this setting before MUAC can be used with confidence in place of weight/height to define prevalence of malnutrition. Trend analysis was also affected by assessments carried out on changing populations and using varying age and height cutoffs.

Weighing supplementary rations, Mazlak IDP camp, Herat 2002
The high dependency on expatriate staff with sometimes limited experience and short contracts, for project management led to lack of continuity of approach. The difficulties experienced by MSFH in human resource management were often related to inability to fill positions by expatriates. Yet MSFH's experience has shown that management capacity can be accessed locally.
At the time MSFH did not have policies which addressed some of the issues that arise in connection with working in the type of longer- term health crisis that exists in Afghanistan. Such a context raises issues on project management, how to work with, through or independently of local partners and infrastructure, the degree of dependence on expatriate staff, coherence of health strategies and methods of withdrawing from programmes. Arguably, this absence of policy led to a modus operandi that was not always appropriate for the context.
Key recommendations
The evaluation contained a number of recommendations related to the above findings. Key recommendations that related specifically to nutrition and food security aspects of the programme were as follows;
- MSFH could further clarify its policy and strategy on implementation of general food distributions based on an analysis of past experience and the internal debate on this issue that has taken place over the past decade. This clarified policy should set out options on what to do in circumstances where food insecurity is severe and undermining MSFH health programmes.
- MSFH could increase capacity to support epidemiological analysis in areas of operation as well as competence in integrating food security, health status and nutritional status analysis.
- Unless the current consensus changes (as evidenced by new research) MUAC assessments and surveys should be treated with the appropriate caution.
1O'Reilly F., Shoham J. MSF Holland in Afghanistan Mission Evaluation: May 2000-May 2002
More like this
FEX: MSF experiences from Afghanistan: Maslakh camp
by Saskia van der Kam Saskia van der Kam is the headquarters nutritionist in MSF Holland. This article draws on her field trips to Afghanistan and a number of other MSF Field...
FEX: MSF Holland
Name MSF Holland Year formed Staff (2003) 1984 Address Plantage Middenlaan 14 PO Box 10014 1001 EA Amsterdam The Netherlands Overseas 795 Telephone 00 31 20 520...
FEX: From emergency food aid to sustainable food security: 10 years of agricultural recovery in Afghanistan
By François Grunewald François Grunewald is an agricultural engineer specialising in the rural economy. He has worked in the field of crisis and post crisis operations since...
FEX: Calculating wasting caseloads and geographic prioritisation of nutrition services in the context of limited data in Afghanistan
View this article as a pdf By Beka Teshome Bongassie, Said M Yaqoob Azimi and Maureen L. Gallagher Beka Teshome is Nutrition Cluster co-lead for Afghanistan with Action...
FEX: Ambulatory treatment of severe malnutrition in Afghanistan
By Emmanuelle Lurqin Emmanuelle is a paediatric nurse and since 2000, has worked with MSF Belgium on nutrition programmes in Angola, Burundi, and Afghanistan. She is currently...
FEX: Targeted Food Distribution to Women and Children in Northern Afghanistan
By Regine Kopplow, Concern Worldwide Regine Kopplow is a senior nutrition advisor with Concern, working in Afghanistan since April 2002. Previous field experiences include...
FEX: Scurvy outbreak in Afghanistan: an investigation by Action Contre la Faim (ACF) and WHO
By Philipe Leborgne, Caroline Wilkinson, Sylvie Montembaut, and Mija Tesse Ververs Dr Philippe Leborgne has been Head of the ACF Medical Department since 1994. Before this he...
FEX: Erratum
The article in Field Exchange, November 2002, Issue 17: Scurvy outbreak in Afghanistan: An investigation by ACF and WHO, by P. Leborgne, C. Wilkinson, S. Montembault and MT...
FEX: Estimating ‘people in need’ from combined GAM in Afghanistan
View this article as a pdf Lisez cet article en français ici By Alexandra Humphreys, Bijoy Sarker, Baidar Bakht Habib, Anteneh Gebremichael Dobamo and Danka...
FEX: Capacity Building in Times of Emergency: Experiences From Malawi
By Claire de Menezes Claire de Menezes is a paediatric nurse specialising in infectious disease. Having spent 16 months working in Sudan with Action Contre la Faim and one...
FEX: Suspected toxic ingestion outbreak in central Afghanistan
By Gerald Martone, International Rescue Committee (IRC) Gerald Martone is the Director of Emergency Response at the International Rescue Committee's Headquarters in New York....
FEX: Lessons learned in West Darfur: challenges in assessment methodologies
By Aranka Anema Aranka Anema has a background in medical anthropology. She is currently working with MSF-Holland as Medical Editor and has been involved with the Nutrition...
FEX: Real-time evaluation of WFPs tsunami response
Summary of published report1 Sorting WFP food in Indonesia The World Food Programme (WFP) recently published the findings of a real-time evaluation (RTE) of their response to...
FEX: Letter on Field Exchange quality control, by André Renzaho
Dear Field Exchange A need for quality control, consistency and diligence for articles published in Field Exchange. As an experienced practitioner and trainer in refugee...
FEX: From the editor
This issue of Field Exchange gives extended coverage to a briefing paper just released by Oxfam and SC UK on the 2011 response to the Horn of Africa crisis. This paper argues...
FEX: Malawi food shortage: how did it happen and could it have been prevented?
By Sarah King Sarah King is currently working as an Emergency Capacity Building Officer with Christian Aid. Having completed a MSc in Public Health Nutrition at LSHTM, she...
FEX: Nutrition assessments in Zimbabwe: a local perspective
By George Kararach For the past two years, George Kararach has worked as a consultant policy analyst in Zimbabwe - for the last year working with UNICEF Zimbabwe. The support...
FEX: From Kigali to Istanbul the long way round – personal reflections on 20-years of humanitarian accountability
By Andy Featherstone Andy Featherstone is an independent consultant with over 20-years of experience in strategic and operational management of international programme, policy...
FEX: Food security assessment of high altitude villages of Badakhshan, Afghanistan
By Salim Sumar, Laila Naz Taj and Iqbal Kermali Dr Salim Sumar heads Focus Humanitarian Assistance Europe Foundation, which is an affiliate of the Aga Khan Development...
FEX: Scale-up of IMAM services in Afghanistan
By Ahmad Nawid Qarizada, Piyali Mustaphi, Jecinter Akinyi Oketch and Shafiqullah Safi View this article as a pdf Lisez cet article en français ici Ahmad Nawid...
Reference this page
Evaluation of MSF Holland mission in Afghanistan. Field Exchange 20, November 2003. p19. www.ennonline.net/fex/20/msf
(ENN_3776)