Menu ENN Search

Peer review of WFP evaluation process

Summary of report1

A mother waits for WFP food aid distribution with her child in Malawi

A recently commissioned peer review of the evaluation process and function within the World Food Programme (WFP) has been completed and posted on the WFP website2. The review assessed the central evaluation function, i.e. the Office of Evaluation (OEDE) of WFP, as its starting point, but also included analysis of decentralised evaluation in WFP. The review was based upon preparatory desk study, field visits to WFP regional bureaux and country offices, a meta-evaluation of twelve OEDE evaluations, a web-based survey of the views of WFP staff (87 responses) and peer panel interviews with selected stakeholders.

The panel concluded that the independence of WFP evaluations was adequate in comparison to similar organisations, however the credibility of products was uneven. While the process of evaluations was somewhat more credible, they were also problematic. The criteria of utility of evaluations were partially met with regard to contributing to programming, but structures and mechanisms to promote utility were weak in most other respects. In addition, OEDE is a strong unit with committed, well-trained and highly motivated staff. Over the past seven years, OEDE has invested much effort on improving evaluation. However, evaluation is of more variable quality at the level of Regional Bureaux and Country Offices, with levels of motivation and invested resources dependent upon the interest and priorities of the offices concerned.

Independence

Evaluation resources are currently safeguarded while OEDE is outside of line management whilst, at the same time, sufficiently integrated into WFP leadership structures to facilitate impact. However, accountability for the implementation of recommendations is unclear. Some OEDE staff are concerned that their careers may be affected by their evaluation role, which could lead to inappropriate risk averse behaviour in their management of sensitive evaluations. There are also insufficient safeguards to prevent partiality and conflicts of interest amongst external evaluators. The role of Regional Bureaux in both oversight and advisory support to Country Offices has problematic implications for the independence of their role in decentralised evaluation. Their evaluation function involves public critique of programming while they must also ultimately maintain collegial day-to-day relationships with Country Offices.

Credibility

Although WFP has an evaluation policy, it is a layered series of documents that detracts from clarity and applicability. Evaluation policy is not sufficiently used to guide practice. Evaluators and Regional Bureaux have been unclear regarding what is expected in terms of quality, due to a lack of specification within OEDE itself and concerns that headquarters expectations do not take into account resource and time constraints in the field. On the whole, OEDE evaluation function is impartial with the views of all stakeholders often sought. However, there appears to be an uneven emphasis on stakeholders who are more accessible and articulate, with beneficiary views, in particular, under represented. The process of preparing for evaluations, management, and advising and supporting teams in the field is handled in a highly professional manner by OEDE. Terms of reference are generally of good quality, but they are, at times, too standardised and over ambitious. The quality of evaluations is mixed but should improve with the planned establishment of new quality standards by OEDE. A failure to take into account the cost implications of recommendations, together with factors related to the nature of priority setting in WFP, has damaged credibility of evaluations among some WFP staff.

Utility

Evaluation is insufficiently integrated into many of the processes by which WFP sets, monitors and analyses policies. Evaluation is primarily focused on outputs, as opposed to outcomes and impact, which reflects the demands of many stakeholders. In a narrow sense of contributing to an understanding of how to 'do things right', evaluation makes a notable contribution to programme design and management. In a wider perspective of learning about 'doing the right thing,' performance is not so good. There have been some efforts within evaluations to present evidence that can stimulate greater reflection within WFP - over the changing role of food aid, for example. However, the corporate view of evaluation has tended to focus primarily on its utility for making modest adjustments to existing approaches. New OEDE plans, to tie evaluation closely to logical frameworks, may enhance utility through a focus on outcomes; but lack of prevailing understanding and use of logical frameworks within WFP will make it difficult. At decentralised levels there is a close link to utility, since there is a direct desire to use evaluation to inform and justify new programmes and phases.

Evaluation makes an inadequate contribution to overall knowledge building within WFP and virtually none among partners. Access to reports and findings through website, debriefings, etc, is acceptable, but promotion of the use of evaluation products is not sufficiently proactive.

Recommendations

OEDE should develop an evaluation policy as a transparent vehicle for promoting greater communication among internal and external stakeholders, regarding the aims and intended utility of evaluations.

OEDE should develop an 'accountability map' of key WFP stakeholders, both internal and external, to help in clarifying roles and responsibilities.

OEDE should look for ways of promoting, and providing incentives for staff to adopt more participatory approaches to evaluation. Engagement with partners at country, regional or global levels is primarily a responsibility of other parts of WFP.

After an evaluation has been submitted to the Executive Director, OEDE should not be involved with drafting or compilation of responses from different parts of the organisation. The management response mechanisms should include rules about the timeframe for the response and procedures for follow up, as well as for reporting to the Executive Board. A similar system for management response should be used for decentralised evaluations. Management response and follow up mechanisms should be transparent with relevant documents easily accessible for WFP and partners and routinely posted in electronic form.

The capacity of OEDE staff should be maintained over time to stimulate interest in the evaluation field and encourage professionalism.

The learning element of evaluations should be linked to a larger organisational knowledge management strategy. OEDE should continue recent efforts to systematically harvest lessons from existing evaluations, as well as external fora such as ALNAP3, the Inter-Agency Standing Committee (IASC)4 and relevant partners.

OEDE should develop a more transparent, rigorous and competitive approach to selection of team leaders. All evaluation teams should include at least one evaluation specialist, preferably the team leader.

In order to address concerns that only a small portion of the overall evaluation budget is within the direct control of OEDE, WFP's senior management should devise ways to safeguard the funding allocated to evaluations for the next biennium. The establishment of a centrally managed fund for both OEDE evaluations and decentralised evaluations should be investigated.

Show footnotes

1Baker. J et al (2007). Peer review of the evaluation function at WFP. Stockholm, November 5th, 2007. The report is available at http://www.uneval.org/indexAction.cfm?module=Library&action=GetFile&DocumentAttachmentID=2003 or on the WFP website (see footnote 2)

2http://wfp.org/operations/evaluation/doclist.asp?section=5&sub_section=8&EBDoc=WFP/EB.1/2008/7-A&Year=2008&ID=WFP153652

3Active Learning Network for Accountability and Performance in Humanitarian Action, http://www.alnap.org

4http://www.humanitarianinfo.org/iasc/

More like this

FEX: Improving impact evaluations

Summary of published research1 The ENN has long-championed the need for more evidence based learning in food and nutrition emergency programming and in particular more rigorous...

en-net: Average Family Size and WFP Emergency Operation in Kenya 2004-2006

I have been reading the WFP evaluation of its Emergency Programme in Kenya EMOP 10374.0 (OEDE/2008/002) but I can find no information on whether rations were based on a...

en-net: Nutrition Specialist, World Food Programme, Nepal (closing 18 November)

TERMS OF REFERENCE U.N. WORLD FOOD PROGRAMME ? NEPAL COUNTRY OFFICE NUTRITION SPECIALIST POSITION: The Nutrition Specialist (Consultant), in addition to the regular duties...

en-net: WFP recruiting P3 level Regional Nutrition Officer

The World Food Programme's Regional Bureau for West and Central Africa is looking for a P3 level Regional Nutrition Officer to assist the Nutrition Unit in...

FEX: Issue 13 Editorial

Dear Readers, The first in a new series of articles appears in this issue of Field Exchange. 'Embracing Cultures', written by Ariana Curdy will, over the next few editions...

FEX: Framework for integration of management of SAM into national health systems

By Katrien Khoos and Anne Berton-Rafael Katrien Ghoos is the Nutrition Specialist on Management of Acute Malnutrition ,Nutrition Information Systems, Emergencies and Disaster...

FEX: SC UK Experiences of Food Security Information Systems

Summary of review1 SC UK recently conducted a review of their experiences of supporting Food Security Information Systems (FSIS) over the past 15-20 years. The review drew on...

FEX: Institutional development in Ethiopia: supporting an improved emergency response

By Jean Gladwin Dr Jean Gladwin is an independent consultant and researcher. She was seconded from the World Food Programme as a technical adviser to the Ethiopian government...

FEX: World Vision

Name World Vision Website www.worldvision.ca Address 1 World Drive, Mississauga, Ontario L5T 2Y4 Director The President of World Vision Canada is Dave Toycen Dr. Carolyn...

FEX: Food assistance for nutrition: Evidence Summit

By Lindsey Green Ongoing food crises around the world underscore the need for effective food assistance (defined as in-kind food aid and/or cash and voucher programming). As...

FEX: Developing a mobile health app to manage acute malnutrition: a five-country experience

By Tine Frank, Emily Keane, Natalie Roschnik, Colleen Emary, Melani O'Leary and Laura Snyder View this article as a pdf Tine Frank is a consultant engaged by Save the...

en-net: Invitation to Tender for evaluation - of GOAL USAID-funded Food Security Projects

GOAL Syria: Evaluation of Humanitarian Action Terms of Reference 1. Introduction 1.1. Background 2011 witnessed the start of nationwide protests in Syria against the...

FEX: The Evolution of Ethiopian Government’s Early Warning System

Dr Kassahun Bedada Beyi Dr Kassahun Bedada Beyi is Early Warning and Response Case Team Coordinator with the Early Warning and Response Directorate. This is located within the...

FEX: Evaluation of CMAM Pakistan: UNICEF country case study

Summary of evaluation1 A flood affected community in Khyber Pakhtunkhwa province CMAM has been implemented in Khyber Pakhtunkhwa province of Pakistan, with scale up in 2010...

FEX: Temporal Integration - Demand driven CTC (Special Supplement 2)

By Steve Collins A small scale market in South Sudan Early CTC programmes prioritised the timely provision of an appropriate level of care to a large proportion of the target...

en-net: 15-465: MONITORING AND EVALUATION DIRECTOR

Essential Job Duties/Scope of Work: BACKGROUND International Medical Corps is a global, humanitarian, nonprofit organization dedicated to saving lives and relieving suffering...

FEX: What does nutrition-sensitive programming mean for WFP?

By Kathryn Ogden, Geraldine Lecuziat, Mutinta Hambayi, Quinn Marshall, Ali Elnawawi and Josephine Lofthouse (WFP nutrition team, Rome) The WFP team acknowledge colleagues from...

FEX: Community Supplementary Feeding Programme (CSFP) Evaluation in Zimbabwe

The first Field Exchange carried an article about the CSFP in Zimbabwe: Nutrition in commercial farms; finding the right plaster for the wound. Since then the findings of a...

FEX: Emergency school feeding programmes

Summary of evaluation1 An emergency school feeding programme in Rumbek, Northern Bahr El Ghazal WFP have recently completed an evaluation of emergency school feeding...

en-net: WFP DRC IS SEEKING A NUTRITION ADVISOR

Terms of Reference: Nutrition Advisor/World Food Prorgamme/Kinshasa, DRC Background Despite large investment and government renewed committed to combat malnutrition, the...

Close

Reference this page

Peer review of WFP evaluation process. Field Exchange 33, June 2008. p29. www.ennonline.net/fex/33/peer