WFP Monitoring and Evaluation of HIV/AIDS Programming in Malawi

By Jeremy Shoham, ENN

Post distribution monitoring GOAL food distribution Chiradzulu 2

This article was written based on a WFP consultation to Malawi in February 2005.

During 2003, WFP Malawi significantly strengthened the monitoring activities for all its programmes in Malawi. The Post Distribution Monitoring (PDM) and Community Household Surveillance (CHS) programmes implemented by WFP are most relevant to targeting in Malawi and provide invaluable insights regarding the role of using proxy indicators to target people living with HIV/AIDS (PLWHA).

Community Household Surveillance (CHS)

The CHS is a regional initiative in six EMOP (Emergency Operation) countries and was initiated in early 2003. CHS is based on sentinel site monitoring and has four main objectives;

The two instruments for the CHS are a household questionnaire and a monthly focus group discussion. CHS is conducted by WFP field monitors, and staff from the National Statistics Office (NSO), Ministry of Agriculture and Irrigation, and NGO implementing partners.

CHS was initially piloted in Malawi in July 2003 in 30 sentinel sites. The first round took place in October 2003, with 30 sentinel sites in six randomly selected EMOP 10290 districts. Four Final Distribution Points (FDPs) per district were selected, with one village randomly selected from serving FDPs. In each village, 22 households were selected, comprising 11 beneficiaries and 11 non-beneficiaries. A second and third round CHS were conducted in February 2004 and October 2004 respectively, with a fourth round planned for February/March 2005.

The initial pilot CHS in July 2003 mainly collected information on the food security situation, including data on coping strategies and household perceptions on need for food aid. There was also some analysis of vulnerability. The first round of surveys collected and analysed data on food security and coping strategies but also focused on targeting. The first round report claimed that social targeting was successful, e.g. 37% and 27% female-headed households amongst beneficiaries and non-beneficiaries respectively. However, it stressed that the 'asset very poor' were less represented than richer households (see figure 1).

The second report, in February 2004, concluded that social targeting, i.e. female headed households, orphan containing households, elderly and disabled, etc, was successful, e.g. 41% of beneficiary households were female headed compared to 37% non-beneficiary households, and 26% beneficiary households had chronically ill compared to 21% non-beneficiary. It also stressed that 9% of beneficiaries had no social inclusion criteria, while 31% of non-beneficiaries had three or more social inclusion criteria. Furthermore, 54% of beneficiaries were asset poor, compared to 48% for non-beneficiaries.

 

Figure 1 Asset ownership categories by beneficiary status and gender of household head
Asset ownership (different assets*) Beneficiary household Non-beneficiary household % of total households
M F M F M F Total
Asset very poor (0-2 assets) 12.9% 24.5% 22.8% 27.6% 18.6% 25.9% 20.9%
Asset poor (3-5 assets) 38.7% 46.4% 37.4% 43.7% 37.7% 45.2% 40.0%
Asset medium (6-8 assets) 35.5% 25.5% 29.1% 18.4% 31.8% 22.3% 28.9%
Asset rich (9+ assets) 12.9% 3.6% 10.6% 10.3% 11.7% 6.6% 10.2%
Total 100% 100% 100% 100% 100% 100% 100%

*excludes livestock

Post Distribution Monitoring (PDM)

The main objectives of PDM are to monitor the use of food aid, satisfaction with food aid, and access to food aid (involving targeting inclusion and exclusion). It involves a household questionnaire and focus group discussions and is conducted by NGOs and WFP field monitors. The first PDM report was compiled in December 2003, containing information for September - December 2003 (EMOP 10290 began in July 2003). There had been considerable work in the development of the PDM prior to this, as well as evaluation of its implementation. A consultant on monitoring and evaluation was taken on at country office level, and experiences of PDMs from other countries with similar programmes were drawn upon.

The first PDM narrative report was based on a total of 1196 households, with a relatively equal distribution between beneficiary and non-beneficiary. Beneficiary households were those receiving food aid through the new EMOP under Food for Work (FFW), or Vulnerable Group Feeding (VGF), i.e. those infected or affected by HIV/AIDS, children, orphans, expectant and nursing women. Social vulnerability categories under this EMOP included female headed households (FHH), elderly headed, households with disabled member or orphans or the chronically ill (CI), or households with a high dependency ratio. Social indicators were combined with food security indicators, i.e. access to land, access to livestock, access to regular income or asset ownership.

The main findings of the first report were that targeting on the basis of social vulnerability was good but weak with respect to economic wealth indicators, i.e. overall access to livestock, income or land was very similar between beneficiaries and non-beneficiaries. This was believed to be due to a high level of inclusion errors. The report recommended refining inclusion criteria through systematic discussions with local communities and traditional chiefs, giving the opportunity to work out local checklists of social and economic indicators tailored to a given local context.

The January 2004 PDM report found a clear improvement of targeting with respect to social and economic welfare indicators following a beneficiary verification exercise in late 20031. The February 2004 PDM report stated that the VGF programme more frequently included households keeping orphans (73%), the chronically ill (31%) and FHH (24.4%) compared to FFW activities (43%, 19% and 43% respectively). It concluded that VGF targeting remains satisfactory with a predominant inclusion of beneficiary households with at least one social vulnerability criteria.

The March 2004 PDM introduced new analytical variables - overall inclusion and exclusion errors and targeting efficiency2. The executive summary of the report states that inclusion error is 16% while exclusion error is 30%, and calculates that the targeting efficiency is 78%3. The June PDM found an inclusion error of 18% and exclusion error of 54%. Thus the targeting efficiency was 64%4.

Informing initiatives

The PDM has had a significant impact on the targeting system in EMOP 10290 and the current Protracted Relief and Recovery Operation (PRRO). Early findings, after the first report, led to an evaluation of targeting (Nsama report in March/April 2004) and an evaluation of the HIV/AIDS programme, with a particular emphasis on targeting in April/May 2004 (Selaphera Consulting Ltd).

Findings from the PDM and CHS have also led to a number of revised monitoring initiatives, including where:

Conclusions

Although there was no rigorous monitoring of targeting during the initial EMOP (1200), partly due to the need to prioritise implementation of the general food distribution, the monitoring evolved rapidly into an extensive system which collected a variety of data needed to assess compliance with targeting. The system has compiled a large quantity of extremely useful data. The level of sophistication achieved with regard to monitoring targeting is unique and could serve as a model for other programmes.

The findings of the monitoring, in particular the weaknesses of targeting on the basis of economic criteria, were rapidly identified and reflected in JEFAP 111 guidelines.

Some of the claims regarding the success of social targeting may have been over-emphasised, i.e. the differences found between beneficiary and non-beneficiary groups were not statistically tested and almost certainly not of sufficient margin to make certain claims. Furthermore, there seems to have been an undue emphasis on inclusion criteria rather than exclusion criteria, which showed very high levels reflecting the large proportion of very poor and poor in Malawian communities. It can be argued that exclusion errors are more important than inclusion errors, because they are divisive at community level.

The monitoring has a number of methodological weaknesses:

Recommendations

Given the lack of success of economic targeting, there is clearly a need for more piloting and research to determine when and where such targeting is feasible and appropriate. For example, where a large proportion of the population are economically 'poor' or 'very poor', it may not be feasible. It may be that in such circumstances, social targeting is more acceptable at community level while economic based criteria will lead to conflict and disagreement. An implication of this may be that 'the first cut' of targeting should be social targeting, with economic targeting subsequently invoked once these households have been identified at community level. In order to assess efficiency of economic targeting, it would have been better to carry this out at the onset of the programme or following new registrations as, during the programme, a process of equalisation is likely to occur and non-beneficiaries are forced to disinvest. Where possible, statistical tests should be applied. It should have been possible to apply non-parametric statistical tests (e.g. pearson coefficients) to compare social inclusion /exclusion for beneficiary and non-beneficiary populations. Had this been done, the findings would have been more credible.

For further information, contact Jeremy Shoham, email: jshoham@easynet.co.uk

Show footnotes

1The percentage of beneficiary households with access to live stock decreased from 30% to 21%. However, there was no improvement with regard to access to land.

2The analysis considers eight major exclusion indicators proposed by the JEFAP monitoring working group. These relate to livestock, acres of food crop and cash crop land, income as medium business and income as formal wage. There are six inclusion socio-vulnerability indicators. These are single headed household, more than seven persons in a HH, elderly headed, female headed, keeping orphans and keeping chronically ill.

3In order to assess the targeting efficiency, a targeting index is derived for each household, which is composed of the difference between its number of inclusion and exclusion criteria. For the purpose of simplicity, each exclusion indicator is given a negative one (-1) and each inclusion indicator a positive one (+1), When the two are aggregated, a deserving household is one that has a positive difference, i.e. more inclusion indicators than exclusion indicators. A negative targeting index will mean that the household has more exclusion criteria than inclusion criteria and hence could be considered as food secure.

4It should be noted that there are different figures in the text of the report for inclusion and exclusion and efficiency (16% and 73% and 78% respectively). Joint Emergency Food Assistance Programme, a country wide consortium of 12 international and local NGOs in Malawi

More like this

FEX: WFP HIV/AIDS Programming in Malawi

By Jeremy Shoham, ENN Households headed by the elderly may be vulnerable This article was written based on a WFP consultation to Malawi in February 2005. The WFP Malawi...

FEX: Targeting Vulnerable Households Within the Context of HIV/AIDS in Malawi

Summary of evaluation1 By Maja Munk and Dr. Neil Fisher Maja Munk has been working in the food security sector for ACF/AAH in Africa and Asia since 2002. Her photographs have...

FEX: Monitoring the targeting system (Special Supplement 1)

In the past, little emphasis has been placed on monitoring what happens to food aid after it has reached the distribution point. Donor reporting has been limited to the...

FEX: Determining eligibility (Special Supplement 1)

Children are often a group targeted in emergencies Eligibility criteria, i.e. the characteristics of those individuals or households to be targeted with food, arise from the...

FEX: Community Based Targeting in Myanmar

By Jeremy Shoham In May/June 2005, Jeremy Shoham was part of a WFP Office of Evaluation (OEDE) team which fielded a mission to Myanmar as part of a five country case study...

FEX: Delivery of Social Protection Programmes in Kenya

By Clemensia Mwiti and Nupur Kukrety Clemensia Mwiti is a Humanitarian Support Professional (HSP) in Emergency Food Security and Livelihoods. Nupur Kukrety is the Social...

FEX: Emergency Food Security and Livelihoods Project in Amhara and Oromia regions

By Sarah Coll-Black and Matt Hobson Sarah Coll-Black is a Social Protection Specialist working with the World Bank in Ethiopia and Kenya. She has been involved with Ethiopia's...

FEX: Food insecurity and mental health among community health volunteers in Ethiopia

By Sarah Coll-Black and Matt Hobson Sarah Coll-Black is a Social Protection Specialist working with the World Bank in Ethiopia and Kenya. She has been involved with Ethiopia's...

FEX: Emergency food-based programming in urban settings

Summary of published research1 Children attending Stara School, Nairobi, that receives WFP food support. The Food and Nutrition Technical Assistance 11 (FANTA-2) Project has...

FEX: Food vouchers in Zimbabwe (Special Supplement 3)

By Ann Witteveen and Lewis Lawrence Musa, Oxfam Children returning from miller with maize bought at the Grain Marketing Board. The food voucher programme was designed to...

FEX: Distributing food (Special Supplement 1)

Food may be distributed in many different ways but the method of distribution will, to a large extent, depend on the eligible groups and the method for identifying them....

FEX: Introduction (Special Supplement 1)

Glossary ACF Action Contre la Faim BMI Body Mass Index CMT Community Managed Targeting CSB Corn Soya Blend DFID Department for International Development FFW Food...

FEX: Evaluation of WFP relief operations in Angola

Summary of evaluation1 The World Food Program mes's (WFP) assistance in Angola has alternated between relief and recovery since the mid-1970s. To date, nine emergency...

FEX: Review of targeting methods in HIV programmes

By Josh Colston Josh Colston studies demography and public health at the London School of Hygiene and Tropical Medicine. In the past he has worked as lecturer in British...

FEX: Food aid targeting in Ethiopia

Summary of published paper Food Distribution, Ethiopia '95 Based on data from a nationally representative survey of 4166 farm households by the Grain Market Research project...

FEX: Review of urban food security targeting methodology and emergency triggers

Summary of report1 People queuing at an Oxfam feeding programme in Mbare, Harare A recent report compiled by Oxfam GB, Concern Worldwide and Action Contre la Faim (ACF)...

FEX: Self-Targeting in Ambon, Indonesia “To be or not to be community minded”

By Carole Lambert Carole Lambert has been a Food Security Officer with Action Contre la Faim for the past four years, actively involved in programmes in the Ivory Coast,...

FEX: Letter on targeting food aid, by Anna Taylor

Dear Editor The targeting of food aid is widely assumed to be the most effective and efficient way of ensuring that the limited food aid resources available in emergencies...

FEX: Swaziland Cash and Food Transfer Programme

By Rosie Jackson Rosie Jackson currently works for Save the Children UK as an Emergency Food Security & Livelihoods Advisor. Based in London, she provides technical support to...

FEX: Milling vouchers in Dafur to optomise food aid

By Hanna Mattinen and Loreto Palmaera Hanna Mattinen has been Food Aid Advisor at the ACF headquarters since 2005, with a focus on food assistance and cash-based...

Close

Reference this page

Jeremy Shoham (2005). WFP Monitoring and Evaluation of HIV/AIDS Programming in Malawi. Field Exchange 25, May 2005. p42. www.ennonline.net/fex/25/wfpmonitoring