Monitoring the targeting system (Special Supplement 1)
In the past, little emphasis has been placed on monitoring what happens to food aid after it has reached the distribution point. Donor reporting has been limited to the delivery of food to its intended destination (Jaspars and Young, 1995). For this reason, "good systems of monitoring and evaluation, to establish whether food aid is indeed reaching intended beneficiaries (and at reasonable cost of delivery), are disturbingly rare." (Barrett, 2002)
As noted in Section 2, targeting is never 100% accurate and there is scope for errors of inclusion and exclusion to arise at every stage of the targeting process. Table 2 highlights some of the key factors that lead to inclusion and exclusion errors at each stage. In the design stages of the targeting system, decisions are made as to which type and degree of error is more acceptable. For example, the aim may be to reach HIV/AIDS affected households by targeting households with a chronically ill household head. If this is done, it must be recognised that this will encompass households with non-HIV/AIDS chronic illness and that there will, therefore, be an unavoidable (although perhaps desirable) inclusion error. The broader and more inclusive eligibility criteria become, the smaller the exclusion error and the greater the inclusion error, and vice versa. Hence, as inclusion errors rise, exclusion errors decline and as exclusion errors rise, inclusion errors decrease.
In the implementation of targeting, i.e. once the quantity of food has been fixed and the size of the targeted population is defined, the relationship between exclusion and inclusion errors changes. That is to say, as inclusion errors increase so do exclusion errors. For example, food distributors could decide to share the food among a greater number of households, including those that are very poor but not affected by chronic illness. This may mean that the numbers of originally- targeted households who receive the food is reduced, or the quantity each household receives is reduced, resulting in a higher exclusion error.
Monitoring exclusion and inclusion assumes that the target group defined in the objectives was correctly identified as the group in greatest need. In addition to inclusion and exclusion errors, a well designed monitoring and evaluation system should determine whether a) the decision to target food within a geographical area was appropriate, b) whether the groups in greatest need were identified by the assessment, and c) whether the objectives were achievable. In practice, inclusion errors may occur because the community has a clearer understanding of need and deliberately subverts the eligibility criteria. In this instance, simply monitoring inclusion and exclusion will show failures in the system whereas in fact, because of the errors, the targeting system may be more effective in meeting people's needs. Thus, it is very important that monitoring allows review of the design of the system itself, as well as checking its implementation.
|Table 2: Overview of monitoring targeting systems|
|Assessing and defining needs||Setting objectives||Determining eligibility||Distribution|
|Key questions for monitoring||Were the needs accurately described?||Were food aid target groups those in greatest need?||Did criteria adequately identify the target group? Were the criteria clear and usable by those responsible for using them?||Did the targeted group receive the right quantity and quality of food at the right time?||Did the targeted group use the food to achieve the objective set?|
|Possible methods||1. Food security monitoring
2. Monitoring the prevalence of malnutrition
|1. Process monitoring and evaluation
2. Food basket monitoring
3. Household profile monitoring
4. Post distribution monitoring: non-beneficiary monitoring
5. Results from complaints mechanism
|1. Food basket monitoring
2. Post distribution monitoring: non-beneficiary monitoring
3. Coverage surveys
4. Process monitoring and evaluation
|1. Food Usage monitoring
2. Market surveys
|Key factors which lead to exclusion and inclusion errors||
. Type of assessment
|. Political context
. Transparent and accountable systems for identifying the eligible
|. Geographical coverage
. Information about programme in the community
. Could it be transported
. Diversion by powerful people
|. Sharing within households
. Redistribution by the community
. Diversion by powerful people
Table 3 summarises the methods that can be used for monitoring the targeting system, and Table 2 indicates at which stages the various methods can be used. These methods are described briefly. It should be noted that there is no single method for determining inclusion and exclusion errors or the appropriateness of the targeting system. Each of the methods listed here provides part of the information necessary to comprehensively monitor a system.
|Table 3: Different types of monitoring which can be used in targeting systems|
|Type of monitoring||Where and when it is done||Purpose|
|Process monitoring (including appeal mechanisms)||Ongoing with beneficiaries, nonbeneficiaries, leaders and authorities||To assess the quality of the implementation and how it is perceived by the population|
|Food basket monitoring||At the distribution point through interviews with beneficiaries||Determines whether the ration received at the distribution point matches the entitlement on ration card|
|Household profile monitoring||
At the distribution point through interviews with beneficiaries
Monitoring of beneficiary household profile relative to eligibility criteria
|Food usage surveys||Post distribution through home interviews with beneficiaries||Determines how recipient households use the food and how long it could last|
|Market surveys||At markets post distribution||To monitor sales and prices of food aid|
|Non-beneficiary monitoring||Post distribution through home interviews with non-beneficiaries||Monitoring perceptions among non-beneficiaries regarding fairness of the targeting and the distribution process|
|Coverage surveys||During a targeted feeding programme, through population surveys||To determine the proportion of the eligible population who are registered for feeding and the proportion who are not|
|Food security monitoring||On an ongoing basis among the whole population||To determine whether the targeting objectives are appropriate/have been achieved|
|Nutrition monitoring||Periodically among the whole population||To determine whether the targeting objectives are appropriate/have been achieved|
The information provided by monitoring should be considered immediately and the implications for the targeting system determined.
Process monitoring and evaluation
Systems should be established for ongoing monitoring of the views of the population on the targeting system. Ongoing methods for participatory monitoring is an integral part of community managed targeting. In systems using administrative targeting, appeal mechanisms can be set up to allow members of the affected population to complain if they have not been targeted or have not received food. These mechanisms can provide an immediate way of monitoring the targeting process. Where appropriate, complaints should be met with an instant response and if necessary, modification of the system.
Food basket monitoring
Food basket monitoring is conducted immediately after receipt of the ration at the distribution point. It monitors whether the ration that is supposed to be being received by a registered beneficiary is indeed received and thereby, provides a measure of exclusion error. As shown in Table 4 (see p25) planned rations often differ from those received because the number of recipients is greater than planned, and so the food is shared 'more thinly' amongst them. Table 4 shows that in Tanzania, the actual ration size was 10.3kg per capita per month, compared to a planned ration of 12 kg per capita per month. This arose because 15.4 per cent of households were underregistered, where not all household members were registered in order to allow more households to be included in the distribution.
Thus, food basket monitoring data is useful if analysed in conjunction with household profile monitoring data and with the targeting objectives. On its own, food basket monitoring does not tell you whether the beneficiary has been correctly targeted (i.e. whether they should have a ration card) nor indeed, whether they may be receiving two rations because they have two ration cards. Neither does it highlight individuals who are incorrectly excluded from the distribution. Food basket monitoring has been used predominantly in refugee situations where ration cards are widely used. (Jaspars and Young, 1995)
Household profile monitoring
Household profile monitoring is very important for monitoring systems which target households, and is a primary measure of the accuracy of the registration process. It also indicates the level of inclusion and exclusion error, assuming the size of the targeted population is known. Household profile monitoring involves determining whether the recipient households meet the eligibility criteria. Box 17 provides some results of household profile monitoring in Zimbabwe.
Box 17: Household profile monitoring results from community based targeting in Zimbabwe 2001
Save the Children UK implemented a targeted community based food distribution in Binga district in Zimbabwe. Eighteen out of the 21 wards were selected to receive food. The wealth breakdown in the community at the time of the assessment was:
Poor - 50-60%
Middle - 25-35%
Better Off - 10-15%
The poorest 50% were targeted to receive a 75% ration in two one-month distributions. Findings of the Household Profile Monitoring, covering 325 households, indicated that the beneficiaries of the food aid programme were in the following wealth groups:
'Middle-Better Off' 2.2%
'Better Off' 1.0%
This can be interpreted as follows:
- 87.3% of beneficiaries are in need of the food aid. However, it is not clear whether 25.8% of them - the "poor-middle" group - are among the most needy.
- 12.7% of beneficiaries do not need the food aid, and should not have been registered. Conversely, therefore, at least 12.7% of the "poor" group should have received food but did not.
Post distribution monitoring
Three types of post distribution monitoring can be used for monitoring targeting:
Project village of a Concern targeted food distribution programme in Afghanistan
Food usage surveys can contribute to an understanding of inclusion errors by highlighting the extent to which food is redistributed by the recipients, either voluntarily or involuntarily post distribution. It can also help to determine whether the food aid is likely to achieve its desired objectives, e.g. if all the oil in the ration is sold in order to purchase more grain, then any nutritional objective may be harder to achieve.
Table 4 shows the data from a food usage survey in Tanzania conducted during community managed food aid targeting. The questionnaire used for the survey was pre-tested and modified when it became clear that the majority of redistribution or sharing of the ration was with unregistered community members, and occurred through the provision of cooked meals rather than dry food. A small proportion of the ration fulfilled social obligations such as kinship support, debt repayment and the cost of ceremonies. The survey showed that the food would last the average household approximately 2 months, rather than the intended 3 months (Save the Children, 1999). The survey was conducted in a sample of households covering a large geographic area, using a similar sampling frame to a 30x30 cluster anthropometric survey.
Market surveys are often conducted to monitor food aid sales. Sale of food aid should not be interpreted as a targeting failure. All households have needs for cash to meet non-food needs and may have to sell some food to obtain cash to buy other essential commodities, such as soap, clothing, fuel etc. Data from market surveys can be analysed alongside food usage surveys, to further contribute to an understanding of whether the targeting system has achieved its objectives.
Non-beneficiary monitoring is important for understanding errors of exclusion. While the scale of exclusion error can be deduced from household profile monitoring, non-beneficiary monitoring allows determination of whether a particular subgroup of the targeted population has been excluded, and the possible reasons for their exclusion. It also may help identify when the needs assessment and eligibility criteria have excluded a needy group (thereby making the population which should be targeted bigger than planned). Box 18 shows examples of some of the reasons why people are excluded from food for work.
Box 18: Typical reasons for the exclusion of needy groups from food for work
- Women may not be able to participate due to the type of work, cultural constraints or childcare responsibilities.
- Illness or disability
- Not being able to wait for payment
- Having share cropping responsibilities
- Participation may be seen as being too risky
- Severity of work norms (i.e. not being able to do other activities later or earlier in the day, degree of flexibility over the length of time to complete a task)
- Distance from the programme
Box 19: Calculation of coverage
Coverage of supplementary and therapeutic feeding is usually calculated using an anthropometric survey. The following formula is used:
Number of individuals in the survey who meet the admission criteria for the programme and report being registered in the programme at the time of the survey, dividied by the total number of individuals in the survey who meet the admission criteria for the programme, multiplied by100
Confidence intervals around these estimates should also be presented.
Box 20: The impact of targeted food aid in Ethiopia in 1999
In north east Amhara region in 1999, the impact of targeted food aid was assessed. The table shows the contribution which food aid made to requirements in Dega areas, North Wollo, and the outstanding deficit experienced by households in the same area. The table goes some way to demonstrate the overall shortfall in food aid available to meet the size of the deficit, and the extent to which exclusion errors occurred particularly for poor people.
|Wealth group||% of total households in 1999||% food aid needs provided by food aid in 1999||Remaining deficit in 1999|
This measure is routinely made for supplementary and therapeutic feeding programmes. Assessment of coverage, using the data required in Box 19, is usually done at the same time as an anthropometric survey. More recently, as greater attention has been paid to increasing the coverage of targeted feeding programmes (see section 4), new methods are being devised (Myatt, 2004, see Figure 6 for some results of this method). The main limitations of the method given in Box 19, are that it does not provide any measure of geographical coverage within the implementation areas (with centre based programmes one would not expect uniform coverage). In addition, the degree of reliability of this method is quite low due to sampling limitations.
Sorting distributed rice in Ethiopia
Impact monitoring and evaluation
Monitoring the impact of food aid targeting allows conclusions to be drawn as to whether the objective has been achieved (for an example, see Box 20). Impact monitoring could include periodic anthropometric and mortality surveys, if the purpose of the targeting was to prevent increases in malnutrition and mortality. It should be remembered, however, that factors other than food aid contribute to reductions in levels of malnutrition, including changes in the public health environment. Similarly, assessment of food entitlement can be conducted pre and post food aid targeting to determine whether, for example, the sale of assets or seed was prevented and to determine whether the food deficit, intended to be filled by food aid, was adequately filled.
Food basket monitoring can be carried out at distribution points
The paucity of published data on targeting systems means that it is not possible to provide a comprehensive indication of the level of error that can be expected in different types of targeting systems. Table 4 provides some evidence of data from community managed targeting and illustrates the accuracy of the registration, the extent to which those registered receive the food they are entitled to, and the ways in which the recipients use the food aid. It can be seen that an error introduced in the registration, for example, has a knock-on effect and results in errors in the distribution, e.g. people get less food because too many are registered. Sphere standards provide an indication of acceptable levels of coverage for supplementary feeding programmes, stating that coverage should be more than 50% in rural areas, 70% in urban areas and 90% in a camp situation.
|Table 4: Example of monitoring data from Save the Children UK community managed targeting systems (Mathys, 2003)|
|Tanzania Singida round 1 (Oct, 1998)||Zimbabwe Kariba (May June, 2003)||Malawi Salima round 1 (March-May, 2002)|
|Planned target population
(% of total population)
|Determining Eligibility||Actual targeted population||153,629||5,843-5,996||36,573|
Percent of registered households correctly registered
|Percent of registered households under-registered||15.4||NA||NA|
Percent of registered households over-registered
|Distributing food||Planned ration size||12kg/cap/month||10kg/cap/month**||50kg/household|
|Ration size announced by Village Committee||10.97kg/cap/month||-||50kg/household|
|Actual ration size||10.3 kg/cap/month||7.1kg/cap/month||50kg/household|
Food usage* % Consumed within the households
68 (consumption and other uses - mostly shared)
*what happened to 3 months supply after 30 days
** maize only
Conclusions for best practice
- Monitoring the targeting system is an integral part of the system itself, not an optional extra. Monitoring provides the mechanism through which errors can be reduced and the system redesigned.
- There is no single method for monitoring targeting, since error can occur at many stages. Multiple methods should be employed to compile comprehensive picture of the effectiveness of the system.
- Monitoring should focus on assessing inclusion and exclusion errors, as well as evaluating the appropriateness of the targeting objectives.
Jaspars and Young, 1995
Save the Children, 2004
More like this
Food may be distributed in many different ways but the method of distribution will, to a large extent, depend on the eligible groups and the method for identifying them....
Glossary ACF Action Contre la Faim BMI Body Mass Index CMT Community Managed Targeting CSB Corn Soya Blend DFID Department for International Development FFW Food...
By Jeremy Shoham, ENN Post distribution monitoring GOAL food distribution Chiradzulu 2 This article was written based on a WFP consultation to Malawi in February...
Children are often a group targeted in emergencies Eligibility criteria, i.e. the characteristics of those individuals or households to be targeted with food, arise from the...
The operational contexts for 'within geographical area targeting' are limited This supplement has focused on targeting food aid in emergencies within geographical areas....
Summary of evaluation1 By Maja Munk and Dr. Neil Fisher Maja Munk has been working in the food security sector for ACF/AAH in Africa and Asia since 2002. Her photographs have...
Agencies deciding to implement food aid targeting in emergencies often face constraints on targeting design. Three such factors are dealt with here: Quantity and quality of...
Dear Editor The targeting of food aid is widely assumed to be the most effective and efficient way of ensuring that the limited food aid resources available in emergencies...
By Jeremy Shoham, ENN Households headed by the elderly may be vulnerable This article was written based on a WFP consultation to Malawi in February 2005. The WFP Malawi...
By Jutta Neitzel TSF distribution in Degehabur, Somali Region Jutta Neitzel is the Head of the Nutrition and Education Section at WFP Ethiopia. She has worked for the...
By Hannah Reed Hannah Reed is the former Assistant Country Director for Programmes in GOAL's Syria Programme. She has over 10 years of experience in the humanitarian and...
Summary of research1 Women collect bags between air drops near the town of Habilla, West Dafur A recent study, carried out by the Feinstein International Centre and...
By Clemensia Mwiti and Nupur Kukrety Clemensia Mwiti is a Humanitarian Support Professional (HSP) in Emergency Food Security and Livelihoods. Nupur Kukrety is the Social...
Post-script by Jeremy Shoham, ENN DFID commissioned an evaluation of the community-managed targeting programme in Singida and Dodoma regions of Central Tanzania in May 1999....
By Carole Lambert Carole Lambert has been a Food Security Officer with Action Contre la Faim for the past four years, actively involved in programmes in the Ivory Coast,...
By Elena Qleibo, Ena’am Abu Nada, Wassem Mushtaha and Julie Campbell Elena Qleibo is the Oxfam Food Security Coordinator for the Gaza programme. She has been working in...
Indian flood victims eat at open air kitchen in Nagari. The objectives of a targeting system arise from the definition of need (Section 1). Given a definition of need, the...
By Lani Trenouth, Jude Powel and Silke Pietzsch Lani Trenouth and Jude Powel were the ACF Food Security and Livelihood programme managers who implemented the programme in...
By Lola Goselow This article is based on a field trip made by Lola Gostelow (SCF HQ emergency advisor) to the SCF programme in Huambo province, Angola in November 1999. The...
By Jeremy Shoham In May/June 2005, Jeremy Shoham was part of a WFP Office of Evaluation (OEDE) team which fielded a mission to Myanmar as part of a five country case study...
Reference this page
Anna Taylor and John Seaman (2004). Monitoring the targeting system (Special Supplement 1). Supplement 1: Targeting food aid in emergencies, July 2004. p22. www.ennonline.net/fex/101/chapter5