The Challenge of Applying CSAS in DRC
By Jason Stobbs, AAH-USA
Jason Stobbs is a long-serving member of Action Against Hunger - USA, volunteering over the past four years in diverse contexts such as Afghanistan, Cambodia, DRC and Chad. A Cambridge Master's graduate in Anthropology and specialist in food security and nutrition issues, he has specifically worked on the development of survey methods in situations of food scarcity.
The contributions of Action Against Hunger - USA and Mark Myatt to this article are gratefully acknowledged.
Some happy faces met along the way
Programme coverage has become an increasingly important indicator for measuring the success of humanitarian interventions. This is especially true in the context of nutritional feeding programmes, where lack of information regarding coverage may seriously dampen the impact of therapeutic and supplementary feeding centres. How coverage is measured is particularly relevant. Practical solutions to the methodological problems of estimating coverage using the 'traditional' two-stage cluster approach (currently considered standard by SPHERE) have been offered by the more recently developed centric systematic area sampling methodology (CSAS)1. Whilst applying this method in the field, Action Against Hunger-USA (AAH-USA) encountered a number of issues in the sampling and in on-the-ground assessment techniques. This article seeks not to expound on the coverage results of the survey, but rather to act as a case study of the concrete application of the methodology in the field. This article details both the problems and successes encountered when employing the CSAS methodology, and offers some suggestions for further fine-tuning.
Figure 1 Quadrat grid positioned on programme target zone
Figure 1 shows the positioning of the grid on the map of Uvira territory. Note that this 'computerised' image is only an approximate rendering of the physical map used during this stage of the survey. Minor cartographic variations between the two maps have been noted.
Notice the topography. A single axis of communication traverses the territory north south, via quadrats 2, 4, 8, 12, 17, 21, 25 and 29. The main town of Uvira is located in quadrat 25. Finally, it is important to know that Uvira is a 'shelved' territory, composed of highlands (5, 6, 9, 10, 14, 15, 19, 23, 26, 27) of middlelands (1, 7, 11, 16, 20, 24, 28) and of plains (2, 8, 12, 13, 17, 18, 21, 22, 25, 29).
You will also notice that where the intervention area corresponds to less than 50% of the area of a particular block, they have been coloured red and have been excluded from the survey. As a result, the territory has been divided into 29 blocks of 133 km_ each.
A mother with her malnourished child identified during the survey
In January 2005, AAH-USA conducted a nutritional coverage survey using CSAS in the Uvira territory located in eastern Democratic Republic of Congo (DRC) in order to assess the level of coverage of the ten nutritional centres located there. The CSAS was particularly chosen as the twostage cluster methodology had previously failed in this territory to give a representative analysis of the malnourished population, where survey results did not concur with the nutritional infrastructure reports.
As per a traditional nutritional coverage survey, the CSAS methodology targets children from 6 to 59 months of age and records weight, height, age, gender, mid-upper arm circumference (MUAC), presence/absence of bilateral oedema and enrolment/ nonenrolment in a nutritional programme. The method can be summarised as a series of steps as follows2:
- First, a correct and accurate map of the target zone for the programme is necessary (in this case, the entire Uvira territory).
- A grid is positioned on this map. This grid must be composed of squares of equal size (quadrats), with the number of quadrats varying according to the size and shape of the programme target zone. The objective is to divide the programme target zone into quadrats of equal size. Each quadrat must be spatially homogeneous, that is, the geographical space covered by the quadrat must be equally accessible to the population within. In addition, the grid is positioned inclusively, that is to say that geographical areas are included if more than 50% of their surface area is within the quadrat.
- When the grid is positioned, the geo graphic centre of each quadrant is determined and an exhaustive list of the villages present is compiled.
- Each village is then classified in order of priority: the 1st is the village closest to the centre of its quadrat, the 2nd is the closest to the 1st, and so on.
- The survey is conducted on all of the quadrats, and a team surveys each square for one entire day (from morning to evening).
- An active case finding method is employed in CSAS. Community health workers and/or local mothers' groups are the best sources of case information. When in the village or villages designated by the methodology, the team asks one or several representatives of the population to take them to all children aged between 6 and 59 months who are:
- malnourished and/or ill. Because there are close links between malnutrition and illness, it is often difficult to tell at a glance whether a child is only ill or whether s/he is also malnourished. In order to obtain an exhaustive sample, every case is measured, and children who are only ill are separated from those who are malnourished during the post-survey analysis.
- enrolled in a nutritional programme, even if they no longer show clinical signs of malnutrition. After the second month spent in a Supplemental Feeding Centre, a malnourished child no longer falls with in the admission criteria, but must still be included in the coverage survey.
An aid worker measuring MUAC during the survey
Anthropometric measurements of children identified as malnourished by the community are then taken. For each case measured, the team asks whether the child is participating in a nutritional programme. When a child claims to be a participant in a programme, the child's name and all relevant information are taken in order to crosscheck whether the child is actually enrolled. If a child who is allegedly malnourished is not at home, the team proceeds to the next house, but returns later. This procedure is continued until there are no more cases known by the community.
When an entire village has been visited, the team goes to the next village on the list, and the next, repeating this process until the end of the day. The number of children measured in a single day is considered sufficient to constitute a representative sample.
- When all of the quadrats have been surveyed, the information is compiled and analysed. The calculation of the formula for the coverage rate is the same as with a two-stage cluster method, but the results are much more representative:
This quadrat included both a Supplementary Feeding Centre (SFC) and Therapeutic Feeding Centre (TFC) located in the main town of Uvira. The quadrat measured 133 km_ and the positioning of its borders was determined in order to exclude, as much as possible, the neighbouring Lake Tanganyika. During the preparation of the survey, it was assumed that this area was spatially homogenous. Yet, the central village of this area, the village of Rugongo (located barely 4 km from Uvira) is located in the middle-lands and not on the plain. By employing the CSAS methodology and selecting villages according to their proximity to the centre of the quadrat (steps 3 and 4,), it followed that all the villages surveyed were also located in the middle-lands and not in the plain. The ensuing results were, if nothing else, surprising: none of the malnutrition cases in these villages were taken care of by SFC/TFC of Uvira, no matter how close by they were.
AAH-USA coverage survey
The AAH-USA nutrition survey took place between January 21st and January 28th 2005, and covered the entire territory of Uvira (including the highlands and middle-lands). Prior to this, two weeks were spent training the five survey teams, which included using the cartographical data to define the quadrats and assigning the itinerary to the teams. The teams slept in the field during the survey period and did not return to base camp at night.
Each team comprised three members (a team leader and two measurers). Each was assigned a number of quadrats, depending on the distance to be travelled and the ease of access. For example, a team assigned to the plains along the axis of communication was assigned five quadrats, while a team assigned to the most remote corners of the highlands was only assigned two. The team leaders were required to act autonomously and gauge, according to their own judgment and the information they obtained locally, the level of threat ahead (the Uvira territory is not a stable area) and the itinerary to employ (whilst still proceeding to villages according to the centric pattern and making way towards the quadrat to be surveyed the next day). Needless to say, such responsibilities required that the teams be thoroughly trained, and relied upon a high degree of initiative and responsibility from the team leaders.
'Spatial homogeneity' assumption
The sampling methodology in a systematic centric area is frequently utilised in ecological studies to determine spatial distribution of animal or vegetable species over a vast surface, as well as in human geography to evaluate the radius of distribution of certain points of sale. Its main advantages are its ease of utilisation, its capacity to derive a representative cross-section for a vast area, the simplicity of the analysis of the data, and the spatial dimension of the analysis which gives it an added value. However, it is built on the hypothesis that each quadrat is spatially homogenous. During our field survey, this assumption clashed with the reality of the area's topography. Uvira is a 'shelved' territory, varying from highlands to middle-lands to plains, as one moves from west to east. Despite experimenting with various quadrat sizes and grid positionings during step 2, no fully satisfactory positioning was found in order to ensure spatial homogeneity. This in turn led to troubling results in the survey, as illustrated by the case of quadrat n°25 (see box). The survey information revealed a striking difference between the middle-lands and the plains in terms of population mobility. However, in following the CSAS methodology, the level of coverage for quadrat n°25 was calculated to be 0%, a clearly misleading figure given the number of beneficiaries in the centres coming from other villages located in the plains.
Although this problem can clearly be attributed to a lack of spatial homogeneity within the quadrat, a viable solution is hard to come by. Indeed, with the lake Tanganyika and the middle- lands barely 4km apart from one another, and the town of Uvira nestled in between, it is difficult to imagine how to reposition the grid in a more suitable way. This difficulty in ensuring spatial homogeneity within the quadrats is symptomatic of the entire area of Uvira, where the presence of a sole axis of communication and of numerous hills and rivers poses certain difficulties in the positioning of the grid. It might be possible to use smaller quadrats, which would have the added benefit of allowing post-survey stratification, However, given that each quadrat takes a day to survey, the ensuing increase in personnel and/or time needed for completing the survey could make the cost prohibitive. Alternatively, the original quadrat size could be kept but a random sample of villages within each quadrat could be used. Another possiblity would be to locate quadrats purposively so as to ensure that plains and mountains were included.
Although this is a case of the method 'failing', it is not a gross failure as it provided useful information. One of the aims of a coverage survey is to identify where coverage is likely to be weak. In this survey, quadrat n°25 gave us good insight into a coverage problem linked to the cultural resistance of the hill folk to bring their children to a centre in the plains, regardless of proximity. This demonstrates how, as in all surveys, this method must be accompanied by a contextual interpretation.
Sampling sizes and active case research
The active case-finding method has the advantage of providing a nearly-exhaustive cross-section of malnutrition cases in the villages surveyed. In addition, it allows for coverage of more villages than in a classic survey. For example, in the AAH-USA survey, 112 villages were visited instead of an average of 30 covered in a classic survey. This guarantees greater accuracy and greater representativeness in the sampling.
In areas of low-density population, where the villages become more distant from one another, the size of the cross-section is very small (e,g. quadrat n°15, where the villages are so far apart from one another that only two villages could be surveyed). Although this is essentially not a problem, insofar as the results reflect the geographic and nutritional realities of the area, a small sampling renders comparison between the areas difficult, if not impossible. For instance, how does one compare the quadrat n°15 with quadrat n°14, where 11 villages were surveyed and 3 cases of malnutrition were confirmed? The exercise is simply impossible, due to the discrepancy in sample size. To diminish the size and to increase the number of quadrats seems, once again, to be the best way to refine the results and avoid this pitfall.
The quality of the cross-section is built entirely on the capacity of health networks or other community representatives to detect malnutrition cases. The voluntary exclusion of malnutrition cases, whether for reasons of discrimination or for taboos surrounding malnutrition, is therefore a reality that must be taken into account during the sampling. This being the case, it is regrettable that no pre-testing was undertaken during this survey. Such a pre-testing would have ensured that the definition of the terms used to define malnutrition cases in the communities was homogeneous and allowed us to compare the number of cases registered with the help of the community with the number of cases registered at the time of an exhaustive screening of the community. However, it appears that the casefinding employed was reasonably sensitive since, prior to their measurement and confirmation of their anthropometric status, many children were put forward who turned out to be 'false-positives' (i.e. children identified as malnourished by the population but not falling within malnutrition criteria). Indeed the final sample size was much smaller than anticipated - out of 858 children measured, 511 were excluded from the final analysis as false positives.
In retrospect, there are a number of pros and cons to using the CSAS methodology in this context. On the positive side, the input and handling of data is considerably more rapid than in a 'classical' two-stage cluster survey. Furthermore, since the cross-section is quite a lot bigger than would have been in a traditional survey, 262 malnutrition cases were detected - considerably more than the 30 cases detected in a previous cluster survey carried out in June 2004. The spatial dimension of the analysis also provided useful information for the orientation of AAH-USA's programmes. For instance, quadrat n°11 turned out to have over 20 cases of malnutrition without any coverage support. This led to the opening of a nutritional infrastructure and the improvement of screening and consciousness-raising activities. Thus, in this particular instance, the coverage survey doubled as a case-finding and referral activity for the programme.
Measuring length during the survey
However, the cost, time and work intensity is considerably greater than that of a classical two-stage cluster method. In the traditional two-stage cluster methodology, a cluster is finished once 30 children have been sampled, whereas in the case of the CSAS methodology, a quadrat takes an entire day to complete, with as many children measured as possible. This demands great effort and planning, especially if villages within a given quadrat are not close to each other. Furthermore, the need to keep quadrats spatially homogeneous entails, in this case, a reduction in size and an increase in the number of quadrats, which in turn increases the cost and length of the survey. Similarly, the disparity in quadrat sample sizes makes for difficult comparisons, unless the size of the quadrats is reduced. Finally, the reliance on village health workers or key informants, although a clear positive in terms of programme integration and sustainability, does in turn create a need for pre-testing in every village, increasing, once again, the time and cost of the activity.
In conclusion, this method has a clear usefulness in the framework of an evaluation of nutritional coverage. However, it requires a high degree of preparation and a thorough knowledge of the area.
1Developed by Mark Myatt, researcher at the Institute of Ophthalmology of London and a consultant for the WHO.
2For a description of the CSAS method, see Field Exchange issue 21, New Method for Estimating Programme Coverage, p11 and 'A Field Trial of a Survey Method for Estimating the Coverage of Selective Feeding Programmes', Mark Mayatt, Teshome Feleke, Kate Sadlet, Steve Collins, Bulletin of the World Health Organization, January 2005, n 83 (1).
More like this
Dear all, We are currently conducting a SQUEAC investigation in two townships of the Northern Rakhine State in Myanmar. We have been doing well so far but we are now facing a...
As we all know, coverage is an internationally recognized indicator of performance for selective feeding program (SFP) and CSAS is the current gold standard for assessing...
en-net: resources on CSAS
I'm trying to learn more about CSAS, but can only find summary or simple documents. Does anyone know where there is a proper, detailed manual on CSAS? That does help. Thanks...
By Mark Myatt Mark Myatt is a consultant epidemiologist and senior fellow at University College London. His areas of expertise include infectious diseases, nutrition, and...
By Simon Kiarie Karanja Simon Karanja is currently the regional nutrition advisor with GOAL in East Africa. Previously he worked as the CTC Coordinator for GOAL Ethiopia and...
By Mark Myatt Mark Myatt is a consultant epidemiologist and senior research fellow at the Division of Ophthalmology, Institute of Ophthalmology, University College London....
By Ernest Guevarra, Saul Guerrero, and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...
By Mark Myatt Mark Myatt is a consultant epidemiologist and senior research fellow at the Division of Epidemiology, Institute of Opthalmology, University College London. His...
By Mark Myatt This article gives an overview of the coverage estimation method developed for the Community Therapeutic Care (CTC) Research Programme in Malawi1. Coverage is...
en-net: Calibration SAM prevalence
Hi, We are currently implementing a coverage survey using SQUEAC methodology in Sierra Leone, Moyamba District. According to the methodology, the calculation of the minimum...
In looking at materials on CSAS, there are several references to S3M but I can't find anything that actually explains it. Does anyone have any resources for that? Hi Mark, I...
By Ben Allen, Mark Myatt, Nikki Williamson, Danka Pantchova and Hassan Ali Ahmed Ben Allen has been Global Coverage Advisor for Action Against Hunger UK for the past two...
By David Rizzi David Rizzi graduated in Pharmacy and later took an MSc in Development at the Rome University La Sapienza, Italy. He holds a second MSc in Public Health...
Resource: A comparison of the programme coverage of the two therapeutic feeding interventions implemented in neighbouring districts of malawi
Published in Public Health Nutrition (2006) 10(9), 907-913 Abstract Objective: To compare therapeutic feeding programme coverage for severely malnourished children achieved...
By David Doledec, ACF-USA A qualified nurse, David Doledec joined Action Contre la Faim in 2002, working in Democratic Republic of Congo, Sudan, Liberia and Uganda before...
en-net: CSAS survey for TSFP only
Dear Sir, We are planing to determine the coverage of a TSFP program that has been runing quite a while. Hence, according to the CSAS sampling, we calculated that we have to...
Hi, We'd like to do a coverage survey in Darfur. We have 8 OTP sites with a total of 6,300 children under 5 in mainly rural areas and we hope to have at least 50% coverage....
Dear all, I hope you are doing fine. Well, I still have an issue and need a bit of guidance. We are in the very final stage of the SQUEAC investigation. I have just compiled...
By Anne-Marie Mayer, Mark Myatt, Myriam Ait Aissa and Nuria Salse Anne-Marie Mayer is a technical consultant for this project and carried out the first field test in Mali with...
Is it advisable to combine the number of children found during small area surveys to the number of children found during wide area surveys? In a recent coverage survey, in...
Reference this page
Jason Stobbs (2006). The Challenge of Applying CSAS in DRC. Field Exchange 27, March 2006. p28. www.ennonline.net/fex/27/drc