Menu ENN Search

A single coverage estimator for use in SQUEAC, SLEAC, and other CMAM coverage assessments

Informal group discussion in a SQUEAC assessmentBy Safari Balegamire (VALID International), Katja Siling (World Food Programme), Jose Luis Alvarez Moran (Coverage Monitoring Network), Ernest Guevarra (VALID International), Sophie Woodhead (Coverage Monitoring Network), Alison Norris (VALID International), Lionella Fieschi (VALID International), Paul Binns (VALID International), and Mark Myatt (Brixton Health)

Location: Global

What we know: There are two different estimators of coverage (point coverage and period coverage) using current coverage assessment methods. Selection of the appropriate estimator can be difficult in multi-district surveys and where contextual information is not readily available. Coverage assessed using different estimators is not comparable.

What this article adds:  A new single coverage estimator is proposed that is a measure of overall programme performance. It can be used with existing survey tools and calculated using previous coverage data. Uncertainty remains regarding the mean length of an untreated SAM episode (included in the calculation); work is ongoing in this regard. The authors recommend the proposed single coverage estimator should be used in preference to using either the point or period coverage estimators.

Introduction

This article has been prompted by the number of support requests received from SQUEAC and SLEAC practitioners regarding the two different estimators of coverage (i.e. point coverage and period coverage, see definitions below) in current use. Both of these estimators can be calculated using data from any of the currently used coverage assessment methods (i.e. CSAS, SQUEAC, SLEAC, S3M) but often produce very different results. The SQUEAC and SLEAC technical reference encourages investigators to select and report only the most appropriate estimator, with the choice of the most appropriate estimator being informed by the context:

Selection of the most appropriate coverage estimator for a programme has proved difficult in multi-district (e.g. national) SLEAC surveys. It is unlikely that one estimator is appropriate for all sub-programmes, and collecting the contextual information needed to select the estimator appropriate to each sub-programme may not be feasible. Even when contextual information is collected, the use of different estimators makes it difficult to make between sub-programme comparisons. Reporting both point and period coverage estimates is a potential solution but this tends to confuse policy makers and has, on occasion, generated resistance to survey results.

This article presents a new single coverage estimator that does not require new survey tools to be developed and tested and can be used with current survey tools and data from previous coverage surveys so as to ensure continuity and comparability.

Coverage estimators

The two coverage estimators in current use are calculated as follows:

    

and:

    

where:

Cin    =    Current SAM cases in the programme

Cout    =    Current SAM cases not in the programme

Rin    =    Recovering SAM cases in the programme

Achieving high levels of coverage requires:

These and other factors often interact with each other in complex ways (see Figure 1).

The two coverage estimators in current use focus on case-finding, recruitment, and retention:

An estimator of coverage that does include both recovering cases that are in the programme and recovering cases that are not in the programme and, thus, provides an unbiased estimator of overall programme performance is:

    

 

 

where:

    Rout    =    Recovering SAM cases not in the programme

The problem with this estimator is that Rout (i.e. the number of recovering cases that are not in the programme) is unknown and may be difficult to collect accurately. A survey might (e.g.) find a moderate acute malnutrition (MAM) case that was not in the programme but it would not be easy to ascertain, except in the case of a defaulter, whether this case had previously been a SAM case and was now a recovering SAM case. Also, we know that the active and adaptive case-finding procedure used in many CMAM coverage surveys is optimally biased to find SAM cases not MAM cases. It could be argued that the MAM cases that we do find using active and adaptive case-finding are likely to be recovering SAM cases. This assumption would not, however, be tenable with house-to-house and door-to-door screening which is often used when case-finding in urban and camp settings. The situation may, however, be complicated by the presence of nutrition programmes distributing lipid nutrient supplements (LNS) such as NutriButter®, Plumpy'DozTM, or Plumpy'SupTM. Despite these complications it may be possible to create a simple instrument to divide MAM cases into “MAM never SAM” and “MAM previously SAM”. Such an instrument need only be accurate “on-average”.

It may also be possible to address the problem of estimating the number of recovering cases not in the programme (Rout) using a simple mathematical model. This article explores the latter approach. The advantage of this approach is that the single coverage estimator could be applied to data from previous coverage surveys allowing comparability of coverage estimates over time and would require no changes to be made to existing methods and tools.

A simple mathematical model

If we assume that incidence and coverage do not change very rapidly over time (i.e. little or no change over a period of less than about two months) then we can also assume that: 

    

 

 

where k is a correction factor (more on this below).

We can express Rout (the unknown variable) in terms of the known variables:

    

 

    

 

    

 

    

 

 

To avoid having a division by zero in the case of a survey finding no current SAM cases in the programme being assessed we would use:

    

 

 

Fractions of cases can be removed by rounding to the nearest whole number using simple truncation (i.e. rounding towards zero):

 

    

 

This leaves us with the problem of deciding a suitable value for the correction factor (k). A reasonable candidate for k is the ratio of the mean length of an untreated episode to the mean length of a CMAM treatment episode. Suitable values might be:

Parameter

Value

Notes

Mean length of an untreated episode

7.5 months**

This value is commonly used when estimating future programme case-loads from prevalence estimates*.

Mean length of a successful treatment episode

2.5 months** (i.e. about 10 weeks)

This value can be estimated from routine programming monitoring data (already collected and calculated in SQUEAC assessments). A value of 2.5 months could be used in the absence of better information or when the validity of routine programme monitoring data is suspect, such as when defaulters are recorded as being cured or when cases are retained in a programme after meeting the criteria for being discharged as cured.

* The value of 7.5 months for the mean untreated episode length is from: Garenne M, Willie D, Maire B, Fontaine O, Eeckels R, Briend A, Van den Broeck J, Incidence and duration of severe wasting in two African populations, Public Health Nutrition, 2009;12(11);1974-1982. Available from http://www.ncbi.nlm.nih.gov/pubmed/19254427

** These values are expressed using the same summary statistic (i.e. the mean) and the same unit of measurement.

Using these values gives:

 

 

and:

 

 

Working with survey data using the computerApplying the model

Applying the model requires two steps:

  1. Calculate Rout using a reasonable estimate for k with Cin, Cout, and Rin from survey data.
  2. Estimate coverage using the single coverage estimator.

To illustrate this process we will apply our model to some familiar example data (see Table 1).

Table 1: Apply the single coverage estimator to example data

 

Source

Survey data

Point coverage

Period coverage

Preferred estimator*

Single coverage estimate

1

FANTA (p105)**

Cin = 0

Cout = 2

Rin = 34

0%

94.4%

Period

2

FANTA (p105)**

Cin = 3

Cout = 9

Rin = 22

25%

73.5%

Point

* The estimator indicated as being the most appropriate by analysis of SQUEAC data.

** The data used in these examples is presented and discussed on page 105 of: Myatt M, Guevarra E, Fieschi L, Norris A, Guerrero S, Schofield L, Jones D, Emru E, Sadler K, Semi-Quantitative Evaluation of Access and Coverage (SQUEAC) / Simplified Lot Quality Assurance Evaluation of Access and Coverage (SLEAC) Technical Reference, Food and Nutritional Technical Assistance III Project (FANTA-III), FHI 360 / FANTA, Washington, DC, October 2012. Available from: http://www.fantaproject.org/monitoring-and-evaluation/squeac-sleac

The inclusion of recovering cases means that the single coverage estimate is mathematically constrained to return a coverage estimate that is greater than or equal to the point coverage estimate. In both of the examples shown in Table 1, the single coverage estimate is higher than the point coverage estimate. The underestimation present in the point coverage estimate has, to some extent, been corrected.

The inclusion of recovering cases that are not in the programme means that the single coverage estimator is mathematically constrained to return a coverage estimate that is less than or equal to the period coverage estimate. In both of the examples shown in Table 1, the single coverage estimate is lower than the period coverage estimate. The overestimation present in the period coverage estimate has, to some extent, been corrected.

We would use the process illustrated in Table 1 with data from simple coverage estimation surveys using (e.g.) the CSAS or S3M survey designs.

Using the single coverage estimator with data from SQUEAC hypothesis testing surveys or SLEAC surveys requires applying the standard LQAS decision rules with d or d1 and d2 calculated from n which is the denominator of the single coverage estimator (see Table 2).

Table 2: Applying the single coverage estimator to example data (SLEAC) with coverage thresholds of p1 = 20% and p2 = 50%

 

Source

n*

d1

d2

Covered

Coverage class

1

FANTA (p105)

0+34+2+22=58

34

HIGH

2 FANTA (p105)

3+22+9+16=50

25

MODERATE

* The denominator for the single coverage estimator. The data and calculations for each n are shown in Table 1.

Using the single coverage estimator in final-stage SQUEAC surveys requires specifying the numerator and denominator of the single coverage estimator as the likelihood (survey) data. Figure 2, for example, shows a Bayesian beta-binomial conjugate analysis performed using the BayesSQUEAC calculator with data from the first example in Table 1 with a Beta(14, 8) prior. In this case the numerator is:

 

 

and the denominator is:

 

 

 

Measuring MUAC during case-finding in a SQUEAC surveyWe also applied the point coverage, period coverage, and single coverage estimators to sixty-four surveys from SQUEAC assessments (i.e. SQUEAC stage three likelihood surveys) undertaken between 2009 and 2014 by the Coverage Monitoring Network (CMN) and VALID, assessing the coverage of CMAM (predominantly SAM treatment) programmes operating in Afghanistan, Angola, Bangladesh, Burkina Faso, Cameroon, Chad, Djibouti, Ethiopia, Haiti, Ivory Coast, Kenya, Mali, Mauritania, Niger, Nigeria, Rwanda, Senegal, Sierra Leone, Somalia, South Sudan, and Sudan. All three estimates were calculated using the conjugate prior used for the original surveys. The single coverage estimates were calculated using k = 3 when calculating Rout for all surveys. Figure 3 shows a scatter-plot of the point coverage and period coverage estimates against the single coverage estimates calculated from each of the sixty-four surveys. All of the plotted points for the point coverage estimates lie on or above the line of equality (if no correction is made then all of the plotted points will fall on this line). This is consistent with the single coverage estimator correcting the tendency towards underestimation of the point coverage estimator. All of the plotted points for the period coverage estimates lie on or below the line of equality (if no correction is made then all of the plotted points will fall on this line). This is consistent with the single coverage estimator correcting the tendency towards overestimation of the period coverage estimator.

Working on a SQUEAC concept mapConclusions

The proposed single coverage estimator provides a measure of overall programme performance with respect to coverage which has face validity, corrects the biases of the point and period coverage estimators, can be applied to data from previous coverage surveys, and requires no change to existing coverage assessment methods and tools.

The single coverage estimator may be used for tracking coverage over time using historical and current coverage survey data. Coverage estimates will need to be calculated using the single coverage estimator with all data. This will usually be possible as most coverage survey reports include the relevant data.

A survey tally sheet from a SLEAC surveyUncertainty remains regarding the mean length of an untreated SAM episode. Work on estimating this for the purposes of estimating programme caseloads in typical emergency and development contexts is currently being done by MSF and Epicentre. The mean length of a successful treatment episode should, whenever possible, be estimated from programme data. The default mean length of a successful treatment episode used in this article (i.e. ten weeks) should be reviewed using programme data and revised accordingly.

The proposed single coverage estimator should be used in preference to using either the point or period coverage estimators.

For more information, contact: Mark Myatt, email: mark[at]brixtonhealth.com

 

More like this

en-net: SLEAC and SQUEAC

Between the SLEAC and SQUEAC, which one should be done first and can SLEAC be done in a SMALL Area, say a district with a population of around 30,000 and geographically the...

en-net: Recovering cases in coverage calculation

I have just been looking at the formulae for coverage assessment, which defines the coverage as:

No. of SAM cases in treatment + No. of recovering cases in...

FEX: Considerations regarding coverage standards for selective feeding programmes

By Ernest Guevarra, Saul Guerrero and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...

FEX: Using SLEAC as a wide-area survey method

By Ernest Guevarra, Saul Guerrero, and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...

en-net: Experience using SQUEAC surveys to assess BSFP/TSFP coverage

Hi all I have read the few discussions in this forum on the use of SQUEAC surveys to assess the coverage of different elements of CMAM. While it seems the SQUEAC methodology...

FEX: How many lives do our CMAM programmes save? A sampling-based approach to estimating the number of deaths averted by the Nigerian CMAM programme

By Assaye Bulti (UNICEF), Stanley Chitekwe (UNICEF), Chloe Puett (ACF), and Mark Myatt (Brixton Health) Assaye Bulti is responsible for managing routine nutrition programme...

en-net: Recovering case

Hello Everyone, Could someone help me understand why in the circumstance described below we consider the child in the "recovering case group" while conducting a SQUEAC...

en-net: Advice on how much SQUEAC and LQAS coverage surveys cost and how long they take

Hi, We'd like to do a coverage survey in Darfur. We have 8 OTP sites with a total of 6,300 children under 5 in mainly rural areas and we hope to have at least 50% coverage....

en-net: Interpretation of rapid MUAC asessment

[i]From Caroline Muthiga:[/i] We conducted a rapid assessment recently using MUAC and measured 120 children. We found 13 children with a MUAC of <11cm, 35 between 11.0- 11.9,...

en-net: Nutrition assessment in small pieces livelihood zones

Unfortunately, our nutrition program fall under three livelihood area. The total program catchment population is about 120,000. The proportion of population in the three...

FEX: High OTP coverage through the Ministry of Health in Chad

By Casie Tesfai Casie Tesfai is currently the Nutrition Technical Advisor for the International Rescue Committee based in New York. She has 10 years of nutrition experience...

en-net: resources on CSAS

I'm trying to learn more about CSAS, but can only find summary or simple documents. Does anyone know where there is a proper, detailed manual on CSAS? That does help. Thanks...

FEX: New Method for Estimating Programme Coverage (Special Supplement 2)

By Mark Myatt This article gives an overview of the coverage estimation method developed for the Community Therapeutic Care (CTC) Research Programme in Malawi1. Coverage is...

FEX: en-net update, August to October 2012

By Tamsin Walters, en-net moderator Forty-seven questions were posted on en-net in the three months August to October 2012 inclusive, eliciting 152 replies. In addition, 18...

en-net: When to conduct Wide Area survey

Whats determines when to do a wide area survey?. A recent small area survey we conducted showed 2 out of 3 children in OTP were covered while 12 out of 25 children children...

en-net: Analysis of Nutritional Status

What are the most sound conclusions on nutrition status that can be drawn if only the following data are available (municipal level)?: 1) Under-5 population estimate, 2)...

en-net: Geographical area for coverage survey

I am planning a SQUEAC survey in Haiti. Our PTAs are spread through 5 Communes (Provinces) which are relatively close together. The problem is that in the middle of this 5...

en-net: How to assess coverage and challenges to coverage in home fortification program

The prevalence of malnutrition like anemia is very high among the under five children in underdeveloped countries. For improving anemia, nutritional status some NGOs were taken...

en-net: Caseload calculations (incidence rate)

I'm currently training some folks on how to calculate expected CMAM caseload using the (prevalence + incidence) * coverage equation. I'm having difficulty explaining to my...

FEX: Causal analysis and the SQUEAC toolbox

By Mara Nwayo and Mark Myatt Mara Nyawo is a nutritionist specialising in nutrition surveys and surveillance. She has nine years experience working in emergency and chronic...

Close

Reference this page

Safari Balegamire, Katja Siling, Jose Luis Alvarez Moran, Ernest Guevarra, Sophie Woodhead, Alison Norris, Lionella Fieschi, Paul Binns, and Mark Myatt (2015). A single coverage estimator for use in SQUEAC, SLEAC, and other CMAM coverage assessments. Field Exchange 49, March 2015. p81. www.ennonline.net/fex/49/singlecoverage