Menu ENN Search

Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?

Lisez cet article en français ici

This is a poscript for research article 'Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?' by Mark Myatt and Ernest Gueverra, which you can read here.

By Sheila Isanaka, Rebecca F. Grais, and Ben G.S. Allen

We thank Mark Myatt and Ernest Gueverra for adding this important work to the ongoing discussion surrounding appropriate coverage methodologies in the management of acute malnutrition. A 2015 review of coverage methodologies highlighted uncertainty in the use of currently recommended methods for coverage assessment and the need for more peer-reviewed evidence to inform global guidance (Epicentre, 2015). We are delighted that our work (Isanaka et al 2018) may have motivated additional consideration of these important issues and hope that stakeholders and policy makers continue to insist on high-quality, evidence-based experience to inform nutrition programming. 

The report by Myatt and Gueverra (2019) shows that coverage estimation using the SQUEAC methodology can yield biased estimations in either direction. For the first time, Myatt and Gueverra quantify the magnitude of this bias, showing only a moderate correlation between the prior and likelihood estimates (Pearson correlation coefficient 95% confidence interval: 0.67 to 0.78). This new evidence is consistent with our findings and supports our conclusion that conflicts between prior and likelihood modes are possible and can lead to biased and imprecise coverage estimates. As discussed by Myatt and Gueverra, the risk of such bias is low when trained staff conduct a survey using appropriate SQUEAC methods. Our work supports this finding, as we similarly show no conflict when an external support team estimated the prior. Our experience, however, goes further than what is possible in the secondary analysis of Myatt and Gueverra to suggest that in resource-limited settings, where sufficient capacity and resources may not be available and the correct methodology may not be faithfully executed, conflict and bias may be more common. 

We wholeheartedly agree that coverage assessments should be done using appropriate methods as outlined in the SQUEAC manual (Myatt et al. 2012). Our SQUEAC assessment followed this guidance, including triangulation by source and method, use of the BBQ tool, sampling to redundancy and a small survey. We used booster and barrier weighting by caregivers of severely acutely malnourished (SAM) children as reported in other contexts (Blanárová et al. 2016) as one component in developing the prior mode. The various prior modes in our analyses were combined to simulate different potential scenarios, including situations where external support is not available. This was done for the pedagogic purpose of the study, and as discussed in the paper does not necessarily reflect typical SQUEAC procedures.

We note that the database employed in the analysis of Myatt and Gueverra is likely comprised of surveys conducted by experienced coverage consultants (those provided, for example, by the Coverage Monitoring Network or Valid International) and therefore include prior modes developed by dedicated consultants using gold standard methods that may be less likely to conflict. The analyses further include data from supplementary feeding programmes and does not standardise calculations of coverage estimate according to current guidance to use single coverage (Balegamire S, 2015), analytical choices which may influence the extent to which conflicts and bias were detected. Nonetheless, we welcome the new evidence presented by Myatt and Gueverra (2019) as an important step towards better understanding of the implications of using the SQUEAC methodology for valid coverage estimation. 

Overall, we look forward to continued evidence-based and peer-reviewed discussion of appropriate coverage methodologies. Several methodologies are available to monitor programme coverage, and the appropriate study design should be selected in consideration of team capacity, resources and reporting requirements. SQUEAC can be a technically demanding method and requires the appropriate capacity to avoid the potential for bias. As both we and Myatt and Gueverra have shown, conflict and biased coverage estimation are possible and should be considered in selecting the appropriate study design and allocating appropriate resources for assessment. 

Read more...

References

Balegamire S, Siling K, Alvarez Moran JS, Guevarra E, Woodhead S, Norris A, Fieschi L, Paul Binns  P, and Myatt M, A single coverage estimator for use in SQUEAC, SLEAC, and other CMAM coverage assessments, 2015, Field Exchange 49 Full text available from: https://www.ennonline.net/fex/49/singlecoverage

Blanárová L, Woodhead S, and Myatt M, Community weighting of barriers and boosters in Democratic Republic of Congo, 2016, Field Exchange 52 Full text available from: https://www.ennonline.net/fex/52/barriersandboostersdrc

Epicentre. Open review of coverage methodologies: comments, questions and ways forward. London: Coverage Monitoring Network; 2015.Isanaka S, Hedt-Gauthier BL, Grais RF, Allen BG, Estimating program coverage in the treatment of severe acute malnutrition: a comparative analysis of the validity and operational feasibility of two methods, Population Health Metrics, 2018,16:100,1-9 Full text available from: https://pophealthmetrics.biomedcentral.com/articles/10.1186/s12963-018-0167-3

Myatt M, Guevarra E, Fieschi L, Norris A, Guerrero S, Schofield L, Jones D, Emru E, Sadler K (2012) Semi-Quantitative Evaluation of Access and Coverage (SQUEAC) / Simplified Lot Quality Assurance Evaluation of Access and Coverage (SLEAC) Technical Reference, Food and Nutritional technical Assistance III Project (FANTA-III), FHI 360 / FANTA, Washington, DC, October 2012 https://www.fantaproject.org/monitoring-and-evaluation/squeac-sleac

More like this

FEX: Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?

View this article as a pdf Lisez cet article en français ici By Mark Myatt and Ernest Gueverra Mark Myatt is a consultant epidemiologist. His areas of expertise...

FEX: Community weighting of barriers and boosters in Democratic Republic of Congo

By Lenka Blanárová, Sophie Woodhead and Mark Myatt Lenka Blanárová is a Community Mobilisation Advisor at ACF-UK, supporting nutrition programmes...

FEX: A single coverage estimator for use in SQUEAC, SLEAC, and other CMAM coverage assessments

By Safari Balegamire (VALID International), Katja Siling (World Food Programme), Jose Luis Alvarez Moran (Coverage Monitoring Network), Ernest Guevarra (VALID International),...

FEX: Coverage assessment forum launched on en-net

En-net is launching a new Coverage Assessment forum area for discussion of methods used in the assessment of programme coverage of the outpatient therapeutic programme (OTP)...

en-net: Sample size estimation for follow up CSAS survey

We are doing a sort of follow up coverage assessment after two years of CMAM program. The project had a baseline coverage estimate in 2011 (used CSAS). The project targets >70%...

en-net: Advice on how much SQUEAC and LQAS coverage surveys cost and how long they take

Hi, We'd like to do a coverage survey in Darfur. We have 8 OTP sites with a total of 6,300 children under 5 in mainly rural areas and we hope to have at least 50% coverage....

FEX: Sampling in insecure environments: Field experiences from coverage assessments in Afghanistan

By Ben Allen, Mark Myatt, Nikki Williamson, Danka Pantchova and Hassan Ali Ahmed Ben Allen has been Global Coverage Advisor for Action Against Hunger UK for the past two...

en-net: SQUEAC - Weighted boosters and barriers

I have tried to check different SQUEAC investigation reports. I am observing inconsistencies in weighing and scoring system. Regardless of the number of boosters and barriers,...

en-net: SQUEAC Coverage survey

We did 2 SQUEAC coverage assessments recently and I have confused with something with in it 1. What is the difference between Alpha and Beta prior ? What is Alpha and Beta...

FEX: Using SLEAC as a wide-area survey method

By Ernest Guevarra, Saul Guerrero, and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...

en-net: How to assess coverage and challenges to coverage in home fortification program

The prevalence of malnutrition like anemia is very high among the under five children in underdeveloped countries. For improving anemia, nutritional status some NGOs were taken...

en-net: Geeting the alpha and beta values

Hi. Im new to squeac assessment. I needed a guidance of knowing how to set the alpha and beta values in the BayeSQUAC. Lets say the assessment team feels the program is between...

en-net: Should results for small and wide area surveys be combined?

Is it advisable to combine the number of children found during small area surveys to the number of children found during wide area surveys? In a recent coverage survey, in...

FEX: Considerations regarding coverage standards for selective feeding programmes

By Ernest Guevarra, Saul Guerrero and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...

en-net: Geographical area for coverage survey

I am planning a SQUEAC survey in Haiti. Our PTAs are spread through 5 Communes (Provinces) which are relatively close together. The problem is that in the middle of this 5...

FEX: Determining predictors for severe acute malnutrition: Causal analysis within a SQUEAC assessment in Chad

By Ruwan Ratnayake, Casie Tesfai and Mark Myatt Ruwan Ratnayake is the Epidemiology Technical Advisor with the International Rescue Committee based in New York. He supports...

FEX: en-net update, August to October 2012

By Tamsin Walters, en-net moderator Forty-seven questions were posted on en-net in the three months August to October 2012 inclusive, eliciting 152 replies. In addition, 18...

FEX: Something for everyone: three perspectives from a recent coverage assessment in Pakistan

Summary of review1 Lady Health Worker and her husband (teacher) working as volunteers Interview by Saul Guerrero, ACF Action Against Hunger, in collaboration with...

en-net: SQUEAC/SMART

Is there an upcoming training on SQUEAC/SMART....anyone any information please share.. Hello, Are there any additional trainings planned for coverage surveys? [b]Workshop...

en-net: Estimating coverage in an exhaustive screening exercise

Dear all

We often conduct exhaustive MUAC screening and we calculate coverage for SAM and MAM using a simple formula which divides the number of children with SAM (or...

Close

Reference this page

Sheila Isanaka, Rebecca F. Grais, and Ben G.S. Allen (2019). Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?. Field Exchange 59, January 2019. p35. www.ennonline.net/fex/59/squeacpostscript