Boosters, Barriers, Questions: an approach to organising and analysing SQUEAC data
By Andrew Prentice (VALID), Balegamire Safari Joseph (VALID), Esther Ogonda McOyoo (Concern), Faith Manee Nzidka (ACF), Hassan Ali Ahmed (Mercy USA), Jackson N Chege (Islamic Relief), Jacqueline Wairimu Macharia (ACF), Kennedy Otieno Musumba (ACF), Lilian Mwikari Kaindi (ACF), Lioko Kiamba (ACF), Mark Murage Gathii (IMC), Muireann Brennan (CDC), Samuel Kirichu (Concern), Salim Athman Abubakar (IMC), Stephen Musembi Kimanzi (IMC) and Mark Myatt (Brixton Health)
This article outlines an approach to organising and analysing collected data and for planning further data collection during a SQUEAC coverage assessment. The approach, known as ‘Boosters, Barriers, Questions (BBQ)’ involves examining the collected data for boosters (i.e. anything that might act to support coverage) and barriers (i.e. anything that might act to undermine coverage) – see Figure 1. The approach was developed during a Coverage Monitoring Network (CMN)1 training on the SQUEAC coverage assessment methodology in Kenya in October and November 2012.
The BBQ approach uses three panes to record (1) boosters, (2) barriers and (3) issues arising that require further data collection (questions). A fourth pane acts as a key to symbols that are used to indicate data sources and data collection methods. Figure 2 shows the parts of the BBQ tools and explains their purpose.
The town of Meti
A large hand-drawn BBQ tool, such as is shown in Figure 3, proved useful for managing a SQUEAC investigation. The BBQ tool provides a summary of the current state of the investigation and serves as a focal point when deciding data collection needs and dividing tasks between team members. The collaborative focus provided by the BBQ tool facilitates team building and improves the quality of the investigation.
When using the BBQ tool, each of the listed boosters and barriers is tagged with symbols that indicate the different sources of data supporting each finding (e.g. programme staff, carers of severe acute malnutrition (SAM) cases, community leaders) and the different methods used to collect the data (e.g. structured interviews, semi-structured interviews, informal group discussions). The use of these symbols allows the easy identification of findings that have, or have not, been validated using triangulation by source and method – see Box 1, Box 2, and Figure 4.
Box 1: Triangulation by source and method
It is important that the collected qualitative data are validated. In practice, this means that data are collected from as many different sources as possible. Data sources are then cross-checked against each other. If data from one source are confirmed by data from another source, then the data can be considered to be useful. If data from one source is not confirmed by data from other sources then more data should be collected, either from the same sources or from new sources, for confirmation. This process is known as triangulation.
There are two types of triangulation: Triangulation by source refers to data confirmed by more than one source. It is better to have data confirmed by more than one type of source (e.g. community leaders and clinic staff ) rather than just by more than one of the same type of source. Type of source may also be defined by demographic, socio-economic, and spatial attributes of informants. Lay informants such as mothers and fathers are sources of differing gender. Lay informants from different economic strata, different ethnic groups, different religious groups, or widely separated locations are also different types of source.
Triangulation by method refers to data confirmed by more than one method. It is better to have data confirmed by more than one method (e.g. semi-structured interviews and informal group discussions) than by a single method.
You should plan data collection to ensure triangulation by both source and method. The BBQ approach is designed to help you do this.
Data collection using triangulation is a purposeful and intelligent process. Data from different sources and methods should be regularly and frequently compared with each other. Discrepancies in the data are then used to inform decisions about whether to collect further data. If further data collection is required, these discrepancies help determine which data to collect, as well as the sources and methods to be used.
Box 2: Example of using the BBQ tool for triangulation by source and methods and hypothesis formation and testing in SQUEAC assessments
On the first day of collecting qualitative data, one of the teams found that carers of young children living in villages near health facilities delivering integrated CMAM (iCMAM) services were more aware of the iCMAM programme than carers of young children living in villages further away from health facilities delivering iCMAM services. This information, collected using informal group discussion with carers of young children in their home villages, was not confirmed by information collected by other teams. This finding was, therefore, placed in the ‘Questions’ section of the BBQ tool. To confirm this finding, questions were developed and incorporated into interview guides for semi-structured interviews intended to be administered to other sources (i.e. teachers and villages leaders and elders) on the following day.
On the second day of collecting qualitative data, information collected using semi-structured interviews with teachers, village leaders and village elders confirmed the original finding that distance was negatively associated with awareness of the iCMAM programme. The finding had, therefore, been confirmed by triangulation by both source and method:
|Carers of young children||Informal group discussion|
|Village chiefs||Semi-structured interview|
|Village elders||Semi-structured interview|
The collected data led to the formation of two linked and formal hypotheses:
Carers of young children living in villages close (i.e. within 1000 metres) to health facilities delivering iCMAM services are aware of the iCMAM programme (i.e. know it exists, know that it treats malnourished children, know that entry is decided by mid-upper arm circumference (MUAC), and know that the programme delivers RUTF).
Carers of young children living in villages far (i.e. further than 5 kilometres) from health facilities delivering iCMAM services are not aware of the iCMAM programme (i.e. do not know it exists, do not know that it treats malnourished children, do not know that entry is decided by MUAC, and do not know that the programme delivers RUTF).
To confirm these hypotheses, small studies were performed by two different teams on days three and four of the SQUEAC assessment. Each team travelled to two villages, one of which was located near (i.e. within 1000 metres) to a health facility delivering iCMAM services and the other located far (i.e. further than 5 kilometres) from a health facility providing iCMAM services. The EPI5 sampling method was used to select five households from each of the selected villages. The EPI5 sampling method was used because it is known to return a sample similar to a simple random sample of households. Carers of young children in each of the selected households were interviewed about their awareness of the programme. An in-depth interview guide was developed for this purpose. In addition, each team was given MUAC tapes and sachets of Ready to Use Therapeutic Food (RUTF) (two types) in order to test whether informants recognised them, reflecting an awareness of the programme.
The data arising from the small studies are summarised below:
|Study team||Village name||Distance class||Distance from iCMAM facility||Number of respondents interviewed||Number of respondents aware of the programme||Number of respondents not aware of the programme|
The summary data were analysed using the simplified Lot Quality Assurance Sampling (LQAS) testing procedure with good awareness defined as more than 50% of carers of young children being aware of the programme.
The first hypothesis (i.e. good awareness if near to an iCMAM facility) would be confirmed if more than:
respondents were aware of the programme in the near villages. The study found ten respondents who were aware of the programme. The first hypothesis was, therefore, confirmed.
The second hypothesis (i.e. poor awareness if far from an iCMAM facility) would be confirmed if:
or fewer respondents were aware of the programme in the far villages. The study found one respondent who was aware of the programme. The second hypothesis was, therefore, confirmed.
Given these results, the SQUEAC assessment team concluded that distance was a factor affecting programme awareness and was likely to be a factor affecting coverage.
The approach outlined here is typical of the SQUEAC investigation process. That is:
- Qualitative data are collected and validated using triangulation by source and method.
- Validated qualitative findings are then used to develop formal hypotheses which are tested using simple quantitative techniques.
These are sometime referred to as Stage I and Stage II of a SQUEAC investigation.
A note on samples sizes and methods: Small sample sizes are common in SQUEAC. This is because the use of prior information acts to reduce both classification and estimation error. In the example small studies presented here, the association between proximity and awareness is very marked and a naïve frequentist analysis (i.e. an analysis that discounts all prior information) testing the null hypothesis that programme awareness was independent of proximity to the programme would return a p-value of p < 0.0001 (one-tailed Fisher Exact Test). This is very strong evidence against the null hypothesis. An estimation approach would return a risk ratio of 10.00 (95% CI = 1.56; 64.20) with proximity as the ‘risk exposure’.
Findings associated with different sources and/or methods can be treated as validated. Figure 5, for example, shows a barrier “Mothers go to traditional healers who are not linked to the programme” revealed by in-depth interviews with carers of SAM cases in the programme and several informal group discussions with traditional birth attendants and traditional healers.
Findings associated with few sources of data and/or few methods of data collection are candidates for further investigation. Specific questions for further investigation are listed in the central ‘Questions’ section of the BBQ tool. Figure 5, for example, shows a potential barrier “Only person who can identify malnutrition is the Community Health Worker” revealed by a single source/method (i.e. informal group discussion with carers of young children in communities) and required, therefore, further investigation by collecting data from different sources and/or similar sources using different methods.
As the investigation proceeds, the BBQ tool is redrafted to (e.g.) combine similar findings and remove invalidated findings. Figure 6 shows the result of redrafting the BBQ tool shown in Figure 3 to combine similar findings. Note how some of the findings related to barriers have been combined using diagrams showing cause and effect linkages between barriers.
Grouping findings by consequence helps with building concept maps that describe the relationships between boosters and barriers in a programme. For example, a programme's failure to recruit traditional healers as community-based case-finders may lead to late admissions, complicated cases requiring long stays or inpatient care (which may lead to defaulting), poor outcomes and negative opinions of the programme. Figure 7 shows a fragment of a programme concept map illustrating these relationships.
Figure 8 shows the first draft of a programme concept map from day five of the SQUEAC investigation. This illustrates the richness of data that arises from SQUEAC investigations and the ability of the BBQ tool to assist with data-analysis and presentation.
Sorting the lists of boosters and barriers into three categories with regard to the likely size of their effect on coverage (i.e. large, moderate, and small effects on coverage) helps with building the prior for the stage three survey.
The BBQ tool proved useful during the CMN training in Kenya and helped trainees make sense of large quantities of data from many and disparate sources. The boosters and barriers model helped trainees maintain the focus of the investigation and to plan data collection. The BBQ tool may be used as an alternative to mindmapping or as a complement to mind-mapping.
For more information, contact: email@example.com
1See news piece in this issue about the CMN Project
More like this
By Lenka Blanárová, Sophie Woodhead and Mark Myatt Lenka Blanárová is a Community Mobilisation Advisor at ACF-UK, supporting nutrition programmes...
By Mark Myatt Mark Myatt is a consultant epidemiologist and senior research fellow at the Division of Ophthalmology, Institute of Ophthalmology, University College London....
en-net: SQUEAC Stage
As per the SQUEAC investigation manual/guide, the purpose of the exercise in Stage 2 is to confirm hypothesis of homogeneity/heterogeneity of coverage in the program area. if...
in SQUEAC technical reference , it's say that late admissions may be investigated by plotting MUAC at admission. In some country we have OTP and ITP, in niger this is...
FEX: Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?
View this article as a pdf Lisez cet article en français ici By Mark Myatt and Ernest Gueverra Mark Myatt is a consultant epidemiologist. His areas of expertise...
By Ben Allen, Mark Myatt, Nikki Williamson, Danka Pantchova and Hassan Ali Ahmed Ben Allen has been Global Coverage Advisor for Action Against Hunger UK for the past two...
FEX: Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage assessments?
Lisez cet article en français ici This is a poscript for research article 'Is there a systematic bias in estimates of programme coverage returned by SQUEAC coverage...
We are conducting SQUEAC for IMAM programme coverage in Garbatulla sub County, we realised that same barriers in previous SQUEAC conducted in 2013 resurfaced again during the...
By Ernest Guevarra, Saul Guerrero, and Mark Myatt Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public...
By Mara Nwayo and Mark Myatt Mara Nyawo is a nutritionist specialising in nutrition surveys and surveillance. She has nine years experience working in emergency and chronic...
en-net: 2nd SQUEAC Planning
We are planning to conduct SQUEAC investigation of CMAM program which is about to end in Dec 2016. Last SQUEAC survey was conducted in 2015 for which the results...
What is the procedure to take after the team encounter not achieving the minimum sample size (number of SAM cases) in the selected villages during the wide area survey...
I'm looking into common practices/guidance around assessing programmatic coverage for Blanket Supplementary Feeding Programs (BSFPs). I read conflicting views of the use of...
By Lily Schofield, Selome Gizaw Lalcha and Terefe Getachew Lily Schofield has worked in many countries in Africa and Asia as a nutrition consultant. She has been involved in...
I am planning a SQUEAC survey in Haiti. Our PTAs are spread through 5 Communes (Provinces) which are relatively close together. The problem is that in the middle of this 5...
By Casie Tesfai Casie Tesfai is a Nutrition Technical Advisor for the International Rescue Committee (IRC) based in New York since 2012. She has over 10 years of humanitarian...
FEX: Determining predictors for severe acute malnutrition: Causal analysis within a SQUEAC assessment in Chad
By Ruwan Ratnayake, Casie Tesfai and Mark Myatt Ruwan Ratnayake is the Epidemiology Technical Advisor with the International Rescue Committee based in New York. He supports...
Summary of review1 Lady Health Worker and her husband (teacher) working as volunteers Interview by Saul Guerrero, ACF Action Against Hunger, in collaboration with...
I would like to know if the qualitative data collected during squeac exercise through interviews/focus group discussions, it contributes in percentage of coverage estimation or...
By Sophie Woodhead, Jose Luis Alvarez Moran, Anne Leavens, Modibo Traore, Anna Horner, Saul Guerrero Based within the ACF-UK Operations Team, Sophie Woodhead coordinates all...
Reference this page
By Andrew Prentice (VALID), Balegamire Safari Joseph (VALID), Esther Ogonda McOyoo (Concern), Faith Manee Nzidka (ACF), Hassan Ali Ahmed (Mercy USA), Jackson N Chege (Islamic Relief), Jacqueline Wairimu Macharia (ACF), Kennedy Otieno Musumba (ACF), L (2013). Boosters, Barriers, Questions: an approach to organising and analysing SQUEAC data. Field Exchange 45, May 2013. p6. www.ennonline.net/fex/45/boosters