Boosters, Barriers, Questions: an approach to organising and analysing SQUEAC data

Box 1: Triangulation by source and method

It is important that the collected qualitative data are validated. In practice, this means that data are collected from as many different sources as possible. Data sources are then cross-checked against each other. If data from one source are confirmed by data from another source, then the data can be considered to be useful. If data from one source is not confirmed by data from other sources then more data should be collected, either from the same sources or from new sources, for confirmation. This process is known as triangulation.

There are two types of triangulation: Triangulation by source refers to data confirmed by more than one source. It is better to have data confirmed by more than one type of source (e.g. community leaders and clinic staff ) rather than just by more than one of the same type of source. Type of source may also be defined by demographic, socio-economic, and spatial attributes of informants. Lay informants such as mothers and fathers are sources of differing gender. Lay informants from different economic strata, different ethnic groups, different religious groups, or widely separated locations are also different types of source.

Triangulation by method refers to data confirmed by more than one method. It is better to have data confirmed by more than one method (e.g. semi-structured interviews and informal group discussions) than by a single method.

You should plan data collection to ensure triangulation by both source and method. The BBQ approach is designed to help you do this.

Data collection using triangulation is a purposeful and intelligent process. Data from different sources and methods should be regularly and frequently compared with each other. Discrepancies in the data are then used to inform decisions about whether to collect further data. If further data collection is required, these discrepancies help determine which data to collect, as well as the sources and methods to be used.

Box 2: Example of using the BBQ tool for triangulation by source and methods and hypothesis formation and testing in SQUEAC assessments

On the first day of collecting qualitative data, one of the teams found that carers of young children living in villages near health facilities delivering integrated CMAM (iCMAM) services were more aware of the iCMAM programme than carers of young children living in villages further away from health facilities delivering iCMAM services. This information, collected using informal group discussion with carers of young children in their home villages, was not confirmed by information collected by other teams. This finding was, therefore, placed in the ‘Questions’ section of the BBQ tool. To confirm this finding, questions were developed and incorporated into interview guides for semi-structured interviews intended to be administered to other sources (i.e. teachers and villages leaders and elders) on the following day.

On the second day of collecting qualitative data, information collected using semi-structured interviews with teachers, village leaders and village elders confirmed the original finding that distance was negatively associated with awareness of the iCMAM programme. The finding had, therefore, been confirmed by triangulation by both source and method:

The collected data led to the formation of two linked and formal hypotheses:

Carers of young children living in villages close (i.e. within 1000 metres) to health facilities delivering iCMAM services are aware of the iCMAM programme (i.e. know it exists, know that it treats malnourished children, know that entry is decided by mid-upper arm circumference (MUAC), and know that the programme delivers RUTF).

and:

Carers of young children living in villages far (i.e. further than 5 kilometres) from health facilities delivering iCMAM services are not aware of the iCMAM programme (i.e. do not know it exists, do not know that it treats malnourished children, do not know that entry is decided by MUAC, and do not know that the programme delivers RUTF).

To confirm these hypotheses, small studies were performed by two different teams on days three and four of the SQUEAC assessment. Each team travelled to two villages, one of which was located near (i.e. within 1000 metres) to a health facility delivering iCMAM services and the other located far (i.e. further than 5 kilometres) from a health facility providing iCMAM services. The EPI5 sampling method was used to select five households from each of the selected villages. The EPI5 sampling method was used because it is known to return a sample similar to a simple random sample of households. Carers of young children in each of the selected households were interviewed about their awareness of the programme. An in-depth interview guide was developed for this purpose. In addition, each team was given MUAC tapes and sachets of Ready to Use Therapeutic Food (RUTF) (two types) in order to test whether informants recognised them, reflecting an awareness of the programme.

The data arising from the small studies are summarised below:

The summary data were analysed using the simplified Lot Quality Assurance Sampling (LQAS) testing procedure with good awareness defined as more than 50% of carers of young children being aware of the programme.

The first hypothesis (i.e. good awareness if near to an iCMAM facility) would be confirmed if more than:

respondents were aware of the programme in the near villages. The study found ten respondents who were aware of the programme. The first hypothesis was, therefore, confirmed.

The second hypothesis (i.e. poor awareness if far from an iCMAM facility) would be confirmed if:

or fewer respondents were aware of the programme in the far villages. The study found one respondent who was aware of the programme. The second hypothesis was, therefore, confirmed.

Given these results, the SQUEAC assessment team concluded that distance was a factor affecting programme awareness and was likely to be a factor affecting coverage.

The approach outlined here is typical of the SQUEAC investigation process. That is:

1. Qualitative data are collected and validated using triangulation by source and method.
2. Validated qualitative findings are then used to develop formal hypotheses which are tested using simple quantitative techniques.

These are sometime referred to as Stage I and Stage II of a SQUEAC investigation.

A note on samples sizes and methods: Small sample sizes are common in SQUEAC. This is because the use of prior information acts to reduce both classification and estimation error. In the example small studies presented here, the association between proximity and awareness is very marked and a naïve frequentist analysis (i.e. an analysis that discounts all prior information) testing the null hypothesis that programme awareness was independent of proximity to the programme would return a p-value of p < 0.0001 (one-tailed Fisher Exact Test). This is very strong evidence against the null hypothesis. An estimation approach would return a risk ratio of 10.00 (95% CI = 1.56; 64.20) with proximity as the ‘risk exposure’.