Study design
The study was conducted in Samia Sub-county, Busia County in western Kenya (Fig. 1). Samia is largely rural with peri-urban pockets and had an estimated population of 119,246 living in 29 sub-locations (the smallest administrative unit in Kenya) in 2016 [23, 24]. Samia is a malaria-endemic sub-county with 12 public health facilities and an active CHV programme [17]. Kenya implemented a national three-phase MC in 2011–2012, with distribution occurring in Samia in mid-2011 [25]. In 2014, 48·3% of households in Busia County had ≥1 LLIN per two people [26]. A subsequent national MC was conducted between July and November 2015 with distribution in Busia County occurring in October 2015.
The original study design, described fully elsewhere [27], intended to compare the costs and effects (i.e. coverage outcomes) in a real-life setting of a single combination of five distribution channels: MC; ante-natal and child health clinics (ANCC); social marketing (SM); commercial outlets (CO) and a pilot continuous LLIN distribution project using community health volunteers to distribute vouchers for nets (CHV LLIN) (Details of each channel are provided in Table S1). The pilot CHV LLIN distribution was initially planned to start directly after the MC to address gaps and maintain UC, with the evaluation planned for 1-year post-MC. Unlike other continuous channels, the CHV channel was (and remains to date) not routinely operational for LLIN distribution in the country. However, two operational factors resulted in modifications to the published study design. First, delays in the MC resulted in CHV distribution preceding the MC. Second, due to insufficient LLINs for the MC, the National Malaria Control Program (NMCP) excluded the area of Samia Sub-county where the CHV channel was operational, from the MC. The CHV channel was implemented in 18 sub-locations (population 63,772) and the MC channel implemented in 11 different sub-locations (population 55,474), with all 29 sub-locations having ANCC, SM and CO. Thus, the modified study design compared two combinations of five delivery channels, herein-after referred to as ‘intervention’ and ‘control’ arms. The intervention arm was comprised of four channels: CHV, ANCC, SM and CO, and the control arm was comprised of MC, ANCC, SM and CO.
The CHV channel consisted of two phases: a 12-month top-up phase (January to December 2014) to increase the number of LLINs in the household to achieve UC and a 6-month continuous distribution phase (January to June 2015) where UC was monitored and maintained. In both phases, householders received vouchers from CHVs and exchanged vouchers for LLINs at health facilities. The CHV network consisted of 211 CHVs linked to health facilities in the intervention arm areas and 140 CHVs in the control arm areas, however these CHVs were not involved in LLIN distribution and data on the linked facilities in the control area was not available. In each intervention arm sub-location, at least one CHV per village distributed vouchers. The ANCC, CO and SM channels ran continuously throughout the study period.
The primary analysis compared two combinations of the five channels, with the difference between arms being the presence of CHV (intervention) or the presence of MC (control). Secondary analyses explored comparisons between individual distribution channels in the entire study area.
Effects
As part of the study, a randomised household survey was conducted from July to August 2016 to measure effects (i.e. LLIN ownership, source channel, coverage and use outcomes). Thirty data collectors were recruited and trained by an author (VW) in the use of the survey tool which was pilot tested prior to use. Household heads had the purpose of the study, time implications, risks and benefits and their right to withdraw at any point explained and were given an opportunity to ask questions. Consented heads were asked to sign two copies of the consent form which was witnessed with a copy retained by the household and the study. Data were collected using user-friendly scannable forms developed using TeleForms software. Completed forms were transported to the KEMRI research centre, logged in, scanned and data transferred to the backend MS Access database into password protected desktop computers that were only accessible to the investigators and KEMRI data management personnel. Data codes were developed, and data queries were regularly run for quality control and assurance. Any missing data were resolved by the supervisor as soon as possible. Where households were not available, field workers called back three times until the data was collected. The source channel for each net owned was ascertained during the household survey. Respondents were asked to show the field workers each net in turn and asked, “Where did you get this net from?”. Field workers validated responses based on either known facts about the net sources (e.g. colour and labelling) or channels (e.g. MD nets were green and stickered, ANCC and CHV nets were blue, CHV nets required a voucher or “paper” to obtain). For nets purchased from a shop (CO) or kiosk (SM), field workers were trained to probe for price and specific shop and kiosk details. Field workers validated responses with known prices for subsidised SM nets and SM kiosks details. Field supervisors resolved any uncertainties related to net source in the field.
Reflecting the modifications described above to the protocol, the sample-size for the household survey was increased to 1000 with consenting households selected randomly from community unit registers for the intervention and control arm sub-locations, and data being scanned into the MS Access database, cleaned and analysed in R and Stata [28, 29]. As per protocol, the main effectiveness measure was LLIN use (the proportion of the population that reported sleeping under an LLIN the previous night). The proportions of children < 5 years of age and pregnant women who reported sleeping under an LLIN the previous night, households with ≥1 LLIN and households with UC were also measured. The UC indicator measures the proportion of households that have enough LLINs to cover all individuals who spent the previous night in the surveyed household, assuming one LLIN per two people. Critically, it describes the intra-household coverage gap, i.e. households which own ≥1 LLIN but have not yet achieved UC [30]. In addition to the protocol measures, household LLIN access (proportion of the population that could have slept under an LLIN, assuming one LLIN is used by two people) and use of existing LLINs (proportion of existing LLINs reportedly used the previous night) were measured.
Household survey data on the aggregate number and source of LLINs was used to calculate the proportion of LLINs by channel, by arm and for the pooled sample. The proportion of LLINs by channel was also analysed at individual household level by arm and for the pooled sample, with households stratified by coverage level (i.e. households with any net, ≥1 net but not UC, and UC). All indicators were calculated separately for intervention, control arms and pooled sample. Statistical tests for association and difference in proportions were estimated with 95% confidence intervals (CI).
Cost and cost-effectiveness
Cost data on each channel were retrospectively collected using questionnaires administered to the distribution channel implementers (i.e. Ministry of Health, non-governmental organisation [NGO] partners and commercial outlets). The research team visited all retail shops and wholesalers in the commercial centre of Samia to identify those selling LLINs and request participation and consent. Total economic and financial costs (unannualised and annualised) by channel were calculated in 2015 United States Dollars ($) and summed to compute total costs of all channels operating in the sub-county. Commodity costs (i.e. LLINs) were stripped out of all cost and cost-effectiveness analysis to allow comparison of distribution costs, independent of potential differences in commodity prices.
Unit cost per channel was calculated by dividing the total cost per channel cost by the number of nets or vouchers distributed per channel using data reported by implementing partners (supply-side) and separately, using data on LLIN source/channel as reported in the household survey (demand-side estimate).
Costs were incurred and measured by channel, but for the purpose of the analysis, per channel costs were allocated to arms. Aggregated per channel costs were used to estimate per arm costs in two ways (planned and observed) illustrated by the logic flow chart (Fig. 2). First, based on the trial design, which assumed no MC costs in the intervention arm and no CHV costs in the control arm (planned cost) and with costs of channels operating in both arms split equally. Second, according to the proportion of LLINs by source/channel in the intervention versus control arms from the household survey (observed cost). For example, 201/1279 nets from MC were observed in intervention arm; therefore, 23·5% of MC costs were allocated to intervention arm with the remainder allocated to control arm. Planned and observed cost-per-arm estimates were then used to calculate unit costs per arm by dividing by reported quantity of vouchers/LLINs distributed using supply-side and separately, demand-side data. Supply-side data was the number of vouchers (CHV and SM) or LLINs (MC, ANCC, CO) distributed as reported by the implementers. Demand-side data was the number of nets by LLIN source (channel) for nets recorded in the household survey, multiplied by the sample proportion.
Cost-effectiveness ratios were calculated for each arm and the pooled sample, and the incremental cost-effectiveness ratio (ICER) was calculated to compare the intervention with the control channels. The ICER represents the ratio of change in costs to change in effects and is interpreted using the cost-effectiveness plane. Reflecting global and national UC policy targets and the importance of ownership as a determinant of LLIN use, the cost-effectiveness indicator was cost per additional household with UC (i.e. costs incurred divided by change in UC achieved) calculated for the intervention arm, control arm and the pooled sample. The change in UC was calculated by subtracting the proportion of households with UC at baseline, assuming the same starting point for each arm taken from the 2014 Demographic and Health Survey [26], from the post-intervention proportion of households with UC as measured by the household survey. While the societal perspective is considered gold standard in economic evaluation, decision makers within the health sector may be more interested in the health system perspective which represents resources that they must commit. Hence the analysis was conducted including and excluding the CO channel costs reflecting the difference between societal and health system perspectives, respectively. Uncertainty estimates on cost-effectiveness were obtained by re-running the analysis using the 95% CI range on the proportion of households with UC.
Equity
Household-level socioeconomic and asset ownership data were used to assign each household to a quintile (1, poorest to 5, least-poor) using principal components analysis [31]. The number and proportion of LLINs per household by quintile were compared by channel, arm and overall. Equity by channel, arm and for the pooled sample (i.e. overall equity) was analysed using concentration indices [32]. A concentration index (C.Ind) gives an indication of equity, independent of contribution to coverage, with a C. Ind of zero implying perfect equity, − 1 the highest degree of pro-poor inequity (distribution favours the poor), and + 1 the highest degree of inequity favouring the least-poor.