Skip to main content

Table 2 Examples of natural experiment methods used for evaluating sweetened beverage taxes

From: How should we evaluate sweetened beverage tax policies? A review of worldwide experience

Study design (with statistical method)

Advantages

Challenges

Example of evaluations

Difference in difference (using a control group)

Reduces biases associated with time-varying factors related to the outcome of interest.

Requires that prior trends of the outcome are similar between treatment and control groups. Difficult to test if no prior data available.

Philadelphia (USA): the evaluation of the tax was based on a difference in difference analysis to estimate changes in sales, using Baltimore as the comparison city [30].

Difference in difference (with propensity score matching)

In absence of an experimental design PSM balances control and treatment comparison groups on basic characteristics using baseline data.

Unable to adjust for unobservable time variant variables.

Philadelphia (USA): Created propensity score weights as inclusion in difference-in-difference models to account for differences in the composition of the four comparison groups and changes in their composition over time [58].

Interrupted time series (ITS)

Creates a counterfactual based on pretax trends. Can be adapted to panel and cross-sectional data.

No control group to adjust for all potential exposures to other policies or factors associated with the outcome of interest.

Mexico: Adapted ITS to a panel of urban households to estimate changes in household beverage purchases, using a fixed effects regression and adding household and contextual variables [29].

UK: Controlled ITS to look at sugar content, prices and beverage product availability from 2 years pre-announcement to 1 year post-implementation [59]; Domestic turnover of UK soft drinks manufacturers pre-post announcement and implementation of the SDIL [60].

ITS with synthetic controls

Creates a synthetic control based on a pool of potential comparison groups.

Requires countries with same data sets for the outcome and variables associated with the outcome prior to the intervention to create the synthetic control. Requires the magnitude and trends in the pretax period are not statistically different between treatment and synthetic control

Mexico: Uses Mexico’s Consumer Price Index price data collected from urban retail outlets across 46 cities to construct a synthetic control product whose pre-tax price most closely tracks that of the treatment product (‘donor’ products comprised of all untaxed non-durables that are neither potential substitutes for taxed drinks nor subject to the concurrent junk food tax) [61].

ITS with correlated random effects

Adjusts for unobserved heterogeneity at the household level. Can be combined with ITS approaches to adjust for pre-intervention trends.

No control group to adjust for all potential exposures to other policies or factors associated with the outcome of interest.

Chile: estimated changes in beverage prices and purchases associated with a tax policy modification in a panel of urban households adapting a ITS model with a correlated random effects model [62].

Regression Discontinuity (RD)

Uses cutoff score on a pre-policy measure to determine allocation of treatment vs control and thus removes potential selection biases and increase internal validity of results.

Requires cutoff to be exogenous (not linked to outcomes). Results more relevant for observations around cutoff (external validity can be difficult to establish)

Denmark: Uses a regression discontinuity (RD) approach to assess the pass-through of the tax changes and a within-household pre-post design to estimate changes in purchases of soft drinks [63].