Increasing active travel: aims, methods and baseline measures of a quasi-experimental study

Alongside intensifying global problems of climate change and energy insecurity sits the more local but connected issue of insufficient physical activity, the health effects of which are evident in rising global rates of diabetes and many other non-communicable diseases [1, 2]. Policy advisers are seeking further evidence on the effectiveness of policy measures, such as promoting walking and cycling, that potentially offer multiple benefits, including enhanced health through physical activity, and reductions in energy use, traffic congestion and carbon emissions [3].

Consistent with the dearth of research on the effectiveness of health policy initiatives [4], several systematic reviews have shown that there is little robust evidence concerning effective interventions to increase walking and cycling [5, 6]. The aim of this paper is to describe the design, implementation, strengths and limitations of a quasi-experimental study of an intervention to increase active travel (walking and cycling).

The New Zealand Model Communities Programme (MCP) was jointly funded from mid-2010 by central and local government and provided an opportunity for a ‘natural experiment’. Our study was initially funded by small grants from the two MCP councils and the researchers’ universities, and then in 2012 by central government as part of a competitive research grant. The research group was independent of both the New Zealand Transport Agency, which channelled the MCP’s central government funding, and the two local governments involved in the MCP’s funding and implementation. However, using the principles and practices of partnership [7, 8], we established strong working relationships with the four local governments involved.

Insufficient physical activity is responsible for an estimated 3.2 million deaths per year globally [2]. Long-term observational studies have reported that individuals who walk and cycle regularly experience lower rates of cardiovascular disease (CVD), cancer, and other diseases [9–11]. However, interventions to promote cycling have not generally resulted in an increase in overall physical activity, or sustained reductions in body mass [12]. Two recent reviews have shown limited evidence for slowing weight gain amongst adults [13], but promising evidence for reducing diabetes [14]. Amongst children and adolescents, walking or cycling to school has been linked with improved cardiorespiratory fitness [15], muscular fitness (equivocal) [16] and lower weight (weak) [17].

New Zealand has high rates of car ownership and an increasing rate of obesity and diabetes. The New Zealand Census and the Household Travel Survey show a decline to around 2006 in active travel and substantial decreases in cycling and walking for children aged 5–14, and in cycling among teenagers, in the last two decades [18, 19]. Recent Travel Survey data suggest that cycling is now starting to increase for larger cities (>100,000 residents), but may still be declining for most smaller cities, and that walking may still be declining.

Research indicates that mode shift towards active travel is difficult to achieve without sustained effort [20, 21]. It is difficult to change long-standing and complex patterns of habitual behaviour in the face of pervasive social, economic and environmental forces such as highway building which maintain the status quo, or support increased car use. In addition, social investment in promoting physical activity, and particularly active travel, is currently limited by a lack of evidence on the nature of the institutional barriers to active travel investment, but it appears that few countries’ institutions effectively integrate public health and urban planning considerations in decision making [11]. Whether this is amenable to integrated national action plan development [22, 23] remains unclear.

Recent reviews have summarised current knowledge about encouraging walking and cycling [21, 24–27]. Ogilvie and colleagues reported that a range of interventions, including publicity campaigns and various engineering measures, have been tested and not been effective in causing a significant shift from car trips to walking and cycling [28]. They later concluded that community-wide promotional activities, in conjunction with improving infrastructure for cycling, may increase cycling by modest amounts, but there is a need for more precise measures of travel activity to assess behaviour change, with a focus on areas without an established cycling culture [12].

A review by Pucher and colleagues concluded that there were clear increases in cycling activity associated with: infrastructure interventions, such as bike-lanes and parking; integration with public transport; education and marketing programmes; bicycle access programmes; and changes in laws related to active transport [21]. They found that targeted interventions have achieved measurable mode shifts in some settings, but concluded that substantial increases in cycling will require a combination of many different interventions including improved physical infrastructure, pro-bicycle educational programmes, supportive land use planning and restrictions on car use. That is, a package may perform better than its constituent parts.

Most of the studies that were included in these systematic reviews were cross-sectional in nature and had significant limitations in their design and measurement. In short, there remain gaps in our understanding of the efficacy and relative importance of interventions to increase walking and cycling. There have been few experimental or quasi-experimental studies [29] and a large US controlled study, with four experimental communities and one control community, detected no significant changes [30]. The iConnect longitudinal study of cycling and walking is being carried out in a number of UK sites, but does not include control sites [31, 32]. In New Zealand, some cross-sectional research exists [33] and in Australia a study is being undertaken to evaluate a cycle system extension in Sydney using a control area and 2-year follow-up, with outcome variables to include quality of life impacts [34]. To our knowledge there are no completed systematic community trials of interventions to increase walking and cycling, which have included the broad range of outcomes we have included in our study.

In early 2010 the New Zealand Transport Agency sought tenders from local councils to establish the MCP, with the aim of encouraging the uptake of walking and cycling through structural changes and educational efforts. The MCP aimed to deliver safe, urban environments that would encourage ‘novice users’ to walk or cycle to school or to work in fully integrated walking and cycling transport networks [54]. Two North Island local governments, New Plymouth District Council and Hastings District Council were selected based on the central government’s criteria, and were funded $4.9 million and $4.3 m. respectively from July 2010 to July 2012. New Plymouth and Hastings are small cities of approximately the same population and latitude (Figure 1); both are largely suburban cities, with very few townhouses or apartment buildings. New Plymouth has a lower proportion of its population who are Māori (the indigenous population) (15%) compared to Hastings (24%), but median incomes are similar, as are median ages (Table 1).

The cities developed their programmes in line with local aims and were branded Let’s Go in New Plymouth and iWay in Hastings, but the strong similarities mean that they can usefully be evaluated side by side. Programme similarities include:

The differences between the programmes are relatively minor: New Plymouth includes visitors as a target group; Hastings focuses more on arterial routes (separated shared pathways that link Hastings to surrounding communities); and some sub-programmes are distinct, such as New Plymouth’s ‘dream street’ concept, an initiative to encourage local communities to re-design their street.

The main level of analysis will be at the level of the city (intervention vs control), mediated by individual exposure to the infrastructure component of the MCP interventions. This exposure is partially determined by geographic factors such as where individuals live, work, study; and how and where they travel, as well as their exposure to traffic. This, in turn, reflects individual demographic and lifestyle factors such as age, sex, social class, income, job, and mode of travel, influencing the way people lead their daily lives.

The follow-up measures will enable calculation of any changes in active travel behaviour attributable to the intervention, as well as any increase in relative inequalities in active travel behaviour. We will fit multi-level regression models on the available survey data on travel behaviour, which include national data being collected at the same time as the survey described here. The models will estimate individual change in mode choice using the data at the household level, extrapolated to the intervention and control cities. As the construction of the infrastructure continued over the period spanned by the survey, a factor in the model will measure whether any changes in the intervention cities relative to the control cities continued incrementally with time.

Secondary analyses will be carried out on outcomes including attitudes to travel, counts of cyclists and pedestrians, and road crashes (injury/death).

Our quasi-experimental ACTIVE study is attempting to increase the robustness of evaluation of walking and cycling programmes, by using a stratified random sample of participants in matched intervention and control cities, most of whom have been followed up for two years. The sample appears reasonably aligned with the 2006 Census: although low-income and younger people are under-represented.

Our baseline measures show that, as expected, driving was the dominant mode; indeed driving (or being driven) was the preferred option for all types of trips, except going to the local park. This preference for car travel does not seem to be predominantly influenced by access to bikes as, though almost everybody had access to a car, over half the sample had access to a bike. A minority of people (14%) reported a physical condition that prevented them from walking or cycling in the last seven days.

There are a number of strengths to this quasi-experimental study. The study has sufficient power to potentially detect a significant change in cycling and walking behaviour overall, and enable cautious causal inferences to be drawn about the efficacy of the programme. We collected the survey data at approximately the same time each year, during winter/early spring. As these seasons are not ideal for cycling and walking, this may exert a downward effect on active travel levels.

The random sample was classified to let us take account of potential confounders such as socio-economic status. Instrumental variables (the distance of a respondent’s dwelling from the major aspect of the intervention, the new structural cycleways and walkways) were measured to enable their potential use in sensitivity analyses.

New Zealand cities’ cycling and walking cultures have weakened over the last few decades, although as noted above, in some larger cities there has recently been an increase in cycling. The focus of this study on four smaller cities, all with relatively low levels of walking and cycling, allows us to illuminate the potential for change starting from a low base and facilitates the generalisation of our results to a range of societies dominated by car travel.

Nonetheless, there are a number of important methodological issue in the study, the first being our choice of control cities. As Ogilvie et al. [5], p.122 noted, ‘It may… be unrealistic to aspire to anything more than “broadly comparable” control areas’ in such studies. While our intervention city samples differ in certain respects from the control city samples, they match them closely in other significant respects, including socio-demographically and ethnically. There were some differences in travel behaviour, with slightly more people walking to work in the intervention cities than in the control cities. On the whole, however, the matching provides confidence that when we later compare the results from the control and intervention cities, the differences are not due to major inter-city differences. A further strength of our study is that we can use other survey data collected by the New Zealand Travel Survey over the study period to supplement the control data collected.

A difficult issue in relation to transport evaluation studies is when to begin collecting baseline data. [5]. Although MCP infrastructural works in principle started from mid-2010, in practice they took time to get underway, so that little had changed on the ground by the time baseline data were collected in mid-2011. However, some promotional activities had taken place, and this may have contributed to some ‘baseline awareness’ of the two MCP programmes by mid-2011, and possibly a diminishing of the extent of measured changes in behaviours. However, in each year of our study, we measured the participants’ awareness of the programmes; for example in mid-2011, only a minority of respondents were aware of the iWay programme.

Other authors of studies in the transport, health and environment area have noted difficulties with low recruitment or response rates (e.g. Saelens et al., [24], p.1557, referring to 15-20% rates); this, and on-line recruitment of samples, may limit generalizability of findings. However, response rates were relatively high in our study compared to existing studies and we collected longitudinal data, which will be more robust to non-response bias than cross-sectional data.

Our research relationship with local authorities was important. Most district council officers were very helpful, but in a control city, council cycling survey data were not provided, despite reminders. Consequently, a consistent set of cycling data from on-road surveys from all areas studied is not available. Our relationship to the councils also related to the practical, funding issues inherent in evaluating a natural experiment. We were not aware of the MCP until the money had been allocated, which meant that our study was initially very constrained for funds. There were two major consequences on the selection of outcome measures and the choice of interviewers. We focused on proxy health outcomes, such as levels of physical activity, rather than independent health indicators, such as BMI or cardiovascular functioning, largely because of the financial resources available. Similarly, although our use of local polytechnic students, who largely matched the socio-demographics of the cities they lived in, may have strengthened research capacity in these areas and enabled us to begin in a timely fashion, it may have reduced our overall response rate; this was boosted in the final year when we were able to employ professional interviewers. This change in interviewers’ experience over the three rounds of the study may also have introduced some instrumentation bias. However, such bias will be the same for both treatment and control cities, so it should not affect the comparisons in travel behaviour that we will estimate.

Data collection in the ACTIVE study has been successfully concluded. Multivariate analyses will include an assessment of the role of socio-demographic variables in explaining changes in levels of active travel to work or study, and physical activity. The data collected will allow comparisons over time and between intervention and control cities, to establish the extent to which the MCP is having an impact on active travel behaviour, and walking and cycling physical activity. We will also be able to establish whether the programme has been successful in significantly affecting crash outcomes, changing travel habits and attitudes, improving reported well-being, and raising public awareness of walking and cycling support.