Primary objective
-
1.
To measure the effectiveness of IRIS alongside a comparator intervention, in general practice, both designed to improve the healthcare response to DVA in primary care.
Secondary objectives
-
2.
To understand the process of IRIS implementation, using a mixed methods approach, including survey and qualitative data.
-
3.
To evaluate IRIS implementation, alongside IRIS’ sustainability, with in depth case studies of two different IRIS areas using interviews, participant observation and document analysis.
Design
A four year observational, pragmatic, mixed methods, implementation MRC phase IV study [27]:
The principal design is a segmented regression analysis of interrupted time series (ITS) data (primarily, referrals received by DVA workers) from general practices that implemented the IRIS intervention and a comparator borough in which the general practices did not implement the IRIS intervention.
There are approximately 386,277 women, aged 16 years and above (patients), registered at the 140 general practices, in the four north-east London implementation borough sites (A, B, C & D), for which IRIS was commissioned; and approximately 77,464 women aged 16 years and above (patients), registered at the 61 general practices, in an adjacent comparator north-east London borough site (E). The comparator borough has not implemented the studied intervention (IRIS) but instead used an alternative DVA initiative, during the time period examined.
Qualitative research is carried out in parallel, including a concurrent embedded, mixed-method process evaluation of IRIS implementation and two in-depth, local, IRIS case studies with a sustainability focus.
All results are integrated, by considering the quantitative and the qualitative results alongside each other, checking that results coincide (for example, are areas with the highest incidence rate ratios for referrals received also the areas in which IRIS has the greatest training reach) whilst reflecting on discordant results, in order to increase understanding about IRIS implementation, sustainability and effectiveness.
Setting
This implementation evaluation involves eight sites. Four of these are northeast London borough implementation sites (A, B, C & D) that commissioned IRIS within the study period. One is a comparator northeast London borough site (E) that did not commission IRIS but an alternative DVA initiative. One is an original IRIS trial intervention borough (F) that commissioned IRIS after the original trial. Additionally, an urban northern IRIS area (G) had an IRIS service started within the study period. The personnel involved included all staff at each general practice (clinical and administrative), DVA service providers’ staff and commissioners, including NHS clinical commissioning group staff (clinical leads for child/adult safeguarding and women’s health) and local council staff (concerned with local DVA strategy and public health).
Population/ participants
Sites are invited to take part in this research due to their geographical location in the North Thames area of London, adjoining the original IRIS intervention trial site with a priori knowledge that areas are interested in IRIS commissioning. One IRIS site outside of London is included as a qualitative case study, as it fulfilled pre-specified inclusion criteria of this work (see Appendix A). IRIS targets women affected by DVA, either currently or historically, from a partner, ex-partner or an adult family member. The eligibility criteria to be included in this observational study are: female patients aged 16 and above, registered at a general practice, at the sites being studied. Women affected by DVA are identified by a clinician and offered a referral to the named IRIS advocate-educator (AE); or women can self-refer to IRIS if they see the publicity material displayed within a surgery.
Intervention
IRIS is a general practice-based DVA training, support and referral programme for primary care staff. The theoretical framework of the training is based on educational outreach, adult learning theory and peer influence. It was developed using the MRC framework for complex interventions, to improve the primary care response to DVA. This involved steps for development, piloting and testing the intervention in a trial design followed by implementation in routine general practice [26].
The IRIS model consists of five core components:
-
1.
Practice based DVA training, with two two-hour sessions for the whole practice team.
-
2.
A local GP, interested in DVA, appointed as an IRIS clinical lead (CL) delivers clinical training alongside an IRIS AE.
-
3.
A named DVA specialist – an IRIS AE - employed and based in a local DVA service, delivers training and also receives referrals from clinicians, responsible for a caseload of work, providing on-going support, refresher training and consultancy for the entire practice team, on a day to day basis when in the practice (preferably attending regular quarterly practice meetings), by phone and email.
-
4.
The AE sees women affected by DVA, providing expert advocacy, including risk assessments, safety planning, emotional & mental health support, housing advice, referring, (e.g. to multi-agency risk assessment conferences (MARACs), child safeguarding services), support on injunctions & criminal justice system, signposting (e.g. to specialist DVA legal services) and accompanying to police or courts.
-
5.
An electronic prompt, reminds clinicians to ask about DVA, considering its multiple dimensions [28] and a template within which to record DVA (encouraging clinicians to (i) assess the immediate safety of the woman and any children, (ii) offer referral, (iii) review within general practice). The electronic prompt is triggered by codes for health conditions or symptoms associated with DVA, such as fatigue, insomnia, anxiety and depression.
The AE is an experienced specialist support DVA worker, with previous training experience. The CL is interested in DVA as a health issue, preferably also with training experience. Both the IRIS AEs and CLs have completed the national IRIS Training for Trainers programme, delivered by IRISi staff,Footnote 1 over three and two days respectively. The IRIS model is based on one full-time AE working with 25 general practices. The model is built upon partnership work, with primary care and specialist third sector agencies coming together to deliver services and promote work across the interdisciplinary gap. Further details about training (e.g. flowcharts showing simple referral pathways and various IRIS publicity materials to display in practices) are given in Additional file 1.
Comparator intervention
The alternative DVA intervention delivered in the comparator site, E, also comprised a DVA education package. This involved training delivered away from general practices, training was delivered by those from an advocacy background (not clinicians), clinicians’ referrals were sent to a One Stop Shop (not named advocates), with no electronic prompt embedded within the electronic medical record, encouraging clinicians to ask about DVA when clinically relevant. The comparator site was not chosen at random. It was invited to participate in this research because it had declined to commission IRIS, but had commissioned an alternative DVA education package.
Outcomes
Effectiveness outcomes
The primary outcome measure is the number of referrals received by DVA service providers from general practices. The denominator is the number of women aged 16 years and above within the practice. This is an evidence-based, intermediate outcome measure, on a causal pathway towards decreased DVA, and possibly better mental health and improved quality of life for women who are referred to specialist DVA services [12].
The secondary outcome measure is the number of women in whose medical records electronic DVA codes, representing DVA identification are used. The denominator is the number of women aged 16 years and above registered within each practice. Other pre-specified secondary outcome measures are the number of referrals received by MARACs from general practice, the type of contact and support offered to women by DVA advocates.
Process outcomes
Context for IRIS implementation, factors influencing IRIS implementation, local adaptations of IRIS and the reach of IRIS training is examined, at the northeast London IRIS sites - A, B, C, D and F.
Qualitative outcomes
Theoretically informed analysis of the features contributing to sustainable implementation of IRIS is conducted at sites B and G (see Appendix A).
Data sources, collection and management
Quantitative data
We use historic and prospective data routinely collected from multiple sources:
-
1.
DVA service providers: number of referrals received by their DVA workers, from general practices, with the date that each referral is received, the general practice from which it is received, the type of contact that is then made and the type of support required.
-
2.
IRIS AEs: record data in referral spreadsheets, with each referral received, the referral date, the practice from which it is received, the contact type and support type, including whether a referral is made to a MARAC. They also record the dates of all IRIS training delivery, the number of practice staff that attend and who specifically delivers the training. These paper based records, are electronically entered on to spread sheets and shared with the researchers either directly or via IRISi.
-
3.
General practices’ electronic medical records (EMR) for DVA codes used, each code with a date stamp and the general practice, at which it was used. EMR searches and data extraction are carried out remotely using the EMIS (Egton Medical Information Systems) web system located in the Centre for Primary Care and Public Health, Queen Mary, University of London.
-
4.
Electronic searches conducted centrally extract the number of women aged 16 years and above registered at each general practice and the Index of Multiple Deprivation score.
-
5.
Online anonymous questionnaire survey, incorporating an implementation measure based on Normalisation Process Theory [29] and administered via the Bristol Online Survey (BOS) tool (https://www.onlinesurveys.ac.uk/).
-
6.
A five-item checklist assesses the fidelity of IRIS, identifying IRIS’ local adaptations.
-
Is DVA training practice based?
-
Is a local GP, the IRIS CL, delivering training alongside the AE?
-
Is the IRIS AE, based in a local DVA service?
-
Is the IRIS AE delivering training & receiving referrals?
-
Is the electronic prompt & recording template, been activated?
-
7.
Practice websites for the total number of practice staff employed. All contained a list of practice staff.
All extracted data are entered into the study database for statistical analysis.
Qualitative data
Qualitative data are derived from the free text comments generated by the online anonymous questionnaire survey (see above), participant observation, document analysis and interviews – semi-structured, using purposive samples of general practice staff (clinical and non-clinical), AEs, local stakeholders, including commissioners and IRIS service users. This work describes and understands rationale for any features unique to a particular site, capturing the views of professionals involved in implementing IRIS and patients who have been referred into the IRIS service.
Data analysis
Quantitative data analysis
Sample size
The sample size was determined by simulation, using the SimSam package in Stata [30]. Simulations were of an analysis of monthly counts with a Poisson distribution. We assumed a typical practice had an average referral rate at baseline of 4.5 per 100,000 per month, based on data from the IRIS cluster randomised trial [9], with this rate varying between practices with a 95% normal range of 1.7 to 12.2 per 100,000 per month. We further assumed there were 3000 eligible women per practice, and 180 practices. We determined the number of months of data required to detect a doubling of the referral rate following the introduction of the intervention, with 90% power at the 5% significance level, assuming that different practices introduced the intervention at different times, uniformly over this period. Simulation showed that 17 months of data were required. In order to capture any seasonality in the referral rates we increased this to 24 months.
We use an interrupted time series segmented regression approach with a mixed effects Poisson regression model, to examine the effect of the IRIS intervention on the primary outcome measure (the number of referrals received by DVA service providers from general practices); and the secondary outcome measure (the number of first times that DVA identification electronic codes are used in medical notes by clinicians). We compare these outcome measures before and after the date of the first IRIS training session, taken from individual general practices (i.e. practice level data), from multiple general practices. This date of the first IRIS training session is taken as the date of IRIS implementation.
Figure 1 depicts an interrupted time series segmented regression approach. Using this approach it is possible to examine whether there is a change in level immediately and/or a change in regression slope following the implementation of the intervention, i.e. a change in the gradient of the slope pre- and post-intervention.
For each practice the outcomes are defined as a daily frequency. The observations occurring on the day that the training is received are excluded, as it is not possible to say whether these occurred before or after training. Graphical displays of the data show average daily rates (per 1000 people) plotted against time centered on IRIS implementation. A moving average smoothing function, with equal weights, is applied to the data, with an unadjusted line of best fit added, before and after the training.
The ITS model includes a random effect of practice and fixed effects of training (pre or post in any given day in any given practice), the slope of the underlying time trend, the change in slope following the training, site (five London boroughs), and month and day of the week to allow for any seasonal effect of time. If for either outcome measure there is evidence of over-dispersion of the daily frequencies compared with a Poisson distribution then we model this with a random effect of day nested within the random effect of practice. We also adjust for the log-transformed number of women aged 16 years and above registered at that practice, every quarter, as an offset variable in the analysis. This allows us to control for the available population size at any one time. The analysis includes only those general practices that received IRIS training. We exclude practices where 50% or more of the list size data are missing, meaning we do not know the number of women aged 16 years and above registered at the practice. Regression coefficients are presented as incidence rate ratios. Data is analysed using the Stata V14 package (StataCorp LP, College Station, TX).
Sub-group analyses
A priori additional analyses include comparing the different implementation sites (A, B, C & D) to each other and the comparator site (E). The ITS analysis is run separately within each site. A forest plot is used to compare these individual analyses. Deprivation scores are used to check that deprivation does not confound the results.
Process evaluation data analysis
Survey data are exported from BOS to Stata V14 and analysed using descriptive statistics (means, standard deviations and frequencies) for the quantitative data.
Qualitative data analysis
Interview data are transcribed verbatim and coded thematically, using a mixture of inductive and deductive approaches. Theoretically informed analysis is conducted. Significant patterns are identified and the data examined for deviant cases.