Study design
The research design for the Moncton site was a parallel-group non-blinded RCT with participants randomly assigned after the baseline interview to receive HF with ACT or TAU. In the four other cities of the AHCS trial, only high need participants received ACT, with moderate need participants receiving ICM. However, at the Moncton site, the relatively small homeless population resulted in only one HF program being tested; this program provided ACT to all participants, regardless of need level. Ethical approval for the study was received from the IRBs of the Université de Moncton and the University of Ottawa and the two Regional Health Authorities (i.e., Horizon and Vitalité) involved with the project. Participants provided signed informed consent before participating in the screening process and before the first interview. None of the authors have any competing interests.
Participants
Participants were recruited through referrals from personnel located in community agencies and institutions, which included shelters, drop-in centers, correctional services, and hospitals who were informed about the study and the eligibility criteria for participation. Eligibility criteria for the study were the following: (1) Legal adult status (age 18 or older); (2) housing status of either absolutely homeless (i.e., lacking a regular fixed physical shelter) or precariously housed (living in a rooming house, single-room occupancy unit, or hotel or motel room and having had two episodes or more of homelessness in the past year); (3) the presence of a serious mental disorder (major depressive, manic or hypomanic episode, post-traumatic stress disorder, mood disorder with psychotic features, psychotic disorder) as determined by DSM-IV criteria on the Mini-International Neuropsychiatric Interview (MINI) [27] at the time of study entry; (4) Canadian citizen, landed immigrant, or refugee claimant; and (5) not receiving ACT or ICM at study entry.
Participants were considered to have a high level of need if they were assessed on the MINI as having a diagnosis of a bipolar disorder or psychotic disorder, scored less than 62 on the Multnomah Community Ability Scale (MCAS) [28] and met at least one of the following criteria: (1) Had been hospitalized twice in any one-year period in the past 5 years, (2) had substance abuse or dependence as assessed on the MINI, or (3) had been arrested or incarcerated in the past 6 months. All other participants were considered to have moderate needs.
Randomization
Randomization was performed by a central data collection system that used an adaptive randomization algorithm to produce equally-sized groups that were balanced on need level. Group assignment was revealed to participants at the end of the first interview. The differences in services received by the two groups precluded blinding, given that data-collection interviews were concerned in part with housing and receipt of services.
Interventions
Housing first
As detailed in the program logic model presented in Table 1, the HF program was based on the Pathways HF approach [6]. Participants received a rent supplement that ensured that they would not contribute more than 30% of their income toward their rent. The housing accessed by HF recipients at the Moncton site was exclusively private market scattered-site rental units. A Housing Coordinator with the program assisted participants in choosing among available units as well as furnishing and moving into them. The Housing Coordinator was also available to mediate if participants encountered difficulties with landlords or neighbors.
In line with the Pathways approach, there were no pre-conditions related to participation in treatment or abstinence from substance use in order to qualify for the HF services. Participants were expected to follow the terms of the lease and make themselves available for one visit per week from program staff. Individualized support was provided to participants by members of a multidisciplinary ACT team. The ACT team operated with a consumer to staff ratio of 10:1, providing services to 100 study participants in the Moncton area. Staff members of the ACT team comprised of a variety of professional disciplines that included a nurse practitioner, psychiatric nurses, an occupational health therapist, a home economist, a social worker, human resource counsellors, a family physician who served as the clinical director, and consulting psychiatrists. The team also added in the later stages of the project two part-time peer support workers who were individuals with lived experience of mental illness and addictions. Additionally, there was an Administrative Manager for the team with training in psychiatric rehabilitation who was available to deliver clinical services to consumers as needed.
An external team made up of Pathways HF staff, the lead researcher of the AHCS trial, and another expert in HF conducted two fidelity assessments, one approximately 10 months after the program began [29] and another when the program was 27 months old [30]. The fidelity assessment entailed visits from three external individuals with expertise in HF who assessed the program on the validated HF with ACT fidelity scale based on observations, interviews with program managers, and focus groups with staff and program participants [29, 30]. Both fidelity assessments showed the program as having good fidelity overall, with 86% of the 37 assessed items rating a score of 3 or more on a 4-point scale in the second fidelity assessment [30].
Treatment as usual
Study participants assigned to the TAU group could access all other existing health and social services in the tri-city area. HF services were not available prior to the demonstration project. The region did have some transitional, long-term supportive housing, and non-profit congregate housing as well as social housing units available [24]. A costing study showed that participants in the TAU group consumed on average only a relatively small amount of services from supportive housing programs [23]. Available publicly-funded mental health services were limited to outpatient and day program services in community health centres and hospitals, and psychiatrists in private practice [24]. At the time of the study, there were no ACT or ICM teams delivering services in Moncton.
Measures
The predicted primary outcomes were housing stability over the study period (i.e., time to first move into stable housing, number of moves, percentage of days housed, perceived housing quality, and being housed consecutively for 6 months or more at the final interview) and community functioning (i.e., level of ability to live independently in the community). The predicted secondary outcomes were self-rated physical and mental health status, substance use, quality of life, community integration, and recovery. The measures for these outcomes were carefully selected based on their history of being used with the population and adequacy of their psychometric properties.
Primary outcomes
Housing stability
The Residential Time-Line Follow-Back Inventory (RTLFB) provides a point-in-time assessment as well as retrospective longitudinal information on housing [31]. Using a calendar, respondents are asked to provide details of their housing for a specific retrospective time period. Specifically, they are asked about the number of moves, the reason for the moves, the type of residence or organization lived in, and the composition of the household. The RTLFB has been shown to have high test-retest reliability and good concurrent validity [31]. In the case of the current study, the RTLFB was administered the first time at the 3-month follow-up asking participants about their housing in the past 6 months in order to determine their housing history just prior to entry into the study. Subsequently, it was administered every 3 months with participants queried about their previous 3 months.
Perceived housing quality
The five-item Perceived Housing Quality (PHQL) scale was administered to housed respondents starting at the 6 month follow-up interview [32]. Respondents were asked about the safety, spaciousness, privacy, friendliness, and overall quality of their housing. For example, an item asked “How would you rate your current home for safety?” Response alternatives ranged from 1 (very bad) to 5 (very good). A summed score on the five items was calculated that can range from 5 to 25. Previous research has shown the measure to have high test-retest reliability [32] and good internal reliability [33]. Cronbach’s alpha for the measure in the current study was 0.88 (6 months), 0.84 (12 months), 0.80 (18 months), and 0.80 (24 months).
Community functioning
The 17-item Multnomah Community Ability Scale (MCAS) was used to measure the level of community functioning of participants [28]. The MCAS measures different aspects of community functioning, notably problems in daily functioning as a result of physical and mental and emotional symptoms, ability to cope with mental illness, level of ability to interact with others, and the presence of any behavioural problems that negatively affect living in the community. Based on the information collected in the interview and in response to relevant questions, ratings were completed on each item by interviewers subsequent to each in-person interview. The MCAS has been shown to have good inter-rater reliability, test-retest reliability, and criterion validity (i.e., predictive of hospital admissions for psychiatric problems) [28]. Ratings on the items ranged from no impairment (1) to extreme impairment (5). The total scale score can range from 17 to 85. Cronbach’s alpha for the MCAS in the current study was 0.77 (baseline), 0.84 (6 months), 0.80 (12 months), 0.82 (18 months), and 0.83 (24 months).
Secondary outcomes
Self-rated health status
Perceived health status was assessed by the EQ-5D, a self-administered 5-item measure that enquires about mobility, self-care, usual activities, pain/discomfort, and anxiety/depression [34]. Each item has three possible responses reflecting no problems, some problems, or extreme problems. A standardized score is calculated across the items that can range from 0 to 2. The EQ-5 has been shown to have good discriminant validity and sensitivity to change in health status [34]. Cronbach’s alpha for the EQ-5D n the current study was 0.70 (baseline), 0.75 (6 months), 0.70 (12 months), 0.72 (18 months), and 0.76 (24 months).
Severity of mental health symptoms
The 14-item Colorado Symptom Index (CSI) was used to measure the severity of mental health symptoms [35]. This self-report measure queries the presence and frequency of mental illness symptoms experienced in the past month. Response alternatives can range from 0 (not at all) to 4 (at least every day) with a possible summed score of 0 to 56. Higher scores reflect a greater severity of symptoms. Research has shown the CSI to have good test-retest reliability, excellent internal consistency, “fair” to “good” discrimination of individuals with psychiatric disabilities, and good convergent and concurrent validity [35]. Cronbach’s alpha for the CSI in the current study was 0.83 (baseline), 0.88 (6 months), 0.87 (12 months), 0.88 (18 months), and 0.88 (24 months).
Severity of substance use problems
The 7-item Global Assessment of Individual Need – Substance Problem Scale (GAIN-SPS) was used to measure substance use severity [36]. Participants were queried about when substance-related problems last occurred. The severity of substance use problems was calculated by summing the number of substance-related problems reported in the past month (range 0–5). The GAIN-SPS has been found to have excellent internal consistency, good concurrent validity with objective measures of substance use, and good discriminant validity for detecting the presence of a substance use diagnosis [36]. Cronbach’s alpha for the GAIN-SPS at baseline in the current study was 0.82 (baseline), 0.88 (6 months), 0.87 (12 months), 0.87 (18 months), and 0.88 (24 months).
Quality of life
Subjective quality of life was measured using the 20-item Quality of Life Interview (QoLI-20) [37]. Items ask participants about their level of satisfaction with different aspects of their lives. Responses are made on a 7-point Likert scale that ranges from terrible (1) to delighted (7). The measure includes a total summed score ranging from 20 to 140, and six subscales measuring satisfaction with living situation, finances, leisure, family, social relations, and safety. As well, one item queries overall satisfaction. Higher values correspond to greater quality of life. Internal reliability for the total score and subscales have been shown to be adequate to very good [37]. Cronbach’s alpha for the QoLI-20 in the current study was 0.89 (baseline), 0.91 (6 months), 0.91 (12 months), 0.92 (18 months), and 0.92 (24 months).
Physical integration
A 7-item version of a measure of external integration [38] was used to measure level of physical integration. The items asked participants if they had participated in different activities in the community in the past month, such as eating in a restaurant, visiting a library, and participating in a volunteer activity. Possible responses were yes (1) or no (0). The total summed score on the scale can range from 0 to 7 with higher scores representing relatively higher levels of physical integration. Cronbach’s alpha for the measure in the current study was 0.57 (baseline), 0.57 (6 months), 0.63 (12 months), 0.49 (18 months), and 0.57 (24 months).
Psychological integration
A four-item version of a psychological integration measure [39] was used to measure level of community integration. Items asked participants about their sense of belonging associated with where they lived. Response alternatives range from strongly disagree (1) to strongly agree (5) with possible total summed scores ranging from 4 to 20. A higher summed score corresponded to a higher level of psychological integration. Previous research has found the measure to have acceptable internal reliability [39]. Cronbach’s alpha for the measure in the current study was 0.74 (baseline), 0.73 (6 months), 0.75 (12 months), 0.67 (18 months), and 0.66 (24 months).
Recovery
The 22-item Recovery Assessment Scale (RAS) measured the perceived level of recovery from having a severe and persistent mental illness [40]. It was administered at baseline and again at 24 months. Previous research has shown the RAS to be made up of several factors that focus on personal confidence and hope, a willingness to ask for help, goal-setting and future orientation, reliance on others, and no domination by symptoms [40]. Response alternatives vary from strongly disagree (1) to strongly agree (5). Higher summed scores across the 22 items reflected greater recovery. Possible scores can range from 22 to 110. The RAS has been shown to have excellent internal reliability, high test-retest reliability, and concurrent validity with related measures [40]. Cronbach’s alpha for the RAS in the current study was 0.91 (baseline) and 0.92 (24 months).
Sample size calculation
The targeted sample size was set at 100 individuals per group. Power analysis determined that a minimum of 63 per group would be sufficient to detect a moderate effect size (ES = 0.50) with α = 0.05 and β = 0.20, allowing for a 35% attrition [15].
Data collection
In order to maintain contact with participants and collect data on housing history and vocational activities, short interviews were conducted mostly by telephone at 3, 9, 15, and 21 months. Longer interviews were conducted in person and included collecting data on all of the other measures with the exception of the recovery measure at baseline, 6, 12, 18, and 21 or 24 months. In the case of the recovery measure, it was administered at baseline and at the last follow-up interview at 21 or 24 months. The original protocol planned a 24 month follow-up interview. However, the final interviews for a small number of participants (38% of participants who completed the study and 30% of the total sample) were completed at 21 months because of time and resource constraints. At the time of enrolment, participants were invited to provide contact information for themselves and for individuals in their social network who would be likely to know where they are located in the future. As well, participants were asked to give consent for the social services department that administers benefit payments to provide study researchers with their address during the study. These contacts were updated as needed at each follow-up interview.
Statistical analysis
An intent-to-treat analysis was conducted. Group- and time-specific means for all outcomes were calculated, with 95% confidence intervals produced by bootstrapping with 2000 replications. Linear regression models were used to analyze time to housing and length of stay in housing. For other outcomes, mixed-effects models were used. Linear models were fitted for continuous outcomes, logistic models for binary outcomes, and negative binomial models for counts of events. In all cases sex, age, and level of need (i.e., moderate or high) were entered as covariates.
Mixed effects models were random-intercept models. Time was treated as a categorical variable by dummy-coding events and interacting each time point with group membership, producing an expected mean and an expected group difference at each time point. Two measures of group differences were produced: differences between groups at the final time point, and the average difference over all the follow-up events (representing differences over the study period as a whole).
Secondary analyses examined group*need level interactions. Due to the large number of interaction terms that would otherwise be necessary, we fit simplified mixed effects models in which outcomes were regressed on their baseline level, group, need, group*need, and covariates. In these models, coefficients for group*need interactions represent overall intervention effect differences over the follow-up period as a whole.
In order to examine for any difficulties associated with missing data, a sensitivity analysis was conducted with a set of 40 imputed data sets for community functioning (i.e., MCAS) developed with sequential regression multivariate imputation. This analysis produced an effect very similar to that of the original model, and no meaningful differences in covariates (e.g., in terms of significance or non-significance). For housing, which was the other primary outcome, the large effects and the relatively small amount of missing data essentially rule out any effects of missing data on inferences.