Table 3 Published descriptions and properties of unique performance indicators for public mental health care
From: Performance indicators for public mental healthcare: a systematic international inventory
Study | Performance indicator | Related criterion | Result |
|---|---|---|---|
Simpson & Lloyd [17] | • Methadone maintenance (MM) drug abuse treatment client evaluation score (composite measure from 7 items) • Therapeutic communities (TC) drug abuse treatment client evaluation score (composite measure from 7 items) • Out-patient drug-free (DF) drug abuse treatment client evaluation score (composite measure from 7 items) • outpatient detoxification (DT) drug abuse treatment client evaluation score (composite measure from 7 items) | 1 year post treatment (high scores more favorable): • opoid use • nonopoid use • marijuana use • alcohol use • employment • jail • return to treatment within 1 year • composite score (all above mentioned criteria) | • More positive MM treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, return to treatment, and the composite score. • More positive TC treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, employment, jail, and the composite score. • More positive DF treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, and the composite score. • More positive DT treatment client evaluation was sig. pos. related to 1-year post treatment return to treatment |
Srebnik et al. [22] | • Satisfaction domain: client satisfaction questionnaire; involvement in treatment; treatment appropriateness; safety at mental health center • Functioning domain: physical; mental; social and leisure; skills for handling stress and symptoms • Quality of life domain: safety; concerns about living condition; goal attainment; victimization • Clinical status domain: four-dimensional classification scale | • Percentage of clients with any meaningful activity • Percentage of clients with a independent living situation • Percentage of clients with no out-of-community (hospital/jail admission) episode | • No sig. associations of satisfaction measures with any of the criteria • Sig. pos. association of functioning measures with living situation • Sig. neg. association of functioning measures with out-of-community episode • Sig. pos. association of quality of life measures with meaningful activity • Sig. pos. association of clinical status measure with meaningful activity, living situation, and out-of-community episode |
Druss et al. [30] | • Promptness and continuity of outpatient follow-up after discharge • Any outpatient follow-up after discharge • Length of stay • Readmission within 30 days • Readmission within in 180 days and total days readmitted within 180 days | Individual level and hospital level measures of satisfaction with: • General service delivery • Alliance with inpatient staff | • Sig. pos. association of promptness and continuity of outpatient follow-up and alliance with inpatient staff at individual level and at hospital level • Sig. pos. association of any outpatient follow-up and alliance with inpatient staff at individual level • Sig. pos. association of length of stay and alliance with inpatient staff at individual level • No sig. association of early readmission with any of the criteria • Sig. neg. association of readmission intensity and general service delivery at individual level |
Macias et al. [28] | • International Center for Clubhouse Development Certification status | • 3 organizational resource variables • 7 survey variables reflective of clubhouse model fidelity | • No sig. association of resource variables and clubhouse certification status • Sig. pos. association with 6 of the 7 fidelity variables. |
Huff [32] | • Crisis service utilization within 30 days after discharge • Median index episode length of stay • Median number of service contacts within 30 days after discharge • Number of providers contact within 30 days after discharge • Diagnostic evaluation services within 30 days after discharge • Early ambulatory contact (within 5 days) • Medication management service within 30 days after discharge • Psychotherapy service within 30 days after discharge | • Readmission for an acute episode of care to any acute mental health provider within a 30-day period after being discharged | • Sig. pos. association of crisis service utilization and 30-day acute relapse risk • No sig. association of length of stay and 30-day acute relapse risk • Sig. pos. association of service contacts and 30-day acute relapse risk • Sig. pos. association of provider contacts and 30-day acute relapse risk • Sig. neg. association of diagnostic evaluation services and 30-day acute relapse risk • Sig. neg. association early ambulatory contact and 30-day acute relapse risk • Sig. neg. association of medication management services and 30-day acute relapse risk • Sig. neg. association of psychotherapy service and 30-day acute relapse risk |
Shipley et al. [91] | • Mean patient satisfaction score (4-item questionnaire) • Mean clinician satisfaction score (4-item questionnaire) • Mean referrer satisfaction score (4-item questionnaire) • Mean time form referral to first appointment • Proportion of patients in which referrer is notified of contact within 6 weeks of referral • Proportion of patient referred and offered appointments who attended | • Five clinical teams of a psychiatric service, one of which was regarded as seriously deficient by an independent review. All teams had important differences in type of referrals and diagnostic group | • Sig. differences in patient satisfaction between teams with which the poorly performing team could be identified • No sig. differences in clinician satisfaction between teams • No sig. differences in referrer satisfaction between teams • Sig. differences in time to first appointment PI that failed to identify deficient team. • Sig. differences referrer notification that failed to identify deficient team. • No sig. differences of patient attending appointments PI between teams. |
Chinman et al. [41] | • Service-need index, incorporating ratings of drug and alcohol use, patient's average acuity score, and GAF-m score | • Average caseload of treatment team or program. • Average number of outpatient service hours provided to patients | • Sig. neg. correlation between service-need index and caseload • Sig. pos. correlation between service-need index and service hours provided |
Dausey et al. [44] | • Preadmission care (binary variable, continuous variable, spline variable) | • Duration of index admission (length of stay). • Use of post discharge aftercare at 30 days. • Readmission at 14, 30 and 180 days. | • Sig. neg. association between preadmission care and length of stay. • Sig. pos. association between preadmission care and aftercare at 30 days • Preadmission care is associated with a slight increase in probability of readmission |
• Percentage of funds allocated to state hospitals to community-based services is at the theoretical optimum proportions (43% to 57%) | • Suicide rate per 100,000 population • Cost per capita for mental health care | • Suicide rate lower in states in which funds allocation proportion are close to the optimum. • Relation between expenditure and cost per capita is only found when states that differ more than 12% from the ideal funding partition are excluded | |
Beaulieu et al. [50] | • NPHPSP Local public health system performance measurement instrument | • Documentary evidence • External judge rating of performance | • Documentation to support agencies' responses to the local instrument validated their responses • External judge ratings were unreliable due to lack of knowledge of local systems |
Edlund et al. [59] | • Percentage of persons with any alcohol, drug, or mental disorder that received at least 4 visits with a mental health specialist or 4 visits with a primary care provider that included counseling for mental health problems and/or that received medication that was efficacious for the individual's disorder and used at a dosage exceeding the minimum recommended dosage for an adequate duration • Active treatment after assessment: use of inpatient, day treatment, or residential care; use of prescribed psychotropic medications daily for a month or more; or a period of potentially therapeutic outpatient treatment for alcohol, drug or mental conditions | • Overall satisfaction with the mental health care available for personal or emotional problems during the past 12 months | • Sig. pos. association of appropriate counseling/appropriate pharmacotherapy, and satisfaction with available mental health services • Sig. pos. association of active treatment, and satisfaction with available mental health services |
Charbonneau et al. [62] | • Dosage adequacy: antidepressant average daily dosage during 3-month profiling period meets guideline-recommended minimum daily dosage • Duration adequacy: inadequate duration defined as > 21% of the profiling period without antidepressants • Follow-up visit adequacy: at least 3 visits to primary care or psychiatry clinics within 3 months of the initial depression encounter; at least 2 visits in addition to the initial one within 3 months of diagnosis | • Inpatient overall, and psychiatric hospitalizations during the 12 months after the depression care period | • No sig. association between dosage adequacy and any criteria • Sig. neg. association between duration adequacy and subsequent overall or psychiatric hospitalizations • No sig. association between follow-up visit adequacy and any criteria |
Druss et al. [64] | • Medication during at least 3 follow-up mental health care visits in the 3 months after a new depressive episode • Ongoing medication treatment in the 3-month period after a new depressive episode • Ongoing medication treatment in the 6 months after a new depressive episode • Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 30 days of hospital discharge • Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 7 days of hospital discharge | • Volume of ambulatory mental health use • Volume of inpatient mental health discharges Volume of inpatient mental health days. | • Sig. pos. association between volume of ambulatory mental health use, and medication management PI, and outpatient follow-up PI • Sig. pos. association of volume of inpatient mental health discharges, and medication management PI, and outpatient follow-up PI • Sig. pos. association of volume of inpatient mental health, and medication management PI, and outpatient follow-up PI |
Leff et al. [66] | • Service fit: the congruence between services prescribed or needed and services received | • Mortality: natural deaths, medico-legal deaths, suicides | • The relationship between service fit and mortality is more apparent in models based on medico-legal deaths and suicides than in the model based on natural deaths |
Nieuwen-huijsen et al. [110] | • Assessment of symptoms (2 criteria), one of both criteria not met within 2 consultations. • Correct diagnosis (3 criteria), one of more criteria not met within 2 consultations. • Evaluation curative care (2 criteria), one of both criteria not met within 2 consultations. • Assessment of work-related causes (2 criteria), one of both criteria not met within 2 consultations. • Evaluation of work disabilities (2 criteria), one of both criteria not met within 2 consultations. • Interventions targeted at the individual (1 criterion), criterion not met within 3 consultations. • Interventions targeted at organization (1 criterion), criterion not met within 3 consultations. • Interventions targeted at providers of care in curative sector (2 criteria), one or both criteria not met within 3 consultations. • Advice on return to work (2 criteria), one or both criteria not met at each consultation. • Timing of consultations (2 criteria), criterion 1 not met at first consultation or criterion 2 not met at consultation 2 or 3. • Summed score over 9 indicators with sufficient content validity and variability. | • Time to return to work • Change in level of fatigue • Patient satisfaction | • No sig. association assessment of symptoms and any criteria • No sig. association correct diagnosis and any criteria • Sig. pos. association evaluation curative care and satisfaction. • No sig. association assessment of work-related causes and any criteria • Sig. neg. association evaluation work disabilities and return to work. • No sig. association organizational interventions and any criteria • Sig. pos. association interventions curative sector and return to work. • No sig. association advice to return to work and any criteria • Sig. neg. association timing of consultations and return to work. • Sig. pos. association overall quality of care and return to work, and satisfaction. |
Rost et al. [70] | • A prescription for an antidepressant medication was noted from up to 30 days before to 14 days after index episode start date; dosage sufficient to take medication for 84 out of 114 days following first prescription; 3 non-emergency visits room visits to a primary care or mental health provider at least one of them had to be with the prescribing provider • 4 or more specialty depression care counseling visits in the 6 months following the index visit | • Absenteeism: lost work hours in the past 4 weeks due to illness or doctor visits | • No sig. association appropriate medication and change in absenteeism over 1 year • Sig. association appropriate psychotherapy and change in absenteeism over 1 year |
Wierdsma et al. [111] | • Clients receive any psychiatric care in the year before involuntary admission | • Length of stay (less than 3 weeks; more than 6 months; mean number of days) • Ambulatory follow-up • Readmission (within 3 months; within 1 year) • Continued care 12 months after involuntary admission | • Sig. pos. association preadmission care and length of stay, and continued care after 12 months. • No sig. association preadmission care and ambulatory follow-up, and readmission |