Skip to main content

Table 3 Published descriptions and properties of unique performance indicators for public mental health care

From: Performance indicators for public mental healthcare: a systematic international inventory

Study Performance indicator Related criterion Result
Simpson & Lloyd [17] • Methadone maintenance (MM) drug abuse treatment client evaluation score (composite measure from 7 items)
• Therapeutic communities (TC) drug abuse treatment client evaluation score (composite measure from 7 items)
• Out-patient drug-free (DF) drug abuse treatment client evaluation score (composite measure from 7 items)
• outpatient detoxification (DT) drug abuse treatment client evaluation score (composite measure from 7 items)
1 year post treatment (high scores more favorable):
• opoid use
• nonopoid use
• marijuana use
• alcohol use
• employment
• jail
• return to treatment within 1 year
• composite score (all above mentioned criteria)
• More positive MM treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, return to treatment, and the composite score.
• More positive TC treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, employment, jail, and the composite score.
• More positive DF treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, and the composite score.
• More positive DT treatment client evaluation was sig. pos. related to 1-year post treatment return to treatment
Srebnik et al. [22] • Satisfaction domain: client satisfaction questionnaire; involvement in treatment; treatment appropriateness; safety at mental health center
• Functioning domain: physical; mental; social and leisure; skills for handling stress and symptoms
• Quality of life domain: safety; concerns about living condition; goal attainment; victimization
• Clinical status domain: four-dimensional classification scale
• Percentage of clients with any meaningful activity
• Percentage of clients with a independent living situation
• Percentage of clients with no out-of-community (hospital/jail admission) episode
• No sig. associations of satisfaction measures with any of the criteria
• Sig. pos. association of functioning measures with living situation
• Sig. neg. association of functioning measures with out-of-community episode
• Sig. pos. association of quality of life measures with meaningful activity
• Sig. pos. association of clinical status measure with meaningful activity, living situation, and out-of-community episode
Druss et al. [30] • Promptness and continuity of outpatient follow-up after discharge
• Any outpatient follow-up after discharge
• Length of stay
• Readmission within 30 days
• Readmission within in 180 days and total days readmitted within 180 days
Individual level and hospital level measures of satisfaction with:
• General service delivery
• Alliance with inpatient staff
• Sig. pos. association of promptness and continuity of outpatient follow-up and alliance with inpatient staff at individual level and at hospital level
• Sig. pos. association of any outpatient follow-up and alliance with inpatient staff at individual level
• Sig. pos. association of length of stay and alliance with inpatient staff at individual level
• No sig. association of early readmission with any of the criteria
• Sig. neg. association of readmission intensity and general service delivery at individual level
Macias et al. [28] • International Center for Clubhouse Development Certification status • 3 organizational resource variables
• 7 survey variables reflective of clubhouse model fidelity
• No sig. association of resource variables and clubhouse certification status
• Sig. pos. association with 6 of the 7 fidelity variables.
Huff [32] • Crisis service utilization within 30 days after discharge
• Median index episode length of stay
• Median number of service contacts within 30 days after discharge
• Number of providers contact within 30 days after discharge
• Diagnostic evaluation services within 30 days after discharge
• Early ambulatory contact (within 5 days)
• Medication management service within 30 days after discharge
• Psychotherapy service within 30 days after discharge
• Readmission for an acute episode of care to any acute mental health provider within a 30-day period after being discharged • Sig. pos. association of crisis service utilization and 30-day acute relapse risk
• No sig. association of length of stay and 30-day acute relapse risk
• Sig. pos. association of service contacts and 30-day acute relapse risk
• Sig. pos. association of provider contacts and 30-day acute relapse risk
• Sig. neg. association of diagnostic evaluation services and 30-day acute relapse risk
• Sig. neg. association early ambulatory contact and 30-day acute relapse risk
• Sig. neg. association of medication management services and 30-day acute relapse risk
• Sig. neg. association of psychotherapy service and 30-day acute relapse risk
Shipley et al. [91] • Mean patient satisfaction score (4-item questionnaire)
• Mean clinician satisfaction score (4-item questionnaire)
• Mean referrer satisfaction score (4-item questionnaire)
• Mean time form referral to first appointment
• Proportion of patients in which referrer is notified of contact within 6 weeks of referral
• Proportion of patient referred and offered appointments who attended
• Five clinical teams of a psychiatric service, one of which was regarded as seriously deficient by an independent review. All teams had important differences in type of referrals and diagnostic group • Sig. differences in patient satisfaction between teams with which the poorly performing team could be identified
• No sig. differences in clinician satisfaction between teams
• No sig. differences in referrer satisfaction between teams
• Sig. differences in time to first appointment PI that failed to identify deficient team.
• Sig. differences referrer notification that failed to identify deficient team.
• No sig. differences of patient attending appointments PI between teams.
Chinman et al. [41] • Service-need index, incorporating ratings of drug and alcohol use, patient's average acuity score, and GAF-m score • Average caseload of treatment team or program. • Average number of outpatient service hours provided to patients • Sig. neg. correlation between service-need index and caseload • Sig. pos. correlation between service-need index and service hours provided
Dausey et al. [44] • Preadmission care (binary variable, continuous variable, spline variable) • Duration of index admission (length of stay).
• Use of post discharge aftercare at 30 days.
• Readmission at 14, 30 and 180 days.
• Sig. neg. association between preadmission care and length of stay.
• Sig. pos. association between preadmission care and aftercare at 30 days
• Preadmission care is associated with a slight increase in probability of readmission
Davis & Lowell [42, 43] • Percentage of funds allocated to state hospitals to community-based services is at the theoretical optimum proportions (43% to 57%) • Suicide rate per 100,000 population
• Cost per capita for mental health care
• Suicide rate lower in states in which funds allocation proportion are close to the optimum.
• Relation between expenditure and cost per capita is only found when states that differ more than 12% from the ideal funding partition are excluded
Beaulieu et al. [50] • NPHPSP Local public health system performance measurement instrument • Documentary evidence
• External judge rating of performance
• Documentation to support agencies' responses to the local instrument validated their responses
• External judge ratings were unreliable due to lack of knowledge of local systems
Edlund et al. [59] • Percentage of persons with any alcohol, drug, or mental disorder that received at least 4 visits with a mental health specialist or 4 visits with a primary care provider that included counseling for mental health problems and/or that received medication that was efficacious for the individual's disorder and used at a dosage exceeding the minimum recommended dosage for an adequate duration • Active treatment after assessment: use of inpatient, day treatment, or residential care; use of prescribed psychotropic medications daily for a month or more; or a period of potentially therapeutic outpatient treatment for alcohol, drug or mental conditions • Overall satisfaction with the mental health care available for personal or emotional problems during the past 12 months • Sig. pos. association of appropriate counseling/appropriate pharmacotherapy, and satisfaction with available mental health services
• Sig. pos. association of active treatment, and satisfaction with available mental health services
Charbonneau et al. [62] • Dosage adequacy: antidepressant average daily dosage during 3-month profiling period meets guideline-recommended minimum daily dosage
• Duration adequacy: inadequate duration defined as > 21% of the profiling period without antidepressants • Follow-up visit adequacy: at least 3 visits to primary care or psychiatry clinics within 3 months of the initial depression encounter; at least 2 visits in addition to the initial one within 3 months of diagnosis
• Inpatient overall, and psychiatric hospitalizations during the 12 months after the depression care period • No sig. association between dosage adequacy and any criteria
• Sig. neg. association between duration adequacy and subsequent overall or psychiatric hospitalizations
• No sig. association between follow-up visit adequacy and any criteria
Druss et al. [64] • Medication during at least 3 follow-up mental health care visits in the 3 months after a new depressive episode
• Ongoing medication treatment in the 3-month period after a new depressive episode
• Ongoing medication treatment in the 6 months after a new depressive episode
• Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 30 days of hospital discharge
• Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 7 days of hospital discharge
• Volume of ambulatory mental health use
• Volume of inpatient mental health discharges Volume of inpatient mental health days.
• Sig. pos. association between volume of ambulatory mental health use, and medication management PI, and outpatient follow-up PI
• Sig. pos. association of volume of inpatient mental health discharges, and medication management PI, and outpatient follow-up PI
• Sig. pos. association of volume of inpatient mental health, and medication management PI, and outpatient follow-up PI
Leff et al. [66] • Service fit: the congruence between services prescribed or needed and services received • Mortality: natural deaths, medico-legal deaths, suicides • The relationship between service fit and mortality is more apparent in models based on medico-legal deaths and suicides than in the model based on natural deaths
Nieuwen-huijsen et al. [110] • Assessment of symptoms (2 criteria), one of both criteria not met within 2 consultations.
• Correct diagnosis (3 criteria), one of more criteria not met within 2 consultations.
• Evaluation curative care (2 criteria), one of both criteria not met within 2 consultations.
• Assessment of work-related causes (2 criteria), one of both criteria not met within 2 consultations.
• Evaluation of work disabilities (2 criteria), one of both criteria not met within 2 consultations.
• Interventions targeted at the individual (1 criterion), criterion not met within 3 consultations.
• Interventions targeted at organization (1 criterion), criterion not met within 3 consultations.
• Interventions targeted at providers of care in curative sector (2 criteria), one or both criteria not met within 3 consultations.
• Advice on return to work (2 criteria), one or both criteria not met at each consultation.
• Timing of consultations (2 criteria), criterion 1 not met at first consultation or criterion 2 not met at consultation 2 or 3.
• Summed score over 9 indicators with sufficient content validity and variability.
• Time to return to work
• Change in level of fatigue
• Patient satisfaction
• No sig. association assessment of symptoms and any criteria
• No sig. association correct diagnosis and any criteria
• Sig. pos. association evaluation curative care and satisfaction.
• No sig. association assessment of work-related causes and any criteria
• Sig. neg. association evaluation work disabilities and return to work.
• No sig. association organizational interventions and any criteria
• Sig. pos. association interventions curative sector and return to work.
• No sig. association advice to return to work and any criteria
• Sig. neg. association timing of consultations and return to work.
• Sig. pos. association overall quality of care and return to work, and satisfaction.
Rost et al. [70] • A prescription for an antidepressant medication was noted from up to 30 days before to 14 days after index episode start date; dosage sufficient to take medication for 84 out of 114 days following first prescription; 3 non-emergency visits room visits to a primary care or mental health provider at least one of them had to be with the prescribing provider
• 4 or more specialty depression care counseling visits in the 6 months following the index visit
• Absenteeism: lost work hours in the past 4 weeks due to illness or doctor visits • No sig. association appropriate medication and change in absenteeism over 1 year
• Sig. association appropriate psychotherapy and change in absenteeism over 1 year
Wierdsma et al. [111] • Clients receive any psychiatric care in the year before involuntary admission • Length of stay (less than 3 weeks; more than 6 months; mean number of days)
• Ambulatory follow-up
• Readmission (within 3 months; within 1 year)
• Continued care 12 months after involuntary admission
• Sig. pos. association preadmission care and length of stay, and continued care after 12 months.
• No sig. association preadmission care and ambulatory follow-up, and readmission