Skip to main content

Table 1 Publications and PMHC quality measurement initiatives per nation/region

From: Performance indicators for public mental healthcare: a systematic international inventory

Nation/region Author/organization (year) Objective of publication/study PI/sets/frameworks Purpose of PI/set/framework
USA     
  Simpson & Lloyd (1979) [17] Cohort study relating client perception of program performance to outcomes Client evaluations of drug abuse treatment in relation to follow-up outcomes Assess drug treatment effectiveness
  Koran & Meinhardt (1984) [18] Assessment of validity of County Need Index Social indicators in statewide mental health planning: lessons from California Promote equity in the distribution of mental health funds
  National Committee for Quality Assurance (since 1993) [19] PI development, assessment of usefulness and feasibility, and implementation Health Plan/Employer Data Information Set (HEDIS) Help employers to evaluate and compare performance among HMOs and other health plans
  McLellan et al. (1994) [20] Exploration of patient and treatment factors in outcomes Similarity of outcome predictors across opiate, cocaine, alcohol treatments; role of treatment services Evaluate effectiveness of substance abuse treatment in reducing substance use, and improving social adjustment.
  Mental Health Statistics Improvement Program (1996) [21] PI development, review of quality measurement performance initiatives MHSIP Consumer-oriented Mental health Report Card Capture and reflect important characteristics of mental health service delivery
  Srebnik et al. (1997) [22] PI development based on literature review and stakeholder-opinion, assessment of PI validity Outcome indicators for monitoring the quality of public mental health care Assess the quality of public mental health care by consumers and providers
  Lyons et al. (1997) [23] Determine whether readmissions can service as a PI for an inpatient psychiatric service Predicting readmission to psychiatric hospital in a managed care environment: implications for quality indicators Provide program managers, third-party payers, and policy makers with information regarding the functioning of health services
  Baker (1998) [24] PI development and presentation of method of quality monitoring A PI spreadsheet for physicians in community mental health centers Demonstrate progress in meeting objectives and implementing strategies for mental health care to legislators and stakeholders
  Carpinello et al. (1998) [25] Explore development, implementation, and early results of using a comprehensive performance management system Managing the performance of mental health managed care: an example from New York State's Prepaid Mental Health Plan Reflect the concerns of multiple stakeholders and form a foundation for continuous quality improvement activities and information-reporting products
  Pandiani et al. (1998) [26] PI development and assessment of PI sensitivity and usefulness Using incarceration rates to measure mental health program performance Provide program administrators with standardized information of program performance in the area of mental health care
  Rosenheck & Cicchetti (1998) [27] PI development and implementation Mental health program report card for public sector programs Tool in improvement of service delivery, mental health system performance, and accountability
  Macias et al. (1999) [28] Assess the worth of mental health certification as a core component of state and regional performance contracting The value of program certification for performance contracting Assess the quality and fidelity of 'clubhouse' psychiatric rehabilitation programs
  Baker (1999) [29] Description of management process for financial and clinical PI PI for physicians in community mental health centers Report clinical and financial performance to payers of mental health services
  Druss et al. (1999) [30] Examine the association between consumer satisfaction- and administrative measures at an individual and a hospital level Patient satisfaction and administrative measures as indicators of the quality of mental health care Provide providers, purchasers and consumers with understandable and measurable information on the quality of health care
  Department of Health and Human Services (2000) [31] Present a comprehensive, nationwide health promotion and disease prevention agenda. Healthy People 2010--Understanding and improving health Guiding instrument for addressing health issues, reversing unfavorable trends, and expanding past achievements in health
  Huff (2000) [32] Assess the association between measures of post-admission outpatient utilization and readmission Outpatient utilization patterns and quality outcomes after first acute ePIode of mental health hospitalization Provide state, patient advocates and service providers with information to ensure outpatient quality of care
  McCorry et al. (2000) [33] PI development and adoption of core set of PI by health plans, private employers, public payers, and accrediting associations The Washington Circle Group core set of PI for alcohol- and other drug services for public- and private sector health plans Promote quality and accountability in the delivery and management of AOD abuse services by public and private organized systems of care
  Vermont's Mental Health Performance Indicator Project Multi-stakeholder Advisory Group (2000) [34] Recommendations for PI to be included in a publicly available mental health report card Indicators of mental health program performance Development of a data based culture of learning about the system of care
  National Association of State Mental Health Program Directors (2000) [35] Provide a guide and a framework for the implementation of PI in mental health systems The NASMHPD framework of mental health PI Address the need for a standardized methodology for evaluating the impact of services provide through the public mental health system
  Siegel et al. (2000) [36] Framework development and selection of performance measures PI of cultural competency in mental health organizations Assess the cultural competency of mental health systems
  American college of Mental Health Administration (2001) [37] PI development, reaching consensus between five national accreditation organizations on quality assessment and measurement A proposed consensus set of PI for behavioral health Advance the partnership between consumers, purchasers, providers and others in quality measurement and improvement
  Young et al. (2001) [38] Estimate the rate of appropriate treatment, and the effect of insurance, provider type and individual characteristics on receipt of appropriate care Survey to assess quality of care for depressive and anxiety disorders in the US Evaluate mental health care quality on a national basis
  California Department of Mental Health (2001) [39] PI development and identify areas that require special study of feasibility of measures PI for California's public mental health system Provide information needed to continuously improve the care provided in California's public mental health system
  Eisen et al. (2001) [40] Provide data that could be used to develop recommendations for an improved consumer survey Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments Assess quality of behavioral health from consumer perspective
  Chinman et al. (2002) [41] Illustrate the utility of a continuous evaluation system in promoting improvements in a mental health treatment system The Connecticut Mental Health Center patient profile project: application of a service need index Defining the characteristics of the patient population to guide management decisions in caseload distribution and service development
  Davis & Lowell (2002a, b) [42, 43] Demonstrate the value of proper proportions of resources a. Expenditure on, and b. fiscal structure of mental health care systems and its relationship to suicide rate Calculate the optimum distribution of community/state psychiatric hospital beds, and cost per capita for mental health care to minimize suicide rate
  Dausey et al. (2002) [44] Examine the relationship between preadmission care and length of inpatient stay, access to aftercare, and re-hospitalization Preadmission care as a new mental health PI Assess the quality, continuity, and intensity of care
  Minnesota Department of Human Services (2002) [45] Inform counties and providers of the implementation of PI PI measures for Adult Rule 79 mental health case management Report on outcomes from the adult mental health system to comply with state's federal mental health block grant application
  Hermann et al. (2002) [46] Assess utility and applicability of process measures for schizophrenia care National inventory of measures of clinical processes proposed or used in the U.S. Assess quality of care for schizophrenia
  Pandiani et al. (2002) [47] Provide a methodological outline for measuring access and identify and discuss a set of decision points in the project Measuring access to mental health care: a multi-indicator approach to program evaluation Assess access to publicly funded systems focusing on both general and special populations
  Druss et al. (2002) [48] Asses the relation between mental health care quality measures and measures of general care quality HEDIS 2000 mental health care PI Provide purchasers a report card for rating and selecting health plans
  CDC--National Public Health Performance Standards Program, (NPHPSP; 2002) [49] Present instruments for assessment of local and state public health systems Local and State public health system performance assessment instruments & Local public health governance performance assessment instrument To improve the practice of public health by comprehensive performance measurement tools keyed to the 10 Essential Services of Public Health
  Beaulieu & Scutchfield (2002) [50] Assess the face and content validity of NPHPSP instrument Local Public Health System Performance Assessment Instrument Ensure the delivery of public health services and support a process of quality improvement
  Beaulieu et al. (2003) [51] Assess the content and criterion validity of NPHPSP instruments Local and State Public Health System Performance Assessment instruments Measure performance of the local and state public health system
  Trutko & Barnow (2003) [52] Explore feasibility of developing a core set of PI measures for DHHS programs that focus on homelessness Core PI for homeless-serving programs administered by the US DHHS Facilitate documentation and analysis of the effectiveness of program interventions
  The Urban Institute (2003) [53] Describe lessons learned from PI development experiment and provide suggestions for other communities Community-wide outcome indicators for specific services Balance outcome-reporting requirements of funders for accountability and providers for improvement of services
  Greenberg & Rosenheck (2003) [54] Examine the association of continuity of care with factors (not) under managerial control Managerial and environmental factors in the continuity of mental health care across institutions Assess the quality of outpatient care for persons with severe mental illness
  Owen et al. (2003) [55] Examine meaningfulness and validity of PI and automated data elements Mental health QUERI initiative: expert ratings of criteria to assess performance for major depressive disorder and schizophrenia Provide clinicians, managers, quality improvement specialists and researchers in the Veterans Health Administration with useful data on clinical practice guidelines compliance
  Siegel et al. (2003) [56] Benchmarking selected performance measures PI of cultural competency in mental health organizations Assess organizational progress in attaining cultural competency (CC) and to provide specific steps for implementing facets of CC.
  Solberg et al. (2003) [57] Understand the process, outcomes and patient satisfaction of primary care patients diagnosed with depression Process, outcomes and satisfaction in primary care for patients with depression Identify quality gaps and serve as a baseline for quality improvements in health plan depression care
  Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Service Administration (SAMHSA), DHHS (2003) [58] Report on 16-state indicator pilot project focused on assessment, refinement an pilot testing comparable mental health performance indicators PI adopted from the NASMHPD Framework of Performance Indicators reflecting much of the MHSIP Report Card Report mental health system performance comparably across states for national reporting, and facilitate planning, policy formulation and decision making at the state level.
  Edlund et al. (2003) [59] Validate the technical quality-satisfaction relationship and examine the effects of selection bias among patients with depressive and anxiety disorders Satisfaction measures as a reflection of technical quality of mental health care Provide health care plan and provider quality information to insurers, providers, and researchers for improvement of quality of care for common mental disorders
  Virginia Department of Mental Health, Mental Retardation and Substance Abuse Services (2003) [60] PI implementation and report on outcomes Virginia's performance outcomes measurement system (POMS) Provide public mental health authorities with information on consumer outcomes and provider performance to contain costs, improve quality and provide greater accountability
  Blank et al. (2004) [61] Assess efficiency of a selection of POMS indicators and develop recommendations for improving POMS Virginia's POMS Continuously improve the quality of services and increase accountability for taxpayer dollars
  Charbonneau et al. (2004) [62] Explore the relationship of process measures with subsequent overall hospitalizations Guideline-based depression process measures Estimate healthcare quality and quantify its benefits
  Stein et al. (2004) [63] Evaluate the process and quality of care and examine patient characteristics that potentially determine quality Quality of care for patients with a broad array of anxiety disorders Assess the quality of care received in primary care settings for efforts at quality improvement
  Druss et al. (2004) [64] Assess relation between mental health care volume and quality HEDIS 2000 mental health care PI Reflect the capacity to treat specialized conditions and as proxy for clinician volume
  McGuire & Rosenheck (2004) [65] Examine the relation between incarceration history and baseline psychosocial problems service utilization, and outcomes of care Criminal history as a prognostic indicator in the treatment of homeless people with severe mental illness Provide clinicians and administrators with information on treatment prospects of former inmates
  Leff et al. (2004) [66] Investigate the relationship between service fit and mortality as a step towards understanding the general relationship between service quality and outcomes Service quality as measured by service fit vs. mortality among public mental health system service recipients Assess and compare programs and systems, the extent to which an intervention has been implemented in program evaluations, an service need in program and resource allocation planning
  Valenstein et al. (2004) [67] Examine providers' views of quality monitoring processes and patient, provider and organizational factors that might be associated with more positive views PI drawn from sets maintained and implemented by various national organizations Provide mental health care providers with feedback about their performance
  Mental health recovery: What helps and what hinders? A National Research Project for the Development of Recovery Facilitating System Performance Indicators (2004) [68] PI development, and assessment of usability and implementation Recovery oriented system indicators (ROSI) Facilitate mental health recovery, and bridge the gap between the principles of recovery and self-help and application of these principles in everyday work of staff and service systems
  Hermann et al. (2004) [69] PI selection and assessment of PI meaningfulness and feasibility Core set of PI for mental and substance-related care Ensure that systems and providers focus on clinically important processes with known variations in quality of care
  Rost et al. (2005) [70] Explore relation between administrative PI and absenteeism Relationship of depression treatment PI to employee absenteeism Provide employers with evidence of the value of the healthcare they purchase.
  Mental Health Statistics Improvement Program (2005) [71] PI development and present toolkit for methodology, implementation and uses MHSIP Quality Report (MQR) Reflect key concerns in mental health systems or organizations performance
  Washington State Department of Social and Health Services--Mental Health Division (2005) [72] PI implementation and report on PI information State-wide publicly funded mental health PI Help system managers and payers understand trends in services delivery systems and change across time
  New York Office of Mental Health (2005) [73] PI development and implementation 2005-2009 Statewide comprehensive plan for mental health services Provide a conceptual framework for performance measurement and improvement
  Garnick et al. (2006) [74] Examine different types of PI, how they fit within the continuum of care, and the types of data that can be used to arrive at these measures PI for alcohol and other drug services Evaluate how well practitioners' actions conform to guidelines, review criteria or standards to improve access, and quality of treatment
  Hermann et al. (2006) [75] Develop statistical benchmarks for quality measures of mental health and substance-related care Selected measures from core set of PI for mental and substance-related care Assess quality of care for Medicaid beneficiaries to inform quality improvement
  Mental health recovery: What helps and what hinders? A National Research Project for the Development of Recovery Facilitating System Performance Indicators (2006) [76] Refinement of self-report survey and administrative profile PI based on feedback from stakeholders Recovery oriented system indicators (ROSI) Measure critical elements and processes of recovery facilitating mental health programs and delivery systems
  Busch et al. (2007) [77, 78] PI development informed by APA guidelines for the treatment of bipolar disorder Quality of care for bipolar I disorder Assess quality of medication and psychotherapy treatment
  Center for Quality Assessment and Improvement in Mental Health (2007) [79] PI development using an adaptation of the RAND appropriateness method, and assess reliability Standards for bipolar excellence (STABLE) PI Advance the quality of care for by supporting improved recognition and promoting evidence-based management
  CDC--National Public Health Performance Standards Program (NPHPSP; 2007) [80] Present the revised instruments for assessment of local and state public health systems Version 2.0 of the Local and State public health system performance assessment instruments and Local public health governance performance assessment instrument Provide users with information to identify strengths and weaknesses of the public health system to determine opportunities for improvement
  Virginia Department of Mental Health, Mental Retardation and Substance Abuse services (2008) [81] PI implementation and report on achieved goals 2008 mental health block grant implementation report PI Monitor the implementation and transformation of a recovery-oriented system
Canada     
  Canadian Institute for Health Information (CIHI; 2001) [82] PI development, assessment of feasibility & usefulness The Roadmap Initiative--Mental health and Addiction Services Roadmap Project. Phase 1 Indicators Maintain and improve Canada's health system
  Federal/Provincial/Territorial Advisory Network on Mental Health (2001) [83] PI development PI for Mental health Services and Supports--A Resource Kit Facilitate ongoing accountability and evaluation of mental health services and supports
  Ontario Ministry of Health and Long-term Care (2003) [84] PI development and mechanisms for implementation Mental Health Accountability Framework Increasing health system accountability to ensure services are as effective and efficient as possible
  Addington et al. (2005) [85] PI selection based on literature review and consensus procedure PI for early psychosis treatment services Evaluate quality, and assist providers in improving quality of health care
Australia     
  NMHWG Information Strategy committee Performance Indicator drafting group (2005) [86] Development conceptual framework of performance & PI Key PI for Australian public mental health services Improve public sector mental health service quality
  Meehan et al. (2007) [87] Assessment of feasibility & usefulness of benchmarking mental health services Input, process, output and outcome PI for inpatient mental health services Benchmarking public sector mental health service organizations
United Kingdom     
  Jenkins (1990) [88] PI development A system of outcome PI for mental health care. Ensure that clinicians district health authorities and directors of public health can monitor and evaluate mental health care
  National Health Service (1999a, b) [89, 90] Framework and PI development A National Service Framework for Mental Health; A New Approach To Social Services Performance Help drive up quality and remove the wide and unacceptable variations in provision.
  Shipley et al. (2000) [91] PI development and validity assessment Patient satisfaction: a valid index of quality of care in a psychiatric service Provide PMHC planners with an independent yardstick for mental health services and determine population mental health
  Audit Commission (2001) [92] PI development and application Library of Local Authority PI Accountability and benchmarking of local authorities by national government
  Jones (2001) [93] Review of pre-existing PI Hospital care pathways for patients with schizophrenia Clarify terms and concepts in schizophrenia care process
  Shield et al. (2003) [94] PI development PI for primary care mental health services Facilitating quality improvement and show variations in care
  Commission for Health Improvement (2003) [95] PI development and implementation Mental health trust balanced scorecard indicators Improve care provided by mental health trusts and promote transparency in PMHC
  Department of Health (2004) [96] PI development National Standards, Local Action--health and social care standards and planning framework Set out the framework for all NHS organizations and social service authorities to use in planning over the next financial three years
  NHS Health Scotland (2007) [11] PI development based on current data, policy, evidence, and expert-opinion Core set of national, sustainable mental health indicators for adults in Scotland Determine whether mental health is improving and track progress
UK (cont.) Care Services Improvement Partnership (2007) [97] PI development Outcome indicators framework for mental health day services Help commissioners and providers to monitor, evaluate, and measure the effectiveness of day services adults with mental health problems
  Healthcare Commission (2007) [98] PI development The Better Metrics Project Provide a common set of requirements to ensure safe and acceptable quality health provision, and provide a framework for continuous improvement
  Department of Communities and Local Government (2007) [99] PI development and application The National Indicator Set (NIS) in Comprehensive Area Assessment (CAA) Performance management of local government by central government
  Association of Public Health Observatories (2007) [100] Present data on the factors which give rise to poor mental health, mental health status of populations, provision of interventions, service user experience and traditional outcomes Indications of public health in the English Regions: Mental Health Provide a resource for regional public health directors, PCT and CSIP directors in making decisions, holding to account those responsible for the delivery and improving mental health of the population.
  Wilkinson et al. (2008) [101] Report on the construction of a set of indicators for mental health and the publication of a report for England's Chief Medical Officer Indications of public health in the English Regions: Mental Health Initiating public health action to improve health at a regional level in England
  London Health Observatory (2008) [102] PI development and implementation Mental health and wellbeing scorecard Support primary care trusts in monitoring delivery of national health improvement objectives, and improvement of mental health and wellbeing
  Care Services Improvement Partnership (2009) [103] Broaden initial framework to provide for application in mental health services more widely Outcome indicators framework for mental health services Ensure the effectiveness and impact of redesigned and refocused services
  Association of Public Health Observatories (2009) [104] PI development, application of pre-existing PI, operationalization of issues, targets and recommendations in policies Indications of public health in the English regions: Drug Use Present information on the relative positions of regions on major health policy areas, highlighting differences, to stimulate practitioners to take action to improve health
Spain     
  Gispert et al. (1998) [105] PI development, assessment of feasibility Mental health expectancy: a global indicator of population mental health Reflect the impact that disability due to mental disorders has on population health
Germany     
  Kunze & Priebe (1998) [106] Development of quality assessment tool Assessing the quality of psychiatric hospital care: a German approach. Assessment of quality of care after political reforms to help promote quality.
  Bramesfeld et al. (2007) [107] Implementation of quality assessment tool Evaluating inpatient and outpatient care in Germany with the WHO responsiveness concept Evaluate performance of mental health care services to improve responsiveness
The Netherlands     
  Roeg et al. (2005) [108] Development of disease-specific concept of quality Conceptual framework of quality for assertive outreach programs for severely impaired substance abuses Improve understanding of the relationship between specific program features and effectiveness
  Nabitz et al. (2005) [109] Development of disease-specific concept of quality A quality framework for addiction treatment programs Clarify the concept of quality for addiction treatment programs
  Nieuwenhuijsen et al. (2005) [110] PI development & validity assessment PI for rehabilitation of workers with mental health problem Assessment of occupational health care to improve the quality of care
  Wierdsma et al. (2006) [111] Application & risk adjustment of PI Utilization indicators for quality of involuntary admission mental health care Assess criteria for involuntary admission to inpatient mental health care
The Netherlands (cont.) Steering Committee--Transparency Mental Healthcare (2007) [112] Improvement of existing PI and PI development Basic Set of PI for Mental Health Care and Addiction Care services Promoting transparency and publication of quality information by mental health and addiction service providers
Italy     
  Bollini et al. (2008) [113] PI development, operationalization of (PORT) guidelines Indicators of conformance with guidelines of schizophrenia treatment in mental health services Monitor the conformance of care with recommend practices and identify areas in need of improvement
South Africa     
  Lund & Fisher (2003) [114] PI development and assessment of PI usefulness Community/hospital indicators in South African public sector mental health services Assess the implementation of policy objectives over time
Singapore     
  Chong et al. (2006) [115] Application of pre-existing PI and operationalization of guidelines Assessment of the quality of care for patients with first- episode psychosis Assess adherence to guidelines in an early psychosis intervention program
International     
  National Research and Development Centre for Welfare and Health (STAKES)--EC Health Monitoring Programme (2002) [8] PI development and assessment of feasibility and usability A set of mental health indicators for European Union Contribute to the establishment of a community monitoring system
  Organisation for Economic Cooperation and Development (OECD; 2004) [10] PI selection and assessment of utility Indicators for the quality of mental health care at the health system level in OECD countries Improve organization and management of care to allow countries to spend their health care dollars more wisely
  World Health Organization (2005) [116] PI development, operationalization of recommendations, assessment of usefulness Assessment Instrument for Mental Health Systems (WHO-AIMS) version 2.2 Collect essential information on the mental health system of a country or region to improve mental health systems
  Saxena et al. (2006) [117] Describe and compare 4 existing high-income country public mental health indicator schemes Healthy People 2010; Mental Health Report Card (MHSIP); Commission for Health Improvement Indicators (CHI); European community Health Indicators (ECHI) Contribute to the development of relevant policies and plans
  Hermann et al. (2006) [118] Report on methods employed to reach consensus on the OECD mental health care indicators Indicators for the quality of mental health care at the health system level in OECD countries Facilitate improvement within organizations, provide oversight of quality by public agencies and private payers, and provide insight into what levels of performance are feasible
  OECD (2008) [119] Provide overview of present mental health care information systems to assess feasibility of performance indicators Indicators for the quality of mental health care at the health system level in OECD countries Monitor changes on effectiveness and safety patients subsequent to reform of mental health services and facilitate benchmarking