Table 1 Publications and PMHC quality measurement initiatives per nation/region
From: Performance indicators for public mental healthcare: a systematic international inventory
Nation/region | Author/organization (year) | Objective of publication/study | PI/sets/frameworks | Purpose of PI/set/framework |
|---|---|---|---|---|
USA | Â | Â | Â | Â |
| Â | Simpson & Lloyd (1979) [17] | Cohort study relating client perception of program performance to outcomes | Client evaluations of drug abuse treatment in relation to follow-up outcomes | Assess drug treatment effectiveness |
| Â | Koran & Meinhardt (1984) [18] | Assessment of validity of County Need Index | Social indicators in statewide mental health planning: lessons from California | Promote equity in the distribution of mental health funds |
| Â | National Committee for Quality Assurance (since 1993) [19] | PI development, assessment of usefulness and feasibility, and implementation | Health Plan/Employer Data Information Set (HEDIS) | Help employers to evaluate and compare performance among HMOs and other health plans |
| Â | McLellan et al. (1994) [20] | Exploration of patient and treatment factors in outcomes | Similarity of outcome predictors across opiate, cocaine, alcohol treatments; role of treatment services | Evaluate effectiveness of substance abuse treatment in reducing substance use, and improving social adjustment. |
| Â | Mental Health Statistics Improvement Program (1996) [21] | PI development, review of quality measurement performance initiatives | MHSIP Consumer-oriented Mental health Report Card | Capture and reflect important characteristics of mental health service delivery |
| Â | Srebnik et al. (1997) [22] | PI development based on literature review and stakeholder-opinion, assessment of PI validity | Outcome indicators for monitoring the quality of public mental health care | Assess the quality of public mental health care by consumers and providers |
| Â | Lyons et al. (1997) [23] | Determine whether readmissions can service as a PI for an inpatient psychiatric service | Predicting readmission to psychiatric hospital in a managed care environment: implications for quality indicators | Provide program managers, third-party payers, and policy makers with information regarding the functioning of health services |
| Â | Baker (1998) [24] | PI development and presentation of method of quality monitoring | A PI spreadsheet for physicians in community mental health centers | Demonstrate progress in meeting objectives and implementing strategies for mental health care to legislators and stakeholders |
| Â | Carpinello et al. (1998) [25] | Explore development, implementation, and early results of using a comprehensive performance management system | Managing the performance of mental health managed care: an example from New York State's Prepaid Mental Health Plan | Reflect the concerns of multiple stakeholders and form a foundation for continuous quality improvement activities and information-reporting products |
| Â | Pandiani et al. (1998) [26] | PI development and assessment of PI sensitivity and usefulness | Using incarceration rates to measure mental health program performance | Provide program administrators with standardized information of program performance in the area of mental health care |
| Â | Rosenheck & Cicchetti (1998) [27] | PI development and implementation | Mental health program report card for public sector programs | Tool in improvement of service delivery, mental health system performance, and accountability |
| Â | Macias et al. (1999) [28] | Assess the worth of mental health certification as a core component of state and regional performance contracting | The value of program certification for performance contracting | Assess the quality and fidelity of 'clubhouse' psychiatric rehabilitation programs |
| Â | Baker (1999) [29] | Description of management process for financial and clinical PI | PI for physicians in community mental health centers | Report clinical and financial performance to payers of mental health services |
| Â | Druss et al. (1999) [30] | Examine the association between consumer satisfaction- and administrative measures at an individual and a hospital level | Patient satisfaction and administrative measures as indicators of the quality of mental health care | Provide providers, purchasers and consumers with understandable and measurable information on the quality of health care |
| Â | Department of Health and Human Services (2000) [31] | Present a comprehensive, nationwide health promotion and disease prevention agenda. | Healthy People 2010--Understanding and improving health | Guiding instrument for addressing health issues, reversing unfavorable trends, and expanding past achievements in health |
| Â | Huff (2000) [32] | Assess the association between measures of post-admission outpatient utilization and readmission | Outpatient utilization patterns and quality outcomes after first acute ePIode of mental health hospitalization | Provide state, patient advocates and service providers with information to ensure outpatient quality of care |
| Â | McCorry et al. (2000) [33] | PI development and adoption of core set of PI by health plans, private employers, public payers, and accrediting associations | The Washington Circle Group core set of PI for alcohol- and other drug services for public- and private sector health plans | Promote quality and accountability in the delivery and management of AOD abuse services by public and private organized systems of care |
| Â | Vermont's Mental Health Performance Indicator Project Multi-stakeholder Advisory Group (2000) [34] | Recommendations for PI to be included in a publicly available mental health report card | Indicators of mental health program performance | Development of a data based culture of learning about the system of care |
| Â | National Association of State Mental Health Program Directors (2000) [35] | Provide a guide and a framework for the implementation of PI in mental health systems | The NASMHPD framework of mental health PI | Address the need for a standardized methodology for evaluating the impact of services provide through the public mental health system |
| Â | Siegel et al. (2000) [36] | Framework development and selection of performance measures | PI of cultural competency in mental health organizations | Assess the cultural competency of mental health systems |
| Â | American college of Mental Health Administration (2001) [37] | PI development, reaching consensus between five national accreditation organizations on quality assessment and measurement | A proposed consensus set of PI for behavioral health | Advance the partnership between consumers, purchasers, providers and others in quality measurement and improvement |
| Â | Young et al. (2001) [38] | Estimate the rate of appropriate treatment, and the effect of insurance, provider type and individual characteristics on receipt of appropriate care | Survey to assess quality of care for depressive and anxiety disorders in the US | Evaluate mental health care quality on a national basis |
| Â | California Department of Mental Health (2001) [39] | PI development and identify areas that require special study of feasibility of measures | PI for California's public mental health system | Provide information needed to continuously improve the care provided in California's public mental health system |
| Â | Eisen et al. (2001) [40] | Provide data that could be used to develop recommendations for an improved consumer survey | Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments | Assess quality of behavioral health from consumer perspective |
| Â | Chinman et al. (2002) [41] | Illustrate the utility of a continuous evaluation system in promoting improvements in a mental health treatment system | The Connecticut Mental Health Center patient profile project: application of a service need index | Defining the characteristics of the patient population to guide management decisions in caseload distribution and service development |
| Â | Demonstrate the value of proper proportions of resources | a. Expenditure on, and b. fiscal structure of mental health care systems and its relationship to suicide rate | Calculate the optimum distribution of community/state psychiatric hospital beds, and cost per capita for mental health care to minimize suicide rate | |
| Â | Dausey et al. (2002) [44] | Examine the relationship between preadmission care and length of inpatient stay, access to aftercare, and re-hospitalization | Preadmission care as a new mental health PI | Assess the quality, continuity, and intensity of care |
| Â | Minnesota Department of Human Services (2002) [45] | Inform counties and providers of the implementation of PI | PI measures for Adult Rule 79 mental health case management | Report on outcomes from the adult mental health system to comply with state's federal mental health block grant application |
| Â | Hermann et al. (2002) [46] | Assess utility and applicability of process measures for schizophrenia care | National inventory of measures of clinical processes proposed or used in the U.S. | Assess quality of care for schizophrenia |
| Â | Pandiani et al. (2002) [47] | Provide a methodological outline for measuring access and identify and discuss a set of decision points in the project | Measuring access to mental health care: a multi-indicator approach to program evaluation | Assess access to publicly funded systems focusing on both general and special populations |
| Â | Druss et al. (2002) [48] | Asses the relation between mental health care quality measures and measures of general care quality | HEDIS 2000 mental health care PI | Provide purchasers a report card for rating and selecting health plans |
| Â | CDC--National Public Health Performance Standards Program, (NPHPSP; 2002) [49] | Present instruments for assessment of local and state public health systems | Local and State public health system performance assessment instruments & Local public health governance performance assessment instrument | To improve the practice of public health by comprehensive performance measurement tools keyed to the 10 Essential Services of Public Health |
| Â | Beaulieu & Scutchfield (2002) [50] | Assess the face and content validity of NPHPSP instrument | Local Public Health System Performance Assessment Instrument | Ensure the delivery of public health services and support a process of quality improvement |
| Â | Beaulieu et al. (2003) [51] | Assess the content and criterion validity of NPHPSP instruments | Local and State Public Health System Performance Assessment instruments | Measure performance of the local and state public health system |
| Â | Trutko & Barnow (2003) [52] | Explore feasibility of developing a core set of PI measures for DHHS programs that focus on homelessness | Core PI for homeless-serving programs administered by the US DHHS | Facilitate documentation and analysis of the effectiveness of program interventions |
| Â | The Urban Institute (2003) [53] | Describe lessons learned from PI development experiment and provide suggestions for other communities | Community-wide outcome indicators for specific services | Balance outcome-reporting requirements of funders for accountability and providers for improvement of services |
| Â | Greenberg & Rosenheck (2003) [54] | Examine the association of continuity of care with factors (not) under managerial control | Managerial and environmental factors in the continuity of mental health care across institutions | Assess the quality of outpatient care for persons with severe mental illness |
| Â | Owen et al. (2003) [55] | Examine meaningfulness and validity of PI and automated data elements | Mental health QUERI initiative: expert ratings of criteria to assess performance for major depressive disorder and schizophrenia | Provide clinicians, managers, quality improvement specialists and researchers in the Veterans Health Administration with useful data on clinical practice guidelines compliance |
| Â | Siegel et al. (2003) [56] | Benchmarking selected performance measures | PI of cultural competency in mental health organizations | Assess organizational progress in attaining cultural competency (CC) and to provide specific steps for implementing facets of CC. |
| Â | Solberg et al. (2003) [57] | Understand the process, outcomes and patient satisfaction of primary care patients diagnosed with depression | Process, outcomes and satisfaction in primary care for patients with depression | Identify quality gaps and serve as a baseline for quality improvements in health plan depression care |
| Â | Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Service Administration (SAMHSA), DHHS (2003) [58] | Report on 16-state indicator pilot project focused on assessment, refinement an pilot testing comparable mental health performance indicators | PI adopted from the NASMHPD Framework of Performance Indicators reflecting much of the MHSIP Report Card | Report mental health system performance comparably across states for national reporting, and facilitate planning, policy formulation and decision making at the state level. |
| Â | Edlund et al. (2003) [59] | Validate the technical quality-satisfaction relationship and examine the effects of selection bias among patients with depressive and anxiety disorders | Satisfaction measures as a reflection of technical quality of mental health care | Provide health care plan and provider quality information to insurers, providers, and researchers for improvement of quality of care for common mental disorders |
| Â | Virginia Department of Mental Health, Mental Retardation and Substance Abuse Services (2003) [60] | PI implementation and report on outcomes | Virginia's performance outcomes measurement system (POMS) | Provide public mental health authorities with information on consumer outcomes and provider performance to contain costs, improve quality and provide greater accountability |
| Â | Blank et al. (2004) [61] | Assess efficiency of a selection of POMS indicators and develop recommendations for improving POMS | Virginia's POMS | Continuously improve the quality of services and increase accountability for taxpayer dollars |
| Â | Charbonneau et al. (2004) [62] | Explore the relationship of process measures with subsequent overall hospitalizations | Guideline-based depression process measures | Estimate healthcare quality and quantify its benefits |
| Â | Stein et al. (2004) [63] | Evaluate the process and quality of care and examine patient characteristics that potentially determine quality | Quality of care for patients with a broad array of anxiety disorders | Assess the quality of care received in primary care settings for efforts at quality improvement |
| Â | Druss et al. (2004) [64] | Assess relation between mental health care volume and quality | HEDIS 2000 mental health care PI | Reflect the capacity to treat specialized conditions and as proxy for clinician volume |
| Â | McGuire & Rosenheck (2004) [65] | Examine the relation between incarceration history and baseline psychosocial problems service utilization, and outcomes of care | Criminal history as a prognostic indicator in the treatment of homeless people with severe mental illness | Provide clinicians and administrators with information on treatment prospects of former inmates |
| Â | Leff et al. (2004) [66] | Investigate the relationship between service fit and mortality as a step towards understanding the general relationship between service quality and outcomes | Service quality as measured by service fit vs. mortality among public mental health system service recipients | Assess and compare programs and systems, the extent to which an intervention has been implemented in program evaluations, an service need in program and resource allocation planning |
| Â | Valenstein et al. (2004) [67] | Examine providers' views of quality monitoring processes and patient, provider and organizational factors that might be associated with more positive views | PI drawn from sets maintained and implemented by various national organizations | Provide mental health care providers with feedback about their performance |
| Â | Mental health recovery: What helps and what hinders? A National Research Project for the Development of Recovery Facilitating System Performance Indicators (2004) [68] | PI development, and assessment of usability and implementation | Recovery oriented system indicators (ROSI) | Facilitate mental health recovery, and bridge the gap between the principles of recovery and self-help and application of these principles in everyday work of staff and service systems |
| Â | Hermann et al. (2004) [69] | PI selection and assessment of PI meaningfulness and feasibility | Core set of PI for mental and substance-related care | Ensure that systems and providers focus on clinically important processes with known variations in quality of care |
| Â | Rost et al. (2005) [70] | Explore relation between administrative PI and absenteeism | Relationship of depression treatment PI to employee absenteeism | Provide employers with evidence of the value of the healthcare they purchase. |
| Â | Mental Health Statistics Improvement Program (2005) [71] | PI development and present toolkit for methodology, implementation and uses | MHSIP Quality Report (MQR) | Reflect key concerns in mental health systems or organizations performance |
| Â | Washington State Department of Social and Health Services--Mental Health Division (2005) [72] | PI implementation and report on PI information | State-wide publicly funded mental health PI | Help system managers and payers understand trends in services delivery systems and change across time |
| Â | New York Office of Mental Health (2005) [73] | PI development and implementation | 2005-2009 Statewide comprehensive plan for mental health services | Provide a conceptual framework for performance measurement and improvement |
| Â | Garnick et al. (2006) [74] | Examine different types of PI, how they fit within the continuum of care, and the types of data that can be used to arrive at these measures | PI for alcohol and other drug services | Evaluate how well practitioners' actions conform to guidelines, review criteria or standards to improve access, and quality of treatment |
| Â | Hermann et al. (2006) [75] | Develop statistical benchmarks for quality measures of mental health and substance-related care | Selected measures from core set of PI for mental and substance-related care | Assess quality of care for Medicaid beneficiaries to inform quality improvement |
| Â | Mental health recovery: What helps and what hinders? A National Research Project for the Development of Recovery Facilitating System Performance Indicators (2006) [76] | Refinement of self-report survey and administrative profile PI based on feedback from stakeholders | Recovery oriented system indicators (ROSI) | Measure critical elements and processes of recovery facilitating mental health programs and delivery systems |
| Â | PI development informed by APA guidelines for the treatment of bipolar disorder | Quality of care for bipolar I disorder | Assess quality of medication and psychotherapy treatment | |
| Â | Center for Quality Assessment and Improvement in Mental Health (2007) [79] | PI development using an adaptation of the RAND appropriateness method, and assess reliability | Standards for bipolar excellence (STABLE) PI | Advance the quality of care for by supporting improved recognition and promoting evidence-based management |
| Â | CDC--National Public Health Performance Standards Program (NPHPSP; 2007) [80] | Present the revised instruments for assessment of local and state public health systems | Version 2.0 of the Local and State public health system performance assessment instruments and Local public health governance performance assessment instrument | Provide users with information to identify strengths and weaknesses of the public health system to determine opportunities for improvement |
| Â | Virginia Department of Mental Health, Mental Retardation and Substance Abuse services (2008) [81] | PI implementation and report on achieved goals | 2008 mental health block grant implementation report PI | Monitor the implementation and transformation of a recovery-oriented system |
Canada | Â | Â | Â | Â |
| Â | Canadian Institute for Health Information (CIHI; 2001) [82] | PI development, assessment of feasibility & usefulness | The Roadmap Initiative--Mental health and Addiction Services Roadmap Project. Phase 1 Indicators | Maintain and improve Canada's health system |
| Â | Federal/Provincial/Territorial Advisory Network on Mental Health (2001) [83] | PI development | PI for Mental health Services and Supports--A Resource Kit | Facilitate ongoing accountability and evaluation of mental health services and supports |
| Â | Ontario Ministry of Health and Long-term Care (2003) [84] | PI development and mechanisms for implementation | Mental Health Accountability Framework | Increasing health system accountability to ensure services are as effective and efficient as possible |
| Â | Addington et al. (2005) [85] | PI selection based on literature review and consensus procedure | PI for early psychosis treatment services | Evaluate quality, and assist providers in improving quality of health care |
Australia | Â | Â | Â | Â |
| Â | NMHWG Information Strategy committee Performance Indicator drafting group (2005) [86] | Development conceptual framework of performance & PI | Key PI for Australian public mental health services | Improve public sector mental health service quality |
| Â | Meehan et al. (2007) [87] | Assessment of feasibility & usefulness of benchmarking mental health services | Input, process, output and outcome PI for inpatient mental health services | Benchmarking public sector mental health service organizations |
United Kingdom | Â | Â | Â | Â |
| Â | Jenkins (1990) [88] | PI development | A system of outcome PI for mental health care. | Ensure that clinicians district health authorities and directors of public health can monitor and evaluate mental health care |
| Â | Framework and PI development | A National Service Framework for Mental Health; A New Approach To Social Services Performance | Help drive up quality and remove the wide and unacceptable variations in provision. | |
| Â | Shipley et al. (2000) [91] | PI development and validity assessment | Patient satisfaction: a valid index of quality of care in a psychiatric service | Provide PMHC planners with an independent yardstick for mental health services and determine population mental health |
| Â | Audit Commission (2001) [92] | PI development and application | Library of Local Authority PI | Accountability and benchmarking of local authorities by national government |
| Â | Jones (2001) [93] | Review of pre-existing PI | Hospital care pathways for patients with schizophrenia | Clarify terms and concepts in schizophrenia care process |
| Â | Shield et al. (2003) [94] | PI development | PI for primary care mental health services | Facilitating quality improvement and show variations in care |
| Â | Commission for Health Improvement (2003) [95] | PI development and implementation | Mental health trust balanced scorecard indicators | Improve care provided by mental health trusts and promote transparency in PMHC |
| Â | Department of Health (2004) [96] | PI development | National Standards, Local Action--health and social care standards and planning framework | Set out the framework for all NHS organizations and social service authorities to use in planning over the next financial three years |
| Â | NHS Health Scotland (2007) [11] | PI development based on current data, policy, evidence, and expert-opinion | Core set of national, sustainable mental health indicators for adults in Scotland | Determine whether mental health is improving and track progress |
UK (cont.) | Care Services Improvement Partnership (2007) [97] | PI development | Outcome indicators framework for mental health day services | Help commissioners and providers to monitor, evaluate, and measure the effectiveness of day services adults with mental health problems |
| Â | Healthcare Commission (2007) [98] | PI development | The Better Metrics Project | Provide a common set of requirements to ensure safe and acceptable quality health provision, and provide a framework for continuous improvement |
| Â | Department of Communities and Local Government (2007) [99] | PI development and application | The National Indicator Set (NIS) in Comprehensive Area Assessment (CAA) | Performance management of local government by central government |
| Â | Association of Public Health Observatories (2007) [100] | Present data on the factors which give rise to poor mental health, mental health status of populations, provision of interventions, service user experience and traditional outcomes | Indications of public health in the English Regions: Mental Health | Provide a resource for regional public health directors, PCT and CSIP directors in making decisions, holding to account those responsible for the delivery and improving mental health of the population. |
| Â | Wilkinson et al. (2008) [101] | Report on the construction of a set of indicators for mental health and the publication of a report for England's Chief Medical Officer | Indications of public health in the English Regions: Mental Health | Initiating public health action to improve health at a regional level in England |
| Â | London Health Observatory (2008) [102] | PI development and implementation | Mental health and wellbeing scorecard | Support primary care trusts in monitoring delivery of national health improvement objectives, and improvement of mental health and wellbeing |
| Â | Care Services Improvement Partnership (2009) [103] | Broaden initial framework to provide for application in mental health services more widely | Outcome indicators framework for mental health services | Ensure the effectiveness and impact of redesigned and refocused services |
| Â | Association of Public Health Observatories (2009) [104] | PI development, application of pre-existing PI, operationalization of issues, targets and recommendations in policies | Indications of public health in the English regions: Drug Use | Present information on the relative positions of regions on major health policy areas, highlighting differences, to stimulate practitioners to take action to improve health |
Spain | Â | Â | Â | Â |
| Â | Gispert et al. (1998) [105] | PI development, assessment of feasibility | Mental health expectancy: a global indicator of population mental health | Reflect the impact that disability due to mental disorders has on population health |
Germany | Â | Â | Â | Â |
| Â | Kunze & Priebe (1998) [106] | Development of quality assessment tool | Assessing the quality of psychiatric hospital care: a German approach. | Assessment of quality of care after political reforms to help promote quality. |
| Â | Bramesfeld et al. (2007) [107] | Implementation of quality assessment tool | Evaluating inpatient and outpatient care in Germany with the WHO responsiveness concept | Evaluate performance of mental health care services to improve responsiveness |
The Netherlands | Â | Â | Â | Â |
| Â | Roeg et al. (2005) [108] | Development of disease-specific concept of quality | Conceptual framework of quality for assertive outreach programs for severely impaired substance abuses | Improve understanding of the relationship between specific program features and effectiveness |
| Â | Nabitz et al. (2005) [109] | Development of disease-specific concept of quality | A quality framework for addiction treatment programs | Clarify the concept of quality for addiction treatment programs |
| Â | Nieuwenhuijsen et al. (2005) [110] | PI development & validity assessment | PI for rehabilitation of workers with mental health problem | Assessment of occupational health care to improve the quality of care |
| Â | Wierdsma et al. (2006) [111] | Application & risk adjustment of PI | Utilization indicators for quality of involuntary admission mental health care | Assess criteria for involuntary admission to inpatient mental health care |
The Netherlands (cont.) | Steering Committee--Transparency Mental Healthcare (2007) [112] | Improvement of existing PI and PI development | Basic Set of PI for Mental Health Care and Addiction Care services | Promoting transparency and publication of quality information by mental health and addiction service providers |
Italy | Â | Â | Â | Â |
| Â | Bollini et al. (2008) [113] | PI development, operationalization of (PORT) guidelines | Indicators of conformance with guidelines of schizophrenia treatment in mental health services | Monitor the conformance of care with recommend practices and identify areas in need of improvement |
South Africa | Â | Â | Â | Â |
| Â | Lund & Fisher (2003) [114] | PI development and assessment of PI usefulness | Community/hospital indicators in South African public sector mental health services | Assess the implementation of policy objectives over time |
Singapore | Â | Â | Â | Â |
| Â | Chong et al. (2006) [115] | Application of pre-existing PI and operationalization of guidelines | Assessment of the quality of care for patients with first- episode psychosis | Assess adherence to guidelines in an early psychosis intervention program |
International | Â | Â | Â | Â |
| Â | National Research and Development Centre for Welfare and Health (STAKES)--EC Health Monitoring Programme (2002) [8] | PI development and assessment of feasibility and usability | A set of mental health indicators for European Union | Contribute to the establishment of a community monitoring system |
| Â | Organisation for Economic Cooperation and Development (OECD; 2004) [10] | PI selection and assessment of utility | Indicators for the quality of mental health care at the health system level in OECD countries | Improve organization and management of care to allow countries to spend their health care dollars more wisely |
| Â | World Health Organization (2005) [116] | PI development, operationalization of recommendations, assessment of usefulness | Assessment Instrument for Mental Health Systems (WHO-AIMS) version 2.2 | Collect essential information on the mental health system of a country or region to improve mental health systems |
| Â | Saxena et al. (2006) [117] | Describe and compare 4 existing high-income country public mental health indicator schemes | Healthy People 2010; Mental Health Report Card (MHSIP); Commission for Health Improvement Indicators (CHI); European community Health Indicators (ECHI) | Contribute to the development of relevant policies and plans |
| Â | Hermann et al. (2006) [118] | Report on methods employed to reach consensus on the OECD mental health care indicators | Indicators for the quality of mental health care at the health system level in OECD countries | Facilitate improvement within organizations, provide oversight of quality by public agencies and private payers, and provide insight into what levels of performance are feasible |
| Â | OECD (2008) [119] | Provide overview of present mental health care information systems to assess feasibility of performance indicators | Indicators for the quality of mental health care at the health system level in OECD countries | Monitor changes on effectiveness and safety patients subsequent to reform of mental health services and facilitate benchmarking |