Improving health care quality for racial/ethnic minorities: a systematic review of the best evidence regarding provider and organization interventions
© Beach et al; licensee BioMed Central Ltd. 2006
Received: 21 October 2005
Accepted: 24 April 2006
Published: 24 April 2006
Despite awareness of inequities in health care quality, little is known about strategies that could improve the quality of healthcare for ethnic minority populations. We conducted a systematic literature review and analysis to synthesize the findings of controlled studies evaluating interventions targeted at health care providers to improve health care quality or reduce disparities in care for racial/ethnic minorities.
We performed electronic and hand searches from 1980 through June 2003 to identify randomized controlled trials or concurrent controlled trials. Reviewers abstracted data from studies to determine study characteristics, results, and quality. We graded the strength of the evidence as excellent, good, fair or poor using predetermined criteria. The main outcome measures were evidence of effectiveness and cost of strategies to improve health care quality or reduce disparities in care for racial/ethnic minorities.
Twenty-seven studies met criteria for review. Almost all (n = 26) took place in the primary care setting, and most (n = 19) focused on improving provision of preventive services. Only two studies were designed specifically to meet the needs of racial/ethnic minority patients. All 10 studies that used a provider reminder system for provision of standardized services (mostly preventive) reported favorable outcomes. The following quality improvement strategies demonstrated favorable results but were used in a small number of studies: bypassing the physician to offer preventive services directly to patients (2 of 2 studies favorable), provider education alone (2 of 2 studies favorable), use of a structured questionnaire to assess adolescent health behaviors (1 of 1 study favorable), and use of remote simultaneous translation (1 of 1 study favorable). Interventions employing more than one main strategy were used in 9 studies with inconsistent results. There were limited data on the costs of these strategies, as only one study reported cost data.
There are several promising strategies that may improve health care quality for racial/ethnic minorities, but a lack of studies specifically targeting disease areas and processes of care for which disparities have been previously documented. Further research and funding is needed to evaluate strategies designed to reduce disparities in health care quality for racial/ethnic minorities.
In recent years, it has become clear that the healthcare system in the United States does not provide the same quality of care for minority populations that it does for the majority white population . The Institute of Medicine (IOM) report "Unequal Treatment" confirmed that racial and ethnic disparities in healthcare are not entirely explained by differences in access, clinical appropriateness, or patient preferences . Additionally, there is increasing evidence that provider behaviors and practice patterns contribute to disparities in care  and that healthcare organizational processes also compromise quality .
Despite awareness of inequities in healthcare quality, little is known about strategies that have the potential to improve the quality of healthcare for ethnic minority populations. For those interested in quality improvement, there is a need for an evaluation and synthesis of the strategies proven to be effective in bettering the quality of healthcare for minorities. Moreover, it is unknown whether strategies specifically designed to reduce disparities in healthcare between racial/ethnic minorities and whites (in contrast to those simply aimed at improving quality) have been implemented successfully.
The purpose of this study was to systematically review the best evidence to determine the effectiveness and costs of interventions designed to improve the quality of healthcare and/or to reduce disparities for racial/ethnic minorities. Our study focuses on evaluations of interventions aimed at healthcare providers, as recent work suggests provider and organizational factors contribute substantially to the inequities . Furthermore, our study focuses on controlled trials, rather than on evaluations with a weaker study designs, because we were interested to learn which quality improvement strategies had been evaluated most rigorously.
We conducted a systematic review of the literature, using formal methods of literature identification, selection of relevant articles, data abstraction, quality assessment and synthesis of results, to determine which strategies improve the quality of care for racial/ethnic minorities have been implemented and are effective, and which strategies have been shown to reduce disparities in care . We separately synthesized articles that evaluated the cultural competence education of health professionals, and report those results elsewhere .
In addition to electronic searching, we identified priority journals that provided the largest number of citations in the electronic searching, and we scanned the tables of content from February 1, 2003 through June 15, 2003. We also scanned the reference lists of key review articles that were identified in our electronic searching and all articles eligible for our report.
The results of the searches were imported into ProCite, a reference management software program. This database was used to store citations, track search results and sources, and track the abstract and article review process.
The following criteria were used to exclude articles: published prior to 1980, not in English, did not include human data, contained no original data, a meeting abstract only (no full article for review), not specific to minority health (overall patient population less than 50% minority or no subgroup analysis), did not occur in the United States, no intervention, intervention not targeted to healthcare providers or organizations, no evaluation of the intervention, not a randomized controlled trial or concurrent controlled trial, or article did not apply to any of the study questions. The term 'minority' was defined as all non-Caucasian or non-white racial and ethnic categories, including African American, Hispanic, American Indian/Alaskan Native, and Asian/Pacific Islander.
Two team members independently reviewed the title and abstract of all citations identified through the literature search. When reviewers agreed that an article was eligible or a decision regarding eligibility could not be made because of insufficient information, the full article was retrieved for review. When reviewers disagreed on eligibility, citations were returned for adjudication.
We developed standardized review forms to (1) confirm eligibility for full article review, (2) assess study and participant characteristics and (3) extract the relevant data to address the study questions. The forms were developed through an iterative process that included the review of forms used for previous systematic reviews, discussions among team members and experts, and pilot testing.
For each eligible study, we abstracted data regarding the targeted health care providers and setting, the patient population, the intervention objectives and methods, the measured outcomes, and the intervention effects. We used a conceptual framework adapted from Cooper et al.  to categorize the types of outcomes into the following categories: use of services, quality of providers, appropriateness of care, efficacy of treatment, patient adherence, patient ratings of care, and health status. For example, 'quality of providers' included such outcomes as provider knowledge and communication behaviors. We also designed several questions to assess methodological rigor of the studies in each of five domains: representativeness, bias/confounding, intervention description, outcome assessment, and analysis.
We used a serial review process such that a primary reviewer completed the quality assessment and data abstraction forms and a second reviewer, after reading the article, checked each item on the form for completeness and accuracy. Differences between primary and secondary reviewers were resolved by adjudication and, when necessary, consultation with the entire team of reviewers.
Once all articles were reviewed and data were synthesized, the strength of the evidence supporting each question was graded into four categories (Grades A-D) based on its quality, quantity, and consistency . To meet the quality criteria for Grade A, there must have been at least one randomized controlled trial. To meet the quality criteria for Grade B, C, or D there must have been at least one controlled trial (not necessarily randomized).
For quantity of studies, there had to be at least 4 studies available in the literature to meet criteria for Grade A, 3 studies to meet criteria for Grade B, 2 studies to meet criteria for Grade C, or fewer than 2 studies to meet criteria for Grade D. For consistency, the results of the studies had to be consistent (either beneficial or harmful results in same direction across studies) to meet criteria for Grade A, reasonably consistent to meet criteria for Grade B (most study results in the same direction), and inconsistent to meet criteria for Grade C. If there were too few studies to judge the consistency (i.e. fewer than three studies), the strength of evidence supporting the question was given a grade of D. The grading of the evidence was discussed at a team meeting and consensus was reached on each criterion. The evidence received a final "grade" that reflected the lowest rank on each of the three criteria (quality, quantity and consistency).
Literature search and review process
Results of the literature search and review process are summarized in Figure 1. Of the 4,833 citations identified through search processes, 3,709 were uniquely identified. Of these, 281 were eligible for full article review. After review, 254 were excluded and a total of 27 articles met eligibility criteria [9–35].
Description of studies
Summary of 27 studies evaluating interventions to improve the quality of healthcare for racial/ethnic minorities
Hospital Outpatient Setting
Community Health Center
>one of the above
Chronic kidney Disease
Acute Respiratory Tract Infections
Emergency Medical Systems
End of Life Care
Targeted Health Providers
Physicians and Non-physicians
>50% African American
>50% Asian/Pacific Islander
>50% American Indian/Alaskan Native
Main Provider Intervention
Structured Patient Questionnaire
Remote Simultaneous Translation
Defibrillators on Emergency Vehicles
> one of the above (Multifaceted)
Publication dates and study designs
Of the 27 studies eligible for review, only three studies were published before 1990 [9–11], 20 were published between 1990 and 1999 [12–31], and four were published after 2000 [32–35]. All studies were randomized controlled trials (n = 20) or concurrent controlled trials (n = 7).
The majority of articles were in the area of prevention. The most common types of preventive services targeted by the interventions were breast cancer screening (n = 11) and cervical cancer screening (n = 6). There were three studies published in the area of mental health (two in depression and one in alcohol abuse) and one each in the area of chronic kidney disease, asthma, acute respiratory tract infections, emergency medical systems, and advance directives.
Subjects and settings
Almost all studies were targeted at physicians. Two studies were not targeted at physicians: one was directed solely at nurses and medical assistants and the other was aimed at emergency medical personnel. The specialty of the targeted physicians was most often internal medicine (n = 7), but there were also general primary care (n = 3), pediatrics (n = 3), family medicine (n = 2), adolescent medicine (n = 1), and one or more of the above (n = 9). The interventions targeted practicing professionals (n = 15), professionals in training (n = 6), or both (n = 6). All interventions occurred in the outpatient setting.
Most studies had more than 50 percent African American patients (n = 19) [9, 10, 12–17, 19, 21, 23, 25–28, 30–32, 35]; only two had patient populations that were specified as more than 50 percent Hispanic [18, 20]. and none had more than 50 percent Asian/Pacific Islander patients or more than 50 percent American Indian/Alaska Native patients. The remaining six studies had diverse groups (but no more than 50 percent in any one racial/ethnic category) [11, 22, 24, 29, 33, 34].
All studies had a provider-targeted intervention, and most studies used more than one provider intervention method. The primary intervention was a tracking/reminder system in ten studies (in which the clinician is given an automated reminder that a particular service might be due), provider education in two studies, bypassing the physician using nurse/nurse practitioners to offer standardized services directly to patients in two studies, provision of a structured patient questionnaire to facilitate patient-provider communication about patient health behaviors in one study, use of remote simultaneous translation (in which the interpreter translates simultaneously with the speaker but is not in the exam room) in one study, use of subspecialty consultation in one study, and use of defibrillators on emergency medical vehicles in one study. There were nine studies that used more than one main method (multifaceted). Approximately half (n = 14) of the studies had a patient intervention component, although these studies varied in whether the patient intervention was provided in addition to the provider intervention (n = 10) or compared to the provider intervention (n = 4). The intervention was tailored specifically for racial/ethnic minorities in only two studies [18, 34].
The most common outcomes were related to healthcare process: use of services (7 studies, 13 outcomes), appropriateness of care (18 studies, 43 outcomes), quality of providers (9 studies, 30 outcomes), patient adherence (4 studies, 9 outcomes), and efficacy of treatment (1 study, 1 outcome). Patient health status (7 studies, 21 outcomes), and patient ratings of care (3 studies, 3 outcomes) were also measured.
Quality of reviewed studies
Summary of selected aspects of study quality for the 27 studies aimed at improving quality of health care for racial/ethnic minorities
Selected Quality Assessment Domains and Items
Articles (Total N = 27)
- Healthcare providers clearly described?
- Adequate comparison group?
- Adequate randomization?
- Complete description (able to replicate)?
- Pre- and post-test?
- Numbers and reasons of non-inclusion?
- Complete statistical analysis?
Effectiveness of interventions to improve the quality of healthcare for minorities
Summary of interventions, clinical areas, and outcomes of 27 studies aimed at improving quality of health care for racial/ethnic minorities
Clinical Area (references)
Total # Studies
Overall Outcome Assessment across Studies
Rating of Evidence
General Adult Prevention (9,10)
All studies reported favorable outcomes.
Cancer Screening (10,15,19,21,25,35)
Tobacco Cessation (30)
End of Life (31)
Adult Cancer Screening (28,29)
Although almost all studies reported some favorable impact on one or more outcome type, results were inconsistent across studies
Tobacco Cessation (27)
Alcohol Cessation (20)
Upper Respiratory Tract Infections (33)
Cancer Screening (12,23)
Both studies reported favorable outcomes in the provision of preventive services to patients
Adult General Prevention (17)
Both studies reported favorable impact on provider counseling behaviors.
Child Injury Prevention (32)
Use of STQ2
Adolescent Health Risks (16)
One study reported favorable impact on provider counseling behavior.
Use of RST3
Well Baby Care (18)
One study reported favorable outcomes on accuracy of translation and provider/patient satisfaction.
Use of Specialty Consult
Chronic Renal Disease (26)
One study did not demonstrate improvement.
Use of Defibrillators
Emergency Medicine (13)
One study did not demonstrate improvement.
Ten studies used tracking and/or reminder systems to improve quality of care. Of these, two were in adult general prevention [9, 10], six were in adult cancer screening [10, 15, 19, 21, 25, 35], one in tobacco cessation , and one was in end-of-life care (completion of advance directives) . All ten studies demonstrated positive outcomes, primarily in the appropriateness of care (such as provision of preventive care, tobacco cessation counseling, or advance directive counseling) category. Overall, there is excellent evidence supporting the use of tracking/reminder systems aimed at providers of racial/ethnic minority patients (Evidence Grade A).
Nine studies used an intervention that we characterize as multifaceted, meaning that there two or more (usually more) main intervention methods [14, 20, 22, 24, 27–29, 33, 34]. Examples of these types of intervention are detailed in the supplementary table [See Additional file 1]. Two of these interventions were in adult cancer screening [28, 29], one in tobacco cessation , one in cholesterol reduction , two in depression [14, 34], one in alcohol cessation , one in acute upper respiratory tract infections , and one in asthma . Outcomes of these studies are mixed, with most studies showing improvements in one or two (but not all) outcomes measured. Overall, there is fair evidence supporting the use of multifaceted interventions aimed at providers of racial/ethnic minority patients (Evidence Grade C).
Bypass the physician
Two studies (both in adult cancer screening) bypassed the physician and had either a nurse or a nurse practitioner offer screening directly to patients, and both studies demonstrated improvements in the provision of preventive services to patients [12, 23]. Overall, there is fair evidence (favorable results but only two studies) supporting the use of bypassing the providers of racial/ethnic minority patients to offer standardized services directly to patients (Evidence Grade C).
Two studies used provider education as the main intervention strategy, one in the area of adult general prevention  and one in prevention of injuries in children . Both studies found improvements in provider counseling behaviors [17, 32], but one measured and did not find any positive effect of the intervention on parental knowledge of injury prevention or parental adherence to provider advice . Overall, there is fair evidence supporting the use of provider education aimed at providers of racial/ethnic minority patients (Evidence Grade C).
Use of Safe Times Questionnaire (STQ)
One study (in the area of prevention for children) used a structured questionnaire to assess adolescent health behaviors, provided the results of the questionnaire to the provider, and demonstrated a positive impact on provider counseling behaviors . However, because of the insufficient number of studies using this method, there is poor evidence supporting the use of structured questionnaires for racial/ethnic minority patients (Evidence Grade D).
Use of Remote Simultaneous Translation (RST)
One study compared the accuracy of translation and quality of patient-physician communication by using remote simultaneous and proximate consecutive interpretation and found fewer translation errors and enhanced patient and physician satisfaction by using the RST method . However, because of the insufficient number of studies using this method, there is poor evidence supporting the use of remote simultaneous translation for racial/ethnic minority patients (Evidence Grade D).
Use of specialty consultation
One study evaluated the use of nephrology consults for patients with chronic kidney disease and found no effect on healthcare process or patient outcomes . However, because of the insufficient number of studies using this method, there is poor evidence supporting the use of specialty consults to improve the quality of care for racial/ethnic minority patients (Evidence Grade D).
Use of defibrillators on emergency medical services
One study evaluated the use of defibrillators on emergency medical services and found no effect on patient outcomes . Overall, there is poor evidence supporting the use of defibrillators on emergency medical services (Evidence Grade D).
Effectiveness of interventions to reduce disparities
Only one study specifically addressed the question of whether an intervention could reduce disparities in healthcare quality between ethnic minority and white persons . The study, which evaluated the impact of two different culturally tailored interventions to improve the quality of depression care compared with a control group that received no intervention, had mixed results. There was no differential effect of the interventions on healthcare process for white versus ethnic minority patients; all patients (African-American, Latino, and white) in the interventions groups were more likely than patients in the control group to receive appropriate therapy. However, there was a mixed effect on health outcomes: there were improvements for African-American and Latino patients in the rate of depression remission compared to controls (with no improvement for white patients), but there were no improvements for African-American and Latino patients in the intervention groups in employment rates compared with controls (with improvement for white patients). Overall, there is poor evidence to determine which interventions might reduce disparities between racial/ethnic minority patients and majority patients (Evidence Grade D).
Costs of quality improvement for racial/ethnic minorities
Only one study reported on the costs of an intervention aimed at improving the quality of healthcare for racial/ethnic minority persons . This study, which provided case management and nephrology consultation for patients with chronic renal insufficiency, estimated that it cost a minimum of US $89,355 yearly in 1998 (or $484 per intervention patient), but it found no health benefits to participants. Overall, there is poor evidence to determine the cost of strategies to improve the quality of care for racial/ethnic minorities (Evidence Grade D).
We rigorously evaluated strategies to improve the quality of care for racial/ethnic minority patients in a select group of studies. Almost all the interventions occurred in the primary care setting, and most focused on the provision of preventive services. Only two of the studies were specifically targeted at the needs of racial/ethnic minority patients [18, 34]; the remaining studies were generic quality improvement strategies. Despite this, there is excellent evidence that provider tracking/reminder systems are effective in improving quality of care for racial/ethnic minority patients (Evidence Grade A), fair evidence that multifaceted interventions, provider education interventions, and interventions that bypass the physician to offer screening services to racial/ethnic minority patients can improve quality of care (Evidence Grade C), and poor evidence for the use of any other of the studied intervention strategies (Evidence Grade D).
Tracking and reminder systems were effective in improving rates of standardized services such as cancer screening [10, 15, 19, 21, 25, 35] and advance directive completion . Although these do not represent all clinical areas for which disparities in care have been documented and thus may hold little potential for addressing overall disparities, the strategy appears effective in improving the process of care for racial/ethnic minorities. In some studies, though, some of the processes of care targeted by tracking and reminder systems were not evidence-based practices for any patient population (for example, oral cavity exams or breast self-examinations for cancer screening) and would therefore be unlikely to improve the quality of care or reduce disparities for racial/ethnic minority patients.
Our review identifies important gaps in knowledge that provide a focus for future research. Regarding strategies worthy of further study, there were several types of interventions with favorable results, but employed in only one or two studies each, thus receiving a grade of either D or C. These are: (1) bypassing the physician to offer standardized services directly to patients [12, 23], (2) use of remote simultaneous translation for patients with limited English proficiency  and (3) the use of a structured questionnaire (Safe Times Questionnaire) for health behaviors risk assessment in adolescents . Of these, the study that evaluated remote simultaneous translation is particularly germane to the needs of some racial/ethnic minorities, and could have widespread impact if the results are replicated in other studies. Also, both of the studies evaluating provider education yielded favorable results [17, 32]; however, other studies have suggested that passive educational interventions are not optimal .
We found poor evidence to determine which strategies are most effective in reducing disparities between minority and white populations (Evidence Grade D). The only study that was specifically designed to do this had mixed results; with improvements in only one of the two outcomes assessed . This represents a critical gap in the literature. More research is needed that is designed specifically to reduce disparities in healthcare quality, for example, research that targets specific diseases (e.g. cardiovascular disease, diabetes, HIV) and healthcare processes known to be a source of racial/ethnic disparities. It may be necessary to distinguish between interventions aimed at improving the quality of care for all persons and those aimed at improving quality of care for racial/ethnic minority populations specifically (such as reducing provider bias or improving intercultural communication skills). When generic quality improvement interventions are done in diverse populations, subgroup analyses that evaluate the effect of the interventions in racial/ethnic minority patients would increase our understanding the effect on equity of treatment.
Many organizations have limited resources to accomplish the goal of improving minority health care quality. Our review found poor evidence to determine the costs of strategies to improve care and reduce disparities for minority populations (Evidence Grade D). Only one study included an estimate of cost. In order to make resource allocation decisions for ethnic minority patient populations in the future, data on the costs of these interventions will be critical.
Additional knowledge gaps to be filled by future research were related to targeted groups, settings, and health outcome assessments. First, there are very few quality improvement interventions that were completed in Hispanic populations and none in American Indians/Alaska Natives or in Asians/Pacific Islanders. Second, almost all studies were done in the primary care setting. Insofar as disparities have been documented in other settings, more studies may be needed in acute care and specialty settings. Finally, few studies measured patient outcomes; most measured healthcare process. This limitation would not be as important if all studies had targeted processes of care that were evidence-based and more closely linked to patient outcomes. Studies need to include patient outcomes, have longer follow-up to determine the sustainability of intervention effects, and link process of care to health outcomes.
There were some challenges in synthesizing this body of literature. First, studies that used multifaceted interventions and did not examine separate components, making it difficult to know exactly what resulted in the beneficial effect observed. Second, each study used slightly different intervention methods, making generalizations across studies difficult. Because no two studies used exactly the same intervention and evaluation strategy, there is a need to replicate and evaluate promising intervention strategies in different healthcare settings and organizations. In addition, multifaceted interventions should be evaluated in such a way so as to distinguish which specific piece of the intervention was most effective.
This review has several limitations. First, eligibility was limited to English language and to published reports of studies. Although our resources did not permit extensive searching of the non-English language and unpublished literature, recent work has suggested that results of reviews with these limits do not differ substantially from reviews with no such limits . There is, however, a possibility of publication bias, in that studies with demonstrated benefit are more likely to be published than those with no benefit. This is a limitation of any systematic review.
Eligibility was also limited to articles published after 1980, and to studies conducted in the United States. There may have been other promising interventions conducted in other countries that are not reflected in this report. Only randomized controlled trials and concurrent controlled trials were included. Although researchers have recommended the use of experimental designs with control groups to evaluate interventions to reduce disparities whenever possible , there have been other worthwhile interventions that have been evaluated with other study designs . Our review, which found a paucity of rigorous clinical trials, suggests that other types of studies should be considered by those interested in designing interventions to reduce disparities. To the extent that other QI strategies are used without a sound evidence base, it may be unfair to hold interventions designed to reduce disparities to a different standard. On the other hand, it is a worthwhile goal that all QI interventions have demonstrated effectiveness. Our review only included interventions targeting providers; interventions directly targeting patients may also be promising strategies to improve the quality of care and reduce racial/ethnic disparities, but they are not reflected in this report.
In conclusion, there are several promising strategies that may improve health care quality for racial/ethnic minorities, but a lack of studies specifically targeting the disease areas and the processes of care for which disparities have been previously documented. This body of research should be a national priority as it will aid in clinical decision-making and health care resource allocation.
This article is based on research conducted by the Johns Hopkins Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality (Contract No. 290-02-0018), Rockville, MD. The authors of this article are responsible for its contents, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or of the U.S. Department of Health and Human Services.
- Mayberry RM, Mili F, Ofili E: Racial and ethnic differences in access to medical care. Med Care Res Rev. 2000, 108-145. 10.1177/107755800773743628. 57 Suppl 1
- Smedley BD, Stith AY, Nelson AR: Unequal Treatment: Confronting racial and ethnic disparities in health care. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. 2003, Washington, DC: Institute of MedicineGoogle Scholar
- van Ryn M: Research on the provider contribution to race/ethnicity disparities in medical care. Med Care. 2002, I140-I151. 40 Suppl 1
- Wennberg DE: Variation in the delivery of health care: the stakes are high. Ann Intern Med. 1998, 128: 866-868.View ArticlePubMedGoogle Scholar
- Beach MC, Cooper LA, Robinson KA, Price EG, Gary TL, Jenckes MW, Gozu A, Smarth C, Palacio A, Feuerstein CJ, Bass EB, Powe NR: Strategies for Improving Minority Healthcare Quality. 2004, Rockville, Maryland: Agency for Healthcare Research and Quality, [Evidence Report/Technology Assessment No. 90], [http://www.ahrq.gov/clinic/epcsums/minqusum.htm]Google Scholar
- Beach MC, Price EG, Gary TL, Robinson KA, Gozu A, Palacio AM, Smarth C, Jenckes MW, Feuerstein CJ, Bass EB, Powe NR, Cooper LA: Cultural Competence: A Systematic Review of Health Care Provider Educational Interventions. Med Care. 2005, 43: 356-373. 10.1097/01.mlr.0000156861.58905.96.View ArticlePubMedPubMed CentralGoogle Scholar
- Cooper LA, Hill MN, Powe NR: Designing and evaluating interventions to eliminate racial and ethnic disparities in health care. J Gen Intern Med. 2002, 17: 477-486. 10.1046/j.1525-1497.2002.10633.x.View ArticlePubMedPubMed CentralGoogle Scholar
- West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, Lux L: Systems to rate the strength of scientific evidence. 2002, Rockville, Maryland: Agency of Healthcare Research and Quality, [Evidence Report/Technology Assessment No. 47], [http://www.ahrq.gov/clinic/epcsums/strengthsum.htm]Google Scholar
- McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP: Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984, 100: 130-138.View ArticlePubMedGoogle Scholar
- Turner BJ, Day SC, Borenstein B: A controlled trial to improve delivery of preventive care: physician or patient reminders?. J Gen Intern Med. 1989, 4: 403-409.View ArticlePubMedGoogle Scholar
- Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM: Microcomputer-generated reminders. Improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989, 29: 273-280.PubMedGoogle Scholar
- Mandelblatt J, Traxler M, Lakin P, Thomas L, Chauhan P, Matseoane S, Kanetsky P: A nurse practitioner intervention to increase breast and cervical cancer screening for poor, elderly black women. The Harlem Study Team. J Gen Intern Med. 1993, 8: 173-178.View ArticlePubMedGoogle Scholar
- Kellermann AL, Hackman BB, Somes G, Kreth TK, Nail L, Dobyns P: Impact of first-responder defibrillation in an urban emergency medical services system. JAMA. 1993, 270: 1708-1713. 10.1001/jama.270.14.1708.View ArticlePubMedGoogle Scholar
- Callahan CM, Hendrie HC, Dittus RS, Brater DC, Hui SL, Tierney WM: Improving treatment of late life depression in primary care: a randomized clinical trial. J Am Geriatr Soc. 1994, 42: 839-846.View ArticlePubMedGoogle Scholar
- Burack RC, Gimotty PA, George J, Stengle W, Warbasse L, Moncrease A: Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994, 32: 609-624.View ArticlePubMedGoogle Scholar
- Schubiner H, Tzelepis A, Wright K, Podany E: The clinical utility of the Safe Times Questionnaire. J Adolesc Health. 1994, 15: 374-382. 10.1016/1054-139X(94)90260-7.View ArticlePubMedGoogle Scholar
- Gemson DH, Ashford AR, Dickey LL, Raymore SH, Roberts JW, Ehrlich MH, Foster BG, Ganz ML, Moon-Howard J, Field LS: Putting prevention into practice. Impact of a multifaceted physician education program on preventive services in the inner city. Arch Intern Med. 1995, 155: 2210-2216. 10.1001/archinte.155.20.2210.View ArticlePubMedGoogle Scholar
- Hornberger JC, Gibson CD, Wood W, Dequeldre C, Corso I, Palla B, Bloch DA: Eliminating language barriers for non-English-speaking patients. Med Care. 1996, 34: 845-856. 10.1097/00005650-199608000-00011.View ArticlePubMedGoogle Scholar
- Burack RC, Gimotty PA, George J, Simon MS, Dews P, Moncrease A: The effect of patient and physician reminders on use of screening mammography in a health maintenance organization. Results of a randomized controlled trial. Cancer. 1996, 78: 1708-1721. 10.1002/(SICI)1097-0142(19961015)78:8<1708::AID-CNCR11>3.0.CO;2-1.View ArticlePubMedGoogle Scholar
- Burge SK, Amodei N, Elkin B, Catala S, Andrew SR, Lane PA, Seale JP: An evaluation of two primary care interventions for alcohol abuse among Mexican-American patients. Addiction. 1997, 92: 1705-1716. 10.1111/j.1360-0443.1997.tb02891.x.View ArticlePubMedGoogle Scholar
- Burack RC, Gimotty PA: Promoting screening mammography in inner-city settings. The sustained effectiveness of computerized reminders in a randomized controlled trial. Med Care. 1997, 35: 921-931. 10.1097/00005650-199709000-00005.View ArticlePubMedGoogle Scholar
- Evans D, Mellins R, Lobach K, Ramos-Bonoan C, Pinkett-Heller M, Wiesemann S, Klein I, Donahue C, Burke D, Levison M, Levin B, Zimmerman B, Clark N: Improving care for minority children with asthma: professional education in public health clinics. Pediatrics. 1997, 99: 157-164. 10.1542/peds.99.2.157.View ArticlePubMedGoogle Scholar
- McCarthy BD, Yood MU, Bolton MB, Boohaker EA, MacWilliam CH, Young MJ: Redesigning primary care processes to improve the offering of mammography. The use of clinic protocols by nonphysicians. J Gen Intern Med. 1997, 12: 357-363. 10.1046/j.1525-1497.1997.00060.x.PubMedPubMed CentralGoogle Scholar
- Keyserling TC, Ammerman AS, Davis CE, Mok MC, Garrett J, Simpson R: A randomized controlled trial of a physician-directed treatment program for low-income patients with high blood cholesterol: the Southeast Cholesterol Project. Arch Fam Med. 1997, 6: 135-145. 10.1001/archfami.6.2.135.View ArticlePubMedGoogle Scholar
- Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, Dews P, Coombs J: How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998, 82: 2391-2400. 10.1002/(SICI)1097-0142(19980615)82:12<2391::AID-CNCR13>3.0.CO;2-K.View ArticlePubMedGoogle Scholar
- Harris LE, Luft FC, Rudy DW, Kesterson JG, Tierney WM: Effects of multidisciplinary case management in patients with chronic renal insufficiency. Am J Med. 1998, 105: 464-471. 10.1016/S0002-9343(98)00329-5.View ArticlePubMedGoogle Scholar
- Allen B, Pederson LL, Leonard EH: Effectiveness of physicians-in-training counseling for smoking cessation in African Americans. J Natl Med Assoc. 1998, 90: 597-604.PubMedPubMed CentralGoogle Scholar
- Manfredi C, Czaja R, Freels S, Trubitt M, Warnecke R, Lacey L: Prescribe for health. Improving cancer screening in physician practices serving low-income and minority populations. Arch Fam Med. 1998, 7: 329-337. 10.1001/archfami.7.4.329.View ArticlePubMedGoogle Scholar
- Dietrich AJ, Tobin JN, Sox CH, Cassels AN, Negron F, Younge RG, Demby NA, Tosteson TD: Cancer early-detection services in community health centers for the underserved. A randomized controlled trial. Arch Fam Med. 1998, 7: 320-327. 10.1001/archfami.7.4.320.View ArticlePubMedGoogle Scholar
- Ahluwalia JS, Gibson CA, Kenney RE, Wallace DD, Resnicow K: Smoking status as a vital sign. J Gen Intern Med. 1999, 14: 402-408. 10.1046/j.1525-1497.1999.09078.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Dexter PR, Wolinsky FD, Gramelspacher GP, Zhou XH, Eckert GJ, Waisburd M, Tierney WM: Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med. 1998, 128: 102-110.View ArticlePubMedGoogle Scholar
- Gielen AC, Wilson ME, McDonald EM, Serwint JR, Andrews JS, Hwang WT, Wang MC: Randomized trial of enhanced anticipatory guidance for injury prevention. Arch Pediatr Adolesc Med. 2001, 155: 42-49.View ArticlePubMedGoogle Scholar
- Harris RH, MacKenzie TD, Leeman-Castillo B, Corbett KK, Batal HA, Maselli JH, Gonzales R: Optimizing antibiotic prescribing for acute respiratory tract infections in an urban urgent care clinic. J Gen Intern Med. 2003, 18: 326-334. 10.1046/j.1525-1497.2003.20410.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Miranda J, Duan N, Sherbourne C, Schoenbaum M, Lagomasino I, Jackson-Triche M, Wells KB: Improving care for minorities: can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized, controlled trial. Health Serv Res. 2003, 38: 613-630. 10.1111/1475-6773.00136.View ArticlePubMedPubMed CentralGoogle Scholar
- Burack RC, Gimotty PA, Simon M, Moncrease A, Dews P: The effect of adding Pap smear information to a mammography reminder system in an HMO: results of randomized controlled trial. Prev Med. 2003, 36: 547-554. 10.1016/S0091-7435(02)00062-2.View ArticlePubMedGoogle Scholar
- Committee on quality of health care in America: Crossing the quality chasm: A new health system for the 21st century. 2001, Washington, DC: National Academy of SciencesGoogle Scholar
- Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA: Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001, 39: II2-45. 10.1097/00005650-200108002-00002.View ArticlePubMedGoogle Scholar
- Egger M, Juni P, Bartlett C, Holenstein F, Sterne J: How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003, 7: 1-76.PubMedGoogle Scholar
- Sehgal AR: Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA. 2003, 289: 996-1000. 10.1001/jama.289.8.996.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/6/104/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.