The effectiveness of knowledge translation strategies used in public health: a systematic review

Background Literature related to the effectiveness of knowledge translation (KT) strategies used in public health is lacking. The capacity to seek, analyze, and synthesize evidence-based information in public health is linked to greater success in making policy choices that have the best potential to yield positive outcomes for populations. The purpose of this systematic review is to identify the effectiveness of KT strategies used to promote evidence-informed decision making (EIDM) among public health decision makers. Methods A search strategy was developed to identify primary studies published between 2000–2010. Studies were obtained from multiple electronic databases (CINAHL, Medline, EMBASE, and the Cochrane Database of Systematic Reviews). Searches were supplemented by hand searching and checking the reference lists of included articles. Two independent review authors screened studies for relevance, assessed methodological quality of relevant studies, and extracted data from studies using standardized tools. Results After removal of duplicates, the search identified 64, 391 titles related to KT strategies. Following title and abstract review, 346 publications were deemed potentially relevant, of which 5 met all relevance criteria on full text screen. The included publications were of moderate quality and consisted of five primary studies (four randomized controlled trials and one interrupted time series analysis). Results were synthesized narratively. Simple or single KT strategies were shown in some circumstances to be as effective as complex, multifaceted ones when changing practice including tailored and targeted messaging. Multifaceted KT strategies led to changes in knowledge but not practice. Knowledge translation strategies shown to be less effective were passive and included access to registries of pre-processed research evidence or print materials. While knowledge brokering did not have a significant effect generally, results suggested that it did have a positive effect on those organizations that at baseline perceived their organization to place little value on evidence-informed decision making. Conclusions No singular KT strategy was shown to be effective in all contexts. Conclusions about interventions cannot be taken on their own without considering the characteristics of the knowledge that was being transferred, providers, participants and organizations.


Participants (continued)
Characteristics: Participants were child and youth mental health practitioners working in publicly funded community based service provider organizations in Ontario. Participants were mostly female (89.2%), and had on average 9 years of experiences as a clinician (7 years among PaU group; 10.8 years among CoP group). Four participants had graduate level education, 8 had bachelors level training, 14 had diplomas or certifications in social work, social service work, child and youth care, or early childhood education, and there was one registered nurse (7 participants did not provide level of education data).
Loss to follow-up: 14 lost to follow up (6 in study group; 8 in control)

Intervention
Interventions: Community of practice-Established group of people sharing knowledge, learning together, and creating common practices.

Description of Intervention:
Session 1: The facilitator explained the purpose of the CoP is to support and develop the practice surrounding the use of the CAFAS tool. Participants were oriented to the various roles that help set-up, develop, nurture, and sustain the community, and set the stage for its sustainability. Members worked together and participated actively. There was also a key role for a content expert, who acted as a resource to the community when needed.
Sessions 2-6: Group invited to shape the agenda for the meetings. Conversation built in which advice, opinions, and information were offered, again situated in practice. Productive inquiry initiated the actions of knowledge access, knowledge exchange, and knowledge creation. The knowledge needed and shared was triggered by a real situation connected to practice.
Practice: 20-item questionnaire regarding respondents self reported use of CAFAS implementation supports reduced to a total CAFAS supports score. Responses were 'yes', 'no', or 'don't know/does not apply' Validity and reliability not reported.
Practice: 10-question Likert scale questionnaire to assess the degree of self-reported change reduced to a total practice change score. Items were rated as 'very much', 'somewhat', 'very little' or 'not at all'. Validity and reliability not reported.

Objective
To evaluate the effectiveness of three dissemination strategies (Pamphlets, CD-ROM, Internet) related to prevention program materials.

Design: Randomized controlled trial
Recruitment: Three adolescent substance abuse prevention programs were identified and illustrative dissemination materials were compiled for each. These materials were disseminated to school personnel, community providers, and policy makers. First by mailed letter invitation, then by telephone follow-up, sites were offered the opportunity to participate in the study.
Inclusion/Exclusion: Sites included schools, community agencies, policy making bodies and youth services agencies. Sites agreeing to participate were asked to identify professionals on staff to complete assessments at planned intervals and to review materials for three youth-oriented substance abuse prevention programs.
Allocation: Grouped by site, consenting professionals were stratified and matched on their constituency (school, agency, policy-making body) and geographic location. Matched triads of sites were randomly assigned to one of three arms: pamphlet, CD-ROM, or Internet.

Participants Total Sample: N=188 professionals
Intervention groups:

Pamphlet n=55
CD-ROM n=64 Internet n=69 Characteristics: The participants were professionals employed in schools, community agencies, and policy-making bodies. Schools were defined as public and independent educational facilities at the middle and junior high levels.
Community agencies were defined as private non-profit organizations that provide youth with human services including school dropout, delinquency, and pregnancy prevention; day treatment, juvenile probation and parole; educational tutoring; and recreational, neighbourhood, and club activities.
Policy-making organizations were government legislative, analytic, funding, and regulatory bodies that were at least in part dedicated to the provision or recommendation of drug abuse prevention services for youth.
Professionals included teachers, social workers, and other management and executive-level personnel who exercised decision-making power over the selection and application of adolescent drug abuse prevention programs. Respondents from target constituencies tended to be female, between the ages of 30 and 49 years, white, and well educated with close to half of respondents (48%) holding graduate degrees.

Intervention
Interventions: Printed materials and information in CD-ROM or internet format tailored to prevention needs

Description of Intervention:
Information was synthesized about three youth-oriented substance abuse prevention programs and a common presentation format for delivering this content via pamphlet, CD-ROM, and Internet was developed.
Materials described the rationale, strategies, and costs to prevent drug abuse, and the roles of schools, professionals, and community groups, and relevant private and government bodies in addressing this problem. Materials were tailored to be responsive to their differing prevention needs. Constituency-specific content was delivered to respondents in the CD-ROM and Internet arms.
Following receipt of completed pre-tests, professionals in the respective study arms were sent the pamphlet, CD-ROM, or logon name, password, and instructions for Internet access.

Description of Control: No control group
Intervention Duration: Participants had 6 months to review materials before first follow up measurement took place Characteristics: Participants were from participating regional and local public health departments in Canada and were directly responsible for making program decisions related to healthy body weight promotion in children. This included program managers and/or coordinators in Ontario, and program directors in the rest of Canada. Participation by province and territory ranged from 29% to 100% with the sample consisting primarily of health departments serving both urban and rural populations (46%).

Loss to follow-up:
Intervention: (TM) n=6 (KB) n=7 Control: (HE) n=7 Follow-up data were collected from 88 of 108 (81.5%) participating public health departments Over seven successive weeks, on the same day each week and the same time of day, participants in the TM group were sent an email indicating that a systematic review related to healthy body weight promotion in children was available in full text at the link provided.
Participants received access to the PDF version of the systematic review, the published abstract of the review, as well as the short summary written. The text of the message was worded to say, 'this message is number XX in a series of seven emails you will receive on healthy body weight promotion in children as part of the KTE strategy you are being exposed to in this randomized controlled trial.
KB group: Included both the HE and TM components and a KB who worked one on one with decision makers in the public health departments. The KBs were Master's prepared, had extensive knowledge and expertise in public health decision making, as well as an understanding of the research process.
Specific tasks conducted by the KB included: ensuring relevant research evidence related to healthy body weight promotion was transferred to the public health decision makers in ways that were most useful to them, assisting them to develop the skill and capacity for evidence-informed decision making, and assisting them in translating evidence into local practice. Approximately twenty percent of KB time was spent facilitating knowledge and skill development either through face-to-face interaction such as workshops or online strategies such as webinars, interactive web enabled meetings, or conferences. Eighty percent of the brokers' time was spent preparing for and directly interacting with participants.

Description of Control:
HE group: Least interactive KTE strategy. HE group had access to health-evidence.ca which is a repository of systematic reviews evaluating any public health intervention. All participants in the study received electronic communication about the availability of this site. Upon searching this site for reviews evaluating strategies to promote healthy body weight in children, those in the HE group would have become aware of the title, citation, and assessment of the methodological quality of seven systematic reviews evaluating the effectiveness of interventions to promote healthy body weight in children. Participants in the HE group also had access to the published abstracts, and the full text articles and a short summary for each of the systematic reviews, written by the research team, with key findings and recommendations for public health policy and practice directly applicable to the types of decisions for which the participants were responsible.

Providers: Researchers, Professionals
Site: Workplace

Change in Practice
Randomized Controlled Trials (4)

Outcome Measurement Tool
Telephone-administered survey (knowledge transfer and exchange data collection tool). Reported reliability 0.65 Cronbach alpha.
Practice: Global Evidence-Informed Decision Making-Mean self report score on the extent to which research evidence was considered in a recent program planning decision in the previous 12 months. Responses ranging from one (not at all) to seven (completely).
Practice: Public Health Policies and Programs-Respondents asked whether the public health policies and programs were being implemented by their health department (yes/no). The total number was summed.

Study Limitations
(Items mentioned by review authors not already identified in risk of bias assessment)

Study authors:
 Self-reported outcome measures  Participants may have not been aware of all public health policies and programs provided by their organization leading to both under and over reporting of this outcome  Variable exposure to intervention-Up to 30% of participants did not engage with the KB at all or to a limited extent  Participants who completed baseline measurements were different in follow up surveys in 30% of departments

Review authors:
 Questionnaire only reported as satisfactory Cronbach alpha of 0.65  Not described how exposure to knowledge broker was estimated  Using two different knowledge brokers could have led to differences between groups using that intervention Study Authors: Forsetlund, L. Bradley, P., Forsen, L., Nordheim, L., Jamtvedt, G., Bjørndal, A.

Objective
The aim of this study was to evaluate whether a tailored theory-based and multifaceted intervention targeted at the whole process of evidence-based practice increased the explicit integration of research in public health physicians' decision-making

Recruitment:
The invitation letters explained that project participants would have free access to a library service. In return, they would be asked to return questionnaires and examples of written reports to be used for programme evaluation. Participants were also informed that some would be asked to co-operate further during the project period. Recruitment was stopped when 73 had been allocated to the intervention group and 75 to the control group, fulfilling the number of the sample size calculations.
Inclusion/exclusion: All public health physicians working in municipalities in Norway with more than 3000 inhabitants (N = 332) were invited to participate in the project.
Allocation: Public health physicians were enrolled by the primary author upon receipt of the consenting letter. Enrolled physicians were subsequently randomized to one of two groups by an independent researcher using computer software.

Control group n=75
Characteristics: Participants were public health physicians working in municipalities in Norway. Public health physicians in Norway are geographically scattered; one physician in each of the country's 435 municipalities. The sample was physicians who were predominately male, were on average 47 years of age and had been working in the field on average 12 years in the intervention group vs 9.5 years in the control group, working experience in rural and urban settings. More physicians in the intervention group had previously attended sessions in critical appraisal.

Intervention
Interventions: workshop, information service, discussion list, access to databases

Description of Intervention:
The intervention program was intended to lead the participants from the first knowledge stage to the confirmation stage when adoption was to occur based on innovation-diffusion process.
Workshop: Interactive small-group setting involving small group problem-based activities and discussion. Involved posing and formulating questions, searching skills, critical appraisal and practical application of research evidence in practice.

Goal-Setting Contract:
Physicians were asked to state three things that they would change when returning to practice.

Intervention Information Services (including library access):
Included ongoing support, access to several databases and consisted of: a question and answer service where upon submitting a questions physicians would receive references or reports based on relevant studies found; access to course material and how to practice evidence-based public health; and links to other sources of information on evidence-based practice.
Discussion List: Discussion stimulated by giving general reminders, providing and asking for feedback and allocating peer discussion. Providers announced when reports had been written and critically appraised selected articles. Participants were reminded of ongoing support services.
Newsletters: Three newsletters reported on principles of evidencebased health care and project activities, including feedback on database use.

Description of Control:
Participants in the control group received free access to library services for one year. First component: a presentation that contained factual statements, statistics and graphs taken from key Department of Health publications highlighting and supporting the guideline evidence base. The presentation was designed to convey positive normative beliefs that all staff adhere to the guideline and expect other staff to adhere.
Second component: group discussion facilitated to ensure that positive normative beliefs were emphasized and any negative normative beliefs challenged.
Third component: comprised group work on two real life vignettes developed in consultation with the professional head of nursing: one depicting an episode of care in which the guideline had been adhered to and a near-miss for a service-user avoided, and one in which the guideline had not been adhered to and there had been a negative outcome.

Providers: Training coordinators'
Site: Conducted at each community mental health teams' base  Some discontinuity occurred between those who returned the questionnaire and those who attended the intervention  Staff turnover was a problem at the intervention site  Through using the audit adherence data aggregated across the mental health directorate it was not being possible to break the data down to the level of the individual health professionals  The timing of the local event made it difficult to isolate the effects of this from the intervention

Review Authors:
 How sites were picked is not addressed  Unclear who control group participants were  Procedure for outcome measurement not stated  Could not use data related to questionnaire because measured "intention"