Skip to main content

Advertisement

Table 1 Process evaluation approaches, implications and lessons from seven case studies

From: Process evaluation in the field: global learnings from seven implementation research hypertension projects in low-and middle-income countries

Study and Country Intervention/ target population Process Evaluation objectives and theoretical approach Methods/ data sources Main findings Implications for project and policy Lessons for process evaluation
Cost effectiveness of salt reduction programs in the Pacific Islands (Fiji and Samoa) Government policy initiatives to reduce salt in foods and meals and community mobilization for behavior change to reduce salt intake. Understanding reach, fidelity, dose, context, feasibility and lessons for future programs Mixed methods: Interviews, collection of routine monitoring data, sub-analysis of population salt survey data Trial outcome: No statistically significant reductions in mean salt intake. Increased awareness of negative impact of salt and some improvements in salt-use behavior. 1. Complex policy interventions need more time and clear implementation strategies 2. Multi-sectoral interventions require strong government leadership 1. Challenging and time consuming to collect and analyse data but adds to learning 2. Stakeholder interviews and measures of the implementation process should be done throughout the intervention period to inform necessary adaptations 3. Using mixed methods and several data sources allows for cross-checking and triangulation to enhance the validity of process evaluation data
A pre-post study design. Approach: MRC process evaluation framework
Project (Fiji): August 2012 –August 2016 Publication: January 2018 Project (Samoa): (January 2013 – December 2015), Publication: (August 2018) Setting: Population wide. Process evaluation: Intervention not implemented with full fidelity within timescale due to contextual factors. However, research capacityincreased, salt reduction mainstreamed in government policies and mechanisms for engaging industry established.
Treating hypertension in rural South Africa: A clinic-based lay health worker to enhance integrated chronic care in Mpumalanga South Africa A cluster randomized controlled trial. Lay health workers (LHW) improving management of hypertension by undertaking simple tasks and freeing nurses to focus on clinical work. Understanding process of developing the intervention, implementation context and mechanisms and processes that led to changes in patient outcomes Mixed methods: Observations, focus group discussions, interviews, diaries Trial outcome: Population control of hypertension did not improve 1. Strong management, skilled LHW, functional equipment and good relations, are essential for success in task shifting 1. Realist evaluations can complement and be combined with randomized controlled trials. 2. Study sites that are within the same geographical area can be diverse in context.
Process evaluation: LHW Intervention made clinics function better and increased patients’ adherence to appointment. Health system factors affected implementation.
Project (April 2013 – December 2015 Publication: (November 2017) Setting: rural clinics Approach: Realist evaluation
Diagnosing hypertension—Engaging Action and Management in Getting Lower BP in Indigenous communities in Canada and rural communities in Tanzania) Health care SMS text messages supporting patient hypertension self-management and facilitating decision support for health care providers. To assess the major active components of the intervention, technology of the intervention, task shifting, cultural congruence and unintended consequences as part of formative research to inform the intervention strategies Mixed methods: Research notes for implementation. Reflective discussion sessions with researchers and community, interviews and focus groups Trial outcomes: Ongoing 1. Important to prepare intervention using local knowledge 2. Need to establish ongoing dialogue between community and researchers 3. Identify strengths and challenges for implementation. 1. Community Based Participatory Research values local knowledge, cultural understanding 2. Formative research should be part of process evaluation in trials.
Formative research: Showed discrepancies between text messages created by researchers and how recipients felt about them. This informed development of text messages congruent to population’s motivation for behavior change.
A prospective randomized trial. Setting: indigenous communities in Canada and rural communities in Tanzania
Project (January 2012 – January 2017) Publication: (April 2017) Approach: Community based participatory research
Optimizing linkage and retention to hypertension care to reduce blood pressure in rural western Kenya - Kosirai and Turbo A cluster randomized controlled trial Project (April 2012 – March 2017) Publication: (January 2016) Community Health Workers (CHW) equipped with a tailored behavioral communication strategy and smartphone technology to increase linkage and retention of people with hypertension Determining fidelity, CHW knowledge, skill retention, attitudes, patient perceptions and barriers to care linkage and retention. Mixed methods: Survey, focus group discussions, collection of process indicators, written tests and clinical examinations. Trial outcomes: Study completed, but analyses still underway. 1. Continuing training, support, and surveillance necessary for program fidelity 1. Multi-modality approach (qualitative and quantitative; combination of observations, discussions, and testing) is critical in complex intervention 2. Process evaluation should start earlier 3. Mixed methods captures areas of data discordance and concordance
Process Evaluation: Low implementation fidelity among CHWs. Initial CHW training significant in some areas and not others. Skills retention among CHWs was sub-optimal
Setting: rural communities in Kenya Approach: RE-AIM framework and Realist evaluation
Salt substitute to reduce blood pressure at the population level in northern Peru A stepped wedge trial Project (March 2012 – March 2017) Publication: (July 2017) Replacing high sodium salt for a salt substitute (low-sodium, high-potassium salt) to reduce blood pressure (BP) among those 18 years and over. To understand strategies for incorporating salt substitute, barriers and facilitators to introducing salt substitute and perceived health impact for salt substitute In-depth interviews Trial outcomes: Population-level BP was reduced, 1. Acceptance of a new product was gradual 2. Important to target those preparing meals 3. Acceptance was increased by providing a substitute for salt without financial cost. 1. Process evaluation data to be collected repeatedly, at the beginning and throughout the implementation time
Process evaluation: Good relations between community and researchers facilitated uptake of the salt substitute. Women did not consult family members when they introduced salt substitute
Setting: six villages Approach: Phenomenology: understanding study participants’ and other stakeholders’ perception
Improving control of hypertension in rural India: Overcoming barriers to diagnosis and effective treatment A cluster randomized controlled feasibility trial Project (January 2014 – October 2016), Publication (May 2018) Training Accredited Social Health Activists (ASHAs) to support patients with hypertension to adopt self-management behaviours and improve BP control Assessing fidelity and factors that might have impacted on the ASHA training outcomes Mixed methods: Interviews, focus group discussions, intervention meeting reports. Trial outcomes: Study completed, but analyses still underway. 1. Need for culturally appropriate training materials for NCDs 2. Importance of adequate interactive and innovative training, retraining, and supervision 3. Need for timely remuneration and incentives for ASHAs 1. Mixed methods gave an in-depth understanding of the intervention 2. Difficult to collect data in 3 different cultural sites using the same methods
Process evaluation: ASHA training was feasible and led to changes in knowledge, skills, and motivation. ASHAs delivered the intervention appropriately. Community appreciated their roles
Setting: 3 regions in rural India Approach: Kirkpatricks’ four level evaluation model
Early use of low-dose triple combination of BP lowering drugs in improving BP control in Sri Lanka A randomized controlled trial • Project: (February 2014–December 2017), Publication: August 2018 Simplified treatment regimen of a half -dose, three-in-one blood pressure lowering pill (Triple pill) for improving hypertension control Investigating factors and their interplay behind the results, patients and providers experience and barriers and facilitators to implementation. Semi-structured interviews Trial outcomes: More patients in the intervention arm reached their blood pressure targets compared to usual care. 1. Training of General Practitioners for the management of hypertension including combination therapy 2. Ensuring the availability of combination therapy 1. Process evaluation data collected before trial results are available, helps in exploring views uninfluenced by trial results.
Setting: patients with hypertension in outpatient departments in 11 urban hospitals Approach: Framework analysis method Process evaluation: Patients and providers liked the triple pill because of its ease of use (single pill, once a day dosing) and significant BP control. No major safety issues were reported.