Skip to main content

Table 4 HDHK process evaluation procedures

From: The 'Healthy Dads, Healthy Kids' community effectiveness trial: study protocol of a community-based healthy lifestyle program for fathers and their children

  Process Evaluation Question Data Sources Tools/Procedures Timing of Data Collection Data Analysis or Synthesis Reporting
Fidelity To what extent was the intervention implemented consistently with the underlying theory and philosophy? # Facilitators
HDHK research staff
* Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content
* Observation checklist of delivered content from independent observer to validate facilitator self-report checklist
* At the conclusion of each session (Checklists will be faxed to research staff post-session)
* At least two observations per seven session program
* Calculated score based on percentage of intended content included
* Percentage difference score of researcher versus facilitator average of intended content included
* Formative - informal feedback to staff prior to the next intervention session
* Summative - reported after the program is complete
Dose delivered To what extent were all intervention components implemented? Facilitators
HDHK research staff
* Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content (theoretical and practical)
* Observation checklist of delivered content from independent observer to validate facilitator self-report checklist
* At the conclusion of each session (Checklists will be faxed to research staff post-delivery)
* At least two observations per seven session program
* Calculated score based on percentage of intended content included
* Percentage difference score of researcher versus facilitator average of intended content included
* Formative - informal feedback to staff prior to the next intervention session
* Summative - reported after the program is complete
Dose received How did participants react to the activities presented during the intervention? Participants * Participant evaluation of each of the attended sessions (enjoyment, amount learnt, ease of understanding content, relevance to self, likelihood of strategy use, likes and suggested changes)
* Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources)
* Post-session
* Post-program
* Quantitative analysis of session scores; Qualitative analysis of likes and suggestion themes per session and across the program * Summative - reported after the program is complete
  To what extent did participants engage in recommended follow-up behavior? Participants * Father's log book and green slip homework activities; Kid's log book * Father's log book (post-program); Kids log book (during and post program); Father's green slip homework (weekly) * Level of follow-up activity completed included as a factor during quantitative analysis * Summative - reported after the program is complete
Reach What proportion of fathers attended greater than 70% of sessions (5 of 7)? Facilitators * Session roll for attendance * Recorded at each session * Level of attendance included as a mediator during quantitative analysis * Summative - reported after the program is complete
Recruitment What recruitment activities were used to attract organizations and participants?
What recruitment activities attracted participants to the program?
HDHK research staff
HDHK research staff
* Marketing spread sheet maintained by research staff (target market, activity, date, response); Recruitment/screening spread sheet recording the date of first contact by a potential participant
* Identification of the most influential marketing method by the potential participant and any other marketing they had been exposed to
* Each time recruitment activities were undertaken
* Recorded at initial phone screening of potential participant
* Narrative description of procedures; graphing of marketing methods and participant response dates to identify successful trends
* Narrative description of influential recruitment strategies; Calculate percentages of most influential strategies leading to participant contact with the research team
* Formative - reported weekly to identify responses to marketing methods used;
Summative - reported after the program is complete to inform next cohort
* Formative - reported weekly to identify responses to marketing methods used; Summative - reported after the program is complete to inform next cohort
Context What barriers to participation were experienced by the participants? Facilitators using participant information * Focus groups as part of the program to identify barriers to implementing the information/activities covered during the program * Session 6 of 7 (facilitators to record the barriers identified by participants) * Themes identified through qualitative analysis * Summative - reported after the program is complete
  What aspects of the program did and didn't appeal to the participants? Participants * Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources) * Post-program * Themes identified through qualitative analysis and statistical analysis of questionnaire * Summative - reported after the program is complete
  1. # Underlying theory and philosophy is embedded in the content given to facilitators to deliver the participants. If all content is delivered as per the program package, consistence of theory and philosophy is said to have been achieved.