Table 4 HDHK process evaluation procedures
| Â | Process Evaluation Question | Data Sources | Tools/Procedures | Timing of Data Collection | Data Analysis or Synthesis | Reporting |
|---|---|---|---|---|---|---|
Fidelity | To what extent was the intervention implemented consistently with the underlying theory and philosophy? # | Facilitators HDHK research staff | * Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content * Observation checklist of delivered content from independent observer to validate facilitator self-report checklist | * At the conclusion of each session (Checklists will be faxed to research staff post-session) * At least two observations per seven session program | * Calculated score based on percentage of intended content included * Percentage difference score of researcher versus facilitator average of intended content included | * Formative - informal feedback to staff prior to the next intervention session * Summative - reported after the program is complete |
Dose delivered | To what extent were all intervention components implemented? | Facilitators HDHK research staff | * Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content (theoretical and practical) * Observation checklist of delivered content from independent observer to validate facilitator self-report checklist | * At the conclusion of each session (Checklists will be faxed to research staff post-delivery) * At least two observations per seven session program | * Calculated score based on percentage of intended content included * Percentage difference score of researcher versus facilitator average of intended content included | * Formative - informal feedback to staff prior to the next intervention session * Summative - reported after the program is complete |
Dose received | How did participants react to the activities presented during the intervention? | Participants | * Participant evaluation of each of the attended sessions (enjoyment, amount learnt, ease of understanding content, relevance to self, likelihood of strategy use, likes and suggested changes) * Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources) | * Post-session * Post-program | * Quantitative analysis of session scores; Qualitative analysis of likes and suggestion themes per session and across the program | * Summative - reported after the program is complete |
| Â | To what extent did participants engage in recommended follow-up behavior? | Participants | * Father's log book and green slip homework activities; Kid's log book | * Father's log book (post-program); Kids log book (during and post program); Father's green slip homework (weekly) | * Level of follow-up activity completed included as a factor during quantitative analysis | * Summative - reported after the program is complete |
Reach | What proportion of fathers attended greater than 70% of sessions (5 of 7)? | Facilitators | * Session roll for attendance | * Recorded at each session | * Level of attendance included as a mediator during quantitative analysis | * Summative - reported after the program is complete |
Recruitment | What recruitment activities were used to attract organizations and participants? What recruitment activities attracted participants to the program? | HDHK research staff HDHK research staff | * Marketing spread sheet maintained by research staff (target market, activity, date, response); Recruitment/screening spread sheet recording the date of first contact by a potential participant * Identification of the most influential marketing method by the potential participant and any other marketing they had been exposed to | * Each time recruitment activities were undertaken * Recorded at initial phone screening of potential participant | * Narrative description of procedures; graphing of marketing methods and participant response dates to identify successful trends * Narrative description of influential recruitment strategies; Calculate percentages of most influential strategies leading to participant contact with the research team | * Formative - reported weekly to identify responses to marketing methods used; Summative - reported after the program is complete to inform next cohort * Formative - reported weekly to identify responses to marketing methods used; Summative - reported after the program is complete to inform next cohort |
Context | What barriers to participation were experienced by the participants? | Facilitators using participant information | * Focus groups as part of the program to identify barriers to implementing the information/activities covered during the program | * Session 6 of 7 (facilitators to record the barriers identified by participants) | * Themes identified through qualitative analysis | * Summative - reported after the program is complete |
| Â | What aspects of the program did and didn't appeal to the participants? | Participants | * Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources) | * Post-program | * Themes identified through qualitative analysis and statistical analysis of questionnaire | * Summative - reported after the program is complete |