Skip to main content

Table 1 Qualitative and quantitative instruments for process, formative, and effectiveness evaluation in the SHIFT project

From: Safety and Health through Integrated, Facilitated Teams (SHIFT): stepped-wedge protocol for prospective, mixed-methods evaluation of the Healthy Workplace Participatory Program

Instrument Description Content Administration Frequency
Process Evaluation
Mobile App, “HWPP Assistant” Meeting assessments and time logs by DT and SC members • DT members and co-facilitators: participationa, knowledge, satisfaction, team effectiveness, engagementa, value
• DT co-facilitators: confidence in facilitating; quality and usability of HWPP materials
• SC members: awareness of team developments, organizational disturbances, team effectiveness, value, engagement
• Log of project-related time (level of participationa)
After each meeting
Coach Notes; Research Assistant Notes Systematic, qualitative observations of DT, SC, and joint DT/SC meetings • Attendancea
• Team dynamicsa
• Quality of Design Team facilitation (DT only)
• Level of management support (DT only)
• Progress through the structured IDEAS intervention design processa
• Any difficulties with the IDEAS process; any adaptations made
After each meeting
Design Team Meeting Records Person-time invested; Fidelity of DT meetings to HWPP protocol • DT meeting datesa
• Time spent in meetingsa
• Number of attendeesa
• Agenda items covered from the protocol
After each meeting
Formative and Effectiveness Evaluation: Short and Medium-term Outcomes
IDEAS Worksheets DT work record of progress on IDEAS steps (summary worksheet) • Content of discussions and interventions at each step: knowledge, breadth of topics covered over timea After IDEAS Step 5
Process Check-up DT and SC members: periodic survey evaluating the HWPP • Organizational support and engagementa
• DT and SC engagementa
• Program facilitation
• Intervention planning/implementation
After IDEAS Step 5, then every 6 months during coaching
Design Team Survey DT survey on members’ attitudes and perceptions of HWPP • Member engagementa; Involvementa
• Managerial support for DT interventions
• Shared DT/SC knowledge and understanding about root causes of safety and health concerns
• Quality of organizational communication
• Expected HWPP impact
• Any unplanned consequences observed (negative or positive)
After IDEAS Step 5, then every 6 months during coaching
Interim Champion & Facilitator Interviews Semi-structured interviews for champions and co-facilitators • Satisfaction regarding program implementation
• Quality of and trends in DT and SC effectiveness, engagementa, involvementa, communication and organizational support
• Expectations regarding impacts of the participatory program overall and from the interventions being designed and implemented by the teams
Every 6 months during coaching
Formative and Effectiveness Evaluation: Longer-term Outcomes
All-Employee Survey Self-administered survey (paper and pencil) for all facility employees • Self-rated health, body weight, health behaviors, perceived health needs, attitudes and beliefs
• Symptoms of burnout and depression, history of injury and chronic disease
• Location/severity of musculoskeletal symptoms, related functional impairment
• Perceptions of working conditions, physical and psychological job features, organizational culture and climate
• Knowledge of and perceived effectiveness of health and safety programs
Baseline & study end
Organizational Readiness Survey SC members, DT co-facilitators, and union leaders: organization’s readiness to form participatory DTs and resources available to support the HWPP (45-item survey) • Existing programs for safety, health, and well-being
• Current program approaches for safety, health, and well-being
• Current resources available for safety, health and wellbeing
• Resources and readiness for change initiatives to improve safety, health and wellbeing
• Resources and readiness for use of teams
• Teamwork in work groups
• Resources and readiness for employee participation
• Management communication about safety, health, and well-being
Baseline & study end
Leadership Interviews Scripted, open-ended interviews for SC members, DT co-facilitators, union leaders • Perceived workforce safety and health concerns
• Employee involvementa in, successes of, and challenges to the existing safety program
• Current strengths, opportunities, and obstacles to the HWPP within the facility
• Shared DT/SC knowledge and understanding about root causes of safety and health concerns
• Expected HWPP impact
• Any unplanned consequences observed (negative or positive)
Baseline & study end
Pre-Post Intervention Surveys DT-administered surveys • Topic-specific measures based on key performance indicators (e.g., reduction of workplace exposures, short-term changes in behavior). Workforce sample determined by scope of intervention. Pre- and post-IDEAS interventions
  1. aprimary outcome