Focus group discussions as a method for investigating group opinions
Focus group discussions were used as the primary method for investigating opinions on the societal impact of research publications in the present study. This approach is useful for obtaining information about knowledge, ideas, opinions and attitudes in a community or group, particularly when little preknowledge about an issue is available [23, 24]. As compared to individual interviews, group discussions enhance creativity because they allow for interaction among group members. They are therefore appropriate for the identification of problem-based solutions [24]. The interactive component has been discussed to "empower" research participants because they allow participants to become an active part in the process of developing ideas [24]. Focus group discussions, however, also have important downsides. The interaction of group members which may on the one hand enhance creativity, may on the other hand silent individual voices of dissent [24]. Therefore, a well-considered approach is necessary when composing groups, conducting the interviews and interpreting results [23, 24].
Formation of the Societal Impact Factor Task Force
The Centre for Public Health comprises 5 departments dedicated to general practice and family medicine, social medicine, environmental health, epidemiology and medical psychology, respectively. An invitation to join a task force for the development of a tool to assess the societal impact of publications based on focus group discussions was sent by email to all scientific employees of the Centre for Public Health, Medical University Vienna in December 2007 by the moderator Manfred Maier (MM). Twenty-four scientific employees, with representatives from all departments, accepted the invitation and formed part of the task force (for names and affiliations, see acknowledgement section).
Mixed purposeful sampling strategy
The outlined composition of the Centre is comparable to many other international academic public health institutions, which makes the Centre an appropriate place to study opinions among a purposeful typical case sample of scientists in an academic public health setting [25]. In order to purposefully eliminate highly influential opinions, which may disproportionally influence the content of the discussions, the opinions of the department chairmen were analyzed in an opinion leader group discussion separate from the other participants [23]. The other groups each involved 5 to 7 scientific employees from different departments. Researchers from different departments were mixed to ensure a more heterogeneous sampling of individuals within the relatively homogenous setting of the Centre, which is expected to result in more diverse opinions and the breaking up of hierarchical structures which may influence the contents of the discussions [23, 24].
Conduction of the focus group discussions
Two rounds of 8 focus group discussions in total, each involving 5 to 7 members of the task force, were conducted between May 2008 and May 2009.
The first round of focus group discussions was dedicated to the questions of (1) what an ideal tool to measure the societal impact should be like, and (2) what specific indicators should be considered when quantitatively assessing a societal impact of publications. The focus groups typically started with an outline of the general purpose of the discussion, and some general warm up questions, followed by a clarification of terms used in the discussion when necessary. The moderator (MM) encouraged all participants to actively participate in the discussion. The discussions involved individual statements by the participants and group interaction.
The contents of the discussions were recorded in writing by an assistant and/or the moderator.
The qualitative data for each group discussion was analyzed inductively with qualitative content analysis, and codes were assigned to the themes related to the explored questions occurring in the discussions. Themes that were mentioned repeatedly were then used to construct a preliminary tool, which was a questionnaire for the self-assessment by authors of the societal impact of own publications. To increase precision, the moderator and 2 other task force members (TN, TED) collaborated on the transformation of the qualitative codes into the self-assessment form. The self assessment form was then sent out to the Task Force members who were asked (1) to apply the form to one own selected publication and (2) to review the self-assessments of two or three colleagues. The members were asked to provide feedback on (1) their perceptions on the suitability of the tool, (2) practical experiences in the application of the tool to his or her own publications, and (3) practical experiences in the assessment of publications from other authors in a further round of focus group meetings, following the same approach as in the first round. This feedback was then used for a revision of the self-assessment questionnaire. A code book defining the terms used for the assessment and the variables (codes) used to assess the societal impact was developed based on the contents of the focus group meetings, in addition to an information sheet for reviewers and authors which aimed at supporting authors and reviewers in their tasks within the application process.
Exercise to evaluate the tool
To perform a first exercise related to the validation of the tool developed, six international external experts with leading roles in academic public health were invited to test the tool and to provide structured comments. TN and TED developed a questionnaire which allowed the experts to rate the comprehensibility, the relevance and the practicability of the variables used in the societal impact factor assessment questionnaire, and the comprehensibility and usefulness of the glossary and of the information to authors and reviewers on 4-degree Likert scales (1: 'comprehensive", 'very relevant", 'very practicable', 'very appropriate', 'very useful', and 4: 'not comprehensive at all', 'not relevant at all", "not practical at all', 'not appropriate at all', 'not useful at all'). Whenever 3 or 4 was ticked, the experts were asked to specify the reasons of their choice. In addition, the questionnaire included open-ended comments related to the tool, e.g. the time needed for the assessment.
This questionnaire, together with three representative publications from different research areas at the Centre for Public Health, the respective self-assessment forms by authors, a short version of the code book to be used as a glossary, and an information sheet for authors and reviewers as a support material was sent to 6 external experts. Publication A used for this test was a national health report on diabetes mellitus type 2 in Austria [26]. In this report epidemiological data on diabetes, risk factors and complications were compiled and analysed, and national prevention strategies and policy implications were made [27]. Publication B dealt with mental health promotion and behavioural medicine. It analysed the impact of media guidelines for reporting on suicides on the quality of suicide reporting and on suicide rates in Austria [28]. Publication C was a survey on the allocation of training posts to applicants for postgraduate medical education in Austria [29]. The experts were selected based on their leading academic roles at departments of Public Health or General Practice (for names and affiliations, see acknowledgement section).
Ethics statement
The study was conducted in compliance with the Helsinki Declaration. All participants in the focus group discussions are investigators and co-author the paper and the drafted tool. The need to obtain ethical consent for this study was specifically waived by the Institutional Review Board at the Medical University of Vienna, Austria.
Comments
View archived comments (1)