Research article | Open | Open Peer Review | Published:
Time trends in the impact factor of Public Health journals
BMC Public Healthvolume 5, Article number: 24 (2005)
Journal impact factor (IF) is linked to the probability of a paper being cited and is progressively becoming incorporated into researchers' curricula vitae. Furthermore, the decision as to which journal a given study should be submitted, may well be based on the trend in the journal's overall quality. This study sought to assess time trends in journal IF in the field of public, environmental and occupational health.
We used the IFs of 80 public health journals that were registered by the Science Citation Index from 1992 through 2003 and had been listed for a minimum period of the previous 3 years. Impact factor time trends were assessed using a linear regression model, in which the dependent variable was IF and the independent variable, the year. The slope of the model and its statistical significance were taken as the indicator of annual change.
The IF range for the journals covered went from 0.18 to 5.2 in 2003. Although there was no statistical association between annual change and mean IF, most of the fastest growing journals registered mean IFs in excess of 1.5, and some represented emerging areas of public health research. Graphs displaying IF trends are shown.
In view of the delay between the publication of IFs and that of any given paper, knowing the trend in IF is essential in order to make a correct choice of journal.
Scientific journal impact factor (IF) is directly linked to the probability of a paper being cited. It is currently accepted that a higher IF indicates a better journal quality, existing a correlation among IF and quality indicators at least in some health disciplines . As a result these indices are progressively becoming incorporated into researchers' curricula vitae. In Spain, publication in top-IF journals has occupational implications in terms of academic careers and obtaining research grants . The most widespread and important bibliometric indicators are those referring to the repercussion of scientific activity, and among these, IF has a leading role .
This pressure means that when it comes to having to select a journal in which to publish their studies, researchers turn to journals with IF, and assess the possibility of publishing in those that have the highest IF possible. Journals having the top IF within each medical specialty tend to be those with the greatest international prestige and highest profile, i.e., the most widely read by researchers and most in demand to publish their studies. However, the publication of extraordinary relevant scientific findings are concentrated in a small list of core journals, most of them not directly related with the specialty of the author [1, 4]
Positive evaluation of IF which failed to take account of its trend over time would tend to favor publications of recent, but not necessarily lasting, interest. In contrast, publications of steadily growing interest and stable impact would be undervalued, though there may be differences of opinion about this .
Hence, in addition to journal quality, the decision as to which journal a manuscript should be submitted may be based on the trend in its IF over time. The aim of this study was to analyze IF time trends of journals in the "Public, Environmental and Occupational Health" category, thereby furnishing a new criterion on which to base the choice of journal for publication.
For study purposes, we selected 80 journals that: were included in the "Public, Environmental and Occupational Health" category of the hard copy version of the Journal Citation Reports (JCR) from 1992 through 2003 ; were listed for a minimum period of 3 consecutive years; and had IFs in the JCR for 2003. We consulted the 2003 JCR-IFs via the ISI Web of Knowledge .
The impact factor is one of the quantitative tools provided by JCR for ranking, evaluating, categorizing, and comparing journals. The annual impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years .
Impact factor time trends were assessed using a linear regression model, in which the dependent variable was IF and the independent variable, the year. The slope of the model (index of annual change-IAC) and its statistical significance were taken as the indicator of year-to-year variation.
Shown in Table 1 is a list of all journals included, ranked by their impact factor in 2003. This table also shows the index of annual change (slope of the model) and its statistical significance. Table 2 shows the same list, but ranked this time according to the IAC. This method of ranking can prove useful, since, by comparing the IF trends between two journals with similar IFs, the better choice would be the journal with the better index of annual change.
The IF range for the journals covered went from 0.18 to 5.2 in 2003. Although there was no statistical association between annual change and mean IF, most of the fastest growing journals registered mean IFs in excess of 1.5 (Journal of Womens Health and Gender Based Medicine IAC = 0.388 p = 0.007, Journal of Urban Health IAC = 0.316 p = 0.002, Epidemiology IAC = 0.250 p < 0.001, Drug Safety IAC = 0.238 p < 0.001), and some represented emerging areas of public health research (Table 2).
Figure 1 depicts IF trends on a multiple graph, thereby allowing a quick idea to be formed of the evolution of the indicator in all the journals included.
If journals that publish review papers are excluded, then the "American Journal of Epidemiology" ranked first in 1991, a position occupied in 1992 by a recently created publication with a fairly specific content matter, the "Cancer Epidemiology Biomarkers & Prevention", in tandem with a journal of similar orientation and seniority, viz., "Cancer Causes and Control". The rise of both these journals may be indicative of current priorities in epidemiologic research [8, 9]. This upward trend pattern among journals addressing cancer epidemiology remained in evidence throughout the study period.
In 2003 the epidemiology journals with the highest IF ranged from 1.5 through 4.5, though it should be stressed that special caution is called for when dealing with the individual journal lists for the respective medical specialties.
The use of IF as an indicator of a journal's profile or prestige has become widespread among researchers, editors, libraries, and even among the agencies that fund research. Nevertheless, this indicator has a number of limitations that have been extensively debated in the literature [5, 10]. Thus, for instance, journals that publish review papers receive a high number of citations and their impact factors are particularly high. It should be pointed out here that the ISI seeks to offer an overview of international science, with the result that journals covering topics or disciplines of more local interest are scarcely covered. Within each category, therefore, it is frequent for journals that are more basic -and thus of universal interest- to be associated with a higher impact factor than those that are more applied -and so of more local interest- given that the latter's circulation is more restricted. The publication of extraordinary relevant scientific findings are concentrated in a small list of core journals, most of them not directly related with the specialty of the author [1, 4]. The mere quality of the documents published by a journal, albeit essential, will not suffice for it to be cited. The number of citations can be enhanced using management techniques, such as expanding a journal's international circulation, raising its profile in databases and on web pages, and increasing the number of papers. It becomes necessary for journals to be known among the international community and attain sufficient prestige to be subsequently cited. The policy pursued in Spain in recent years aimed at fostering high-quality, competitive science has induced Spanish scientists to bypass Spanish journals and send their publications instead to journals enjoying a wide international circulation, something that is often associated with journals having the greatest IF.
Our results suggest that it would be of interest to add the index of annual change to the criteria used for selecting a journal for publication. Studies conducted in another JCR area have shown the importance of analyzing IF trends by category . IF trend might, however, be determined by certain factors that should be discussed.
The journals analyzed may be assigned to more than two categories in the SCI-JCR. The Public Health category changed names in 1996, and from 1997 onwards was called "Public, Environmental and Occupational Health", thus explaining why the number of titles jumped from 61 to 73 from one year to the next (Table 3).
Similarly, until the year 2000, the "Epidemiology" category came within the umbrella of "Public Health", thereafter disappearing and falling within the category of study without any specific entry. The categories covered by the JCR have changed in name and number over the years and, logically, this is equally true of the journals that comprise them.
In addition to the comments regarding the list of journals included in the category of study, it would be a wise decision on the ISI's part if allocation of journals to categories were made in agreement with the researchers . However, it is not the intention of this paper to dwell on the use and abuse of the IF or the arbitrariness of the study-category journal list, an aspect previously analyzed in connection with the public health sphere .
This study solely included journals listed for 3 years in the JCR. Recently launched journals with a policy geared to novel publication, such as the BMC group, were not taken into consideration. The "BMC Public Health" journal has been listed for two years, with IFs of 0.29 in 2002 and 0.93 in 2003, and plots a growth pattern similar to that of journals addressing emerging issues.
In general terms and in view of Figure 1, the linear model seems to be adequate, inasmuch as there are very few journals with trends that display evident turning points. Many of the journals maintain their trend over the years. Most of them (60%) add 0.03 points or more to their IFs every year and there are very few that register a decline with time; indeed this is statistically significant in only one instance. A number of categories can be drawn up based on the trend pattern, namely: long-standing journals with the top IF, which maintain their trend; new journals focused on emerging issues, which seem to enjoy good acceptance; journals that maintain a very stable intermediate ranking; and a small group that has witnessed a decrease in their respective IFs.
The reasons why a journal changes its IF trend has been commented before and some of them does not have any relationship with a better quality of its papers. Probably a better or worst management of the journal also is related with the changes in the citations trend and could deserve some study.
In the publication of a scientific paper, a long time elapses between deciding upon a journal and the date of publication: on average, more than one year can go by. In view of the stability of the indicators in the area of study targeted, relying upon IF time trends in order to choose a particular journal might perhaps not be very relevant. Yet it may be a critical aspect in other areas of science where there are increases of around one point per year. For researchers/authors who know only too well how costly it can prove to see their paper published and are, moreover, aware that it is going to be evaluated on the basis of concepts as abstract as the impact and number of their publications, this decision is important.
Leaving aside the speculative components of the choice, and given the delay that accumulates between the publication of IFs and that of any given paper, knowing the trend in the IF is yet another factor that will help authors make the correct choice of journal.
McKibbon KA, Wilczynski NL, Haynes RB: What do evidence-based secondary journals tell us about the publication of clinically important articles in primary healthcare journals?. BMC Med. 2 (1): 33-10.1186/1741-7015-2-33. 2004 Sep 06
Camí J, Suñén E, Méndez R: Caracterización bibliométrica de Grupos de Investigación Biomédica en España. IMIM-UPF-ISCIII. 2003, [http://www.isciii.es/paginas/fis/mapa/index.htm]
Rice RE, Borgman CL, Bednarski D, Hart PJ: Journal-to-journal citation data: issues of validity and reliability. Scientometrics. 1989, 15: 582-8.
Weiner JM, Shirley S, Gilman NJ, Stowe SM, Wolf RM: Access to data and the information explosion: oral contraceptives and risk of cancer. Contraception. 1981, 24 (3): 301-13. 10.1016/0010-7824(81)90043-3.
Seglen PO: Why the impact factor of journals should not be used for evaluating research. BMJ. 1997, 314: 498-502.
INSTITUTE FOR SCIENTIFIC INFORMATION: Journal Citation Reports on microfiche 2001. Science Edition. Philadelphia (versions relating to previous years also consulted). 2002
ISI Web of Knowledge: Journal Citation Reports, (database on the Internet, consulted in September 2004). INSTITUTE FOR SCIENTIFIC INFORMATION, Philadelphia. 2003, [http://go5.isiknowledge.com/]
Hasbrouck LM, Taliano J, Hirshon JM, Dannenberg AL: Use of epidemiology in clinical medical publications, 1983–1999: a citation analysis. Am J Epidemiol. 2003, 157: 399-408. 10.1093/aje/kwf218.
Fernández E: Factor de impacto bibliográfico 1993. Salud pública y epidemiología. Gaceta Sanitaria. 1995, 9: 213-214.
Porta Serra M: Factor de impacto bibliográfico (Science Citation Index and Social Sciences Citation Index) de las principales revistas de medicina preventiva, salud pública y biomedicina. Algunas cifras, algunas impresiones. Revisiones en Salud Pública. 1993, 3: 313-347.
Jemec GB: Impact factors of dermatological journals for 1991–2000. BMC Dermatol. 2001, 1: 7-10.1186/1471-5945-1-7.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/5/24/prepub
Study supported in part by ISCIII-RCESP C03/09 (Spanish Network for Cooperative Research in Epidemiology and Public Health).
The author(s) declare that they have no competing interests.
CMT was responsible for the development of intellectual content and the study design, collected the data, data coding and entry, interpretation of the results, manuscript drafting and the critical revisions of manuscript. GLA was responsible for the development of intellectual content, statistical analyses, interpretation of the results and manuscript drafting. All authors read and approved the final manuscript.