THE THOMSON REUTERS JOURNAL SELECTION PROCESS
By Jim Testa, VP, Editorial & Publisher Relations
Thomson Reuters is committed to providing comprehensive coverage of the world's most important and influential journals to meet its subscribers' current awareness and retrospective information retrieval needs. Today Web of Science SM covers over 12,000 top tier international and regional journals in every area of the natural sciences, social sciences, and arts and humanities.
But comprehensive does not necessarily mean all-inclusive.1
Why Be Selective?*
It would appear that to be comprehensive, an index of the scholarly journal literature might be expected to cover all journals published. It has been demonstrated, however, that a relatively small number of journals publish the majority of significant scholarly results. This principle is often referred to as Bradford's Law.2
In the mid-1930's, British mathematician and librarian S.C. Bradford realized that the core literature for any given scientific discipline was composed of fewer than 1,000 journals. Of these 1,000 journals, there are relatively few with a very strong relevance to the given topic, whereas there are many with a weaker relevance to it. Those with a weak relevance to the given discipline or topic, however, typically have a strong relevance to some other discipline. Thus, the core scientific literature can form itself around various topics, with individual journals becoming more or less relevant depending on the topic. Bradford understood that an essential core of journals forms the literature basis for all disciplines and that most of the important papers are published in relatively few journals.3,4
More recently Thomson Reuters analyzed the 7,621 journals covered in the 2008 Journal Citations Report®. The analysis found that 50% of all citations generated by this collection came from only 300 of the journals. In addition, these 300 top journals produced 30% of all articles published by the total collection. Furthermore, this core is not static. Its basic composition changes constantly, reflecting the evolution of scholarly topics. Our mission is to update journal coverage in Web of Science by identifying and evaluating promising new journals and, whenever necessary, deleting journals that have become less useful.
The Evaluation Process5
Many factors are taken into account when evaluating journals for coverage in Web of Science, ranging from the qualitative to the quantitative. The journal's basic publishing standards, its editorial content, the international diversity of its authorship, and the citation data associated with it are all considered. No one factor is considered in isolation, but by combination and interrelation of data, the Thomson Reuters editor is able to determine the journal's overall strengths and weaknesses.
The Thomson Reuters editors who perform journal evaluations have educational backgrounds relevant to their areas of responsibility. Because they monitor virtually every new scholarly journal published, they are also experts in the literature of their fields. The evaluation of a journal for coverage in Web of Science begins with the submission of current issues. The publisher must deliver three consecutive current issues, one at a time as they are published, to Thomson Reuters. Issues may be submitted in print, online, or both. Publishers may send print issues to the following address: Publication Processing, Thomson Reuters, 1500 Spring Garden Street, Fourth Floor, Philadelphia, PA 19130. For submission of an online journal, the publisher must provide access information at http://ip-science.thomsonreuters.com/
info/journalsubmission/. This form is also appropriate for all types of online journals, including those that publish articles one at a time rather than collected in periodic issues.
Once timeliness has been established, Thomson Reuters has the option to proceed with the other aspects of the evaluation process. This process rarely begins immediately, however, due to the high volume of journal submissions and the editorial priorities set by Thomson Reuters for Web of Science coverage. When the evaluation has begun, all issues received will be considered. It is important, therefore, that all issues be received in a timely manner. Publishers or editors should continue sending timely issues until the evaluation process has been concluded. If at any time the journal publisher or editor seeks an update on the status of the evaluation for a specific journal, an inquiry may be sent via the form at http://ip-science.thomsonreuters.com/info/jrneval-status/.
Journal evaluation is ongoing at Thomson Reuters with journals added to and deleted from Web of Science throughout the year. Each year, the Thomson Reuters editorial staff reviews over 2,000 journal titles for inclusion in Web of Science. Around 10-12% of the journals evaluated are accepted for coverage. Moreover, existing journal coverage in Thomson Reuters products is constantly under review. Journals now covered are monitored to ensure that they are maintaining high standards and a clear relevance to the products in which they are covered. The journal selection process described here is applied to all journals in Web of Science, whether covered in Science Citation Index Expanded TM, Social Sciences Citation Index®, or Arts & Humanities Citation Index®. The application of citation analysis is specific to the subject area or category of the journal's editorial content. For example, citation metrics generated for fast-moving biomedical topics would never be applied to a journal dealing with Social Sciences topics nor to an Arts & Humanities title, where citation analysis plays a very different role. These special considerations are noted below.
Basic Publishing Standards
Timeliness of publication is a basic criterion in the evaluation process. As noted above, it is of primary and fundamental importance. A journal must be publishing according to its stated frequency to be considered for inclusion in Web of Science. The ability to publish on time implies a healthy backlog of manuscripts essential for ongoing viability. It is not acceptable for a journal to appear chronically late, weeks or months after its cover date. To measure timeliness we need to see three consecutive current issues, one after another, as soon as they are published.
Timeliness is also essential for electronic journals, or e-journals. If the e-journal is publishing distinct issues at a stated frequency, these issues should appear online in a timely manner.
However, as noted above, when an e-journal publishes articles one at a time rather than collecting articles for release as an 'issue', Thomson Reuters takes a different approach. In these cases, the editor looks for a steady flow of articles over a nine-month period. The appropriate number of articles for the journal will be determined by the norms for its particular Web of Science category.
International Editorial Conventions
Thomson Reuters also determines if the journal follows international editorial conventions, which are intended to optimize retrievability of source articles. These conventions include informative journal titles, fully descriptive article titles and author abstracts, complete bibliographic information for all cited references, and full address information for every author.
Full Text English
English is the universal language of science. For this reason Thomson Reuters focuses on journals that publish full text in English, or at very least, bibliographic information in English. There are many journals covered in Web of Science that publish articles with bibliographic information in English and full text in another language. However, going forward, it is clear that the journals most important to the international research community will publish full text in English. This is especially true in the natural sciences. There are notable exceptions to this rule in the Arts & Humanities and in Social Sciences topics. This is discussed further below. Nonetheless, full text English is highly desirable, especially if the journal intends to serve an international community of researchers. In addition, all journals must have cited references in the Roman alphabet.
Application of the peer-review process is another indication of journal standards and signifies overall quality of the research presented and the completeness of cited references.6 Inclusion of Funding Acknowledgements is also strongly recommended. Not only do they help create a greater context for the journal, these acknowledgements also function as a confirmation of the importance of the research presented.
As mentioned above, an essential core of scientific literature forms the basis for all scholarly disciplines. However, this core is not static — scientific research continues to give rise to specialized fields of studies, and new journals emerge as published research on new topics achieves critical mass. Thomson Reuters editors determine if the content of a journal under evaluation will enrich the database or if the topic is already adequately addressed in existing coverage.
With an enormous amount of citation data readily available to them, combined with their daily observation of virtually every new scholarly journal published, Thomson Reuters editors are uniquely well positioned to spot emerging topics and active fields in the literature.
Thomson Reuters editors look for international diversity among the journal's contributing authors, editors, and editorial advisory board members. This is particularly important in journals targeting an international audience. Today's scientific research takes place in a global context, and an internationally diverse journal is more likely to have importance in the international community of researchers.
Thomson Reuters is also interested in excellent regional journals and is able to include a relatively small proportion of these each year. Regional journals often target a local rather than an international audience, requiring less emphasis on extensive international diversity. Citation analyses may also play a different role in the evaluation of regional journals whose citation impact is usually modest. Otherwise, the selection criteria for regional journals are the same as for international journals. The importance
of the regional journal would be measured more in terms of the specificity of its content. Will it enrich our coverage of a particular subject or provide studies with a specific regional perspective? All regional journals selected must be publishing on time, have English-language bibliographic information (title, abstract, keywords), and be peer reviewed. Cited references must be in the Roman alphabet. For more information on the Thomson Reuters approach to Regional Coverage in recent years read the essay at http://thomsonreuters.com/products_services/science/free/essays/regional_content_expansion_wos/.
The Thomson Reuters Journal Selection Process is unique in that Thomson Reuters editors have a wealth of citation data available to them. Because Web of Science is a true citation index, all cited references from every item in every journal covered in Web of Science are indexed whether or not the cited work is also covered as a source publication. Through these data it is possible to measure the citation impact of journals under evaluation.
The importance of interpreting and understanding these data correctly cannot be emphasized too strongly. Using quantitative citation data to measure impact is meaningful only in the context of journals in the same general discipline. For example, smaller fields like Agricultural Engineering do not generate as many articles or citations as larger fields like Biotechnology or Genetics. In some areas, particularly in the Arts & Humanities, it may take a relatively long time for an article to attract a meaningful number of citations. But in other areas, such as the life sciences, it is not unusual for citations to accrue rapidly and peak after two or three years.7 These facts must be taken into consideration if citation data are to be used correctly.
Citation analysis takes place on at least two levels. Thomson Reuters looks for citations to the journal itself, as expressed by impact factor and/ or total citations received. For new journals that do not yet have a citation history at the publication level, analysts examine the citation record of the contributing authors and editorial board members. This allows them to see if the journal is able to attract contributions from scholars whose prior work has been useful to the research community.
Likewise, established journals that are not covered are often re-evaluated. These journals can experience new growth in citation impact resulting from changes such as translation into English, change in editorial focus, change in publisher, publishing medium, etc.
As noted above, because Thomson Reuters captures all cited references from each of the over 12,000 journals covered, citation information is available on journals not covered as well as those that are covered.
Self-citation rates are also taken into consideration. The self-cited rate relates a journal's self-citations to the number of times it is cited by all journals, including itself. For example, journal X was cited 15,000 times by all journals, including the 2,000 times it cited itself. Its self-cited rate is 2/15 or 13%.
It is entirely normal for authors to reference the prior work that is most relevant to their current results, regardless of the source journal in which the work was published. However, there are journals in which the observed rate of self-citation is a dominant influence in the total level of citation. For these journals, self-citation has the potential to distort the true role of the title in the context of the literature of its subject.8
Among all journals listed in the 2010 JCR Science Edition, for example, 85% have self-citation rates of less than 15%. This shows that self-citation is quite normal for most journals. Significant deviation from this normal rate, however, prompts an examination by Thomson Reuters to determine if excessive self-citations are being used to artificially inflate the impact factor. If we determine that self-citations are being used improperly, the journal's impact factor will be suppressed for at least two years and the journal may be considered for deselection from Web of Science.
All social science journals undergo the same thorough evaluation as journals in the natural sciences. Publishing standards, editorial content, international diversity, and citation data are all considered. Standard citation metrics, at both journal and author levels, are analyzed while keeping in mind that overall citation rates in the social sciences are generally lower than those in the natural sciences.
Regional studies have special importance in the social sciences, as topics of local rather than global interest are often the subject of scholarly research.
Arts & Humanities
Publishing standards, including timeliness, are important in the evaluation of arts and humanities journals. Citations patterns in the arts and humanities, however, do not necessarily follow the same predictable pattern as citations to social sciences and natural sciences articles. In addition, arts and humanities journal articles frequently reference non-journal sources (e.g., books, musical compositions, works of art, and literature). English-language text is not a requirement in some areas of arts and humanities scholarship where the national focus of the study precludes the need for it. An example of this is studies in regional or national literatures.
As stated earlier, the basic mission of Thomson Reuters is to provide access to the world's most important and influential journals regardless of the media in which they are published.
Publishing standards, editorial content, international diversity, and citation analysis are all considered when evaluating a purely electronic journal.
Evaluating the timeliness of publication of an e-journal may require a somewhat different approach. If the e-journal is publishing distinct issues, the Thomson Reuters editor looks for the appearance of these in a timely manner. However, when an e-journal publishes articles one at a time rather than collecting articles for release as an issue, the editor simply looks for a steady flow of articles over several months.
The format of electronic journals is extremely important to Thomson Reuters. Following is a set of guidelines for electronic journal formats. Following these guidelines helps ensure correct citation of articles and reduces the possibility of ambiguity in citation of articles.
Publishers should insure that it is easy to identify the following elements:
- Journal Title
- Year of publication
- Volume and/or Issue Number (if applicable)
- Article Title
- Page Number or Article Number (one or the other is required; article number should not be the DOI) If your journal has page numbers and article numbers, list them separately and not merged together (for example: Art. #23, pp. 6-10 and not 23.6-23.10).
- Authors names and addresses
- Label all article identifiers such as DOIs, PIIs and Article Numbers
- A complete table of contents for each issue that includes the page/article number for each article (unless journal is being published as single articles)
Labeling these identifiers in both source articles and in citations helps insure their proper use by those referencing the article and correct labeling by abstracting and indexing firms such as Thomson Reuters.
- Each article must be assigned a unique page number or article number (whichever numbering scheme is being used) within any one given issue. Additionally, article numbers must be unique within an entire volume number. If the same article numbers are repeated in each issue within a volume, ambiguities will result when citing the original article. A citation to V 20, art. 1, May 2002, (even when adding the author's name), would be difficult to find if V 20, art. 1 June 2002 also exists. Avoid duplication of article numbers.
- Consider references to your journal — instruct authors to include the following information when citing your e-journal:
- Journal title (use one standard abbreviation for your journal; avoid acronyms that may be confused with other titles)
- Volume number (if applicable)
- Issue Number (if applicable; within parenthesis)
- Page number and/or article number (clearly identifying the article number as such)
- Year of publication
How to submit a journal for evaluation
Thomson Reuters welcomes journal submissions for evaluation. If you would like to submit a journal for evaluation, please follow this link and provide all necessary information as well as access to your journal either electronically or in print.
Credits and Footnotes
Special thanks to Editorial Development staff members Katherine Junkins, Maureen Handel, Mariana Boletta, Anne-Marie Hinds, Kathleen Michael, Rodney Chonka, Chang Liu, and Luisa Rojo for their critical reading and comments.
- Garfield, E., "How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations," Current Contents, May 28, 1990.
- Garfield, E., Citation Indexing — Its Theory and Application in Science, Technology, and Humanities, New York: John Wiley & Sons, 1979.
- Garfield, E., "The Significant Scientific Literature Appears in a Small Core of Journals," The Scientist, (17), September 2, 1996.
- Garfield, E., "How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations," Current Contents, May 28, 1990.
- McVeigh, M., "Journal Self-Citation in the Journal Citation Reports – Science Edition", 2002.