THE THOMSON REUTERS JOURNAL SELECTION PROCESS
By James Testa, Vice President, Editorial Development & Publisher Relations
At the center of Web of Science Core Collection are three flagship Citation Indexes, namely, the Science Citation Index Expanded™ (SCIE), Social Sciences Citation Index ® (SSCI) and the Arts & Humanities Citation Index ® (AHCI). These Citation Indexes cover the world’s top- tier international and regional journals whose evaluation and selection is governed by the Web of Science Core Collection Journal Selection Process, a well-established set of criteria that have been applied consistently for over fifty years. Citation impact is one of the defining characteristics of journals covered in SCIE and SSCI. Measurable citation impact, as expressed in the Journal Impact Factor, is published annually for journals covered in SCIE and SSCI.
Emerging Sources Citation Index is a new edition in Web of Science Core Collection. ESCI is a multidisciplinary Citation Index covering all areas of the scholarly literature of the sciences, social sciences, and arts and humanities. The Core Collection Journal Selection Process now includes a set of fundamental criteria that govern the selection of Journals for ESCI. The selection process for ESCI is related to the process applied to SCIE, SSCI and AHCI. Journals accepted for coverage in ESCI must be peer reviewed, follow ethical publishing practices, meet the our technical requirements, have English language bibliographic information, and be recommended or requested by a scholarly audience of Web of Science users.
It is important to note here that coverage of journals included in ESCI is never duplicated in SCIE, SSCI or AHCI. Coverage in ESCI is entirely separate from coverage in one or more of the Core Collection flagship indexes (SCIE, SSCI, AHCI). In addition, we do not calculate Journal Impact Factor metrics for journals covered in ESCI. Only journals that are selected for SCIE and SSCI will be listed in the Journal Citation Reports with Journal Impact Factor and related metrics. (Journal Impact Factors are not calculated for journals covered solely in AHCI.)
Because ESCI is a true Citation Index it is possible for users to track citation activity at the article and publication levels. Moreover, the citation activity for journals covered in ESCI will be used in the Journal Selection Process for SCIE, SSCI and AHCI and will enhance the transparency of that aspect of the Process. We will not, however, publish an official Impact Factor for publications covered in ESCI. Impact Factors are published annually in the Journal Citation Reports for SCIE and SSCI journals exclusively.
The Web of Science Core Collection Journal Selection Process now includes those criteria that Editorial Development applies to journals evaluated and selected (or rejected) for the ESCI. The Journal Selection Process for the central focus of Core Collection, namely, SCIE, SSCI and AHCI, remains fundamentally unchanged and consistent. SCIE, SSCI and AHCI continue to define the highly selective and central focus of Core Collection.
But comprehensive does not necessarily mean all-inclusive.1
Why Be Selective?
One might suppose that to be comprehensive, an index of the scholarly journal literature might be expected to cover all journals published. It has been demonstrated, however, that a relatively small number of journals publish the majority of significant scholarly results. This principle is often referred to as Bradford’s Law.2
S.C. Bradford realized that the core literature for any given scientific discipline was composed of fewer than 1,000 journals. Of these 1,000 journals, there are relatively few with a very strong relevance to the given topic, whereas there are many with a weaker relevance to it. Those with a weak relevance to the given discipline or topic, however, typically have a strong relevance to some other discipline. Thus, the core scientific literature can form itself around various topics, with individual journals becoming more or less relevant depending on the topic. Bradford understood that an essential core of journals forms the literature basis for each discipline and that most of the important papers are published in relatively few journals.3, 4
More recently, analysis of 11,813 journals across all categories of the natural and social sciences covered in the 2014 JCR ® revealed that as few as 525 journals account for 50% of what is cited and more than 25% of what is published. A core of 4,470 of these journals accounts for 80% of published articles and nearly 85% of cited articles. This analysis illustrates a principle known as Garfield’s Law of Concentration, namely that the core literature for all scholarly disciplines may be concentrated in a relatively small number of journals. 5 This is the principle upon which the Web of Science Core Collection Journal Selection Process was founded, and it continues to be the guiding force behind the Process. Furthermore, this core is not static. Its basic composition changes constantly, reflecting the evolution of scholarly topics. Our mission is to update journal coverage in the Web of Science Core Collection by identifying and evaluating promising new journals and, whenever necessary, deleting journals that have become less useful. With the addition of the ESCI to Core Collection it will be possible for many journals under evaluation for SCIE, SSCI and AHCI to be visible in Web of Science during the evaluation process. Additionally, many journals that need to be deselected from SCIE, SSCI and AHCI may continue to be indexed in the ESCI.
THE EVALUATION PROCESS FOR SCIE, SSCI and AHCI6
Overview of the Process
Many factors are taken into account when evaluating journals for coverage, ranging from the qualitative to the quantitative. These include the following:
- Basic publishing standards
- Editorial content
- International focus
- Citation Analysis
No one factor is considered in isolation, but by combination and interrelation of data, our editors are able to determine the journal's overall strengths and weaknesses.
Once timeliness has been established, we have the option to proceed with the other aspects of the evaluation process. This process rarely begins immediately, however, due to the high volume of journal submissions and the editorial priorities set for Web of Science Core Collection coverage. (As mentioned above, many journals under evaluation for SCIE, SSCI or AHCI may be covered initially in ESCI.) For journals evaluated for possible coverage in SCIE, SSCI or AHCI it is important that all issues be received/ posted in a timely manner. (After three consecutive issues have been received/ posted, the journal publisher may request an update on the status of the evaluation of that journal. For journals that are not publishing issues but are posting articles individually, a status update may be requested after nine months of content has been published. See below for instructions on requesting a status update for a journal under evaluation. Please do not request an evaluation status update until after three consecutive issues have been received/ posted, or nine months’ content has been posted.)
Journal evaluation is ongoing with journals added to and deleted from the Web of Science Core Collection throughout the year. Each year, the Editorial Development staff reviews over 3,500 journal titles for inclusion in SCIE, SSCI and AHCI. Only around 10% of these journals are accepted for coverage. Moreover, existing journal coverage in the Web of Science Core Collection is constantly under review. Journals now covered are monitored to ensure that they are maintaining high standards and a clear relevance to the products in which they are covered.
The journal selection process described here is applied to all journals evaluated for Science Citation Index Expanded, Social Sciences Citation Index , or Arts & Humanities Citation Index . The application of citation analysis is specific to the subject area or category of the journal’s editorial content. For example, citation metrics generated for fast moving biomedical topics would never be applied to a journal dealing with Social Sciences topics, and in the Arts & Humanities citation analysis plays a very minor role in evaluation. These special considerations are noted further below.
Application of the peer-review process is another indication of journal standards and signifies the overall quality and integrity of the research presented and the completeness of bibliographic elements, especially cited references.6 Inclusion of funding acknowledgements is also strongly recommended. Not only do they help create a greater context for the journal, they also function as a confirmation of the importance of the research presented.
Ethical Publishing Practices
Evidence of unethical policies such as predatory publishing practices or editorial instructions leading to excessive, inauthentic journal self-citation or any other fraudulent practices are not acceptable in any journal under evaluation and result in immediate rejection. When discovered in a journal already covered in Web of Science Core Collection they may result in deselection or the suppression of any affected citation metrics.
Journals published in print or electronic formats (XML, PDF) are eligible for evaluation. A technical evaluation is applied to all journals accepted for coverage in their electronic format to assure that access to their content is compatible with our indexing systems.
Timeliness of publication is a basic criterion in the evaluation process. As noted above, it is of primary and fundamental importance. A journal must be publishing according to its stated frequency to be considered for inclusion in SCIE, SSCI or AHCI. The ability to publish on time implies a healthy backlog of manuscripts essential for ongoing viability. It is not acceptable for a journal to appear chronically late, weeks or months after its cover date. To measure timeliness we need to see three consecutive current issues, one after another, as soon as they are published online or in print.
However, when a journal publishes articles online one at a time rather than collecting articles for release as an “issue,” we take a different approach. In these cases, the editor looks for a steady flow of articles over a nine-month period. The appropriate number of articles for the journal will be determined by the norms for its particular Web of Science category.
International Editorial Conventions
We also determine if the journal follows international editorial conventions, which are intended to optimize retrievability of source articles. These conventions include informative journal titles, fully descriptive article titles and author abstracts, complete bibliographic information for all cited references, and full address information for every author.
Full Text English
English is the universal language of science. For this reason our focus is on journals that publish full text in English or, at the very least, bibliographic information in English. There are many journals covered in Web of Science Core Collection that publish articles with bibliographic information in English and full text in another language. However, it is clear that the journals most important to the international research community are publishing full text in English. This is especially true in the natural sciences. There are notable exceptions to this rule in the arts and humanities and in social sciences topics. This is discussed further below. Nonetheless, full text English is highly desirable, especially if the journal intends to serve an international community of researchers. In addition, all journals must have cited references in the Roman alphabet.
As mentioned above, an essential core of scientific literature forms the basis for all scholarly disciplines. However, this core is not static — scientific research continues to give rise to specialized fields of studies, and new journals emerge as published research on new topics achieves critical mass. Our editors determine if the content of a journal under evaluation will enrich the database or if the topic is already adequately addressed in existing coverage.
With an enormous amount of citation data readily available to them, combined with their daily observation of virtually every new scholarly journal published, our editors are uniquely well positioned to spot emerging topics and active fields in the literature.
Editors look for international diversity among the journal’s contributing authors, editors, and editorial advisory board members at a level that is appropriate for the journal’s target audience. If the journal’s content is aimed at an international audience then we expect to find an internationally diverse group of authors, editors and editorial advisory board members contributing to it.
We are also interested in excellent regional Journals and are able to include a relatively small proportion of these each year. Regional journals often target a local, rather than an international, audience requiring less emphasis on extensive international diversity. Citation analyses may also play a different role in the evaluation of regional Journals whose citation impact is characteristically modest. Otherwise, the selection criteria for regional journals are the same as for international journals. The importance of a regional journal is measured more in terms of the specificity of its content. Will it enrich our coverage of a particular subject or provide studies with a specific regional perspective? All regional journals selected must be publishing on time, have English-language bibliographic information (title, abstract, keywords), and be peer reviewed. Cited references must be in the Roman alphabet. For more information on our approach to regional coverage in recent years read the essay at http://thomsonreuters.com/products_services/science/free/essays/regional_content_expansion_wos/.
Because Web of Science Core Collection is a true citation index, all cited references from every item in every journal covered are indexed whether or not the cited work is also covered as a source publication. As a result, the Web of Science Core Collection Journal Selection Process is unique in that our editors have a wealth of citation data available to them. Through these data it is possible to measure the citation impact of journals that are still under evaluation.7
We use citation analysis to determine the importance and influence of a journal in the surrounding literature of its subject. Citation analysis takes place on at least two levels. We analyze Total Citation counts to determine the integration of the journal into the surrounding literature over its entire publishing history. We use Impact Factor to determine the recent effect of the journal on the literature of its subject.
We also look at the citation history of contributing authors and editorial board members to determine if the journal is able to attract established scholars in the field of study.
Citation data and metrics are interpreted and understood in the editorial context of the journal under evaluation. Using quantitative citation data to measure impact is meaningful only in the context of journals in the same category. For example, smaller fields like Agricultural Engineering do not generate as many articles or citations as larger fields like Biotechnology or Genetics. In some areas, particularly in the Social Sciences, it may take a relatively long time for an article to attract a meaningful number of citations. But in other areas, such as the Life Sciences, it is not unusual for citations to accrue rapidly and peak after two or three years.8 These facts must be taken into consideration for the correct use of citation data.
Likewise, established journals that have been rejected in the past may be re-evaluated. These journals can experience new growth in citation impact resulting from changes such as translation into English, change in editorial focus, change in publisher, publishing medium, etc.
Self-citation rates are also taken into consideration. The self-cited rate relates a journal’s self-citations to the number of times it is cited by all journals, including itself. For example, journal X was cited 15,000 times by all journals, including the 2,000 times it cited itself. Its self-cited rate is 2/15 or 13%.
It is entirely normal for authors to reference the prior work that is most relevant to their current results, regardless of the source journal in which the work was published. However, there are journals where the observed rate of self-citation is a dominant influence in the total level of citation. For these journals, self-citation may distort their true role in the surrounding literature of its subject.9
Among all journals listed in the 2014 JCR Science Edition, for example, 85% have self-citation rates of 15% or less. This shows that self-citation is quite normal for most journals. Significant deviation from this normal rate, however, prompts an examination by Editorial Development to determine if excessive self-citations result in an artificial inflation of the impact factor. If we determine that self-citations are causing this effect, the journal’s impact factor will be suppressed for at least one year and the journal may be considered for deselection from the Web of Science Core Collection.
SPECIFIC CONSIDERATIONS FOR THE SOCIAL SCIENCES
All social science journals undergo the same thorough evaluation as journals in the natural sciences. Publishing standards, editorial content, international focus, and citation data are all considered. Standard citation metrics, at both journal and author levels, are analyzed while keeping in mind that overall citation rates in the social sciences are generally lower than those in the natural sciences.
Regional studies are often the subject of scholarly research. These studies have special importance in the social sciences as topics of local, rather than global, interest.
SPECIFIC CONSIDERATIONS FOR THE ARTS & HUMANITIES
Publishing standards, including timeliness, are important in the evaluation of arts and humanities journals. Citation patterns in the arts and humanities, however, do not necessarily follow the same predictable pattern as citations to social sciences and natural sciences articles. In addition, arts and humanities journal articles frequently reference non-journal sources (e.g., books, musical compositions, works of art, and literature). As a result, citation metrics do not normally play a leading role in the evaluation of arts and humanities journals.
English Language full text is also not always required in some areas of arts and humanities scholarship where the national focus of the study precludes the need for it. An example of this is studies in regional or national literatures.
What is most important in the arts and humanities is novel, well focused content that is well produced and well communicated.
EVALUATION PROCESS FOR THE ESCI
The evaluation criteria for coverage in ESCI are focused on consideration of Publishing Standards and Editorial Content. These include the following:
- Peer review. Peer review, as an indicator of the integrity of the scholarly content, must be present in all journals accepted for coverage in ESCI.
- Ethical publishing practices. Evidence of unethical publishing practices (see above) is an obstacle to coverage in ESCI.
- Electronic format. Only journals publishing in electronic formats (XML, PDF) compatible with our systems are eligible for coverage in ESCI. No print-only journals are eligible.
- English language bibliographic information is required for all journals seeking coverage in ESCI.
- Recommendation or request for coverage by Web of Science users. Journals of particular importance to Web of Science users are given the highest priority in evaluation and selection for ESCI.
Timeliness is not considered as formal a criterion in the evaluation of ESCI journals as it is in the evaluation of journals for SCIE, SSCI and AHCI. It is of essential importance, however, that any journal under evaluation for ESCI publish actively with current issues and articles posted regularly. Journals that have not published issues or articles for an extended period are not viable candidates for coverage in ESCI.
Many journals that are under evaluation for SCIE, SSCI or AHCI are considered first for coverage in ESCI. If these journals meet the criteria mentioned above they may begin coverage in ESCI with full evaluation for possible coverage in SCIE, SSCI and AHCI occurring at a later date.
Journals that need to be deselected for coverage in SCIE, SSCI or AHCI may continue coverage in ESCI at the discretion of the Editorial Development staff.
Because ESCI is a true citation index, every issue and every item published in each covered journal will be indexed. All cited references in each ESCI journal will also be captured and indexed. As with all journals covered in Web of Science Core Collection, citations to its articles are captured and displayed as Times Cited. These metrics will be visible to Web of Science users and will be used by Editorial Development as the basis for the citation-analysis aspect of the classic Web of Science Core Collection Journal Selection Process if the journal is evaluated subsequently for possible coverage in SCIE, SSCI or AHCI.
An official Journal Impact Factor will not be published for journals covered in ESCI. Nor will citations from ESCI journals be considered as part of the Impact Factor calculation for journals covered in SCIE or SSCI.
Some Guidelines for Electronic Journals
As stated earlier, our basic mission is to provide access to the world's most important and influential journals regardless of the media in which they are published.
The format of electronic journals is extremely important. The following guidelines help to ensure correct citation of articles and reduce the possibility of ambiguity when citing articles.
- Ensure that it is easy to identify the following elements:
- Journal title
- Year of publication
- Volume and/or issue number (if applicable)
- Article title
- All article identifiers, including page number, article number, DOI and PII (see item #2 below)
- Authors names and addresses
- A complete table of contents for each issue that includes the page or article number for each article (unless the journal is being published as single articles)
- Follow these guidelines for article identifiers in both source articles and in citations; this helps ensure their proper use by those referencing the article:
- Each article must be assigned a unique article number or include continuous pagination (whichever numbering scheme is being used). If the same article numbers are repeated in each issue within a volume, ambiguities will result when citing the original article
- If your journal has page numbers and article numbers, list them separately and not merged together (for example: Art. #23, pp. 6-10 and not 23.6-23.10)
- Include a DOI. The article number should not be the DOI
- Instruct authors to include the following information when citing your e-journal:
- Journal title (use one standard abbreviation for your journal; avoid acronyms that may be confused with other titles)
- Volume number (if applicable)
- Issue number (if applicable; within parenthesis)
- Page number and/or article number (clearly identifying the article number as such)
- Year of publication
The editors who perform journal evaluations have educational backgrounds relevant to their areas of responsibility. Because they monitor virtually every new scholarly journal published, they are also experts in the literature of their fields.
The evaluation of a journal for coverage in Web of Science begins with the submission of recently published content. For detailed instruction on submitting a journal for evaluation see below.
How to Submit a Journal for Evaluation
Credits and Footnotes
Special thanks to Editorial Development leaders Joelle Masciulli, Mariana Boletta, Kathleen Michael and Rodney Chonka for their critical reading and comments, to Publisher Relations management Marian Hollingsworth and Thomas Ciavarella for their active promotion of ESCI standards in the publishing community, and to Nina Chang for her insight on the integration of ESCI into the Core Collection in Web of Science.
1. Garfield, E., "How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations," Current Contents, May 28, 1990.
2. Garfield, E., Citation Indexing -- Its Theory and Application in Science, Technology, and Humanities, New York: John Wiley & Sons, 1979.
4. Garfield, E., "The Significant Scientific Literature Appears in a Small Core of Journals," The Scientist, V10 (17), September 2, 1996.
5. Garfield, E.,”The Mystery of the Transposed Journal Lists – Wherein Bradford’s Law of Scattering is Generalized According to Garfield’s Law of Concentration.” In Essays of an Information Scientist, Vol. 1(Philadelphia: ISI Press, 1977). Pp. 222-223.
6. "How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations," Current Contents, May 28, 1990.
9. McVeigh, M., Journal Self-Citation in the Journal Citation Reports – Science Edition, 2002.