Is the Impact Factor Impacting You? Why Scientists Are Ditching This Metric
"Discover how the scientific community is moving away from the traditional Impact Factor to embrace more meaningful measures of research influence."
For decades, the Impact Factor (IF) has reigned supreme as a metric to determine the “importance” of scientific journals. Academic institutions have historically relied on it for decisions regarding promotion and tenure, while other organizations have used it to evaluate funding applications. Despite numerous editorials and commentaries highlighting its misuse, the IF continues to persist, often becoming a source of frustration for journal editors due to its inherent inaccuracies and potential for manipulation.
Understanding how the IF is calculated quickly reveals its flaws in accurately assessing a journal's true impact. The IF is determined by dividing the number of citations a journal receives over a two-year period by the total number of articles published in that journal during those years. However, the numerator includes all journal citations regardless of content type, while the denominator only counts primary research and review articles. This 'apples to oranges' comparison creates a skewed representation of a journal's actual influence.
Furthermore, a single highly cited article can disproportionately inflate a journal's IF. There are also ways to manipulate the IF, such as reducing the number of primary research articles published, increasing the publication of review articles (which tend to be cited more frequently), or even negotiating article designations with the company that calculates the IF. These issues highlight the IF's fundamental flaws as a reliable metric.
Why the Metric Matters
The impact factor remains widely used as a shorthand for journal influence, but its effects should be interpreted cautiously. A journal-level average cannot by itself establish the quality or significance of an individual paper. Its influence may therefore be substantial while its meaning for any single researcher or study remains limited.
A Limited Measure
The standard approach uses citation-based indicators to summarize journal performance. Such measures can provide one view of scholarly attention, but they do not capture every dimension of research quality, relevance, or social value. Their usefulness depends on careful interpretation and should not be confused with a complete evaluation of individual work.
A Metric That Reshaped Science
Since its introduction more than 40 years ago, the journal impact factor has become one of the most discussed bibliometric indicators. The sources describe major effects on the scientific ecosystem, including changes to publishing, hiring practices, resource allocation, research activity, and dissemination. They also indicate that debates about the limitations and appropriate uses of journal performance indicators have accompanied the history of scientometrics and citation measures. Journals with higher impact-factor values are commonly regarded as more prestigious or important within their fields.
Why the Scientific Community Is Saying 'No' to the Impact Factor
So, how can we diminish the over-reliance on the IF? Casadevall and Fang proposed several strategies, including supporting the San Francisco Declaration on Research Assessment (DORA), boycotting high-IF journals, developing diverse metrics, and reforming review criteria for promotion, tenure, and research funding. The American Society of Microbiology, boasting over 30,000 members, recently removed the IF from more than a dozen of its journals – a move applauded by many.
- Inaccurate Representation: The IF calculation method compares citations of all content types (numerator) to only primary research and review articles (denominator), creating a skewed representation.
- Manipulation Potential: Journals can manipulate their IF by reducing the number of primary research articles, increasing review articles, or negotiating article designations.
- Disproportionate Influence: A single, highly cited article can significantly inflate a journal's IF, regardless of the overall quality of other publications.
- Focus on Quantity Over Quality: The IF emphasizes citation counts rather than the actual scientific merit, novelty, or impact of individual articles.
An Ongoing Debate
Recent discussion continues to question whether journal-level citation measures are sufficient for evaluating research. Reviews of the subject generally frame the issue as one of appropriate use rather than simple acceptance or rejection. A cautious approach treats the impact factor as one indicator among several, rather than as a direct measure of the value of a particular study.
Where the Metric Breaks Down
Citation-based journal metrics vary across disciplines because some fields cite rapidly while others, such as mathematics, develop more durable literatures with fewer short-term citations. Critics also identify questionable editorial practices, misuse and manipulation, and the distortion caused when a few highly cited papers raise a journal's average. The metric measures journal citation averages, not the quality of every paper published there. These limitations weaken its use as a stand-alone basis for judging researchers or research.
Beyond a Single Number
Comparing research through one journal-level number can obscure differences among disciplines, publication types, and individual papers. A broader assessment is likely to be more informative because citation averages do not fully represent research quality or influence. The impact factor can still offer contextual information, but it is best treated as one limited signal within a larger evaluation.
Time to Rethink How We Measure Scientific Success?
It's time for readers and fellow editors to critically re-evaluate the use and importance (or lack thereof) of the IF. This move sends a clear message: the IF is outdated and misused, and we need to explore alternative metrics to assess the true importance and scientific impact of research publications.
Use With Caution
The evidence supports a measured view of the impact factor: it is influential, familiar, and easy to compare, but incomplete. Its interpretation is especially sensitive to disciplinary citation practices and the difference between journal averages and individual research quality. Scientists and institutions therefore have reason to use it cautiously and alongside other forms of assessment.
A Broader Metrics Landscape
The Science family provides journal profiles, turnaround times, citation distributions, and citation-based metrics, with its metrics page updated twice a year. Separately, a 2024 study explored forecasting scientific publication trends using heterogeneous sources, including historical PubMed publication time series, research and review articles, and pre-trained models. These developments point toward evaluation systems that combine multiple forms of evidence rather than relying only on the impact factor. Guidance on metrics for 2025 also highlights altmetrics and other newer evaluation tools.
Changing Incentives
Because metrics can influence publishing, hiring, and funding decisions, their design has consequences beyond measurement. A system centered too heavily on one indicator may reward visibility in a journal rather than the full value of the underlying work. More balanced evaluation would need to account for disciplinary differences and recognize forms of contribution that citation averages leave out.
Impact Beyond Academia
Academic metrics primarily aim to represent how knowledge is disseminated among scientists, not how research affects the wider world. Research impact can also extend to the economy, society, health, and legislation. This distinction explains why journal citations alone may fail to capture the practical importance of a study. Evaluating real-world influence requires attention to outcomes beyond academic publishing.