Is the 'Credibility Revolution' in Economics Leaving Some Fields Behind?
"A new analysis reveals that while some areas of economics have embraced rigorous research methods, others are lagging, potentially impacting the reliability of their findings."
For the past two decades, economics has undergone a significant transformation known as the 'credibility revolution'. This movement emphasizes the use of transparent, credible research designs, leveraging new data to generate profound insights. Pioneered by figures like Joshua Angrist and Jörn-Steffen Pischke, the revolution aims to enhance the reliability and validity of economic research, addressing pressing questions from economic growth to the impacts of social and educational policies.
However, a recent study casts light on a concerning trend: the uneven adoption of these rigorous methods across different fields within economics. While some areas have fully embraced the credibility revolution, others are lagging behind, potentially undermining the robustness of their conclusions. This raises critical questions about whether the movement's initial momentum, identified by Angrist and Pischke, is still continuing apace, and whether certain empirical techniques are being favored over others.
Building on the work of Currie, Kleven, and Zwiers, this analysis examines the credibility revolution across various fields, including finance and macroeconomics. By analyzing over 32,000 National Bureau of Economic Research (NBER) working papers, the study identifies the frequency of phrases related to different empirical techniques, providing a comprehensive view of how these methods are being applied—or not—across the discipline.
Measuring an Uneven Transformation
Quantifying the reach of the so-called 'credibility revolution' in economics is difficult, because no single authoritative measure tracks how widely its methods have been adopted across subfields. Available indicators, such as publication patterns, citation practices, or training changes, paint an incomplete and sometimes contradictory picture. Some observers suggest that fields closer to the design-based approaches favored by the movement have changed quickly, while others appear to have adapted more slowly. Given this uncertainty, any precise numbers offered about the movement's impact should be regarded as provisional rather than definitive.
Methods, Assumptions, and Their Boundaries
The accepted methods associated with the credibility revolution generally center on using natural experiments and other research designs to estimate causal effects, treating identification of a treatment effect as the core hurdle. In practice, these approaches require strong assumptions about comparability between treated and untreated groups, the validity of instruments, or the randomness of treatment assignment. Because those assumptions are rarely fully testable, results depend on judgment as much as on technique. A significant limitation is that the emphasis on credible identification can steer researchers toward questions that lend themselves to clean designs, even while many economically important questions do not.
The Roots of Credibility
Dictionaries consistently define credibility in terms of believability and trustworthiness, and these definitions are remarkably stable across reference works. Merriam-Webster, for instance, describes credibility as "the quality or power of inspiring belief," while Cambridge defines it as the fact that someone or something can be believed or trusted. Wikipedia extends the definition by identifying two key components, trustworthiness and expertise, each of which carries both objective and subjective dimensions. This shared foundation suggests that the concept at the heart of economics' credibility movement has long rested on a broadly agreed-upon meaning.
The Uneven Landscape of Credibility in Economics
The study reveals that while the overall trends identified by Currie et al. continue to advance, significant heterogeneity exists across fields. Applied microeconomics has wholeheartedly embraced empirical techniques that emphasize research design, such as difference-in-differences, event studies, and randomized trials. In contrast, finance and macroeconomics are lagging in the uptake of these methods.
- Difference-in-differences dominate finance and macroeconomics.
- Bartik and shift-share instruments are primarily used in applied micro areas.
- Synthetic controls have plateaued in popularity.
Emerging Work and Open Questions
Recent work in this space does not yet lend itself to a single summary, since research on the credibility movement is scattered across methodological reviews, replication studies, and field-specific appraisals. Much of the literature proceeds by example, documenting how particular natural experiments have reshaped understanding of specific economic questions. Reviews of the movement often note that its influence is measured more through practice than through formal pronouncements. Accordingly, general claims about the latest findings should be read as impressions of an ongoing conversation rather than settled conclusions.
When Trust Is Lost
At least one dictionary frames credibility in plainly behavioral terms, noting that a person "has credibility when you seem totally trustworthy or believable" and loses it by "lying, cheating and acting rather shady." This framing suggests that credibility is not fixed but earned and forfeited through conduct. In the context of economics, it implies that a research field's standing can erode when its practices, or the actions of its practitioners, stop inspiring belief. The definition offers a useful lens for understanding how failures, not just methodological advances, shape whether an approach is trusted.
Comparing Approaches Across Fields
Comparing the credibility movement across fields of economics is inherently difficult, because different areas face different data environments, institutional constraints, and traditions of evidence. Fields with abundant administrative or experimental data may have been easier for design-based methods to penetrate, whereas those reliant on structural modeling or historical narrative may pose greater challenges. Researchers offer only partial accounts of these differences, and rigorous cross-field comparisons are relatively scarce. Any strong claim that one field has been left behind should therefore be treated as a hypothesis in need of direct evidence.
The Path Forward: Diversifying Research Methods
The growing interest in and impact of difference-in-differences research across economics highlights how a single empirical technique, when widely adopted, can meaningfully shift the trajectory of an entire academic field. However, given some of the recent econometrics work flagging sensitivities and weaknesses in difference-in-differences, there may be value in researchers attempting to more broadly diversify their research methods portfolio. It is also quite striking that given the popularity of difference-in-difference that synthetic control methods have not grown further, as these methods have very similar properties.
A Contested Legacy
Across various discussions of the credibility movement, a recurring theme is that its legacy is contested rather than settled. Supporters tend to emphasize gains in rigor and transparency, while critics point to what they see as narrowing or misdirected incentives. There appears to be little consensus among commentators about whether the movement has changed economics for the better overall. Given this range of views, expert commentary on the topic is best read as reflecting genuine disagreement rather than an established verdict.
What Comes Next
What the next phase of the credibility movement will bring remains genuinely uncertain. Some commentators anticipate continued refinement of identification strategies and greater emphasis on replication and transparency, while others foresee growing attention to areas where design-based methods are harder to apply. The frontier may also be shaped by new data sources, computational tools, and institutional incentives that are difficult to predict from current vantage points. Projections in this direction should therefore be framed as possibilities rather than forecasts.
Institutional Pressures and Structural Forces
Wider systemic forces, such as publication incentives, funding structures, and the training of new researchers, likely shape how the credibility movement unfolds in different fields, though these connections are not fully documented. When incentives reward papers with clean, novel identification, researchers may gravitate disproportionately toward such work. These pressures can interact with the methodological demands of particular fields in ways that are hard to isolate empirically. Any assessment of systemic challenges must acknowledge that much of this reasoning is necessarily speculative.
People, Practice, and Consequences
Behind the methodological debates are researchers making daily choices about research design, and the conclusions that emerge carry implications for real decisions in policy and business. How individuals weigh rigor against relevance, or career incentives against intellectual curiosity, is not well captured by survey statistics. The human costs of methodological drift, such as disaffected researchers or policy built on fragile findings, are largely anecdotal at this stage. These human dimensions warrant attention even though they resist straightforward measurement.