Regression Discontinuity Design: How Spillovers Can Impact Policy Evaluations
"Uncover the hidden ripple effects in RDD and how to accurately assess policy outcomes in interconnected environments."
In an interconnected world, policy evaluations often face a complex challenge: spillovers. Regression Discontinuity Design (RDD) is a popular method for estimating the causal effects of policies or interventions. It leverages a cutoff point to create a quasi-experimental setting, comparing outcomes for those just above and below the threshold. However, the classic RDD framework typically assumes that there are no spillovers—meaning that the treatment only affects those directly receiving it.
This assumption can be unrealistic. Policies rarely operate in a vacuum. Their effects can ripple outwards, influencing individuals or groups not directly targeted. For example, a job training program in one region might indirectly benefit neighboring areas through increased economic activity. Neglecting these spillovers can lead to biased and misleading conclusions about a policy's true impact.
Recent research has begun to address the challenges that spillovers pose to RDD. By acknowledging and accounting for these spillover effects, policy evaluators can gain a more accurate understanding of policy outcomes and make better-informed decisions. This article explores how spillovers can influence RDD estimates and introduces methods to mitigate these biases, offering practical insights for researchers and policymakers alike.
Cutoff-Based Evidence and the Role of Modeling Choices
Recent analyses in this area rely on designs that mirror the logic of cutoff-based evaluation. One illustrative study tracked patient outcomes over time using hierarchical regression models that treated the hospital as a random intercept and year as the primary exposure, expressed as categorical terms relative to a 2010 reference year. This setup highlights how the choice of reference period and the clustering of patients within facilities can materially shape estimated effects. For regression discontinuity work, such clustering raises the risk that spillovers—such as patients treated across neighboring hospitals or programs influencing one another—violate the independence assumptions these models carry.
The Standard Design and Its Assumptions
The standard regression discontinuity design compares units just above and just below a treatment threshold, treating the cutoff as quasi-random because units close to it are presumed similar on average. Local polynomial regressions and optimal bandwidth selection are widely used to estimate the discontinuity. These approaches generally rest on the assumption that units cannot precisely manipulate their score near the cutoff and that no other changes occur at the threshold. Spillovers between treated and untreated units can undercut these assumptions, since outcomes for units on one side may be influenced by nearby units on the other.
A Long Evolution, Cautiously Characterized
The regression discontinuity approach has a long intellectual history, generally traced to evaluation work conducted in the mid-twentieth century on educational programs and scholarships. Over subsequent decades, the method moved from a relatively specialized tool into mainstream applied economics and policy evaluation. This evolution was driven by advances in computing, richer administrative data, and formal results on bandwidth selection and inference. Any detailed historical account should be treated cautiously, as exact origin stories and priority claims vary across sources.
Understanding RDD and the Spillover Problem
Regression Discontinuity Design relies on a clear threshold to create a comparison group. Units just above the threshold receive the treatment, while those just below do not. By comparing outcomes near this cutoff, researchers can estimate the treatment effect, assuming that potential outcomes are continuous around the cutoff (Hahn et al. 2001). This approach is valuable because it mimics random assignment near the threshold, reducing concerns about selection bias (Lee and Lemieux 2010).
- Exogenous Spillovers: Imagine a media campaign designed to increase voter turnout. Households directly exposed to the campaign might discuss it with friends and family in other areas, influencing their voting behavior as well.
- Endogenous Spillovers: Consider a policy that incentivizes renewable energy adoption. If one household installs solar panels, their neighbors might feel social pressure to do the same, leading to a ripple effect of adoption.
Modern Emphasis on Robustness and Spillovers
Recent research on regression discontinuity has emphasized robustness alongside the classic focus on estimating local effects. Methods for handling multiple cutoffs, fuzzy assignment, and covariates continue to develop, and many reviews stress how sensitive estimates can be to specification choices such as bandwidth and polynomial order. In spillover settings in particular, newer work explores the conditions under which spillovers can still be identified or at least bounded. Because much of this work is recent and evolving, findings should be read as provisional.
Known Failure Modes of the Design
Critics of regression discontinuity point to a fundamental trade-off: the local, cutoff-specific estimate often says little about the average treatment effect for the full population. The design is also vulnerable when individuals can manipulate their assignment score or when other interventions coincide with the threshold. Spillovers represent a particularly pointed failure mode, since they can break the clean contrast between treated and untreated units that the design relies on. These concerns are widely discussed in the literature, though their practical relevance depends heavily on the setting.
RDD Against Other Approaches
Compared with randomized controlled trials, regression discontinuity trades external validity for credibility closer to the cutoff: it typically uses observational data but exploits a known assignment rule. Relative to other quasi-experimental methods such as difference-in-differences or instrumental variables, RDD arguably needs weaker assumptions within the narrow window around the threshold. The choice among methods generally hinges on data availability, the nature of the assignment rule, and how concerned one is about unmeasured confounders. Direct head-to-head comparisons are rare, so conclusions about the superiority of any single method should be drawn cautiously.
Moving Forward: Towards More Accurate Policy Evaluations
Accounting for spillovers in RDD is essential for robust policy evaluation. By acknowledging the interconnectedness of modern environments and adopting appropriate methodologies, researchers can obtain more accurate estimates of policy effects, leading to better-informed decisions and more effective interventions. As the complexity of policy challenges grows, so too must the sophistication of our evaluation techniques. Continued research and methodological development in this area are crucial for ensuring that policy decisions are based on sound evidence and a comprehensive understanding of real-world impacts.
Expert Consensus on Credibility and Limits
Most expert commentary converges on the view that regression discontinuity is among the most credible non-experimental approaches when assignment is rule-based. At the same time, commentators caution that results are local and that validity depends on assumptions that spilled-over treatment effects are absent or fully accounted for. A recurring theme is the need for transparent robustness checks and thoughtful bandwidth selection. Because much of this commentary reflects accumulated professional judgment rather than a single definitive source, it should be read as guidance rather than settled fact.
Frontiers in Spillover-Aware Inference
Looking ahead, the frontier for regression discontinuity lies in richer data settings—such as administrative records and spatial or network data—where spillovers can be measured and modeled directly. Methodological innovation around spillover-aware estimators and credible bounds on local effects is likely to continue. Greater emphasis on external validity, through methods that extrapolate from the cutoff while tracking assumptions, also appears to be on the horizon. These directions are promising but still largely experimental, so concrete claims about their success should be treated as projections.
Systemic Pressures Around Causal Evidence
Beyond technique, the challenges surrounding regression discontinuity reflect broader systemic pressures in policy evaluation. Evaluations must balance statistical rigor with the cost and difficulty of collecting data around thresholds, and institutional realities constrain when clean cutoffs exist at all. Funding priorities, data-sharing restrictions, and the incentives of stakeholders can all shape which evaluations are feasible. In such an environment, spillover concerns are one piece of a larger puzzle about producing credible causal evidence in real-world settings.
The Human Stakes Behind the Estimates
Behind any evaluation design are the people whose lives are affected by the policy under study. When spillovers occur, the practical consequence is that some individuals receive benefits or burdens the evaluation never fully attributes to the treatment. Decision-makers relying on regression discontinuity results therefore risk misjudging who truly benefits, which can shape funding, program design, and public trust. Grounding technical methods in these human stakes is essential to translating causal estimates into responsible policy choices.