arXiv 30 Apr 2025 · Econometrics
arXiv:2504.21595 · PDF · DOI · OpenAlex · Extracted main text
Counterfactual mean estimators such as difference-in-differences and synthetic control have grown into workhorse tools for program evaluation. Inference for these estimators is well-developed in settings where all post-treatment data is available at the time of analysis. However, in settings where data arrives sequentially, these tests do not permit real-time inference, as they require a pre-specified sample size T. We introduce real-time inference for program evaluation through anytime-valid rank tests. Our methodology relies on interpreting the absence of a treatment effect as exchangeability of the treatment estimates. We then convert these treatment estimates into sequential ranks, and construct optimal finite-sample valid sequential tests for exchangeability. We illustrate our methods in the context of difference-in-differences and synthetic control. In simulations, they control size even under mild exchangeability violations. While our methods suffer slight power loss at T, they allow for early rejection (before T) and preserve the ability to reject later (after T).
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Abadie, A. and Zhao, J (2021) Synthetic controls for experimental design | 1.000 | 11 | 5 | 100% |
| 2 | Chernozhukov, V., Wüthrich, K., and Zhu, Y (2021) An exact and robust conformal inference method for counterfactual and synthetic controls | 1.000 | 8 | 3 | 100% |
| 3 | Grünwald, P., de Heide, R., and Koolen, W (2024) Safe testing | 0.874 | 5 | 2 | 100% |
| 4 | Abadie, A. and Gardeazabal, J (2003) The economic costs of conflict: A case study of the basque country | 0.843 | 3 | 3 | 100% |
| 5 | Fedorova, V., Gammerman, A., Nouretdinov, I., and Vovk, V (2012) Plug-in martingales for testing exchangeability on-line | 0.811 | 4 | 2 | 100% |
| 6 | Bai, J (2009) Panel data models with interactive fixed effects | 0.644 | 2 | 2 | 100% |
| 7 | Koning, N. W (2024) Measuring evidence with a continuous test self | 0.644 | 2 | 2 | 100% |
| 8 | Koning, N. W (2024) Post-hoc and anytime valid permutation and group invariance testing self | 0.644 | 2 | 2 | 100% |
| 9 | Vovk, V., Nouretdinov, I., and Gammerman, A (2003) Testing exchangeability on-line | 0.644 | 2 | 2 | 100% |
| 10 | Abadie, A., Diamond, A., and Hainmueller, J (2010) Synthetic control methods for comparative case studies: Estimating the effect of california’s tobacco control program | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 33 scored citations.