Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects

Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects

\setlength\abovedisplayskip{2pt plus 0pt minus 2pt} \setlength\belowdisplayskip{2pt plus 0pt minus 2pt}

\thispagestyle{fancy}

Introduction

Treatment effects of certain policy interventions on economic entities are often of major interest in economic and econometric studies. In panel data models, estimation of and inference on treatment effects can be formulated as missing data problems. For instance, consider a sample of $N$ units with a policy intervention on units $N_{0}+1,\ldots ,N$ at time $T_{0}$, so that the pre-treatment periods are $1,\ldots ,T_{0}$, the post-treatment periods are $T_{0}+1,\ldots ,T$, and the control units are $1,\ldots ,N_{0}$. Let $ y_{i,t}$ be the potential outcome in the absence of policy intervention for unit $i$ at time $t$, and $y^{+}_{i,t}=y_{i,t}+\Delta _{i,t}$ be the potential outcome under policy intervention for unit $i$ at time $t$. We are interest in the treatment effect $\Delta _{i,t}$ for $(i,t)\in \mathcal{I}_{1}=\{N_{0}+1,\ldots ,N\}\times \left\{ T_{0}+1,\ldots ,T\right\} $, and it is easy to see that calculating $\Delta _{i,t}= y^{+}_{i,t}-y_{i,t}$ requires knowing $y_{i,t}$, which is actually unobservable (missing) for $(i,t)\in \mathcal{I}_{1}$.

Many existing methods of estimating $\Delta _{i,t}$ make efforts to construct “counterfactuals" for $y_{i,t}$, $(i,t)\in \mathcal{I}_{1}$ (e.g., abadie2010synthetic; hsiao2012panel; bai2021matrix; athey2021matrix; among others). The underlying models in counterfactual frameworks can be simplified as $y_{i,t}=\xi _{i,t}+e_{i,t}$, where $\xi _{i,t}$ is the systematic part and $e_{i,t}$ is the idiosyncratic error. Then the predicted values $\left\{ \widehat{\xi }_{i,t}:(i,t)\in \mathcal{I} _{1}\right\} $ serve the role as the counterfactuals for $\left\{ y_{i,t}:(i,t)\in \mathcal{I}_{1}\right\} $, and the treatment effects are estimated by $\widehat{\Delta }_{i,t}=y^{+}_{i,t}-\widehat{\xi }_{i,t}$ for every $(i,t)\in \mathcal{I}_{1}$. According to model specifications, assumptions, and estimation strategies, we can roughly classify the existing methods into the following 4 categories.

The first category is the synthetic control method proposed by abadie2003economic and abadie2010synthetic. It approximates $y_{N,t}$ by a convex combination of $\left( y_{1,t},\ldots ,y_{N-1,t}\right) $, which can be obtained from a constrained regression. Generalisations of the synthetic control method can be found in, for example, amjad2018robust, abadie2021penalized, arkhangelsky2021synthetic, ben2021augmented, ben2021synthetic, kellogg2021combining, and masini2021counterfactual. One can see abadie2021using for a comprehensive review. The second category is the the panel data approach proposed by hsiao2012panel, which constructs the counterfactual by solving a linear regression of $y_{N,t}$ on $\left( y_{1,t},\ldots ,y_{N-1,t}\right)$. This approach is later extended by ouyang2015treatment, li2017estimation, and hsiao2022multiple. Methods in the third category use the factor model technique (e.g., bai2002determining; bai2003inferential; pesaran2006estimation; bai2009panel; moon2015linear) to construct counterfactuals. Examples include kim2014divorce, xu2017generalized, and bai2021matrix. The last category starts from the statistical learning literature on matrix completion (e.g., candes2009exact; candes2010matrix; mazumder2010spectral; gamarnik2016note), and a recent example is athey2021matrix, which completes the potential outcome matrix via nuclear norm regularisation.

In practice, a mere point estimator of the treatment effect $\Delta _{i,t}$ is not sufficient either theoretically or empirically, and a valid inference is needed, which usually requires the (asymptotic) distribution of $\left( \widehat{ \Delta }_{i,t}-\Delta _{i,t}\right) $. As $\widehat{\Delta }_{i,t}-\Delta _{i,t}=\left( \xi _{i,t}-\widehat{\xi }_{i,t}\right) +e_{i,t}$, the idiosyncratic error term $e_{i,t}$ will dominate the distribution of $\left( \widehat{\Delta }_{i,t}-\Delta _{i,t}\right) $ as $ N_{0},T_{0}\rightarrow \infty $ whenever the systematic part can be consistently approximated. Due to this issue, most studies make inferences on time series average treatment effects as the number of post-treatment periods $\left( T-T_0 \right) \to\infty$ (e.g., hsiao2012panel; li2017estimation; li2020statistical, and chernozhukov2018t), or make inferences on cross-sectional average treatment effects as the number of treated units $\left( N-N_0 \right) \to\infty$ (e.g., xu2017generalized; arkhangelsky2021synthetic; ben2021synthetic; bai2021matrix), or assume the normality of error terms (e.g., fujiki2015disentangling; bai2021matrix). However, these approaches can not simultaneously meet the following needs in empirical studies.

To the best of our knowledge, there are only few inferential studies simultaneously satisfy the three conditions above. abadie2010synthetic use a cross-sectional permutation strategy, where they apply the synthetic control method to every potential control unit to approximate the true distribution of treatment effect estimator under the null hypothesis of $\Delta _{N,t}=0$. Obviously, the validity of this permutation approach relies on the cross-sectional exchangeability of the units. Furthermore, if one wants to construct the confidence intervals of the treatment effects in this fashion, then the equality between distributions of outcomes in the absence of and under intervention up to a location shift is required. chernozhukov2021exact apply time series permutation test to the inference on treatment effects. For a given null hypothesis $H_{0}:\Delta _{N}=\delta _{N}$, they plug $\delta_{N}$ into the observed data matrix to compute the full outcome matrix in the absence of intervention, and then perform time series permutations of the residuals from the treated unit to obtain an approximation of the distribution of the test statistic under $ H_{0}$ and the critical value. Confidence intervals can be obtained by grid searching on a set of different values of $\delta_{N}$, which may lead to intensive computation. As the latest work, cattaneo2021prediction build prediction intervals for synthetic control methods on conditional prediction intervals and non-asymptotic concentration. Since concentration inequalities only guarantee lower bounds of coverage probabilities, the proposed prediction intervals may suffer from conservativeness.

This paper aims to provide a theoretically non-conservative and computationally simple inferential procedure for the estimated treatment effects that simultaneously satisfy the above conditions, without imposing the cross-sectional exchangeability assumption. To be more specific, we apply bootstrap method to the treatment effect estimators proposed by bai2021matrix. Note that bai2021matrix's model keeps a simple factor structure but is quite general in the sense that it allows for multiple treated units and heterogeneous time of intervention. Bootstrapping is one of most commonly used tools to approximate an unknown distribution via resampling, and is shown to be asymptotically valid under mild conditions. We construct bootstrap-based confidence intervals of treatment effects in panel data models with interactive fixed effects. Specifically, we first apply the factor-based matrix completion technique proposed by bai2021matrix to obtain point estimators of treatment effects. Then we follow gonccalves2017bootstrap to approximate the distribution of $\widehat{ \Delta }_{i,t}-\Delta _{i,t}$ by (block) wild bootstrap and bootstrap, and use the quantiles of bootstrapped distribution to construct confidence intervals of $\Delta _{i,t}$ {for every} $(i,t)\in \mathcal{I}_{1}$.

Our paper contributes to the literature in the following folds. First, we establish the validity of confidence intervals by proving that they have asymptotically correct coverage probabilities as $\left( N_{0},T_{0}\right) \rightarrow \infty $, and our proposed confidence intervals do not require any specific distributional assumption on $e_{i,t}$ and are robust to small number of post-treatment periods. Thus, it is theoretically non-conservative and can meet the three needs for inferential purpose of treatment effects estimation in empirical studies. Second, we extend the bootstrap procedure in gonccalves2017bootstrap to a more general case. Since gonccalves2017bootstrap intend to make inference on factor-augmented regression models bai2006confidence where the factors serves as intermediate variables, they only require valid bootstrap approximations of factors. In this paper, we intend to make inference on the factor model (with missing values) itself, and the estimators involve products of estimated factors, their associated loadings, and a rotation matrix. This implies that we need valid bootstrap approximations of factors, loadings, and the rotation matrix, which creates theoretical challenges. Nevertheless, we successfully establish the asymptotic validity of our proposed bootstrap for such a scenario.

The finite sample properties of proposed bootstrap construction of confidence intervals for estimated treatment effects are investigated through Monte Carlo simulation, using different data generating processes (DGPs) for models with or without exogenous covariates, and for models with heteroscedastic errors or serially correlated errors. The simulation results show that the proposed bootstrap procedure works remarkably well for constructing confidence intervals for estimated treatments. Namely, the empirical coverage ratios are quite close to nominal values in all cases we considered regardless of whether the number of unobserved factors are priorly given or estimated from data.

Finally, we revisit the benefits of political and economic integration of Hong Kong with Mainland China in hsiao2012panel, as well as the effectiveness of the California Tobacco Control Program (CTCP) on per capita cigarettes consumption and personal health expenditures in abadie2010synthetic. We note that the treatment effects estimated using our new method are of similar magnitudes to those in the literature, while our bootstrap procedure provides reliable confidence intervals showing that the impacts of CTCP were significant over time for both per capita cigarettes consumption and personal health expenditures, and that the impacts of political and economic integration of Hong Kong with Mainland China vanished over time, although significant in the first few years.

The rest of this article is organised as follows. Section (ref) establishes the model and the assumptions. Section (ref) focuses on the estimation and inference in a model without covariates, and Section (ref) deals with a model with covariates. We conduct Monte Carlo simulations in Section (ref), and apply our method to classical datasets in Section (ref). Section (ref) concludes. Mathematical proofs of main results are left to online appendices.

Notations: we introduce some notations that are frequently used throughout this paper. $\mathbb{Z}_{+}=\{1,2,\ldots \}$ is the set of positive integers. $I_r$ is the $r\times r$ identity matrix. $A^{\mathsf{T}}$ denotes the transpose of matrix $A$. For a vector $z$, let $\left\Vert z\right\Vert =\sqrt{z^{\mathsf{T}}z}$ be the Euclidean norm of $z$. For a matrix $A$, let $\left\Vert A\right\Vert =\sqrt{ \mathrm{tr}\left( A^{\mathsf{T}}A\right) }$ be the Frobenius norm of $A$. $ \mathbb{N}\left( \mu ,\Sigma \right) $ stands for a normal distribution with mean $\mu $ and variance $\Sigma $. And $\xrightarrow{\mathrm{a.s.}}$, $ \xrightarrow{\mathbb{P}}$ and $\xrightarrow{\mathrm{d}}$ denote almost sure convergence, convergence in probability and convergence in distribution, respectively. We let $M$ denote a generic finite positive constant, whose value does not depend on $N$ or $T,$ and may vary case by case.

The Model

Suppose there are observations $\left( \mathscr{Y}_{i,t},x_{i,t}\right) $ for $i=1,\ldots ,N$ and $t=1,\ldots ,T$. Let the dummy variable $d_{i,t}$ indicate the $i$-th unit's treatment status at time $t$ with $d_{i,t}=1$ if under the treatment and $d_{i,t}=0$ if not. The observed data takes the form

where $y_{i,t}$ is the latent outcome of unit $i$ at time $t$ in the absence of treatment, and $\Delta_{i,t}$ is the treatment effect on unit $i$ at time $t$. For the ease of exposition, we assume $d_{i,t}=1$ for all $(i,t)\in \mathcal{I}_{1}$ and $d_{i,t}=0$ for all $(i,t)\in \mathcal{I}\backslash \mathcal{I}_{1}$, where

with $1\leq N_{0}<N$ and $1\leq T_{0}<T$. That is, we assume the last $ N-N_{0}$ units are intervened by the treatment at time $T_{0}$. Therefore, the observed matrix of outcomes is

Note that the above specification can be generated to allow for heterogeneous intervention time periods by letting $T_{0}$ be the time of the earliest intervention. This is consistent with the definition of $\mathcal{I}_{1}$ in bai2021matrix that $\mathcal{I}_{1}$ corresponds to the \textquotedblleft largest possible" missing block of the matrix.

We are interested in measuring the treatment effects of the policy intervention for the treated units after time $T_{0}$, which are $\Delta _{i,t}$ for $(i,t)\in \mathcal{I}_{1}$. Note that $\Delta _{i,t}$ measures the difference of the outcomes with and without the intervention of the treatment in the post-treatment periods. Unfortunately, the outcomes with and without the intervention of treatment cannot be simultaneously observed in reality for the same unit at a given time. This is because, once the policy intervention is in effect, then the researchers can only observe $\mathscr{Y} _{i,t}=y_{i,t}+\Delta _{i,t}$, not $y_{i,t}$. Thus, in order to estimate $ \Delta _{i,t}$, we need to generate the counterfactual of $y_{i,t}$ for $ (i,t)\in \mathcal{I}_{1}$, denoted as $\widehat{y}_{i,t}$, and thus the treatment effect can be estimated as

Our goal in this paper is to estimate $\Delta _{i,t}$ for $(i,t)\in \mathcal{ I}_{1}$ when neither treated units $N_{1}=N-N_{0}$ nor the post treated periods $T_{1}=T-T_{0}$ is large, where the former can be formulated as average treatment effects across units (see hsiao2022multiple for the aggregation of multiple treatment effects), and the latter can be formulated as the average treatment effects across post-treatment periods (see fujiki2015disentangling and li2017estimation).

Following the literature of treatment effects estimation using panels (e.g., abadie2010synthetic; hsiao2012panel; and bai2021matrix), we assume that $y_{i,t}$ is generated by the following panel data model with interactive fixed effects:

where $x_{i,t}$ is a $p$-dimensional vector of observed covariates of unit $ i $ at time $t$, $e_{i,t}$ is the idiosyncratic error term of unit $i$ at time $t$, $f_{t}$ is an $r$-dimensional time-variant unobserved factor at time $t$, and $\lambda _{i}$ is an $r$-dimensional individual specific factor loading of unit $i$, where $r$ is the number of unobserved factors and is usually unknown to researchers. \footnote{ Even if $r$ is unknown in practice, it can be consistently estimated using the method described in Section 4 and 5 of bai2002determining or the method proposed by alessi2010improved, and thus we can treat $r$ as known in the theoretical analyses below. This can be formally justified as follows. Let $\theta $ be a parameter in the model of interest and $\widehat{ \theta }$ be the estimator of $\theta $ based on estimated number of factors $\widehat{r}$. Note that $r$ takes values in $\mathbb{Z}_{+}$, and then the consistency of its estimator $\widehat{r}$ implies that $\mathbb{P}\left( \widehat{r}=r\right) \rightarrow 1$ as $\left( N,T\right) \rightarrow \infty $. Therefore,

}

Let $y_{i}=\left( y_{i,1},\ldots ,y_{i,T}\right) ^{\mathsf{T}}$, $ X_{i}=\left( x_{i,1},\ldots ,x_{i,T}\right) ^{\mathsf{T}}$, and $ e_{i}=\left( e_{i,1},\ldots ,e_{i,T}\right) ^{\mathsf{T}}$ for every $i\in \{1,\ldots ,N\}$. Define $Y=\left( y_{1},\ldots ,y_{N}\right) $, $F=\left( f_{1},\ldots ,f_{T}\right) ^{\mathsf{T}}$, and $\Lambda =\left( \lambda _{1},\ldots ,\lambda _{N}\right) ^{\mathsf{T}}$. We make the assumptions below.

The moment and convergence conditions in Assumptions (ref), (ref), (ref), (ref), and (ref) are used to establish probability bounds and asymptotic distributions. They are quite standard in the literature for panel models with interactive fixed effects, see, for example, bai2003inferential , bai2009panel and bai2021matrix, among others. The distribution and order conditions in Assumptions (ref), (ref), and (ref) mainly work for the validity of our bootstrap procedure.

We make further remarks on some of the assumptions. Assumption (ref)(4) requires the strict exogeneity of treatment status, and one can find similar conditions in Assumption 5 of hsiao2012panel and Assumption 2 of xu2017generalized. Assumption (ref)(3) states that the factors in the pre-treatment subsample and post-treatment subsample have the same asymptotic second sample moment matrices as those in the full sample. Assumption (ref)(4) states that the factor loadings in the control subsample and treated subsample have the same asymptotic second sample moment matrices as those in the full sample. When either $T-T_0$ or $N-N_0$ is finite, Assumption (ref)(3) or (ref)(4) can be replaced with equality as in the identifying restriction PC1 on Page 19 of bai2013principal to fulfil the condition that either the factors or factor loadings possess the same properties in the full sample as well as in the post-treatment periods or for the treated units.

Estimation and Inference in a Model without Covariates

To highlight the essence of our approach for estimation and inference of treatment effects, in this section we focus on a special case of Model (ref), where there is no covariate (i.e., $ \beta=0$) and the model is reduced to a pure approximate factor model:

We will extend our suggested approach to the full model (ref) in Section (ref).

Estimation of Treatment Effects

Following bai2021matrix, we consider the factor-based approach to estimate $y_{i,t}$ for $(i,t)\in\mathcal{I}_1$. Let $Y$, $F$, and $\Lambda$ follow their definitions in Section (ref). Since $y_{i,t}$ is unobserved for all $(i,t)\in \mathcal{I}_1$, the south-east block of $Y$ is missing in reality. Formally, we define two sub-matrices of $Y$:

That is, $Y_\mathrm{tall}$ corresponds to the control subsample, and $Y_\mathrm{wide}$ corresponds to the pre-treatment subsample. Let $\left( {F}_\mathrm{tall}, {\Lambda}_\mathrm{tall} \right)$ and $\left( { F}_\mathrm{wide}, {\Lambda}_\mathrm{wide} \right)$ be the factor and loading matrices associated with $Y_\mathrm{tall}$ and $Y_\mathrm{wide}$, respectively. It is easy to see that ${F}_\mathrm{tall}=F$, ${\Lambda}_ \mathrm{wide}=\Lambda$, ${F}_\mathrm{wide}$ is a sub-matrix formed by the first $T_0$ rows of $F$, and ${\Lambda}_\mathrm{tall}$ is a sub-matrix formed by the first $N_0$ rows of $\Lambda$.

In lieu of bai2021matrix, we can use the algorithm below to estimate the treatment effects $\Delta_{i,t}$ for all $(i,t)\in\mathcal{I}_1 $.

Construction of Confidence Intervals

When neither $N_1$ nor $T_1$ is large, the average treatment effects either across treated units or post-treatment periods may not be a good measure for inferential purpose since it is hard to establish the asymptotic properties of such effects. In order to provide inferential procedure for individual treatment effects especially when neither $N_1$ nor $T_1$ is large, motivated by gonccalves2017bootstrap, we adapt bootstrap method to construct the confidence intervals of $\Delta_{i,t}$ for $(i,t)\in\mathcal{I} _1$, without a specific distributional assumption on the error term $e_{i,t}$ or particular quantity on $N_1$ or $T_1$ .

Under Assumptions (ref)--(ref) in this paper, the asymptotic properties of the bootstrapped confidence intervals (ref) and (ref) are summarized in the following theorem.

From Theorem (ref), we can conclude that the bootstrap confidence intervals will provide a correct coverage for the estimated treatment effects in the post-treatment periods, and thus one could conduct statistical inference about the treatment effects based on the bootstrap confidence intervals. It also worths noting that even if our bootstrap procedure follows the idea of gonccalves2017bootstrap, the proof of bootstrap validity becomes quite complicated due to the differences in purposes and model specifications. Since gonccalves2017bootstrap intend to make inference on factor-augmented regression models bai2006confidence where the factors serves as intermediate variables, they only require valid bootstrap approximations of factors. In this paper, we intend to make inference on the factor model (with missing values) itself, and the estimators involve products of estimated factors, their associated loadings, and a rotation matrix. This implies that we need valid bootstrap approximations of factors, loadings, and the rotation matrix, which leads to laborious works.

Estimation and Inference in a Model with Covariates

In the above section, we discussed the estimation of the treatment effects and associated bootstrap confidence intervals for a pure factor model. Besides the unobserved factors, in practice, the outcome variable could also be affected by some exogenous regressors. To accommodate such a scenario, we extend the factor-based approach to a model with covariates, i.e., Model (ref) in Section (ref).

Estimation of Treatment Effects

For a factor model with covariates, i.e., a panel data model with interactive fixed effect, our estimation strategies involve an interactive fixed effect estimation (IFEE), which is described and studied in bai2009panel. If $\left\{ y_{i,t}: (i,t)\in\mathcal{I} \right\}$ were fully observed, we could directly use IFEE to estimate Model (ref). The general framework of IFEE is summarised in the algorithm to follow.

Now we turn to the estimation of treatment effect when $\left\{ y_{i,t}:(i,t)\in\mathcal{I} \right\}$ are not observable. Let $Y$, $Y_ \mathrm{tall}$, $Y_\mathrm{wide}$, $X_i$, $F$, ${F}_\mathrm{tall}$, ${F}_ \mathrm{wide}$, $\Lambda$, ${\Lambda}_\mathrm{tall}$, and ${\Lambda}_\mathrm{ wide}$ follow their definitions in Sections (ref) and (ref). Let $ X=\left( X_1, \ldots, X_N \right)$, $X_\mathrm{tall}= \left( X_1, \ldots, X_{N_0} \right)$, and $X_\mathrm{wide}$ to be a sub-matrix formed by the first $T_0$ rows of $X$.

Then we can use the algorithm below to estimate the treatment effects $ \Delta_{i,t}$ for all $(i,t)\in\mathcal{I}_1$.

Construction of Confidence Intervals

For the estimated $\widehat{\Delta}_{i,t}$ for $(i,t)\in\mathcal{I}_1$, we shall again use the bootstrap procedure discussed above to construct their confidence intervals.

As above, we can establish the validity of the proposed confidence intervals (ref) and (ref) in the sense that they have asymptotically correct coverage probabilities as $N_0,T_0\to \infty$.

Theorem (ref) suggests that we can apply the proposed bootstrap procedure to conduct statistical inference for estimated treated effects from a panel with interactive fixed effects model.

Simulation Studies

In this section, we conduct several Monte Carlo experiments to investigate the finite sample properties of the proposed confidence intervals. \footnote{MATLAB codes for simulation are available from the authors upon request.}

In the data generating processes below, we assume that the common factors $ \left\{ f_{t}\right\} $ are i.i.d.\ as $\mathbb{N}\left( 0,I_{3}\right) $, and the factor loadings $\left\{ \lambda _{i}\right\} $ are also i.i.d.\ as $ \mathbb{N}\left( 0,I_{3}\right) $. For the model with covariates, we assume that the covariates $\left\{ x_{i,t}\right\} $ are i.i.d.\ as $\mathbb{N} \left( 0,AA^{\mathsf{T}}\right) $, where each entry of the $2\times 2$ matrix $A$ is drawn from $\mathbb{N}(0,1)$. Let $\beta \sim \mathbb{N}\left( 0,I_{2}\right) $. We consider the following data generating processes. \footnote{We also consider AR(1) factors, exponentially distributed errors, other variance structures, and dependence between factors and covariates. These results are available from the authors upon request.}

The error term is defined as

where $v_{i,t}=\rho_i v_{i,t-1}+\varepsilon_{i,t}$ and $\left\{ \varepsilon_{i,t} \right\}$ are i.i.d.\ drawn from $\mathbb{N} (0,1)$. We specify two variance structures of $\left\{ e_{i,t} \right\}$.

Moreover, we consider the following marginal distributions of $\left\{ v_{i,t}\right\}$.

For demonstration, we assume there is only one treated unit $i=N$ and $5$ post-treatment periods. The treatment effects are assumed to be constants equal to 1, i.e., $ \Delta _{N,t}=1$ for $t=T_{0}+1,\ldots ,T_{0}+5$. The number of control units $N_{0}\in \{30,50,100\}$ and the number of pre-treatment periods $ T_{0}\in \{20,40\}$. We construct the $90\%$ and $95\%$, equal-tailed and symmetric confidence intervals for $\Delta _{N,t}$, $t=T_{0}+1,\ldots ,T_{0}+5$. In Step 2(1) of Algorithms (ref) and (ref), we use ordinary wild bootstrap procedure for Case 1, and block wild bootstrap procedure with block width equal to 4 for Case 2. The number of factors is either treated as known or estimated using the method of bai2002determining. For computational simplicity, a warp-speed method giacomini2013warp is applied with 2000 replications for each scenario. We report the coverage rates (in percent) of confidence intervals in Tables (ref)--(ref). In these tables, EQ stands for equal-tailed confidence intervals and SY stands for symmetric confidence intervals.

Several interesting findings can be observed from the simulation results. On the first hand, the results in Table (ref)--(ref) clearly show that the empirical coverage ratio for treatment effects from a pure factor model is quite close to the nominal values (both 90% and 95% level) when the post-treatment period is short, regardless of whether the idiosyncratic errors are heteroscedastic or serially correlated, or whether the number of unobserved factors is known or estimated from the data. On the other hand, when exogenous covariates are included for treatment effects estimation, Tables (ref)--(ref) also show that our proposed bootstrapped confidence intervals are able to provide accurate coverage ratios for the estimated treatment effects in a panel model with exogenous regressors and with heteroscedastic or serially correlated errors. In general, the simulation results confirm the validity of our proposed bootstrap procedure in providing accurate and robust confidence intervals for estimated treatment effects using a panel with interactive fixed effects.

Empirical Applications

In this section, we re-evaluate the impacts of Hong Kong's Political and Economic Integration with Mainland China as well as the effects of California's Tobacco Control Program using our proposed bootstrap procedure. \footnote{MATLAB, Python and R codes for application are available from the authors upon request.}

Hong Kong's Political and Economic Integration with Mainland China Revisited

In this subsection, we revisit the impacts of political and economic integration of Hong Kong with Mainland China, which has been analysed in hsiao2012panel. Since hsiao2012panel specify a pure factor model without covariates, we apply the methods in Section (ref) of this paper to the dataset of hsiao2012panel. For the results about political integration, quarterly real GDP growth rates from 1993Q1 to 1997Q2 of 10 countries and districts are used to form the counter-factual path of Hong Kong from 1997Q3 up to 2003Q4. The 10 countries and districts are Mainland China, Indonesia, Japan, Korea, Malaysia, Philippines, Singapore, Taiwan, Thailand and US. For the analysis of economic integration, quarterly real GDP growth rates of 24 countries and districts from 1993Q1 to 2003Q4 are used to form counter-factual path of Hong Kong from 2004Q1 to 2008Q1.

The results for the impact of political integration and economic integration with Mainland China on Hong Kong's economic growth are provided in (ref) and (ref), respectively.

As is shown in Figure (ref), the estimated treatment effects of Hong Kong political integration are of similar magnitudes and patterns as those in hsiao2012panel. In Figure (ref), the estimated treatment effects of Hong Kong economic integration are positive, as in hsiao2012panel, and confidence intervals cover 0 in most of the periods. This suggests that the impact of political and economic integration of Hong Kong with Mainland China is significant in the first few years after integration, and vanishes afterward. This observation is in general consistent with the insignificant average treatment effects across post-treatment periods in hsiao2012panel.

California's Tobacco Control Program Revisited

Now we revisit the effectiveness of CTCP on per capita cigarettes consumption and personal healthcare expenditures using the methods discussed in Section 4 of this article. In November 1988, California passed the Proposition 99, which increased California's cigarettes tax by 25 cents per pack and earmarked the tax revenue for health and anti-smoking measures. Proposition 99 triggered a wave of local clean-air ordinances in California. abadie2010synthetic used the synthetic control method for the period 1970-2000 to show that the California Tobacco Control Program had a significant impact on per capita cigarette consumption for the period 1989-2000, and that its impact continued to be enhanced over time.

Note that in the model and dataset of abadie2010synthetic, covariates do not change over time, which violates Assumption (ref)(2) of this study. To accommodate time-variant covariates, we use the dataset of hsiao2019panel, who also revisit the impact of CTCP on per capita cigarettes consumption and personal healthcare expenditures but use a set of time-variant covariates: per capita GDP obtained from abadie2010synthetic; poverty rates obtained from the National Census Bureau; educational attainment, defined as the percentage of obtaining college degree of population 25 years and over, obtained from the National Census Bureau.

In our analysis, the number of factors is estimated using the method proposed by alessi2010improved, which shows better performance than bai2002determining in this and the next applications. Both ordinary wild bootstrap procedure and block wild bootstrap procedure with block width equal to 3 are considered, and equal tailed confidence intervals are reported. The results for the impact of CTCP on per capita cigarette consumption and personal healthcare expenditures are provided in (ref) and (ref), respectively.

Figure (ref) shows that the estimated treatment effects are of similar magnitudes as those in abadie2010synthetic; and that the confidence intervals indicate negative and significant impacts over time, consistent with their findings using permutation tests. Figure (ref) shows that the estimated treatment effects of CTCP on health expenditure are of similar magnitudes as reported in hsiao2019panel. Consistent with their findings, confidence intervals indicate that the effects are short-lived. In both figures, we can see that the confidence intervals based on ordinary and block wild bootstrap procedures produce similar results.

Conclusion

In this paper, we consider the construction of confidence intervals for treatment effects estimated in panel models with interactive fixed effects, which serves as an alternative inferential approach. We first use the factor-based matrix completion technique proposed by bai2021matrix for panel models to estimate the treatment effects, and then use bootstrap method to construct confidence intervals of the treatment effects for treated units at each post-treatment period. Our construction of confidence intervals requires neither specific distributional assumptions on the error terms nor large number of post-treatment periods. We also establish the validity of proposed bootstrap procedure that these confidence intervals have asymptotically correct coverage probabilities. Simulation studies show that these confidence intervals have satisfactory finite sample performances, and empirical applications using classical datasets yield treatment effect estimates of similar magnitude and reliable confidence intervals.