Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
68,116 characters · 18 sections · 68 citation commands
Many average partial effects: with an application to text regression
\address[Harold D. Chiang]{Department of Economics, University of Wisconsin-Madison, United States}
\allowdisplaybreaks
Binary response models are some of the most commonly used nonlinear econometric models. When studying such models, the average partial effect, henceforth APE, is a popular target parameter of interest. Under big data environments, as often happens in text analysis, dimension reduction via lasso, or other type of machine learning algorithms, is often unavoidable. Failure to account for the model selection step often leads to severely biased estimates, which invalidate the usual inference procedures (see Figure 1 for an illustration). Few results are available for valid post-selection inference for a single nonlinear functional of high-dimensional nuisance parameters, such as APE, let alone simultaneous inference for potentially many of such parameters. To fill this void, this paper considers simultaneous inference and confidence intervals for lasso Logit APEs. In addition, as we illustrate in simulation studies in Section (ref) that ignoring cluster sampling can lead to severely distorted testing results, even when cluster sizes are small. The size distortion is further aggravated in testing multiple hypotheses. Importantly, all our theoretical results stay valid for cluster-sampled data with heterogeneous cluster sizes.
To our knowledge, this is the first paper handling multiple testing and simultaneous confidence interval problems for more than a single APE under high-dimensional or big-data environments. In addition, cluster sampling with heterogeneous cluster sizes is allowed. Using the Neyman orthogonalization technique, we propose a new lasso-based post-double selection APE estimator. To accompany the main theoretical results, we propose valid nuisance parameter estimators as well as their practical tuning parameter selection algorithms that are compatible with our theory. To address the multiple-testing problem, we develop a new, simple-to-implement, multiplier cluster bootstrap. We provide simple algorithms for testing high-dimensional hypotheses and constructing simultaneously valid confidence intervals. Simulation studies suggest the proposed methods have favorable finite-sample performance. We illustrate the applicability of our theoretical results through examining a claim of Wu18 on the presence of genderally biased use of language following Wu's text regression model using internet forum textual data from Economics Job Market Rumors (EJMR) forum - see the following section.
Text analysis using machine learning algorithms has become a useful alternative to the more traditional data analysis used in economics and other social sciences. Popular categories of text analysis models include text regression models, generative models, dictionary-based methods and word embeddings. The first two categories link attributes and word counts through conditional probabilities\footnote{Roughly speaking, given attributes $v_i$ and word counts $c_i$, a text regression model considers ${\mathrm{P}}(v_i|c_i)$ and a generative model considers ${\mathrm{P}}(c_i|v_i)$. } and, therefore, naturally relate to common econometric models. Notable examples of applications using text regression include stock prices prediction (e.g. JW13) and the Google Flu Trends, which is summarized in G_etal09, among others. GST19 is a representative recent example for generative models applied to economics. For more details and applications, see GKT19 for an up-to-date review.
Using a text regression model, Wu18 examines how women and men are discussed and depicted in the anonymous Economics Job Market Rumors forum. The author first extracted a list of female/male classifier vocabularies. According to Wu, a post is considered to be female if it contains any female classifier and male if it contains any male classifier\footnote{Wu makes use of a classification procedure to decide the posts that contains both female and male classifiers. See Section II A of Wu18 for more details}. Let $Female_i$ be an indicator of whether post $i$ is female. $X_i$ denotes a vector of counts for each of the top 10,000 most common words\footnote{It is also possible to use frequency and $n$-grams in place of word count and words, respectively, as suggested in GKT19. } (excluding all gender classifiers) that are present in gendered post $i$. Wu considers the text regression model with the logistic\footnote{For text regression models with binary attributes, a penalized logistic model is recommended by GKT19; see their Section 3.1.1 for more details.} specification,
where $\Lambda$ is the logistic function, using a lasso Logit procedure. The $Male$ counterpart is estimated analogously. For interpretability, Wu computes estimates for the APE for each of the $9,540$ words, where the APE for the word count of the $k$-th word is defined as
Some top estimates of Wu's are listed in Table (ref). Based on these results, Wu concludes that the words that predict a post about a woman are typically about physical appearance or personal information, whereas those most predictive of a post about a man tend to focus on academic or professional characteristics.
Wu18 focuses on estimation. To further examine the magnitude and statistical significance of these estimates, the researcher may be interested in conducting hypotheses testing or constructing confidence intervals. To do so, several issues need to be carefully accounted for. First, as posts in EJMR data of Wu18 are sampled from different threads of various discussion topics, it is likely that posts coming from the same thread are highly correlated. Therefore, statistical testing should be conducted using a cluster robust inference method. Secondly, Wu18 highlights that females are often described with words about appearance or personal information. To formally quantify such statements, one may want to conduct multiple testing for APEs of a (potentially large) set of vocabularies related to appearance or personal information. Furthermore, in many cases, words with the same or close meaning are double-counted in this dataset, e.g. “attractive" and “attractiveness" or “homo", “homosexual," and “gay."\footnote{If the researcher is only concerned about joint testing, an easy alternative is to combine these words. However, this is not desirable when one wants to obtain separate estimates.} Thus, the researcher may want to consider a joint test that controls family-wise error rates for APEs of these words. This results in a multiple testing problem. Therefore, the testing procedure needs to be able to control the family-wise error rate while testing potentially many variables. To our best knowledge, no method in the literature is capable of addressing all these issues simultaneously. This paper attempts to provide a useful and easy-to-implement method that can be applied to such problems.
Our main contribution is to provide a theory for high-dimensional multiple-testing and simultaneous confidence intervals for APEs of binomial and fractional response regression models under clustered data. To our best knowledge, no results were previously available for this purpose. As a by-product, this paper also complements existing papers by proposing a practical method for studying low-dimensional APEs of interest under high-dimensional settings. Furthermore, cluster sizes are allowed to be heterogeneous - this is essential to our application as number of posts varies from thread to thread. Inference and construction of confidence intervals for such models are practically challenging; despite that methods are proposed in the literature, no simulation evidence for inference of even a single APE under lasso-regularization with these methods is available. In addition, we present practical and theoretically justified penalty choices for all the lasso estimators. Furthermore, easy-to-implement bootstrap procedures are also provided for inference/confidence intervals that hold valid, regardless of whether the researcher is interested in one or multiple APEs.
The past decade has seen an explosive development in the literature of post-selection inference for lasso-based high-dimensional methods. This includes BCCH12 for instrumental variable models, BCH14, JM14, ZZ14, Farrell15, CK18 and AIW18 for linear regression/treatment effects models. Post-selection inference for generalized linear models such as Logit has been studied by vdGBRD14, BCK15, BCW16, BCFH17 and BCCW18, to list a few. This line of research predominately focuses on regression coefficients of the generalized linear models rather than nonlinear functionals such as an APE. Recently, CNS18 study $L_2$-continuous functionals using lasso and Dantzig selector. While focusing on affine-functionals, they provide an extension of their method to nonlinear functionals. Their method makes use of a linear Riesz representer to approximate the linearization of a nonlinear functional, which differs from our approach. In addition, all of the aforementioned papers are based on i.i.d. or independent sampling assumptions. On the other hand, there are some results available for high-dimensional linear panel data. This includes BCHK16, Kock16 and KT18.
Cluster-robust inference under various fixed-dimensional parametric settings has been well-studied and widely applied in the literature. See Wooldridge10 and CM15 for textbook treatment and comprehensive reviews. There has been recent literary focus on cluster-robust bootstrap inference. This includes KS12, Hagemann17, MW17 and DMN18, among others. These results cannot be generalized in a straightforward manner to high-dimensional settings as the delta-method does not, in general, hold in an asymptotic framework with increasing dimensionality, see Caner17 for more details.
APE for binomial/fractional regression models has been discussed extensively in the literature (cf chamberlain84, wooldridge05 and wooldridge18, etc). Inference for APEs of lasso-based binomial regression models are first studied by WZ17 under a short (balanced) panel data setting. They make use of a single-selection step with a lasso Probit estimator and propose a de-biased estimator for a single APE and obtain asymptotic normality. More recently, HW18 highlight the estimator of WZ17 for its requirement of a “soft" beta-min assumption that rules out regularization bias asymptotically\footnote{Such post model selection inference issues are widely discussed in the literature (see e.g. PL09 and the reference within).}. For i.i.d. data, HW18 provide an alternative augmented minimax estimator based on the novel framework for linear functionals developed in HS17. However, no variance estimator for this approach is proposed. Also, the aforementioned results are available only for a single APE; multiple testing and simultaneous confidence intervals for more than one APE remain unavailable. In addition, implementing inference for even a single APE under such settings presents practical challenges; to our best knowledge, there has been no simulation evidence presented for the proposed estimators in the aforementioned papers.
This paper aims to address all the aforementioned issues simultaneously. To do so, we extend the general framework for i.i.d. data developed in the important works of BCK15 and BCCW18 to allow for cluster sampling and adapt it to the studies of APEs. The pointwise/simultaneous inference and confidence intervals are based on a multiplier cluster bootstrap which is built upon the high-dimensional central limit theorem of CCK13.
Denote $(\Omega,\mathcal A)$ the underlying measurable space and for each $G\in \mathbb N$, $\mathcal P_G$ is a set of probability measures ${\mathrm{P}} \in \mathcal P_G$ defined on $\mathcal A$. Consider triangular array data $\{W^G_{g}: g=1,...,G, G=1,2,3,...\}$ defined on probability space $(\Omega, \mathcal{A}, {\mathrm{P}})$, where ${\mathrm{P}}$ depends on $G$ through $\mathcal P_G$. Each $W^G_{g}= \{ W^G_{ig}: 1\le i \le n_g \}$, is a random vector that is independent across $g$, but not necessarily identically distributed. All parameters that characterize the distribution of $\{W^G_g; g=1,...,G\}$ are implicitly indexed by ${\mathrm{P}}_G$ and thus by $G$. This dependence is henceforth omitted for simplicity. $W_{ig}=(Y_{ig},X_{ig}')'$ takes values in $\mathbb R^{p+1}$. For each $g\le G$, $G\in\mathds N$, the deterministic size of cluster $n_g$ satisfies $1\le n_g\le\bar n$ for a constant $\bar n$ independent of $G$. Therefore, for $i$ such that $n_g<i \le \bar n$, we can set $W_{ig}=0$ and thus each $W_g$ can be represented as a $\bar n(p+1)$-dimensional random vector. Let ${\mathrm{E}}_\mathrm{P}$ be the expectation with respect to law ${\mathrm{P}}$.
For a vector $\beta$, the $k$-th component is denoted as $\beta_k$. For vectors, denote the $\ell_1$-norm as $\|\cdot\|_1$, ${l}_2$-norm as $\|\cdot\|$, $\ell_\infty$-norm as $\|\cdot\|_\infty$, and the “$\ell_0$-norm" as $\|\cdot\|_0$ to denote the number of non-zero components. For a matrix $A$, let $A'$ be the transpose of $A$. For $1\le q < \infty$, $\|A\|_{q}$ denotes the induced ${l}_q$-norm and $\|A\|_\infty=\max_{1\le j,k\le p}|A_{j,k}|$. For a vector $\delta \in {\mathbb{R}}^{p}$ and given data, $\| X_{ig}'\delta \|_{G} = \sqrt{ \frac{1}{G} \sum_{g=1}^G \sum_{i=1}^{n_g}(X_{ig}'\delta)^{2}}$ denotes the prediction norm of $\delta$. Let $e_j$ be the $j$-th vector of the standard basis for $\mathbb R^p$. Given a vector $\delta \in {\mathbb{R}}^p$, and a set of indices $T \subseteq \{1,\ldots,p\}$, denote $\delta_T \in {\mathbb{R}}^p$ the vector such that $(\delta_{T})_{j} = \delta_j$ if $j\in T$ and $(\delta_{T})_{j}=0$ if $j \notin T$. The support of $\delta$ is defined as ${\rm support}(\delta) = \{ j \in \{1,...,p\}: \delta_j \neq 0\} $. We denote $a \vee b = \max\{ a, b\}$, and $a \wedge b = \min\{ a , b \}$. The notaion $[k]=\{1,...,k\}$ is used for $k\in\mathbbm N$. We use $c$, $C$ to denote strictly positive constants that is independent of $G$ and ${\mathrm{P}}\in \mathcal P_G$. Their values may change at each presence. The notation $a_G \lesssim b_G$ denotes $a_G \le C b_G$ for all $G$ and some $C>0$ that does not depend on $G$. $a_G=o(1)$ means that there exists a sequence $(b_G)_{G\ge 1}$ of positive numbers that do not depend on ${\mathrm{P}}\in \mathcal P_G$ for all $G$ such that $|a_G|\le b_G$ for all $G$ and $b_G = o(1) $ as $G $ converges to zero. $a_G\lesssim_P b_G$ means that for any $\epsilon>0$, there exists $C$ such that ${\mathrm{P}}_{\mathrm{P}}(a_G > C b_G)\le \epsilon$ for all $G$. Throughout the paper we assume $G \ge 3$.
The rest of the paper is structured as follows. In Section (ref), an overview of the method and algorithms are given. Section (ref) contains the main asymptotic results. Section (ref) covers algorithms for penalty choices and the auxiliary results for theoretical performance of nuisance parameters. Results of simulation studies are demonstrated in Section (ref). In Section (ref), we apply the proposed method to conduct simultaneous testing to verify a statement about gendered language in Wu18. We concludes in Section (ref). All the mathematical proofs and additional details are delegated to the appendix.
Recall that $W_{ig}=(Y_{ig},X'_{ig})'$. Suppose that the researcher observes data sampled from $G$ clusters, $\{W_{ig}:i=1,...,n_g,\:g=1,...,G\}$. Each cluster size $n_g$ is considered non-random, and $1\le n_g\le \bar n<\infty$ for a constant $\bar n$ that does not depend on $G$. Denote $n=\sum_{g=1}^G n_g$. Throughout the paper, we assume that the conditional expectation of $Y$ given $X$ follows the following single-index structure
for each cluster $g$. Any $W_{i_1 g},W_{i_2 g}$ can be arbitrarily correlated while any $W_{i_1 g_1}, W_{i_2 g_2}$ are independent if $g_1 \ne g_2$. The dimensionality of $\beta^0$ is allowed to increase with $G$. This is the population-averaged approach as $\beta^0$ represents an averaged parameter after integrating out heterogeneity. The target parameter is the APE with respect to the $k$-th continuous covariate of interest,
where $\Lambda'$ stands for the derivative of $\Lambda$. As $n_g \le \bar n $, it suffices to consider $\alpha_k$, the rescaled APE\footnote{The original APE can be simply recovered by $\text{APE}_k=(G/n)\cdot\alpha_k $.} defined as
We now summarize the estimation, inference and construction of simultaneous confidence intervals procedures based on the theoretical results to be presented in Section (ref) and (ref) ahead. First, we describe the procedures for computing the proposed APE estimators. Set $\alpha_k$ as the parameter of interest. The post-double-selection estimator for $\alpha_k$ is defined as
where $\check \beta^k$ is the pooled Logit estimate with its support restricted to the set of covariates
and $\widehat \beta$, $\widehat \zeta^k$ and $\widehat \gamma^k$ are nuisance parameter estimators to be defined below. Therefore, once $\widetilde T_k$ is obtained, estimation of $\widetilde \alpha_k$ becomes a standard pooled Logit problem.
Suppose that we have some generic penalty tuning parameters $\lambda$, $\lambda^\gamma_k$ and $\lambda_k^\zeta$ and, in addition, $\widehat \Psi$, $\widehat\Psi^\gamma_k$, $\widehat\Psi_k^\zeta$, diagonal normalization matrices of dimensions $p$, $p-1$ and $p$, respectively. Formal and theoretically justified choices of these objects are delayed to Section (ref).
First, $\widehat \beta$ and its two post-lasso counterparts are defined as
Using the above post-lasso estimates, we compute $\widehat f^2_{ig}=\Lambda'(X_{ig}'\widetilde\beta)$ and $\widehat S_{ig}^k= \widetilde\beta^k_k \cdot \{1-2\Lambda(X_{ig}'\widetilde\beta)\}$. Throughout the rest of this paper, denote $D^j_{ig}=X_{ig,j}$, the $j$-th component of $X_{ig}$, and $X^j_{ig}=X_{ig,-j}'$, the remaining $p-1$ variables. Using these quantities, the remaining two nuisance parameter estimates can be obtained as
Now $\widetilde T_k$ can be calculated following ((ref)) and thus
and $\widetilde \alpha_k$ can be obtained following equation ((ref)).
Suppose that the researcher is interested in $\alpha_k$ for a set of continuous covariates with $k\in A$ for an index set $A \subset [p]$\footnote{There is no restriction on the cardinality of $A$. $A=[p]$ is also allowed.}. We present a concrete estimation procedure as the following algorithm.
For inference, let us define the post-lasso counterparts of $\widehat \gamma^{k}$ and $\widehat \zeta^{k}$
and the nuisance parameter estimate
where each $\widehat \tau^2_k$ is calculated using
Define the additional nuisance parameter estimate
Finally, define the variance estimate as
We are now ready to introduce a procedure for simultaneous inference. Suppose that the null hypothesis of interest is
for some values $(\alpha^0_k)_{k\in A}$. We present a concrete simultaneous inference procedure as the following algorithm.
Finally, we illustrate the procedure for constructing simultaneously valid confidence intervals with $(1-a)$ coverage probability for $\alpha_k$, $k\in A$.
In this section, we present our main theoretical results for simultaneous inference and construction of confidence intervals. These results justify the validity of the algorithms proposed in Section (ref). First, we introduce some notations. Recall
Define the Neyman orthogonal score for $\alpha_k$ by
where $\psi_k (W_{ig},\eta)= \beta_k \Lambda'(X_{ig}'\beta)-\mu'X_{ig} \{Y_{ig}-\Lambda(X_{ig}'\beta)\}$. In addition, let the “ideal" population nuisance parameters\footnote{See Section (ref) in the Appendix for derivation of this moment condition.} for $\alpha_k$ be $\eta^k=(\beta^{0\prime},\mu^{k\prime})'\in \mathbb R^{2p}$ with
where $S^k_{ig}=\beta_k^0 (1-2\Lambda(X_{ig}'\beta^0))$ is an auxiliary regressor and $f^2_{ig}=\Lambda'(X_{ig}'\beta^0)$ is a regression weight. Also denote the population nodewise regression coefficients for the $j$-th covariate as $\gamma^j$. We can also rewrite the population nuisance parameter regression coefficients $\zeta^j$ as a weighted projection of $S^j_{ig}$ on $X_{ig}$. Thus, we have the following
Denote the projection errors by $Z^j_{ig}=D^j_{ig}-X^j_{ig}\gamma^j$ and $\varepsilon^j_{ig}=S^j_{ig}-X_{ig}\zeta^j$. Let $q>4$ be a constant independent of $G$. Let $c_1$ and $C_1$ be some strictly positive constants independent of $G$. Furthermore, let $a_G=p \vee G$ and $\check \delta_G $ be a sequence of positive constants that converge to zero. $M_{G,1}\ge 1$ and $M_{G,2}\ge 1$ be some sequence of positive constants possibly diverging to infinity. $s=s_G$ is a non-decreasing sequence of constants. We make the follow assumptions.
For the following assumption, define $U_{gk}= \bar n\cdot \max_{i \in [n_g]} |X_{ig,k}|$, $U_g=[U_{gk}]_{k \in [p]}$ and $V^j_g=\max_{i \in [n_g]} (|Z^j_{ig}|\vee |\varepsilon^j_{ig}|)$.
A proof can be found in Section (ref) in the Appendix. Now, it remains to find a valid variance estimator. Recall the variance estimator $\widetilde \sigma_k^2 $ defined in ((ref)). Denote $\widetilde \sigma_k = \{\widetilde \sigma_k^2\}^{1/2} $.
A proof can be found in Section (ref) in the Appendix.
\\ Recall that the “ideal" nuisance parameter vector $\eta^k=(\beta^{0\prime},\mu^{k\prime})'$, where
In this section, we propose estimators for these nuisance parameters as well as some theoretically justified choices of penalty tuning parameters. The choices here are based on the moderate deviation theory of self-normalized sums, which is first adapted for penalty selection of lasso by BCCH12. Throughout this section, we fix a positive integer $\bar m \ge 1$ as the number of iterations used in the algorithms for choosing penalty tuning parameters.
We now establish an asymptotic theory for estimation of $\beta^0$, which plays a central role in estimation of APE. The identification of $\beta^0$ follows from quasi-maximum likelihood and the assumption of population-averaged approach ${\mathrm{E}}[Y_{ig}|X_{ig}]=\Lambda(X_{ig}'\beta^0)$. Define the negative partial log-likelihood function by
Then, one has
We propose the following algorithm for the choice of $\widehat \Psi$.
The following result provides convergence rates of $\widetilde \beta$ and $\widetilde \beta^k$.
A proof can be found in Section (ref) in the Appendix.
We now establish asymptotic theory for weighted post-lasso with estimated weights that will be essential for Sections (ref) and (ref). We propose the following algorithm for the choices of penalty tuning parameters.
The following result provides convergence rates of $\widetilde\gamma^j$, which plays an important role in Section (ref).
A proof can be found in Section (ref) in the Appendix.
Now we provide estimators for $\theta^k$ that are built upon the method of cluster nodewise post-lasso estimator for approximately inverting a singular matrix. The theory developed here is based on applying the weighted post-lasso with estimated weights from BCCW18 to the panel nodewise regressions of Kock16. Recall that each nuisance parameter vector $\theta^k$ contains the matrix
Its sample counterpart is not invertible if $p>n$ and could be very unstable if $p$ is only moderately larger than $n$. Here, we take advantage of Assumption (ref) to construct a high quality approximate inverse estimate. Denote $\Theta_{j}$ for the $j$-th row written as a column vector. If we can find some reasonable estimator $\widehat\Theta_k$ for $\Theta_k$, then intuitively an estimator for $\theta^k$ can be defined as
We propose a cluster nodewise post-lasso procedure to estimate $\Theta$. Recall that the error $Z^j_{ig}=D^{j}_{ig}- X^j_{ig}\gamma^{j}$ which satisfies $ {\mathrm{E}}_\mathrm{P}[\frac{1}{G}\sum_{g=1}^{G} \sum_{i=1}^{n_g} f^2_{ig} X^j_{ig} Z^j_{ig}]=0$. Define the error variance $\tau^2_{j}= {\mathrm{E}}_\mathrm{P}[\frac{1}{G}\sum_{g=1}^{G} \sum_{i=1}^{n_g} f^2_{ig} ( Z_{ig}^{j})^2]$. Some properties of $\tau_j^2$ can be found in Section (ref). Note that $\gamma^j$ has a sparse approximation $\bar \gamma^j$ under Assumption (ref). Then, we can use post-lasso estimate $\widetilde \gamma^j$ for $\gamma^j$ from Section (ref) and construct a $p\times p$ matrix $\widehat C$ by
That is, the off-diagonal spots of the $j$-th row of $\widehat C$ consist of components of $- \widetilde \gamma^j$ and the diagonal entries are set to $1$. Also, denote
where $ \widehat \tau_{j}^2$ is defined in ((ref)). Now, the cluster nodewise post-lasso estimator for $\Theta$ is defined as
which in turn gives the expression of ((ref)). The following results provide validity of $\widehat \Theta$ and $\widetilde \theta^k$.
Proofs for the above two results can be found in Sections (ref) and (ref) in the Appendix.
Recall that the nuisance parameters $\zeta^k$ is identified by
We propose the following algorithm for choice of the penalty tuning parameters.
The following result provides convergence rates of $\widetilde\zeta^k$.
A proof can be found in Section (ref) in the Appendix.
In this section, we conduct simulation studies to investigate the finite-sample performance of the proposed procedures. Set the number of total observations to $n$, and each observation is then randomly assigned into $G_0$ clusters. The empty clusters, if they exist, are then discarded and thus $G\le G_0$. For DGP1, let the number of covariates for each observation be $p=1.5\cdot G_0$ and
The first component of each covariate vector is set to $1$ and the rest of the subvector, $X_{ig,-1}$, can be decomposed into an idiosyncratic part $X^1_{ig}$ and a cluster-wise component $ X^2_g$ as
and both $X^1_{ig}$ and $X^2_g$ are i.i.d. following a multivariate normal distribution with mean $0$ and a Toeplitz covariance matrix:
So the larger $\rho$ is, the more correlated the covariates are. The outcome variable is generated by
where the error term can also be decomposed into an idiosyncratic term and a cluster-wise term as
where both $U^1_{ig} $ and $U^2_g$ are i.i.d. following the normal $N(0,1/2)$ distribution. Thus, $U_{ig}$ is a standard logistic distribution. Thus both covariates and errors are correlated within each cluster. To consider “outliers" and substantial skew and kurtosis in marginal distribution of independent variables, we also consider alternative DGPs inspired by KS12 by setting $X^1_{ig}$ and $X^2_g$ to follow a mixture between two distributions, $N(0,\Sigma(\rho))$ with probability 0.9 and a $N(0,\Sigma(\rho))-1.5\times N(1,\Sigma(\rho))$ with probability $0.1$.
Note that for the DGPs with high $\rho$ values, such as M4, M5, M9 and M10, the approximate sparsity conditions in Assumption (ref) are violated. We conduct three sets of simulations. First we examine one-dimensional confidence interval coverage for $\alpha_2$ with true underlying $\beta_2\in\{0,0.25,0.5,0.75,1\}$. Our second goal is to construct simultaneous confidence intervals that control the family-wise error rate for $\alpha_k$ for $k\in A$, $A$ is set to be
where the APE with respect to the intercept is always omitted. In this group of simulations, we set $\beta_2=0.5$. Finally, we examine the asymptotic behaviors of coverage probabilities of simultaneous intervals for $A_{10}$.
The estimation of all lasso and lasso Logit are conducted using R package glmnet and the penalty choices follow Algorithms (ref), (ref) and (ref) in Section (ref) with $\bar m=1 $. For each iteration of the simulation, we set the number of bootstrap iterations to $B=600$. We then simulate $1,000$ times for each DGP. The simultaneous confidence intervals are constructed following Algorithm (ref) and without normalization by $\widetilde \sigma_k$ for simplicity. The true $\alpha_k$ are computed using $3,000,000$ additional observations generated independently from data following the same marginal distribution as $X_{ig}$. The nominal coverage probability is set to be $0.95$. The results for one-dimensional confidence intervals are presented in Table (ref). \qquad\\
\quad\\ We now illustrate the necessity of cluster robust method under many-small-cluster asymptotics. Consider DGP M1-M5. Suppose we implement observation-wise multiplier bootstrap (i.e. in each iternation, i.i.d. standard normal r.v.'s $\xi_i^b$ are generated for each observation) in place of the proposed multiplier cluster bootstrap. The results are presented in Table 4. One can see that in most cases, the non-cluster robust method is severely oversized while the multiplier cluster bootstrap has close to nominal coverage rates consistently. \qquad\\
\quad\\
We now present the coverage probabilities for simultaneous confidence intervals for different sets of covariates. For this part of experiments, we focus on models M1 to M5 with $\beta_2=0.5$. The results are shown in Table (ref).
\qquad\\
As before, we now illustrate the need of cluster robust method in simultaneous inference. As Table 6 shows, the oversize problem of non-cluster robust method is aggravated in simultaneous inference.
Finally, we investigate the asymptotic behaviors of the case with $A_{10}$, one of the worst-performing cases in the above simulations for simultaneous confidence intervals, to examine whether the performance improves as sample size increases. In this set of simulations, set $\beta_2=0.5$ for number of nominal clusters $G_0=200$, $400$, $600$ and $800$, $p=1.5\cdot G_0$, and $n=2.5\cdot G_0$. The results are presented in Table (ref).
\quad\\
In all of the simulation experiments, the coverage probabilities are mostly reasonably close to the nominal coverage rate when $\rho$ is not close to one. When $\rho$ is high, the approximate sparsity of nuisance parameters in Assumption (ref) is potentially violated. Thus some of the coverage probabilities deviate away from the nominal rate. In addition, the coverage probabilities improve as sample size increases. In summary, the outcomes of these experiments are consistent with our theoretical results.
In this section, we apply our method of simultaneous inference for APEs in the text regression model of Wu18 introduced in Section (ref). We make use of the pronoun sample (gendered posts including either female or male pronouns) from Wu18. Following Wu, using the EJMR dataset\footnote{The dataset is publicly available at url:https://www.aeaweb.org/articles?id=10.1257/pandp.20181101}, we exclude the same list of words from the 10,000, including all gender classifiers, plus names of non-economist celebrities. We conduct our analysis based on the subset of non-duplicate posts that are used as the test sample for selecting optimal probability threshold in the original paper (the posts with index labelled as test0) for classification of posts that contains both female and male classifiers. We consider only pronoun sample. This leaves 46,502 posts sampled from 31,739 threads and 9541 covariates\footnote{Since the number of observations is larger than dimensionality of parameters, regular Logit and even OLS can be applied here. We have attempted to implement Logit using glm package in R. However, it did not finish after 70 minutes. OLS on the other hand takes 55 minutes to complete. In contrast, the proposed estimation and inference algorithms, when applied to the testing problem in this section, takes about two minutes to complete.} that consists of an intercept and the word counts of 9,540 non-excluded vocabularies.
Wu18 highlights that posts about males include more academically and professionally oriented vocabularies, such as “adviser," “supervisor," and “Nobel." To see the joint significance of these words' APE in terms of predicting female, we test
Following the penalty choices of Algorithms (ref), (ref) and (ref), the estimates of APEs of these words calculated using Algorithm (ref) are listed in Table (ref). These estimates are qualitatively similar to the corresponding estimates in Wu18. Using multiplier cluster bootstrap with $10,000$ bootstrap iterations, we obtain the test results listed in Table (ref). Note that under all three confidence levels, we reject the null hypothesis and the statistical evidence supports Wu's statement \footnote{One may be concerned about the high-correlation between "supervisor" and "adviser." However, removing either one of them does not change the significance of the tests at $99$% confidence level.}.
\qquad\\
\qquad\\
In this paper, we study logistic average partial effects with lasso regularization when data is sampled under clustering. We proposed two valid estimators along with their theoretically justified lasso penalty choices. Based on these estimators, we provide easy-to-implement algorithms for simultaneous inference and confidence intervals and establish their asymptotic validity. Simulation studies demonstrate that the proposed procedures work as predicted by the theory in finite sample. We then apply the proposed method to conduct analysis of textual data to examine the presence of gendered language on the EJMR forum following the text regression model of Wu18. Our analysis provides further statistical evidence to support Wu's finding.
\allowdisplaybreaks