Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
45,713 characters · 7 sections · 20 citation commands
Assessing Outcome-to-Outcome Interference in Sibling Fixed Effects Models
\pagenumbering{gobble}
Sibling fixed effects (FE) models are useful for estimating causal treatment effects while offsetting unobserved sibling-invariant confounding. However, treatment estimates are biased if an individual’s outcome affects their sibling’s outcome. We propose a robustness test for assessing the presence of outcome-to-outcome interference in linear two-sibling FE models. We regress a gain-score—the difference between siblings' continuous outcomes—on both siblings' treatments and on a pre-treatment observed FE. Under certain restrictions, the observed FE’s partial regression coefficient signals the presence of outcome-to-outcome interference. Monte Carlo simulations demonstrated the robustness test under several models. We found that an observed FE signaled outcome-to-outcome spillover if it was directly associated with a sibling-invariant confounder of treatments and outcomes, directly associated with a sibling’s treatment, or directly and equally associated with both siblings’ outcomes. However, the robustness test collapsed if the observed FE was directly but differentially associated with siblings’ outcomes or if outcomes affected siblings’ treatments.
\scriptsize
Acknowledgements: This work was supported by the Eunice Kennedy Shriver National Institute for Child Health and Human Development through the Population Research Center at the University of Texas-Austin (P2CHD042849) and through the Center for Demography at the University of Wisconsin-Madison (T32 HD007014-42). This content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health. I thank Felix Elwert for comments and discussions that informed this manuscript. Conflicts of interest: none.
\pagenumbering{arabic} \setcounter{page}{1}
Sibling fixed effects (FE) models are commonplace in observational epidemiologic research for estimating causal treatment effects while controlling for unobserved sibling-invariant confounding, such as shared genetic risk factors.frisell2012, khashan2014, khashan2015, hvolgaard2016, sjolander2016, hanley2017, axelsson2019, mallinson2020, petersen2020, frisell2021, mallinson2021 These models rely on strong assumptions—in particular, an individual's outcome cannot affect their sibling's treatment nor outcome.frisell2012, sjolander2016, frisell2021, mallinson2021 Such assumptions are precarious because siblings' health are interdependent.feinberg2012, deneve2017 One can prevent outcome-to-treatment interference by measuring all treatments before all outcomes within sibships. However, study design cannot circumvent outcome-to-outcome interference, thereby threatening the validity of any sibling FE analysis. Statistical tests like regressions with negative controls can assess potential bias,lipsitch2010, cunninghambook although there is little guidance for their use in gauging outcome-to-outcome interference in sibling comparison studies.
We explain a robustness test for assessing outcome-to-outcome interference in linear two-sibling FE models. The method involves regressing a gain-score—the difference in siblings' outcomes—on both siblings' treatments and on a pre-treatment observed FE that is related to siblings’ treatments or outcomes. Using graphical causal modeling and Monte Carlo simulations, we demonstrate when the partial regression coefficient for the observed FE signals outcome-to-outcome interference across several models.
Our baseline sibling FE model (Figure 1A) reflects a typical sibling comparison design.sjolander2016 Subscripts $i=\left\{1,\ldots,N\right\}$ and $j=\left\{1,2\right\}$ denote cluster and sibling, respectively. $T_{ij}$ is a binary or continuous treatment, $Y_{ij}$ is a continuous outcome, $U_i$ is an unobserved FE (e.g., family health history), and $D_i=Y_{i2}-Y_{i1}$ is the gain-score. Arrows and adjacent Greek letters denote causal effects: $\delta$ is the treatment effect ($T_{ij}\rightarrow Y_{ij}$); $\psi$ is the unobserved confounding effect on outcomes ($U_i\rightarrow Y_{ij}$); and $\chi$ ($U_i\rightarrow T_{i1}$) and $\gamma$ ($U_i\rightarrow T_{i2}$) are unobserved confounding effects on treatments. This model embeds two assumptions. First, the treatment effect $\delta$ and the confounding effect $\psi$ are equal for both siblings. Second, all effects are linear and homogeneous. Both assumptions align with conventional linear FE models.sjolander2016, kim2019, kim2021, gunasekara2014,imai2019 For accessibility, we omitted sibling-specific covariates, which are compatible with the baseline model and all subsequent models as long as one can condition on them without loss of generality.
\afterpage{
}
Following the baseline model, $\delta$ is our estimand. Regressing $Y_{ij}$ on $T_{ij}$ will not identify $\delta$, covariate adjustment notwithstanding, because $U_i$ confounds both variables. This motivates the utility of FE estimators that isolate treatment effects while controlling for certain types of unobserved confounding.sjolander2016, mallinson2021,kim2019,kim2021,gunasekara2014,imai2019 The gain-score is one such estimator, and we demonstrate its mechanics and susceptibility to outcome-to-outcome interference in simple linear regression. We regress the gain-score on siblings’ treatments,
where $e_i$ is the residual. Through outcome differencing, the gain-score estimator offsets unobserved confounding bias such that the treatment-conditional covariance between $U_i$ and $D_i$ ($\sigma_{U_iD_i\cdot T_{i1}T_{i2}}$) is zero, thereby isolating the treatment effect.mallinson2021, kim2019, kim2021 The baseline model includes four pathways that “transmit” bias from $U_i$ into $D_i$:
Covariate adjustment on $T_{i1}$ and $T_{i2}$ closes paths 1 and 2, and paths 3 and 4 cancel each other out exactly because the association that flows along path 3 ($-\psi$) is equal to the negative value of the association that flows along path 4 ($\psi$). Thus, $\sigma_{U_iD_i\cdot T_{i1}T_{i2}}=0$, and $b_2=\delta$ identifies the treatment effect precisely (likewise, $b_1=-\delta$). See Appendix for details.
The estimator’s validity relies on absent outcome-to-outcome interference. Otherwise, we cannot identify $\delta$. The model in Figure 1B introduces outcome-to-outcome interference $\eta$ ($Y_{i1}\rightarrow Y_{i2}$). There are now two additional pathways from $U_i$ to $D_i$:
Covariate adjustment on $T_{i1}$ blocks path 5. However, path 6 is open as a result of outcome-to-outcome interference, so $\sigma_{U_iD_i\cdot T_{i1}T_{i2}}\neq0$, thereby preventing treatment identification (i.e., $b_2\neq\delta$ and $b_1\neq-\delta$). See Appendix for details.
Given that outcome-to-outcome interference in sibling FE models biases treatment identification, we may consider statistical tests for gauging this source of bias. An ideal test is a gain-score regression that adjusts for $U_i$:
Under the models in Figures 1A and 1B, $b_U=0$ indicates no outcome-to-outcome spillover ($\eta=0$) because $\sigma_{U_iD_i\cdot T_{i1}T_{i2}}=0$, whereas $b_U\neq0$ indicates outcome-to-outcome spillover ($\eta\neq0$) because $\sigma_{U_iD_i\cdot T_{i1}T_{i2}}\neq0$. This hypothetical test is straightforward but impossible; $U_i$ is unobserved, so we cannot adjust for it. Indeed, adjustment for $U_i$ would render sibling FE analysis obsolete—simply regress $Y_{ij}$ on $T_{ij}$ and $U_i$ to identify $\delta$.
Still, we may measure cluster-level characteristics, raising the possibility of adjusting for an observed FE in the gain-score regression. The model in Figure 1C introduces $C_i$, a binary or continuous observed pre-treatment FE, and $U_i^\prime$, an unobserved FE that is equivalent to $U_i$ without $C_i$. The dashed line between $U_i^\prime$ and $C_i$ indicates a direct and linear association that is equal to $\pi$. An example of a potential observed FE is maternal nativity, which is a sibling-invariant characteristic that is associated with family health history and pregnancy-related outcomes in the United States.qin2010, creanga2012 Additionally, $C_i$ should precede siblings’ treatments to prevent bias from conditioning on a mediator variable between the treatment and outcome or from conditioning on a post-treatment collider.elwert2014, vwbook
Under the model in Figure 1C, we repeat the gain-score regression and adjust for $C_i$:
There are four pathways from $C_i$ to $D_{i}$:
Covariate adjustment on siblings’ treatments blocks paths 1 and 2, and outcome differencing offsets the associations from paths 3 and 4. Therefore, $\sigma_{C_iD_i\cdot T_{i1}T_{i2}}=0$, so $b_C=0$.
We then consider the model in Figure 1D, which includes $C_i$ and outcome-to-outcome interference. Two additional pathways link $C_i$ to $D_i$:
While covariate adjustment closes path 5, path 6 remains open, so $\sigma_{C_iD_i\cdot T_{i1}T_{i2}}\neq0$ and $b_C\neq0$. Theoretically, this robustness test signals outcome-to-outcome interference in a basic sibling FE model. We now turn to simulation analyses to exercise this test in multiple settings.
Monte Carlo simulations demonstrated the robustness test’s performance in the model of Figure 1D and across its subsequent variations in Figure 2.adkins2012 Model alterations included treatment-to-treatment interference, direct associations between $C_i$ and siblings’ treatments or outcomes, treatment-to-outcome interference, or outcome-to-treatment interference.
\afterpage{
}
Our simulation model follows:
\[U^\prime_{i},\upsilon_{c},\upsilon_{i1},\upsilon_{i2}\sim N(0,1)\] \[\boldsymbol\alpha=\{\lambda,\omega\}\] \[C_{i}=
\] \[T_{i1}=
\] \[T_{i2}=
\] \[Y_{i1}=\delta T_{i1}+\kappa T_{i2}+\lambda C_{i}+\psi U^\prime_{i}+\upsilon_{i1}\] \[Y_{i2}=\delta T_{i2}+\theta T_{i1}+\eta Y_{i1}+\boldsymbol\alpha C_{i}+\psi U^\prime_{i}+\upsilon_{i2}\] \[D_i=Y_{i2}-Y_{i1}\]
For each sibling FE model, we ran two simulation sets, each consisting of two simulations with 1,000 runs of 5,000 observations (sibling pairs). Simulations varied on outcome-to-outcome interference ($\eta$) and on the association between $C_i$ and $U_i^\prime$ ($\pi$):
Baseline parameters were fixed across simulations: $\delta=3$, $\chi=1$, $\gamma=2$, and $\psi=5$. Unless specified, we set additional parameters to zero. Within samples, we ran the gain-score regression in equation (3). We conducted simulations in Stata Statistical Software: Release 16.statasoft See Appendix for simulation code.
Table 1 displays simulation inputs and results for the observed FE, and Appendix Table 1 contains full results. We first consider simulations 1.1 and 1.2. Under the model of Figure 1D, $\widehat{b_C}=0$ (95% confidence interval: -0.105, 0.105) without outcome-to-outcome interference, and the coverage probability—the percent of samples with confidence intervals that overlapped zero—exceeded 95%. Conversely, $\widehat{b_C}=0.264$ (95% confidence interval: 0.159, 0.370) with outcome-to-outcome interference, and the coverage probability was \textless1%. These results persisted in the presence of treatment-to-treatment interference (Figure 2A), direct associations between $C_i$ and siblings’ treatments (Figure 2B), equivalent and direct associations between $C_i$ and siblings’ outcomes (Figure 2C), and treatment-to-outcome interference (\textbf{Figure 2D}). However, $\widehat{b_C}\neq0$ with direct non-equivalent associations between $C_i$ and siblings’ outcomes (\textbf{Figure 2E}) or with outcome-to-treatment interference (\textbf{Figure 2F}), regardless if outcome-to-outcome interference was present.
\afterpage{
}
Results from simulations 2.1 and 2.2 indicate that the robustness test was only valid under the models in Figures 2B and 2C if $\pi=0$; if $C_i$ and $U_i^\prime$ were not directly associated, then $C_i$ must have been directly associated with a treatment or directly and equally associated with both outcomes for the robustness test to work. Otherwise, the test was invalid because $C_i$ was not associated with any other variable (Figures 1D, 2A, 2D, and 2F) or because $C_i$ was differentially associated with siblings’ outcomes (Figure 2E).
In a post-hoc analysis, we re-ran all simulation sets under the model in Figure 2A such that $C_i$ was directly associated with only one sibling’s treatment: $\nu=1.5$ ($C_i\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i2}$) and $\tau=0$ ($C_i\hdashrule[0.5ex]{0.6cm}{0.5pt}{2pt}T_{i1}$), or $\nu=0$ and $\tau=0.7$. Results were consistent with main findings, so $C_i$ is a valid observed FE in these settings (see Appendix).
In this paper, we demonstrated a simple robustness test for assessing bias from outcome-to-outcome interference in linear two-sibling FE models. The test involves regressing a gain-score on a pre-treatment observed FE and on both siblings’ treatments. For the test to work, the observed FE must be directly associated with an sibling-invariant confounder on treatments and outcomes, directly associated with at least one sibling’s treatment, or directly and equally associated with both siblings’ outcomes. Further, the robustness test collapses if the observed FE is directly and differentially associated with siblings’ outcomes or if an individual’s outcome affects their sibling’s treatment, although the latter violates the treatment estimation in a sibling FE model regardless.frisell2012, sjolander2016, frisell2021, mallinson2021 Failure to meet this criterion does not necessarily warrant a variable’s exclusion from a FE regression model. For example, the regression should adjust for an observed cluster-level variable that differentially affects siblings’ treatments and outcomes because it is a confounder, but it is not a valid observed FE for our robustness test.
Finding an observed FE for the robustness test is data dependent. Birth registries are commonplace in sibling FE analyseskhashan2014, khashan2015, hvolgaard2016, axelsson2019, mallinson2020, mallinson2021 and may provide several candidate variables. For instance, United States birth records record maternal demographic information that are fixed and related to family health history, such as nativity and ethnicity.qin2010, creanga2012, nchs Other sources may not have a similarly wide breadth of sibling-shared characteristics. Additionally, these shared characteristics may have complex links to siblings’ characteristics, and researchers must interrogate how candidate observed FEs relate the treatments and outcomes. For this reason, we considered only pre-treatment variables as candidate observed FEs for the robustness test. Recent methodological guidance discourages conditioning on post-treatment variables to minimize bias in treatment estimates.elwert2014, vwbook In our sibling FE model, conditioning on an observed FE that is causally affected by both siblings’ treatments would induce such bias in the treatment estimate, so it is also possible that it would invalidate the observed FE’s performance in the robustness test. Using an observed FE that is measured prior to siblings’ treatments alleviates this concern. More broadly, we recommend graphical modeling to determine a variable’s usefulness for the robustness testpearl2013, tennant2021 and, when possible, running multiple robustness tests with valid observed FEs to thoroughly screen possible outcome-to-outcome interference.
We also highlight the flexibility of this robustness test. Despite relying on gain-scores, it is applicable to studies with different FE strategies, such as dummy score regressionkim2021 or conditional maximum likelihood estimation.sjolander2016 The data may require reshaping such that each observation consists of a dyadic pair. Furthermore, the robustness test may be used in studies of other types of dyadic pairs and not only in sibling comparison analyses.
There are some limitations to this paper. We focused on settings with linear homogenous effects. Our test may be invalid in models with nonlinear relationships, including those with binary outcomes. Furthermore, we restricted our attention to dyadic sibling models, although this test may work in data with clusters of three or more siblings—for example, one could run the robustness test on randomly selected pairs from sibships. Finally, we omitted models with shared mediator or collider variables, which can induce bias in sibling FE models.sjolander2017 These factors may also invalidate our robustness test.
In sum, our gain-score robustness test is a useful tool for validating two-sibling FE models. Researchers who conduct sibling comparison studies should take advantage of this test to evaluate potential bias from outcome-to-outcome interference.