Discordant Relaxations of Misspecified Models

Discordant Relaxations of Misspecified Models

Introduction

A central challenge in the structural estimation of economic models is that the hypothesized structure often fails to identify a single generating process for the data, either due to multiple equilibria or data observability constraints. In such a context, the econometrics of partially identified models have been trying to obtain a tractable characterization of parameters compatible with the available data and maintained assumptions (hereafter identified set). A question of particular relevance in applied work is that it is often very difficult to find a tractable characterization of the identified set and then to obtain a valid confidence region for it. To avoid this difficulty, a large part of the literature has been trying to provide a confidence region for an outer set, i.e., a collection of values for the parameter of interest that contains the identified set but may also contain additional values.\footnote{See Molinari2020 for a detailed discussion.} Because of its tractability, constructing a confidence region for an outer set has been entertained in various topics of studies where the parameters of interest are only partially identified, see for instance Blundell2007, Ciliberto2009, Aucejo2017, Sheng2020, dePaula2018, Dickstein2018, Honore2020, Chesher2020, Gualdani2021, and BerryCompiani2022, among many others.

In most empirical studies, obtaining a tight outer set is very often interpreted as evidence for a small and informative identified set.\footnote{A tight outer set here refers to an outer set that is very small and informative.} This is because, under correct specification, any outer set contains the identified set. In this paper, we examine the implications of using outer sets for models that could be misspecified. We say a model is misspecified if the identified set of the model parameters is empty. We use refutation and misspecification interchangeably in this paper.

The first main contribution of this paper is to characterize a class of models for which outer sets based on non-sharp identification conditions may be discordant. For this class of models, as long as the model is misspecified, there always exist two sets of non-sharp identification conditions that fail to detect the violation of the model and at the same time yield outer sets that are disjoint with each other. Our result covers a large class of models studied in the partial identification literature, including models whose identified set is characterized by intersection bounds, conditional moment inequalities, or the Artstein1983 inequalities. The discordancy that we find is a negative property because the result provided by an outer set could entirely be driven by the set of non-sharp identification conditions chosen by the researcher, and that we could always consider an alternative choice that provides a result that conflicts with the initial one.

Discordant outer sets only exist when the model is misspecified. In theory, a researcher could run a model specification test before using an outer set. However, in practice, although it is possible to construct a non-sharp specification test that only checks the sufficient conditions for model misspecification, constructing a sharp specification test, which checks the necessary and sufficient conditions for the emptiness of the identified set, is as challenging as obtaining a sharp characterization of the identified set. The non-tractability of the latter is often the motivation to use outer sets in the first place.\footnote{See for instance, Sheng2020, Gualdani2021, and the empirical application in BerryCompiani2022.}

Therefore, our result shows that the usage of outer sets based on non-sharp identification conditions is an unreliable compromise. It suggests that looking for the sharp characterization of the identified set must not only be viewed as a theoretical exercise but also has important empirical relevance. The identified set not only exhausts all the identification restrictions in the model structure and assumptions but is also immune to the possible misleading conclusions of discordant submodels.

Our warning against the usage of outer sets based on non-sharp identification conditions should not discourage researchers from relaxing stringent primitive model assumptions and replacing them with weaker ones. Although the identified set of a weaker set of primitive model assumptions is necessarily an outer set of the stringent original model, it is different from an outer set based on non-sharp identification conditions of the original model. The identified set, derived from weaker primitive assumptions, has a clear and precise interpretation akin to the empirical content inherent in weaker primitive assumptions due to its sharpness. In contrast, the choice of non-sharp identification conditions is often driven by analytical or computational tractability. Researchers often lack primitive interpretations for the outer set based on these non-sharp identification conditions. These outer sets are only relevant to the empirical analysis when viewed as conservative bounds for the sharp results. Yet, this view of conservative bounds could be misleading because, as we show in this paper, the conclusion drawn from these outer sets can be driven entirely by the choice of non-sharp identification conditions instead of the empirical content of the model.

However, discordant outer sets do not exist in all refuted models, especially when they are based on weaker primitive assumptions instead of non-sharp identification conditions. We then derive sufficient conditions for the non-existence of discordant submodels. This second result characterizes a class of models for which constructing outer sets cannot lead to misleading interpretations. In this case, outer sets would be conservative but always robust.

Prior to our work, various papers have been concerned about misspecification in partially identified models. An important focus has been dedicated to analyzing the impact of model misspecification on standard confidence regions used for set-identified models. Bugni_etal2012 analyze the behavior of usual inferential methods for moment inequality models under local model misspecification. Ponomareva2011 and Kaido2013 consider the impact of misspecification on semiparametric partially identified models, respectively, in the linear regression model with an interval-valued outcome and in a framework where some nonparametric moment inequalities are correctly specified and misspecification is due to a parametric functional form. See also Allen2020 who propose a method for statistical inference on the minimum approximation error needed to explain aggregate data in quasilinear utility models. It is worth noting that if one tries to find a confidence region for an outer set, none of the inference methods, including those developed in the previously cited papers, can resolve the specific issue we are raising here. This is because two non-empty outer sets derived from the same underlying model structure can lead to discordant results. Adopting one of these outer sets without checking the validity of the underlying model could lead to misleading conclusions. Therefore, we need to suggest a more primitive approach to deal with these discordant results in this paper.

This objective leads to our second main contribution, which consists of providing a method to salvage models that are possibly misspecified because of the existence of discordant misspecified submodels or discordant nonempty outer sets. The main intuition is to construct some minimum relaxation of the full model by removing discordant submodels until all remaining submodels are compatible. Because there could be multiple ways to relax a model to restore data consistency, we take the union of the identified set of all these relaxed models. By doing so, we construct what we call the misspecification robust bound. We provide general sufficient conditions under which our misspecification robust bound exists and also provide an intuitive empirical interpretation for it. Intuitively, we will say that a hypothesis is robust to misspecification if the hypothesis is compatible with all relaxed models that are data consistent and is implied by at least one of those data-consistent relaxed models.

The misspecification robust bound concept is related to the minimally relaxed identified set introduced in Andrews2019, and to the falsification adaptive set concept introduced in Masten2020. The primary departure from Masten2020 lies in our emphasis on discrete relaxations, while Masten2020 focused exclusively on relaxing assumptions in a continuous manner. In general, the use of discrete or continuous relaxation depends on the empirical application under scrutiny. We explore various features of discrete relaxations beyond its formal definition.

It is worth noting that discrete relaxations of misspecified models have been entertained in various existing papers, see for instance, Manski2000, Manski2009, Blundell2007, Kreideral2012, Chen2018, Kedagni2021, Mourifie2020, among many others. In these papers, when the initial model is too stringent, they suggested alternative weaker assumptions that are believed to be more compatible with the empirical application under scrutiny and for which the identified/outer set is not empty. However, some alternative reasonable relaxations may generate results that are discordant with what they suggested. To mitigate this issue, our misspecification robust bound approach suggests to collect information from all reasonable discordant minimum relaxations of the initial model.

We organize the rest of the paper as follows. Section (ref) introduces two simple leading examples that will illustrate our main contributions. Section (ref) presents our general setting and main results on the characterization of discordant submodels. Section (ref) discusses a class of models for which constructing outer sets do not lead to misleading interpretations. Section (ref) introduces the misspecification robust bound used to salvage misspecified models. Section (ref) provides a numerical illustration of the discordancy issue by visiting the widely used entry game model studied in Ciliberto2009, and also illustrates our misspecification robust bounds in a return to education example. The last section concludes, and additional results and proofs are relegated to the appendix.

Introductory examples

Although the main idea of this paper can be applied to general models, we begin with these two straightforward examples to illustrate our main contributions.

First leading example: Intersection bounds

Let us consider a special case of the intersection bounds in CLR2013 in which a parameter $\theta$ is bounded by the conditional mean of an upper and lower bounds,

where $\underline{Y}$ and $\overline{Y}$ are two observable random bounds and $Z$ is a vector of instrumental variables. Let $\mathcal{Z}$ be the support of $Z$, and define\footnote{The $\sup$ and $\inf$ operators in (ref) should be understood as essential supremum and essential infimum respectively. }

The identified set of $\theta$ is the interval $[\underline{\gamma}, \overline{\gamma}]$ when $\underline{\gamma} \leq \overline{\gamma}$. We assume the following regularity condition holds in this example.

This simple framework encompasses some important treatment effect models, including discrete and continuous treatment models, see for instance Manski1990, Manski1994 and Kim2018 among many others.\footnote{For the sake of conciseness those examples are discussed in more details in Appendix (ref).} In practice, model (ref) is sometimes implemented by solving its unconditional version,

where $h$ is some nonnegative function mapping its input to $\mathbb{R}_+^m$ with $m < \infty$, and the inequalities in (ref) are vector inequalities. The inference for (ref) is typically much simpler than the inference for the original model (ref), especially when $Z$ is multi-dimensional. Let $\widetilde{\Theta}(h)$ be the identified set for $\theta$ in model (ref). As made explicit in the notation, $\widetilde{\Theta}(h)$ depends on the choice of instrumental function $h$. However, since (ref) implies (ref), we know that for every choice of $h$, $\widetilde{\Theta}(h)$ is always an outer set of the interval $[\underline{\gamma}, \overline{\gamma}]$, the identified set for $\theta$ in model (ref), i.e., $[\underline{\gamma}, \overline{\gamma}] \subseteq \widetilde{\Theta}(h)$. This inclusion relation is often used as a justification for using model (ref). Its identification result $\widetilde{\Theta}(h)$ is often viewed as a conservative bound for $[\underline{\gamma}, \overline{\gamma}]$, the identified set for model (ref).

Our first observation is that the result based on $\widetilde{\Theta}(h)$ is not always reliable. Later in Section (ref), we show that when the identified set of (ref) is empty, i.e., $\underline{\gamma} > \overline{\gamma}$, there always exist two $h$ and $h'$ such that both $\widetilde{\Theta}(h)$ and $\widetilde{\Theta}(h')$ are nonempty but $\widetilde{\Theta}(h)\cap \widetilde{\Theta}(h')$ is empty. Thus, two researchers could apply the same model on the same data set and yet draw completely different conclusions from the outer sets by choosing different $h$ functions. For example, if one observes $\widetilde{\Theta}(h) \subseteq (0, +\infty)$ for some $h$, one should not jump directly to the conclusion that the sign of $\theta$ is positive without verifying the non-emptiness of the identified set, since in the case of emptiness, there are circumstances under which another researcher may choose an alternative $h'$ such that $\widetilde{\Theta}(h) \subseteq (-\infty,0)$.

This caveat of outer sets is somewhat overlooked in the literature. As we listed some papers in the introduction, it is common for researchers to construct a confidence interval for an outer set and draw conclusions based solely on its result. If the model is refutable and a researcher only studies an outer set in the empirical analysis without knowing whether the identified set is empty or not, results based on an outer set could be misleading in the intersection-bound model. In section (ref), we show that this caveat is indeed a concern for some widely used partial identification models, and in Section (ref), we illustrate this discordancy issue using the entry game model studied in Ciliberto2009.

On the other hand, there exist models for which constructing outer sets does not lead to misleading interpretations. In the following, we will introduce a model that belongs to this category as our second leading example. We study this class of models in more detail later in Section (ref).

Second leading example: Adaptive Monotone IV (AMIV)

Consider the following potential outcome model: $Y = \sum_{z\in \mathcal{Z}}\mathbbm{1}(Z = z)[Y_{1z}D + Y_{0z}(1-D)]$, where the treatment $D$ is binary and the support $\mathcal{Z}$ of instrument $Z$ is discrete and finite. $Y_{dz}$ is the potential outcome when the treatment and the instrument are externally set to $d$ and $z$, respectively. Assume $Z$ is one-dimensional and assume, without loss of generality that $\mathcal{Z} = \{1, ..., k\}$. We are interested in the average potential outcomes $\theta_d = \sum_{z}P(Z = z)E Y_{dz}$ for $d \in \{0,1\}$, and then average treatment effect (ATE), i.e., $\theta_1-\theta_0$. In this framework, the seminal work of Manski1990 derived sharp bounds on the ATE under three main assumptions: (ref) the bounded support of the potential outcomes, (ref) the exclusion restriction, i.e., $E Y_{dz}=E Y_{dz'}$ for $z\neq z'$, and (ref) the mean independence assumption, i.e., $E[Y_{dz}|Z] = E[Y_{dz}]$ for all $z \in \mathcal Z$. However, there are some empirical evidences ---for instance Ginther2000, where the identified for the ATE proposed by Manski is empty ---suggesting a violation of the set of these assumptions. To be able to say something meaningful on the ATE in such a context, we introduce the following assumptions, which is one way to relax Manski1990's (Manski1990) assumptions.

For any $z\in \{1, ..., k\}$, define assumption $a_z$ to be the collection of the following assumptions:

Each assumption $a_z$ has three parts. $\ref{enu:support}$ requires the potential outcomes to have a bounded support. (ref) is a mean independence assumption associated to the potential outcome $Y_{dz}$. The novelty here is (ref), which is an adaptive relaxation of the exclusion restriction. Indeed, in the extreme case when $z=1$, (ref) is equivalent to the full exclusion restriction, i.e., $Y_{dz} = Y_{dz'}$ for all $d$, $z$ and $z'$, then (ref) and (ref) are equivalent to $E[Y_{d}|Z] = E[Y_{d}]$, which is the restriction under which Manski1990 derived bounds on the ATE. On the other extreme, when $z=k$, (ref) and (ref) imply the MIV assumption introduced in Manski2000, i.e., $z_1 < z_2 \Rightarrow E[Y_d\vert Z=z_1] \leq E[Y_d\vert Z=z_2]$. However, when $1<z< k$, we are in a middle-ground situation where the exclusion restriction is relaxed in such a way that $Y_{dz'}$ is monotone in $z'$, but remains flat for $z'\ge z$. See Figure (ref) for an illustration of how $Y_{dz}$ depends on $z$ under (ref).

Because the cut-off point $z$ would be set based on the data through techniques elaborated in Section (ref), we refer to this assumption as the Adaptive Monotone IV (AMIV) assumption. The economic rationality of the AMIV is that, even if $Z$ is not a valid IV because it could positively affect the potential outcome, in some empirical contexts, it could be reasonable to consider that the marginal effect of the IV on the potential outcome becomes null after a certain cut-off point.

For example, let us consider the well-studied topic of measuring the returns to a college degree. In this empirical context, researchers have often used parental education as an instrumental variable for college education. Nonetheless, some argue that this instrument may not be valid. They point out that parents with higher education tend to have children with better unobserved skills that significantly impact potential earnings, despite the possibility that the marginal benefit of one extra year of a parent's education may be diminishing. The AMIV assumption introduced in this subsection combines these two features. In terms of our notation, $Y$ is observed wage, $D$ is an indicator for college attendance, $Z$ is parental education, and $Y_{dz}$ is the potential wage when college attendance $D$ and parental education $Z$ are externally set to $d$ and $z$, respectively. In the context here, the AMIV assumption acknowledges that parental education may positively influence children, while also accommodating the potential for its marginal impact to vanish beyond a certain threshold. We elaborate on this application in Section (ref).

Notice that by construction, for all $z=1,...,k-1$, $a_{z}$ implies $a_{z+1}$. Therefore, we have $\Theta\left(\{a_{z}\}\right) \subseteq \Theta\left(\{a_{z+1}\}\right)$ for all $z \in \{1,...,k-1\}$. In section (ref), we will show that this nested structure of the assumptions may lead to a situation where we cannot have discordant submodels. More generally, we will derive sufficient conditions for the non-existence of discordant submodels.

When comparing these two leading examples, it is important to recognize that the motivation that leads to the construction of the outer sets or the relaxation of the initial model is different. In the first example, the outer sets are generated from non-sharp restrictions and are only used as a device aiming to provide useful information about the identified set. This situation usually occurs when one does not know how to conduct inference directly for the identified set or when the inference for the identified set is computationally intractable to implement. In Section (ref) below, we will explore why such an approach could lead to unreliable results. In the second example, the outer sets are introduced in a more constructive manner aiming to relax a model refuted by the data. They are based on more primitive assumptions on the latent variables. In such a case, it is often possible to provide a series of outer sets that are not discordant with others. We will discuss this case in Section (ref).

Misleading Submodels

We view a model as a collection of constraints on the latent, observable variables, and the parameters. Throughout the paper, the parameter space $\Theta$ is assumed to be some subset in a metric space, which can be of finite or infinite dimensions. Let $A$ be some nonempty collection of these constraints. We consider $A$ as the full model (or simply model when there is no confusion) and, $A' \subsetneq A$ as a submodel. For any nonempty subset $A'\subseteq A$, we use $\Theta_I(A')$ to denote the set of parameter values that satisfy all the constraints in $A'$. For each $a\in A$, we abbreviate $\Theta_I(\{a\})$ as $\Theta_I(a)$. Let $\emptyset$ denote the empty set. For our purpose, $\Theta_I(\emptyset)$ can be an arbitrary nonempty subset of $\Theta$ such that $\Theta_I(a)\subseteq \Theta_I(\emptyset)$ for any $a \in A$. For the sake of simplicity, when there is no confusion, we will just use $\Theta$ to refer to $\Theta_I(\emptyset)$.

By definition, $\Theta_I(A') \subseteq \Theta_I(A'')$ if $A'' \subseteq A'$. As a result, for any $A' \subseteq A$, $\Theta_I(A')$ is an outer set of $\Theta_I(A)$. Moreover, we say a submodel $A'$ is data-consistent if $\Theta_I(A')$ is nonempty, and call it refuted or misspecified if the reverse is true. Since $\Theta_I(A) \subseteq \Theta_I(A')$ for any $A'\subseteq A$, if the model $A$ is data-consistent, we know that each $A'\subseteq A$ is also data consistent, and $\Theta_I(A')\cap \Theta_I(A'')$ is nonempty for any two submodels $A'$ and $A''$.

In this section, we focus on $A$ that consists of constraints that could be written in terms of the observable variables and model parameters only. This includes models where the identified sets are entirely characterized by a set of moment (in)equalities, including generalized method of moments (GMM) models, or Artstein1983 inequalities involving only observables and the parameter of interest. We refer to this type of constraints as identification conditions. These identification conditions are often derived from primitive assumptions involving the latent variables. In this restricted framework, we view $A$ as the sharp identification conditions, and we call $A'$ nonsharp identification conditions if $\Theta_I(A) \subsetneq \Theta_I(A')$. For instance, in the first leading example, the role of $A$ is played by the set of moment inequalities (ref) indexed by all instrumental functions $h$, with $\Theta_I(A)=[\underline{\gamma},\overline{\gamma}]$. A submodel $A' \subseteq A$ could refer to (ref) for a specific function $h$, with $\Theta_I(A')=\widetilde{\Theta}(h)$. If $[\underline{\gamma}, \overline{\gamma}] \subsetneq \widetilde{\Theta}(h)$, then $A'$ is a nonsharp identification condition.

When identification conditions are written in terms of the observable variables and model parameters only, they satisfy the following assumption.

For example, all moment (in)equality models satisfy this assumption. This assumption may not hold when some $a \in A$ involves primitive restrictions on latent variables, as is the case in our second leading example.

In empirical works with partially identified models, researchers often use nonsharp identification conditions instead of the sharp ones. This is often motivated by two reasons: (i) sometimes the researchers may not even know the sharp characterization of the identified set, and (ii) the sharp identification conditions might be computationally intractable given the existing inferential methods.\footnote{See, for example, Berry and Compiani (2022, Section 6, footnote 42).} When empirical results are based on an outer set obtained from nonsharp identification conditions, they are traditionally viewed as conservative yet valid because these outer sets are always supersets of the sharp identified set obtained using the sharp identification conditions. However, we are going to present a theorem that shows that outer sets obtained from nonsharp identification conditions are not always reliable and could potentially be misleading in the presence of model misspecification. We need the following assumption for this formal result.

To clarify this assumption, we start with the simple case where all the conditions can be verified for $\mathscr{C} = \{\{a\}: a\in A\}$. In such a scenario, Assumption (ref) breaks down into the following two parts: (i) every $a\in A$ is data-consistent, and (ii) either $A$ is finite or $\Theta_I(a)$ is compact for each $a\in A$. Part (i) ensures that $A$ will not be refuted because a specific $a\in A$ is refuted. When $A$ is finite, this is enough to ensure that identification condtions are not mutually compatible when $A$ is refuted. This reasoning can also apply to cases when $A$ has infinite elements, given the compactness condition in part (ii) is true. In Assumption (ref), we permit $\mathscr{C}$ to be formulated in ways beyond $\mathscr{C} = \{\{a\}: a\in A\}$. By doing so, $\Theta_I(a)$ need not necessarily be compact for each $a\in A$ in infinite cases. We only require the identified set for some finite combinations of $a\in A$ is compact. This flexibility is helpful, for example, when some identification conditions determine the lower bound for parameters, while others define the upper bound.

We would verify Assumption (ref) with $\mathscr{C} = \{\{a\}: a\in A\}$ for the first leading example in the next subsection. For an example of $\mathscr{C}$ being constructed in other ways, see Appendix (ref) where Assumption (ref) is verified for conditional moment inequalities models. We are now ready to state the formal result.

Theorem (ref) tells us that when the model is refuted, an outer set derived from one set of nonsharp identification conditions could be completely different from an outer set obtained from a different set of nonsharp identification conditions. In such a case, the information delivered by an outer set depends mainly on which nonsharp identification conditions the researcher decides to use. Therefore, for this class of models, applied researchers must be very careful in interpreting outer sets based on nonsharp identification conditions. We will illustrate this point further with the first leading example in the next subsection. Later, in Section (ref), we provide a numerical illustration of Theorem (ref) when applied to an entry game example.

Moreover, Theorem (ref) shows that, for any data-consistent nonsharp identification condition, there always exists another set of nonsharp identification conditions that are discordant with some of its strengthened versions. This suggests that the issue of discordancy is not confined to certain pairs of outer sets. When the model is refuted, we can start with any data-consistent outer set and further tighten its bounds by applying more restrictions. After incorporating just a finite number of additional restrictions, it will inevitably be in conflict with another data-consistent outer set, even if the cardinalities of $A$ and $\mathscr{C}$ are uncountably infinite.

Although Theorem (ref) seems to focus essentially on partially identified models, it also applies to point-identified models. Suppose that each outer set derived from each single identification condition is a singleton, i.e., $\Theta_I(a)$ is a singleton for all $a \in A$. Then, Theorem (ref) says that the model is misspecified if and only if there exist two different identification conditions $a, a' \in A$ such that $\Theta_I(a) \neq \Theta_I(a')$. This is related to a long-existing observation in point-identified models: whenever a model is over-identified, one can test the model specification by comparing the point-estimates obtained from different identification conditions. In point-identified models, the issue of over-identification or misspecification is a direct concern for the researchers. In partially identified models, however, applied researchers tend to believe that their results are more credible and thus less sensitive to misspecification. Therefore, they often interpret the tightness of an outer set as a signal of an informative identified set, as discussed in Molinari2020 regarding the “usefulness" of outer sets. Theorem (ref) provides a different perspective that, for a certain class of models, outer sets could be misleading. Therefore, applied researchers should be very careful in interpreting outer sets based on nonsharp identification conditions, even if the model is only partially identified. In the following, we analyze the implications of Theorem (ref) on our first leading example.

Intersection bounds example continued

To begin, let us construct $A$ in this example. Define $\mathcal{H}^+_m$ to be the space of all nonnegative instrumental functions with dimension $m$. More formally, let $\mathcal{H}^+_m \equiv \{h: \mathcal{Z}\mapsto \mathbb R^m_+ \text{ such that } E\left\Vert h(Z)\right\Vert < \infty$, $E\left\Vert \underline{Y}h(Z)\right\Vert < \infty$, $E\left\Vert \overline{Y}h(Z)\right\Vert < \infty \text{ and } E[h_i(Z)] > 0, \; \forall \; i=1,...,m \}$. Let $A$ be the set of all identification conditions (ref) indexed by $h\in \mathcal{H}^+_1$. The set $A$ constructed here represents sharp identification conditions, because (ref) holds if and only if (ref) holds for all $h\in \mathcal{H}^+_1$.

Next, we verify Assumptions (ref) and (ref) hold. Note that, for each $a\in A$, $\Theta_I(a)$ is nonempty and compact. This is because each $a\in A$ corresponds to an $h\in \mathcal{H}^+_1$, and because for all $h\in \mathcal{H}^+_1$, the identified set $\widetilde{\Theta}(h)$ for model (ref) is equal to the following interval

which is nonempty and compact under Assumption (ref). We can let $\mathscr{C} = \{\{a\}: a\in A\}$. Then, $\Theta_I(A')$ is nonempty and compact for all $A'\in \mathscr{C}$. Moreover, $\Theta_I(A) = \Theta_I(\cup_{A'\in \mathscr{C}}A')$ by the construction of $\mathscr{C}$. Thus, Assumption (ref) is satisfied. Since (ref) are moment inequalities which only depend on observables and the parameter, Assumption (ref) is also satisfied.

Note that, in this example, if $B\subseteq A$ and $B$ consists of $m$ assumptions, then $B$ refers to the submodel that (ref) holds for some $h\in \mathcal{H}^+_m$. As a result, Theorem (ref) implies that when (ref) is refuted, there must exist some $h_1 \in \mathcal{H}^+_{m_1}$ and $h_2\in \mathcal{H}^+_{m_2}$, such that $\widetilde{\Theta}(h_1)\neq \emptyset$ and $\widetilde{\Theta}(h_2)\neq \emptyset$ but $\widetilde{\Theta}(h_1)\cap \widetilde{\Theta}(h_2)$ is empty. In fact, because of the specific structures of this example, we can even obtain the following stronger result.

When (ref) is refuted, Proposition (ref) shows that the unconditional moment restrictions can point identify any element in the crossed bound $(\overline{\gamma}, \underline{\gamma})$ with a properly chosen instrumental function. The width of $(\overline{\gamma}, \underline{\gamma})$ depends on the extent of the model violation: the worse the violation is, the wider this interval would be. In the extreme case where the mean independence condition is significantly violated such that $[E\underline{Y}, E\overline{Y}]\subseteq (\overline{\gamma}, \underline{\gamma})$, it implies that any point in the Manski worst-case bounds can be selected as the point identification result by a suitable choice of $h$.

Proposition (ref) also sheds some light on the implementation of the inference procedure in AS2013, which is one of the most popular inference procedures used for conditional moment inequalities. AS2013 converts the conditional moment inequalities into unconditional moment inequalities in the same way as we transformed, in the first leading example, (ref) into (ref). A notable distinction in AS2013's (AS2013) approach is that the number of instruments, i.e., the dimension of $h$ in our notation, increases to infinity as the sample size grows. Our results show that the usage of infinite number of instruments in the limit is crucial to achieve reliable inference results. In contrast, if researchers implement their inference but choose an instrumental function with a finite and fixed dimension, their results could be spuriously informative and misleading, as underscored in Proposition (ref). See also the formal results in Appendix (ref).

It is worth noting that Theorem (ref) applies to much more general frameworks. More precisely, in Appendices (ref) and (ref), we provide sufficient conditions under which Theorem (ref) applies to two widely used classes of partially identified models. The first is a class of models where the identified set is characterized by the following type of conditional moment inequalities:

Here, $X \in \mathbb{R}^{k_1}$ and $Z\in \mathbb{R}^{k_2}$ are observable random variables, and $m(\cdot, \cdot\ ;\theta) \in \mathbb{R}$ is some known integrable function for each $\theta$. If researchers construct outer sets by transforming ((ref)) into finite-dimensional unconditional moment inequalities, then a similar discordancy issue may happen, as the one we have seen in the first leading example. See Appendix (ref) for more details.

The second class of models that Theorem (ref) could be applied to is the class for which the identified set can be characterized by Artstein1983's (Artstein1983) inequalities. In her recent survey, Molinari2020 shows that this class of models includes simultaneous-move finite games with multiple equilibria, auction models with independent private values, network formation models, treatment effect models, etc. In many of those cases, the number of Artstein1983's (Artstein1983) inequalities that characterize the (sharp) identified set is extremely high (very often much higher than the sample size of the data under use). In practice, for the sake of computational feasibility, empirical researchers often pre-select a finite collection of Artstein's inequalities to obtain an outer set. As examples, we could cite Ciliberto2009, Haile2003, Sheng2020, Chesher2020, and BerryCompiani2022, among many others. In Appendix (ref), we show that those pre-selected nonsharp moment inequalities suffer the same issue pointed out in Theorem (ref).

In Section (ref), we provide a numerical illustration of the discordancy issue by visiting the widely used entry game model studied in Ciliberto2009. We will explore in more detail the consequences of this pre-selection procedure when the original model might be refuted and illustrate the implication of our Theorem (ref) in this widely used framework.

Compatible Submodels and Minimum Data-Consistent Relaxation

As discussed in the previous section, there could be discordant submodels when the full model is refuted. However, the falsification of the full model does not necessarily lead to discordance of the submodels. Unlike in the previous section, we now consider $A$ that consists of primitive assumptions on latent variables in addition to those on observable variables and the parameter of interest. In this wider class of models, we will present a sufficient condition that ensures that all data-consistent submodels are always compatible with each other. In this section, for the sake of simplicity, we focus on the case where $A$ is finite. For the case where $A$ is infinite, similar results could be derived under additional conditions. We elaborate on those conditions and results in Appendix (ref).

To state our result, we need to introduce a new concept. When the full model is refuted, we can obtain a data-consistent submodel by dropping or relaxing some of the assumptions. We say that a data-consistent submodel is a minimum relaxation if we relax the minimum number of assumptions needed to restore data consistency.

It is worth noting that the concept of a minimum data-consistent relaxation depends on how the researcher defines each of the simple assumptions $a$ that constitute $A$. Therefore, all the subsequent results relying on the minimum data-consistent relaxation depend on the way the researcher constructs $A$. We will return to this point in Section (ref). Furthermore, a minimum data-consistent relaxation always exists when $A$ is finite; its existence requires additional conditions when $A$ is infinite. See Appendix (ref). To illustrate this concept, let us consider a simple example where $A = \{a_1, a_2, a_3\}$. The identified sets of each $a_i$ are all closed intervals in $\mathbb{R}$ as shown in Figure (ref) with $\Theta_I(a_1) = [b,c]$, $\Theta_I(a_2) = [d, e]$, $\Theta_I(a_3) = [f, g]$ and $f\le b \le c < d \le e \le g$. Assume also for the purpose of illustration that $\Theta_I(\{a, a'\}) = \Theta_I(a)\cap\Theta_I(a')$ for $a,a'\in \{a_1, a_2, a_3\}$.

In this example, both $\{a_1, a_3\}$ and $\{a_2, a_3\}$ are minimum data-consistent relaxations. And, $\{a_3\}$ is not a minimum data-consistent relaxation, since it will remain data-consistent after including $a_1$ or $a_2$. In general, minimum data-consistent relaxations may or may not be unique. We will defer the discussion of multiple minimum data-consistent relaxations to the next section. In this section, we focus on the situation where there exists a unique minimum data-consistent relaxation. In fact, the uniqueness of the minimum data-consistent relaxation ensures the absence of discordancy issues discussed in the previous section. More precisely, in Appendix (ref), we show that the existence of two discordant submodels is equivalent to the existence of two minimum data-consistent relaxations that are also discordant each other under some regularity conditions. Therefore, the uniqueness of the minimum data-consistent relaxation ensures the absence of discordant sub-models.

The following result describes the conditions under which there exists a unique minimum data-consistent relaxation.

Theorem (ref) through its condition (ref) provides a way to check whether there exists a unique minimum data-consistent relaxation $A^*$. Note that Condition (ref) does not hold in every model: in a general model, each $a\in A'$ being data-consistent is necessary but not sufficient for $A'$ to be data-consistent, because individual data-consistent assumptions might not be mutually compatible when combined. Condition (ref) can be verified by investigating when each $A' \subseteq A$ is data-consistent, which could be done even without seeing the data. In terms of interpretation, condition (ref) implies that all data-consistent submodels are compatible with each other. It also implies that the set of data-consistent submodels, i.e., $\{A'\subseteq A: \Theta_I(A')\neq \emptyset\}$, is closed under the union operation: the union of data-consistent submodels remains data-consistent.

When the full model $A$ is data-consistent, the unique minimum data-consistent relaxation $A^*$ is just equal to $A$. When $A$ is refuted, $A^*$ can be viewed as the model learned from the data by removing all refuted assumptions in $A$ while keeping all the data-consistent ones. Indeed, condition (ref) suggests that $A^*=\{a \in A:\Theta_I(a)\neq \emptyset\}$. The interpretation of $A^*$ and its role as the unique minimum data-consistent relaxation will be studied further in Appendix (ref). One way to illustrate condition (ref) is to consider our second leading example.

AMIV example continued

Recall that by construction, for all $z=1,...,k-1$, $a_{z}$ implies $a_{z+1}$. In addition, define $a^\dagger$ as the collection of (ref) and (ref). Let $A = \{a_1,...,a_k, a^\dagger\}$ be the collection of all assumptions. Then, the full model $A$ is the classic mean independence assumption considered in Manski1990. Within this second leading example, all assumptions are nested. That is, for any two $a,a'\in A$, either $a$ implies $a'$ or $a'$ implies $a$. Therefore, the data-consistency of a set of assumptions is equal to the data-consistency of the strongest assumption in that set, which implies the validity of (ref).\footnote{However, it is worth noting this nested structure is not necessary for condition (ref) to hold.} Therefore, (ref) holds in this example. Theorem (ref) then implies that all data-consistent submodels will be compatible with each other and there exists a unique minimum data-consistent relaxation $A^*$. The following result characterizes the identified set of $A^*$. To state the result, we use the following notations: $\underline{Y}_d = Y\mathbbm{1}(D = d) + \underline{y}_d \mathbbm{1}(D \neq d) $, $\overline{Y}_d = Y\mathbbm{1}(D = d) + \overline{y}_d \mathbbm{1}(D \neq d)$, $\underline{q}_{dz}=E[\underline{Y}_d|Z = z]$, and $\overline{q}_{dz} = E[\overline{Y}_{d}|Z = z]$.

Misspecification Robust Bounds

In this section, we consider cases where there are multiple data-consistent relaxations. According to Theorem (ref), the multiplicity of minimum data-consistent relaxations is a necessary condition for the existence of discordant submodels. Indeed, whenever there are two mutually incompatible data-consistent submodels, there are at least two minimum data-consistent relaxations. If there is no reason to favor one submodel over another ex-ante, it is reasonable to consider all of these relaxations.

The misspecification robust bound concept is similar to the falsification adaptive set concept introduced in Masten2020. However, a distinctive feature of this section is that we focus on discrete relaxations, where an assumption is either dropped or kept, while Masten2020 focuses exclusively on relaxing assumptions in a continuous way. In general, the type of relaxation depends on the empirical question under study. In the following, we derive the misspecification robust bound for our two leading examples.

Intersection bounds example continued

For the model (ref), the misspecification robust bound is given in the following result.

A direct implication of Proposition (ref) is that if $P(E[\underline{Y}|Z]\le \overline{\gamma}) > 0$ and $P(E[\overline{Y}|Z]\ge \underline{\gamma}) > 0$ hold, which are mild technical requirements, the misspecification robust bound simplifies to $\Theta^*_I=[\min(\underline{\gamma},\overline{\gamma}), \max (\underline{\gamma},\overline{\gamma})]$ whether or not the full model is refuted.

AMIV example continued

A direct implication of Proposition (ref) is that the AMIV model has a unique minimum data-consistent relaxation $A^*$ which can be summarized as follows: $A^*=\{a^\dagger\}\cup\{a_z: \text{ Equations } (\ref{Test1}) \text{ } \& \text{ } (\ref{Test2}) \text{ hold}\}$. Therefore, the misspecification robust bound for the AMIV model is:

where $z^* = \min\{z: a_z \in A^*\}$.

Empirical interpretation of the misspecification robust bound

As we pointed out earlier, $\Theta^*_I$ will depend on how the researcher decides to define the assumptions $a$ that constitute $A$. Different constructions of $A$ correspond to different ways to relax a refuted model. It is inevitable that there almost always exist multiple ways to relax a stringent and refuted model, and different ways of relaxations would lead to different results. Instead of drawing a general conclusion about which relaxation approach is superior, we believe it is important to offer an empirical interpretation of $\Theta^*_I$ for a given $A$. In this way, even if different researchers may construct $A$ based on their own economic interpretations of the model, they would have a clear interpretation of their results.

In Theorem (ref) in Appendix (ref), we show that the misspecification robust bound $\Theta^*_I$ is both rationalizable and nonconflicting in the following sense:

When the full model is refuted, different data-consistent submodels can imply different and potentially discordant statements on $\theta$. Among all possible statements on $\theta$, we think that being rationalizable and nonconflicting is a minimum requirement for a statement to be robust to model misspecification. If a statement fails to be rationalizable, then it is not implied by any of the data-consistent submodels. If a statement is not nonconflicting, then it is rejected by some data-consistent submodels.

The fact that $\Theta^*_I$ is both rationalizable and nonconflicting gives it an interesting empirical interpretation. Consider the simple case where $\theta$ is a scalar. Suppose we are interested in the sign of $\theta$. And, suppose $\Theta^*_I$ turns out to be within the positive real line, i.e., $\Theta^*_I\subseteq \mathbb{R}{++}$. Then, it means that some submodels identify the sign of $\theta$ to be positive, and whenever the sign of $\theta$ can be identified by a submodel, the sign of $\theta$ is always positive.

In some cases, $\Theta^*_I$ is the smallest set that is both rationalizable and nonconflicting. That is, for any $\widetilde{\Theta}\subseteq \Theta$, $\widetilde{\Theta}$ is both rationalizable and nonconflicting if and only if $\Theta^*_I \subseteq \widetilde{\Theta}$. In this case, $\Theta^*_I$ could have richer interpretations. Consider the previous simple example again. Suppose it turns out that $\Theta^*_I\cap \mathbb{R}_{++} \neq \emptyset$ and $\Theta^*_I\cap \mathbb{R}_{--} \neq \emptyset$ so that $\theta\in \Theta^*_I$ does not imply the sign of $\theta$. If we know $\Theta^*_I$ is the smallest rationalizable and nonconflicting set, then we have the following conclusion: neither $\theta$ is positive nor $\theta$ is negative are rationalizable and nonconflicting statements. In other words, the value of $\Theta^*_I$ in this case implies that the data and the model cannot provide a clear statement on the sign of $\theta$. Finally, in Theorem (ref) in Appendix (ref), we show that $\Theta^*_I$ would be the smallest rationalizable and nonconflicting set if there exists a unique minimum data-consistent relaxation or the identified set for each minimum data-consistent relaxation is a singleton.

Discrete Relaxation versus Continuous Relaxation

As can be seen, the misspecification robust bound relaxes a refuted model in a discrete way: an assumption is either fully kept or dropped during the relaxation. There are many other ways to relax and salvage a refuted model. One can also relax assumptions continuously as in Masten2020. In general, different relaxations will lead to different results, and it is hard to compare all the possible approaches. However, there does exist a special case where discrete relaxation always leads to more informative results than any other ways of relaxations.

In order to make an adequate comparison, we need to introduce the terminology used in Masten2020. For any $\epsilon \in [0, 1]$ and any $a\in A$, let $a_{\epsilon}$ denote the assumption after relaxing assumption $a$. The degree of relaxation is measured by $\epsilon$: when $\epsilon = 0$, $a_{\epsilon} = a$; when $\epsilon\in (0, 1)$, the assumption $a$ is partially relaxed but the exact form of $a_{\epsilon}$ would depend on the specific way of relaxation chosen by the researcher; when $\epsilon = 1$, the assumption $a$ is completely relaxed and $a_{\epsilon}$ is a null assumption which does not impose any restriction. Assume the relaxation is monotone: if $\epsilon_1 \le \epsilon_2$, $a_{\epsilon_1}$ is stronger than $a_{\epsilon_2}$, in the sense that $a_{\epsilon_1}$ implies $a_{\epsilon_2}$. For any $\delta: A\to [0, 1]$, define $A(\delta) \equiv \{a_{\delta(a)}: a\in A\}$ as the perturbed full model. For any two $\delta_1: A\to[0, 1]$ and $\delta_2: A\to[0, 1]$, we write $\delta_1 < \delta_2$ if $\delta_1(a) \le \delta_2(a)$ for all $a\in A$ and $\delta_1(a) < \delta_2(a)$ for some $a\in A$. Then, the falsification frontier (FF) in Masten2020 can be defined as $FF = \{\delta: A\to [0, 1]\ : \Theta_I(A(\delta))\neq \emptyset\text{ and there does not exist }\delta'\text{ such that } \Theta_I(A(\delta'))\neq \emptyset, \Theta_I(A(\delta')) \subsetneq \Theta_I(A(\delta)) \text{ and } \delta' < \delta\}$. We slightly modified the definition of the falsification frontier of Masten2020 to ensure the nonemptiness of $FF$ in some special cases.\footnote{ The original definition in Masten2020, written in our notation, is $FF = \{\delta: A\to [0, 1]\ : \Theta_I(A(\delta))\neq \emptyset\text{ and there does not exist }\delta'\text{ such that } \Theta_I(A(\delta'))\neq \emptyset \text{ and } \delta' < \delta\}$. With our modified definition, we do not need to worry about the possibility that there is a sequence of $\{\delta_i:i \geq 1\}$ such that $\delta_n \rightarrow \delta^{\ast}$, $\Theta_I(A( \delta^{\ast}))= \emptyset$, $\Theta_I(A( \delta_n))=\Theta_I(A( \delta_1)) \neq \emptyset$ and $\delta_{n+1} < \delta_{n}$ for all $n\geq 1$.} Then, the falsification adaptive set $\Theta^\dagger_I$ is defined as $\Theta^\dagger_I = \cup_{\delta\in FF}\Theta_I(A(\delta))$.

Note that $\Theta^\dagger_I$ depends on the specific way that one chooses to relax the assumptions. If one chooses to relax them discretely, i.e., if $a_{\epsilon} = a_{\mathbbm{1}(\epsilon > 0)}$ for any $\epsilon$ and $a$, then $\Theta^\dagger_I$ is equal to the minimum data-consistent relaxation $\Theta^*_I$. If one chooses a different way of relaxation, the $\Theta^\dagger_I$ is generally different. In some special cases, however, $\Theta^*_I$ is always included in $\Theta^\dagger_I$ no matter which way of relaxation is chosen. More precisely, whenever for any minimum data-consistent relaxation $\widetilde{A}$, $\Theta_I(\widetilde{A})$ is a singleton, it can be shown that $\Theta^*_I\subseteq \Theta^\dagger_I$ for any type of relaxation chosen by the researcher. We formally state and prove this result, respectively, in Theorem (ref) in Appendix (ref).

Numerical and Empirical Illustrations

In this section, through numerical exercises, we illustrate two of our main theoretical results. In Section (ref), we consider the entry game model studied in Ciliberto2009. We simulate a misspecified entry game model such that the sharp identification conditions deliver an empty identified set. In such a context, we show that it is possible to generate multiple non-empty conflicting outer sets by just selecting different sets of nonsharp identification conditions. This illustrates the discordancy issue raised in Theorem (ref). In Section (ref), we revisit a return to college application and report the estimated identified set of the minimum data-consistent relaxation derived under the assumption that parental education satisfies the AMIV assumption.

Numerical illustration of discordant outer sets in an entry game model

Consider an entry game model with $m$ players. Each player $i$ chooses $Y_i\in \{0, 1\}$ to maximize its payoff:

where $\delta^{i}_j$ is player $j$'s competition impact on player $i$, $\epsilon = (\epsilon_1, ..., \epsilon_m)\sim N(0, I_m)$, and $I_m$ is the identity matrix. Denote $\alpha = (\alpha_i: i=1,...,m)$, $\delta = (\delta^{i}_j: i\neq j)$, and, let $\theta = (\alpha, \beta, \delta)$ collect all the parameters. We model the players' behaviour as pure-strategy Nash equilibrium. Therefore, we restrict the parameter space $\Theta$ to be the set of parameters where pure-strategy equilibria exist with probability $1$. This is the same empirical model used in Ciliberto2009, except for two simplifications: (i) we assume that $\beta$ is the same for all players; (ii) we assume that the distribution of $\epsilon$ is known.

Let $\mathcal{K}$ be the collection of all subsets of $\mathcal{Y} \equiv \{0, 1\}^{m}$. Let $F$ denote the joint distribution of $(Y, X)$ in the data. Define $\Gamma(x, \epsilon;\theta)$ as the set of all pure-strategy Nash equilibria given $(x, \epsilon)$, the parameter $\theta$ and the payoff function in ((ref)). As shown in Galichon and Henry (2011) and equivalently in Beresteanu, Molchanov, and Molinari (2011), $\theta$ belongs to the identified set $\Theta_I(F)$ if and only if

Equation ((ref)) characterized the Artstein1983 inequalities associated to the entry game model under study. For each fix covariate $x$, we have $2^{2^m}$ inequalities to be checked. In this case, our full model $A$ is defined by the whole set of Artstein1983's inequalities, and then its cardinality is $2^{2^m} \times Card(X)$. This becomes easily non-tractable even for a relatively small number of firms, i.e., for instance, for a fixed $x$, and 5 firms we have $2^{2^5}=4, 294, 967, 296$ inequalities to be checked. Therefore, in practice, outer sets are almost always used. Let $A'$ be a subset of $\mathcal{K}$. Then, the outer set associated with $A'$ is

$A'$ is a nonsharp identification condition whenever $\Theta_I(F, A')\neq \Theta_I(F, \mathcal{K})\equiv \Theta_I(F)$. In CT, for each fixed $x$, they considered the outer set associated with $A_{ct}$ defined as $A_{ct} = \{\{y\}: y\in \mathcal{Y}\}\cup \{\{y\}^c: y\in \mathcal{Y}\}$ where $\{y\}^c$ stands for the complement set of $\{y\}$ in $\mathcal{Y}$. It is worth noting that whenever $m>2$, $A_{ct}$ is a nonsharp identification condition.

Data generating process (DGP)

In order to illustrate the issue of non-reliability of the outer sets in presence of misspecification, we generate a joint distribution $F$ from a game that might be different from the model (ref). Assume that, in the data generating process, each player $i$ chooses $Y_i\in \{0, 1\}$ to maximize the following payoff instead of ((ref)):

The extra vector of parameter $\gamma\equiv \{\gamma^{i}_{j_1, j_2}: \text{where }(i,j_1, j_2) \text{ are mutually different}\}$ captures the second-order competition effect. When $\gamma = 0$, the model in (ref) is correctly specified. If $\gamma\neq 0$, the model in (ref) is misspecified. To complete the model, we assume that, whenever there are multiple pure-strategy Nash equilibria, players will choose each equilibrium with the same probability. We assume that the support of $X$ is a bounded interval $\mathcal{X}$ in $\mathbb{R}^{\dim(X)}$. Without loss of generality, we assume $X$ is distributed uniformly in its support.

In the simulation, we focus on the simple case where $\gamma^1_{j_1,j_2} = \gamma^*$ for all $(j_1, j_2)$, and $\gamma^i_{j_1,j_2} = 0$ for all $(i\neq 1, j_1, j_2)$. Then, the joint distribution of $(Y, X)$ generated from this data generating process is indexed by $(\theta, \gamma^*)$, which we write as $F_{\theta, \gamma^*}$. Note that $\gamma^*$ measures the degree of misspecification: the larger the value of $\gamma^*$ is, the larger the degree of misspecification of model in (ref) is. In the simulation design we impose that $m=3$, $\dim X=1$, $X\sim U[-1,1]$, $\beta=0.1$, $\alpha=[1,1,1]$, and $\delta^{i}_j =1$ for all $(i,j)$. In the following, we construct outer sets both for the parameter $\delta$ and also for a counterfactual outcome in a counterfactual experiment. In both cases, we will show that we will be able to generate three outer sets (including the CT outer set) that are discordant with each other and none of them contains the true value.

Discordant nonsharp identification conditions for $\delta$

Our first objective is to illustrate the existence of discordant nonsharp identification conditions. Here, we focus on the (projected) identified set for $\delta^1_2$. Given the DGP with $(\theta, \gamma^*)$, the outer set for $\delta^1_2$ associated with the nonsharp identification conditions in $A'$ is characterized as follows: $\Lambda(F_{\theta, \gamma^*}, A') = \{t: \exists (\alpha, \beta, \delta) \in \Theta_I(F_{\theta, \gamma^*}, A') \text{ s.t. }\delta^{1}_2 = t\}$. Now, let us denote by $\overline{\gamma}^*_{ct}$ the maximum degree of misspecification that could not be detected by the submodel $A_{ct}$. More precisely, we define $\overline{\gamma}^*_{ct} \coloneqq \sup\{\gamma^*: \Lambda(F_{\theta, \gamma^*}, A_{ct}) \neq \emptyset\}$, therefore, $\Lambda(F_{\theta, \gamma^*}, A_{ct})$ is a nonempty outer set when $\gamma^* \leq \overline{\gamma}^*_{ct}$, and it becomes empty whenever $\gamma^* > \overline{\gamma}^*_{ct}$. In Figure (ref), we plot in blue the CT outer set, i.e., $\Lambda(F_{\theta, \gamma^*}, A_{ct})$, at various degree of model misspecification, for $\gamma^* \in [0,\overline{\gamma}^*_{ct}]$, where $0$ corresponds to no misspecification and $\overline{\gamma}^*_{ct}$ corresponds to the maximum degree of misspecification that is not detectable with $A_{ct}$.

A first remark is that the CT outer set shrinks when the degree of misspecification increases and at some point it no longer contains the true value. This illustrates that the tightness of the outer set should not systematically be interpreted as a signal of an informative identified set but it could just signal a presence of misspecification.

In Appendix (ref), we explain why the findings of Theorem (ref) apply to the entry game example. So, according to Theorem (ref), there must exist some other $A'$ nonsharp identification condition which is discordant with $A_{ct}$. Indeed, we are able to find two sets of nonsharp identification conditions, denoted as $A_1$ and $A_2$, which result in discordant identification results with $A_{ct}$. In Figure (ref), we plot $\Lambda(F_{\theta, \gamma^*}, A_1)$ in orange and $\Lambda(F_{\theta, \gamma^*}, A_2)$ in green. $\Lambda(F_{\theta, \gamma^*}, A_1)$ suggests values for $\delta^{1}_2$ that are higher than those suggested by $\Lambda(F_{\theta, \gamma^*}, A_{ct})$ while $\Lambda(F_{\theta, \gamma^*}, A_2)$ suggests values that are lower. When the degree of misspecification is higher than $0.4$, these three outer sets have no overlap.

Discordant counterfactual predictions

In many of the empirical games applications, applied researchers are very often interested in implementing counterfactual analyses. Interestingly, we observe not only discordant results for the parameters, but we also observe this phenomenon for counterfactuals. Below, we illustrate a scenario where different outer sets lead to discordant counterfactual outcomes when the full model is misspecified. Therefore, we illustrate the fact that the discordancy issue also applies to counterfactual outcomes. Let us consider a counterfactual where firm $3$ is no longer a potential entrant of the market. This type of counterfactuals would arise, for example, when firm $3$ is a foreign firm and is banned from the home market due to a trade policy, or when firm $3$ is merged with other firms.

Figure (ref) plots the different counterfactual predictions of submodels $A_{ct}$, $A_1$, and $A_2$ for the probability that only one firm enters the market, i.e., the probability of the presence of a monopoly in a market with characteristics $x_0 = (0, 0, 0)$. As we can see clearly, three submodels give counterfactual predictions that are discordant with each other.

An empirical illustration for compatible submodels

Context and Data

Estimating the causal impact of college education on later earnings has always been troublesome for economists because of the endogeneity of the level of education. To evaluate the returns to schooling, different approaches have been proposed, and most of them rely on the validity of instruments such as parental education, tuition fees, quarter of birth, distance to college, etc. The validity of all these IVs has been widely criticized because of their potential correlation with the children's unobserved skills.

In order to accommodate potentially invalid instruments, Manski2000, Manski2009 introduced the monotone IV (MIV) that does not require the IV to be valid but only imposes a positive dependence relationship between the IV and potential earnings. For instance, parental education may not be independent of potential wages, but plausibly does not negatively affect future earnings. In such a context, bounds on the average return to education can be derived.

In this application, we will consider the AMIV assumption introduced in Section (ref). We consider that parental education can have a positive effect on children's future earnings, but this marginal positive effect could plausibly become null after some cut-off. The particularity of our method is to let this cut-off be determined by the data using our misspecification robust bounds.

We consider the data used in Heckman2001. The data consist of a sample of 1,230 white males taken from the National Longitudinal Survey of Youth of 1979 (NSLY79)\footnote{The NLSY79 survey is sponsored and directed by the U.S. Bureau of Labor Statistics, and managed by the Center for Human Resource Research (CHRR) at The Ohio State University. Interviews are conducted by the National Opinion Research Center (NORC) at the University of Chicago. See bureau_of_labor_statistics_national_2019 for more details.}. The data contain information on the log weekly wage, college education, father's education, mother's education, among many other variables. Following HTV, we consider the college enrollment indicator as the treatment: it is equal to 1 if the individual has completed at least 13 years of education and 0 otherwise. In this empirical exercise, we use the maximum of parental education as the candidate instrumental variable. Some summary statistics are reported in Table (ref).

Methodology and results

We start by constructing the 95% confidence region for the identified sets of the average structural functions $E[Y_d]$, $d \in \{0,1\}$ and the average treatment effect $E[Y_1-Y_0]$ under the Manski1990 mean independence assumption, denoted as $\Theta_I(MI)$, and under the MIV assumption, denoted as $\Theta_I(MIV)$. In addition, we construct an estimate of our misspecification robust bounds under the AMIV assumption, denoted as $\Theta^*_I(AMIV)$, using the following steps:

The same procedure is applied to get an estimate for $\Theta_I(MIV)$ except that $z^*_d$ is set to 20 for each $d \in \{0,1\}$. Finally, since the identified set for the ATE under the mean independence assumption denoted as $\Theta_I(MI)$ takes the form of standard intersection bounds, we use the CKLRstata package to obtain its 95% confidence bounds.

The results are summarized in Table (ref). Column (1) shows that the 95% confidence region for $\Theta_I(MI)$ is empty. In other words, the data shows clear evidence against the use of parental education as a valid IV. On the other hand, column (4) shows the result for $\Theta_I(MIV)$.\footnote{We even test the validity of the MIV using the test proposed by Hsu2019, we do not reject the MIV assumption even at 10 % level.} As can be seen, we move from an empty identification region to a wide and non-informative identification region. In contrast, our misspecification robust bounds provide a nonempty yet relatively smaller set estimate for the ATE. Column (2) shows estimates of our misspecification robust bounds $\Theta_I^*(AMIV)$ when we allow the cut-offs to differ across potential outcomes as discussed in Remark (ref), while column (3) shows estimates where the cut-offs are restricted to be the same for both potential outcomes as in Proposition (ref). In the former case, we see that our proposed approach almost identifies the sign of the ATE.

Discussion

In this paper, we demonstrate the existence of discordant submodels in a wide range of models in the presence of model misspecification. This provides another reason why one should use the sharp characterization of the identified set whenever possible. The identified set not only exhausts all the identification restrictions in the model structure and assumptions but also is immune to the possible misleading conclusions of discordant submodels. Unlike an outer set, the identified set will be empty when the model is refuted by the data.

In empirical applications where a sharp characterization of the identified set is not tractable, our results suggest that empirical researchers should exercise caution when working with nonsharp identification conditions, especially when the bounds they obtain are very tight. For example, as a robustness check, one could construct the outer sets in different ways and check for any discordance between them.

Salvaging a refuted model is usually a challenging task, as it often involves some arbitrariness in how the model gets relaxed, and it could sometimes be computationally intractable. However, things get much easier when the minimum data-consistent relaxation is unique. In this case, it is apparent which assumptions are consistent with the data and which assumptions are not because all the data-consistent assumptions are compatible with each other (Theorems (ref) and (ref)). Moreover, the identified set of any data-consistent submodel can be viewed as a conservative bound for the misspecification robust bound in this case, making the computation a lot easier.

When the uniqueness of the minimum data-consistent relaxation is beyond reach, one can still choose to find the misspecification robust bound we proposed in this paper. It always leads to rationalizable and non-conflicting statements (Theorem (ref)), and it is sometimes the most informative rationalizable and non-conflicting statement (Theorem (ref)). We work out the misspecification robust bound in some simple examples, but its exact solution could be too complicated to solve when the underlying model involves many structures. In those challenging cases, it might be possible to construct an outer set that always covers the misspecification robust bound proposed in this paper. This type of outer sets will be immune to the issue raised in this paper. It remains unclear how to construct such outer sets, but this could be one reasonable step beyond the findings in this paper.