Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
76,348 characters · 16 sections · 46 citation commands
Rationalizing Rational Expectations: Characterizations and Tests
How individuals form their beliefs about uncertain future outcomes is critical to understanding decision making. Despite longstanding critiques \citep*[see, among many others,][]{pesaran1987limits,Manski2004}, rational expectations remain by far the most popular framework to describe belief formation Muth61. This theory states that agents have expectations that do not systematically differ from the realized outcomes, and efficiently process all private information to form these expectations. Rational expectations (RE) are a key building block in many macro- and micro-economic models, and in particular in most of the dynamic microeconomic models that have been estimated over the last two decades AM10,blundell17.
In this paper, we build a new test of RE. Our test only requires having access to the marginal distributions of subjective beliefs and realizations, and, as such, can be applied quite broadly. In particular, this test can be used in a data combination context, where individual realizations and subjective beliefs are observed in two different datasets that cannot be matched. Such situations are common in practice \citep*[see, e.g.,][]{Delavande2008, AHK2012, AHMR2014, Stinebrickner14b,gennaioli2016expectations,kuchler2015personal,BR18,BBRR20}. Besides, even in surveys for which an explicit aim is to measure subjective expectations, such as the Michigan Survey of Consumers or the Survey of Consumer Expectations of the New York Fed, expectations and realizations can typically only be matched for a subset of the respondents. And of course, regardless of attrition, whenever one seeks to measure long or medium-term outcomes, matching beliefs with realizations does require waiting for a long period of time before the data can be made available to researchers.\footnote{Situations where realizations can be perfectly predicted beforehand, such as in school choice settings where assignments are a known function of observed inputs, are notable exceptions.}
The tests of RE implemented so far in this context only use specific implications of the RE hypothesis. In contrast, we develop a test that exploits all possible implications of RE. Using the key insight that we can rationalize RE if and only if the distribution of realizations is a mean-preserving spread of the distribution of beliefs, we show that rationalizing RE is equivalent to satisfying one moment equality and (infinitely) many moment inequalities.\footnote{Interestingly, the equivalence on which we rely, which is based on Strassen's theorem strassen1965, is also used in the microeconomic risk theory literature, see in particular rothschild1970increasing.} As a consequence, if these moment conditions hold, RE cannot be rejected, given the data at our disposal. By exhausting all relevant implications of RE, our test is able to detect much more violations of rational expectations than existing tests.
To develop a statistical test of RE rationalization, we build on the recent literature on inference based on moment inequalities, and more specifically, on andrews2016inference. By applying their results to our context, we show that our test controls size asymptotically and is consistent over fixed alternatives. We also provide conditions under which the test is not conservative.
We then consider several extensions to our baseline test. First, we show that by using a set of covariates that are common to both datasets, we can increase our ability to detect violations of RE. Another important issue is that of unanticipated aggregate shocks. Even if individuals have rational expectations, the mean of observed outcomes may differ from the mean of individual beliefs simply because of aggregate shocks. We show that our test can be easily adapted to account for such shocks.
Finally, we prove that our test is robust to measurement errors in the following sense. If individuals have rational expectations but both beliefs and outcomes are measured with (classical) errors, then our test does not reject RE provided that the amount of measurement errors on beliefs does not exceed the amount of intervening transitory shocks plus the measurement errors on the realized outcomes. In that specific sense, imperfect data quality does not jeopardize the validity of our test. In particular, this allows for elicited beliefs to be noisier than realized outcomes. This provides a rationale for our test even in cases where realizations and beliefs are observed in the same dataset, since a direct test based on a regression of the outcome on the beliefs (see, e.g., Lovell86, Lovell86) is, at least at the population level, not robust to any amount of measurement errors on the subjective beliefs.
We apply our framework to test for rational expectations about future earnings. To do so, we combine elicited beliefs about future earnings with realized earnings, using data from the Labor Market module of the Survey of Consumer Expectations (SCE, New York Fed), and test whether household heads form rational expectations on their annual labor earnings. While a naive test of equality of means between earnings beliefs and realizations shows that earnings expectations are realistic in the sense of not being significantly biased, thus not rejecting the rational expectations hypothesis, our test does reject rational expectations at the 1% level. Taken together, our findings illustrate the practical importance of incorporating the additional restrictions of rational expectations that are embedded in our test. The results of our test also indicate that the RE hypothesis is more credible for certain subpopulations than others. For instance, we reject RE for individuals without a college degree, who exhibit substantial deviations from RE. On the other hand, we fail to reject the hypothesis that college-educated workers have rational expectations on their future earnings.
By developing a test of rational expectations in a setting where realizations and subjective beliefs are observed in two different datasets, we bring together the literature on data combination (see, e.g., CM02, CM02, MP06, MP06, \citeauthor*{fan2014identifying}, fan2014identifying, \citeauthor*{BLL16}, BLL16, and RM2007, RM2007 for a survey), and the literature on testing for rational expectations in a micro environment Lovell86,gourieroux1986direct,ivaldi1992survey.
On the empirical side, we contribute to a rapidly growing literature on the use of subjective expectations data in economics \citep*[see, e.g.,][]{Manski2004,Delavande2008,KW2008,vanderKlaauw2012,AHMR2014,PST2014, Stinebrickner14,WZ2015}. In this paper, we show how to incorporate all of the relevant information from subjective beliefs combined with realized data to test for rational expectations.
The remainder of the paper is organized as follows. In Section (ref), we present the general set-up and the main theoretical equivalences underlying our RE test. In Section (ref), we introduce the corresponding statistical tests and study their asymptotic properties. Section (ref) illustrates the finite sample properties of our tests through Monte Carlo simulations. Section (ref) applies our framework to expectations about future earnings. Finally, Section (ref) concludes. The appendix collects various theoretical extensions, additional simulation results, additional material on the application, and all the proofs. The companion R package \href{https://CRAN.R-project.org/package=RationalExp}{RationalExp}, described in the user guide \citep*[][]{RationalExp_R18}, performs the test of RE.
We assume that the researcher has access to a first dataset containing the individual outcome variable of interest, which we denote by $Y$. She also observes, through a second dataset drawn from the same population, the elicited individual expectation on $Y$, denoted by $\psi$. The two datasets, however, cannot be matched. We focus on situations where the researcher has access to elicited beliefs about mean outcomes, as opposed to probabilistic expectations about the full distribution of outcomes. The type of subjective expectations data we consider in the paper has been collected in various contexts, and used in a number of prior studies \citep*[see, among others,][]{Delavande2008,zafar11,AHK2012,AHMR2014,HoffmanBurks17}.
Formally, $\psi=\mathcal{E}[Y|\mathcal{I}]$, where $\mathcal{I}$ denotes the $\sigma$-algebra corresponding to the agent's information set and $\mathcal{E}\left[ \cdot | \mathcal{I} \right]$ is the subjective expectation operator (i.e. for any $U$, $\mathcal{E}\left[ U| \mathcal{I} \right]$ is a $\mathcal{I}$-measurable random variable). We are interested in testing the rational expectations (RE) hypothesis $\psi={\mathbb E}[Y|\mathcal{I}]$, where $ {\mathbb E}\left[\cdot | \mathcal{I}\right] $ is the conditional expectation operator generated by the true data generating process. Importantly, we remain agnostic throughout most of our analysis on the information set $\mathcal{I}$. Our setting is also compatible with heterogeneity in the information different agents use to form their expectations. To see this, let $(U_1,...,U_m)$ denote $m$ variables that agents may or may not observe when they form their expectations, and let $D_k=1$ if $U_k$ is observed, 0 otherwise. Then, if $\mathcal{I}$ is the information set generated by $(D_1 U_1,...,D_mU_m)$, agents will use different subsets of the $(U_k)_{k=1...m}$ (i.e., different pieces of information) depending on the values of the $(D_k)_{k=1...m}$. Our setup encompasses a wide variety of situations, where individuals have private information and form their beliefs based on their information set. This includes various contexts where individuals form their expectations about future outcomes, including education, labor market as well as health outcomes. By remaining agnostic on the information set, our analysis complements several studies which primarily focus on testing for different information sets, while maintaining the rational expectations assumption (see CunhaHeckman07, CunhaHeckman07, for a survey).
It is easy to see that the RE hypothesis imposes restrictions on the joint distribution of realizations $Y$ and beliefs $\psi$. In this data combination context, the relevant question of interest is then whether one can rationalize RE, in the sense that there exists a triplet $(Y',\psi',\mathcal{I}')$ such that $(i)$ the pair of random variables $(Y',\psi')$ are compatible with the marginal distributions of $Y$ and $\psi$; and $(ii)$ $\psi'$ correspond to the rational expectations of $Y'$, given the information set $\mathcal{I}'$, i.e., ${\mathbb E}(Y'|\mathcal{I}')=\psi'$. Hence, we consider the test of the following hypothesis:
where $\sim$ denotes equality in distribution. Rationalizing RE does not mean that the true realizations $Y$, beliefs $\psi$ and information set $\mathcal{I}$ are such that ${\mathbb E}\left[Y|\mathcal{I}\right]=\psi$. Instead, it means that there exists a triplet $\left(Y',\psi', \mathcal{I}'\right)$ consistent with the data and such that ${\mathbb E}\left[Y'|\mathcal{I}'\right]=\psi'$. In other words, rejecting $\text{H}_0$ implies that RE does not hold, in the sense that the true realizations, beliefs, and information set do not satisfy RE ($ {\mathbb E}\left[Y|\mathcal{I}\right]\neq \psi$). The converse, however, is not true.
Let $F_{\psi}$ and $F_{Y}$ denote the cumulative distribution functions (cdf) of $\psi$ and $Y$, $x^+ = \max(0, x)$, and define $$\Delta(y)= \int_{-\infty}^y F_{Y} (t) - F_{\psi} (t) dt.$$ Throughout most of our analysis, we impose the following regularity conditions on the distributions of realized outcomes ($Y$) and subjective beliefs ($\psi$):
The following preliminary result will be useful subsequently.
Lemma (ref) states that in order to test for H$_0$, we can focus on the constraints on the joint distribution of $Y$ and $\psi$, and ignore those related to the information set. This is intuitive given that we impose no restrictions on this set. Our main result is Theorem (ref) below. It states that rationalizing RE (i.e., $\text{H}_0$) is equivalent to a continuum of moment inequalities, and one moment equality.
The implication (i) $\Rightarrow$ (iii) and the equivalence between (ii) and (iii) are simple to establish. The key part of the result is to prove that (iii) implies (i). To show this, we first use Lemma (ref), which states that H$_0$ is equivalent to the existence of $(Y',\psi')$ such that $Y'\sim Y$, $\psi'\sim \psi$ and ${\mathbb E}\left[Y'\middle|\psi'\right]=\psi'$. Then the result essentially follows from Strassen's theorem strassen1965.
It is interesting to note that Theorem (ref) is related to the theory of risk in microeconomic theory. In particular, using the terminology of rothschild1970increasing, (ii) states that realizations ($Y$) are more risky than beliefs ($\psi$). The main value of Theorem (ref), from a statistical point of view, is to transform $\text{H}_0$ into the set of moment inequality (and equality) restrictions given by (iii). We show in Section (ref) how to build a statistical test of these conditions.
\paragraph{Comparison with alternative approaches}
We now compare our approach with alternative ones that have been proposed in the literature. In the following discussion, as in this whole section, we reason at the population level and thus ignore statistical uncertainty. Accordingly, the “tests” we consider here are formally deterministic, and we compare them in terms of data generating processes violating the null hypothesis associated with each of them.
Our approach can clearly detect many more violations of rational expectations than the “naive” approach based solely on the equality ${\mathbb E}(Y)={\mathbb E}(\psi)$. It also detects more violations than the approach based on the restrictions ${\mathbb E}(Y)={\mathbb E}(\psi)$ and $\mathbb{V}(Y)\geq\mathbb{V}(\psi)$ (approach based on the variance), which has been considered in particular in the macroeconomic literature on the accuracy and rationality of forecasts patton2012forecast. On the other hand, and as expected since it relies on the joint distribution of $(Y,\psi)$, the “direct” approach for testing RE, based on ${\mathbb E}(Y|\psi)=\psi$, can detect more violations of rational expectations than ours.
To better understand the differences between these four different approaches (“naive”, variance, “direct”, and ours), it is helpful to consider important particular cases. Of course, if $\psi={\mathbb E}\left[Y\middle|\mathcal{I}\right]$, individuals are rational and none of the four approaches leads to reject RE. Next, consider departures from rational expectations of the form $\psi={\mathbb E}\left[Y\middle|\mathcal{I}\right]+\eta$, with $\eta$ independent of ${\mathbb E}\left[Y\middle|\mathcal{I}\right]$. If ${\mathbb E}(\eta)\neq 0$, subjective beliefs are biased, and individuals are on average either over-pessimistic or over-optimistic. It follows that ${\mathbb E}(Y)\neq {\mathbb E}(\psi)$, implying that all four approaches lead to reject RE.
More interestingly, if ${\mathbb E}(\eta)=0$, individuals' expectations are right on average, and the naive approach does not lead to reject RE. However, it is easy to show that, as long as deviations from RE are heterogeneous in the population ($\mathbb{V}(\eta)>0$), the direct approach always leads to a rejection. In this setting, our approach constitutes a middle ground, in which rejection of RE depends on the degree of dispersion of the deviations from RE ($\eta$) relative to the uncertainty shocks ($\varepsilon=Y - {\mathbb E}(Y|\mathcal{I})$). In other words and intuitively, we reject RE whenever departures from RE dominate the uncertainty shocks affecting the outcome. Formally, and using similar arguments as in Proposition (ref) in Subsection (ref), one can show that if $\varepsilon$ is independent of ${\mathbb E}\left[Y\middle|\mathcal{I}\right]$, we reject $\text{H}_0$ as long as the distribution of the uncertainty shocks stochastically dominates at the second-order the distribution of the deviations from RE.
Specifically, if $\varepsilon \sim \mathcal{N}(0,\sigma^2_\varepsilon)$ and $\eta \sim \mathcal{N}(0,\sigma^2_\eta)$, we reject RE if and only if $\sigma^2_\eta > \sigma^2_\varepsilon$. In such a case, our approach boils down to the variance approach mentioned above: we reject whenever $\mathbb{V}(\psi)>\mathbb{V}(Y)$. But interestingly, if the discrepancy ($\eta$) between beliefs and RE is not normally distributed, we can reject H$_0$ even if $\mathbb{V}(\psi) \leq \mathbb{V}(Y)$. Suppose for instance that $\varepsilon\sim\mathcal{N}(0,1)$ and $$\eta = a \left(-\mathds{1}\{U \leq 0.1\} + \mathds{1}\{U \geq 0.9\}\right), \; U \sim \mathcal{U}[0,1] \text{ and } a>0.$$ In other words, 80% of individuals are rational, 10% are over-pessimistic and form expectations equal to ${\mathbb E}\left[Y\middle|\mathcal{I}\right]-a$, whereas 10% are over-optimistic and expect ${\mathbb E}\left[Y\middle|\mathcal{I}\right] + a$. Then one can show that our approach leads to reject RE when $a\geq 1.755$, while for $a= 1.755$, $\mathbb{V}(\eta)\simeq 0.616 < \mathbb{V}(\varepsilon)=1$.
\paragraph{Binary outcome}
Our equivalence result does not require the outcome $Y$ to be continuously distributed. In the particular case where $Y$ is binary, our test reduces to the naive test of ${\mathbb E}(Y)={\mathbb E}(\psi)$. Indeed, when $Y$ is a binary outcome and $\psi\in[0,1]$, one can easily show that as long as ${\mathbb E}(Y)={\mathbb E}(\psi)$, the inequalities ${\mathbb E}\left[ \left(y-Y\right)^+ - \left(y-\psi\right)^+\right] \geq 0$ automatically hold for all $y\in {\mathbb R}$. This applies to expectations about binary events, such as, e.g., being employed or not at a given date.
\paragraph{Interpretation of the boundary condition} To shed further light on our test and on the interpretation of $\text{H}_0$, it is instructive to derive the distributions of $Y|\psi$ that correspond to the boundary condition ($\Delta(y)=0$). The proposition below shows that, in the presence of rational expectations, agents whose beliefs $\psi$ lies at the boundary of $\text{H}_0$ have perfect foresight, i.e. $\psi ={\mathbb E}[Y|\mathcal{I}]= Y$.\footnote{For any cdf $F$, we let $F^{-1}$ denote its quantile function, namely $F^{-1}(\tau)=\inf\{x:F(x)\geq \tau\}$.}
In practice we may observe additional variables $X\in {\mathbb R}^{d_X}$ in both datasets. Assuming that $X$ is in the agent's information set, we modify H$_0$ as follows:\footnote{See complementary work by GHP16, who use subjective expectations data to relax the rational expectations assumption, and propose a method allowing to test whether specific covariates are included in the agents' information sets.}
Adding covariates increases the number of restrictions that are implied by the rational expectation hypothesis, thus improving our ability to detect violations of rational expectations. Proposition (ref) below formalizes this idea and shows that H$_{0X}$ can be expressed as a continuum of conditional moment inequalities, and one conditional moment equality.
Oftentimes, the outcome variable is affected not only by individual-specific shocks, but also by aggregate shocks. We denote by $C$ the random variable corresponding to the aggregate shocks. The issue, in this case, is that we observe a single realization of $C$ ($c$, say), along with the outcome variable conditional on that realization $C=c$. In other words, we only identify $F_{Y|C=c}$ rather than $F_Y$, as the latter would require to integrate over the distribution of all possible aggregate shocks. Moreover, the restriction ${\mathbb E}\left[Y\middle|C=c, \psi\right]= \psi$ is generally violated, even though the rational expectations hypothesis holds. It follows that one cannot directly apply our previous results by simply replacing $F_Y$ by $F_{Y|C=c}$. In such a case, one has to make additional assumptions on how the aggregate shocks affect the outcome.
To illustrate our approach, let us consider the example of individual income. Suppose that the logarithm of income of individual $i$ at period $t$, denoted by $Y_{it}$, satisfies a Restricted Income Profile model: $$Y_{it} = \alpha_i + \beta_t + \varepsilon_{it},$$ where $\beta_t$ capture aggregate (macroeconomic) shocks, $\varepsilon_{it}$ follows a zero-mean random walk, and $\alpha_i$, $(\beta_{t})_{t}$ and $(\varepsilon_{it})_t$ are assumed to be mutually independent. Let $\mathcal{I}_{it-1}$ denote individual $i$'s information set at time $t-1$, and suppose that $\mathcal{I}_{it-1}=\sigma\left(\alpha_i, (\beta_{t-k})_{k\geq 1}, (\varepsilon_{it-k})_{k\geq 1}\right)$. If individuals form rational expectations on their future outcomes, their beliefs in period $t-1$ about their future log-income in period $t$ are given by $$\psi_{it} = {\mathbb E}\left[Y_{it}\middle|\mathcal{I}_{it-1}\right] = \alpha_i + {\mathbb E}\left[\beta_t\middle|(\beta_{t-k})_{k\geq 1}\right] + \varepsilon_{it-1}.$$ Thus, $Y_{it} = \psi_{it} + C_t + \varepsilon_{it}-\varepsilon_{it-1}$, with $C_t =\beta_t - {\mathbb E}\left[\beta_t\middle|(\beta_{t-k})_{k\geq 1}\right]$. The corresponding conditional expectation is given by: $${\mathbb E}\left[Y_{it} \middle|\mathcal{I}_{it-1},C_t=c_t\right]=\psi_{it} + c_t \neq \psi_{it}.$$
To get closer to our initial set-up, we now drop indexes $i$ and $t$ and maintain the conditioning on the aggregate shocks $C=c$ implicit. Under these conventions, rationalizing RE does not correspond to ${\mathbb E}\left[Y\middle|\mathcal{I}\right]=\psi$, but instead to ${\mathbb E}\left[Y\middle|\mathcal{I}\right]=c_0 + \psi$ for some $c_0\in{\mathbb R}$. A similar reasoning applies to multiplicative instead of additive aggregate shocks. In such a case, the null takes the form ${\mathbb E}\left[Y\middle|\mathcal{I}\right]=c_0\psi$, for some $c_0>0$. In these two examples, $c_0$ is identifiable: by $c_0 = {\mathbb E}(Y) - {\mathbb E}(\psi)$ in the additive case, by $c_0={\mathbb E}(Y)/{\mathbb E}(\psi)$ in the multiplicative case. Moreover, there exists in both cases a known function $q(y,c)$ such that ${\mathbb E}(q(Y,c_0))={\mathbb E}(\psi)$, namely $q(y,c)=y-c$ and $q(y,c)=y/c$ for additive and multiplicative shocks, respectively.
More generally, we consider the following null hypothesis for testing RE in the presence of aggregate shocks:
where $q(.,.)$ is a known function supposed to satisfy the following restrictions.
By applying our main equivalence result (Theorem (ref)) to $q(Y,c_0)$ and $\psi$, we obtain the following result.
A few remarks on this proposition are in order. First, this result can be extended in a straightforward way to a setting with covariates. This is important not only to increase the ability of our test to detect violations of RE, but also because this allows for aggregate shocks that differ across observable groups. We discuss further this extension, and the corresponding statistical test, in Appendix (ref). Second, in the presence of aggregate shocks, the null hypothesis does not involve a moment equality restriction anymore; the corresponding moment is used instead to identify $c_0$. Related, a clear limitation of the naive test (${\mathbb E}(Y)={\mathbb E}(\psi)$) is that, unlike our test, it is not robust to aggregate shocks. In this case, rejecting the null could either stem from violations of the rational expectation hypothesis, or simply from the presence of aggregate shocks. Third, in Appendix (ref), we examine whether one can extend the results above to test for RE when aggregate shocks affect the outcomes in a more general way. Proposition (ref) establishes a negative result in this respect: as long as one allows for a sufficiently flexible dependence between the outcome and the aggregate shocks, any given distribution of subjective expectations is arbitrarily close to a distribution for which RE can be rationalized. This implies that, within this more general class of outcome models, there does not exist any almost-surely continuous RE test that has non-trivial power.
We have assumed so far that $Y$ and $\psi$ were perfectly observed; yet measurement errors in survey data are pervasive \citep*[see, e.g.][]{BBM2001}. We explore in the following the extent to which our test is robust to measurement errors. By robust, we mean that the test does not incorrectly reject RE, when they in fact hold. Specifically, assume that the true variables ($\psi$ and $Y)$ are unobserved. Instead, we only observe $\widehat{\psi}$ and $\widehat{Y}$, which are affected by classical measurement errors.\footnote{See zafar2011can who does not find evidence of non-classical measurement errors on subjective beliefs elicited from a sample of Northwestern undergraduate students. We conjecture that our test is robust to some forms of non-classical measurement errors. However, it seems difficult in this case to obtain a general result similar to the one in Proposition (ref).} Namely:
The following proposition shows that our test is robust to a certain degree of measurement errors on the beliefs.
The key condition is that $F_{\xi_\psi}$ dominates at the second order $F_{\xi_Y+\varepsilon}$, or, equivalently here, that $F_{\xi_Y+\varepsilon}$ is a mean-preserving spread of $F_{\xi_\psi}$. Recall that in the case of normal variables, $\xi_\psi \sim \mathcal{N}(0,\sigma^2_1)$ and $\xi_Y+\varepsilon \sim \mathcal{N}(0,\sigma^2_2)$, this is in turn equivalent to imposing $\sigma^2_1 \leq \sigma^2_2$. Thus, even if there is no measurement error on $Y$, so that $\xi_Y=0$, this condition may hold provided that the variance of measurement errors on $\psi$ is smaller than the variance of the uncertainty shocks on $Y$. More generally, this allows elicited beliefs to be - potentially much - noisier than realized outcomes, a setting which is likely to be relevant in practice. One should not infer, however, that measurement errors are innocuous in our set-up. Indeed, the converse of Proposition (ref) does not hold: we may reject $H_0$ with $Y$ and $\psi$, but not with $\widehat{Y}$ and $\widehat{\psi}$. As a simple example, suppose that $Y\sim\mathcal{N}(0,\sigma^2_Y)$, $\psi\sim\mathcal{N}(0,\sigma^2_\psi)$, $\xi_Y\sim \mathcal{N}(0,\sigma^2_3)$, $\xi_\psi=0$ and $\sigma^2_\psi \in (\sigma^2_Y, \sigma^2_Y+\sigma^2_3]$. Then, $\widehat{Y}$ and $\widehat{\psi}$ satisfy $H_0$, since $\sigma^2_\psi \leq \sigma^2_Y+\sigma^2_3$, whereas $Y$ and $\psi$ do not, since $\sigma^2_\psi >\sigma^2_Y$. Importantly though, Proposition (ref) does show that our test is conservative in the sense that measurement errors cannot result in incorrectly concluding that the RE hypothesis does not hold.
In situations where $(\widehat{Y},\widehat{\psi})$ are jointly observed, one could in principle alternatively implement the direct test. However, in contrast to our test, the direct test is not robust to any measurement errors on the subjective beliefs $\psi$. Indeed, if RE holds, so that ${\mathbb E}\left[Y|\psi\right]=\psi$, it is nevertheless the case that ${\mathbb E}\left[\widehat{Y}\middle|\widehat{\psi}\right]\neq \widehat{\psi}$, as long as $\text{Cov}(\xi_Y,\widehat{\psi})=\text{Cov}(\xi_\psi, Y)=0$ and $\mathbb{V}(\xi_\psi)>0$. In other words, even if individuals have rational expectations, the direct test will reject the null hypothesis in the presence of even an arbitrarily small degree of measurement errors on the elicited beliefs.
Also, it is unclear whether, in the presence of measurement errors on the elicited beliefs and beyond the restrictions on the marginal distributions, there are restrictions on the copula of $(\widehat{Y},\widehat{\psi})$ that are implied by RE. For instance, we show in Proposition (ref) in Appendix (ref) that under RE, and without imposing restrictions on the dependence between $\xi_Y+\varepsilon$ and $\xi_\psi$, the coefficient of the (theoretical) linear regression of $\widehat{Y}$ on $\widehat{\psi}$ remains unrestricted.\footnote{There might of course possibly be additional relevant information in the higher-order moments, although we have not been able to find any.} On the other hand, if one assumes that $\text{Cov}(\xi_Y+\varepsilon, \xi_\psi)\geq 0$ and $\mathbb{V}(\psi)/\mathbb{V}(\xi_\psi)\geq \underline{\lambda}$ for some $\underline{\lambda}\geq 0$, Proposition (ref) also shows that the coefficient of the linear regression of $\widehat{Y}$ on $\widehat{\psi}$ is bounded from below under RE. Such a restriction, which does require to take a stand on the signal-to-noise ratio $\mathbb{V}(\psi)/\mathbb{V}(\xi_\psi)$, can be easily added to the moment inequalities of our test if $(\widehat{Y},\widehat{\psi})$ is observed.
We now briefly discuss other relevant directions in which Theorem (ref) can be extended. First, another potential source of uncertainty on $\psi$ is rounding. Rounding practices by interviewees are common in the case of subjective beliefs. Under additional restrictions, it is possible in such a case to construct bounds on the true beliefs $\psi$ manski2010rounding. We show in Appendix (ref) that our test can be generalized to accommodate this rounding practice.
Second, we have implicitly maintained the assumption so far that subjective beliefs and realized outcomes are drawn from the same population. In Appendix (ref), we relax this assumption and show that our test can be easily extended to allow for sample selection under unconfoundedness, through an appropriate reweighting of the observations.
Third, our equivalence result and our test can be extended to accommodate situations with multiple outcomes $(Y_k)_{k=1,..,K}$ and multiple subjective beliefs $(\psi_k)_{k=1,..,K}$ associated with each of these outcomes. Specifically, whether one can rationalize rational expectations in this environment can be written as:
which, in turn, is equivalent to the distribution of the outcomes $Y_k$ being a mean-preserving spread of the distribution of the beliefs $\psi_k$. This situation arises in various contexts, including cases where respondents declare their subjective probabilities of making particular choices among $K+1$ possible alternatives. This also arises in situations where expectations about the distribution of a continuous outcome $Y$ are elicited through questions of the form “what do you think is the percent chance that [Y] will be greater than [y]?”, for different values $(y_k)_{k=1,..,K}$. In such cases, it is natural to build a RE test based on the multiple outcomes $(\mathds{1}\{Y> y_k\})_{k=1,..,K}$ and subjective beliefs $(\psi_k)_{k=1,..,K}$, where $\psi_k$ is the subjective survival function of $Y$ evaluated at $y_k$.
We now propose a testing procedure for H$_{0X}$, which can be easily adapted to the case where no covariate common to both datasets is available to the analyst. To simplify notation, we use a potential outcome framework to describe our data combination problem. Specifically, instead of observing $(Y,\psi)$, we suppose to observe only, in addition to the covariates $X$, $\widetilde{Y} = DY + (1-D)\psi $ and $D$, where $D=1$ (resp. $D=0$) if the unit belongs to the dataset of $Y$ (resp. $\psi$). As in Subsection (ref), we assume that the two samples are drawn from the same population, which amounts to supposing that $D\protect\mathpalette{\protect\independenT}{\perp} (X, Y,\psi)$ (see Assumption (ref)-(i) below). In order to build our test, we use the characterization (ii) of Proposition (ref): $$ {\mathbb E}\left[ \left(y-Y\right)^+ - \left(y-\psi\right)^+\middle| X \right] \geq 0 \quad \forall y\in {\mathbb R} \; \text{ and} \quad {\mathbb E}\left[ Y - \psi\middle| X\right] =0 .$$ Equivalently but written more compactly with $\widetilde{Y}$ only, $${\mathbb E}\left[ W \left(y-\widetilde{Y}\right)^+\middle|X\right]\geq 0 \quad \forall y\in {\mathbb R} \; \text{ and} \quad {\mathbb E}\left[W\widetilde{Y}\middle|X\right]= 0,$$ where $W=D/{\mathbb E}(D)-(1-D)/{\mathbb E}(1-D)$. This formulation of the null hypothesis allows us to apply the instrumental functions approach of andrews2016inference (andrews2016inference, AS), who consider the issue of testing many conditional moment inequalities and equalities. We then build on their results to establish that our test controls size asymptotically and is consistent over fixed alternatives.\footnote{Other testing procedures could be used to implement our test, such as that proposed by linton2010improved.} The initial step is to transform the conditional moments into the following unconditional moments conditions: $${\mathbb E}\left[W\left(y-\widetilde{Y}\right)^+ g(X)\right] \geq 0, \quad {\mathbb E}\left[\left( Y - \psi\right)g(X)\right] =0 ,$$ for all $y\in {\mathbb R}$ and $g$ belonging to a suitable class of non-negative functions.
We suppose to observe a sample $(D_i, X_i, \widetilde{Y}_i)_{i=1...n}$ of $n$ i.i.d. copies of $(D,X,\widetilde{Y})$. We consider instrumental functions $g$ that are indicators of belonging to specific hypercubes within $[0,1]^{d_X}$, hence we tranform the variables $X_i$ to lie in $[0,1]^{d_X}$. For notational convenience, we let $\widetilde{X}_i$ denote the nontransformed vector of covariates, and redefine $X_i$ as: $$X_i=\Phi_0\left( \widehat{\Sigma}_{ \widetilde{X},n}^{-1/2}\left( \widetilde{X}_{i} - \overline{ \widetilde{X}}_{i}\right)\right),$$ where, for any $x=(x_1,\dots,x_{d_X})$, we let $\Phi_0(x)=\left( \Phi(x_1) , \dots, \Phi\left(x_{d_X} \right)\right)^{\top}$. Here $ \Phi $ denotes the standard normal cdf, $ \widehat{\Sigma}_{\widetilde{X},n} $ is the sample covariance matrix of $ \left(\widetilde{X}_i\right)_{i=1...n}$ and $ \overline{\widetilde{X}}_n $ its sample mean.
Specifically, we consider instrumental functions $g$ belonging to the class of functions $\mathcal{G}_r= \left\{g_{a,r}, \; a\in A_r\right\}$, with $A_r= \left\{1,2,\dots, 2r\right\}^{d_X}$ ($r\geq 1$), $g_{a,r}(x) = {\rm {\large 1}\hspace{-2.3pt}{\large l}}\left\{x \in C_{a,r}\right\}$ and, for any $a=(a_1,...,a_{d_X})^{\top}\in A_r$, $$C_{a,r} = \prod_{u=1}^{d_X} \left(\dfrac{a_u -1 }{2r}, \dfrac{a_u }{2r}\right].$$
Finally, to define the test statistic $T$, we need to introduce additional notations. First, let $w_i=nD_i/\sum_{j=1}^n D_j - n(1-D_i)/\sum_{j=1}^n( 1-D_j)$ and define, for any $y\in{\mathbb R}$,
Let $\overline{m}_n (g,y)= \sum_{i=1}^{n} m\left(D_i,\widetilde{Y}_i, X_i, g,y\right)/n $ and define similarly $\overline{m}_{n,j}$ for $j=1,2$. For any function $g$ and any $y\in{\mathbb R}$, we also define, for some $\epsilon>0$, $$ \overline{\Sigma}_n(g,y) = \widehat{\Sigma}_n(g,y) + \epsilon \mathrm{Diag}\left( \widehat{\mathbb{V}}\left(\widetilde{Y} \right) , \widehat{\mathbb{V}}\left(\widetilde{Y} \right) \right),$$ where $\widehat{\Sigma}_n(g,y) $ is the sample covariance matrix of $ \sqrt{n}\overline{m}_n\left( g,y\right) $ and $\widehat{\mathbb{V}}\left(\widetilde{Y} \right)$ is the empirical variance of $\widetilde{Y}$. We then denote by $\overline{\Sigma}_{n,jj}(g,y) \ (j=1,2)$ the $j$-th diagonal term of $\overline{\Sigma}_{n}(g,y)$.
Then the (Cram\'{e}r-von-Mises) test statistic $T$ is defined by
where $ \widehat{\mathcal{Y}} =\left[\underset{i=1,\dots,n}{\min}\widetilde{Y}_i,\underset{i=1,\dots,n}{\max}\widetilde{Y}_i \right] $, $p\in (0,1)$ is a parameter weighting the moments inequalities versus equalities and $(r_n)_{n\in{\mathbb N}}$ is a deterministic sequence tending to infinity.
To test for rational expectations in the absence of covariates, we set the instrumental function equal to the constant function $g(X)=1$, and the test statistic is simply written as: $$ T= \underset{y \in \widehat{\mathcal{Y}}}{\sup}\bigg[ \left(1-p\right)\left( - \frac{\sqrt{n} \overline{m}_{n,1}(y) }{\overline{\Sigma}_{n,11}(y)^{1/2} } \right)^{+2} + p \left( \frac{\sqrt{n} \overline{m}_{n,2}(y) }{\overline{\Sigma}_{n,22}(y)^{1/2}} \right)^2\bigg],$$ where, using the notations introduced above, $\overline{m}_{n,j}(y)=\overline{m}_{n,j}(1,y)$ and $\overline{\Sigma}_{n,jj}(y)=\overline{\Sigma}_{n,jj}(1,y)$ $(j=1,2)$.
Whether or not covariates are included, the resulting test is of the form $\varphi_{n,\alpha} = {\rm {\large 1}\hspace{-2.3pt}{\large l}}\left\{T> c^*_{n,\alpha}\right\}$ where the estimated critical value $c^*_{n,\alpha}$ is obtained by bootstrap using as in AS the Generalized Moment Selection method. Specifically, we follow three steps:
Note that, despite the multiple steps involved, the testing procedure remains computationally easily tractable. In particular, for the baseline sample we use in our application (see Section (ref)), the RE test only takes 2 minutes.\footnote{This CPU time is obtained using our companion R package, on an Intel Xeon CPU E5-2643, 3.30GHz with 256Gb of RAM.}
We now turn to the asymptotic properties of the test. For that purpose, it is convenient to introduce additional notations. Let $\mathcal{Y}$ and $\mathcal{X}$ denote the support of $Y$ and $X$ respectively, and $$\mathcal{L}_F= \left\{ \left(y,g_{a,r}\right): y\in \mathcal{Y}, \ (a,r)\in A_r\times{\mathbb N}: \ {\mathbb E}_{F}\left[ W \left(y -\widetilde{Y}\right)^+ g_{a,r}(X)\right]=0 \right\},$$ where, to make the dependence on the underlying probability measure explicit, ${\mathbb E}_F$ denotes the expectation with respect to the distribution $F$ of $\left(D,\widetilde{Y},X\right)$. Finally, let $\mathcal{F}$ denote a subset of all possible cumulative distribution functions of $\left(D,\widetilde{Y},X\right)$ and $\mathcal{F}_0$ be the subset of $\mathcal{F}$ such that H$_{0X}$ holds. We impose the following conditions on $\mathcal{F}$ and $\mathcal{F}_0$.
The main result of this section is Theorem (ref). It shows that, under Assumption (ref), the test $\varphi_{n,\alpha}$ controls the asymptotic size and is consistent over fixed alternatives.
Theorem (ref) (i) is closely related to Theorem 5.1 and Lemma 2 in AS. It shows that the test $\varphi_{n,\alpha}$ controls the asymptotic size, in the sense that the supremum over ${\mathcal F}_0$ of its level is asymptotically lower or equal to $\alpha$. To prove this result, the key is to establish that, under Assumption (ref), the class of transformed unconditional moment restrictions that characterize the null hypothesis satisfies a manageability condition pollard1990empirical. Using arguments from hsu2011consistent, we then exhibit cases of equality in Theorem (ref) (ii), showing that, under mild additional regularity conditions, the test has asymptotically exact size (when letting $\eta$ tend to zero). Finally, Theorem (ref) (iii), which is based on Theorem 6.1 in AS, shows that the test is consistent over fixed alternatives.
\paragraph{Extension to account for aggregate shocks}
This testing procedure can be easily modified to accommodate unanticipated aggregate shocks. Specifically, using the notation defined in Section (ref), we consider the same test as above after replacing $\widetilde{Y} $ by $ \widetilde{Y}_{\widehat{c}} = D q(Y,\widehat{c}) + (1-D) \psi$, where $\widehat{c}$ denotes a consistent estimator of $c_0$. The resulting test is given by $\varphi_{n,\alpha, \widehat{c}} = {\rm {\large 1}\hspace{-2.3pt}{\large l}}\left\{T(\widehat{c})> c^*_{n,\alpha}\right\}$ (where $T(\widehat{c})$ is obtained by replacing $\widetilde{Y} $ by $\widetilde{Y}_{\widehat{c}}$ in the original test statistic). Such tests have the same properties as those above under some mild regularity conditions on $q(\cdot,\cdot)$, which hold in particular for the leading examples of additive and multiplicative shocks ($q(y,c)=y-c$ and $q(y,c)=y/c$). We refer the reader to Appendix (ref) for a detailed discussion of this extension.
In the following we study the finite sample performances of the test without covariates through Monte Carlo simulations. The finite sample performances of the version of our test that accounts for covariates are reported and discussed in Appendix (ref).
We suppose that the outcome $Y$ is given by $$ Y = \rho \psi + \varepsilon,$$ with $ \rho \in [0,1] $, $\psi \sim \mathcal{N}(0,1)$ and $$\varepsilon = \zeta \left(-{\rm {\large 1}\hspace{-2.3pt}{\large l}}\{U \leq 0.1\} + {\rm {\large 1}\hspace{-2.3pt}{\large l}}\{U \geq 0.9\}\right),$$ where $\zeta$, $U$ and $\psi$ are mutually independent, $\zeta \sim \mathcal{N}(2, 0.1)$ and $U\sim \mathcal{U}[0,1]$. In this setup, ${\mathbb E}(Y|\psi)=\rho\psi$ and expectations are rational if and only if $\rho=1$. But since we observe $Y$ and $\psi$ in two different datasets, there are values of $\rho\neq 1$ for which our test cannot reject the null hypothesis. More precisely, we can show that as the sample size $n$ grows to infinity, we reject the null if and only if $\rho \leq \rho^* \simeq 0.616$. Besides, given this data generating process, the naive test ${\mathbb E}(Y)={\mathbb E}(\psi)$ always fails to reject RE, while the RE test based on variances is only able to detect a subset of violations of RE that correspond to $ \rho < 0.445$.
To compute our test, we need to choose the tuning parameters $b_0$, $\kappa$, $\epsilon$ and $\eta$ (see Section (ref) for definitions). As mentioned in Section (ref), we set $\epsilon=0.05$ and $\eta=10^{-6}$, following andrews2016inference. andrews2013inference show that there exists in practice a large range of admissible values for the other tuning parameters parameters. Regarding $b_0$ and $\kappa$, we follow beare2015improved (beare2015improved, Section 4.2) and compute, for a grid of candidate parameters, the rejection rate under the null and under one alternative (namely, $\rho=0.5$), through Monte Carlo simulations. Then, we set $(b_0,\kappa)$ so as to maximize the power subject to the constraint that the rejection rate under the null is below the nominal size 0.05. That way, we obtain $b_0=0.3$ and $\kappa=0.001$. The parameter $p$ has a distinct effect, in that its choice does not affect size, at least asymptotically. Rather, this parameter selects to what extent the test aims power at the equality constraint $E(Y-\psi)=0$ versus the inequalities $E[(y-Y)^+ - (y-\psi)^+]\geq 0 \ (y\in{\mathbb R})$. Setting $p$ to $0.05$ leads to slightly higher power in our DGP, but values of $p$ in $[0,0.31]$ provide similar finite sample performances, with power always greater than 90% of the maximal power.
Results reported in Figure (ref) show the power curves of the test $ \varphi_{\alpha} $ for five different sample sizes ($n_{Y}=n_{\psi}=n \in \{400; 800; 1,200; 1,600; 3,200\}$) as a function of the parameter $\rho$, using 800 simulations for each value of $\rho$. We use 500 bootstrap simulations to compute the critical values of the test.
Several remarks are in order. First, as expected, under the alternative (i.e. for values of $\rho \leq \rho^* = 0.616$), rejection frequencies increase with the sample size $n$. In particular, for the largest sample size $n=3,200$, our test always results in rejection of the RE hypothesis for values of $\rho$ as large as .45. Second, in this setting, our test is conservative in the sense that rejection frequencies under the null are smaller than $\alpha=0.05$, for all sample sizes. This should not necessarily come as a surprise since the test proposed by AS has been shown to be conservative in alternative finite-sample settings (see, e.g. Table 1 p.22 in AS for the case of first-order stochastic dominance tests). However, for the version of our test that accounts for covariates and for the data generating process considered in Appendix (ref), rejection frequencies under the null are very close to the nominal level.
Using the tests developed in Section (ref), we now investigate whether household heads form rational expectations on their future earnings. We use for this purpose data from the Survey of Consumer Expectations (SCE), a monthly household survey that has been conducted by the Federal Reserve Bank of New York since 2012 (see \citeauthor*{armantier2017overview}, armantier2017overview, for a detailed description of the survey, and kuchler2015personal, kuchler2015personal; \citeauthor*{CPWZ18}, CPWZ18; \citeauthor*{FKZ18}, FKZ18 for recent articles using the SCE). The SCE is conducted with the primary goal of eliciting consumer expectations about inflation, household finance, labor market, as well as housing market. It is a rotating internet-based panel of about 1,200 household heads, in which respondents participate for up to twelve months.\footnote{Each survey takes on average about fifteen minutes to complete, and respondents are paid \$15 per survey completed.} Each month, the panel consists of about 180 entrants, and 1,100 repeated respondents. While entrants are overall fairly similar to the repeated respondents, they are slightly older and also have slightly lower incomes (see Table 1 in armantier2017overview, armantier2017overview).
Of particular interest for this paper is the supplementary module on labor market expectations. This module is repeated every four months since March 2014. Since March 2015, respondents are asked the following question about labor market earnings expectations ($\psi$) over the next four months: “What do you believe your annual earnings will be in four months?”. Implicit throughout the rest of our analysis is the assumption that these elicited beliefs correspond to the mean of the subjective beliefs distribution.\footnote{This assumption, while often made in the subjective expectations literature, is a priori restrictive. In this application, for the vast majority of the sub-groups of the population, the mean of $\psi$ cannot be statistically distinguished from the one of $Y$ (see Table (ref) below). This provides empirical support for this assumption.} In this module, respondents are also asked about current job outcomes, including their current annual earnings ($Y$), through the following question: “How much do you make before taxes and other deductions at your [main/current] job, on an annual basis?”.
Specifically, we use for our baseline test the elicited earnings expectations ($\psi$), which are available for two cross-sectional samples of household heads who were working either full-time or part-time at the time of the survey, and responded to the labor market module in March 2015 and July 2015 respectively. We combine this data with current earnings ($Y$) declared in July 2015 and November 2015 by the respondents who are working full-time or part-time at the time of the survey.\footnote{Throughout our analysis (with the exception of the number of observations reported in Table (ref)) we use the monthly survey weights of the SCE in order to obtain an estimation sample that is representative of the population of U.S. household heads. See armantier2017overview for more details on the construction of these weights. We also Winsorize the top 5 percentile of the distributions of realized earnings and earnings beliefs.} This leaves us with a final sample of 2,993 observations, which is composed of 1,565 earnings expectation observations, and 1,428 realized earnings observations. 51% (1,536) of these observations correspond to the sub-sample of respondents who are reinterviewed at least once. We refer to Table (ref) for additional details on our sample.
We summarize how we implemented the test in practice, either on the overall sample or on each subsample corresponding to the binary covariates in Table (ref). For each case, we start by winsorizing the distribution of realized earnings ($Y$) and earnings beliefs ($\psi$) at the 95% level.\footnote{We show in Table (ref) in Appendix (ref) that our results are robust to other levels of Winsorization.} Then, we perform the test without covariates, where we allow for multiplicative aggregate shock and thus test $H_{0S}$, with $q(y; c) = y/c$.\footnote{In our application, the parameter $c$ is estimated using survey weights from the SCE.} Then, we use the function $\mathtt{test}$ of our companion R package \href{https://CRAN.R-project.org/package=RationalExp}{RationalExp}.\footnote{See Section 3 in our user's guide RationalExp_R18 for details on this function.} We choose the same values for the tuning parameters $b_0=0.3$ and $\kappa=0.001$ as in the Monte-Carlo simulations in Section (ref). We also set $p=0.05$, $\epsilon= 0.05$, and $\eta= 10^{-6}$. Following andrews2016inference, the interval $ \widehat{\mathcal{Y}}$ is approximated by a grid of length $100$ from $\underset{i=1,\dots,n}{\min}\widetilde{Y}_i$ to $\underset{i=1,\dots,n}{\max}\widetilde{Y}_i$. Finally, we use 5,000 bootstrap simulations to compute the critical values of the test.
In Table (ref) below, we report the results from the naive test of RE (${\mathbb E}(Y)={\mathbb E}(\psi)$), and our preferred test (“Full RE”), where we allow for multiplicative aggregate shocks. We implement the tests both on the overall population and on separate subgroups. The latter approach allows us not only to identify which groups fail to rationalize RE, but also, and importantly, to account for the possibility that aggregate shocks may in fact differ across subgroups.
Several remarks are in order. First, using our test, we reject for the whole population, at any standard level, the hypothesis that agents form rational expectations over their future earnings. Second, we also reject RE (at the 5% level) when we apply our test separately for whites (non-Hispanics) and minorities, as well as low vs. high numeracy test scores.\footnote{Respondents' numeracy is evaluated in the SCE through five questions involving computation of sales, interests on savings, chance of winning lottery, of getting a disease and being affected by a viral infection. Respondents are then partitioned into two categories: “High numeracy” (4 or 5 correct answers), and “low numeracy” (3 or fewer correct answers).}
Third, the results from our test point to beliefs formation being heterogeneous across schooling (college degree vs. no college degree) and tenure (more or less than 6 months spent in current job) levels. In particular, we cannot rule out that the beliefs about future earnings of individuals with more schooling experience correspond to rational expectations with respect to some information set. Similarly, while we reject RE at any standard level for the subgroup of workers who have accumulated less than 6 months of experience in their current job, we can only marginally reject at the 10% level RE for those who have been in their current job for a longer period of time. As such, these findings complement some of the recent evidence from the economics of education and labor economics literatures that individuals have more accurate beliefs about their ability as they progress through their schooling and work careers \citep*[see, e.g.,][]{Stinebrickner12,AAMR16}.
Fourth, using the naive test of equality of means between earnings beliefs and realizations, one would instead generally not reject the null at any standard levels. The one exception is the subgroup of workers without a college degree, for whom the naive test yields rejection of RE at the 5% level. But, as discussed before, one cannot rule out that such a rejection is due to aggregate shocks.
Even though individuals in the overall sample form expectations over their earnings in the near future that are realistic, in the sense of not being significantly biased, the result from our preferred test shows that earnings expectations are nonetheless not rational. Taken together, these findings highlight the importance of incorporating the additional restrictions of rational expectations that are embedded in our test, using the distributions of subjective beliefs and realized outcomes to detect violations of rational expectations. That the variance test of RE never rejects the null at any standard levels indicates that it is important in practice to go beyond the first moments, and exploit instead the full distributions of beliefs and outcomes to detect departures from rational expectations. These results also suggest that, in order to rationalize the realized and expected earnings data, one should consider alternative models of expectation formation that primarily differ from RE in their third, or higher-order moments.
The results of the direct test of RE on the subsample of individuals who are followed over four months are reported in Table (ref) below. While these results generally paint a similar picture to the results of our test, there are some differences. In particular, the direct test rejects RE at the 5% level for men and at 1% for individuals with tenure greater than 6 months, whereas we do not reject RE for the former group and only marginally so, at the 10% level, for the latter. The direct test also rejects with less power than our test for certain groups (low numeracy, tenure lower than 6 months, and minorities). This lower power may seem surprising given that the direct test can exploit the joint distribution of $(Y,\psi)$, but is simply due to the important reduction in sample size when focusing on the subsample of individuals who are followed over four months results.
There are also important issues associated with the direct test, which generally warrant caution when interpreting the results from this test. Most importantly, as already discussed in Section (ref), the direct test is not robust to measurement errors on the subjective beliefs $\psi$. As shown in Proposition (ref) in Appendix (ref), it is however possible to derive a restriction on $\beta$ under RE. Specifically, if $\xi_\psi$ is positively correlated with $\varepsilon + \xi_Y$, we have, under RE,
where $\underline{\lambda}$ is a lower bound on the signal-to-noise ratio $\mathbb{V}(\psi)/\mathbb{V}(\xi_\psi)$. Table (ref) also reports the results of tests combining (ref) with the restrictions on the marginal distributions used in our full RE test. Adding the restriction (ref) does not change the results for values of signal-to-noise ratio between 5 and 20 (i.e., for noise-to-signal ratios between 5% and 20%). Overall, using the subsample of linked data $(Y,\psi)$ through this additional restriction does not add much to our test, at least once we account for possible measurement errors on the elicited beliefs. Another significant concern with the direct test, and, more generally, the use of linked data on $(Y,\psi)$, is that attrition may be endogenous. We discuss this issue in more details in Appendix (ref).
Coming back to our test, the rejection of RE for the overall population but also for most of the subpopulations are, in view of Proposition (ref), unlikely to be due to data quality issues. In that sense, these results may be seen as robust evidence against the RE hypothesis for individual earnings, at least in this context. As a result, conclusions of behavioral models based on the assumption that agents form rational expectations about their future earnings may be misleading. Exploring this important question requires one to go beyond testing though, by quantifying the extent to which model predictions are actually sensitive to the violations from rational expectations that have been detected with our test. We investigate this issue in DGM in the context of a life-cycle consumption model.
In this paper, we develop a new test of rational expectations that can be used in a broad range of empirical settings. In particular, our test only requires having access to the marginal distributions of realizations and subjective beliefs. As such, it can be applied in frequent cases where realizations and beliefs are observed in two separate datasets, or only observed for a selected sub-population. By bypassing the need to link beliefs to future realizations, our approach also enables to test for rational expectations without having to wait until the outcomes of interest are realized and made available to researchers. We establish that whether one can rationalize rational expectations is equivalent to the distribution of realizations being a mean-preserving spread of the distribution of beliefs, a condition which can be tested using recent tools from the moment inequalities literature. We show that our test can easily accommodate covariates and aggregate shocks, and, importantly for practical purpose, is robust to some degree of measurement errors on the elicited beliefs. We apply our method to test for rational expectations about future earnings, using data from the Survey of Consumer Expectations. While individuals tend to be right on average about their future earnings, our test strongly rejects rational expectations.
Beyond testing, in this application as in any other situations where rational expectations are violated, a natural next step is to evaluate the deviations from rational expectations that one can rationalize from the available data. In the context of structural analysis, a central question then becomes to which extent the main predictions of the model are sensitive to those departures from rational expectations. We explore this important issue and propose in DGM a tractable sensitivity analysis framework on the assumed form of expectations.