Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
79,409 characters · 16 sections · 66 citation commands
Did Harold Zuercher Have Time-Separable Preferences?
Keywords: dynamic discrete choice, Epstein-Zin preferences, preference for the timing of resolution of uncertainty
It is well-known that in the standard model of expected utility with separable time preferences, an agent's risk preference and her preference for intertemporal substitution are both captured by the curvature of her Bernoulli utility function. As a result, greater risk aversion is correlated with greater complementarity of consumption across different time periods. Risk and time however are two distinct phenomena and it would be natural to have different preferences over the two. Furthermore, time-separable models are widely recognized for imposing an unrealistic constraint on agents' preferences regarding the timing of uncertainty resolution. As rust1994structural pointed out, time-separable “expected-utility models imply that agents are indifferent about the timing of the resolution of uncertain events, whereas human decision-makers seem to have definite preferences over the time at which uncertainty is resolved.”
Models with non-separable time preferences such as epsein1989sub allow for a clean separation of the two: the elasticity of intertemporal substitution is independent of risk aversion. Moreover, the models allow agents to have nontrivial preferences regarding the timing of the resolution of uncertainty, as suggested by the experimental literature (see, e.g., nielsen2020preferences, meissner2022measuring). These feature makes them popular in fields such as macroeconomics and finance, where they have been used to explain phenomena such as the equity premium puzzle (see, e.g., mehra1985equity and bansalyaron2004).
However, current empirical research has yet to incorporate non-separable preferences within the dynamic discrete choice (DDC) framework. Misspecifying an agent's preference as time-separable when it is not can result in biased inferences about structural primitives and misleading policy recommendations. Our paper aims to bridge this gap. We begin by outlining a set of theoretical results crucial for empirical analysis in the DDC framework. Following this, we introduce an empirical model of dynamic discrete choice that incorporates non-separable time preferences, extending the well-known bus engine replacement model analyzed in rust1987optimal. Our model includes the standard time-separable model, commonly employed in the DDC literature, as a special case. This is an important and empirically relevant feature as it allows us to statistically test whether the decision maker (Harold Zuercher in the bus engine replacement example) has separable time preferences or not.
While our model allows for more general and richer time preferences, we show that it can be solved and estimated similarly as the standard DDC model. In the paper, we first show that agents with our nonseparable time-preferences have well-defined optimal dynamic choices. Based on the Bellman equation with our nonseparable time-preferences, we first prove the existence of the value function. Specifically, we apply lattice theory and Tarski's fixed point theorem to show the existence of the smallest and largest fixed points of a mapping in the space of value functions. This allows us to use a simulated nested fixed point algorithm to find the smallest and largest fixed points for model estimation; if the two coincide, then the value function is unique. We also provide general sufficient conditions that ensure a contraction mapping and uniqueness of the value function. These conditions reduce to the standard $\beta <1$ condition for separable preferences but we also consider specific parametrizations (CARA and CRRA) of Epstein-Zin preferences.
We apply our framework to the bus engine replacement data in rust1987optimal. In his original work, Rust assumes that the manager Harold Zuercher is risk neutral with separable time preferences. At the beginning of each period, Harold makes a dynamic choice for each bus engine by trading off between an immediate lump sum cost of replacing it and higher maintenance costs for keeping it. In addition to allowing Harold to have non-separable time preferences, we generalize the Rust model in several other aspects. We assume that the manager may be risk-averse and may benefit from operating the bus (e.g., collecting passenger fares proportional to the additional mileage in each month).
We estimate the model with non-separable and separable time preferences, respectively, under the CARA parameterization. Our estimation results of the non-separable model suggest that the agent is likely to prefer late resolution of uncertainty. Because the model with separable preferences is nested in the non-separable model, comparing the likelihood from the two models is straightforward, and we are able to statistically reject the null hypothesis that Harold has separable time preferences. Moreover, our estimates for the agent's risk attitudes and the payoff parameters are significantly different when allowing for non-separable preferences. We find that, given the time-separable model, the risk preference parameter is overestimated, while the maintenance cost per mileage is underestimated.
To better interpret the model estimates, we compute the certainty equivalent given different model specifications. Specifically, we consider a counterfactual scenario in which a subsidy program assists the agent in smoothing revenue across periods. We solve for the monetary payoff that makes the agent (Harold) indifferent when the uncertainty about incremental mileage in each period is eliminated. Our general model with non-separable preferences implies that Harold would accept a sure payment of \$120 each month to stay indifferent. In contrast, the certainty equivalent under the misspecified model with separable preferences is \$61.6, which is 48.7% lower than the value implied by the general model.
\paragraph{Related Literature} There are many empirical applications of DDC models. See miller1984job and buchholz2021semiparametric for dynamic choices of workers, pakes1986patents for patenting decisions, crawford2005uncertainty, hendel2006measuring, and gowrisankaran2012dynamics for dynamic consumer demand. As far as we know, there is no existing work in the DDC literature that has incorporated non-separable time preferences. Our main methodological advance is in developing a feasible specification of DDC models with EZ preferences, arguably the most popular model of non-separable time preferences. Following rust1987optimal, we propose a nested fixed point algorithm for estimating our model. Our proposed algorithm can be applied to estimate models with separable and non-separable preferences, and it therefore provides a useful tool for testing various model specifications.
The use of non-separable models has been common in the macrofinance literature since kreps1978temporal and epsein1989sub. For example, bansalyaron2004 provide a unified explanation for several long-standing puzzles in asset markets using a recursive utility model. epstein2014much explore the quantitative implications of recursive utility on temporal resolution of uncertainty. We find that the risk preference parameter is significantly overestimated under the misspecified model with separable preferences, suggesting the importance of separating preferences for risk and intertemporal substitution.
In the theoretical literature, frick2019dynamic provide an axiomatic analysis of dynamic random utility, focusing on the history-dependency of choices and the type of randomness inherent in these models. lu2020_repeated provide a theoretical model of dynamic stochastic choice in which the agent has non-separable preferences. They provide axiomatic foundations and also consider EZ preferences in an application focusing on its choice-theoretic implications. In contrast, our paper focuses on the DDC model as the workhorse model for analyzing dynamic decision processes in structural econometrics.
The rest of the paper is organized as follows. We describe EZ preferences and the dynamic discrete choice model setup in Section (ref). Theoretical results regarding the existence and uniqueness of the value function are provided in Section (ref). We discuss the estimation strategy and our empirical application in Section (ref) and (ref), respectively. Section (ref) concludes. The proofs are delegated to the appendices.
In this section, we introduce non-separable time preferences in an infinite-horizon recursive framework. First, recall that under standard risk and intertemporal preferences, the utility function is given by
Here, $u$ is a strictly increasing utility function over a set $\mathcal{C}$ of real-valued payoffs, $\beta\in\left[0,1\right]$ is the discount factor and $v$ is the future continuation value.\footnote{Note that since the value function is recursive, this is equivalent to $u\left(c\right)+\beta v$ as the $1-\beta$ term is a scalar that can be factored out.} Let $\mathcal{U}$ denote the range of $u$. It is easy to see here that risk is additively-separable across time.\footnote{To see this point, notice that because of the linearity of the model, ((ref)) reduces to $(1-\beta )\mathbb{E}_c[u(c)]+ \beta \mathbb{E}_v[v]$, where the first expectation captures the risk over current consumption; while the second expectation captures the risk over future consumption.}
Following kreps1978temporal and epsein1989sub, we consider a generalization of standard preferences that allow for non-separable time preferences, i.e., where the risk is not additively-separable across time.\footnote{Due to the non-linearity of $\phi$, the utility function ((ref)) cannot be expressed as the additive form of two expectations, unlike the time-separable model described in footnote (ref).}
We consider the following representation:
where $\phi$ is a strictly increasing aggregator function that is either concave or convex.\footnote{ epsein1989sub consider an even more general formulation.} Refer to Section 2.2 for parametric examples of the utility function ((ref)), which will aid in understanding the general formulation presented in ((ref)).
As we will see below, the curvature of $\phi$ captures the agent's attitudes towards how risk is resolved across different time periods. When $\phi$ is linear, this reduces to the standard time-separable preferences, which implies the the indifference to the timing of resolution of uncertainty, otherwise, risk cannot be additively separated across time as in Equation ((ref)). Note that although the above formula is recursive and assumes infinite periods, the functional form can be easily applied to finite time periods by calculating utilities from the last period via backwards induction.
The essential difference from separable preferences is the role of the aggregator $\phi$, which allows for greater flexibility in modeling risk and time preferences. In fact, agents may exhibit non-trivial preferences over when risk is resolved. Consider the two lotteries shown in Figure (ref). Both lotteries are the same in terms of probabilities and outcomes: they differ only in the timing of when uncertainty is resolved. In the left lottery $P$, the uncertainty about period 2 outcome ($10$ or $0$) is resolved in period 1. In the right lottery $Q$, on the other hand, the uncertainty is resolved in period 2. If the agent always prefers a lottery where uncertainty is resolved in period 1 (e.g., $P$) to a lottery where uncertainty is resolved in period 2 (e.g., $Q$), then we say that he prefers early resolution of uncertainty. If the agent always prefers a lottery where uncertainty is resolved in period 2 (e.g. $Q$) to a lottery where uncertainty is resolved in period 1 (e.g. $P$), then we say he prefers late resolution of uncertainty. If he is indifferent between the two lotteries, then we say that he is indifferent to the timing of resolution of uncertainty.
In our model, preferences for the timing of resolution of uncertainty are fully captured by the the curvature of of $\phi$. To see why, consider the example above. Applying (ref) to two-periods, the value of lottery $P$ is given by \[ V\left(P\right)=\mathbb{E}\left[\phi\left(\left(1-\beta\right)u\left(5\right)+\beta z\right)\right] \] where $z$ is a random variable that takes on either $\left(1-\beta\right)u\left(0\right)$ or $\left(1-\beta\right)u\left(10\right)$. Note here that all uncertainty about $z$ is resolved in the first period. On the other hand, the value of lottery $Q$ is given by \[ V\left(Q\right)=\phi\left(\left(1-\beta\right)u\left(5\right)+\beta\phi^{-1}\left(\mathbb{E}\left[\phi\left(z\right)\right]\right)\right) \] Here, the uncertainty about $z$ is not resolved until the second period, so the expectation is taken inside the function $\phi^{-1}\left(\cdot\right)$. If we define $\tilde{\phi}\left(z\right):=\phi\left(\left(1-\beta\right)u\left(5\right)+\beta z\right)$, then the agent prefers early resolution of uncertainty if $V\left(P\right)\geq V\left(Q\right)$ or
which holds if $\tilde{\phi}$ is less concave than $\phi$.
We can now generalize how the curvature of $\phi$ captures preference for early or late resolution of uncertainty. Recall that the Arrow-Pratt measure of a function $\phi$ is given by \[ A_{\phi}\left(z\right):=-\frac{\phi^{\prime\prime}\left(z\right)}{\phi^{\prime}\left(z\right)}. \]
To see the condition in Proposition (ref) more explicitly, note that \[ A_{\phi_{y}}\left(z\right)=\beta A_{\phi}\left(\left(1-\beta\right)y+\beta z\right) \] so the agent prefers early resolution of uncertainty iff \[ \beta A_{\phi}\left(\left(1-\beta\right)y+\beta z\right)\leq A_{\phi}\left(z\right) \] for all $y\in\mathcal{U}$. The condition for preference for late resolution is the same but with the inequality reversed.
Preferences over the timing of resolution of uncertainty can play an important role in agents' decision making process in dynamic environments. Different choices are not only associated with different levels of uncertainty, but are also related to when that uncertainty is realized over time. For example, in Rust's bus engine replacement problem, replacing an engine entails current (sure) cost but makes the future uncertainty of engine failure resolved earlier. On the other hand, not replacing the engine leads to the future uncertainty of engine failure resolved later.
There is no conclusive experimental evidence on whether individuals have preference for early or late resolution of uncertainty. In fact, agents' preference about the timing of resolution of uncertainty could be highly context-dependent. For example, when uncertainty is over positive consumption levels, a preference for early resolution may be more reasonable. Indeed, in the macro-finance literature, the assumption of preference for early resolution of uncertainty is often imposed. On the other hand, in cases when payoffs can be very negative, individuals may prefer to avoid early resolution of uncertainty. As a heuristic example, consider a pronouncement of a fatal disease from a doctor. Some patients may prefer to learn the news later because this information can induce a great deal of disappointment, fear and anxiety. These negative emotions, while may not have instrumental values, could still directly affect their utility.\footnote{Note that in the definition of a preference for the timing of resolution of uncertainty, the information must be non instrumental, i.e., the agent cannot do anything given the information.}
We now present several parametrizations of non-separable time preferences that have been used in the literature.
By far the most common parametric form of Epstein-Zin preferences is when both $u$ and $\phi$ have CRRA (or power) functional forms. Here, the utility is given by $u\left(c\right)=c^{1-\rho}$ and the time aggregator is given by \[ \phi\left(z\right)=\left(z^{\frac{1}{1-\rho}}\right)^{1-\alpha}=z^{\frac{1-\alpha}{1-\rho}} \] where $\alpha\leq1$ and $\rho\leq1$ are parameters that characterize intertemporal and risk preferences respectively. Under these assumptions, the value function ((ref)) yields the following form:
To see how $\rho$ and $\alpha$ characterize intertemporal and risk preferences, first consider the case where there is no risk so we can remove the expectations in (ref). Redefining a new value function as $\tilde{v}=\phi^{-1}\left(v\right)$, we obtain from (ref) that the new value function is given by \[ \left(1-\beta\right)u\left(c\right)+\beta\tilde{v}=\left(1-\beta\right)c^{1-\rho}+\beta \tilde{v} \] where intertemporal preferences are determined solely by $\rho$. Here, $\rho$ measures the elasticity of substitution, defined as the percentage change of the growth rate of consumption (i.e. $c_{t+1}/c_{t}$) to the percentage change of the intertemporal marginal rate of substitution (MRS):\footnote{Recall that MRS is defined as $u^{\prime}\left(c_{t+1}\right)/u^{\prime}\left(c_{t}\right)$.}
For risk preferences, consider a one-time shock that resolves within a single-period followed by zero consumption forever. It straightforward to see that the Bernoulli utility in (ref) reduces to $c^{1-\alpha}$, which has standard constant relative risk aversion (CRRA) parameter $\alpha$. The fact that both intertemporal and risk preferences can be so neatly separated here (which is not possible with standard separable preferences) speaks to its popularity in the literature.
Proposition (ref) reduces to the following in this parametrization.
An alternative parameterization is when $u$ has the CARA (or exponential) functional form. Here, the utility is given by $u\left(c\right)=u_{\rho}\left(c\right)$ where
and the time aggregator is given by \[ \phi\left(z\right)=u_{\alpha}\left(u_{\rho}^{-1}\left(z\right)\right)=\alpha^{-1}\left(1-\left(1-\rho z\right)^{\frac{\alpha}{\rho}}\right) \] for parameters $\alpha\geq0$ and $\rho\geq0$ that characterize intertemporal and risk preferences. Under these assumptions, the value function ((ref)) yields the following form
These preferences have appeared in skiadas2009asset and are exactly the CARA analog of CRRA Epstein-Zin preferences. One advantage of this formulation is that it allows for negative payoffs, which CRRA cannot accommodate. For example, in rust1987optimal, the agent needs to pay a large cost when choosing to replace the bus engine resulting in net negative revenue that period.
As in CRRA Epstein-Zin, the parameters $\rho$ and $\alpha$ characterize intertemporal and risk preferences. First, in the absence of risk, we can follow the same reasoning as for CRRA Epstein-Zin above by redefining a new value function $\tilde{v}=\phi^{-1}\left(v\right)$. We now obtain from (ref) that the new value function is given by \[ \left(1-\beta\right)u\left(c\right)+\beta\tilde{v}=\left(1-\beta\right)\rho^{-1}\left(1-e^{-\rho c}\right)+\beta \tilde{v} \] so intertemporal preferences are determined solely by $\rho$. Here, in contrast to CRRA Epstein-Zin, $\rho$ measures the responsiveness of the difference in consumption between two periods (i.e. $c_{t+1}-c_{t}$) to the percentage change of MRS:
For risk preferences, as before, consider a one-time shock that resolves within a single-period followed by zero consumption forever. It straightforward to see that the Bernoulli utility in (ref) has Arrow-Pratt coefficient \[ \frac{\left(1-\beta\right)\alpha+\beta e^{\rho c}\rho}{\left(1-\beta\right)+\beta e^{\rho c}}. \] This is a convex combination between $\alpha$ and $\rho$ where the weight on $\alpha$ is decreasing in $\beta$ and consumption $c$. Note that holding all else constant, risk aversion is increasing in $\alpha$. Unlike CRRA Epstein-Zin preferences, while intertemporal preferences depend solely on $\rho$, risk preferences here depend on both $\alpha$ and $\rho$.
Proposition (ref) reduces to the following in this parametrization.
Finally, both CRRA and CARA Epstein-Zin preferences nest separable preferences as special cases. When $\alpha=\rho$ in either case, the time aggregator becomes linear, i.e. $\phi\left(z\right)=z$, and the value function becomes
where $u$ is either of the CRRA or CARA functional form. Furthermore, when $\alpha=\rho\rightarrow0$ under either, $u\left(c\right)=c$ so this reduces to the separable risk-neutral case. This is the specification commonly adopted in many empirical studies, including the engine replacement model in rust1987optimal.
The fact that separable preferences, i.e. $\alpha=\rho$, is nested in our more general parametrizations allows us to statistically test whether the agent has separable preferences. While separability provides computational convenience, misspecifying the agent's preference as additively time separable when it is not can lead to systematic biases in inferences of the structural primitives of the model. We see this for the case of CARA Epstein-Zin preferences in Section (ref).
We now apply the general non-separable preferences defined in the previous section to a general dynamic discrete choice framework. We provide two key theoretical results. The first result pertains to the existence of a value function, while the second addresses its uniqueness.
Denote the set of states by $\mathcal{X}$ and sets of actions by $\mathcal{D}$. We assume $\mathcal{D}$ is finite. The timing of our setup is as follows. At the beginning of period $t$, the state $x_t$ realizes and then an idiosyncratic shock $\varepsilon_{t}$ realizes. The agent then makes a choice $d_{t}\in \mathcal{D}$. Finally, the uncertainty about another variable $\Delta_t$, which affects the current period consumption is realized. The agent consumes the realized consumption that period, which is given by
where $\pi(d_{t},x_{t},\Delta_{t})$ represents the deterministic consumption. We assume that $\varepsilon_{t}(d_{t})$ is a random shock for alternative $d_t$. The vector of random shocks $\varepsilon_t = (\varepsilon_t(1), \cdots, \varepsilon_t(| \mathcal{D} |))$ is drawn from a cumulative distribution function $G(\cdot)$.
We represent the agent's optimization problem using the Bellman’s equation as follows.
The first expectation is due to uncertainty in the current-period payoff shock $\Delta_{t}$, and the second expectation is due to uncertainty in the future state $(x_{t+1}, \varepsilon_{t+1})$. We follow the dynamic discrete choice literature to assume that (1) the random shock $\varepsilon$'s are independent over time and (2) conditional on the current state $x_t$ and choice $d_t$, the future state $x_{t+1}$ is independent of the unobserved state $\varepsilon_t$. Given these assumptions, the distribution of future state $(x_{t+1}, \varepsilon_{t+1})$ does not depend on $\varepsilon_t$. We therefore omit $\varepsilon_t$ in the second expectation $\mathbb{E}_{x_{t+1},\varepsilon_{t+1} | d, x_t, \Delta_t}$.
From the Bellman's Equation in ((ref)), we define the ex-ante value function as
Note that $V(x_{t+1})$ represent the agent's expected value at the beginning of period $t+1$ before the realization of the future shock $\varepsilon_{t+1}$. If the agent chooses $d_t$, the value she receives can be represented as
Note that $v\left(d_{t},x_{t},\varepsilon_{t}\right)$ is essentially the “choice-specific” value function defined in the dynamic discrete choice literature. The value functions in Equation (ref) are non-separable in random shocks $\varepsilon_{t}$ even if we assume that those shocks enter the current payoff additively as in Equation ((ref)). In the special case when $\phi$ is linear and the agent is risk-neutral, $\varepsilon_t$ is additively separable from other expressions in the value function and this reduces to the standard discrete choice setup.
The decision maker's optimal choice is defined as \[ d_{t}^{\ast}=\arg\max_{d \in \mathcal{D}}\text{ } v(d,x_{t},\varepsilon_{t}), \] which leads to a Conditional Choice Probability (CCP) of choosing action $d_t$
With the non-separability of random shocks, the CCP in Equation ((ref)) and the ex-ante value function in Equation ((ref)) do not have closed-form solutions even if we assume that $\varepsilon$'s follow extreme value distributions.
Combining Equations (ref) and (ref), we can define a value function iteration operator $T$ in the space of the ex-ante value functions. To simplify notation, define \[ r(d_{t},x_{t}, \Delta_t,\varepsilon_{t},V):=\phi\left(\left(1-\beta\right)u\left(c\left(d_{t},x_{t}, \Delta_t, \varepsilon_{t}\right)\right)+\beta\phi^{-1}\left( \mathbb{E}_{x_{t+1}| d_t, x_t, \Delta_t}V\left(x_{t+1}\right)\right)\right). \] Then the operator $T$ is defined as
where $F_{d_{t},x_{t}}(\Delta_t)$ represents the cumulative distribution function of $\Delta_{t}$ given the current state $x_{t}$ and the agent's choice $d_{t}$. The ex-ante continuation value $V$ is obtained as a solution of the functional equation $T(V)=V$.
In this section, we establish existence of a value function. We show that the operator $T$ defined in Equation (ref) always has a fixed point under some mild assumptions. Moreover, we demonstrate how to obtain the largest and smallest fixed points.
Let $\underline{\pi}$ and $\bar{\pi}$ denote the smallest and largest possible value of profit $\pi$. Define
where $\hat{\varepsilon}:=\max_{d\in\mathcal{D}}\varepsilon_{d}$. These are bounds for the largest and smallest possible values of continuation values. Suppose $v^{\ast}$ and $v_{\ast}$ are both finite and let $\mathcal{V}$ be the set of value functions bounded between some $\underline{v}\leq v_{\ast}$ and $\bar{v}\geq v^{\ast}$, that is, the set of $V$ such that $\underline{v}\leq V\left(x\right)\leq\bar{v}$ for all $x\in\mathcal{X}$.
We first show that $T:\mathcal{V}\rightarrow\mathcal{V}$ is well-defined (see Appendix (ref)). We then define a complete lattice $\left(\mathcal{V},\geq\right)$, where $\geq$ is the partial order on $\mathcal{V}$ such that $V\geq V^{\prime}$ if $V\left(x\right)\geq V^{\prime}\left(x\right)$ for all $x\in\mathcal{X}$.\footnote{A partially ordered set $\left(\mathcal{V},\geq\right)$ is a complete lattice if every subset has both a supremum and an infimum in $\mathcal{V}$ according to the partial order $\geq$.} It is easy to see that $T$ is monotonic, that is, $V\geq V^{\prime}$ implies $T\left(V\right)\geq T\left(V^{\prime}\right)$.
Since $T$ is monotonic, Tarski's fixed point theorem (see AB2006infinite) guarantees that the set of fixed points of $T$ is non-empty and a complete lattice. Moreover, if we start iterating from the smallest and largest possible values of $V$, then we obtain the respective smallest and largest fixed points of $T$. We summarize this as follows.
Existing results assume separable time preferences. Our contribution is demonstrating that for non-separable time preferences, we can apply Tarski's theorem as long as the $T$ mapping is monotonic (i.e. increasing) in the value function.\footnote{ jia2008 also uses Tarski's fixed point theorem but for a different problem (entry game). She also assumes separable time preferences.} This also provides a way to compute the smallest and largest value functions by iterating $T$. Although this does not guarantee a unique fixed point, we can easily check by calculating the smallest and largest fixed points and seeing if they coincide.
How about specific parametrizations of non-separable preferences? Theorem (ref) implies the following for CRRA Epstein-Zin preferences.
With CARA Epstein-Zin preferences, Theorem (ref) implies the following.
The condition in Corollary (ref) is weak; it is equivalent to saying that the moment generating function for $-\varepsilon$ always exists. This is true if $\varepsilon$ is Normal or Extreme-Value distributed, which we note below.
In this section, we present conditions under which $T$ constitutes a contraction, ensuring the uniqueness of the value function. First, recall from Proposition (ref) that $\phi_{y}\left(z\right)=\phi\left(\left(1-\beta\right)y+\beta z\right)$ and define \[ \psi_{y}\left(z\right):=\phi_{y}\left(\phi^{-1}\left(z\right)\right). \] Recall that $\underline{\pi}$ and $\bar{\pi}$ are the smallest and largest possible value of profit $\pi$. Also recall that $\underline{v}$ and $\bar{v}$ are the bounds of the value function (which can be infinite). We now present our second theorem.
Theorem (ref) implies that when the bound $M<1$, the ex-ante value function $V(\cdot)$ is a unique fixed point of the contraction mapping $T$ as defined in equation ((ref)). For intuition, note that $\psi_{y}\left(z\right)$ captures the value given current period utility $y$ and future continuation value $z$. For instance, if $\phi\left(z\right)=z$ as in the standard separable case, then \[ \psi_{y}\left(z\right)=\left(1-\beta\right)y+\beta z. \] In this case, $\psi_{y}\left(\cdot\right)$ is linear so $\psi_{y}^{\prime}\left(z\right)=\beta$. This immediately yields the standard result that coincides with Rust's condition for a contraction mapping.
In general, when the agent does not have separable preferences, the bounds on $\psi_{y}^{\prime}\left(z\right)$ would depend on the curvature of $\phi$. More explicitly, we have
where $\tilde{z}=\phi^{-1}\left(z\right)$. We thus have \[ \psi_{y}^{\prime\prime}\left(z\right)=\frac{\phi_{y}^{\prime}\left(\tilde{z}\right)}{\phi^{\prime}\left(\tilde{z}\right)^{2}}\left(A_{\phi}\left(\tilde{z}\right)-A_{\phi_{y}}\left(\tilde{z}\right)\right). \] From Proposition (ref), this implies that $\psi_{y}^{\prime}\left(\cdot\right)$ is increasing (decreasing) if the agent prefers early (resp. late) resolution of uncertainty. This implies the following.
Note that this bound can be further simplified if we know whether $\phi$ is convex or concave. For instance, if $\phi$ is convex, then $\psi_{y}^{\prime}\left(z\right)$ is increasing in $y$ so we can use $\pi=\bar{\pi}$ as the bound when evaluating $M$ in Equation ((ref)). The case for concave $\phi$ is symmetric.
How about specific parametrizations of non-separable preferences? Theorem (ref) implies the following for CRRA Epstein Zin preferences.
Note that in the other $\rho>\alpha$ case, we can still use Theorem (ref) to obtain a sufficient condition for a contraction mapping but the bound $M$ in Equation ((ref)) would depend on the error distribution. The reason we are able to obtain a simple sufficient condition for when the agent prefers early resolution of uncertainty ($\rho\leq\alpha$) is that the value function under CRRA Epstein-Zin is bounded below (since $u(c)$ has to be non-negative). The fact that values are bounded below and $\phi$ is concave given $\rho\leq\alpha$ (see Corollary (ref)), allows us to create bounds to ensure a contraction mapping.
With CARA Epstein-Zin preferences, Theorem (ref) implies the following.
In other words, for an agent with CARA Epstein-Zin preferences and prefers late resolution of uncertainty (i.e. $\rho\geq\alpha$), $T$ is a contraction mapping as long as $\beta$ is sufficiently small (i.e. $\beta^{\frac{\alpha}{\rho}}<1$). The fixed point would be unique. As with the CRRA Epstein-Zin case, we can still use Theorem (ref) for the case when $\rho<\alpha$ but the bound $M$ would again depend on the error distribution. Note that in contrast to CRRA Epstein-Zin, values are bounded above under CARA Epstein-Zin, so it is when the agent prefers late resolution of uncertainty ($\rho\geq\alpha$) where we can obtain simple sufficient conditions for a contraction mapping.
We propose to use an estimation approach similar to rust1987optimal's Nested Fixed Point algorithm to estimate the structural parameters in the dynamic model described in Section (ref).\footnote{Since rust1987optimal, several estimation methods for standard DDC models became available, such as the CCP approach of hotz1993conditional, the nested pseudo-likelihood approach of aguirregabiria2002swapping and their extensions to various settings. See aguirregabiria2010dynamic for a survey of methodologies. Recently, igami2017estimating and igami2020mergers extend the Nested Fixed Point algorithm to dynamic oligopoly games with sequential or stochastically alternating moves.} In our model, the choice-specific value function is not additively separable in the idiosyncratic shock $\varepsilon$ due to the nonlinear nature of the non-separable time preferences. Therefore, the well-known multinomial logit formula for the conditional choice probabilities and the McFadden's social surplus function are no longer applicable. We describe in this section how we obtain the conditional choice probabilities using simulation methods. Our nested fixed point algorithm contains an inner loop that solves a DDC problem for a given set of parameter values and an outer loop that maximizes the log-likelihood.
We focus on the estimation of two parameterizations of the model introduced in Section (ref). Let $\theta = (\alpha, \rho, \theta_\pi)$ denote the vector of parameters to be estimated, where $\alpha$ and $\rho$ are parameters in the utility function $u$ and aggregator function $\phi$ in either the CRRA or CARA Epstein-Zin case. These two parameters capture the agent's risk and intertemporal preferences. $\theta_\pi$ represents the parameters in the payoff function $\pi(d_t, x_t, \Delta_t)$. Throughout the estimation, we assume that the value of the discount factor $\beta$ is known to the econometrician.\footnote{The identification and estimation of the discount factor $\beta$ in standard time-separable models is known to be difficult and often requires exclusion restrictions or the availability of terminating actions. See discussions in fang2015estimating, bajari2016identification, komarova2018joint, abbring2020identifying, schneider2021identification.} Let $x_{it}$ and $d_{it}$ denote the observable state and choice for agent $i=1, 2, \cdots, N$ at period $t=1, 2, \cdots, T$. $\Delta_{it}$ represents the realization of current period uncertainty. We derive the likelihood of the choice pattern and state transition process observed in the data:
Note that $Pr(d_{it} | x_{it}; \theta)$ represents the choice probability conditional on the observable state $x_{it}$ and $Pr(x_{it} | d_{it-1}, x_{it-1}, \Delta_{it-1})$ represents the state transition probability, which can be estimated separately outside of the structural model.
For a given parameter $\theta$, we solve the ex-ante value function $V(x_t)$ through an iterative procedure outlined as follows.
The contraction property proved in Theorem (ref) guarantees the uniqueness of the value function. For our specific parameterizations of non-separable preferences, Corollaries (ref) and (ref) provide sufficient conditions for a contraction mapping when agents have certain preference for the temporal resolution of uncertainty under CRRA and CARA Epstein-Zin preferences, respectively. When the bound $M$ in Theorem (ref) is difficult to compute, we can check empirically whether the largest and the smallest fixed points coincide with each other. Theorem (ref) guarantees that we obtain the largest (smallest) fixed point of $T$ through the iterative procedure above if we start iterating from the largest (smallest) possible values of $V$. In all of our simulation exercises and empirical application, the fixed points are all unique.
At the point of convergence, the conditional choice probabilities $Pr(d_{it}|x_{it};\theta)$ can also be computed numerically.
In the outer loop of the algorithm, we search for the value of $\theta \in \Theta$ to maximize the log-likelihood function in Equation ((ref)), i.e., the estimator for the structural parameter is $\hat{\theta}=\max_{\theta \in \Theta} LL(\theta)$.
Before moving on to the empirical example, we briefly discuss identification of the model parameters. With the introduction of non-separable time preferences, our model is much richer than the standard additively-separable model. Most importantly, the value functions in our model are non-separable in the random shocks $\varepsilon_t$. Existing identification results for time separable models (e.g., magnac2002identifying, magnac2002identifying; arcidiacono2020identifying, arcidiacono2020identifying) rely on the inversion from CCPs to value functions given additively separable random shocks (hotz1993conditional, hotz1993conditional), and thus do not directly apply to our model.
Our empirical model imposes specific parametric assumptions on $u$ and $\phi$. These parameters affect the agent's risk and intertemporal preferences in non-trivial ways so deriving generic identification results is not a straightforward exercise. All model parameters (i.e., $\theta$) jointly determine the conditional choice probabilities $p(d_t|x_t)$ in Equation (ref). Matching the model-implied CCPs with their empirical counterparts for various $x_t$, we essentially characterize $\theta$ as the solution to a system of nonlinear equations. Given the high nonlinearity of the dynamic model, one possibility is to at least locally identify $\theta$ given certain rank conditions (see lewbel2019identification for more detailed discussions and examples on local identification). Since the main purpose of our paper is to provide a general framework of dynamic discrete choice models that allow for non-separable time preferences, we leave a thorough investigation of identification for future research.
We apply our model to the dataset of bus engine replacement decisions originally studied in rust1987optimal. The manager (Harold Zuercher) makes a dynamic choice for each bus engine by trading off between an immediate lump sum cost of replacing it and higher maintenance costs for keeping it at the beginning of each period. We extend the Rust model to allow for non-separable time preferences, agents being risk-averse, and earning revenues from operating the bus.
The observable state variable $x_t$ represents the accumulated mileage at the beginning of period $t$. Let $\varepsilon_t$ denote the unobserved payoff shocks and $\Delta_t$ denote the incremental mileage realized within period $t$. The agent decides whether or not to replace the engine upon observing $(x_t, \varepsilon_t)$. Let $d_t=1$ represent the case where the engine is replaced, and $d_t=0$ otherwise. The timeline of the model is shown in Figure (ref).
We first specify the agent's payoff function
In Equation ((ref)), $RC$ represents the replacement cost and $\theta_d \Delta_t$ represents the revenue collected by the agent within a month for the incremental mileage $\Delta_t$. When the bus engine is not replaced, the agent has to pay a maintenance cost proportional to the accumulated mileage, i.e., $\theta_x x_t$. The realized consumption agent has in period $t$ is given by
where $\sigma$ is the standard deviation of the random shock and $\varepsilon_t(d_t)$ is the shock that the agent receives for alternative $d_t \in \{0, 1\}$. Note that $\varepsilon_t=(\varepsilon_t(1), \varepsilon_t(0))$ and it is randomly drawn from a cumulative distribution function $G(\cdot)$. Note that in this empirical application, the payoff agent receives might be negative. For example, if the agent chooses to replace the engine, he needs to pay an immediate replacement cost. We adopt the CARA Epstein-Zin preferences described in Section (ref) as it allows for negative payoffs.
Applying the CARA parameterization to the choice-specific value function defined in Equation ((ref)), we have
where $V(x_{t+1})$ is the ex-ante value function given by
and the next period accumulated mileage $x_{t+1}$ is updated following:
Intuitively, if the agent chooses to replace the engine, i.e., $d_t=1$, the current accumulated mileage is reset to zero, so the future mileage equals to the incremental mileage $\Delta_t$. If the agent does not replace the engine, the next-period accumulated mileage is increased by $\Delta_t$. Note that in this engine replacement example, the future state variable $x_{t+1}$ is uniquely determined once $d_t$, $x_t$ and $\Delta_t$ are realized. In other words, there is no uncertainty with respect to $x_{t+1}$ conditional on $(d_t, x_t, \Delta_t)$. We therefore omit the expectation $\mathbb{E}_{x_{t+1}|d_t, x_t, \Delta_t}$ in front of $V(x_{t+1})$. The probability that an engine is replaced conditional on the observed state $x_t$ is
{ Based on the empirical model described above, we conduct a numerical exercise to compare the conditional choice probabilities under separable and non-separable preferences. In Figure (ref), the red and blue curve represent the probabilities of replacing the engine when the agent has separable and non-separable time preferences, respectively. Throughout the exercise, we fix the intertemporal preference parameter $\rho$ to be 0.5 and change the value of risk preference parameter $\alpha$. Note that when $\alpha=\rho=0.5$, the two models are equivalent, resulting in the same conditional choice probabilities (i.e., the red and blue curves intersect at $\alpha=0.5$). When the agent prefers late resolution of uncertainty (i.e., $\rho > \alpha$), if we misspecify the agent's preference to be time-separable and use the observed CCP to estimate the agent's risk preference, we would over-estimate $\alpha$. Conversely, we would under-estimate $\alpha$ if the agent prefers early resolution of uncertainty (i.e., $\rho<\alpha$). This exercise highlights the consequences of misspecifying the agent's preference to be time-separable when it is in fact not. Importantly, the direction of the bias depends on whether the agent prefers early or late resolution of uncertainty.
In the estimation, we discretize mileage into 130 intervals of length 3000 (miles) and assume that $\varepsilon$'s are drawn from a standard normal distribution. To identify the scale of the unobserved random shock, we fix the $RC$ to be 8 based on the cost of engine replacement reported by Harold Zurcher (see rust1987optimal Table III). We set the discount factor $\beta=0.9$ and use 2,500 simulation draws when approximating value functions in the iteration process. To summarize, the structural primitives to be estimated include: parameters for risk-aversion and intertemporal preferences, $\alpha$ and $\rho$; coefficient for revenue and maintenance cost, $\theta_d$ and $\theta_x$; and standard deviation of the idiosyncratic shock, $\sigma$.
In Table (ref), we present the estimation results for four model specifications with standard errors provided in the parenthesis. We first estimate a model with non-separable time preferences, where we impose no restrictions on the values of $\alpha$ and $\rho$. Second, we restrict that $\alpha=\rho$, which leads to a model with separable time preferences and risk aversion. Next, we consider adding revenue to the original Rust model. Finally, for comparison purposes we also estimate the original Rust model\footnote{In rust1987optimal, the distribution of $\varepsilon$ is assumed to be Type I extreme values, mainly for computational convenience. We use normal distribution for the shocks $\varepsilon$.}.
Note that in the Rust model with revenue, the choice-specific value function is define as
This is nested in the non-separable case where $\alpha = \rho \rightarrow 0$. The original Rust model further removes the revenue from the payoff function $\pi$ (or in other words, restricting $\theta_d=0$), so that the choice-specific value function is reduced to:
From the first column of Table (ref), we find that $\rho > \alpha$, suggesting that the agent is likely to prefer late resolution of uncertainty. Comparing the log-likelihood between Columns (1) and (2), we are able to reject the hypothesis that the agent has a separable preference at a 10% significant level (the likelihood ratio test statistic = 2.747, $p$-value = 0.097). When allowing for non-separable time preferences, our estimate for $\alpha$ is smaller. This is consistent with our simulation exercises: when agents prefer late resolution of uncertainty, misspecifying the agent's preference to be time-separable results in an overestimate of the risk preference parameter $\alpha$.
Our estimates for the payoff parameters $\theta_d$ and $\theta_x$ are also significantly different when allowing for non-separable preferences. In particular, we underestimate the maintenance cost parameter $\theta_x$, while overestimate the revenue parameter $\theta_d$ under time-separable preferences. This result highlights that misspecifying agents' preferences can lead to biased inferences about structural primitives, and potentially misleading policy recommendations. Comparing Columns (3)-(4), we find that adding revenue to the original rust model has a minimal impact.
The parameters that capture agents' risk and time preferences ($\alpha$ and $\rho$) are important for predicting choices under uncertainty in dynamic settings and for making policy recommendations. To better interpret the estimated values of $\alpha$ and $\rho$, we compute the certainty equivalent given different model specifications. In the engine replacement model, the agent faces uncertainty about the incremental mileage in each period, which not only affects the current-period revenue, but also affects future maintenance costs. We compute the monetary payoff that makes the agent indifferent when the uncertainty about incremental mileage in each period is removed. Specifically, we assume that in a counterfactual scenario there is a subsidy program that helps the agent to smooth revenue across periods. The agent receives $C$ each period no matter what the incremental mileage is. This is similar to the “capacity payment" often seen for power plants in the electricity industry and pipelines in the natural gas industry.\footnote{Regulators have adopted capacity payment mechanisms to encourage capacity investment and promote system reliability. See details of capacity payment mechanisms in electricity markets in pfeifenberger2009comparison.}
We compute the certainty equivalent using estimates from both separable and non-separable models. We find that agents with separable preferences are willing to accept a sure payment of \$120 each month to stay indifferent. However, for agents with non-separable time preferences, the calibrated subsidy payment is \$61.6, almost a half of what an agent with separable time preferences would require. This result is intuitive, because when the agent prefers late resolution of uncertainty, the original setting where there is uncertainty in revenue is less unfavorable. This exercise confirms that misspecifying agents’ preferences as time-separable when it is in fact not can give rise to misleading policy implications.
In this paper, we propose an empirical model of dynamic discrete choice with Epstein-Zin preferences, generalizing the well-known bus engine replacement model studied in rust1987optimal. Based on the Bellman equation with Epstein-Zin preferences, we prove the existence of the value function and provide conditions under which it is the unique fixed point of a contraction mapping. We propose a simulated nested fixed point algorithm for model estimation based on the theoretical results. Since Epstein-Zin preferences include separable expected utility as a special case, our framework allows us to test whether an agent has separable time preferences or not.
We apply the framework to the bus engine replacement data and show that we can reject the null hypothesis that Harold has time-separable preferences. Based on our estimates, the agent is likely to prefer late resolution of uncertainty. Misspecifying the agent's preference to be time separable when it is in fact not leads to a biased inference of agents' risk and intertemporal preferences as well as other key structural parameters. This has non-trivial consequences for making policy evaluations.
The tools we develop in this paper can be easily applied to analyze many empirical dynamic discrete choice models in industrial organization, environmental, health, and labor economics while allowing for more general non-separable time preferences. For example, a recent empirical literature studies the impact of uncertainty on economic outcomes such as oil drilling, firm investment, technology adoption, and exit decisions (see kellogg2014effect, kellogg2014effect; dorsey2019waiting, dorsey2019waiting; handley2020measuring, handley2020measuring; gowrisankaran2022policy, gowrisankaran2012dynamics). In these applications, preferences for the temporal resolution of uncertainty would naturally play an important role in firms' dynamic responses to policy changes, such as resolving policy uncertainty early. Allowing for more general non-separable time preferences could lead to different welfare implications.