Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
106,937 characters · 24 sections · 95 citation commands
Identification and Estimation of Dynamic Games with Unknown Information Structure
\newgeometry{left=1.25in,right=1.25in,top=0.75in,bottom=1.25in}
Keywords: Dynamic games, Markov, correlated equilibrium, information, partial identification
\restoregeometry
This paper develops a computationally tractable econometric framework for estimating empirical dynamic discrete game models with weak assumptions on players' information. Let
describe the data-generating process under Markov perfect equilibrium, where $\Theta$ is the set of structural parameters, $\mathcal{S}$ is the set of information structures, and $\mathcal{D}$ is the data; the correspondence in (ref) reflects the possibility of multiple equilibria. The standard identification argument relies on inverting (ref) under a specific “incomplete information” assumption---namely that players observe their own payoff shocks but not those of their rivals---which simplifies the estimation problem by making it closely parallel to that of single-agent models \citep*{Rust1994, Aguirregabiria2007, Bajari2007, Pakes2007, pesendorfer2008asymptotic, bugni2021iterated, dearing2024efficient}.\footnote{See \citet*{aguirregabiria2021dynamic} for a recent survey of the literature.}
While this assumption has enabled computationally tractable estimation of dynamic games, researchers often have little evidence to justify it. Unlike observable outcomes such as prices, sales, or entry decisions, information sets are inherently unobservable and may reflect complex mixtures of private knowledge, noisy signals, and shared information transmitted through suppliers, trade associations, or industry channels. Firms may also differ in their ability to gather market intelligence: some obtain precise signals about profitability, while others rely on noisy information. In addition, the timing and granularity of what firms know about their rivals can vary widely.
A growing body of research shows that assuming the information structure is known, while misspecifying players' information, can generate substantial bias in both static (grieco2014discrete; pakes2015moment; \citealp*{Syrgkanis2021}; Magnolfi2023; koh2023stable; Gualdani2024) and dynamic settings (Fershtman2012; Aguirregabiria2020; \citealp*{Asker2020}). The concern is particularly acute in the empirical literature on dynamic games, where most works adopt a specific assumption that limits the role of players' private information in strategic interactions. With the exception of Pakes1995, which assumes complete information, we are not aware of empirical approaches that allow for alternative information structures. Achieving informationally robust identification by directly recovering the model primitives from equation (ref) is, however, infeasible: the set of possible information structures is vast and infinite-dimensional, and the task is further complicated by the inherent challenges of estimating dynamic models of strategic interaction.
In this paper, we develop a tractable empirical framework for analyzing dynamic discrete games while leaving the within-stage information structure unknown to the analyst. We assume that the researcher knows only the minimal information available to players but remains agnostic about the possibility that the players may have access to additional signals.\footnote{This assumption restricts the set of possible information structure $\mathcal{S}$ in (ref).} The framework typically delivers a set of partially identified parameters that are robust to misspecification of the within-stage information structure. It can be used to construct confidence sets for model parameters that are robust to informational assumptions, thereby assessing the sensitivity of parameter estimates to the researcher's assumptions about players' information.
The main innovation of this paper is the development of Markov correlated equilibrium, a dynamic (Markovian) analog of Bayes correlated equilibrium Bergemann2016. We show that Markov correlated equilibrium can capture the set of Markov perfect equilibrium predictions (the joint distribution of actions and states) that can arise when the players might observe more signals than assumed by the analyst. We establish the informational robustness property of Markov correlated equilibrium by formulating Markov perfect equilibrium as the Bayes Nash equilibrium of the associated reduced one-shot game via the one-shot deviation principle and invoking Theorem 1 of Bergemann2016.
We build on the theory of Markov correlated equilibrium to propose computationally tractable approaches for estimation and inference. Roughly speaking, we prove
where $\Theta_I^\mathit{MCE}$ is the identified set of parameters after assuming that a Markov correlated equilibrium generated the data, and $\Theta_I^\mathit{MPE}(\tilde{S})$ is the Markov perfect equilibrium identified set when the information structure is assumed to be $\tilde{S}$. Directly inverting (ref) requires constructing the right-hand side of (ref), which is computationally infeasible when $\mathcal{S}$ is large. However, we show that constructing the left-hand side of (ref) is computationally tractable. In sum, treating the data as if they were generated by a Markov correlated equilibrium enables an informationally robust econometric analysis. The identified set (ref) is usually non-singleton. We propose a computationally attractive strategy for estimation and inference by formulating the estimation problem as mathematical programs with equilibrium constraints (MPEC) su2012constrained and applying the inference strategy of HorowitzLee2022 by nesting the confidence set for the conditional choice probabilities into the MPEC problem.\footnote{The standard estimation strategies for dynamic games, such as the nested fixed-point algorithm of rust1987optimal or the conditional choice probability inversion of hotz1993conditional, are not applicable.}
We apply our framework to study the dynamic entry and exit decisions of the two largest coffee chains in the US, Starbucks and Dunkin'. As a benchmark, we first estimate the structural parameters under Markov perfect equilibrium, assuming that each player observes their own idiosyncratic payoff shock but not the opponent's, which corresponds to the standard “incomplete information” assumption in the literature. We then employ our Markov correlated equilibrium framework to estimate an identified set that captures all possible parameters that could generate the data under Markov perfect equilibrium when the players may additionally observe information about the opponent's payoff shock. We also estimate an identified set that allows for the possibility that the players may not even observe their payoff shock; in this case, $\mathcal{S}$ spans the entire set of information structures, and the identified set is fully robust to any potential misspecification of the information structure.
Comparison between the estimates from the standard incomplete information Markov perfect equilibrium assumption and our informationally robust Markov correlated equilibrium assumption suggests that the standard incomplete information assumption may be too strong. The estimates from the standard approach suggest that the presence of an opponent has a positive spillover effect on profitability. On the other hand, the Markov correlated equilibrium identified sets do not rule out both positive and negative spillover effects.
We conduct a counterfactual experiment in which we reduce the firms' entry costs by 10% and simulate the evolution of the number of active firms, averaged over all markets. The possible outcomes range from 0 (no firms active in any market) to 2 (both firms active in all markets). To illustrate the range of counterfactual predictions under our framework, we consider two extreme scenarios. The low (resp. high) scenario considers the expected number of active firms in equilibrium using the lower (resp. upper) bounds of the 95% CIs of the competitive effects parameters, while holding all other parameters at their point estimates.
The standard Markov perfect equilibrium approach predicts that reducing entry costs would raise the average number of active firms from the baseline of 0.37 to between 0.55 (low) and 0.58 (high) by the end of the sample period. Allowing for the possibility that firms observe not only their own but also their rival's payoff shock expands the range to 0.35--0.87 under our Markov correlated equilibrium framework. The range becomes even wider, 0.26--1.32, once we incorporate the uncertainty of the remaining structural parameters, and it grows further if we allow for the possibility that firms may not observe (or may only partially observe) their own payoff shocks.
These results show that our framework can generate policy implications that are both robust to informational assumptions and informative, though the degree of informativeness depends on how flexibly one models the information structure and accounts for parameter uncertainty. At the same time, the robustness provided by Markov correlated equilibrium may admit too many possibilities to be empirically useful, a limitation also noted by koh2023stable, Magnolfi2023, and Gualdani2024 in their application of Bayes correlated equilibrium.
Our work advances the literature on the empirical analysis of game-theoretic models in several ways. Our work is most closely related to the econometric frameworks that pursue robustness to misspecification on information structures (grieco2014discrete; pakes2015moment; \citealp*{Syrgkanis2021}; koh2023stable; Magnolfi2023; canen2024decomposition; Gualdani2024; \citealp*{han2024testing}). Building on this literature, we develop a novel framework for estimating a class of dynamic games. More broadly, our work contributes to the literature on tractable econometric methods for partially identified game-theoretic models under weak assumptions on equilibrium selection Tamer2003, ciliberto2009market, beresteanu2011sharp, galichon2011set.
We contribute to the dynamic games literature by developing a new solution concept with attractive computational properties. Markov correlated equilibrium extends Bayes correlated equilibrium of Bergemann2013, Bergemann2016 to dynamic Markovian games in the tradition of maskin2001markov. While related notions of correlated equilibrium in dynamic games have been studied Nowak1992, Duffie1994, solan2001characterization, solan2002correlated, mailath2006repeated, von2008extensive, He2017, forges2020correlated, our formulation is distinct in extending Bayes correlated equilibrium to dynamic stochastic Markov games with incomplete information. Importantly, our solution concept is tailored to the class of dynamic games that serve as the workhorse for empirical studies of dynamic competition aguirregabiria2021dynamic.
Our framework maintains tractability by focusing on robustness to within-stage information assumptions. Although one could allow for across-stage incomplete information and dynamic belief updating (e.g., Doval2020; makris2023information), such extensions are typically infeasible in empirical work. Persistent information asymmetries render estimation intractable Fershtman2012, and broadening the solution concept would further enlarge the identified set, reducing its empirical informativeness. Our results already demonstrate that Markov correlated equilibrium admits a wide range of outcomes, which can significantly weaken empirical content, as observed by koh2023stable, Magnolfi2023, and Gualdani2024 in static settings. For this reason, we restrict attention to the informational assumptions most relevant to existing empirical frameworks, rather than pursuing broader forms of robustness.
Finally, this paper contributes to the empirical literature on dynamic competition and spatial expansion of retail chains \citep*{Aguirregabiria2007, jia2008happens, holmes2011diffusion, ellickson2013estimating, nishida2015estimating, aguirregabiria2016empirical, arcidiacono2016estimation, yang2020learning, beresteanu2024dynamics}. We examine a novel application: the dynamic expansion of the two largest coffee chains in the US, Starbucks and Dunkin'. The study most closely related to ours is fang2024measuring, who analyze entry-deterrence motives of the four largest coffee chains in Toronto, Canada, using a dynamic oligopoly framework.
The rest of the paper is organized as follows. In Section (ref), we define Markov correlated equilibrium and establish a connection between Markov correlated equilibria and Markov perfect equilibria in a class of dynamic games and discuss its properties. In Section (ref), we introduce an econometric model and discuss informationally robust identification. Section (ref) proposes strategies for computing the identified set. Section (ref) discusses our approach to inference. Section (ref) applies our framework to the dynamic entry/exit game by Starbucks and Dunkin' in the US. Section (ref) concludes. All proofs are in Appendix (ref).
We consider a class of dynamic Markovian games that has been used as a standard framework for empirical works. We introduce the concept of Markov correlated equilibrium for informationally robust analysis and discuss its properties.
Let $t=1,2,...,\infty$ denote discrete time. A stationary dynamic Markov game of incomplete information is defined by a pair of basic game $G$ and information structure $S$. A basic game $G = \langle \mathcal{I}, (\mathcal{A}_i, u_i)_{i \in \mathcal{I}}, \mathcal{X}, \mathcal{E}, \psi, f, \delta \rangle$ specifies the payoff-relevant primitives of the model: $i \in \mathcal{I} = \{1, 2, ..., I \}$ indexes the players; $a_{it} \in \mathcal{A}_i$ denotes player $i$'s action at time $t$, where $\mathcal{A}_i$ is a finite set of actions available to player $i$; $a_t = (a_{1t}, ..., a_{It}) \in \mathcal{A} \equiv \times_{i \in \mathcal{I}} \mathcal{A}_i$ denotes an action profile; $x_t \in \mathcal{X}$ denotes a state variable that is publicly observed by the players, and $\mathcal{X}$ is assumed to be finite; $\varepsilon_t \in \mathcal{E}$ denotes a latent state variable that is not directly observed by the players, and $\mathcal{E}$ is assumed to be finite\footnote{The assumption of a finite state space is non-substantive: it simplifies notation and matches the discretization used in estimation. The theory readily extends to a more general state space; see \citet*{Syrgkanis2021} and Magnolfi2023 for the case of Bayes correlated equilibrium.}; $\psi_{\varepsilon_t \vert x_t}$ represents the players' prior on the probability of $\varepsilon_t$ conditional on $x_t$; $f_{x_{t+1} \vert a_t, x_t, \varepsilon_t}$ specifies the probability of transitioning to state $x_{t+1}$ conditional on $(a_t, x_t, \varepsilon_t)$; $u_i: \mathcal{A} \times \mathcal{X}\times \mathcal{E} \to \mathbb{R}$ is the flow payoff function of player $i$; $\delta \in [0, 1)$ is the players' discount factor.
An information structure $S = \left( (\mathcal{T}_i)_{i\in \mathcal{I}} , \pi \right)$ specifies the information-relevant primitives; $\tau_{it} \in \mathcal{T}_i$ denotes player $i$'s private signal at period $t$, where $\mathcal{T}_i$ represents a finite set of private signals; $\tau_t = (\tau_{1t}, ..., \tau_{It}) \in \mathcal{T} \equiv \times_{i \in \mathcal{I}} \mathcal{T}_i$ is a signal profile; $\pi : \mathcal{X} \times \mathcal{E} \to \Delta (\mathcal{T})$ is a signal distribution that maps the state of the world to a signal profile. Note that players' signals are allowed to be correlated. The interpretation is that each player $i$ does not directly observe the latent state $\varepsilon_t$ but receives a private signal $\tau_{it}$ whose informativeness about $\varepsilon_t$ depends on $\pi$.
The game $(G,S)$ is common knowledge to the players. The model proceeds as follows. At the beginning of period $t$, the common knowledge state $x_t \in \mathcal{X}$ is given and publicly observed. Conditional on $x_t$, the latent state $\varepsilon_t \in \mathcal{E}$ is drawn from the probability distribution $\psi_{\varepsilon_t \vert x_t}$. Next, a profile of private signals $\tau_t = (\tau_{1t}, ..., \tau_{It}) \in \mathcal{T}$ is drawn from the signal distribution $\pi_{\tau_t \vert x_t, \varepsilon_t}$. At this point, each player $i$ observes $(x_t, \tau_{it})$ and uses Bayes' rule to infer $\varepsilon_t$. Then, the players simultaneously choose actions $a_{it}$, $i=1,...,I$, and each player $i$ receives period $t$ payoff $u_i(a_t, x_t, \varepsilon_t) \in \mathbb{R}$. Finally, the observable state $x_t$ transitions to $x_{t+1}$ via the probability kernel $f_{x_{t+1} \vert a_t, x_t, \varepsilon_t}$. Given $x_{t+1}$, period $t+1$ begins.
Note that the primitives imply that the transition probability of state variables factors as $\mathrm{Pr}(x_{t+1}, \varepsilon_{t+1} \vert a_t, x_t, \varepsilon_t) = \psi_{\varepsilon_{t+1} \vert x_{t+1}} f_{x_{t+1} \vert a_t, x_t, \varepsilon_t}$. Assuming that the latent variables are generated independently of the previous-period states is standard in the empirical literature.\footnote{When latent variables are correlated over time, empirical analysis quickly becomes intractable, as computation of perfect Bayesian equilibria requires tracking the full history of players' information and updating beliefs at every contingency Fershtman2012.} The assumption also plays a vital role in this paper as it simplifies the characterization of correlated equilibria in dynamic settings.
The players are rational and forward-looking. In period $t$, each player $i$ chooses action plans to maximize the expected discounted sum of intertemporal payoffs
The conditioning on $(x_t, \tau_{it})$ reflects the assumption that $i$ observes the public state $x_t$ and her signal $\tau_{it}$ before choosing period $t$ action. Since the environment is stationary, we suppress time subscripts unless necessary.
A Markov strategy of player $i$ is a mapping $\beta_i: \mathcal{X} \times \mathcal{T}_i \to \Delta (\mathcal{A}_i)$ that specifies a probability distribution over actions at each realization of observable state and private signal.\footnote{ While the standard Markov strategy assumption only allows players to condition on payoff-relevant state maskin2001markov, our definition allows players' strategy to depend on signals that may be unrelated to the fundamentals (e.g., a public correlation device under complete information), which is a consequence of considering arbitrary information structures. However, our assumptions align with the spirit of maskin2001markov because we rule out arbitrary strategies by requiring actions to depend only on signals of a memoryless latent state. Moreover, we are unaware of a treatment for the construction of the maximally coarse partition of the action space in an incomplete information setup. } A Markov strategy profile $\beta = (\beta_1, ..., \beta_I)$ is a Markov perfect equilibrium if it is a subgame perfect equilibrium.\footnote{This definition is standard in the literature. Beliefs are determined by the signal distribution $\pi$, conditional on publicly observed states. With public transitions and common knowledge of probabilities, beliefs need not enter the equilibrium definition. Subgames are well-defined under Markov strategies.}
To leverage the results in static games to a dynamic environment, it is useful to characterize the Markov perfect equilibrium conditions using the one-shot deviation principle.\footnote{According to the one-shot deviation principle, a strategy profile is subgame perfect precisely when no player can gain from deviating at a single information set. For infinite-horizon games, this principle holds under the “continuity at infinity” condition, which ensures that far-off events matter little fudenberg1991game. Dynamic discrete game models satisfy this condition because payoffs are discounted sums of uniformly bounded expected stage payoffs, even if shocks themselves have unbounded support.} The one-shot deviation principle formulation has two advantages. First, we can leverage results established in static environments as it translates a Markov perfect equilibrium as a Bayes-Nash equilibrium of the associated normal-form game. Second, it facilitates equilibrium computation during econometric analysis.
Suppose a strategy profile $\beta = (\beta_1, ..., \beta_I)$ is given. Define the ex-ante value function $V_i^\beta(x) \in \mathbb{R}$ as the expected payoff to player $i$ at state $x$ prior to the realization of $\varepsilon$ when all players follow the prescriptions in $\beta$. For each $i \in \mathcal{I}$, $V_i^\beta \in \mathbb{R}^{\vert \mathcal{X} \vert}$ is the unique solution to
where $\beta_{a \vert x, \tau} \equiv \prod_{i=1}^I \beta_i (a_i \vert x, \tau_i)$ denotes the conditional distribution over action profiles induced by the strategy profile $\beta$. Next, define the outcome-specific payoff function given strategy profile $\beta$ as
which represents the continuation payoff to player $i$ when $(a,x,\varepsilon)$ is realized today, and all players follow the prescriptions in $\beta$ from tomorrow onward.
A strategy profile $\beta$ in game $(G,S) $ induces a reduced normal-form basic game $G^\beta = \langle \mathcal{I}, (\mathcal{A}_i, v_i^\beta )_{i \in \mathcal{I}}, \mathcal{X}, \mathcal{E}, \psi \rangle$. Then $(G^\beta, S)$ describes a “static" game in which player $i$'s “static” payoff function is given by $v_i^\beta : \mathcal{A} \times \mathcal{X} \times \mathcal{E} \to \mathbb{R}$. The following lemma allows us to import the results in static environments into dynamic environments.
We introduce a dynamic analog of Bayes correlated equilibrium for dynamic discrete games. A stationary Markov decision rule in $(G,S)$ is a mapping
which specifies a probability distribution over action profiles at each realization of state and signals. It is instructive to think of $\sigma$ as a recommendation strategy of an omniscient mediator who observes $(x, \varepsilon, \tau)$ and privately recommends action to each player. Suppose the mediator commits to a recommendation strategy $\sigma$ and announces it to the players. After the state and players' signal $(x, \varepsilon, \tau)$ are realized, an action profile $a=(a_1,...,a_I)$ is drawn from the probability distribution $\sigma_{a \vert x, \varepsilon, \tau}$, and each $a_i$ is privately recommended to each player $i$. Each player $i$, having observed $(x, \tau_i, a_i)$, decides whether to obey (play $a_i$) or not (deviate to $a_i' \neq a_i$). If the decision rule $\sigma$ is such that the players are always obedient, we call $\sigma$ a Markov correlated equilibrium of $(G,S)$.
We formalize the equilibrium condition as follows. Let $V_i^\sigma \in \mathbb{R}^{\vert \mathcal{X} \vert}$ denote the vector of ex-ante value functions induced by $\sigma$, obtained as a unique solution to
Define the outcome-specific value function associated with $\sigma$ as
which represents the payoff to player $i$ if $(a,x,\varepsilon)$ is realized today and $\sigma$ determines the players' actions in the future. Let $\mathbb{E}^\sigma [ v_i^\sigma (a_i', a_{-i}, x, \varepsilon) \vert x, \tau_i, a_i]$ be the expected payoff to player $i$ from choosing $a_i'$ when $i$ observes $(x,\tau_i)$ and receives recommendation $a_i$. The following definition states that $\sigma$ is a Markov correlated equilibrium of $(G,S)$ if the players do not deviate from the mediator's recommendations.
Since
canceling out the denominator---which is constant across all possible realizations of $(\varepsilon, \tau_{-i}, a_{-i})$---lets us rewrite the obedience condition (ref) as, for each $i$,
Equation (ref) serves as the operational definition of Markov correlated equilibrium in our econometric analysis.
A Markov correlated equilibrium always exists for any signal distribution $\pi$. Nowak1992 establishes the existence of a stationary correlated equilibrium in a discounted stochastic game with complete information under standard regularity conditions. We will show in the following sections that the set of Markov correlated equilibria under arbitrary information structure always includes a complete information stationary correlated equilibrium.\footnote{Existence results for Markov perfect equilibrium under incomplete information do not necessarily imply the existence of a Markov correlated equilibrium for certain information structures. For this reason, we rely on the result of Nowak1992.} Thus, the set of Markov correlated equilibria is always non-empty.
As in Section (ref), a decision rule $\sigma$ in game $(G,S)$ induces a reduced normal-form basic game $G^\sigma = \langle \mathcal{I}, (\mathcal{A}_i, v_i^\sigma)_{i\in \mathcal{I}}, \mathcal{X}, \mathcal{E}, \psi \rangle$. Then (ref) shows that if $\sigma$ is a Markov correlated equilibrium of $(G,S)$, it is a Bayes correlated equilibrium of $(G^\sigma,S)$.
When players fully discount the future ($\delta = 0$), a Markov correlated equilibrium collapses to a Bayes correlated equilibrium of the stage game: for each $i$,
Thus, Markov correlated equilibrium is a proper dynamic analog of Bayes correlated equilibrium.
Unfortunately, contrary to its static analog, the Markov correlated equilibrium conditions are not linear with respect to the decision rule, which creates two challenges that are absent in the static case. First, Markov correlated equilibrium is more difficult to compute. While computing a Bayes correlated equilibrium amounts to solving a linear program, computing a Markov correlated equilibrium generally requires solving a non-convex program. Second, the set of Bayes correlated equilibria is convex, but the set of Markov correlated equilibria is generally non-convex. The convexity of the set of Bayes correlated equilibria implies that a mixture of multiple equilibria is observationally equivalent to a single equilibrium; \citet*{Syrgkanis2021} and Magnolfi2023 use this property to stay agnostic about the true equilibrium selection rule in the data generating process. However, since Markov correlated equilibrium does not share this property, we assume that a single equilibrium is selected.
We establish the informational robustness property of Markov correlated equilibrium. Let an equilibrium prediction refer to the joint distribution on actions, states, and signals that are implied by either an equilibrium strategy profile or decision rule. We prove the following claim.
In the empirical application, we consider a scenario where the analyst knows that each firm observes its firm-specific payoff shock but cannot rule out the possibility that they receive some signals about opponents' shocks. Claim (ref) says that the analyst can capture the set of all predictions that can arise in such a scenario by resorting to Markov correlated equilibrium. Thus, as in Bayes correlated equilibrium, Markov correlated equilibrium attains a form of informational robustness that allows the analyst to be agnostic about the players observing additional signals after specifying players' minimal information. We formalize Claim (ref) by following Bergemann2016's framework and connecting Markov correlated equilibrium to Bayes correlated equilibrium via the one-shot deviation principle.
First, we introduce a partial order on the set of information structures using the notion of expansion as in Bergemann2016. Let $\omega \in \Omega \equiv \mathcal{X}\times \mathcal{E}$ for the sake of exposition.
Intuitively, if $S^* \succsim_E S$, the players receive extra signals in $S^*$ than they do in $S$. Specifically, complete information (players observe the latent state $\varepsilon$ almost surely) is an expansion of an arbitrary information structure.
Next, we claim that if a strategy profile induces the same joint distribution over states and actions as a decision rule, the two induce the same reduced normal-form games. Let $S^*$ be an expansion of $S$. A strategy profile $\beta$ in $(G,S^*)$ induces a decision rule $\sigma$ for $(G,S)$ if
The strategy profile $\beta$ in a game with more signals induces the same joint distribution over actions, states, and signals as the decision rule $\sigma$ in a game with fewer signals.
Lemma (ref) is intuitive: if $\beta$ induces $\sigma$, then $\beta$ and $\sigma$ induce identical distributions over action profiles at each state, so the associated ex-ante value functions must be identical to each other, i.e., $V_i^\beta = V_i^\sigma$. Then, the outcome-specific payoff functions are also identical to each other, i.e., $v_i^\beta = v_i^\sigma$, making all primitives of the basic games identical to each other.
Finally, a solution concept generates a prediction defined as a probability distribution over actions at each state and signal. Let $\mathcal{P}_{a \vert x,\varepsilon,\tau}^{\mathit{MPE}}\left(G,S\right)$ represent the set of predictions that can be induced by a Markov perfect equilibrium in $\left(G,S\right)$.\footnote{We slightly abuse the notation and use $\mathcal{P}_{a\vert x, \varepsilon, \tau}^{\mathit{MPE}} (G,S)$ to denote a set of conditional distributions over $\mathcal{A}$ defined at each $(x,\varepsilon,\tau)$.} Let $\mathcal{P}_{a\vert x,\varepsilon,\tau}^{\mathit{MCE}}\left(G,S\right)$ be defined similarly for a Markov correlated equilibrium.
We are ready to state our main theorem. Suppose the analyst knows that the players observe signals specified by $S$ but does not know whether the players have access to extra signals. What is the set of feasible Markov perfect equilibrium predictions? The following theorem says that the set of Markov correlated equilibria of $(G,S)$ captures all Markov perfect equilibrium predictions when the players can observe more signals than in $S$.
Theorem (ref) is attractive because $\mathcal{P}_{a\vert x,\varepsilon,\tau}^{\mathit{MCE}}\left(G,S\right)$ is far easier to characterize and compute than $\bigcup_{S^{*}\succsim_{E}S}\mathcal{P}_{a\vert x,\varepsilon,\tau}^{\mathit{MPE}}$; the former does not require searching over the set of information structures.\footnote{makris2023information obtain a related result in the context of multistage games. However, since their focus is on finite-stage games and they allow for varying information structures across stages, our result is not a direct corollary of theirs. Moreover, applying their framework in empirical settings would generally be infeasible, as it requires keeping track of the entire history.} Note that when $\delta = 0$, Theorem (ref) reduces to
which is analogous to Bergemann2016 Theorem 1.
In empirical works, it is common to assume that the analyst observes the conditional choice probabilities that represent the probability of each action profile at each state that is observable to the econometrician. Thus, it is useful to characterize the implications of Theorem (ref) in terms of conditional choice probabilities. Let $\mathcal{P}_{a\vert x}^{\mathit{MPE}}\left(G,S\right)$ denote the set of feasible conditional choice probabilities that can arise under a Markov perfect equilibrium of $\left(G,S\right)$. Let $\mathcal{P}_{a\vert x}^{\mathit{MCE}}\left(G,S\right)$ be defined similarly.
In Section (ref), we discuss how results from Theorem (ref) and Corollary (ref) can be applied to derive results on the identification of the model parameters.
As in the case of Bayes correlated equilibrium, if players observe more signals, the set of Markov correlated equilibria becomes smaller. In other words, more information implies a smaller set of Markov correlated equilibria.
We continue Example (ref) to illustrate the Markov correlated equilibrium identified set. The true parameters in (ref) are set to $(\pi^m,\pi^d,c,\kappa)=(1.2,-1.2,-0.2,0.1)$, and we use the discount factor $\delta = 0.9$. We generate data by finding a Markov perfect equilibrium under the standard assumption that $\varepsilon_i$ is private information to player $i$. pesendorfer2008asymptotic finds five equilibria, two of which are symmetric to other equilibria. We report the conditional choice probabilities of equilibria (i), (ii), and (iii) of pesendorfer2008asymptotic in Table (ref).\footnote{We use dearing2024efficient's MATLAB code to generate the equilibrium conditional choice probabilities.}
To characterize the bounds on the game outcomes that can arise in a Markov correlated equilibrium, we compute the Markov correlated equilibria that maximize and minimize the expected number of active firms.\footnote{The procedure is similar to the counterfactual equilibrium calculations in the empirical application detailed in Online Appendix (ref).} Using the associated conditional choice probabilities, we simulate the number of active firms over $T=6$ periods, assuming that the initial state has no incumbent, i.e., $x_1 = (0,0)$.
Figure (ref) plots the simulated average number of active firms for $T=6$ periods. Note that the outcomes of pesendorfer2008asymptotic's three Markov perfect equilibria are bounded by those of the Markov correlated equilibria. The Markov correlated equilibrium bounds characterize the sharp bounds on the Markov perfect equilibrium outcomes that can arise when players can receive information about the opponent's payoff shock.
In this section, we specialize the model for econometric analysis and discuss informationally robust identification in dynamic discrete games using Markov correlated equilibrium.
To facilitate econometric analyses, we impose the following assumptions on the basic game primitives. As commonly assumed in empirical works, we let $\varepsilon \equiv (\varepsilon_1, \varepsilon_2, ..., \varepsilon_I)$ where $\varepsilon_i \in \mathcal{E}_i$ only enters player $i$'s payoff and $f_{x'\vert a,x,\varepsilon}=f_{x'\vert a,x}$, i.e., the transition probability of public state $x$ is independent of latent state $\varepsilon$. We assume that the payoff functions and the prior distribution are parameterized by a finite-dimensional vector $\theta$ so that $u_{i}=u_{i}^{\theta}$ and $\psi=\psi^{\theta}$.
The econometrician observes data $$\left\{ a_{m,t},x_{m,t}:\ m=1,2,...,M,\ t=1,2,...,T\right\},$$ which have players' actions $a_{m,t}$ and common knowledge state variables $x_{m,t}$ across $M$ independent markets over $T$ periods. We assume that $M$ is large and that the conditional choice probabilities $\phi:\mathcal{X}\to\Delta\left(\mathcal{A}\right)$ and the transitional probability function $f:\mathcal{A}\times\mathcal{X}\to\Delta\left(\mathcal{X}\right)$ can be non-parametrically identified from the data. Finally, we assume that the common discount factor $\delta \in [0,1)$ is known to the econometrician. In sum, we treat the basic game $G^\theta = \langle \mathcal{I}, (\mathcal{A}_i, u_i^\theta)_{i\in \mathcal{I}}, \mathcal{X}, \mathcal{E}, \psi^\theta, f, \delta \rangle$ as known to the econometrician up to $\theta$.
We summarize the baseline assumptions for econometric analysis as follows.
The only “unconventional" assumption is that $\mathcal{E}$ is finite. Although unnecessary for the identification arguments, the discrete support assumption is necessary for feasible estimation and makes the connection to the previous section straightforward. Other econometric papers that use Bayes correlated equilibrium also discretize the state space for feasible estimation (\citealp*{Syrgkanis2021}; Magnolfi2023; Gualdani2024). The player-specific payoff shock assumption, which is a conventional assumption, makes the role of the latent state clear, but it is also not required for our identification arguments.
Given a game $(G^\theta,S)$, a solution concept determines a set of feasible predictions, which in turn induces a set of feasible conditional choice probabilities. A standard econometric approach is to assume that the information structure $S$ is known and identify the model parameters by inverting the mapping from the model parameters to the observed conditional choice probability vector $\phi$.
In words, a candidate parameter $\theta$ enters the identified set if the observed conditional choice probabilities $\phi$ can be rationalized by some equilibrium (defined by the solution concept) of the model.
We consider a scenario where a Markov perfect equilibrium generates the data, but the true information structure is unknown to the analyst; misspecifying the information structure leads to biased parameter estimates. We assume that the analyst only knows that the players minimally observe signals in $S$ but may observe more. For instance, as assumed in Magnolfi2023, the analyst might know that each player $i$ observes at least $\varepsilon_{i}$ but does not know whether they can receive extra information about opponents' payoff shocks.
It is well-understood that working directly with Assumption (ref) is computationally infeasible; when the analyst knows $S$ but does not know the true information structure $S^0$, the analyst has to consider the set of all information structures that are expansions of $S$, but the set is too large. We can make the identification problem computationally feasible by replacing Assumption (ref) with the following.
Theorem (ref) says that an econometrician who does not know the true information structure underlying the data-generating process can proceed by treating the data as if they were generated by a Markov correlated equilibrium. \citet*{Syrgkanis2021}, Magnolfi2023, koh2023stable, and Gualdani2024 use similar results in static settings. Theorem (ref) extends the results to a class of dynamic games. Note that a Markov correlated equilibrium identified set always includes the parameters that can be obtained under the complete information Markov perfect equilibrium assumption with mixed strategies Pakes1995, Doraszelski2010.
\paragraph{Identified Set Shrinks with Stronger Informational Assumptions} The Markov correlated equilibrium identified set depends on the baseline information structure set by the analyst, which specifies what players minimally observe. Theorem (ref) has shown that a stronger assumption on players' information leads to a tighter set of predictions. The following translates this intuition to identified sets.
Theorem (ref) implies a trade-off between informational robustness and the tightness of the identified set. Using weaker assumptions on information is more robust but increases the size of the identified set. The econometrician obtains the tightest identified set when players' baseline information is complete; the econometrician obtains the loosest identified set when players minimally observe nothing about $\varepsilon$.
\paragraph{Markov Correlated Equilibrium is Not Robust to Multiple Equilibria Issue} In the static case, assuming that a single Bayes correlated equilibrium generates the data is innocuous because any mixture of Bayes correlated equilibria is a Bayes correlated equilibrium. However, this property does not carry over to the dynamic case because a mixture of Markov correlated equilibria need not be a Markov correlated equilibrium. This occurs because the Bayes correlated equilibrium obedience condition is linear in the decision rule while the Markov correlated equilibrium obedience condition is nonlinear (non-convex) in the decision rule. Thus, although we remain agnostic about the equilibrium selection rule, we assume that the data are generated by a single equilibrium. Identification with data generated by multiple equilibria is possible (e.g., by using \citet*{beresteanu2011sharp}), but far more computationally challenging.\footnote{Our framework already incorporates public randomization that is independent across stages. However, because of the relationship between stage-game payoffs and value functions induced by the Markov structure, public randomization does not enable us to convexify the objective set of distributions.}
\paragraph{Excluded Variables with Large Support Ensure Point-Identification} While the tightness of the identified set is largely an empirical question, the identified set can shrink substantially when excluded variables have large support. In a scenario where players know their own payoff shock, Magnolfi2023 prove that players' payoff parameters are point-identified under Bayes correlated equilibrium when there are player-specific excluded variables with large support.\footnote{Parameter governing the correlation of players' payoff types may not be point-identified.} Consider, for example, a two-player entry game. Driving a time-persistent variable that enters only player $j$'s payoff to either positive or negative infinity ensures that the player always stays in or stays out, rendering the other player's decision problem a single-agent binary choice problem. The econometrician can use the single-agent data to identify the payoff parameters, including the competitive effects parameters Tamer2003. The same intuition holds in the dynamic case: in markets where one player always stays in or out, the other player solves a single-agent dynamic discrete choice problem while taking the other player's action as fixed. The players' payoff parameters can be identified and estimated using the single-agent dynamic discrete choice data if our interest lies in the case where players know their own payoff shock.\footnote{If we are interested in the case where players may not have complete knowledge about their own shock, we will need to extend Theorem 1 of Bergemann2016 to a single-agent dynamic discrete choice setup.}
Following pesendorfer2008asymptotic, we assume that the econometrician knows the values of the scrap value $\kappa=0.1$ and the discount factor $\delta=0.9$. However, instead of assuming that the researcher knows the information structure exactly, we assume that the researcher only knows that $i$ minimally observes $\varepsilon_i$ and leaves open the possibility that $i$ receives extra signals about $\varepsilon_{-i}$.
Table (ref) reports the projection intervals of the Markov correlated equilibrium identified sets obtained under each equilibrium. In all cases, the Markov correlated equilibrium identified set contains the true parameter vector, as expected. However, the size of the identified set can vary depending on which equilibrium generated data. We find that the Markov correlated equilibrium identified set obtained from equilibrium (i)---which is the only nested pseudo-likelihood-stable equilibrium and generates highly asymmetric choice probabilities across players---is the tightest.\footnote{The nested pseudo-likelihood-stability has been the subject of investigation in the dynamic games estimation literature (see dearing2024efficient for a review). Equilibrium (i) is nested pseudo-likelihood-stable while equilibria (ii) and (iii) are not. Our Markov correlated equilibrium estimator is not sensitive to the nested pseudo-likelihood-stability of the underlying equilibrium.} The identified sets under equilibria (ii) and (iii) are indistinguishable because the equilibrium conditional choice probabilities are similar to each other.
Figure (ref) visualizes the results by plotting the convex hull of the sharp identified set corresponding to equilibrium (ii). In each subfigure, the shaded area represents the convex hull of the projected identified sets, and the star represents the location of the true parameters. As expected, the identified set is non-singleton and contains the true parameter.
Our numerical example shows that a strong assumption on information is far from innocuous: removing arbitrary assumptions on who observes what may result in a significant loss of identifying power. Robustness to misspecification on players' information structure expands the identified set, but the degree to which the identified set expands can vary depending on which equilibrium is selected in the data generating process.
We propose computationally feasible estimation strategies by formulating a mathematical program with equilibrium constraints \`{a} la su2012constrained. We assume the econometrician knows the true conditional choice probability vector $\phi$ (we relax this assumption in the next section). We also assume that the baseline information structure $S$ is fixed and denote the sharp identified set as $\Theta_I \equiv \Theta_I^{MCE}(S)$.
For a candidate parameter $\theta$, we have $\theta \in \Theta_I$ if and only if there exists a decision rule $\sigma$ that satisfies the Markov correlated equilibrium obedience conditions and induces the observed conditional choice probabilities. We characterize the Markov correlated equilibrium as a set of constraints that the econometrician can plug into optimization software.
Constraints (ref) mean that $\sigma$ is a proper conditional probability distribution, as required by the definition of a decision rule; (ref) requires that $\sigma$ induces the observed conditional choice probabilities $\phi$; (ref) is the Markov correlated equilibrium obedience condition; (ref) is the ex-ante value function equation described in (ref).\footnote{Note (ref) also allows us to express each $V_i$ as a closed-form function of $\sigma$. This is similar to expressing the ex-ante value function as a closed-form function of conditional choice probabilities in Aguirregabiria2007.} Note that $\psi$, $\pi$, $\phi$, $\delta$, and $f$ are treated as known objects in the mathematical program.\footnote{These conditions bear some resemblance to those in abreu1990toward and athey2004collusion. Their approach characterizes the set of continuation payoffs using such conditions, raising the question of whether it could also apply in our setting. However, the Markov structure in our framework imposes a tight restriction between stage-game and continuation payoffs, which renders their approach inapplicable.}
\paragraph{Fully Robust Identified Set} In the special case where the analyst sets the baseline information structure to players having no minimal signals, $S^{null}$, the corresponding identified set---“fully robust identified set”---allows for the underlying information structure to be completely arbitrary. It turns out that determining whether a candidate parameter enters the fully robust identified set is computationally easier because the associated program is linear.
The expressions above reflect the assumption that the players receive no signals about the latent state $\varepsilon$. They only rely on their common prior to update their beliefs after observing private recommendations by the information designer. Since $\Theta_I^{MCE}(S) \subseteq \Theta_I^{MCE} (S^{null})$ for any information structure $S$ by Theorem (ref), the analyst can exploit computational tractability by first estimating the fully robust identified set and then analyzing cases with minimal information specified, as the search area can be restricted. However, the fully robust identified set may be quite large and empirically uninformative Magnolfi2023,koh2023stable.
The inference problem in this paper is characterized by a moment inequality model with a large number of moment inequalities and a high-dimensional nuisance parameter, making it difficult to apply many inference methods for partially identified models.\footnote{See canay2017practical and \citet*{canay2023user} for recent surveys of the inference methods for partially identified models.} Inference methods used by \citet*{Syrgkanis2021}, Magnolfi2023, and Gualdani2024 (e.g., \citet*{Chernozhukov2007}'s subsampling approach) are computationally challenging to implement in our setting because the Markov correlated equilibrium constraints are non-convex.
We propose a computationally attractive strategy for inference based on the key insights from HorowitzLee2022.\footnote{koh2023stable uses a similar approach in static discrete games settings.} Our approach is intended to minimize the need for repeatedly solving non-convex optimization problems. For simplicity, let us assume that the transition probability function is known to the researcher. However, an extension to the case where $f$ needs to be estimated is straightforward. Our approach is also applicable to Bayes correlated equilibrium.
In our setting, the sampling errors are only associated with the reduced-form parameters, namely the conditional choice probabilities (CCPs) $\phi$. If $\phi$ is known, the identified set can be estimated without statistical uncertainty. Thus, we propose controlling for the sampling uncertainty using a confidence set for the CCPs.\footnote{HorowitzLee2022 proposes methods for carrying out (non-asymptotic) inference when the partially identified parameters are characterized by constraints that involve unknown population means that can be estimated from data. Our approach follows their insights in treating the CCPs as unknown parameters that determine the partially identified set. A similar insight has been employed by kline2016bayesian, who propose a Bayesian approach.}
Suppose $\Phi_\alpha$ is a confidence set for the CCPs $\phi$ with the property that $\phi \in \Phi_\alpha $ with probability at least $1-\alpha$ as the sample size $n$ goes to infinity. Leading examples of $\Phi_\alpha$ are box constraints or ellipsoids. Let $\Theta_I(\phi)$ be the identified set when the CCPs are $\phi$. Define the confidence set as
If $\Phi_\alpha$ covers the true $\phi$ with high probability, then $\widehat{\Theta}_I^\alpha$ covers the true $\Theta_I$ with high probability. Furthermore, checking whether $\theta \in \widehat{\Theta}_I^\alpha$ amounts to solving a nonlinear program in Theorem (ref), but with $\phi$ as part of the optimization variables subject to the constraint $\phi \in \Phi_\alpha $.
We briefly discuss possible approaches for constructing $\Phi_\alpha$. While there are many possible approaches available, it is helpful to construct $\Phi_\alpha$ as linear constraints because they are computationally easy to handle. For example, one can construct simultaneous confidence intervals of the form
where the vectors $\hat{L}, \hat{U} \in \mathbb{R}^{\vert \mathcal{A} \vert \times \vert \mathcal{X} \vert}$ are determined by data. Examples include simultaneous confidence intervals for multinomial probabilities proposed by fitzpatrick1987quick or sup-$t$ band by montiel2019simultaneous; the former is easier to compute since the endpoints of the intervals can be computed in closed-form, but the latter can be tighter.
An alternative is to take a parametric approach.\footnote{With limited observations relative to covariate dimension, nonparametric CCP estimation may fail, as some cells contain no data. A flexible multinomial logit specification, though inconsistent under misspecification, offers lower variance and often outperforms nonparametric estimators in small samples. Aguirregabiria2007 show this approach performs well in a two-step pseudo–maximum likelihood framework.} Suppose $\phi = \phi(\gamma)$, where $\gamma$ is the parameter vector that governs the CCPs. One may estimate $\gamma$ and construct simultaneous confidence intervals of the form
where $\hat{L},\hat{U} \in \mathbb{R}^{\dim(\gamma)}$ can be computed using the asymptotic distribution of $\hat{\gamma}$. One may also consider replacing $\phi$ with its linear approximation $\phi(\gamma) \approx \phi(\hat{\gamma}) + \nabla \phi(\hat{\gamma})^\top (\gamma - \hat{\gamma})$ to further simplify the computation. In the identified set search, we also add the constraints defining $\phi$ as proper conditional distributions, i.e., $\phi_{a\vert x} \geq 0$, $\forall a,x$, and $\sum_{a} \phi_{a\vert x} = 1$, $\forall x$.
In our empirical application, we construct simultaneous confidence intervals using the sup-$t$ band approach, following montiel2019simultaneous.
We study a dynamic entry game by the two largest players in the US coffee chain industry, Starbucks and Dunkin', to illustrate the usefulness of our framework. The US coffee chain industry has grown significantly in recent decades, driven by increasing demand for specialty coffee and the rise of a culture emphasizing quality, convenience, and socialization. While companies compete for market share through pricing, product differentiation, marketing strategies, and in-store service, this market is highly concentrated. In particular, Starbucks and Dunkin' have significantly outpaced other competitors by a wide margin. Starbucks and Dunkin' accounted for 65% and 28% (resp. 55% and 35%) of total sales (resp. outlets) from the top 8 coffee chains in the US in 2019 technomic2019.\footnote{The third and fourth chains were Tim Hortons and Dutch Bros., which accounted for 2.2% and 1.7% (resp. 2.6% and 1.3%) of the sales (resp. outlets) out of the top 8 chains.}
Little is publicly known about how coffee chains predict opponents' strategic entry decisions. It is reasonable to expect that firms have good information about their stores' (potential) profitability. Still, less is known about how firms learn about their opponents' profitability. Resorting to the standard “incomplete” information assumption---firms observe their payoff shocks but observe nothing about their opponents' shocks---will likely result in a model misspecification. Whether this misspecification is an empirical concern depends on how much it may change the structural estimates and counterfactual outcomes. We apply our Markov correlated equilibrium framework to estimate the parameters and conduct a counterfactual experiment with weaker assumptions on players' information.
Our primary dataset is Data Axle's U.S. Historic Business Database, which has annual location information of all business establishments in the U.S. from 1997 to 2023. Figure (ref) shows that the number of Starbucks and Dunkin' stores in the US increased dramatically from 1997 to 2023. Figure (ref) illustrates how the firms dramatically expanded their geographic footprint over time.
We define markets as four-digit 2010 census tracts.\footnote{Four-digit census tracts are census-defined geographic units that are contiguous and strike a balance between county (which is too large) and six-digit census tracts (which is too small).} Our geographic market definition assumes that coffee chains compete in small geographic areas.\footnote{guler2018inferring shows that demand for Starbucks is highly localized. thomadsen2005effect shows evidence that only fast-food chains within approximately 0.5 miles---which amounts to 10 minutes walking distance---compete as close substitutes in California; it is likely that coffee consumers travel even less.} A firm is active in a market when we observe the firm's outlet in the market. We supplement the firm-location data with the demographic/geographic data from the Longitudinal Track Data Base \citep*{logan2014interpolating} and the neighborhood characteristics data from the National Neighborhood Data Archive. We limit our sample period to 2003--2017 to match the sample period of the neighborhood characteristics data.\footnote{We still use the business location data from 2018 to reduce the possibility of incorrectly determining firms not being active due to missing data. We consider a firm to have switched from active to not active in a market (i.e., exited the market) when we do not observe the firm's outlet in the market for the following five years.} We exclude the markets without any eating places during the sample period.
Table (ref) Panel A shows the active probabilities of each firm from 2018--2023 when the firm either did not have any outlets or had at least one outlet in the corresponding county from 2003--2017. The {\it ex-post} active probability in a county is much lower when the firm does not have any outlets in that county compared to when it does (1% vs. 20%). Thus, it is reasonable to consider that a firm does not take into account the competitor's situation if the competitor does not have any outlets in the county, and we define a firm as a potential entrant in each market if the firm had at least one outlet in the corresponding county during the sample period. We also use own-chain outlets in the county as firm-specific excluded variables that capture the network effects associated with economies of density guler2018inferring. Table (ref) Panel B demonstrates that the number of own-chain outlets in the county has a positive effect on the active probability in markets where both Starbucks and Dunkin' are potential entrants (two-player markets). In contrast, the number of competitor-chain outlets in the county affects the active probability little. We interpret the weak negative relationship as stemming from the firm's strategic behavior in response to the higher competitor's active probability.
Table (ref) reports the summary statistics of a balanced panel of $M=38,687$ markets observed over $T = 15$ years. Both Starbucks and Dunkin' are potential entrants in 79% of the markets, and 17% and 4% respectively have Starbucks and Dunkin' as a single potential entrant. The firms are active in 14% and 11% of the observations, while entries and exits are rare (1% and 0.2%, respectively), indicating a high correlation in the incumbency status. We assume the firms take the market characteristics as exogenous and time-invariant. For each market, we average characteristics over the observation period and treat them as fixed attributes. The key common characteristic is the number of eating places, which captures overall market attractiveness, as customer traffic tends to rise with the density of food establishments.
Our model follows the standard dynamic oligopoly entry/exit model with heterogeneous firms \citep*{Aguirregabiria2007,Pakes2007,pesendorfer2008asymptotic}. In each market $m$ and time period $t$, firm $i$ decides whether to operate in the market ($a_{imt}=1$) or not ($a_{imt}=0$). An active firm's flow profit is
Here, public state $x_{mt}$ includes three components, common time-invariant market characteristics $w_{m}$, firm-specific time-invariant excluded variables $(v_{im}, v_{jm})$, and incumbency status $(z_{imt} \equiv a_{im,t-1}, z_{jmt} \equiv a_{jm,t-1})$. $w_{m}$ includes a constant and log number of eating places, and $v_{im}$ is log number of own-chain outlets in the county. Parameters $\theta_{w}$, $\theta_{i,v}$, $\theta_{ec}$, and $\theta_{i,ce}$ represent coefficients on market characteristics, excluded variable, entry cost, and competitive effects, respectively. We allow the coefficients on the excluded variable and competitive effects to be asymmetric. $\varepsilon_{imt}$ is the idiosyncratic per-period payoff shock, which is assumed to be independently drawn from the standard logistic distribution. The state transition is deterministic since the only dynamic state variable is the incumbency status, determined by the firms' actions in the previous period. We set the discount factor to $\delta = 0.9$ and the flow payoff from being inactive to zero.\footnote{Normalizing the flow profit from the outside option to zero is not without loss of generality for counterfactual analysis \citep*{aguirregabiria2014identification, kalouptsidi2017non, kalouptsidi2021identification}. We show that our counterfactual experiment of reducing a proportion of entry cost is equivalent to providing an entry subsidy equal to the same proportion of the entry cost net of scrap value without this normalization in Online Appendix (ref). } We discretize market characteristics $(w_{m},v_{im})$ into two bins by grouping the observations above and below the median and replacing the values with within-bin means.
We first estimate the parameters under the standard incomplete information Markov perfect equilibrium assumption using the $K$-pseudo maximum likelihood approach Aguirregabiria2007, bugni2021iterated. Each player observes their own idiosyncratic shock $\varepsilon_{imt}$ but not the opponent's shock $\varepsilon_{jmt}$. We then estimate the Markov correlated equilibrium identified set, maintaining this standard informational assumption. The identified set is informationally robust in the sense that it captures all possible parameters that could generate the data under Markov perfect equilibrium, allowing for the possibility that the players know their own $\varepsilon_{imt}$ and may observe information about the opponent's $\varepsilon_{jmt}$ before making decisions. We also estimate the fully robust identified set, which additionally encompasses the possibility that the players do not observe or are partially informed about their $\varepsilon_{imt}$ (formally defined in Section (ref)). The identified set is fully robust to any misspecification of the information structure regarding the idiosyncratic shocks $\varepsilon_{mt} = (\varepsilon_{imt},\varepsilon_{jmt}) $. Lastly, we estimate the model without the excluded variable $v_{im}$ with these three information structures above to gauge the importance of the exclusion restriction. We outline our estimation strategy of the Markov correlated equilibrium identified sets and relegate the computational details to Online Appendix (ref).
We leverage the excluded variable with large support as discussed in Section (ref). Our excluded variable $v_{im}$, the log number of own-chain outlets in the county, has large support because it has no lower bound.\footnote{Given the length of our observation period, we assume that this variable sufficiently captures the time-persistent match effect between firms and counties. While our estimation procedure relies on a specific parametric form of $v_{im}$, sufficient variation in the excluded variable will tighten the identified set even if we impose another parametric form (see e.g., Magnolfi2023).} Firm $i$ is not a potential entrant (i.e., $i$'s active probability is $0$) if the number of own-chain outlets in the county is $0$, since $v_{im} = \log 0 = -\infty$. This model specification is motivated by the fact that a firm will be rarely active in markets where the firm did not have any outlets in the relevant county (Table (ref) Panel A). In markets where firm $j$ is not a potential entrant or $a_{jmt}$ is always $0$, firm $i$ solves a single-agent dynamic decision problem, as the opponent's idiosyncratic shock $\varepsilon_{jmt}$ becomes payoff-irrelevant. We estimate the single-agent discrete choice model, which contains the payoff parameters except the competitive effects parameters in (ref),\footnote{We can identify the competitive effects parameters if there is a subset of markets defined by excluded variables where the opponent is always active additionally. It would be challenging to find a sufficient number of such markets to reasonably estimate the competitive effects parameters in our setting (see Table (ref) Panel B for the effect of our excluded variable on the active probability).} using the two-step pseudo-likelihood method Aguirregabiria2002. This step allows us to obtain estimates for non-competitive effects parameters, $\theta_w, \theta_{i,v}, \theta_{j,v}$, and $\theta_{ec}$.
We then apply Theorem (ref) to estimate the identified set of the competitive effects parameters using markets where both firms are potential entrants. We adopt the non-competitive effects parameters estimated from the single-agent model in the Markov correlated equilibrium identified set estimation to demonstrate the identifying power of our Markov correlated equilibrium framework.\footnote{aradillas2022inference also takes a similar two-step approach in static ordered-response games. The authors point identify the non-competitive effects parameters and use the estimates to partially identify the competitive effects parameters.} We estimate the CCPs using flexible logit specification and construct their simultaneous confidence intervals with a plug-in sup-$t$ implementation montiel2019simultaneous. We estimate a criterion function $Q(\theta)$ that measures the degree of violation in the obedience condition (ref) for given competitive effects parameters $\theta = (\theta_{i,ce},\theta_{j,ce})$. We construct identified set estimates by grid searching $\theta$ that satisfies the estimated $Q(\theta) \leq c$ for a small threshold $c > 0$. We use non-zero $c$ to accommodate misspecifications in the structural and CCP logit models. Larger values of $c$ yield more conservative identified sets. We do not incorporate the sampling error associated with the first-stage single-agent model estimation.
We also consider a model without the excluded variables $v_{im}$ and $v_{jm}$. We grid search for all the payoff parameters $\theta = (\theta_{w},\theta_{ec},\theta_{i,ce},\theta_{j,ce}) $ for this model.
Table (ref) reports our estimation results with the exclusion restriction. The presence of an opponent has a positive spillover effect on profitability under the standard incomplete information Markov perfect equilibrium assumption (Column (1)). The 95% confidence intervals (CIs) of the competitive effects parameters become wider as we make the information structure more flexible (compare columns from left to right).\footnote{We set the criterion function threshold $c$ so that the CIs of the competitive effects parameters become column (1) $\subset$ column (2) $\subset$ column (3).} The Markov correlated equilibrium estimates suggest the possibility of a large negative opponent effect (columns (2) and (3)). However, the positive effect of Starbucks' presence on Dunkin's profitability is tightly bounded above. The exclusion restriction helps narrow the direction of Dunkin's competitive effects parameter (compare with the results without the excluded variable in Table (ref) in Appendix (ref)).
The market characteristics coefficients and entry cost from the two-player model (column (1)) and the single-agent model (columns (2) and (3) display the same estimates) are similar, which justifies the usage of different subsets of markets to estimate the same parameters.\footnote{The CIs do not overlap for some parameters. The discrepancies can be attributed to either misspecification in the information structure or the differences in the parameters in different markets. We do not provide a formal test of misspecification in the information structure. See \citet*{han2024testing} for a related test in the Bayes correlated equilibrium context.} The market characteristics coefficient estimates show positive effects of attractive markets and economies of density on profitability. The entry cost estimates are relatively large compared to the other parameter estimates, which is consistent with the low entry probabilities.
As a counterfactual experiment, we reduce each firm's entry cost by 10% and simulate the number of active firms from 2003 under three assumptions on information structures used in the estimation. The first is the standard incomplete information Markov perfect equilibrium predictions. We then apply the Markov correlated equilibrium to make informationally robust predictions, assuming either the standard incomplete information or the fully robust specification. For each case, we use the structural parameters estimated under the corresponding solution concept and information structure and apply the same framework to compute the counterfactual predictions.\footnote{In the counterfactual Markov correlated equilibrium predictions, we do not use the latent information structures that generate the Markov perfect equilibrium predictions compatible with the data for each parameter value in the Markov correlated equilibrium identified set estimates. \citet*{Bergemann2022} shows how to conduct counterfactual analysis with the latent information structure fixed at the pre-counterfactual Markov perfect equilibrium. This approach would not tighten our standard incomplete information Markov correlated equilibrium predictions since the predictions for a given set of structural parameters are tight (Table (ref)). It may appreciably narrow the fully robust predictions by restricting the range of predictions for a given set of structural parameters.}
We consider two extreme scenarios, low and high, to get a sense of a reasonably possible range of the counterfactual predictions. For the low (high) scenario, we use the lower (upper) bounds of the 95% CIs of the competitive effects parameters and find an equilibrium that minimizes (maximizes) the expected number of active firms.\footnote{Note that the pair of values at the lower bounds of the competitive effects parameters for Starbucks and Dunkin' may not necessarily belong to the confidence set.} As a separate exercise, we also make counterfactual predictions using the lower and upper bounds of the 95% CIs of all structural parameters, rather than only the competitive effects parameters, to incorporate the uncertainty of the non-competitive effects parameters. We use the counterfactual equilibrium CCPs to simulate the evolution of active firms. We detail the counterfactual equilibrium calculations in Online Appendix (ref).
Figure (ref) plots the counterfactual market evolution from 2003--2017. The y-axis takes the average of all markets, so the minimum and maximum possible values are 0 (no firms active) and 2 (both firms active), respectively. The no policy baseline with the standard incomplete information Markov perfect equilibrium predicts an increase from $0.15$ to $0.37$ during the sample period (line Base). The Markov perfect equilibrium predicts that the counterfactual policy intervention of reducing the entry cost by 10% increases the number of active firms by 50% in 2017 (line MPE). The range of the two scenarios (dashed curves) is very tight using the CIs of the competitive effects parameters (Figure (ref), average number of active firms in 2017: low, $0.55$; high, $0.58$) and become somewhat wider using the CIs of all structural parameters (Figure (ref), low, $0.46$; high, $0.69$).
The counterfactual Markov correlated equilibrium predictions, keeping the standard incomplete information, of the average number of active firms in 2017 range from $0.35 $ (low) to $0.87 $ (high) using the CIs of the competitive effects parameters (Figure (ref), line MCE). If we are concerned about the informational robustness regarding the competitive effects parameters, this range provides a reasonably plausible set of outcomes, allowing for robust yet still informative policy implications. In contrast, the predictions become much more spread out and less informative, ranging from $0.26$ (low) to $1.32$ (high), using the CIs of all parameters (Figure (ref), line MCE). The fully robust Markov correlated equilibrium generates very wide predictions even right after the start of the counterfactual policy intervention (line Fully robust). Moreover, without the exclusion restriction, the counterfactual Markov correlated equilibrium predictions cover almost the entire range of the average number of active firms, $[0,2]$, even under the standard incomplete information (Figure (ref) in Appendix (ref)).
The counterfactual Markov correlated equilibrium predictions involve two sources of multiplicity: partial identification and estimation uncertainty of structural parameters; and flexible counterfactual information structure. The flexible information structure can generate various counterfactual equilibria, resulting in a wide range of predictions. Table (ref) tabulates the range of counterfactual predictions for a given set of structural parameters to examine the contribution of these multiplicities. The minimum and maximum number of active firms that can arise from the counterfactual Markov perfect equilibrium predictions coincide for a given set of structural parameters (columns under MPE Standard incomplete), which suggest that we do not have multiple equilibria for a fixed vector of parameters and information structure. Thus, the range of Markov perfect equilibrium predictions is entirely driven by the estimation uncertainty (or 95% CIs) of structural parameters.
The prediction ranges for a given set of structural parameters remain very tight using the Markov correlated equilibrium, using the standard incomplete information as the baseline (columns under MCE Standard incomplete). These prediction ranges encompass the counterfactual outcomes arising from any Markov perfect equilibrium when the players minimally observe their payoff shocks and may observe more information about opponents' payoff shocks. The tight counterfactual outcome bounds indicate that the ability to obtain more information about opponents' payoff shocks and the possibility that such information is correlated across firms have little impact on the outcome, given the model parameters fixed. The small competitive effects parameters (in absolute values) can explain the tight counterfactual bounds. If the competitive effects are null, then the firms do not care about the opponents' behavior, and there is no room for an information designer to affect the total number of entrants, on average.\footnote{The information designer can still induce correlation in the firms’ actions. As a result, the CCPs may deviate from those predicted under the standard MPE scenario.} Here, the competitive effects parameters range between $-0.30$ and $0.14$, which is small considering the standard deviation of the idiosyncratic payoff shock (the standard deviation of the standard logistic random variable is $\sqrt{\pi^2/3} \approx 1.81$). Consequently, the degree to which the counterfactual outcomes can vary depends largely on the structural parameter values. In contrast, the prediction ranges, when fixing structural parameters, become much wider by allowing the possibility that players may not observe or be partially informed about their own payoff shocks (columns under MCE Fully robust).
In this paper, we have studied the identification, estimation, and counterfactual analyses of a common class of dynamic games for empirical analysis, assuming the researcher knows some minimal information available to players. To facilitate informationally robust econometric analysis of dynamic games, we have defined Markov correlated equilibrium, a dynamic analog of Bayes correlated equilibrium, and studied its properties. We have discussed new challenges that arise in dynamic environments and proposed practical solutions. We have applied our proposed method to study the dynamic entry and exit decisions of the two largest coffee chains in the US.
We conclude this paper with a discussion of future research directions. First, further refinement of Markov correlated equilibrium (and thus Bayes correlated equilibrium) will attract greater interest from empiricists. The type of informational robustness offered by Markov/Bayes correlated equilibrium admits too many possibilities. Markov correlated equilibrium identified sets and counterfactual outcomes can span a wide range, which may not be empirically informative; Magnolfi2023 find similar results in their empirical application. Numerical examples from koh2023stable and Magnolfi2023 show that even complete information correlated equilibrium identified sets can be large, indicating that correlation in actions can also weaken the identifying power substantially. In some settings, we may be able to reasonably exclude some information structures from consideration. Additionally, we may want to restrict attention to some equilibrium selection mechanisms to narrow down the possibilities. It will be fruitful to find a tractable approach that can accommodate further assumptions on the underlying information structure and equilibrium selection mechanism.
Second, estimation with Markov correlated equilibrium remains computationally challenging. A similar problem exists for Bayes correlated equilibrium since the analyst needs to run a grid search. Studying how to conduct computationally tractable estimation and inference in this class of partially identified models will be crucial for broadening the scope of applications.
\paragraph{Disclosure Statement} During the preparation of this work the authors used Open AI's ChatGPT 5.0 in order to check grammar and improve clarity of the writing. After using this tool/service, the authors reviewed and edited the content as needed and takes full responsibility for the content of the published article. Wharton Research Data Services (WRDS) was used in preparing part of the data set used in the research reported in this manuscript. This service and the data available thereon constitute valuable intellectual property and trade secrets of WRDS and/or its third-party suppliers. The authors declare that no financial or personal conflicts of interest influenced the research presented in this paper.