Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
108,173 characters · 34 sections · 53 citation commands
Robust Market Interventions
\setcounter{page}{1}
Market power has recently attracted renewed attention and is thought to have significant and growing welfare implications syverson2019macroeconomics. While many applied analyses of competition confine the analysis of market power to tightly defined product markets, it is becoming clear that important welfare-relevant spillovers operate across such markets baqaee2020productivity, azarvives2021,pelligrino2021,edererpelligrino2021. Our theoretical understanding of such spillovers in environments with many firms interacting via general demand systems remains limited. This paper is about the welfare theory of such interactions.
Consider many profit-maximizing, single-product firms simultaneously setting prices, with arbitrary complementarities and substitutabilities across products. For instance, one firm's product---e.g., a Samsung smartphone---may be a substitute to some products---e.g., Apple smartphones---and a complement to others---compatible accessories such as earbuds, watches, and smart home appliances, which may in turn be substitutes or complements to one another.
We are interested in the nature of inefficiencies in such an environment and policies to respond to them. We consider these issues from the perspective of an authority that recognizes the possibility of inefficiency due to market power and can intervene through taxes and subsidies on firms' sales. For concreteness, we will think of this authority as the operator of a large marketplace, such as Amazon, that mediates retail sales and aims to increase the equilibrium economic surplus generated by the marketplace.\footnote{The aim of this might be to deliver more value to consumers and producers to keep the platform competitive, or to reclaim the surplus via lump-sum fixed fees.} What principles should guide the design of such interventions? When can such policies be implemented under realistic uncertainty about market parameters?
What makes the problem challenging is that, once we broaden our perspective beyond one traditionally defined market (e.g., smartphones) and consider spillovers to a variety of other complements and substitutes, there is a kind of curse of dimensionality. In a marketplace with numerous and changing goods, the demand system is high-dimensional and a priori unstructured. As any firm's cost changes, the number of potential effects to consider is equal to the number of products; thus, the number of interactions quadratically in this number. Realistic signals will leave substantial uncertainty about many aspects of the structure of the game among the firms (see (ref) for a detailed discussion). In particular, the authority will have nothing close to a precise estimate of the entire demand system. This raises the question is whether there are policies that robustly improve surplus despite this uncertainty.
Our main result is that if demand satisfies a property that we call recoverable structure, then there are feasible intervention rules that robustly (i.e., with high probability) increase equilibrium total surplus despite large errors in observing every detail of the system. Moreover, within a natural class of interventions---those that do not reduce consumer surplus\footnote{This constraint is natural for a platform that could lose customers to other marketplaces or a government that can lose political support.}---our feasible interventions achieve the largest gain in surplus that is possible for a given level of subsidy expenditure. Hence, within this class, these interventions are as good as those that could be designed by an authority with perfect information.
Furthermore, our results provide tight conditions for robust intervention in the following sense. First, the property of recoverable structure cannot be dispensed with; there are reasonable demand systems without recoverable structure for which the authority cannot robustly increase total surplus. Second, we show that there are settings with recoverable structure in which any intervention rule that robustly increases total surplus is equivalent, in terms of how it allocates surplus, to the interventions identified by our main result. These tightness results show that robust interventions must, in general, be tailored to the marketplace---there is no simple rule of thumb that always works. On the other hand, under conditions that we identify, it is possible to tailor rules well, despite large uncertainty in many aspects of demand.
The key condition in the paper is recoverable structure. We now explain what it means for demand to have recoverable structure and how this property is used in the construction of robust interventions.
The demand structure is encoded in a matrix $\bm{D}$ of demand derivatives, which in our setting is equal to the Slutsky matrix. A given cell $D_{ij}$ in this matrix is the derivative of product $i$'s demand with respect to product $j$'s price. Thus, the matrix specifies the complementarity and substitutability relationships across products. Mathematically, the recoverable structure property requires that $\bm{D}$ can be written as a rank-one matrix with large norm plus a matrix orthogonal to this. This rank-one piece can be thought of as a large principal component: a part of the demand system described by a single vector that accounts for a large amount of demand behavior. In terms of the economic intuition, we will show that recoverable structure entails substantial large-scale complementarities. This manifests as the ability of small subsidies to have large spillover effects that raise the consumption of many goods by significant amounts.
The key point here is that recoverable structure is large-scale structure. When large-scale complementarities are present, they correspond to marketplace-wide double marginalization problems, where many goods all exert externalities on one another. Such broad externalities create the potential for interventions that substantially improve welfare.
The statistical implications of recoverable structure are then central to actually taking advantage of this potential when the Slutsky matrix is observed imperfectly. The authority's signal consists of noisy estimates of the entries of this matrix, with noise magnitudes in each entry comparable to the entries themselves. This noise creates large uncertainty in the operation of a given intervention. We show that, nevertheless, in large markets with recoverable structure, such noisy observation of $\bm{D}$ can be used to precisely predict the effects of some well-chosen interventions---specifically, those operating in the space of eigenvectors associated with the largest eigenvalues of $\bm{D}$. The key tool for this is the Davis--Kahan theorem davis1970rotation.
Combining the economic and statistical implications of recoverable structure allows us to establish our main result. In markets possessing such structure, the authority can recover precise information about large-eigenvalue components of the spectral decomposition, and interventions based on this information have highly predictable surplus implications.
At a technical level, to perform this analysis we develop a new spectral description of the pass-through of an intervention. That is, we diagonalize the Slutsky matrix to obtain a specific orthonormal basis in which we can express the implications of any intervention as a linear combination of orthogonal effects. These effects correspond to the projection of the intervention onto each eigenvector of the Slutsky matrix $\bm{D}$. By characterizing the pass-throughs of subsidies to prices, quantities, and welfare separately across these principal components, we are able to prove that targeting the high-eigenvalue principal components yields precisely predictable results achieving our claimed welfare properties. The spectral decomposition may be of independent interest, yielding a useful basis in which price and welfare pass-throughs of cost shocks behave intuitively despite the complexity of a system with arbitrary spillovers.
Our paper contributes to the literature on the structure and theoretical properties of market power. For an early theoretical paper, see dixit1986comparative; more recent studies include, for example, vives1999oligopoly, azarvives2021, nocke2018Multiproduct, and nocke2022Merger. A recent strand of research in macroeconomics and industrial organization uses differentiated oligopoly network models---similar to the one we consider here---to provide empirical estimates of efficiency losses due to market power (e.g., pelligrino2021 and edererpelligrino2021).\footnote{See also elliott2019role for related arguments about how network methods can be useful for competition authorities in developing antitrust investigations.}
Given these estimates of inefficiencies, a natural theoretical question is: What feasible interventions can improve welfare? Our main contribution is to analyze interventions from the perspective of an authority uncertain about the demand structure. Our analysis combines new spectral pass-through formulas with results building on the statistical theory of large matrices, and we identify conditions on the demand structure that ensure the robust achievement of welfare improvements even when many aspects of the demand structure cannot be accurately estimated.\footnote{Our focus on pass-through builds on work emphasizing the value of pass-through as a conceptual tool, e.g., marshall1890, Pigou1920, dixit1979price and, more recently, weyl2013pass, miklos2021pass and JN2024.} This approach has significant implications for understanding which kinds of empirical models are needed to design interventions in large markets with many goods. We elaborate on these issues in (ref).
Methods in high-dimensional statistics are currently attracting considerable interest in econometric settings with high-dimensional covariates (see, e.g., \citet*{athey2021matrix} and \citet*{chernozhukov2023inference}), and there is work applying related statistical models to informational or behavioral spillovers in social networks \citep*{golub2012homophily, dasaratha2020distributions, cai2022,parise2023graphon, chandrasekhar2024non}. However, we know little about when noisy data can be effectively used in order to implement desirable interventions in the presence of strategic spillovers, particularly in market settings. We show that, in a large oligopoly market, the concepts developed in the literature on large network recovery can be useful for designing socially desirable interventions.
Our paper contributes to the theory of network interventions. Early contributions include Borgatti2006, \citet*{Ballesteretal2006}, and goyal1996interaction.\footnote{The literature on this subject is very large. Intervention design has been studied in models of information diffusion, advertising, finance, security, and pricing, among other topics---see e.g., \citet*{banerjee2013diffusion}, BlochQuerou2013, \citet*{Candoganetal2012}, BelhajDeroian, GDemange, DziubinskiGoyal2017, GaleottiGoyal2009, and \citet*{leduc2017pricing}.} Spectral methods have recently been applied to optimal intervention problems when spillovers are known \citep*{galeotti2020targeting, gaitonde2021polarization,liu2024dynamic}.\footnote{Some recent work uses spectral analysis to derive conditions for core-selecting re-allocative auctions MarzenaR2023, and robust implementation ollar2023network. See also aguiar2017slutsky on spectral methods to study Slutsky matrices in a consumer theory setting.} By contrast, in the present paper, the authority observes strategic spillovers with significant noise.\footnote{We share with several prior papers the idea that decision-makers act under partial information about the network. For instance, \citet*{Galeottietal2010} study large network games where players have incomplete information about the network structure, described by a random graph; \citet*{akbarpour2020just} considers seeding in a large random graph; the diffusion process there lacks any form of complementarity. Our questions and methods of analysis are very different from these papers.} The methods we develop for robust interventions can be applied to other network games more generally and we briefly discuss this in (ref). Our analysis of perturbations of taxes and subsidies is related to the classic “tax reform approach” in public finance feldstein1976theory,tirole1981tax; the study of uncertain spillovers distinguishes our work.
Our approach to robustness is conceptually related to, but methodologically distinct from, an extensive literature in economic theory. That literature focuses on understanding the design of mechanisms and contracts that achieve desired outcomes even when assumptions about the environment (e.g., agents' preferences, beliefs, and rationality) are relaxed; see carroll2019robustness for a survey. Our definition of robustness aligns with the spirit of this literature. However, in our context, the motivation for analyzing robust interventions arises from the high-dimensional nature of the market state, and use methods that align with statistical work in this type of setting.
In this section, we present the framework for our study. The foundation is a simple differentiated oligopoly game. Within this game, we introduce a class of interventions available to the authority and calculate the surplus outcomes associated with these interventions. Finally, we introduce the statistical framework describing the signals available to the authority and a notion of rules that use these signals to achieve good outcomes robustly.
There is a set $\{1,\ldots,n\}$ of distinct products. The demand for these products arises from the consumption choices of a fixed, finite number of optimizing households. Each household $h\in \{1, \dots, H\}$ takes prices as given and has a choice utility that is quasilinear in a numeraire $m$, $$ {U}^h(\tilde{\bm{q}}^h,m) = V^h(\tilde{\bm{q}}^h) + m,$$ where $V^h$ is a twice-differentiable and strictly concave function of the consumption profile $\tilde{\bm{q}}^h \in \mathbb{R}^n$ and $m$ is a numeraire (“money”), in which all prices are denominated. Given a price profile $\tilde{\bm{p}}$, the household's problem is to choose a bundle $\tilde{\bm{q}}^h$ to maximize ${U}^h(\tilde{\bm{q}}^h,m)-\tilde{\bm{p}}\cdot \tilde{\bm{q}}^h$. Letting $\bm{q}^h(\tilde{\bm{p}})$ be the solution to household $h$'s problem (unique by concavity of $V^h$), total market demand is\footnote{ Because the households' utilities are quasilinear in money, one can derive the same aggregate demand from a representative consumer, and this description is sufficient for studying producer and aggregate consumer surplus. However, some of our results will provide more refined results about the effects on individual consumers, showing that no household is significantly hurt.} $$ \bm{q}(\tilde{\bm{p}}) = \sum_{h=1}^H \bm{q}^h(\tilde{\bm{p}}).$$ Note that we use tilde notation for an arbitrary price or quantity, and then drop the tilde for these variables to indicate some optimal or equilibrium solution.
There is a firm associated with each product: Firm $i$ produces product $i$. Firms play a simultaneous pricing game; each firm chooses $\tilde{p}_i \geq 0$. For any realized profile of prices $\tilde{\bm{p}}$, firm $i$'s profit is
where $c_i$ is the (constant) marginal cost of production.
We fix a vector $\bm{c}^0$ of marginal costs and a pure-strategy Nash equilibrium $\bm{p}^0$, and we refer to these as the status quo marginal costs and equilibrium, respectively.\footnote{Existence of a pure strategy equilibrium is guaranteed if profits are quasi-concave in prices (see Theorem 1.2 in fudenberg1991game). A sufficient condition for this is that the functions $1/q_i(\tilde{\bm{p}})$ are convex in $\tilde{p}_i$ (vives1999oligopoly page 149). This holds, for instance, under the much stronger condition that demand is linear in prices; in this case the equilibrium is also unique. The general sufficient condition for local uniqueness is nonsingularity of the Jacobian of best responses at equilibrium, which will hold generically in our setting mclennan2018advanced.} To facilitate unambiguous local comparative statics, we make the following assumption.
From now on, we confine attention to cost perturbations within the set discussed in (ref), and when we refer to an equilibrium at any cost profile, we mean the locally unique one entailed by this assumption.
An authority---an institution that oversees a marketplace---can intervene in the market. We focus on a canonical set of interventions: per-unit subsidies and taxes. For a consumption profile $\tilde{\bm{q}}$, a per-unit subsidy intervention $$\bm{\sigma}=(\sigma_1,\ldots,\sigma_n)$$ consists of a transfer $\sigma_i \tilde{q}_i$ from the authority to firm $i$; a positive $\sigma_i$ corresponds to a subsidy to firm $i$, while a negative $\sigma_i$ corresponds to a tax.
We will focus on demand functions that are linear in a neighborhood of the status quo equilibrium, as captured by the following assumption.
This assumption facilitates our analysis of interventions, yielding simple formulas for comparative statics. It also captures much of the economics of our robust interventions problem, despite its simplicity. We discuss how to extend the analysis for the case of non-linear demand in (ref).
The firms' first-order conditions imply that equilibrium prices $\bm{p}$ around the status quo $\bm{p}^0$ satisfy $$ {q}_i(\bm{p})=-\frac{\partial q_i}{ \partial p_i}(\bm{p}) ({p}_i-{c}_i). $$ The linearity assumption implies that $\frac{\partial q_i}{ \partial p_i}(\bm{p})$ remains constant when prices change locally around $\bm{p}^0$. Hence, we can replace the $\bm{p}$-dependent partial derivative in the above equation with the constant $\frac{\partial q_i}{ \partial p_i}(\bm{p}^0)$. By strict concavity of the consumer utility functions, this is a negative number; from now on, we maintain a normalization (by choosing suitable units in which to express the quantity produced by firm $i$) that $\frac{\partial q_i}{ \partial p_i}(\bm{p}^0)=-1$ (see (ref)). After this normalization, equilibrium behavior is summarized by the following system of equations:
Implicitly differentiating this (linear) system, we obtain that the effect of a small intervention $\bm{\sigma}$ on prices is determined by the following system of equations:
where $\bm{\sigma}=\bm{c}^0-\bm{c}$ is the intervention (i.e., tax or subsidy offered by the authority), $\dot{\bm{p}}$ is the derivative of $\bm{p}$ in the direction of $\bm{\sigma}$ (see (ref)), and $\bm{D}=\bm{D}(\bm{p}^0)$ is the Slutsky matrix (in the normalized units): $$ D_{ij}= {\frac{\partial q_i }{\partial p_j}}(\bm{p}^0).$$ For $i\neq j$, if $D_{ij}>0$ (resp. $D_{ij}<0$) then, around the equilibrium, products $i$ and $j$ are substitutes (resp. complements).\footnote{In general, the matrix of derivatives of Marshallian demand need not be the same as the Slutsky matrix (which works with compensated demand). However, in this demand system, the wealth effect is zero due to the fact that the goods utility and money are additively separable. Thus, the two matrices coincide nocke2017quasi, and so we use the term “Slutsky matrix” throughout.} Note also that quantity changes following the intervention $\bm{\sigma}$ are pinned down by
The local linearity of demand implies that comparative statics of prices and quantities are fully determined by the Slutsky matrix $\bm{D}$. We note that $\bm{D}$ satisfies the following property nocke2017quasi.
This property holds because the demand function can be taken to arise from a representative household (with a twice-differentiable utility function for goods equal to the sum of the consumers' utilities, $V^h$).
Finally, the following helpful normalization is without loss of generality---it holds by suitably adjusting the units of the numeraire $m$.
In what follows, we maintain Assumptions 1--3 unless stated otherwise.
The authority's net spending associated with an intervention $\bm{\sigma}$ is denoted by $S=\bm{\sigma}\cdot \tilde{\bm{q}}$. Recalling (ref), we define the set of feasible interventions as\footnote{We restrict attention to deterministic interventions but this is immaterial to our results.} $$\bm{\Sigma} = \{ \bm{\sigma} \in \mathbb{R}^n : \Vert \bm{\sigma} \Vert < \nu\}.$$
The authority cares about the surplus that different market participants obtain in equilibrium. We focus on three canonical metrics: consumer surplus $C$, producer surplus $P$, and total surplus $W$ (accounting for the intervention expenditure $S$). Given an intervention $\bm{\sigma}$, and quantity profiles $\{\bm{q}^h\}_{h=1, \dots, n}$ and $\bm{q}:=\sum_h \bm{q}^h$, these are: \[ C=\sum_{h} {C}^h \quad \text{where} \quad {C}^h= V^h(\bm{q}^h) - \bm{q}^h \cdot \bm{p}, \]
We evaluate the effect of an intervention $\bm{\sigma}$ on an outcome variable $Y$ by its first derivative. Formally, the first-order effect on any outcome variable $Y$ (e.g., $Y=C$ or $P$) of changing subsidies in the direction $\bm{\sigma}$ is defined by\footnote{We often omit the subscript $\bm{\sigma}$ when there is no ambiguity about the relevant intervention.}
Fixing the parameters of the economy, the set of possible surplus outcomes is defined to be the set of tuples $$\{ (\dot{C}_{\bm{\sigma}},\dot{P}_{\bm{\sigma}}, \dot{S}_{\bm{\sigma}}) : \bm{\sigma} \in \mathbb{R}^n \}$$ of surplus outcomes corresponding to some intervention. The following proposition characterizes the possible surplus outcomes.
Part (1) says that, for a given level of spending $\dot{S}$, the set of possible outcomes is a line in $(\dot{P},\dot{C})$ space. This “Pareto frontier” tells us what is possible in principle, and in particular implies that market surplus cannot be increased by more than twice the level of expenditure without reducing consumer surplus.
While this result characterizes what post-intervention outcomes are possible in equilibrium, it does not discuss how to attain them. The following lemma is an important input in the answer to this question, and is used in the proof of the “if” direction of (ref)(1).
The proof works by combining the formulas in the proof of the above proposition with the formulas in ((ref)) and ((ref)) to make price and quantity derivatives explicit.
Formula ((ref)) can be interpreted as a pass-through equation: entry $i$ of the row vector $\bm{w}^\mathsf{T}=(\bm{q}^0)^\mathsf{T} \bm{D} [\bm{I}-\bm{D}]^{-1}$ gives the impact on total surplus of increasing the subsidy $\sigma_i$. The authority aims to achieve a desired total surplus effect $\dot{W}_{\bm{\sigma}}$, possibly subject to additional requirements, such as holding spending $\dot{S}_{\bm{\sigma}}$ constant.
Matrix inverses such as $[\bm{I}-\bm{D}]^{-1}$ can be extremely sensitive to entries of $\bm{D}$. Therefore, without precise knowledge of $\bm{D}$ and $\bm{q}^0$, the authority may not be able to implement a desired point on the line defined by ((ref)). For example, the authority may not be sure that a given intervention will increase total surplus ($\dot{W}_{\bm{\sigma}}>0$) rather than decrease it ($\dot{W}_{\bm{\sigma}}<0$). Indeed, it seems hard to justify the detailed study of comparative statics such as (ref) when $n$ is large without confronting the uncertainty about the ingredients of the formula an analyst or authority is likely to face.
These observations motivate the central question of this paper: Which interventions have surplus effects that can be predicted with confidence by an authority facing substantial uncertainty about market primitives?
To formalize this question, we introduce a simple model of noisy observation of $\bm{D}$ and $\bm{q}^0$.
In the linear oligopoly environment, (ref) establishes that to determine the first-order surplus effects of any intervention $\bm{\sigma}$, the only additional data needed is the tuple $(\bm{D},\bm{q}^0)$, where $\bm{D}$ is a negative semidefinite matrix with diagonal entries $-1$ and $\bm{q}^0$ is a vector of norm at most $1$. We call such a tuple a market state and denote it by $\bm{\theta}$. The set of possible market states is denoted by $\bm{\Theta}$.
The authority receives a signal, denoted by $\widehat{\bm{\theta}} \in \widehat{\bm{\Theta}}$, about the market state.\footnote{(ref) discusses some practical examples.} This signal consists of random variables $$ \widehat{\bm{D}} = \bm{D} + \bm{E} \text{ and } \widehat{\bm{q}}^0 = \bm{q}^0 + \bm{\varepsilon}. $$ We will later detail assumptions on the error terms. For now, a canonical setting to keep in mind is one where all error draws are mean-zero and independent, with each error having a magnitude comparable to the underlying entry $D_{ij}$ or $q^0_{i}$. Note that the signal need not lie in the same set as the state; for example, under our assumptions, $\bm{D}$ is negative semidefinite, but the signal $\widehat{\bm{D}}$ might not be.
Let $\varphi_{\bm{\theta}} \in \Delta(\widehat{\bm{\Theta}})$ denote the probability measure over signals when the state is $\bm{\theta}$.
The authority designs an intervention rule\footnote{This should be measurable in a suitable sense, which is clear in our application.} $$ \bm{R} : \mathcal{T} \to \bm{\Sigma}, $$ prescribing an intervention $\bm{\sigma} \in \bm{\Sigma}$ for every possible signal $\widehat{\bm{\theta}}$. We now define a notion of such a rule robustly achieving a desired property.
A market outcome is a tuple $(\bm{\theta},\bm{\sigma})$ consisting of a market state and an intervention. We call this pair the outcome because it determines production, consumption, and transfers. A property is a measurable subset $\mathscr{P} \subseteq \bm{\Theta}\times \bm{\Sigma}$ of all possible outcomes. An important example of a property is increasing total surplus, given by ((ref)):
We are interested in understanding which properties can be achieved with high probability in all market states that the authority considers possible:
The only randomness in the definition is in the signal draw---recall $\varphi_{\bm{\theta}}$ is the distribution of the signal given the true state $\bm{\theta}$.
We close with some remarks on our modeling choices.
The main result of this paper says that under conditions on the set of possible market states $\bm{\Theta}$ and conditions on the distribution of errors, there are intervention rules that improve surplus robustly. The content of the result lies in specifying the assumptions on $\bm{\Theta}$ and error distributions. While these conditions are somewhat involved to state in full generality, a useful preview can be presented in an important class of examples, related to the classic stochastic block model in network theory.
There is a fixed, finite set of product types, $M=\{1,2,\ldots,m\}$. The interactions of products are determined by their types, and given by entries of an $m$-by-$m$ type-level matrix $\bar{\bm{D}}$ satisfying Property NSD. Quantities are also determined by types, according to a vector $\bar{\bm{q}} \in \mathbb{R}^m$. Let the type of good $i$ be $k(i) \in M$.\footnote{We let $N(t)$ be the number of products of type $t$, which depends on the total number of products $n$, and assume $N(t)/n$ is a convergent sequence as $n\to\infty$ for each $t \in M$.} Finally, fixing $\gamma(n) \in (0,1]$, let
An explicit example of a stochastic block model appears in (ref). The errors $\varepsilon_i$ in observing $\bm{q}^0$ and the errors $E_{ij}$ in observing $\bm{D}$ are i.i.d. and mean zero, with bounded support, satisfying $\operatorname{Var}[E_{ij}] < \overline{\sigma}$ and $\operatorname{Var}[\Vert \bm{\varepsilon}\Vert] < \overline{\varsigma} \Vert \bm{q}^0 \Vert $ for some fixed real numbers $\overline{\sigma}$ and $\overline{\varsigma}$. This entails that errors in demand signals and quantity signals are of the same order of magnitude as the underlying parameters. Within this model, we have:
Note that (ref)(2) implies that the intervention rules guaranteed by (ref) achieve the largest possible increase in total surplus subject to not reducing consumer surplus. It is also worth noting that the partitioning of goods into types need not be known in advance to achieve the target surplus outcome.
In this setting, the assumption that $\gamma(n)n^{1/2}$ is large ensures that some of the entries of $\bar{\bm{D}}$ can be recovered despite noisy observation. Notice, however, that other entries of $\bar{\bm{D}}$ may not be recoverable precisely. One extreme example of this occurs if some types have only a single good, or more generally a number of goods uniformly bounded in $n$. The substance of the result is that the information that can be recovered about $\bar{\bm{D}}$ suffices to robustly achieve the indicated outcome.
The result raises several questions: First, how is recoverable information used to design effective interventions, and what determines the limits of this strategy? More fundamentally, how can it be extended beyond the specific structure of the stochastic block model? The assumption that there are arbitrarily large\footnote{When there are $|M|$ types of products, then some type must contain at least $n/|M|$ products.} “blocks” of products whose exact relationships to other goods (the entries $D_{ij}$) are identical within type is restrictive. Are there more flexible structures that permit robust interventions---e.g., in cases where no entry of $\bm{D}$ can be recovered precisely? A third question that this result raises is whether outcomes that allocate surplus differently can be robustly achieved. The remainder of the paper addresses the three issues we have raised.
The next subsection presents the assumption on $\bm{\Theta}$, called recoverable structure, that underlies our analysis of the general robust intervention problem. We then present the key method---a spectral price theory decomposition---that enables our use of this assumption, and illustrate the ideas throughout in relation to the stochastic block model special case.
Recoverable structure imposes conditions on the pattern of complements and substitutes (which we will call interactions) among products, as summarized by the Slutsky matrix $\bm{D}$. It requires that there is a strong latent pattern of product interactions and, simultaneously, this latent structure “has enough correlation” with the vector of market quantities.
We now define this notion formally. A vector in $\mathbb{R}^n$ describes a bundle of products. Given $\bm{D}$, we are interested in the subspace of bundles spanned by eigenvectors of $\bm{D}$ with large eigenvalues. Formally, let $\mathcal{L}(\bm{D},M) \subseteq \mathbb{R}^n$ be the subspace of the bundle space spanned by the eigenvectors of $\bm{D}$ with eigenvalues at least $M$ in absolute value.
To understand this definition, note that a Slutsky matrix, by virtue of being symmetric, can be orthogonally diagonalized: it can be written as a linear combination of orthogonal rank-one matrices: \[ \bm{D} = -\sum_{\bm{u}^\ell\in \mathcal{L}(\bm{D},M)} |\lambda_\ell| \underbrace{{\bm{u}}^\ell (\bm{u}^\ell)^\mathsf{T}}_{\text{rank-1 matrix}}-\sum_{\bm{u}^\ell\notin \mathcal{L}(\bm{D},M)} |\lambda_\ell| \underbrace{{\bm{u}}^\ell (\bm{u}^\ell)^\mathsf{T}}_{\text{rank-1 matrix}} \] where $\lambda_1, \lambda_2, \ldots, \lambda_n$ are the eigenvalues of $\bm{D}$ (which are nonpositive numbers because $\bm{D}$ is negative semidefinite), ordered from greatest to least in absolute value, and $(\bm{u}^1, \dots, \bm{u}^n)$ is a corresponding basis of orthonormal eigenvectors. All the summands are orthogonal to each other, and $|\lambda_\ell|$ is the norm of the contribution of the corresponding summand.
Having market states with $(M,\delta)$-recoverable structure means that (i) $\bm{D}$ has eigenvectors with eigenvalues larger than $M$ in absolute value, ensuring the first summation is nonzero, and (ii) the vectors $\bm{u}^\ell$ in that sum can jointly account for a non-negligible portion of the status quo quantities. We will call the eigenvectors $\bm{u}^\ell$ associated with the eigenvalues such that $|\lambda_\ell| \geq M$ the top eigenvectors.
As we will detail later, if we set $M$ to be larger than the norm of the noise $\bm{E}$ in the signal of $\bm{D}$, then condition (i) ensures that the space of top eigenvectors of $\bm{D}$ can be estimated precisely. We now explain what recoverable structure with a large $M$ means economically before explaining the intuition behind requirement (ii), which we defer to (ref).
We start with a simple example.
Recoverable structure is closely related to large-scale complementarities more generally. To see this, let $\Vert \bm{x} \Vert $ be the Euclidean norm of vector $\bm{x}$ and note that: $$ |\lambda_1| =\sup_{\dot{\bm{p}} \neq \bm{0}}\frac{\Vert \bm{D} \dot{\bm{p}} \Vert}{\Vert \dot{\bm{p}}\Vert}= \sup_{\dot{\bm{p}} \neq \bm{0}}\frac{\Vert \dot{\bm{q}} \Vert}{\Vert \dot{\bm{p}}\Vert},$$ where the first equality is the well-known Courant--Fischer characterization of the spectral radius of a symmetric matrix, and the second equality follows from of equilibrium condition, $\dot{\bm{q}}=-\bm{D} \dot{\bm{p}}$. The solution to the maximization problem over $\dot{\bm{p}}$ is to choose a change in price equal to the dominant eigenvector of $\bm{D}$, denoted by $\bm{u}^1$, which is the eigenvector associated with the largest eigenvalue in absolute value. Thus, the largest eigenvalue measures how much a price shock can be amplified in terms of its effect on demand and the corresponding eigenvector gives this extremal price shock achieving this effect.
This gives us a useful perspective on the meaning of a large $|\lambda_1|$. Note that if there were no demand spillovers across products (i.e., all products were independent), then by our normalization we would have $\bm{D}=-\bm{I}$ and each good's quantity would change by an amount equal to the price change. When there are demand spillovers across products, and we change prices in the direction of $\bm{u}^1$, each good's quantity changes by $-|\lambda_1|$ times the price change, a large negative multiple. Hence, the case $|\lambda_1| > 1$ indicates that there are price changes where the downward effect on quantities is larger than in the independent case. This corresponds to complementarities, in that reductions in demand reinforce one another.\footnote{In contrast, when $|\lambda_1| < 1$, the effect on demand is lower than in the independent case, corresponding to an economy where products are globally substitutes.}
This interpretation of the dominant eigenvector and its eigenvalue extends to the other eigenvectors and eigenvalues of $\bm{D}$. Let $\bm{u}^{\ell}$ be the eigenvector associated with the $\ell^{\text{th}}$ largest eigenvalue, denoted $\lambda_{\ell}$. A direct implication of the Courant--Fischer theorem is that the price change that maximizes the change in quantities relative to the change in price across all price changes orthogonal to ${\bm{u}^1, \bm{u}^2, \ldots, \bm{u}^{\ell-1}}$ is exactly $\bm{u}^\ell$, and $$ \sup_{\dot{\bm{p}} \perp {\bm{u}^x}{x=1}^{\ell-1}}\frac{\Vert \dot{\bm{q}} \Vert}{\Vert \dot{\bm{p}}\Vert}=|\lambda_\ell|. $$ Hence, the spectral decomposition of $\bm{D}$ captures a set of $n$ orthogonal price changes in the economy, each representing the maximum induced change in quantity that is feasible within the system given the orthogonality constraint.
We now make explicit the relationship between the general recoverable structure condition and the stochastic block model. In the stochastic block model, fixing $\bar{\bm{D}}$, $\bar{\bm{q}}$, and a sequence $\gamma(n)$, the set of market states is the set of all $(\bm{D},\bm{q}^0)$ (for any number of firms) satisfying the block specification ((ref)) for some function $k$ partitioning products into types.
The intuition is straightforward: because there are only finitely many types, some blocks in $\bm{D}$ must be large; as in (ref), a large block with entries of order $\gamma(n)$ gives rise to an eigenvalue of magnitude $\gamma(n)n$. The proof also shows that the projection onto the corresponding eigenspace of $\bm{q}^0$ arising from a generic $\bar{\bm{q}}^0$ has norm bounded below by a number independent of $n$.
To see how the presence of aggregate structure relates to large-scale complementarities in the example, recall that we assumed $\bar{\bm{D}}$ itself satisfies NSD, and so its diagonal entries are $-1$. This means that there is a large block on the diagonal of $\bm{D}$ with negative entries of magnitude $\gamma(n)$---large-scale complementarities just as in (ref).
Thus, we can now keep the stochastic block model in mind as a canonical example of recoverable structure with $M(n) \gg \sqrt{n}$.
When recoverable structure is present, information about the top eigenvectors of $\bm{D}$ will be recoverable with high precision---under suitable assumptions on errors---using standard statistical results. We now describe how an authority can use information solely about top eigenvectors to design an intervention that increases surplus robustly.
The key tool is a decomposition of surplus pass-through in spectral terms, building on a spectral description of the pass-through of an intervention to prices and quantities. Denoting by $\bm{U}$ the matrix whose $\ell^{\text{th}}$ column is the $\ell^{\text{th}}$ eigenvector $\bm{u}^{\ell}$ of $\bm{D}$, and by $\bm{\Lambda}$ the matrix whose non-diagonal elements are zero and whose $\ell^{\text{th}}$ diagonal element is $\lambda_{\ell}$, we have: $$\bm{D}=\bm{U}\bm{\Lambda}\bm{U}^{\mathsf{T}}.$$
An intervention $\bm{\sigma}$ that subsidizes (or taxes) a single product will in general affect not only the prices and quantities of that product but also those of other products, whose equilibrium values are all connected through strategic interactions. If we think of the eigenvectors $\bm{u}^\ell$ as representing bundles, then these bundles have the important property that an intervention $\bm{\sigma} \propto \bm{u}^\ell$ in the direction of such a bundle will only affect the price $\bm{u}^\ell \cdot \bm{p}$ and quantity $\bm{u}^\ell \cdot \bm{q}$ of that bundle, leaving the prices and quantities of the bundles corresponding to the other eigenvectors unchanged. Generally, we can decompose $\bm{\sigma}=\sum_\ell (\bm{u}^{\ell}\cdot \bm{\sigma}) \bm{u}^\ell $ into a combination of $n$ orthogonal interventions, each in the direction of an eigenvector. We can use this decomposition to solve the oligopoly game and obtain simple expressions for the pass-through of the intervention in terms of the eigenvalues of $\bm{D}$.
Thus, we can study the price and quantity pass-throughs of each of these $n$ interventions separately across eigenvectors. In particular, each unit of subsidy in direction $\bm{u}^{\ell}$ exclusively passes through to the price and quantities of bundle $\bm{u}^\ell$, and it does so with coefficients $-(1+|\lambda_\ell|)^{-1}$ and $|\lambda_\ell|(1+|\lambda_\ell|)^{-1}$, respectively.
Note that the magnitudes of the price and quantity pass-throughs in the different $\bm{u}^\ell$ are ordered according to their corresponding eigenvalues: The larger is $|\lambda_{\ell}|$, the less a given subsidy $\bm{u}^{\ell}\cdot \bm{\sigma}$ reduces prices, but the more it increases quantities. This asymmetry is the result of two opposing forces: On the one hand, the strategic interactions among firms imply that the equilibrium price $\bm{u}^{\ell}\cdot \bm{p}^0$ is less sensitive to the subsidy $\bm{u}^{\ell}\cdot \bm{\sigma }$ the larger is $|\lambda_{\ell}|$. On the other, the demand $\bm{u}^{\ell}\cdot \bm{q}^0$ is more sensitive to the price $\bm{u}^{\ell}\cdot \bm{p}^0$ the larger\footnote{Indeed, it follows from ((ref)) that $\bm{U}^{\mathsf{T}}\bm{q}^0=\bm{\Lambda}\bm{U}^{\mathsf{T}}\bm{p}^0$, so the slope of the demand $\bm{u}^{\ell}\cdot \bm{q}^0$ with respect to own price $\bm{u}^{\ell}\cdot \bm{p}^0$ is equal to $\lambda_{\ell}$.} is $|\lambda_{\ell}|$; this is just a fact about the market's demand function, rather than equilibrium pricing. (ref) shows that the second effect dominates the first in the sense that the larger is $|\lambda_{\ell}|$, the more sensitive is the equilibrium quantity $\bm{u}^{\ell}\cdot \bm{q}^0$ to the subsidy $\bm{u}^{\ell}\cdot \bm{\sigma}$.
We now combine the spectral decomposition with the surplus formulas to deduce the following second lemma.
(ref) shows that the effect of an intervention on consumer, producer, or total surplus is a weighted sum of pass-throughs to each of the eigenvectors $\bm{u}^\ell$---with the weight being the corresponding bundle's quantity.
We now use (ref) to illustrate how---by setting $\bm{\sigma}$ equal to $\bm{u}^1$---the authority may achieve the highest total surplus per dollar spent possible subject to the constraint that the change in consumer surplus is not negative (recall (ref)). Because $\bm{u}^1$ is orthogonal to all the other $\bm{u}^\ell$ with $\ell\neq 1$, such intervention changes only the price and quantity of the bundle $\bm{u}^1$: $$ \bm{u}^1\cdot \dot{\bm{p}}=-\frac{1}{1+|\lambda_1|} \quad \text{and} \quad \bm{u}^1\cdot \dot{\bm{q}}=\frac{|\lambda_1|}{1+|\lambda_1|}, $$ leading to an overall change in consumer and producer surplus equal to
Recoverable structure requires that $|\lambda_1| \to \infty$ with $n$, and so (by inspection of the equations) the interventions will achieve $\dot{C}/\dot{S} \to 0$ and $\dot{P}/\dot{S} \to 2$.
In (ref), we reported a complete information benchmark that shows what is possible with no observation errors. In (ref), we provided an example in which a natural constrained-efficient outcome among these is achievable under considerable observation errors. This section presents our main result, which generalizes this example.
We now add to our maintained assumptions the following assumption about the authority's signal. Let $\Vert \bm{E} \Vert $ denote the spectral norm of a matrix $\bm{E}$, which for a symmetric matrix is equal to its largest eigenvalue. Fix a sequence $b(n)$ and a positive constant $\overline{V}$.
The first part of the assumption bounds the matrix norm of the errors in estimating the normalized Slutsky matrix. In (ref), we provide a simple procedure for sampling market data independently across product pairs $(i,j)$ under which this assumption holds with $b(n)= \gamma n^{1/2}$ (for a positive constant $\gamma>0$). We can think of this example as generating $E_{ij}$ that are essentially independent, with variance of constant order (i.e., neither growing nor decaying with $n$). This shows that the assumption can hold even when there is no entry of $\bm{D}$ that can be recovered accurately.
If $b(n) = \gamma n^\beta$ with \(\beta \in (1/2, 1) \), there are error structures consistent with (ref) where some terms in $\widehat{\bm{D}}$ have large covariances. For example, spatially correlated errors, with sufficiently “distant” parts of the matrix being at most slightly correlated, would satisfy this assumption.\footnote{The idea is analogous to ergodicity-type conditions in time-series settings.} An example that would violate part (1) of (ref) is the entries of $\bm{E}$ all having correlation bounded away from zero (e.g., arising from a common shock to measured complementarities).
The second part of the assumption requires independence of errors across different products' quantities. The purpose of the assumption is to apply a law of large numbers for estimating outcomes such as the average quantity, as well as various linear combinations of quantities that are important for intervention outcomes. Independence is stronger than we need for our main result stated in (ref), and is made to facilitate exposition. When we use this assumption in the proof of our main result, (ref), we rely on a substantially weaker but more technical condition that (ref)(2) implies (see (ref)). Regarding the assumption on the norm of $\bm{\varepsilon}$, recall that we assume that $\Vert\bm{q}^0\Vert \leq 1$; the assumption on $\bm{\varepsilon}$ scales the error to be of the same order of magnitude as the quantity vector.
Our main result, (ref), examines what can be implemented robustly when the authority has partial information and the economy has a recoverable structure that is strong enough relative to the noise. It shows that interventions exist that robustly protect consumers from surplus loss and implement market surplus equal to the upper bound achievable by an omniscient authority (the upper bound given by (ref)(2)).
Recall that we have fixed a sequence $b(n)$ under which (ref) holds---an upper bound on the noise in observations of the demand system.
The condition in (ref) stipulates that the market has $( M(n),\delta)$-re\-cover\-able structure for some $M(n)$ that asymptotically dominates $b(n)$. This lower bound on $M(n)$ ensures that the recoverable structure in $\bm{D}$ is substantially larger than the norm of the error matrix $\bm{E}$, which is $O(b(n))$ under (ref). Note also that the assumptions on errors are satisfied by the stochastic block model of (ref), so (ref) is a direct corollary of (ref).\footnote{This follows since the entries of $\bm{E}$ were taken to be independent dallaporta2012eigenvalue.}
Part (i) states that, when this condition is satisfied, the authority can robustly achieve approximately two dollars of surplus gain per dollar spent.\footnote{Recalling the definition $\dot{W} = \dot{P} + \dot{C} - \dot{S}$, this implies that every dollar spent yields approximately one unit increase in $\dot{W}$, net total surplus.}
Point (ii) states that it is possible to achieve this while leaving all households' welfare essentially unchanged; indeed, under the policy we construct, producers fully capture the surplus gains. Note that by (ref), the welfare gain described in point (i) is essentially the maximum total surplus change that an omniscient authority could implement with the same expenditure without reducing consumer surplus.
Finally, point (iii) says the authority can precisely target the realized expenditure (and thus, total surplus impact) of the policy.
Our notion of $\epsilon$--robustness means that these surplus properties are achieved ex post with high probability. A natural question is whether they are also achieved in expectation (since, in principle, realizations with very negative surplus could occur with low probability). In our setting, it turns out that all the analysis would be unaffected if we added good ex ante expected performance to the definition of robustness.\footnote{This is because the surplus pass-throughs are supported on $[0,1]$ and all quantities in the proofs are bounded. Thus, convergence in probability is equivalent to convergence in $L^1$, and so our proofs extend to show close approximations to the omniscient benchmark in terms of ex ante surplus.}
To prove (ref), we apply a statistical method that accurately identifies a subspace spanned by top eigenvectors of the Slutsky matrix from noisy observations. We then show that interventions projecting exclusively onto these recoverable subspaces possess the desirable welfare properties stated in (ref).
Recall that if $M(n) \gg b(n)$, then $(M(n),\delta)$-recoverable structure requires that the normalized Slutsky matrix $\bm{D}$ has eigenvalues that, in absolute value, are much larger than $b(n)$; we will refer to such eigenvalues simply as “large” from now on.
The key tool in our statistical exercise that leverages this assumption is the Davis--Kahan theorem. Under the hypothesis that some eigenvalues of $\bm{D}$ are large, this theorem guarantees that, despite the noise in $\bm{E}$, the large eigenvalues of the observed matrix are good approximations of the true large eigenvalues. In other words, the noise in $\bm{E}$ cannot cause the large eigenvalues of $\bm{D}$ to become “mixed up” with the eigenvalues far away in the spectrum; see Figure (ref) for an illustration. The theorem also permits the recovery of eigenvectors. More precisely, this theorem has the following two central implications in our setting:
This powerful result forms the core of our strategy to recover and use structure underlying the oligopoly demand robustly under noise. To facilitate the illustration, we impose a stronger assumption on $\bm{D}$: that the largest eigenvalue of $\bm{D}$ is sufficiently well-separated from all other eigenvalues by a “gap” much larger than $b(n)$. Under this condition, the Davis--Kahan Theorem yields an even stronger implication: we can use $\widehat{\bm{D}}$ to recover a normalized eigenvector $\widehat{\bm{u}}^1$ that correlates almost perfectly with the corresponding eigenvector\footnote{In (ref) we develop (ref) to illustrate our main result. (ref) illustrates the similarity between the true $\bm{u}^1$ (in panel B) and the estimated $\widehat{\bm{u}}^1$ (panel D) for a case where this stronger “gap" condition holds.} $\bm{u}^1$.
To use this, observe that (ref) and (ref) together imply
while expenditure is
Let us use the recovered $\widehat{\bm{u}}^1$ to design an intervention with $\bm{\sigma} \propto \widehat{\bm{u}}^1$. The Davis--Kahan Theorem allows us to treat $\widehat{\bm{u}}^1$ as effectively equal to its true counterpart $\bm{u}^1$ with a very small error, so from now on we will treat $\bm{u}^1$ as known. By choosing the sign of $\bm{\sigma}$ to ensure $(\bm{u}^1 \cdot \bm{q}^0)(\bm{u}^1 \cdot \bm{\sigma})$ is positive. Then we can see from the equations above that $\dot{W}$ will closely approximate $\dot{S}$, since $\lambda_1 \gg b(n)$ implies $\frac{|\lambda_1|}{1+|\lambda_1|} \approx 1$. Moreover, if we know $\bm{u}^1 \cdot \bm{q}^0$ and this differs from zero (which is a requirement of recoverable structure), we can scale the intervention to be of the size that we desire, and achieve $\dot{S}=s$.
This argument contains some wishful thinking, however. When we arranged the sign of $\bm{\sigma}$ so that $(\bm{u}^1 \cdot \bm{q}^0)(\bm{u}^1 \cdot \bm{\sigma})$ is positive, we did not consider that we have only a noisy observation $\widehat{\bm{q}}^0$ of $\bm{q}^0$. So part of the challenge of the proof is to manage the observation error that makes $\widehat{\bm{q}}^0$ different from ${\bm{q}}^0$, and to show that we can obtain a correct estimate of the sign with probability tending to $1$ as $n\to\infty$. If we fail to do this correctly, our intervention actually decreases efficiency with positive probability. This explains the need for the second part of Assumption (ref) on the error $\bm{\varepsilon}$ in the quantity signal.
However, bounded noise alone is insufficient: if $\bm{u}^1 \cdot \bm{q}^0$ is very small, there may be no hope for consistently recovering the true magnitude or sign of $\bm{u}^1 \cdot \bm{q}^0$ from the signal $\widehat{\bm{u}}^1 \cdot (\bm{q}^0 + \bm{\varepsilon})$ even with well-behaved noise: the asymptotically small noise could still overwhelm a similarly decaying underlying mean $\bm{u}^1 \cdot \bm{q}^0$. Such an unrecoverability would make it impossible to orient and scale our intervention appropriately. The definition of recoverable structure prevents this problem by requiring that the projection of $\bm{q}^0$ onto eigenvectors with large eigenvalues is bounded away from zero.
The special case of our main result that this discussion makes plausible is: If the largest eigenvalue of $\bm{D}$ is well-separated from others and if $\bm{q}^0 \cdot \bm{u}^1$ is not vanishingly small, then a subsidy profile proportional to $\bm{u}^1$ can, if it is suitably scaled, achieve all the properties of (ref).
The proof of the main result improves on this sketch in two ways. First, it does not rely only on the eigenspace spanned by $\bm{u}^1$. Instead, it uses a potentially much larger eigenspace of $\widehat{\bm{D}}$. The general intervention projects $\widehat{\bm{q}}^0$ onto $\mathcal{L}(\bm{D},M(n))$, the eigenspace of all eigenvectors of $\bm{D}$ with eigenvalues larger than $M(n)$. This makes it easier for the analog of $\bm{u}^1 \cdot \bm{q}^0$ not to be too small, since the projection of $\bm{q}^0$ onto a larger eigenspace will have a larger norm. Second, the general proof dispenses with assuming that any eigenvalues are well-separated. Instead, it handles any possible spectrum of $\bm{D}$ subject to our maintained assumptions. This introduces considerable complexity, as it is no longer generally possible to recover any true eigenvector $\bm{u}^\ell$ with any accuracy. We instead work directly with a recovered eigenspace that generalizes the span of $\widehat{\bm{u}}^1$. We show that despite limited knowledge of individual true eigenvectors of $\bm{D}$ underlying this space, we can use the fact that all of them have large eigenvalues to generalize our argument for showing that ((ref)) and ((ref)) can be made very close and nonzero with a feasible intervention. This is where the arguments go beyond standard applications of the Davis--Kahan theorem.
We begin this section by providing an example illustrating the notion of recoverable structure, and then use this example to demonstrate the effects of the intervention rule that taxes and subsidizes firms in the direction of the eigenvector associated with the largest eigenvalue of the observed Slutsky matrix.
In the context of Example (ref), we focus on the following intervention rule: Recover the eigenvector associated to the largest eigenvalue, in absolute terms, of the estimated Slutsky matrix $\widehat{\bm{D}}$. Intervene to subsidize firms in proportion to this eigenvector. This intervention aligns with the intervention rule behind our Theorem (ref).\footnote{In this example, for low values of $\gamma$ the largest eigenvalue of $\bm{D}$ is sufficiently well-separated from all other eigenvalues and, consequently, the authority can use the first eigenvector of $\widehat{\bm{D}}$ as a good approximation of $\bm{u}^1$ (see the illustrative discussion in (ref)).}
In order to meaningfully compare the effects of such intervention for different realizations of the market state, we scale the size of all interventions that we consider by requiring that they have the same expenditure based on the observed quantity vector.\footnote{More precisely, we first project the observed quantity vector onto the recovered eigenvector, and use that to predict the expenditure size.}
The true initial quantity vector $\bm{q}^0$ has some regular block structure but also some idiosyncratic heterogeneity. It is constructed as follows: \[ q^0_i=(\bm{q}_{\text{block}})_i {X}_i\] Here, the quantity vector $\bm{q}_{\text{block}}$ provides a base quantity for each product that depends on its associated block ($0.1$ for products in the first two blocks, and $3$ for the products in the third block). The random variable ${X}_i$ is drawn independently of all others, and its logarithm is normal with variance $0.1$ and mean $1$. We use a multiplicative perturbation to avoid negative quantities.
The observed quantities are given by $$\widehat{q}^0_i={q}_i^0 Y_i $$ where $Y_i$ is an independent error with the same distribution as $X_i$. This can be rewritten in terms of our additive error model, with $\varepsilon_i = q^0_i (Y_i-1)$. Here again, the multiplicative error model avoids negative quantities.
We consider different values of $\gamma\in[0,1]$. For each of these values we generate $3000$ market states according to the above description and we compute the changes in consumer and producer surplus under the true market state. (ref) summarizes this exercise: for each value of $\gamma$ considered, it reports the median (blue dot) of the change in consumer surplus (panel A) and of the change in producer surplus (panel B) and the respective 5th and 95th percentiles associated with the $3000$ market state realizations.
(ref) shows a sharp transition in the performance of the intervention. For $\gamma$ less than roughly $0.7$ the true market state has very large eigenvalues and so the authority can use the estimate of $\bm{D}$ to precisely identify the underlying main eigenvector(see (ref) for $\gamma=0.3$.) Note also that products in category $3$ are the ones that are most substitutable with other products in categories $1$ and $2$ and so they are highly represented in the first eigenvector (i.e., they have a high eigenvector centrality). This implies that the estimated first eigenvector is sufficiently correlated with the status quo market quantity. Hence, for low $\gamma$, the true market state has recoverable structure. This allows the authority to implement interventions that robustly have the pass-through properties characteristic of high-eigenvalue eigenvectors---negligible impact on prices and hence on consumers, along with an effect on producer surplus equal to twice the authority's spending , which is normalized to one unit.
However, as $\gamma$ grows larger than $0.7$, the property of recoverable structure fails (see (ref) for $\gamma=0.9$) with the consequence that an intervention that taxes and subsidizes firms based on the estimated first eigenvector is very unpredictable and risky. The unpredictability is shown by the fast widening of the error bars as $\gamma$ increases beyond $0.7$. The riskiness is shown by the fact that for over a third of the outcomes, the realized change in producer surplus, and hence in total surplus, is negative.
In this example, the demand for products in block 3 is significantly higher than the demand for the other products. Hence, to a first approximation, consumer surplus increases when the price of products in block 3 decreases, while producer and total surplus increase when the quantity of these products increases.
When the authority can estimate the first eigenvector accurately, the first eigenvector intervention turns out to subsidize products in block $3$ and tax all the other products. Subsidizing products in block 3 leads to a decrease in their price and hence an increase in their demand. Taxing products in blocks 1 and 2 leads to an increase of the price of these products, and hence an increase in the demand of products in block 3. Combining the effects, the intervention leads to a relatively high increase in the demand of good 3, while keeping prices roughly constant. As a result, both producer and total surplus increase dramatically without sizable changes in consumer surplus.
This management of the spillovers can be achieved only by statistically identifying some relevant latent market structure from the noisy demand measurements. While in this example that structure takes the simple form of product categories, in general it might be less easy to describe, and yet equally useful for the design of robust interventions.
We have established that if demand has recoverable structure, the authority can robustly achieve the maximum possible total surplus per dollar spent subject to the constraint that consumers are not harmed. The associated intervention rule boosts production with minimal price changes, resulting in firms capturing all efficiency gains.
This raises two natural questions. First, can we find interventions that robustly increase total surplus when the demand does not have recoverable structure? Second, when there is recoverable structure, is the structure of robust interventions that we have just described in any sense necessary? For example, could we have found interventions that robustly increase consumer surplus, rather than leaving it unchanged?
In this section, we work with the case we have focused on in other illustrations, where $\bm{E}$ has i.i.d. entries with a standard deviation that does not depend on $n$, giving $b(n) \sim n^{1/2}$; this choice is immaterial and the results apply to a wide range of alternative noise structures.
In each case, (ref) describes the limits on what can be achieved by an authority with noisy information. It also notes that these limits really are about information: part (ii) of each case states that an omniscient authority would not be subject to the same limitation.
In more detail, Part 1 of (ref) tells us that we may not be able to design interventions that robustly increase total surplus $\dot{W}=\dot{C}+\dot{P}-\dot{S}$. (Since, by (ref) we know this can be done under the recoverable structure assumption, our construction must lack recoverable structure.) Intuitively, in this case the information about the market state learned from the signal can be very imprecise and, therefore, there are market states in which any intervention will lead to undesirable outcomes. The proof constructs a set of market states such that, with an uninformative signal, for any intervention there is a market state with $\dot{W}<0$. The basic idea is to use the total surplus decomposition:
and construct the example so that the authority cannot accurately predict the signs of the terms for any given $\bm{\sigma}$.
Part 2 of (ref) tells us that even if the market state has recoverable structure, it may be impossible to design interventions that robustly increase total surplus and allow consumers to capture some of the resulting efficiency gains. (ref) tells us that to achieve such an outcome, the intervention must project onto some $\bm{u}^{\ell}$ where $\lambda_\ell$ is not too large, since only those eigenvectors have nonvanishing pass-through to consumer surplus. However, the noisy observation of $\widehat{\bm{D}}$ and $\bm{q}^0$ gives very noisy estimates of the constituents of ((ref)) corresponding to these eigenvectors. So, by targeting them, there is a substantial chance (at least in some environments) that the policy will have negative consequences for consumers.
We have developed a theory of robust interventions in large oligopolies. We identify a condition on demand under which an authority can robustly increase the total surplus per dollar spent as much as would be possible under perfect information subject to the constraint that consumers are not harmed. The methodological contribution lies in developing spectral methods to analyze pass-through in oligopolies, and applying these methods to gain leverage on statistical problems about oligopolies observed with noise.
We conclude with some observations about the scope of our analysis and connections to related research.
In our model, the authority has “big data” about the Slutsky matrix $\bm{D}$ that may not allow precise estimation of any pairwise demand interactions or hedonic model parameters. This is natural for markets with a large and changing collection of goods, such as those hosted on large online marketplaces. Such marketplaces collect immense amounts of data---about consumer browsing behavior, timing of purchases, consideration sets, etc.---and apply machine learning techniques to these data to form informative but imperfect estimates of interactions among various goods \citep*{athey2018impact,wager2021experimenting,cai2022recommender,bajari2023experimental}. This is modeled by our notion of a market signal.\footnote{The distributional properties of $\bm{E}$ describing the errors in these estimates would depend on the application, as would the conditions for $\bm{E}$ that would bound its norm. It would be interesting to investigate these issues in specific applications.} Our paper offers an approach for calculating suitable statistics that suffice for effective interventions despite the noise in this signal.
Our approach contrasts with a standard one in empirical industrial organization, where markets tend to be defined tightly so that each contains only a relatively small number of similar goods with strong demand interactions, and then a small number of hedonic parameters and demand elasticities are precisely estimated. That approach would correspond in our notation to a very precise signal (i.e., an error matrix $\bm{E}$ with small norm) and a small number of products.
We have focused on total market surplus as a canonical objective. But we also saw that our robust intervention maximizes the change in total surplus by increasing producer surplus while holding consumer surplus constant. In this sense, the intervention maximizes the increase in producer surplus under the constraint that consumers do not lose---a reasonable objective for an operator of a marketplace that collects revenue proportional to sellers’ profits. We leave the study to other objective functions for the authority to future research.
Our model permits flexible marginal interventions. This modeling choice is suited to online marketplaces, because the operators in charge of them can finely target policies that function as taxes and subsidies, including commission rates, discount coupons, free advertising, etc.---and regularly experiment with such perturbations. It is worth noting, however, that the policies our analysis recommends need not be specific to individual products. This is because when we take a large matrix (in our case, the Slutsky matrix) reflecting relationships among units (in our case, products) and look at eigenvectors with large eigenvalues, the coordinates of those eigenvectors typically yield low-dimensional embeddings capturing substantively natural categories \citep*{chen2021spectral}. For instance, in our Example 1, the top two eigenvectors are sufficient to recover the blocks to which the goods belong. Relatedly, spectral clustering analyses based on the top few eigenvectors sort items into natural “similarity” classes, where similarity is defined by having similar relationships to other classes spielman1996spectral. Once again, the spectral statistics used in these techniques tend to pick up interpretable “broad” features of the products, rather than idiosyncrasies specific to individual products. As a result, the policies our interventions recommend---which project all variation onto these eigenvectors---will often be close to a policy that depends mostly on category---e.g., a subsidy on smartphones along with a tax on certain types of accessories. Though the policies will not be perfectly regular (note the irregularities of Panel D of (ref)) the above observations lead us to conjecture that an authority constrained to design policies that discriminate only at a coarse product level could, under natural assumptions, achieve a substantial amount of the gains of our policies. We leave these interesting considerations to future work.
Lastly, we mention the problem of predicting the effects of general perturbations to markets in settings such as ours. This is related to, but distinct from, the problem we have studied. The problem of robust intervention is importantly easier, because the authority chooses the perturbation to make a prediction about. Nevertheless, the spectral decomposition of intervention effects appears likely to be useful for descriptive comparative statics in cases where a cost shock $\bm{\sigma}$ or other change is exogenous.
Recent work by pelligrino2021 and edererpelligrino2021 uses an oligopoly model to empirically quantify the evolution of market power. The relationship between our model and their work sheds light on the types of empirical models that can capture recoverable structure.
In pelligrino2021, the model of demand is hedonic, in the spirit of lancaster1966new: the household's utility is additively separable in the contributions of various characteristics, and a product provides a bundle of these characteristics. The Slutsky matrix $\bm{D}$ derived from this demand model can be expressed as a transformation of the cosine similarity matrix of products' characteristics, which hoberg2016text estimated for a large set of consumer goods using text data.\footnote{pelligrino2021 and edererpelligrino2021 consider quantity competition, but the Slutsky matrix does not depend on this choice.} We have calculated that in the Slutsky matrix derived this way, the eigenvalues are all small and the recoverable structure condition fails.\footnote{There are more than $3000$ products in the data and for the case of i.i.d. noise we would require that the largest eigenvalue is considerably larger than $b(n)=\sqrt{n} \approx 54$; this fails as largest eigenvalue of $\bm{D}$ in absolute value is about $2$.} It is useful to reflect on why this is the case.
In the model of pelligrino2021, a simple calculation shows that it is impossible for the Slutsky matrix to have large eigenvalues.\footnote{The (un-normalized) Slutsky matrix in pelligrino2021 is $-\bm{B}^{-1}$, where $\bm{B}=\bm{I} + \alpha(\bm{\Sigma}-\bm{I})$. The matrix $\bm{\Sigma}$ is positive semidefinite because it can be written as $\bm{V}^\mathsf{T} \bm{V}$, where the columns of $\bm{V}$ are the characteristic vectors of various products. Thus all eigenvalues of $\bm{B}$ are real numbers bounded below by $1-\alpha$, and all eigenvalues of $-\bm{B}^{-1}$ are at most $1/(1-\alpha)$ in magnitude. Pellegrino uses the value $\alpha=0.12$, which prevents any eigenvalue from exceeding $1.13$. We do not work with exactly the same Slutsky matrix because of the normalization in (ref). Its eigenvalues are a bit different, but they can still be bounded by a constant by elaborating this argument. Numerically we see that the normalization makes little difference. } For an economic intuition, note that in the lancaster1966new type of model, the “direct” relationship between any pair of goods is substitution. With substitution, if some demand is diverted from one good due to an increase in its price, the total effect on all substitute goods is bounded, since, loosely speaking, the demand gained by these other goods must come out of the demand lost by the more expensive one. This bounds the sum of positive entries in $\bm{D}$ corresponding to this effect, which in turn bounds any complementarities in the Slutsky matrix.\footnote{Note that complementarity (where one good's demand decreases in the price of the other) can arise in pelligrino2021 model. This happens through indirect effects: the substitute of my substitute can be my complement. However, since the “direct” substitution effect is bounded in magnitude, so are the indirect consequences.} In essence, in a hedonic model where the basic force is substitution, overall spillovers remain bounded, and the fact that $\bm{D}$ has no large eigenvalues is the mathematical manifestation of this.
Quite different behavior emerges in models where utility arises directly from consuming goods together, and such complementarities are central to our examples of recoverable structure. A leading practical example comes from the use of computers: a consumer's utility from a computer depends on the hardware, operating system, and applications. Two firms selling distinct components---a hardware device and an operating system, for instance---supply complementary goods, while two firms selling the same component (say, operating systems) supply substitute goods matutes1988mix,matutes1992compatibility. Our illustrative (ref) in (ref) shows how Slutsky matrices with large eigenvalues arise naturally in such settings.\footnote{The complementarities there happen to be within-category, but that is not important for our point here.} But, as we have seen, it is impossible to produce the same patterns in models of the lancaster1966new type, because they cannot generate large eigenvalues; one would need to incorporate terms reflecting that some characteristics provide greater value when enjoyed together. There is a straightforward economic intuition for why such complementarities more readily produce recoverable structure: when one good's price decreases, all its complements can experience comparable nonvanishing increases in demand. This creates the clusters of nonvanishing entries in $\bm{D}$ that are the hallmark of recoverable structure.
In summary, direct complementarities seem practically important and can naturally yield the recoverable structure central to our results. We hope these observations will motivate further empirical research on the structure of large-scale oligopoly models with complementary goods.
One can view our exercise as a special case of an intervention, under noisy information, in a game among a large number of agents. In our case, the game comes from a standard oligopoly pricing model. Under the assumption of linear demand, the pricing game can be seen as a network game with linear best replies where the Slutsky matrix defines the network. Our analysis shows that if the oligopoly exhibits recoverable structure, then there are robust interventions for particular economic objectives. The methods we have developed can be extended to other settings. For example, in a public goods setting, interventions would aim to realign private marginal returns with social marginal returns. The literature has developed tools to understand how to do this when the authority has precise information on the spillovers causing the underprovision of public goods.\footnote{See, for instance, bramoulle2014strategic. } However, we know little about designing interventions under noisy information about such externalities. Similarly, in contracting for teams under moral hazard, network methods have recently been developed for locally perturbing contracts to achieve better outcomes for a principal IncentiveSpillovers. But it is a considerable challenge to extend these results to the realistic case where the strategic interactions among members of an organization are only imperfectly known. General games will lack some of the structure we have leveraged, including the properties of the spillovers structure coming from a symmetric, positive semidefinite Slutsky matrix. So there are challenges to overcome in extending our results. We hope this paper stimulates research in these directions.
We have assumed that demand is exactly linear in a neighborhood around the status quo equilibrium point. This assumption implies that the pass-through of an intervention to prices and quantities and, therefore, to welfare, depends only on the Slutsky matrix $\bm{D}$. We use this simplification to develop new concepts useful for robust market interventions. These concepts can be extended to nonlinear demand settings. We briefly explain how.
When demand is not locally linear, the pass-through of marginal cost shocks depends not only on the Slutsky matrix (which is the Jacobian of demand) but also on the Hessian of the demand function, the matrix whose $(i,j)$ entry is $\partial^2 q_i(\bm{p})/\partial p_i \partial p_j$ (see, e.g., miklos2021pass). Thus, our calculations would change, and there is no guarantee that our linear tax/subsidy interventions (based only on the Slutsky matrix at the status quo) would perform as they do in the linear model.
However, our main result can be extended once we allow the authority to use nonlinear interventions, i.e., to commit a vector of functions specifying a payment to each producer $i$ as a function of all prices and quantities realized after the intervention. With this broader set of instruments, the authority can use nonlinear rebates based on post-intervention quantities to reduce the problem to the one we have studied. The key idea is to effectively linearize the demand the firms face around the status quo by using transfers to make up the difference between realized demand and a linear demand function. Once demand has been “linearized” in this way, the problem that firms face becomes equivalent to the one we have studied and we can use the results developed to design per-unit tax/subsidy interventions with desirable welfare properties. If we assume that the curvature of the demand of each product is locally bounded by a known constant, the payments needed to linearize demand can be bounded by a small fraction of the first-order gains of an intervention, so our welfare guarantees remain valid. Such assumptions on curvature also allow us to specify concrete sizes of interventions that achieve a given level of welfare gain, rather than just characterizing the behavior of derivatives.