Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
12,179 characters · 6 sections · 31 citation commands
-3emVariational Bayes and non-Bayesian Updating
Let $S$ be a finite state space (parameter space) and $p\in \Delta(S)$ be the prior of the agent. The agent gets information: let $X$ (finite) be the set of possible signal realizations (messages). Let $f:S\to\Delta(X)$ be the likelihood function (a theorist may call $f$ a Blackwell experiment). Let $q_x\in \Delta(S)$ be the agent's belief upon observing data $x$. As we all know, the Bayes rule says that
The denominator, the marginal on $x$, is often hard to compute in practice when $X$ is continuously distributed and the sum is an integral. In the next note I will talk about one particular computational approach to this problem, called variational Bayes. For now, to get rid of the denominator, define $\propto$ as “equal up to a multiplicative factor” and write (ref) as
Researchers in psychology have found that experimental subjects deviate from the Bayes rule in systematic ways.\footnote{edwards1964man; phillips1966conservatism; tversky1971belief; kahneman1972subjective.} Such deviations are classified as normatively incorrect (biases, errors, etc.). Benjamin19 reviews this evidence and uses the following two-parameter generalization of (ref) as a descriptive model to categorize various errors in probability updating. AR also use this model to obtain a similar classification.
For lack of better terminology I will call this the exponential updating rule
where $\alpha, \beta >0$. The exponent $\alpha$ measures how much the agent over/under weighs the prior and the exponent $\beta$ measures how much they overweigh the evidence. In particular, $\alpha<1$ is classified as underinference (or conservatism), $\alpha>1$ as overinference, $\beta<1$ as base-rate neglect, and $\beta>1$ as confirmation bias. As noticed by Grether80 if the researcher runs a log linear regression of posteriors over priors and likelihoods (without an intercept), then they are de facto estimating a exponential updating rule.
Interestingly, the case of $\alpha=1$, $\beta<1$ has been proposed as a normative model in Bayesian statistics\footnote{ Grunwald12; BissiriHolmesWalker16; Bhattacharya19.} and machine learning.\footnote{Huang2018improving; higgins2017beta; Pepe.} Thus, what would be characterized by a behavioral economist as an error in probability updating, to a statistician may be an appealing way to guard against model misspecification. This way of modeling concern for misspecification is distinct from the economics literature, where the focus is not directly on misspecification of the likelihood function $f$, but on misspecification of the of predictive posterior distribution.\footnote{HS; cerreiavioglio2022making; lanzani2002dynamic.}
Consider now a different approach, where instead of an explicit formula for updating beliefs, the agent is choosing her beliefs to minimize some loss function. Fix the prior $p\in \Delta(S)$. Recall the relative entropy (Kullback-Leibler divergence) $D:\Delta(S)\times\Delta(S)\to [0, +\infty]$, given by $$D(q\parallel p)=\sum_{s\in S}q(s)\log\frac{q(s)}{p(s)}$$ when $q\ll p$ and $+\infty$ otherwise.\footnote{We use the convention that $0\log\frac{0}{c}=0$ for any $c\geq 0$. For any $q\in \Delta(S)$ we write $q\ll p$ whenever $p(s)=0$ implies $q(s)=0$ (that is $q$ does not invent new states that $p$ does not believe in). The Bayesian posterior has this property and any alternative updated belief considered by the agent will also have it.} Relative entropy is a notion of “distance” between $q$ and $p$. If we solve $$\min_{q\in \Delta(S)} D(q\parallel p),$$ we will get $q=p$.\footnote{This is known as the “information inequality,” Theorem 2.6.3. of CoverThomas91.}
So far we have gotten nowhere in terms of updating. We started with a prior and our procedure ended up with a prior, so we have not updated at all. This is because we have not used the data yet. Consider now a different optimization problem: $$\max_{q\in \Delta(S)} \sum_{s\in S} q(s) \log f(x|s).$$ Here we are choosing $q$ to best fit the data. The solution will concentrate all its mass on the states that lead to maximum likelihood $f (x|s)$. Maximum likelihood estimation is the classical approach to statistics.
Now the key point is that we get Bayesian updating if we combine the two problems:
Thus, Bayesian updating can be understood as striking a balance between being close to the prior and being close to the evidence. This way of looking at things is known as variational Bayes (peterson1987mean; hinton1993keeping; Bleietal17).\footnote{The adjective “variational” is here because we are solving an optimization problem (by calculus of variations when $S$ is infinite).} To be more precise: It is as hard to solve (ref) as it is to compute (ref) (since we are getting the same answer). The computational advantage in variational Bayes comes from regularizing the problem, which means adding a set of constraints (e.g., $q$ is a product measure on $S=\mathbb{R}^d$) or adding additional terms to the objective function. I will abstract from this and focus on the exact equivalence as in Proposition (ref).
So far we understood that Bayesians are striking a balance between priors and data. But why equal weights? Consider this problem:
where $\lambda>0$ is a parameter.
Here $\lambda$ measures the attitude of our agent toward model misspecification. If the agent has full confidence in the model, she sets $\lambda=1$ and gets Bayes. But with imperfect confidence she might want to set $\lambda<1$. As mentioned before, this has been proposed as a normative model in Bayesian statistics. On the other extreme is $\lambda=\infty$, which as we saw above, corresponds to maximum likelihood estimation, a procedure in classical statistics.
This is exponential updating with $\alpha=1$ and $\beta=\lambda$. Thus, taking Bayes as a point of reference, $\lambda<1$ leads to underweighting the evidence, whereas $\lambda>1$ leads to overweighting the evidence.
Overweighting the evidence seems similar to underweighting the prior. This is true at the level of “preferences over beliefs” represented by (ref): increasing $\lambda$ puts more weight on the evidence and less relative weight on the prior. However, at the level of the updating rule, this is not exactly true: in the exponential updating formula there is a separate parameter, $\alpha$, which measures how much we over/underweight the prior. What force does this correspond to at the level of “preferences over beliefs?” I explore this in the next section.
Recall that the entropy of $q$ is defined by $$H(q):=-\sum_{s\in S} q(s)\log q(s),$$ with the convention $0\log 0=0$. The function $H$ measures how chaotic or uncertain the distribution $q$ is.
Suppose now that our decision maker has a taste for entropy and solves:
for some parameters $\lambda>0, \mu\in \mathbb{R}$.
The case $\mu<0$ (uncertainty loving) was studied by Rava under the assumption that $\lambda=1$ (no model misspecification).\footnote{Formally, their agent minimizes $D(q\parallel q^{Bayes}_x) + \mu H(q)$ for some $\mu<0$.} This agent dislikes having beliefs that are close to certainty. A related notion of extreme-belief aversion was studied by benjamin2016model. On the other hand, if the agent prefers to have simple theories of the world and holding complex beliefs is more costly enke2023cognitive, then we have $\mu>0$ (uncertainty aversion).
The next proposition shows that such an agent either follows the exponential updating rule (if they are not too averse toward uncertainty) or puts all posterior probability on one state (if they dislike uncertainty too much).
Behavior “flips” around $\mu=1$. The objective function is convex as long as $\mu< 1$. For $\mu\geq 1$ the objective function is concave and the minimum value is attained at the extreme points of the simplex. The case of $\mu\geq 1, \lambda=1$ corresponds to Maximum a Posteriori (MAP) estimation in Bayesian statistics, i.e., the mode of the posterior distribution robert2007bayesian.
Dominiak2023inertial study a model of updating where the agent has a state space $\Omega$ and receives information in the form of $E\subseteq \Omega$. Their agent chooses their posterior to minimize a general notion of distance between their posterior and their prior, subject to the constraint that the posterior puts probability one on event $E$. They show that in the case when $\Omega=S\times X$ and $E=S\times \{x\}$, a specific cost function leads to the exponential updating rule considered here. My Proposition (ref) accomplishes the same, but with the added value of decomposing the preferences into attitudes toward model misspecification and toward uncertainty.
zhao2022pseudo studies a related model of updating upon qualitative information, where the update is a solution to minimizing relative entropy. His result is an analogue of Proposition (ref).
Geffroy study non-Bayesian persuasion where the receiver uses a updating rule such that the updated belief is a function only of the prior belief and the correct Bayesian posterior. They show that exponential discounting satisfies this property.
cripps2018divisible studies a general class of updating rules where the order of signal matters. The $\alpha=1$ case of exponential updating satisfies this property, but for $\alpha\neq 1$ the order of signals matters.
EpsteinNoorSandroni study a model where the updated belief is an affine combination of the prior and the Bayesian posterior. In this note the mixing operates in the log-probability space, not the probability space.
In the literature on motivated beliefs agents optimally choose their beliefs subject to some constraints or costs (BrunnermeierParker05; BenabouTirole16). From this point of view, Proposition (ref) identifies the objective function that leads to the benchmark case of Bayes updating.