Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
Rendered from LaTeX for readability, not typeset faithfully. Citation keys are highlighted; maths is left as source; figures, tables and equation environments are summarised rather than reproduced; unrecognised commands are greyed out so nothing is silently dropped. Email addresses are removed.
Measurability of functionals and of ideal point forecasts
abstractAbstract.
The ideal probabilistic forecast for a random variable $Y$ based on an information set $\ensuremath{\mathcal{F}}$ is the conditional distribution of $Y$ given $\ensuremath{\mathcal{F}}$. In the context of point forecasts aiming to specify a functional $T$ such as the mean, a quantile or a risk measure, the ideal point forecast is the respective functional applied to the conditional distribution.
This paper provides a theoretical justification why this ideal forecast is actually a forecast, that is, an $\ensuremath{\mathcal{F}}$-measurable random variable. To that end, the appropriate notion of measurability of $T$ is clarified and this measurability is established for a large class of practically relevant functionals, including elicitable ones.
More generally, the measurability of $T$ implies the measurability of any point forecast which arises by applying $T$ to a probabilistic forecast.
Similar measurability results are established for proper scoring rules, the main tool to evaluate the predictive accuracy of probabilistic forecasts.
Keywords:
Bayes act;
elicitability;
forecast;
information set;
scoring function;
scoring rule
AMS 2020 Subject Classification: 62C99; 91B06
Introduction
A typical aspect of decision making, be it in business, politics, or private life, is that decisions should account for unknown or future events.
Hence, these decisions commonly base on predictions or forecasts for these events.
Fortunately, one regularly has at least partial information about the events of interest, such as regressors or feature variables in a cross-sectional setting or past observations of the event in a time series framework, which help to improve the predictions.
Expressed in terms of mathematical statistics,
the event of interest is a random element $Y$ on a probability space which often attains values in the real numbers, and the given information is modelled as a sub-$\sigma$-algebra $\ensuremath{\mathcal{F}}$ of the $\sigma$-algebra of events.
It has been argued in the literature that probabilistic forecasts, taking the form of probability measures, distributions, or densities, are most informative and should be preferred, since they directly express the uncertainty about the future event Dawid1984, GneitingETAL2007. The ideal or truthful probabilistic forecast of $Y$ based on $\ensuremath{\mathcal{F}}$ is the conditional distribution of $Y$ given $\ensuremath{\mathcal{F}}$, denoted by $\ensuremath{\mathbb{P}}_{Y|\ensuremath{\mathcal{F}}}$.
“However, practical situations may require single-valued point forecasts, for reasons of decision making, reporting requirements, or communications, among others” GneitingKatzfuss2014.
In such situations, one summarises the uncertainty in a typically real-valued functional $T$ of the distribution of $Y$ such as the mean, a quantile, an expectile, or a law-determined risk measure such as Expected Shortfall.
Then, the ideal or truthful point forecast for $Y$ based on $\ensuremath{\mathcal{F}}$ is the functional $T$ applied to the conditional distribution of $Y$ given $\ensuremath{\mathcal{F}}$, that is, $T(Y|\ensuremath{\mathcal{F}}) := T(\ensuremath{\mathbb{P}}_{Y|\ensuremath{\mathcal{F}}})$ NoldeZiegel2017.
This concatenation also turns out to be the main idea in building dynamic or conditional risk measures Weber2006.
Ideal forecasts based on $\ensuremath{\mathcal{F}}$ need to be $\ensuremath{\mathcal{F}}$-measurable random elements, or random variables, for the case of point forecasts.
On the one hand, this ensures that they actually only exploit the information in $\ensuremath{\mathcal{F}}$ and no additional sources. From a more technical perspective, on the other hand, measurability ensures that one can assign probabilities to events induced by forecasts and, more generally, that forecasts are amenable to statistical analysis.
This paper studies when an ideal point forecast based on $\ensuremath{\mathcal{F}}$, or more generally, a forecast which arises by applying the functional to a probabilistic forecast based on $\ensuremath{\mathcal{F}}$, is indeed $\ensuremath{\mathcal{F}}$-measurable. To this end, the appropriate notion of measurability of the target functional $T$, viewed as a map from a class of probability measures to the reals, is clarified. Moreover, we establish corresponding measurability results for many practically relevant examples, including the class of elicitable ones, which can be written as the minimiser of an expected loss function (Theorem (ref)).
Our results provide a formal justification for the $\ensuremath{\mathcal{F}}$-measurability of ideal $\ensuremath{\mathcal{F}}$-based point forecasts which seems to have been taken for granted in large parts of the forecast evaluation literate, sometimes tacitly, sometimes with somewhat incomplete arguments, or which has been established under restrictive assumptions Weber2006, HolzmannEulert2014, NoldeZiegel2017, Pohle2020, FisslerHoga2021, GneitingResin2021, FFHR2021, HogaDimitriadis2021.
From a regression or machine learning perspective where $Y$ is observed together with explanatory variables $X$, our results justify the measurability of the oracle regression function $x\mapsto T(Y|X=x)$ beyond the classical situation of mean regression to quantile regression Koenker2005, in expectile regression NeweyPowell1987 or in Expected Shortfall regression DimiBayer2019.
In the field of sensitivity analysis, measuring the information content with score improvements also calls for the measurability results established in our paper BorgonovoETAL2021, FisslerPesenti2022.
In Section (ref), we provide the formal definitions and concepts of the article. The appropriate concept of measurability of functionals is discussed in Section (ref) which leads to the $\ensuremath{\mathcal{F}}$-measurability of the ideal $\ensuremath{\mathcal{F}}$-based forecast.
We show measurability of important functionals such as moments or weighted quantiles and discuss the connection to continuity and robustness results discussed in statistics and finance. Measurability of elicitable functionals is discussed and established in Section (ref).
Finally, we provide sufficient conditions for the measurability of scoring rules in Section (ref). They are used to measure the predictive accuracy of probabilistic forecasts, and
map a pair of a predictive distribution $\ensuremath{\mathrm{P}}$ and an observation $y$ to the real number $\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}},y)$. Hence, technically speaking, for fixed $y$ they constitute a real-valued statistical functional, which aligns them with the main topic of this paper.
Regarding the application, the measurability of scoring rules formally justifies common statistical practices such as computing Diebold--Mariano tests which test predictive dominance DieboldMariano1995.
Statistical functionals and forecasting
Real-valued statistical functionals map probability measures on the Borel-$\sigma$-algebra of events of the observation domain to real numbers. More formally, the observation domain $O \subseteq \ensuremath{\mathbb{R}}^d$ is a Borel-measurable subset of a Euclidean space $\ensuremath{\mathbb{R}}^d$, where we will mainly consider $d=1$, with Borel-$\sigma$-algebra $\mathcal O$ of $O$. A statistical functional on some family $\ensuremath{\mathcal{P}}$ of (Borel-)probability measures on $(O, \mathcal O)$ is a map $T: \ensuremath{\mathcal{P}} \to \ensuremath{\mathbb{R}}$ from $\ensuremath{\mathcal{P}}$ to the real numbers.
A prediction space setting GneitingRanjan2013 consists of a probability space $(\Omega, \ensuremath{\mathcal{A}}, \ensuremath{\mathbb{P}})$ together with a random variable $Y : \Omega \to O$, the response variable modelling the quantity of interest, and a sub-$\sigma$-algebra $\ensuremath{\mathcal{F}}$ of $\ensuremath{\mathcal{A}}$, the information set on which the forecast is based. $\ensuremath{\mathcal{F}}$ can be generated, for example, by observable explanatory variables, often called, regressors or features, which can also contain past observations of $Y$ in a time series setting.
Then, the conditional distribution $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ of $Y$ given $\ensuremath{\mathcal{F}}$ is the ideal probabilistic forecast. If a real-valued quantity shall be issued as a forecast, it is essential to specify which statistical functional $T$ of the conditional distribution $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ is the target.
A general forecast for $T$ of $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ is an $\ensuremath{\mathcal{F}}$-measurable random variable, and the ideal forecast is $T(Y|\ensuremath{\mathcal{F}}) : = T\big(\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}} \big)$. For $T(Y|\ensuremath{\mathcal{F}})$ to be defined, we first require that conditional distributions in $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ are contained in $\ensuremath{\mathcal{P}}$, the domain of $T$. Second, we require the $\ensuremath{\mathcal{F}}$-measurability of the forecast $T(Y|\ensuremath{\mathcal{F}})$.
Our aim is to provide conditions on $T$ which guarantee that, when well-defined, $T(Y|\ensuremath{\mathcal{F}})$ is automatically a forecast, that is, $\ensuremath{\mathcal{F}}$-measurable, and to show that these conditions are satisfied by essentially all statistical functionals which arise in applications.
To proceed, let us recall the formal definition of the conditional distribution $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$. A Markov kernel from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$ is a map $\kappa: \Omega \times \mathcal O \to [0,1]$ such that
enumerate• for each $\omega \in \Omega$, the map $B \mapsto \kappa(\omega, B)$, $B \in \mathcal O$, is a probability measure on $(O, \mathcal O)$,
• for each $B \in \mathcal O$, the map $\omega \mapsto \kappa(\omega, B)$, $\omega \in \Omega$, is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{B}}[0,1]$-measurable.
A regular version of the conditional probability of $Y$ given $\ensuremath{\mathcal{F}}$, denoted by $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$, is a Markov kernel from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$ such that for each $F \in \ensuremath{\mathcal{F}}$ and $B \in \mathcal O$,
\[ \ensuremath{\mathbb{P}}\big(F \cap \{ Y \in B\} \big) = \int_F \ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}(\omega; B)\, \ensuremath{{\mathrm d}} \ensuremath{\mathbb{P}}(\omega).\]
Recall that $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ is unique only up to almost sure equality.
When forecasting the functional $T$ for $Y$ based on $\ensuremath{\mathcal{F}}$, we aim at the map
equation[equation omitted — 206 chars of source]
and we shall assume that the version $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ can be and is chosen such that $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}(\omega;\cdot) \in \ensuremath{\mathcal{P}}$ for all $\omega \in \Omega$. If $\kappa$ is a probabilistic forecast based on $\ensuremath{\mathcal{F}}$, that is, a Markov kernel from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$ for which $\kappa(\omega, \cdot) \in \ensuremath{\mathcal{P}}$ for $\omega \in \Omega$, we may form the resulting point forecast by
$$\omega \mapsto T\big(\kappa(\omega;\cdot) \big), \qquad \omega \in \Omega.$$
To show that $T(Y|\ensuremath{\mathcal{F}})$ or, more generally, that $T\big(\kappa(\cdot;\cdot) \big)$ is a forecast, that is $\ensuremath{\mathcal{F}}$-measurable, it suffices to find a $\sigma$-algebra $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$ such that (a) $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ or $\kappa$, considered as a map from $\Omega$ to $\ensuremath{\mathcal{P}}$, is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurable, and (b) to show that $T : \ensuremath{\mathcal{P}} \to \ensuremath{\mathbb{R}}$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable, where $\ensuremath{\mathcal{B}}$ is the Borel-$\sigma$-algebra of $\ensuremath{\mathbb{R}}$.
Measurability of functionals
We shall choose $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$ as the projection $\sigma$-algebra. More formally, let
$\ensuremath{\mathcal{Q}}$ be the family of all probability measures $\ensuremath{\mathrm{P}}$ on $(O, \mathcal O)$, and
for each $B \in \mathcal O$, consider the evaluation map
\[ \pi_B(\ensuremath{\mathrm{P}}) = \ensuremath{\mathrm{P}}(B), \qquad \ensuremath{\mathrm{P}} \in \ensuremath{\mathcal{Q}}.\]
Then, on $\ensuremath{\mathcal{Q}}$ we can consider the smallest $\sigma$-algebra $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{Q}})$ which makes all evaluation maps $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{Q}}) - \ensuremath{\mathcal{B}}[0,1]$-measurable. For a subset $\ensuremath{\mathcal{P}} \subseteq \ensuremath{\mathcal{Q}}$ we denote by $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) = \{ \ensuremath{\mathcal{P}} \cap A \mid A \in \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{Q}})\}$ the trace $\sigma$-algebra.
Note that $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$ is also the $\sigma$-algebra generated by the restrictions of the evaluation maps $\pi_B$ to $\ensuremath{\mathcal{P}}$, which follows from the simple fact that for any $C\in \ensuremath{\mathcal{B}}[0,1]$ and any $B\in \mathcal O$ we have $\pi_{B|\ensuremath{\mathcal{P}}}^{-1}(C) = \ensuremath{\mathcal{P}} \cap \pi_B^{-1}(C)$.
Moreover, $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$ can be regarded as the Borel-$\sigma$-algebra induced by the topology of setwise convergence.
lemmaLet $\kappa: \Omega \times \mathcal O \to [0,1]$ be a map such that for each $\omega\in \Omega$, $\kappa(\omega, \cdot) \in \ensuremath{\mathcal{P}}$. Then $\kappa$ is a Markov kernel from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$ if and only if the map $\omega \to \kappa(\omega, \cdot)$ from $\Omega$ to $\ensuremath{\mathcal{P}}$ is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurable.
proof$\kappa$ is a Markov kernel if and only if for each $B \in \mathcal O$, the map $\omega \to \kappa(\omega, B) = \pi_B(\kappa(\omega, \cdot))$ is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{B}}[0,1]$-measurable, which is the case if and only if $\omega \to \kappa(\omega, \cdot)$ is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurable.
Concerning the measurability of the forecasts $T(Y|\ensuremath{\mathcal{F}})$ of functionals $T$, we observe the following.
lemmaIf the functional $T: \ensuremath{\mathcal{P}} \to \ensuremath{\mathbb{R}}$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable, then $T(Y|\ensuremath{\mathcal{F}})$ defined in (ref) is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{B}}$-measurable.
proofThis is clear since by Lemma (ref), the conditional distribution $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$ is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurable as a map from $\Omega$ to $\ensuremath{\mathcal{P}}$, and $T$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable by assumption.
Similarly to Lemma (ref), the general forecast $\omega \mapsto T\big(\kappa(\omega;\cdot) \big)$, $\omega \in \Omega$, where $\kappa$ is a Markov kernel from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$, is $\ensuremath{\mathcal{F}} - \ensuremath{\mathcal{B}}$-measurable.
Thus, to clarify the measurability of $T\big(\kappa(\cdot;\cdot) \big)$ and of $T(Y|\ensuremath{\mathcal{F}})$ in (ref) it suffices to investigate $T$.
Before we turn to a general result for elicitable functionals, let us investigate several important examples.
In the following we identify any probability distribution $\ensuremath{\mathrm{P}}\in\ensuremath{\mathcal{Q}}$ with its distribution function $F = F_{\ensuremath{\mathrm{P}}}$ defined by $F(x) = \ensuremath{\mathrm{P}}(O \cap (-\infty, x])$, $x \in O$.
example[Moments]
Let $h: O \to \ensuremath{\mathbb{R}}$ be a measurably function.
Let $\ensuremath{\mathcal{P}}_h$ be a family of probability measures on $(O, \mathcal O)$ satisfying $\int_O \, |h(y)|\, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}(y) < \infty$, and consider the mean functional
\[ T_h(\ensuremath{\mathrm{P}}) = \int_O \, h(y)\, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}(y), \qquad \ensuremath{\mathrm{P}}\in\ensuremath{\mathcal{P}}_h.\]
Then $T_h$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}_h)-\ensuremath{\mathcal{B}}$-measurable.
Indeed, if $h(y) = \mathds{1}_B(y)$ for some $B \in \mathcal O$, then $T_h = \pi_B$ and the claim follows from the measurability of the evaluation map.
If $h$ is a simple function, the resulting functional will be a finite linear combination of evaluation maps, and hence also measurable. For $h\geq 0$, there is a sequence of non-negative simple functions $(h_n)$ with $h_n \uparrow h$ pointwise, and then $T_{h_n}(\ensuremath{\mathrm{P}}) \uparrow T_{h}(\ensuremath{\mathrm{P}})$ for every $\ensuremath{\mathrm{P}}$. Hence, as a pointwise limit of the measurable functions $T_{h_n}$, $T_h$ is also measurable. For a general $h$, the measurability follows by the decomposition $T_h = T_{h^+} - T_{h^-}$.
Example (ref) directly yields the measurability of Borel-measurable functions of multiple moments. This yields, for example, the measurability of the variance, the skewness, the kurtosis, or of the Sharpe ratio.
example[Quantiles]
Let $O = \ensuremath{\mathbb{R}}$,
fix $\alpha \in [0,1]$ and let
\begin{align*}
T(F) = q_\alpha^-(F)
& = \inf\{ x \in \ensuremath{\mathbb{R}} \mid F(x) \geq \alpha\} \in \bar \ensuremath{\mathbb{R}} = \ensuremath{\mathbb{R}}\cup\{-\infty, \infty\}, \qquad F \in \ensuremath{\mathcal{P}},
\end{align*}
be the lower $\alpha$-quantile of $F$, which is its essential supremum if $\alpha=1$.
Recall that $ \ensuremath{\mathcal{B}}(\bar \ensuremath{\mathbb{R}}) = \{A\cup E\mid A\in\ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}}), \ E\subseteq \{-\infty, \infty\}\}$.
To show that $T$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}(\bar \ensuremath{\mathbb{R}})$-measurable,
consider the family of evaluation maps $\pi_{(-\infty,x]}(F) = F(x)$, for $F \in \ensuremath{\mathcal{P}}$, $x \in \ensuremath{\mathbb{R}}$. Each $\pi_{(-\infty,x]}$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}([0,1])$-measurable.
Now, given $p \in \ensuremath{\mathbb{R}}$, the lower $\alpha$-quantile $q_\alpha^-(F)$ of $F$ is strictly larger than $p$, $q_\alpha^-(F) > p$, if and only if $\alpha > F(p)$. Therefore
\begin{align*}
\{F \in \ensuremath{\mathcal{P}} \mid q_\alpha^-(F) > p\} & = \{F \in \ensuremath{\mathcal{P}} \mid \pi_{(-\infty,p]}(F) < \alpha\},
\end{align*}
showing measurability of the lower $\alpha$-quantile, where we exploit the fact that it suffices to show measurability on a generator of $\ensuremath{\mathcal{B}}(\bar \ensuremath{\mathbb{R}})$.
Similar considerations yield the $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}(\bar \ensuremath{\mathbb{R}})$-measurability of the upper $\alpha$-quantile, $\alpha\in[0,1]$,
\begin{align*}
T(F) = q_\alpha^+(F)
& = \sup\{ x \in \ensuremath{\mathbb{R}} \mid \lim_{t\uparrow x} F(t) \leq \alpha\} \in \bar \ensuremath{\mathbb{R}}, \qquad F \in \ensuremath{\mathcal{P}},
\end{align*}
where $q_0^+(F)$ corresponds to the essential infimum of $F$.
For $\alpha\in(0,1)$, we can also consider the $\alpha$-quantile, $q_\alpha$, as an interval-valued functional $q_\alpha(F) = [q_\alpha^-(F), q_\alpha^+(F)]\subset \ensuremath{\mathbb{R}}$. Then, the previously established measurability results yield that $q_\alpha$ is an Effros measurable closed-valued multifunction in the sense that for all open sets $G\subseteq\ensuremath{\mathbb{R}}$, $\{F\in\ensuremath{\mathcal{P}}\mid q_\alpha(F)\cap G \neq \emptyset\}\in \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$, see Definition 1.3.1 in Molchanov2017. Therefore, the ideal forecast $q_\alpha(Y|\ensuremath{\mathcal{F}})$ is a random closed set in the sense of Definition 1.1.1 in Molchanov2017.
This complements the measurability argument of the quantile provided recently in CastroETAL2021.
In the risk management literature, the lower $\alpha$-quantile is known as Value at Risk at level $\alpha$, $\mathrm{VaR}_\alpha$.
The following example is concerned with another important quantitative risk measure, the Expected Shortfall.
example[Weighted averages of quantiles]
Let $w\colon[0,1]\to[0,\infty)$ be a measurable weight function such that the left-sided and the right-sided limits exists for all $x\in[0,1]$.
Consider the functional
\begin{equation*}
T_w(F) = \int_0^1 q_\gamma^-(F)w(\gamma)\, \ensuremath{{\mathrm d}} \gamma \in \bar \ensuremath{\mathbb{R}}, \qquad F\in \ensuremath{\mathcal{P}}_w,
\end{equation*}
where $\ensuremath{\mathcal{P}}_w$ is the domain such that $T_w$ is well-defined. The measurability of $q_\gamma^-$ from Example (ref) and a Riemann approximation argument yield that $T_w$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}_w) - \ensuremath{\mathcal{B}}(\bar \ensuremath{\mathbb{R}})$-measurable.
This directly yields the measurability of the lower and upper Expected Shortfall (by choosing $w = \mathds{1}_{[0,\alpha]}/\alpha$ or $w = \mathds{1}_{[\alpha,1]}/(1-\alpha)$) and of the Range Value at Risk ContDeguestETAL2010, FisslerZiegel2021 (by choosing $w= \mathds{1}_{[\alpha,\beta]}/(\beta - \alpha)$ for $0<\alpha<\beta<1$).
To treat the next Example (ref), we first need to establish the following technical lemma.
lemmaLet $T\colon \ensuremath{\mathcal{P}}\to\ensuremath{\mathbb{R}}$ be an $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}})$-measurable functional. Then the composed evaluation maps $\ensuremath{\mathcal{P}}\to[0,1]$,
\begin{equation}
F\mapsto \pi_{(T(F),\infty)}(F),
\qquad F\mapsto \pi_{(-\infty,T(F))}(F),
\qquad F\mapsto \pi_{\{T(F)\}}(F)
\end{equation}
are $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}([0,1])$-measurable.
proofFor the first map in (ref), let $p\in[0,1]$ and consider
\begin{align*}
\{F\in \ensuremath{\mathcal{P}}\mid \pi_{(T(F),\infty)}(F) \le p\}
&= \{F\in \ensuremath{\mathcal{P}}\mid F(T(F))\ge 1- p\}
= \{F\in \ensuremath{\mathcal{P}}\mid T(F)\ge q_{1- p}^-(F)\} \\
&= \{F\in \ensuremath{\mathcal{P}}\mid (T -q_{1- p}^-)(F) \ge 0\}
\in \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}).
\end{align*}
The last assertion comes from the fact that $q_{1-p}^-$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}})$-measurable and that differences of measurable functionals are also measurable.\\
For the second map in (ref), the arguments are similar. Using the shorthand $F(x-) := \lim_{t\uparrow x}F(t)$ for the left-sided limit, we obtain for $p\in[0,1]$
\begin{align*}
\{F\in \ensuremath{\mathcal{P}}\mid \pi_{(-\infty,T(F))}(F) \le p\}
&= \{F\in \ensuremath{\mathcal{P}}\mid F(T(F)-)\le p\}
= \{F\in \ensuremath{\mathcal{P}}\mid T(F)\le q_p^+(F) \} \\
&= \{F\in \ensuremath{\mathcal{P}}\mid (T - q_p^+)(F)\le 0 \} \in \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}).
\end{align*}
Finally, note that $\pi_{\{T(F)\}}(F) = 1-\pi_{(T(F),\infty)}(F) - \pi_{(-\infty,T(F))}(F)$.
exampleConditional Value at Risk (CoVaR) and conditional Expected Shortfall (CoES) are influential systemic risk measures due to AB16.
In a nutshell, they assess the riskiness of a position $Y$ given that a reference position $X$ is at risk. Following GT13, the latter conditioning event is interpreted as $X$ exceeding its Value at Risk at level $\beta$, $\mathrm{VaR}_\beta$.
Hence, these risk measures can be viewed as functionals, first mapping a bivariate distribution $\ensuremath{\mathrm{P}}_{X,Y}$ to the univariate conditional distribution $\ensuremath{\mathrm{P}}_{Y\mid X \ge \mathrm{VaR}_{\beta}(X)}$ and then applying a risk measure such as $\mathrm{VaR}_{\alpha}$ (in the case of CoVaR) or $\mathrm{ES}_\alpha$ (in the case of CoES) to this univariate distribution.
Since $\mathrm{VaR}_{\alpha}$ and $\mathrm{ES}_\alpha$ are measurable functionals from univariate distributions to $\ensuremath{\mathbb{R}}$, we obtain the measurability of these conditional risk measures if the map $\ensuremath{\mathrm{P}}_{X,Y}\mapsto \ensuremath{\mathrm{P}}_{Y\mid X \ge \mathrm{VaR}_{\beta}(X)}$ is measurable.
To account for possible discontinuities in the marginal distribution of the first component $X$, we follow Remark C.1 in FisslerHoga2021 and incorporate a correction term to this map.
Hence, we formally consider
$\eta\colon \ensuremath{\mathcal{P}}^2 \to \ensuremath{\mathcal{P}}$, where $\ensuremath{\mathcal{P}}^2$ is a set of Borel-distributions on $\ensuremath{\mathbb{R}}^2$ and $\ensuremath{\mathcal{P}}$ is a set of such distributions on $\ensuremath{\mathbb{R}}$, such that for a Borel set $B\in\ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}})$ we get
\begin{align*}
&\eta(\ensuremath{\mathrm{P}}_{X,Y})(B)=
\ensuremath{\mathrm{P}}_{Y\mid X \succcurlyeq \mathrm{VaR}_\beta(X)}(B)\\ \nonumber
&:=
\frac{1}{1-\beta}\Big[\ensuremath{\mathrm{P}}_{X,Y}\big((\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X),\infty) \times B\big) \\ \nonumber
&\qquad + \Big(1-\beta - \ensuremath{\mathrm{P}}_X \big((\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X),\infty)\big) \Big)\ensuremath{\mathrm{P}}_{X,Y}\big(\{\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X)\} \times B\big)/\ensuremath{\mathrm{P}}_X \big(\{\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X)\}\big)\Big]
\end{align*}
For the last term, note that $\ensuremath{\mathrm{P}}_X \big(\{\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X)\}\big)=0$ implies that $1-\beta - \ensuremath{\mathrm{P}}_X \big((\mathrm{VaR}_{\beta}(\ensuremath{\mathrm{P}}_X),\infty)\big)=0$ for which case we set $0/0=0$.
To show the $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}^2)-\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurability of $\eta$ it is sufficient to show that for any $B\in\ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}})$, the concatenation with the evaluation map $\pi_B\circ \eta$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}^2)-\ensuremath{\mathcal{B}}([0,1])$-measurable.
This measurability follows from Lemma (ref) and the fact that for any $C\in\ensuremath{\mathcal{B}}(\ensuremath{\mathbb{R}})$ the map $\ensuremath{\mathrm{P}}_{X,Y}\mapsto \ensuremath{\mathrm{P}}(\cdot \times C)$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}^2)-\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurable.
remark[Continuity of statistical functionals]
Apart from mere measurability, continuity or even differentiability properties of statistical functionals are of interest in statistics, finance and econometrics. A statistical functional is weakly continuous at $\ensuremath{\mathrm{P}} \in \ensuremath{\mathcal{P}}$ if the weak convergence $\ensuremath{\mathrm{P}}_n \Rightarrow \ensuremath{\mathrm{P}}$ of a sequence $(\ensuremath{\mathrm{P}}_n)$ in $\ensuremath{\mathcal{P}}$ to $\ensuremath{\mathrm{P}}$ implies that $T(\ensuremath{\mathrm{P}}_n) \to T(\ensuremath{\mathrm{P}})$. Let $\ensuremath{\mathcal{B}}(\ensuremath{\mathcal{P}})$ denote the Borel-$\sigma$-algebra generated by the weak topology on $\ensuremath{\mathcal{P}}$.
If a statistical functional $T$ is weakly continuous at every $\ensuremath{\mathrm{P}}\in \ensuremath{\mathcal{P}}$ it is also $\ensuremath{\mathcal{B}}(\ensuremath{\mathcal{P}})$-measurable.
Since $\ensuremath{\mathcal{B}}(\ensuremath{\mathcal{P}}) \subseteq \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) $, this readily implies that $T$ is also measurable in the sense defined above.
However, apart from the Range Value at Risk the functionals discussed above are known not to be continuous at every $\ensuremath{\mathrm{P}}$ under the general assumptions that we impose: For the mean, the discontinuity follows from HuberRonchetti2009, while the lower $\alpha$-quantile, $\alpha\in(0,1)$, is weakly continuous at $F$ if and only if the quantile function $\gamma\mapsto q_\gamma^-(F)$ is continuous at $\alpha$, that is, if and only if $q_\alpha^-(F)=q_\alpha^+(F)$; see VanderVaart1998.
In the finance literature, statistical functionals arise from law-determined (often called law-invariant) risk measures and are often called risk functionals KratschmeSchiedETAL2014. Assuming monotonicity with respect to first order stochastic dominance and translation equivariance, Weber2006 establishes the weak continuity of risk functionals on the class of probability distributions with compact support, implying the measurability.
While this applies to quantiles and weighted averages of quantiles with weight function $w$ satisfying $\int w(\gamma)\, \ensuremath{{\mathrm d}} \gamma=1$, we do not require compactly supported distributions.\\
Generalizing Weber2006, KratschmeSchiedETAL2014 show that monotone and translation equivariant risk functionals arising from convex risk measures on $\ensuremath{\mathcal{P}}\subseteq \ensuremath{\mathcal{Q}}^p = \{\ensuremath{\mathrm{P}}\in\ensuremath{\mathcal{Q}} \mid \int |x|^p \, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}(x)\}$ are continuous in the Wasserstein metric of order $p\in[1,\infty)$.
Similarly, KieselETAL2016 show the continuity of $L$-functionals in the Wasserstein metric of order $p\in[1,\infty)$.
Since a sequence of probability measures $(\ensuremath{\mathrm{P}}_n)$ in $\ensuremath{\mathcal{P}}$ converges to $\ensuremath{\mathrm{P}}$ in the Wasserstein metric of order $p$ if and only if it converges weakly and $\int |x|^p \, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}_n(x) \to \int |x|^p \, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}(x)$, the continuity in the Wasserstein metric of order $p$ implies $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurability.
This yields an alternative argument for the $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurability of the lower and upper expected shortfall.
Elicitable functionals
Let us now turn to general elicitable functionals. These are functionals such that the ideal forecast corresponds to a Bayes act in the sense that it minimizes an expected loss function.
Again, let $O \subseteq \ensuremath{\mathbb{R}}$ be a measurable subset, let $A$ be an interval, and let $\ensuremath{\mathcal{P}}$ be a family of Borel probability measures on $(O, \mathcal O)$.
Let
\[
\mathrm{L} : A \times O \to \ensuremath{\mathbb{R}}
\]
be a scoring or loss function with the following properties:
enumerate• For each $a \in A$ and $\ensuremath{\mathrm{P}} \in \ensuremath{\mathcal{P}}$, $\mathrm{L}(a,\cdot)$ is $\ensuremath{\mathrm{P}}$-integrable. Set
\[ \overline{\mathrm{L}}(a,\ensuremath{\mathrm{P}}) = \int_O \mathrm{L}(a,y)\, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{P}}(y).\]
We also write $\overline{\mathrm{L}}(a,F)$ if $F$ is a distribution function of a distribution $\ensuremath{\mathrm{P}}$ in $\ensuremath{\mathcal{P}}$.
• For each $F \in \ensuremath{\mathcal{P}}$, the function $a \mapsto \overline{\mathrm{L}}(a,F)$, $a \in A$,
\begin{enumerate}
• is continuous,
• has a compact, non-empty interval of minimizers, denoted by $I_\mathrm{L}(F)$,
• is monotonically decreasing to the left of $I_\mathrm{L}(F)$, and monotonically increasing to the right.
\end{enumerate}
Then $\mathrm{L}$ is $\ensuremath{\mathcal{P}}$-consistent for the functionals $T_{\min}(F) = \min I_\mathrm{L}(F)$ and $T_{\max}(F) = \max I_\mathrm{L}(F)$ meaning that $T_{\min}(F)$ and $T_{\max}(F)$ minimize the expected loss function $a\mapsto \overline{\mathrm{L}} (a,F)$ for each $F\in\ensuremath{\mathcal{P}}$.
If $I_\mathrm{L}(F)$ is a singleton for each $F \in \ensuremath{\mathcal{P}}$, then $T_{\min} = T_{\max}=:T$, then $\mathrm{L}$ is strictly $\ensuremath{\mathcal{P}}$-consistent for $T$, meaning that $T$ uniquely minimizes the expected loss.
A functional is elicitable on $\ensuremath{\mathcal{P}}$ if there exists a strictly $\ensuremath{\mathcal{P}}$-consistent scoring function for it.
If the class $\ensuremath{\mathcal{P}}$ is convex and if the continuity assumption a) on the expected loss holds, then the strict $\ensuremath{\mathcal{P}}$-consistency of $\mathrm{L}$ for $T_{\min} = T_{\max}$ already implies properties b) and c) as shown in FisslerZiegel2019, Nau1985, BelliniBignozzi2015, Lambert2019; see also SteinwartPasinETAL2014 for a related result.
Properties b) and c) are known as order-sensitivity or accuracy-rewarding in the literature; see LambertETAL2008 and the references above.
As noted in the discussion below Proposition 2.2 in FisslerZiegel2019, a sufficient condition for the continuity of the expected loss $a\mapsto \overline{\mathrm{L}}(a,F)$ for each $F\in\ensuremath{\mathcal{P}}$ are conditions a), b) and c) on the level of the loss $a\mapsto \mathrm{L}(a,y)$ itself for each $y\inO$.
The continuity of $\mathrm{L}(a,y)$ in its first argument is a common regularity condition; see e.g. Gneiting2011.
Define the entropy, uncertainty, or Bayes risk of the $\ensuremath{\mathcal{P}}$-consistent loss $\mathrm{L}$ for $T$ as $T_{\mathrm{ent}}(F) = \overline{\mathrm{L}}\big(T(F),F\big)$, $F\in\ensuremath{\mathcal{P}}$.
theoremUnder the above assumptions, the functionals $T_{\min}$, $T_{\max}$ and $T_{\mathrm{ent}}$ are $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable.
proofFirst consider $T_{\min}$.
By Example (ref), for each $a\inA$ the functional $F\mapsto \overline{\mathrm{L}}(a,F)$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable. We then argue similarly as in Example (ref): Let $a \in A$ and $F \in \ensuremath{\mathcal{P}}$. We claim that under our assumptions, $T_{\min}(F) > a$ holds if and only if there exists $b \in A \cap \ensuremath{\mathbb{Q}}$ with $b > a$ such that $\overline{\mathrm{L}}(a,F) > \overline{\mathrm{L}}(b,F)$.
Indeed, by definition of $T_{\min}$, $T_{\min}(F) > a$ holds if and only if
$a$ is to the left of $I_\mathrm{L}(F)$. By monotonicity and continuity of $\overline{\mathrm{L}}(\cdot,F)$,
this holds if and only if there exists $b \in (a,\infty)\cap \ensuremath{\mathbb{Q}} \cap A$ such that $\overline{\mathrm{L}}(a,F) > \overline{\mathrm{L}}(b,F)$.
Therefore,
\begin{align*}
\{F\in\ensuremath{\mathcal{P}} \mid T_{\min}(F) >a \} & = \bigcup_{b \in (a,\infty)\cap \ensuremath{\mathbb{Q}} \cap A} \{ F\in \ensuremath{\mathcal{P}} \mid \overline{\mathrm{L}}(a,F) - \overline{\mathrm{L}}(b,F) >0\},
\end{align*}
showing measurability of $T_{\min}(F)$. The arguments for $T_{\max}$ work similarly.
For $T_{\mathrm{ent}}$, we obtain the measurability upon noting that for any $x\in\ensuremath{\mathbb{R}}$
\begin{align*}
\{F\in\ensuremath{\mathcal{P}} \mid T_{\mathrm{ent}}(F) < x \} & = \bigcup_{a \in \ensuremath{\mathbb{Q}} \cap A} \{ F\in \ensuremath{\mathcal{P}} \mid \overline{\mathrm{L}}(a,F) < x\}.
\end{align*}
example[Expectiles and generalized quantiles]
On the class of distributions with finite mean, NeweyPowell1987 introduced the $\tau$-expectile of a distribution $\ensuremath{\mathrm{P}}$, $e_\tau(\ensuremath{\mathrm{P}})$, $\tau\in(0,1)$, as the unique solution to the equation
\[
\tau \int_{(x,\infty)} (y-x) \,\mathrm{d}\ensuremath{\mathrm{P}}(y) = (1-\tau) \int_{(-\infty,x]} (x-y) \,\mathrm{d}\ensuremath{\mathrm{P}}(y)
\]
in $x$. On the class of square-integrable distributions, $e_\tau$ is elicitable with the asymmetric piecewise quadratic loss $\mathrm{L}_\tau(a,y) = |\mathds{1}\{y\le a\} - \tau|(a-y)^2$ as a strictly consistent loss. Hence, Theorem (ref) establishes the measurability of $e_\tau$ on this class.
For the measurability on the class of integrable distributions, one can consider the slightly modified loss function $\widetilde \mathrm{L}_\tau(a,y) = \mathrm{L}_\tau(a,y) - \mathrm{L}_\tau(0,y)$.
BelliniKlarETAL2014 study generalized $\tau$-quantiles which arise as minimizers of the expected loss
$L_{\tau,\phi_1,\phi_2} (a,y) = \mathds{1}\{y\le a\} (1 - \tau)\phi_1(|a-y|) + \mathds{1}\{y> a\}\tau \phi_2(|a-y|)$ for two convex and strictly increasing functions $\phi_1, \phi_2 : [0,\infty)\to[0,\infty)$. Clearly, if $\phi_1$ and $\phi_2$ are both the absolute function (the squared function) the $\tau$-quantile ($\tau$-expectile) arises. ElliottETAL2005 studied the situation of general power functions.
Theorem (ref) then establishes the $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurability of these generalized $\tau$-quantiles.
example[Variance and Expected Shortfall]
On the class of square-integrable distributions $\ensuremath{\mathcal{P}}$, the squared loss $L(a,y) = (a-y)^2$ is a strictly $\ensuremath{\mathcal{P}}$-consistent loss for the mean functional.
The corresponding entropy functional $T_{\mathrm{ent}}$ is the variance functional.
Therefore, Theorem (ref) establishes the $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurability of the variance as an alternative argument to the direct application of Example (ref).
For the lower or the upper Expected Shortfall at level $\alpha$ it is well known EmbrechtsWang2015 that both versions can be written as the entropy of a consistent loss function for $q_\alpha^-$.
Therefore, Theorem (ref) establishes an alternative argument for the measurability to that provided in Example (ref).
An alternative approach to establish the measurability of a functional is to exploit identifiability instead of elicitability.
A functional is called identifiable if there exists a moment or identification function $\mathrm{V}\colonA\timesO\to\ensuremath{\mathbb{R}}$ such that the functional is the unique zero of the expected identification function.
Indeed, under some technical assumptions, elicitability and identifiability are equivalent for one-dimensional functionals SteinwartPasinETAL2014, and concerning measurability, one ends up with similar results and examples as considered in this section.
remarkProposition 3.7 in BelliniBignozzi2015 establishes $\psi$-weak continuity of elicitable functionals on classes of probability measures with compact support under additional regularity conditions on the corresponding strictly consistent loss function.
$\psi$-weak continuity has been discussed in KratschmeSchiedETAL2014 and generalizes continuity in the Wasserstein metric. It implies $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}})$-measurability in the sense studied in this paper.
Measurability of scoring rules
In this section we briefly discuss measurability of scoring rules, which are used to compare probabilistic forecasts.
Consider maps
equation[equation omitted — 119 chars of source]
such that for each $\ensuremath{\mathrm{P}} \in \ensuremath{\mathcal{P}}$, $\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}};\cdot)$ is Borel-measurable and that for each $\ensuremath{\mathrm{Q}} \in \ensuremath{\mathcal{P}}$,
\[ \ensuremath{\mathbb{E}}_\ensuremath{\mathrm{Q}}\big[\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}};Y) \big] = \int_O \ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}};y)\, \ensuremath{{\mathrm d}} \ensuremath{\mathrm{Q}}(y) \]
exists.
Note that we allow $\infty$ in the range of $\ensuremath{\mathrm{S}}$ in (ref) to treat the common logarithmic score, see Example (ref).
$\ensuremath{\mathrm{S}}$ is called a strictly $\ensuremath{\mathcal{P}}$-proper scoring rule if
$$ \ensuremath{\mathbb{E}}_\ensuremath{\mathrm{Q}}\big[\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{Q}};Y) \big] < \ensuremath{\mathbb{E}}_\ensuremath{\mathrm{Q}}\big[\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}};Y) \big] \qquad \text{for all } \ \ensuremath{\mathrm{P}}, \ensuremath{\mathrm{Q}} \in \ensuremath{\mathcal{P}},\ \ensuremath{\mathrm{P}} \not= \ensuremath{\mathrm{Q}},$$
meaning that forecasting the true distribution $\ensuremath{\mathrm{Q}}$ uniquely minimizes the expected score.
Recall that the ideal probabilistic forecast based on an information set $\ensuremath{\mathcal{F}}$ is the conditional distribution $\ensuremath{\mathbb{P}}_{Y | \ensuremath{\mathcal{F}}}$.
That means in a forecasting situation, with available information $\ensuremath{\mathcal{F}}$ (e.g. given by regressors), it is natural that the forecasts have the form of Markov kernels from $(\Omega, \ensuremath{\mathcal{F}})$ to $(O, \mathcal O)$.
Inserting a Markov kernel $\kappa$ for $\ensuremath{\mathrm{P}}$ leads to the map
equation[equation omitted — 115 chars of source]
and averages of such expressions, which estimate expected values, are used when comparing different forecasts.
But in order to form the expected value in (ref) we in particular require measurability, i.e. we need that (ref) is an $\ensuremath{\mathcal{A}} - \ensuremath{\mathcal{B}}$-measurable random variable. Since $\omega \mapsto \big(\kappa(\omega,\cdot);Y(\omega)\big)$ is $\ensuremath{\mathcal{A}} - \ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) \otimes \mathcal O$-measurable,
this will be guaranteed if we assume that $\ensuremath{\mathrm{S}}$ in (ref) is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) \otimes \mathcal O - \ensuremath{\mathcal{B}}$-measurable.
lemmaSuppose that $O$ is a one-dimensional interval,
and that
\begin{enumerate}
• for each $y \in O$, the map $\ensuremath{\mathrm{S}}(\cdot; y) : \ensuremath{\mathcal{P}} \to (-\infty, \infty]$ is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) - \ensuremath{\mathcal{B}}$-measurable,
• for each $\ensuremath{\mathrm{P}} \in \ensuremath{\mathcal{P}}$, the map $\ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}}; \cdot)$ is right-continuous (or left-continuous).
\end{enumerate}
Then $\ensuremath{\mathrm{S}}$ in (ref) is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) \otimes \mathcal O - \ensuremath{\mathcal{B}}$-measurable.
proofFor a finite interval, say $O = (0,1]$, for $n \in \ensuremath{\mathbb{N}}$, by (i) the map
$$ \ensuremath{\mathrm{S}}_n(\ensuremath{\mathrm{P}},y) = \sum_{k=1}^{2^n}\, \ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}}; k/2^n)\, \mathds{1}_{((k-1)/2^n, k/2^n]}(y)$$
is $\ensuremath{\mathcal{A}}(\ensuremath{\mathcal{P}}) \otimes \mathcal O - \ensuremath{\mathcal{B}}$-measurable and by (ii), $\ensuremath{\mathrm{S}}_n(\ensuremath{\mathrm{P}},y) \to \ensuremath{\mathrm{S}}(\ensuremath{\mathrm{P}},y)$ for $n \to \infty$. An infinite interval can be treated analogously by extending the sum to an infinite one.
remarkThe argument in Lemma (ref) can be used to obtain a similar result for multivariate observations, where right-continuity is defined, as for multivariate distribution functions, by taking limits from above in each coordinate. However, the form of the observation domain $O$ is also an issue, and to proceed as above we require that $O$ is a possibly infinite hyperrectangle.
example[Continuous ranked probability score]
The continuous ranked probability score (CRPS) of a distribution $F$ with finite first moment is
\begin{equation}
\operatorname{CRPS}(F,y) = \int_{- \infty}^{\infty} \big(F(z) - \mathds{1}\{y \leq z\} \big)^2\, \mathrm{d} z,\quad F \in \ensuremath{\mathcal{P}},
\end{equation}
see GneitingRaftery2007.
It has an alternative representation in terms of an integrated quantile loss, and also as
\begin{align}
\operatorname{CRPS}(F,y) & = \ensuremath{\mathbb{E}}_F\big[ \big|y- X \big|\big] - \frac{1}{2}\, \ensuremath{\mathbb{E}}_F \big[\big|X'- X \big|\big],
\end{align}
where $X$ and $X'$ are independent copies with distribution $F$.
From both (ref) and (ref) one can readily see that $\operatorname{CRPS}(F,y)$ is continuous as a function of $y$, invoking the dominated convergence theorem.
Further, for fixed $y$, (ref) implies measurability of $\operatorname{CRPS}(F,y)$ as a map in $F$, while from (ref) the measurability can be obtained with a Riemann sum approximation similarly as in Example (ref). Similar arguments can be made and hence the measurability is obtained for weighted versions of the CRPS as introduced in GneitingRanjan2011 and HolzmannKlar2017.
example[Logarithmic score and local scoring rules]
Assume that $\ensuremath{\mathcal{P}}$ is a family of right-continuous Lebesgue densities on the real line. We write $\ensuremath{\mathrm{p}} \in \ensuremath{\mathcal{P}}$ and denote the distribution function associated with $\ensuremath{\mathrm{p}}$ by $F_\ensuremath{\mathrm{p}}$.
The logarithmic score is
\[
\operatorname{LOGS}(\ensuremath{\mathrm{p}},y) = - \log \ensuremath{\mathrm{p}}(y),\qquad \ensuremath{\mathrm{p}} \in \ensuremath{\mathcal{P}},
\]
see Good1952 and GneitingRaftery2007.
To apply Lemma (ref), observe that $\operatorname{LOGS}(\ensuremath{\mathrm{p}},y)$ is right-continuous as a function of $y$ by our assumption on $\ensuremath{\mathrm{p}}$ and continuity of the logarithm.
For the measurability in $\ensuremath{\mathrm{p}}$ for fixed $y$, note that by right-continuity,
$$\lim_{n \to \infty} \log\Big(n\, \big(F_\ensuremath{\mathrm{p}}(y+1/n) - F_\ensuremath{\mathrm{p}}(y)\big) \Big) = \log \ensuremath{\mathrm{p}}(y),$$
showing measurability, since the evaluation of the distribution function is measurable. Measurability extends to weighted versions of the logarithmic score as in DiksETAL2011 and in HolzmannKlar2017, and under appropriate assumptions on higher derivatives to higher-order local scoring rules EhmGneiting2012, ParryDawidLauritzen2012 such as the Hyv\"arinen score Hyvaerinen2005.
Acknowledgements
We would like to thank Johannes Resin for valuable feedback on an earlier version of the paper.
thebibliography\bibitem[Adrian and Brunnermeier, 2016]{AB16}
Adrian, T. and Brunnermeier, M. K. (2016).
\newblock {CoVaR}.
\newblock {\em Am. Econ. Rev.}, 106(7):1705--1741.
\bibitem[Bellini and Bignozzi, 2015]{BelliniBignozzi2015}
Bellini, F. and Bignozzi, V. (2015).
\newblock On elicitable risk measures.
\newblock {\em Quant. Finance}, 15(5):725--733.
\bibitem[Bellini et al., 2014]{BelliniKlarETAL2014}
Bellini, F., Klar, B., M\"uller, A., and {Rosazza Gianin}, E. (2014).
\newblock Generalized quantiles as risk measures.
\newblock {\em Insurance Math. Econom.}, 54:41--48.
\bibitem[Borgonovo et al., 2021]{BorgonovoETAL2021}
Borgonovo, E., Hazen, G. B., Jose, V. R. R., and Plischke, E. (2021).
\newblock Probabilistic sensitivity measures as information value.
\newblock {\em Eur. J. Oper. Res.}, 289(2):595--610.
\bibitem[Cont et al., 2010]{ContDeguestETAL2010}
Cont, R., Deguest, R., and Scandolo, G. (2010).
\newblock Robustness and sensitivity analysis of risk measurement procedures.
\newblock {\em Quant. Finance}, 10:593--606.
\bibitem[Dawid, 1984]{Dawid1984}
Dawid, A. P. (1984).
\newblock {Present Position and Potential Developments: Some Personal Views:
Sta\-tistical Theory: The Prequential Approach}.
\newblock {\em J. Roy. Statist. Soc. Ser. A}, 147(2):278--292.
\bibitem[de Castro et al., 2021]{CastroETAL2021}
de Castro, L. I., Costa, B. N., Galvao, A. F., and Zubelli, J. (2021).
\newblock Conditional quantiles: An operator-theoretical approach.
\newblock {\em Preprint}.
\newblock \url{https://dx.doi.org/10.2139/ssrn.3924597}.
\bibitem[Diebold and Mariano, 1995]{DieboldMariano1995}
Diebold, F. X. and Mariano, R. S. (1995).
\newblock Comparing predictive accuracy.
\newblock {\em J. Bus. Econom. Statist.}, 13:253--263.
\bibitem[Diks et al., 2011]{DiksETAL2011}
Diks, C., Panchenko, V., and van Dijk, D. (2011).
\newblock Likelihood-based scoring rules for comparing density forecasts in
tails.
\newblock {\em J. Econom.}, 163(2):215--230.
\bibitem[Dimitriadis and Bayer, 2019]{DimiBayer2019}
Dimitriadis, T. and Bayer, S. (2019).
\newblock A joint quantile and expected shortfall regression framework.
\newblock {\em Electron. J. Statist.}, 13(1):1823--1871.
\bibitem[Ehm and Gneiting, 2012]{EhmGneiting2012}
Ehm, W. and Gneiting, T. (2012).
\newblock {Local proper scoring rules of order two}.
\newblock {\em Ann. Statist.}, 40(1):609 -- 637.
\bibitem[Elliott et al., 2005]{ElliottETAL2005}
Elliott, G., Timmermann, A., and Komunjer, I. (2005).
\newblock {Estimation and Testing of Forecast Rationality under Flexible Loss}.
\newblock {\em Rev. Econ. Stud.}, 72(4):1107--1125.
\bibitem[Embrechts and Wang, 2015]{EmbrechtsWang2015}
Embrechts, P. and Wang, R. (2015).
\newblock Seven proofs for the subadditivity of expected shortfall.
\newblock {\em Depend. Model.}, 3:126--140.
\bibitem[Fissler et al., 2021]{FFHR2021}
Fissler, T., Frongillo, R., Hlavinov\'a, J., and Rudloff, B. (2021).
\newblock Forecast evaluation of quantiles, prediction intervals, and other
set-valued functionals.
\newblock {\em Electron. J. Statist.}, 15(1):1034--1084.
\bibitem[Fissler and Hoga, 2021]{FisslerHoga2021}
Fissler, T. and Hoga, Y. (2021).
\newblock Backtesting systemic risk forecasts using multi-objective
elicitability.
\newblock {\em Preprint}.
\newblock \url{https://arxiv.org/abs/2104.10673}.
\bibitem[Fissler and Pesenti, 2022]{FisslerPesenti2022}
Fissler, T. and Pesenti, S. M. (2022).
\newblock Sensitivity measures based on scoring functions.
\newblock {\em Preprint}.
\newblock \url{https://doi.org/10.48550/arXiv.2203.00460}.
\bibitem[Fissler and Ziegel, 2019]{FisslerZiegel2019}
Fissler, T. and Ziegel, J. F. (2019).
\newblock Order-sensitivity and equivariance of scoring functions.
\newblock {\em Electron. J. Stat.}, 13(1):1166--1211.
\bibitem[Fissler and Ziegel, 2021]{FisslerZiegel2021}
Fissler, T. and Ziegel, J. F. (2021).
\newblock On the elicitability of range value at risk.
\newblock {\em Stat. Risk Model.}, 38(1-2):25--46.
\bibitem[Girardi and Tolga Erg{\"u}n, 2013]{GT13}
Girardi, G. and Tolga Erg{\"u}n, A. (2013).
\newblock Systemic risk measurement: Multivariate {GARCH} estimation of
{CoVaR}.
\newblock {\em J. Bank. Financ.}, 37(8):3169--3180.
\bibitem[Gneiting, 2011]{Gneiting2011}
Gneiting, T. (2011).
\newblock {Making and Evaluating Point Forecasts}.
\newblock {\em J. Amer. Statist. Assoc.}, 106:746--762.
\bibitem[Gneiting et al., 2007]{GneitingETAL2007}
Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007).
\newblock Probabilistic forecasts, calibration and sharpness.
\newblock {\em J. R. Stat. Soc. Ser. B. Stat. Methodol.}, 69:243--268.
\bibitem[Gneiting and Katzfuss, 2014]{GneitingKatzfuss2014}
Gneiting, T. and Katzfuss, M. (2014).
\newblock Probabilistic forecasting.
\newblock {\em Annu. Rev. Stat. Appl.}, 1(1):125--151.
\bibitem[Gneiting and Raftery, 2007]{GneitingRaftery2007}
Gneiting, T. and Raftery, A. (2007).
\newblock {Strictly Proper Scoring Rules, Prediction, and Estimation}.
\newblock {\em J. Amer. Statist. Assoc.}, 102:359--378.
\bibitem[Gneiting and Ranjan, 2011]{GneitingRanjan2011}
Gneiting, T. and Ranjan, R. (2011).
\newblock {Comparing Density Forecasts Using Threshold- and Quantile-Weighted
Scoring Rules}.
\newblock {\em J. Bus. Econom. Statist.}, 29(3):411--422.
\bibitem[Gneiting and Ranjan, 2013]{GneitingRanjan2013}
Gneiting, T. and Ranjan, R. (2013).
\newblock Combining predictive distributions.
\newblock {\em Electron. J. Statist.}, pages 1747--1782.
\bibitem[Gneiting and Resin, 2021]{GneitingResin2021}
Gneiting, T. and Resin, J. (2021).
\newblock Regression diagnostics meets forecast evaluation: Conditional
calibration, reliability diagrams, and coefficient of determination.
\newblock {\em Preprint}.
\newblock \url{https://doi.org/10.48550/arXiv.2108.03210}.
\bibitem[Good, 1952]{Good1952}
Good, I. J. (1952).
\newblock Rational decisions.
\newblock {\em J. R. Stat. Soc. Ser. B. Stat. Methodol.}, 14(1):107--114.
\bibitem[Hoga and Dimitriadis, 2021]{HogaDimitriadis2021}
Hoga, Y. and Dimitriadis, T. (2021).
\newblock On testing equal conditional predictive ability under measurement
error.
\newblock {\em Preprint}.
\newblock \url{https://arxiv.org/abs/2106.11104}.
\bibitem[Holzmann and Eulert, 2014]{HolzmannEulert2014}
Holzmann, H. and Eulert, M. (2014).
\newblock The role of the information set for forecasting -- with applications
to risk management.
\newblock {\em Ann. Appl. Stat.}, 8:79--83.
\bibitem[Holzmann and Klar, 2017]{HolzmannKlar2017}
Holzmann, H. and Klar, B. (2017).
\newblock Focusing on regions of interest in forecast evaluation.
\newblock {\em Ann. Appl. Stat.}, 11(4):2404--2431.
\bibitem[Huber and Ronchetti, 2009]{HuberRonchetti2009}
Huber, P. J. and Ronchetti, E. M. (2009).
\newblock {\em Robust Statistics}.
\newblock John Wiley & Sons, Inc., Hoboken, New Jersey, second edition.
\bibitem[Hyv{{\"a}}rinen, 2005]{Hyvaerinen2005}
Hyv{{\"a}}rinen, A. (2005).
\newblock Estimation of non-normalized statistical models by score matching.
\newblock {\em J. Mach. Learn. Res.}, 6(24):695--709.
\bibitem[Kiesel et al., 2016]{KieselETAL2016}
Kiesel, R., R\"uhlicke, R., Stahl, G., and Zheng, J. (2016).
\newblock The wasserstein metric and robustness in risk management.
\newblock {\em Risks}, 4(32).
\bibitem[Koenker, 2005]{Koenker2005}
Koenker, R. (2005).
\newblock {\em {Quantile Regression}}.
\newblock Cambridge University Press, Cambridge.
\bibitem[Kr\"{a}tschmer et al., 2014]{KratschmeSchiedETAL2014}
Kr\"{a}tschmer, V., Schied, A., and Z\"{a}hle, H. (2014).
\newblock Comparative and qualitative robustness for law-invariant risk
measures.
\newblock {\em Finance Stoch.}, 18:271--295.
\bibitem[Lambert, 2019]{Lambert2019}
Lambert, N. (2019).
\newblock {Elicitation and Evaluation of Statistical Functionals}.
\newblock {\em Preprint}.
\newblock \url{http://ai.stanford.edu/ nlambert/papers/elicitability.pdf}.
\bibitem[Lambert et al., 2008]{LambertETAL2008}
Lambert, N., Pennock, D. M., and Shoham, Y. (2008).
\newblock Eliciting properties of probability distributions.
\newblock In {\em {Proceedings of the 9th ACM Conference on Electronic
Commerce}}, pages 129--138, Chicago, Il, USA. ACM.
\bibitem[Molchanov, 2017]{Molchanov2017}
Molchanov, I. (2017).
\newblock {\em Theory of Random Sets}.
\newblock Probability Theory and Stochastic Modelling. Springer-Verlag London,
London, 2 edition.
\bibitem[Nau, 1985]{Nau1985}
Nau, R. F. (1985).
\newblock {Should Scoring Rules Be `Effective'?}
\newblock {\em Manag. Sci.}, 31(5):527--535.
\bibitem[Newey and Powell, 1987]{NeweyPowell1987}
Newey, W. K. and Powell, J. L. (1987).
\newblock {Asymmetric Least Squares Estimation and Testing}.
\newblock {\em Econometrica}, 55:819--847.
\bibitem[Nolde and Ziegel, 2017]{NoldeZiegel2017}
Nolde, N. and Ziegel, J. F. (2017).
\newblock {Elicitability and backtesting: Perspectives for banking regulation}.
\newblock {\em Ann. Appl. Stat.}, 11(4):1833--1874.
\bibitem[Parry et al., 2012]{ParryDawidLauritzen2012}
Parry, M., Dawid, A. P., and Lauritzen, S. (2012).
\newblock {Proper local scoring rules}.
\newblock {\em Ann. Statist.}, 40(1):561--592.
\bibitem[Pohle, 2020]{Pohle2020}
Pohle, M.-O. (2020).
\newblock The {M}urphy decomposition and the calibration-resolution principle:
A new perspective on forecast evaluation.
\newblock {\em Preprint}.
\newblock \url{https://doi.org/10.48550/arXiv.2005.01835}.
\bibitem[Steinwart et al., 2014]{SteinwartPasinETAL2014}
Steinwart, I., Pasin, C., Williamson, R., and Zhang, S. (2014).
\newblock {Elicitation and Identification of Properties}.
\newblock {\em JMLR Workshop Conf. Proc.}, 35:1--45.
\bibitem[{van der Vaart}, 1998]{VanderVaart1998}
{van der Vaart}, A. W. (1998).
\newblock {\em {A}symptotic statistics}.
\newblock Cambridge Series in Statistical and Probabilistic Mathematics.
Cambridge University Press.
\bibitem[Weber, 2006]{Weber2006}
Weber, S. (2006).
\newblock {Distribution-Invariant Risk Measures, Information, and Dynamic
Consistency}.
\newblock {\em Math. Finance}, 16:419--441.