Quantile Graphical Models: Prediction and Conditional Independence with Applications to Systemic Risk

Quantile Graphical Models: Prediction and Conditional Independence with Applications to Systemic Risk

Introduction

Co-movements, dependence and influence between variables are fundamental in economics and finance for decision and policy making as well as prediction. To this end, we propose Quantile Graphical Models (QGMs) as a modeling framework and consider their usefulness in three main applications. First, empirical auction models often rely on independent private values or affiliated private values, detecting collusion in these auctions is a form of testing conditional independence. Examples of studying entry, market power, or collusion can be found in bajari2003deciding, porter2005detecting, harrington2008detecting. Second, QGMs can be used to identify and measure systemic tail risk. The recent bank and sovereign crisis in the US and Europe have also boosted the interest in the important role of network spill-over effects in contagion and shaping systemic risk (Acemoglu2010, Acemoglu2013, elliott2014financial, hansen2014challenges). Many measures of systemic risk focus on spill-overs fit naturally in our QGM setting (Adrian2011, Andersen2013, hautsch2014forecasting, hautsch2015financial, hardle2016tenet). We apply these insights to re-evaluate international financial contagion in volatilities (Claessens2001).\footnote{Works on economic and financial networks include Billio2010, bonaldi2015empirical, kastl2017recent. We refer to de2017econometrics for an excellent review on the econometrics literature on networks.} Third, QGMs can measure dependence between stock returns for hedging strategies. In financial management settings, risk quantification is crucial, and advanced hedging decisions are typically focused on the tail of the distribution of stock returns rather than the mean. Moreover, such strategies aiming to reduce risk are critical precisely during the market downside movement. Empirical evidence (Ang2006,Ang2002,Patton2004) points to the non-Gaussianity of the distribution of stock returns, especially during market downturns. Therefore, it is also instructive to understand how dependence (and policy impact) would change as the downside movement of the market becomes more extreme. The proposed QGMs are flexible enough to cover all these cases.

QGMs can be viewed as part of graphical models which have been successively applied to estimate and visualize relationship (maathuis2018handbook,heckman2015causal). Graphical models are widely used in machine learning, statistical learning, and social science to model the statistical dependence among the components a $d$-dimensional random vector $X_{V}$, in the form of a graph or network $G=(V, E)$. Here $V$ is the node set contains the labels of the components and $E$ is the edge set represents unknown statistical relationships that need to be estimated, thus poses novel problems of statistical inference. In the case of Gaussian Graphical Models (GGMs), which assuming $X_{V}$ are jointly Gaussian distributed, the conditional independence structure is completely characterized by the support of the inverse of the covariance matrix of $X_{V}$. Notably in this case, the same graph will characterize conditional independence and the best linear prediction. However, in non-Gaussian settings, not only it is harder to characterize conditional independence, there are no reasons for the same graph to characterize both conditional independence and best linear prediction.

QGMs provide an alternative route to learn conditional independence and prediction under asymmetric loss functions which is appealing in non-Gaussian settings. As in non-Gaussian cases these notions do not coincide and there are needs for different estimation approaches. We propose two different QGMs to handle different types of applications. First, we propose Conditional Independence Quantile Graphical Models (CIQGMs) to characterize the conditional independence of distributions through evaluating the distributional dependence structure at each quantile index. Second, we propose Prediction Quantile Graphical Models (PQGMs) in which predictive relationship is the main focus. Note, QGMs also enable us to focus on specific parts of the distributions of variables, which play an important role in applications like financial contagion and measuring systemic risk contributions where extreme events are the main interests for practitioners.

CIQGMs can be used for validation of the graph structure in the causal graphical models (pearl2009causality, robins1986new, heckman2015causal). Conditional independence has a long history in statistical models with consequences towards parameter identification, causal inference, prediction sufficiency, and many others, see dawid1979conditional. CIQGMs aim to characterize conditional independence via the conditional quantile functions. In such models, we consider a flexible specification that can approximate well the conditional quantile functions (up to a vanishing approximation error). In turn, this allows detecting which variables have a strong or near zero impact on others which can then be used to provide guidance on conditional independence.

PQGMs focus on the prediction of a variable based on linear combinations of other variables (a reduced form relation) under asymmetric losses. An important motivation for proposing PQGMs is to allow for misspecification as the conditional quantile function is typically non-linear in non-Gaussian settings. The linear specification is widely used in practice despite possible misspecification which motivates an analysis for accomodating these issues. We characterize the uniform prediction properties under a family of asymmetric loss functions, which this family enables practitioners to investigate different parts of the tail distribution. Other papers investigated the impact of misspecification on quantile functions are abadie2014inference, angrist2006quantile, knight2008asymptotics, lee2009efficiency. Our analysis also contributes to the high-dimensional quantile regression by allowing non-vanishing misspecification.

Broadly speaking, QGMs enhance our understanding of statistical dependence among $X_V$. For example, for each quantile index $\tau$, they provide visualization of the dependence via graphs whose edges represent conditional (quantile) relationships. Given that for each specific quantile index $\tau$ we will obtain one such graph, we could have a graph process indexed by $\tau \in (0,1)$. The structure represented by a $\tau$-quantile graph represents a local relation and can be valuable in cases where tail interdependence might be of special interest.\footnote{This is similar to the contrast between quantile regression and linear regression, where the latter provides information only on the conditional mean, while the former can provide a more complete description of the distribution of the outcome.} The graph process induced by QGMs has several important features. First, a $\tau$-quantile graph enables different values of edge strength in different directions. This is important because for undirected networks, the distinction is unclear. Second, QGMs can capture the tail interdependence through estimating at a high or low quantile index. \footnote{Examples of high or low quantile index can be $\tau = 0.95$ or $\tau = 0.05$ respectively. The analysis extends to the case of $s^3\log^5p = o(n \tau (1-\tau))$. The case of $n \tau = C$, even in the fixed dimension case, leads to a substantially different analysis and limiting distributions, as shown in chernozhukov2005extremal. Here $s$ is the sparsity parameter, $p$ is the dimension of conditional variables, $n$ is the sample size, $C$ is a constant.} Third, QGMs can capture the asymmetric dependence structure at different quantiles, which can be particularly useful in empirical applications (e.g., stock market dependence, exchange rate dependence). By considering all the quantiles at once we can characterize conditional independence structure for a set of variables that are not jointly Gaussian.

We also provide and study the estimation procedures that allow us to learn QGMs from the observed data. Our techniques are geared for covering high-dimensional settings where the size of the model is potentially larger than the sample size. These techniques are based on $\ell_{1}$-penalized quantile regression and Neyman orthogonal equations. For CIQGMs, under mild regularity conditions, we provide rates of convergence and edge properties of the estimated graph that hold uniformly over a large class of data generating processes. We provide simultaneously valid confidence regions (post-selection) for the coefficients of the CIQGM that are uniformly valid, despite of possible model selection mistakes. Based on proper thresholding, recovery of CIQGMs patterns is possible when coefficients are well separated from zero which parallel the results for graph recovery in the Gaussian case.\footnote{Similar to graph recovery in the Gaussian case such exact recovery is subject to the lack of uniformity validity critiques of Leeb and P\"otscher leeb2008can.} For PQGMs, we provide an estimator that achieves an adaptive rate of convergence, which might differ under different conditioning events. Therefore we contribute to the recent active literature on simultaneous valid confidence regions post-model selection, BCCH2012sparse,BCH2014inference,BCFH2013program,Farrell:JMP,CanerKock:HDCI,CHS:PnP GRD2014,zhang2014confidence,belloni2015uniform,BCK2013robustQR,jankova2015confidence,ning2017general,wang2016inference; in particular, the penalty choices and theoretical results are uniformly valid and adaptive to the relevant conditioning events.

Although we build upon the quantile regression literature (Koenker2005,BCK2013robustQR), we derive new results for penalized quantile regression in high dimensional settings that are uniformly valid, robust to small coefficients (e.g. allowing for model selection mistakes), allow possibly non-vanishing misspecification, many controls and a continuum of additional conditioning events. These results contribute to a growing literature that relies on quantile based models to characterize the data generating process. zheng2015globally considers a globally adaptive quantile regression model, establishes oracle properties, and improved rates of convergence for the high-dimensional case. Screening procedures based on moment conditions motivated by the quantile models have been proposed and analyzed in he2013quantile, wu2015conditional in the high-dimensional case. joe1997multivariate considers tail dependence defined via conditional probabilities in a low dimensional setting.

Finally, we view QGMs as complementary to a large body of works on GGMs (Dempster1972,Lauritzen1996,Cox1996,Edwards2000,Drton2004,Drton2007,Drton2008, meinshausen2006high, Yuan2007,Banerjee2008,Friedman2008, Yuan2010,Cai2011,Liu2012a,Sun2012,Liu2012b,chiong2017estimation). Our work is also complementary to other works trying to relax the joint Gaussian assumption. Liu2009,Liu2012,Xue2012 work with so-called nonparanormal models or semiparametric Gaussian copula models, i.e., the variables follow a joint Gaussian distribution after monotone transformations. Ravikumar2011, peng2009partial work with Sub-Gaussian data, which restricted fatness of the tails. ravikumar2010high, Xue2012a, loh2012structure work with discrete-valued random variable, and few types of exponential families. yang2015graphical provides results for M-estimators for a subclass of exponential family graphical models. QGMs allow for different sets of distributions.

The rest of the paper is organized as follows. Section (ref) provides main motivating examples. Section (ref) lays out the foundation of the conceptual framework of QGMs. Section (ref) contains estimators for QGMs while Section (ref) contains the theoretical guarantees of the estimators. Section (ref) provides an empirical application of QGMs to measure systemic risk contribution. Finally, the appendix contains proofs, simulations, and implementation details of the estimators.

{\bf Notation}. For an integer $k$, we let $[k]:=\{1,\ldots,k\}$ denote the set of integers from $1$ to $k$. For a random variable $X$ we denote by $\mathcal{X}$ its support. We use the notation $a\vee b=\max\{a,b\}$ and $a\wedge b=\min\{a,b\}$. We use $\|v\|_{p}$ to denote the $p$-norm of a vector $v$. In particular, the $\ell_2$-norm is denoted by $\Vert \cdot \Vert$; the $\ell_{0}$-“norm” $\Vert\cdot\Vert_{0}$ denotes the number of non-zero components. Given a vector $\delta\in\mathbb{R}^{p}$, and a set of indices $T\subset\{1,...,d\}$, we denote by $\delta_{T}$ the vector in which $\delta_{Tj}=\delta_{j}$ if $j\in T$, $\delta_{Tj}=0$ if $j\notin T$. We use $\mathbb{E}_{n}$ to abbreviate the notation $n^{-1}\sum_{i=1}^n$; for example, $\mathbb{E}_{n}[f]:=\mathbb{E}_{n}[f(\omega_i)]:=n^{-1}\sum_{i=1}^nf(\omega_i)$.

Motivating Examples

Screening for Collusion

One application of CIQGM is in the empirical auction literature on the examination of entry or bidding prices patterns to detect coordinating groups. As shown in bajari2003deciding, firms’ bids, after controlling for all information about costs, are jointly independent under the competitive model and lack of independence is taken as evidence consistent with collusion. Although, collusion is only one alternative explanation, testing conditional independence can be viewed as a screening device to determine whether further investigation is warranted.

Mathematically, denote $Y_{a,t}$ as the amount bid by firm $a$ on project $t$, and $Z_{a,t}$ as covariates observed in dataset. We define $X_{a,t}$ to be residuals after projecting out $Z_{a,t}$. bajari2003deciding test whether $X_{a}$ is independent of $X_{b}$, using Fisher's Z-transformation of the coefficient of correlation between $X_{a}$ and $X_{b}$. For two Gaussian random variables, this is equivalent to test pairwise independence. In our terminology, this corresponds to an edge $(a,b)$ not being contained in the graph if and only if

Pairwise independence, however, needs not imply joint independence. CIQGM can also handle joint independence in the non-Gaussian setting. This is because CIQGM works with the following case

namely an edge $(a,b)$ is not contained in the graph if and only if $X_{b}$ and $X_{a}$ are independent conditional on all remaining variables $X_{V\backslash\{a,b\}}=\{X_{k};k\in V\backslash\{a,b\}\}$, via the equivalence between conditional probabilities and conditional quantiles (details can be found in Section (ref)).

Systemic Tail Risk

Measuring systemic risk taking into account tail risk network spillover effects is another application of QGMs, e.g. our framework complements to the systemic risk measure CoVaR Adrian2011 which ignore tail risk dependencies induced by the underlying financial network structure. Our framework also allows for large scale networks.

Traditional tail risk measures, such as Value of Risk (VaR), focus on the loss of an individual institution. CoVaR attempts to measure the VaR of the whole financial system or a particular financial institution by conditioning on another being in distress. Formally, Adrian2011 define institution $b$'s CoVaR at level $\tau$ conditional on a particular outcome of institution $a$, as the value of $CoVaR_{\tau}^{b\vert a}$ that solves

for some event $ \mathbb{C}(X_{a}) $ based on $X_{a}$. A special case is $\mathbb{C}(X_{a})=\{ X_a = VaR_{\tau}^{a}\}$ which, as interpreted by Adrian2011, means with probability $\tau$ institution $b$ is in trouble given that institution $a$ is in trouble.

QGMs can work with the case

with the main difference here is the conditioning events (or variables), i.e. from $\mathbb{C}(X_{a})$ to $\mathbb{C}(X_{a}, X_{V\backslash\{a,b\}})$ with the latter could be high dimensional. Hence, our QGMs take into account risk spillovers from other institutions driving the CoVaR. The identified risk spillovers between all financial institutions constitute a financial tail risk network which, as shown later, can be used for measuring institutions' systemic tail risk contributions. In summary, QGMs can take into account the system-wide network spillover effects via incorporating tail network spillover effects into risk measuring, thus relate systemic risk to tail spillover effects from individual institutions to the whole system.

Another related definitions of tail risk, $\Delta CoVaR$, is defined as the change in the VaR of the whole financial system conditional on a institution being under distress relative to its median state. In terms of estimation, replacing covariate $X_{a}$ by the difference between its $\tau$-th quantile (denoted as $VaR_{\tau}^{a}$), and its median (denoted as $VaR_{50\%}^{a}$), yields

where $\widehat{\beta}_a^{b}(\tau)$ comes from pairwise quantile regression of $X_b$ on $X_a$, and

where ${\check{\beta}}^{b}(\tau)$ is estimated via Algorithm (ref) or (ref), inference procedures are based on Corollary (ref).

After learning a tail risk network, we can use our new network-cooperated $\Delta CoVaR$ to measure the systemic risk contribution of each institution. The systemic risk contribution of institution $a$ can be measured by it "to" or "from" degree, see Andersen2013; or by other network centrality measures, see jackson2010social, Kolaczyk2009, newman2010networks. To-degrees measure contributions of individual institutions to the overall risk of systemic network events, for institution $a$ is defined as $\delta_{a}^{to} = \sum_{k}\Delta CoVaR_{\tau}^{k\vert a,V\backslash\{a,k\}}$. From-degrees measure exposure of individual institutions to systemic shocks from the network, for institution $a$ it is defined as $\delta_{a}^{from} = \sum_{k}\Delta CoVaR_{\tau}^{a\vert k,V\backslash\{a,k\}}$. The net contribution of institution $a$ is defined as net-$\Delta CoVaR^{a}=\mbox{\ensuremath{\delta}}_{a}^{to}-\delta_{a}^{from}$.

In Section (ref), we revisit the analysis of international financial contagion through the volatility spillovers perspective. We visualize the tail risk interdependence via PQGMs as they allow for heteroskedasticity and asymmetric responses, can be used to model nonlinear tail interdependence, and to visualize potential asymmetric changes in conditional correlations. The estimated contagion network taking into account global interconnectedness is important for Eurozone financial regulators who want to identify globally systemically important EU countries, or for global financial portofolio diversification. Our the systemic tail risk analysis tools mentioned in previous paragraphs, can help with achieve those goals.

Stock Returns Under Market Downside Movements

Hedging decisions rely on the dependence of various stocks returns. Moreover, hedging is even more relevant during market downside movements, which motivates us to understand interdependence conditional on those events. Stock returns are in general non-Gaussian in those settings, as shown in the empirical finance literature, e.g. Ang2002, Longin2001,Patton2006. We can parameterize the downside movements by using a random variable $W$, which could be the market index, and conditional on the event $\Omega_\varpi = \{W \leqslant \varpi\}$. This allows us to define a $\varpi$-conditional-CIQGM as $G^I(\tau,\varpi)=(V,E^I(\tau,\varpi))$ and a $\varpi$-conditional-PQGM as $G^P(\tau,\varpi)=(V,E^P(\tau,\varpi))$, for each $\varpi \in \mathcal{W}$. We might be interest in a fixed $\varpi$ or on a family of values $\varpi\in(-\bar \varpi, 0]$. The latter induces $\mathcal{W} = \{\Omega_\varpi=\{W\leqslant \varpi\} : \varpi\in(-\bar \varpi, 0] \}$.

Figure (ref) provides an example using $ \mathcal{W}$-Conditional QGM with \sloppy$W \text{= \{Market Index Returns\}}$, $\varpi$ as the $\tau_m$-th quantile of the market index returns, and $\tau_m = \{0.15, 0.5, 0.75, 0.9\}$. We obtain daily stock returns from CRSP and use S&P 500 as the market index. The full sample consists of 2769 observations for 86 stocks from Jan 2, 2003 to December 31, 2013. The total number of stocks is 86 due to data availability. We define market movement as when the market index returns are below a pre-specified level (e.g. $\tau_m$-th quantile), hence conditioning on a particular $\varpi$ corresponds to consider the subsample based on whether the corresponding date's market return is less equal to the $\tau_m$-th quantile of the market index returns. The results show higher interdependence under market downside moments, pose different hedging decisions.

Quantile Graphical Models

In this section we describe quantile graphical models associated with a $d$-dimensional random vector $X_V$ where the set $V=[d]=\{1,\ldots,d\}$ denotes the labels of the components. These models aim to provide a description of the dependence between the random variables in $X_V$. In particular, these models induce graphs that allow for visualizing dependence structures. Nonetheless, because of the non-Gaussianity, we consider two fundamentally distinct models (one geared towards conditional independence and one geared towards prediction).

Conditional Independence Quantile Graphical Models

Conditional independence graphs have been used to provide visualization and insight on the dependence structure between random variables. Each node of the graph is associated with a component of $X_V$. We denote the conditional independence graph as $G^I=(V,E^I)$ where $G^I$ is an undirected graph with vertex set $V$ and edge set $E^I$ which is represented by an adjacency matrix ($E^I_{a,b}=1$ if the edge $(a,b)\in G^I$, and $E^I_{a,b}=0$ otherwise). An edge $(a,b)$ is not contained in the graph if and only if

namely $X_{b}$ and $X_{a}$ are independent conditional on all remaining variables $X_{V\backslash\{a,b\}}=\{X_{k};k\in V\backslash\{a,b\}\}$.

In order to achieve a tractable concept for non-Gaussian settings, we use that ((ref)) occurs if and only if

In turn, by the equivalence between conditional probabilities and conditional quantiles to characterize a random variable, we have that ((ref)) occurs if and only if

For a quantile index $\tau \in (0,1)$, the $\tau$-quantile conditional independence graph is a directed graph $G^I(\tau)=(V,E^I(\tau))$ with vertex set $V$ and edge set $E^I(\tau)$. An edge $(a,b)$ is not contained in the edge set $E^I(\tau)$ if and only if

By the equivalence between ((ref)) and ((ref)), the union of $\tau$-quantile graphs over $\tau\in(0,1)$ represents the conditional independence structure of $X$, namely $E^I=\cup_{\tau \in (0,1)} E^I(\tau)$. We also consider a relaxation of ((ref)). For a set of quantile indices $\mathcal{T}\subset (0,1)$, we say that

$X_{a}$ and $X_{b}$ are $\mathcal{T}$-conditionally independent given $X_{V\backslash\{a,b\}}$, if ((ref)) holds for all $\tau \in \mathcal{T}$. Thus, we have that ((ref)) implies ((ref)). We define the $\mathcal{T}$-quantile graph as $G^I(\mathcal{T})=(V,E^I(\mathcal{T}))$ where $$ E^I(\mathcal{T})= \cup_{\tau \in \mathcal{T}}E^I(\tau).$$ Although the conditional independence concept relates to all quantile indices, the quantile characterization described above also lends itself to quantile specific impacts which can be of independent interest.\footnote{For example, we might be interested in some extreme events which typically correspond to crises in financial systems.}

Prediction Quantile Graphical Models

Prediction Quantile Graphical Models (PQGMs) are motivated by prediction accuracy under an asymmetric loss function (instead of conditional independence as in Section (ref)). More precisely, for each $a\in V$, we are interested in predicting $X_a$ based on linear combinations of the remaining variables, $X_{V\backslash\{a\}}$, where accuracy is measured with respect to an asymmetric loss function. Formally, PQGMs measure accuracy as

where $X_{-a}=(1,X_{V\backslash\{a\}}')'$, and the asymmetric loss function $\rho_{\tau}(t)=(\ensuremath{\tau-1\{t\leqslant0\}})t$ is the check function used in KoenkerBassett1978.

Importantly, PQGMs are concerned with the best linear predictor under the asymmetric loss function $\rho_\tau$ which is a specification that is widely used in practice. This is a fundamental distinction with respect to CIQGMs discussed in Section (ref) where the specification of the conditional quantile was approximately a linear function of transformations of $X_{V\backslash \{a\}}$.\footnote{In Section (ref) the vector $Z^a$ in equation ((ref)) collects the functions of the vector $X_{V\backslash \{a\}}$.} Indeed, we note that under suitable conditions the linear predictor that solves the minimization problem in ((ref)) approximates the conditional quantile regression as shown in BCF2011. (In fact, the conditional quantile function would be linear if $X_V$ was jointly Gaussian distributed.) However, PQGMs do not assume that the conditional quantile function of $X_a$ is well approximated by a linear function and instead it focuses on the best linear predictor.

We define that $X_b$ is predictively uninformative for $X_a$ given $X_{V\backslash\{a,b\}}$ if $$ \mathcal{L}_a(\tau \mid V\backslash\{a\}) = \mathcal{L}_a(\tau \mid V\backslash\{a,b\}) \ \ \ \mbox{for all} \ \ \tau \in (0,1),$$ i.e., considering a linear function of $X_b$ will not improve our performance of predicting $X_a$ with respect to the asymmetric loss function $\rho_\tau$.

Again we can visualize the predictive relationship through a graph process indexed by $\tau \in (0,1)$. That is, for each $\tau \in (0,1)$ we have a directed graph $G^P(\tau)=(V,E^P(\tau))$, where an edge $(a,b)\in G^P(\tau)$ only if $X_b$ is predictively informative for $X_a$ given $X_{V\backslash\{a,b\}}$ at the quantile $\tau$. Finally, it is also convenient to define the PQGM associated with a subset $\mathcal{T}\subset(0,1)$ as $G^P(\mathcal{T})=(V,E^P(\mathcal{T}))$ where $$E^P(\mathcal{T})=\cup_{\tau \in \mathcal{T}}E^P(\tau).$$

$\mathcal{W}$-Conditional Quantile Graphical Models

In what follows, we discuss an extension of the QGMs discussed in Sections (ref) and (ref) to allows for conditioning on a (possible infinity) family of events $\varpi \in \mathcal{W}$.\footnote{With a slight abuse of notation, we let $\varpi$ to denote the event and also the index of such event. For example, we write ${\mathrm{P}}(\varpi)$ as a shorthand for ${\mathrm{P}}( W \in \Omega_\varpi)$.} Such extension is motivated by several applications in which the interdependence between the random variables in $X_V$ maybe substantially impacted by additional observable events (e.g. downside movements of the market). This general framework allows different forms of conditioning. The main implication of this extension is that QGMs are now graph processes indexed by $\tau \in \mathcal{T} \subset (0,1)$ and $\varpi \in \mathcal{W}$.

We define $X_a$ and $X_b$ are $(\mathcal{T},\varpi)$-conditionally independent,

if for all $\tau \in \mathcal{T}$ we have

The conditional independence edge set associated with $(\tau,\varpi)$ is defined analogously as before. We denote them by $E^I(\tau,\varpi)$ and $E^I(\mathcal{T},\varpi) = \cup_{\tau \in \mathcal{T}} E^I(\tau,\varpi)$ for each $\varpi \in \mathcal{W}$.

The extension of PQGMs proceeds by defining the accuracy under the asymmetric loss function conditionally on $\varpi$. More precisely, we define

The prediction edge set associated with $(\tau,\varpi)$ is also defined analogously as before. We denote them by $E^P(\tau,\varpi)$ and $E^P(\mathcal{T},\varpi) = \cup_{\tau \in \mathcal{T}} E^P(\tau,\varpi)$, for each $\varpi \in \mathcal{W}$.

Estimators for High-Dimensional Quantile Graphical Models

In this section, we propose and discuss estimators for QGMs introduced in Section (ref). Throughout it is assumed that we observe a $d$-dimensional i.i.d. random vector $X_V$, namely $\{X_{iV} : i=1,\ldots,n\}$. Based on the data observed, unless additional assumptions are imposed we cannot estimate the quantities of interest for all $\tau \in (0,1)$. Instead, in what follows we will consider a (compact) set of quantile index $\mathcal{T}\subset (0,1)$. The estimators are intended to handle high dimensional models and a continuum of conditioning events in $\mathcal{W}$.

Estimators for CIQGMs

We discuss the specification and propose an estimator for CIQGMs. Although in general it is potentially hard to correctly specify coherent models, the following are simple examples.

Although a linear specification is correct for Examples (ref) and (ref), Example (ref) and (ref) illustrate that we need to consider a more general transformation of the covariates $X_V$ in the specification for each conditional quantile function. Nonetheless, specifications with additional non-linear terms can approximate non-drastic departures from normality.

We will consider a conditional quantile representation for each $a\in V$. It is based on transformations of the original covariates $X_{V\backslash\{a\}}$ that create a $p$-dimensional random vector $Z^a=Z^a(X_{V\backslash\{a\}})$ such that

where $r_{a\tau}$ denotes a small approximation error. For $b\in V\backslash\{a\}$ we let $I_a(b):= \{ j : Z^a_{j} \ \mbox{depends on} \ X_b\}$. That is, $I_a(b)$ contains the components of $Z^a$ that are functions of $X_b$. Under correct specification, if $X_a$ and $X_b$ are conditionally independent, we have $\beta_{a\tau j} = 0$ for all $j\in I_a(b)$, $\tau \in (0,1)$.

This allows us to connect the conditional independence quantile graph estimation problem with model selection with quantile regression. Indeed, the representation ((ref)) has been used in several quantile regression models, see Koenker2005. Under mild conditions this model allows us to identify the process $(\beta_{a\tau})_{\tau \in \mathcal{T}}$ as the solution of the following moment equation

In order to allow for a flexible specification, so that the approximation errors are negligible, it is attractive to consider a high-dimensional $Z^a$ where its dimension $p$ is possibly larger than the sample size $n$. In turn, having a large number of technical controls creates an estimation challenge if the number of coefficients $p$ is not negligible with respect to the sample size $n$. In such a high dimensional setting, a widely applicable condition that makes estimation possible is approximate sparsity fan2011sparse,BCCH2012sparse,BCH2014inference. Formally we require

where the sparsity parameter $s$ of the model is allowed to grow (at a slower rate) as $n$ grows, and $f_{a\tau}=f_{X_a\mid X_{V\backslash\{a\}}}(Q_{X_a}(\tau\vert X_{V\backslash\{a\}})\vert X_{V\backslash\{a\}})$ denotes the conditional density function evaluated at the corresponding conditional quantile value. This sparsity also has implications on the maximum degree of the associated quantile graph.

Algorithm (ref) below contains our proposal to estimate $\beta_{a\tau}$, $a\in V$, $\tau \in \mathcal{T}$. It is based on three procedures in order to overcome high-dimensionality. In the first step, we apply a (post-)$\ell_1$-penalized quantile regression. The second step applies (post-)Lasso where the data is weighted by the conditional density function at the conditional quantile.\footnote{We note that an estimate for $f_{a\tau}$ is available from $\ell_1$-penalized quantile regression estimators for $\tau+h$ and $\tau-h$ where $h$ is a bandwidth parameter, see Koenker2005,BCK2013robustQR and Comment (ref).} Finally, the third step relies on constructing (orthogonal) score function that provides immunity to (unavoidable) model selection mistakes.

There are several parameters that need to be specified for Algorithm (ref). The penalty parameter $\lambda_{V\mathcal{T}}$ is chosen to be larger than the $\ell_\infty$-norm of the (rescaled) score at the true quantile function. The work in BC-SparseQR exploits the fact that this quantity is pivotal in their setting. Here, additional correlation structure would have an impact and the distribution is pivotal only for each $a\in V$. The penalty is based on the maximum of the quantiles of the following random variables (each with pivotal distribution), for $a\in V$

where $\{U_i: i=1,\ldots,n\}$ are i.i.d. uniform $(0,1)$ random variables, and $\widehat\sigma_{aj}^Z=\{{\mathbb{E}_n}[(Z_j^a)^2]\}^{1/2}$ for $j\in[p]$. The penalty parameter $\lambda_{V\mathcal{T}}$ is defined as $$\lambda_{V\mathcal{T}} := \max_{a\in V} \Lambda_{a\mathcal{T}}(1-\xi/|V| \mid Z^a ), $$ that is, the maximum of the $1-\xi/|V|$ conditional quantile of $\Lambda_{a\mathcal{T}}$ given in ((ref)). Regarding the penalty term for the weighted Lasso in Step 2, we recommend a (theoretically valid) iterative choice. We refer to Appendix (ref) for the implementation details of the algorithm. We denote $\|\beta\|_{1,\widehat\sigma^Z}:=\sum_j \widehat\sigma_{aj}^Z|\beta_j|$ the standardized version of the $\ell_1$-norm.

Algorithm (ref) above has been studied in BCK2013robustQR where it is applied to a single triple $(a,\tau, j)$, and we have used the following parameter space for $\alpha$, $\mathcal{A}_{a\tau j} = \{ \alpha \in {\mathbb{R}} : |\alpha - \widetilde \beta_{a\tau j}| \leqslant 10 / \{\widehat \sigma_{aj}^Z \log n \} \}$. Under similar conditions, results that hold uniformly over $(a,\tau, j) \in V\times \mathcal{T}\times [p]$ are achievable (as shown in the next sections) building upon the tools developed in BC-SparseQR and chernozhukov2012gaussian. Algorithm (ref) is tailored to achieve good rates of convergence in the $\ell_\infty$-norm. In particular, under standard regularity conditions, with probability approaching to 1 we have $$ \sup_{\tau \in \mathcal{T}} \| \beta_{a\tau} - \check\beta_{a\tau} \|_\infty \lesssim \sqrt{\frac{\log (p|V|n)}{n}}.$$ In order to create an estimate of $E^I(\tau)=\{ (a,b) \in V\times V: \max_{j\in I_a(b)}|\beta_{a\tau j}|>0\}$, we define $$\widehat E^I(\tau) = \left\{ (a,b)\in V\times V : \ \max_{j\in I_a(b)}\frac{|\check\beta_{a\tau j}|}{\mbox{se}(\check\beta_{a\tau j})} > \overline{\rm cv}\right\}$$ where $\mbox{se}(\check\beta_{a\tau j})=\{\tau(1-\tau){\mathbb{E}_n}[\widetilde v^2_{ia\tau j}]^{-1}\}^{1/2}$ with $\tilde{v}_{ia\tau j} = \widehat{f}_{ia\tau}\{ Z^a_{ij} - Z^a_{i,-j}\tilde{\gamma}^j_{a\tau}\}$, is an estimate of the standard deviation of the estimator, and the critical value $\overline{\rm cv}$ is set to account for the uniformity over $a\in V$, $\tau \in \mathcal{T}$, and $j\in[p]$. We discuss in the following sections a data driven procedure based on multiplier bootstrap that is theoretically valid in this high dimensional setting.

Estimators for PQGMs

In this section we propose an estimator for PQGMs in which case we are interested in the prediction of $X_a$, $a\in V$, using a linear combination of $X_{V\backslash\{a\}}$ under the asymmetric loss discussed in ((ref)). We will add an intercept as one of the variables for the sake of notation so that $X_{-a}=(1,X_{V\backslash\{a\}}')'$. Given the loss function $\rho_\tau$, the target $d$-dimensional vector of parameters $\beta_{a\tau}$ is defined as (part of) the solution of the following optimization problem

As we are interested in the case that $d$ is large, the use of high-dimensional tools to achieve consistent estimators is needed. The estimation procedure we proposed is based on $\ell_1$-penalized quantile regression but additional issues need to be considered to cope with the (non-vanishing) difference between the best linear predictor and the conditional quantile function. Again we consider models that satisfy an approximately sparse condition. Formally, we require the existence of sparse coefficients $\{\bar\beta_{a\tau}:a\in V, \tau \in \mathcal{T}\}$ such that

where (again) the sparsity parameter $s$ of the model is allowed to grow as $n$ grows. The high-dimensionality prevents us from using (standard) quantile regression methods and regularization methods are needed to achieve good prediction properties.

A key issue is to set the penalty parameter properly so that it bounds from above

However, it is important to note that we do not assume that the conditional quantile of $X_a$ is a linear function of $X_{-a}$. Under correct linear specification of the conditional quantile function, $\ell_1$-penalized quantile regression estimator has been studied in BC-SparseQR. The case that the conditional quantile function differs from a linear specification by vanishing approximation errors has been considered in kato2011 and BCK2013robustQR. The analysis proposed here aims to allow for non-vanishing misspecification of the quantile function relative to a linear specification while still guarantees good rates of convergence in the $\ell_2$-norm to the best linear specification. Thus the penalty parameter in the penalized quantile regression needs to account for such misspecification and is no longer pivotal as in BC-SparseQR.

In order to handle this issue we propose a two step estimation procedure. In the first step, the penalty parameter $\lambda_0$ is conservative and is set via bounds constructed based on symmetrization arguments, similar in spirit to vdGeer2008,Volume2013. This leads to $\lambda_0 = 2(1+1/16) \sqrt{2\log(8|V|^2/\xi)/n}$. Although this is conservative, under mild conditions this would lead to estimates that can be leverage to fine tune the penalty choice. The second step uses the preliminary estimator to bootstrap ((ref)) based on the tools in chernozhukov2013gaussian as follows. Specifically, for estimates $\widehat\varepsilon_{ia\tau}$ of the “noise" $\varepsilon_{ia\tau}=1\{X_{ia} \leqslant X_{i,-a}'\beta_{a\tau}\}-\tau$ for $i\in[n]$, for $a\in V$ define

where $(g_i)_{i=1}^n$ is a sequence of i.i.d. standard Gaussian random variables. The new penalty parameter $\bar\lambda_{V\mathcal{T}}$ is defined as

that is, the maximum of the $(1-\xi)$ conditional quantile of $\bar{\Lambda}_{a\mathcal{T}}$. The penalty choice above adapts to the unknown correlation structure across components and quantile indices. The following algorithm states the procedure where we denote weighted $\ell_1$-norms by $\|\beta\|_{1,\widehat\sigma^X}:=\sum_j\widehat{\sigma}_{aj}^X|\beta_j|$ with $\widehat{\sigma}_{aj}^X=\{{\mathbb{E}_n}[X_{j}^2]\}^{1/2}$ the standardized version of the $\ell_1$-norm and $\|\beta\|_{1,\widehat\varepsilon}:=\sum_j \widehat{\sigma}_{a\tau j}^{\varepsilon X}|\beta_j|$ with $ \widehat{\sigma}_{a\tau j}^{\varepsilon X} = \{{\mathbb{E}_n}[\widehat\varepsilon_{a\tau}^2X_{-a,j}^2]\}^{1/2}$ a norm based on the estimated residuals.

Under regularity conditions stated in Section (ref), with probability approaching 1, we have $$ \max_{a\in V} \sup_{\tau \in \mathcal{T}} \| \beta_{a\tau} - \check\beta_{a\tau} \| \lesssim \sqrt{\frac{s\log (|V|n)}{n}}.$$ The estimate of the prediction quantile graph is given by the support of $(\check\beta_{a\tau})_{a\in V, \tau \in \mathcal{T}}$, namely $$\widehat E^P(\tau) = \left\{ (a,b)\in V\times V : \ |\widehat\beta_{a\tau b}| > \bar \lambda_{V\mathcal{T}} / \widehat{\sigma}_{a\tau b}^{\varepsilon X} \right \}.$$ That is, it is induced by covariates selected by the $\ell_1$-penalized estimator. Those thresholded estimators not only have the same rates of convergence as of the original penalized estimators but also possess additional sparsity guarantees.

Estimators for $\mathcal{W}$-Conditional Quantile Graphical Models

In order to handle the additional conditioning events $\Omega_\varpi$, $\varpi\in\mathcal{W}$, we propose to modify Algorithms (ref) and (ref) based on kernel smoothing. To that extent, we assume the observed data is of the form $\{(X_{iV}, W_i) : i=1,\ldots, n\}$, where $W_i$ might be defined through additional variables. Furthermore, we assume for each conditioning event $\varpi\in \mathcal{W}$ we have access to a kernel function $K_\varpi$ that is applied to $W$, to represent the relevant observations associated with $\varpi$ (recall that we denote ${\mathrm{P}}( W \in \Omega_\varpi)$ as ${\mathrm{P}}(\varpi)$). We assume that $K_\varpi(W) = 1\{W \in \Omega_\varpi\}$.

This framework encompasses the previous framework by having $K_\varpi(W) = 1$ for all $W$. Moreover, it allows for a richer class of estimands which require estimators whose properties should hold uniformly over $\varpi \in \mathcal{W}$ as well. Next we propose estimators for this setting, i.e. we generalize the previous methods to account for the additional conditioning on $\varpi\in \mathcal{W}$. In what follows, with a slight abuse of notation we use $\varpi$ to denote not only the index but also the event $\Omega_\varpi$. For further notational convenience, we denote $u=(a,\tau,\varpi)\in \mathcal{U} := V\times\mathcal{T}\times\mathcal{W}$ so that the set $\mathcal{U}$ collects all the three relevant indices. With $\widehat{\sigma}^{Z}_{a\varpi j} = \{{\mathbb{E}_n}[K_\varpi(W)(Z_j^a)^2]\}^{1/2}$, we define the following weighted $\ell_1$-norm $ \|\beta\|_{1,\varpi} = \sum_{j\in [p]} \widehat{\sigma}^{Z}_{a\varpi j} |\beta_j|.$ This norm is $\varpi$ dependent and provides the proper adjustments as we condition on different events associated with different $\varpi$'s.

We first consider estimators of CIQGMs conditional on the events in $\mathcal{W}$. In this setting, the model is correctly specified up to small approximation errors. The definition of the penalty parameter will be based on the random variable $$ \Lambda_{a\mathcal{T}\mathcal{W}} = \sup_{\tau \in \mathcal{T}, \varpi \in \mathcal{W}} \max_{ j\in[p]} \left|\frac{{\mathbb{E}_n}[K_\varpi(W)(1\{U \leqslant \tau\} - \tau) Z^a_j]}{\sqrt{\tau(1-\tau)} \widehat{\sigma}^{Z}_{a\varpi j}}\right| $$ where $U_i$ are independent uniform $(0,1)$ random variables, and set the penalty $$\lambda_{V\mathcal{T}\mathcal{W}} = \max_{a\in V}\Lambda_{a\mathcal{T}\mathcal{W}}(1-\xi/\{|V|n^{1+2d_W}\}\vert Z^a, W),$$ that is, the maximum of the $(1-\xi/\{|V|n^{1+2d_W}\})$ conditional quantile of $\Lambda_{a\mathcal{T}\mathcal{W}} $. Algorithm (ref) provides the definition of the estimator. Here $\mathcal{A}_{uj} = \{ \alpha \in {\mathbb{R}} : |\alpha - \widetilde \beta_{uj}| \leqslant 10 / \{\widehat{\sigma}^{Z}_{a\varpi j} \log n \} \}$, and denote $\lambda_u := \lambda_{V\mathcal{T}\mathcal{W}}\sqrt{\tau(1-\tau)}$.

Next we consider estimators of PQGMs conditional on the events in $\mathcal{W}$. Similar to the previous case, for $a\in V$ define

where $(g_i)_{i=1}^n$ is a sequence of i.i.d. standard Gaussian random variables. The new penalty parameter $\bar\lambda_{V\mathcal{T}}$ is defined as

that is, the maximum of the $(1-\xi)$ conditional quantile of $\bar{\Lambda}_{a\mathcal{T}\mathcal{W}}$. It will also be useful to define another weighted $\ell_1$-norm, $\|\beta\|_{1,\varpi\widehat\varepsilon}:= \sum_{j} \widehat{\sigma}_{a\tau \varpi j}^{\varepsilon X}|\beta_j|$ with $ \widehat{\sigma}_{a\tau \varpi j}^{\varepsilon X} = \{{\mathbb{E}_n}[K_\varpi(W)\widehat\varepsilon_{a\tau\varpi}^2X_{-a, j}^2]\}^{1/2}$. We also denote $\widehat{\sigma}^{X}_{a\varpi j}=\{{\mathbb{E}_n}[K_\varpi(W)X_{-a,j}^2]\}^{1/2}$. The penalty choice and weighted $\ell_1$-norm adapt to the unknown correlation structure across components and quantile indices. The following algorithm states the procedure, with $ \lambda_{0\mathcal{W}} = 2(1+1/16) \sqrt{2\log(8|V|^2\{ne/d_W\}^{2d_W}/\xi)/n}$.

Main Theoretical Results

This section is devoted to theoretical guarantees associated with the proposed estimators. We will establish rates of convergence results for the proposed estimators as well as the (uniform) validity of confidence regions. These results build upon and contribute to an increasing literature on the estimation of many processes of interest with (high-dimensional) nuisance parameters.

Throughout, we will provide results for the estimators of the $\mathcal{W}$-conditional quantile graphical models as those can be generalized the other models by setting $K_\varpi(W) = 1$. Although some of the tools are similar, CIQGMs and PQGMs require different estimators and are subject to different assumptions. Thus, substantial different analyses are required.

$\mathcal{W}$-Conditional CIQGM

For $u=(a,\tau,\varpi)\in \mathcal{U}$, define the $\tau$-conditional quantile function of $X_a$ given $X_{V\backslash\{a\}}$ and $\varpi$ as

where $Z^a$ is a $p$-dimensional vector of (known) transformations of $X_{V\backslash \{a\}}$, and $r_u$ is an approximation error. The event $\varpi \in \mathcal{W}$ will be used for further conditioning through the function $K_\varpi(W) = 1\{ W \in \varpi\}$.

We let $f_{X_a\mid X_{V\backslash \{a\}}, \varpi }(\cdot\vert X_{V\backslash \{a\}},\varpi)$ denote the conditional density function of $X_a$ given $X_{V\backslash \{a\}}$ and $\varpi\in\mathcal{W}$. We define $f_{u}:=f_{X_a\vert X_{V\backslash \{a\}}, \varpi }(Q_{X_{a}}(\tau\vert X_{V\backslash \{a\}}, \varpi )\vert X_{V\backslash \{a\}}, \varpi )$ as the value of the conditional density function evaluated at the $\tau$-conditional quantile. In our analysis we will consider for $u\in\mathcal{U}$

Moreover, for each $u\in\mathcal{U}$ and $j\in[p]$ we define

This provides a weighted projection to construct the residuals $$ v_{uj} = f_u(Z^a_j - Z^a_{-j}\gamma_{u}^j) $$ that satisfy ${\mathrm{E}}[ f_{u} Z^a_{-j} v_{uj} \vert \varpi] =0$ for each $(u,j)\in \mathcal{U}\times[p]$.

The estimands of interest are $\beta_u \in {\mathbb{R}}^p $, $u \in \mathcal{U}$, and can be written as the solution of (a continuum of) moment equations. Letting $\beta_{uj}$ denote the $j$th component of $\beta_u$ so that $\beta_{uj} \in {\mathbb{R}}$ solves $$ {\mathrm{E}}[\psi_{uj}(X,W,\beta,\eta_{uj}) ] = 0, $$ where the function $\psi_{uj}$ is given by $$ \psi_{uj}(X,W,\beta,\eta_{uj}) = K_\varpi(W)( \tau - 1\{ X_a \leqslant Z^a_j \beta + Z^a_{-j}\eta_{uj}^{(1)} + \eta_{uj}^{(3)} \})f_u(Z_j^a-Z^a_{-j}\eta_{uj}^{(2)}),$$ and the true value of the nuisance parameter is given by $\eta_{uj}=(\eta_{uj}^{(1)},\eta_{uj}^{(2)},\eta_{uj}^{(3)})$ with $\eta_{uj}^{(1)}=\beta_{u,-j}$, $\eta_{uj}^{(2)}=\gamma^j_u$, and $\eta_{uj}^{(3)}=r_u$. In what follows $c, C$ denote some fixed constant, $\delta_n$ and $\Delta_n$ denote sequences go to zero with $\delta_n = n^{-\mu}$ for some sufficiently small $\mu$. Denote $\mu_\mathcal{W}=\inf_{\varpi\in\mathcal{W}}{\mathrm{P}}(\varpi)$.

{\bf Condition CI.} Let $u=(a, \tau, \varpi) \in \mathcal{U} := V \times \mathcal{T}\times \mathcal{W}$ and $(X_i,W_i)_{i=1}^n$ denote a sequence of independent and identically distributed random vectors generated accordingly to models ((ref)) and ((ref)):

(i) Suppose $\sup_{u\in\mathcal{U}, j\in[p]}\{\|\beta_u\|+\|\gamma^j_u\|\}\leqslant C$ and $\mathcal{T}$ is a fixed compact set: (a) there exists $s=s_n$ such that $\sup_{u\in\mathcal{U}, j\in[p]}\{\|\beta_u\|_0 + \|\bar \gamma^j_u\|_0\}\leqslant s$, $\sup_{u\in\mathcal{U}, j\in[p]}\|\bar \gamma^j_u-\gamma^j_u\|+s^{-1/2}\|\bar \gamma^{j}_u-\gamma^{j}_u\|_1\leqslant C \{n^{-1}s\log(|V| p n)\}^{1/2}$, where $\bar \gamma^j_u$ is approximately sparse; (b) the conditional distribution function of $X_a$ given $X_{V\backslash \{a\}}$ and $\varpi$ is absolutely continuous with continuously differentiable density $f_{X_a\vert X_{V\backslash \{a\}},\varpi}(t\vert X_{V\backslash \{a\}},\varpi)$ bounded by $\bar f$ and its derivative bounded by $\bar f'$ uniformly over $u\in\mathcal{U}$; (c) $|f_u-f_{u'}|\leqslant L_f\|u-u'\|$, $\|\beta_u-\beta_{u'}\|\leqslant L_\beta \|u-u'\|^\kappa$ with $\kappa \in [1/2,1]$, and ${\mathrm{E}}[ |K_\varpi(W)-K_{\varpi'}(W)|] \leqslant L_K\|\varpi-\varpi'\|$; (d) the VC dimension $d_W$ of the set $\mathcal{W}$ is fixed, $\{Q_{X_a}(\tau\vert X_{V\backslash \{a\}},\varpi) : (\tau,\varpi)\in \mathcal{T} \times \mathcal{W}\}$ is a VC-subgraph with VC-dimension $1+Cd_W$ for every $a\in V$;

(ii) The following moment conditions hold uniformly over $u\in\mathcal{U}$ and $j\in[p]$: ${\mathrm{E}}[|f_uv_{uj}Z^a_k|^2\vert \varpi]^{1/2}\leqslant C{\underline{f}}_u$, $\min_{a\in V}\inf_{\|\delta\|=1}{\mathrm{E}}[\{(X_a,Z^a)\delta\}^2 \vert \varpi]\geqslant c$, $ \max_{a\in V}\sup_{\|\delta\|=1}{\mathrm{E}}[ \{(X_a,Z^a)\delta\}^4 \vert \varpi ]\leqslant C$, ${\mathrm{E}}[f_u^2 (Z^a\delta)^{2}\vert \varpi] \leqslant C{\underline{f}}_u^2{\mathrm{E}}[(Z^a\delta)^{2}\vert \varpi ]$, $\max_{j,k}\frac{{\mathrm{E}}[|f_uv_{uj}Z^a_k|^3\vert \varpi]^{1/3}}{{\mathrm{E}}[|f_uv_{uj}Z^a_k|^2\vert \varpi]^{1/2}}\log^{1/2}(pn|V|) \leqslant \delta_n \{n {\mathrm{P}}(\varpi)\}^{1/6}$;

(iii) Furthermore, for some fixed $q\geqslant 4\vee (1+2d_W)$, $\sup_{u\in\,\mathcal{U},\|\delta\|=1} {\mathrm{E}}[|(X_a,Z^a)\delta|^2r_u^2 \vert\varpi]\leqslant C {\mathrm{E}}[r_u^2 \vert \varpi] \leqslant Cs/n$, $\max_{u\in\mathcal{U}, j\in[p]}|{\mathrm{E}}[f_ur_uv_{uj}\vert \varpi]|\leqslant \delta_n n^{-1/2}$, ${\mathrm{E}}[ \max_{i\leqslant n} \sup_{u\in \mathcal{U}}|K_\varpi(W)r_{iu}|^q ] \leqslant C$, and with probability $1-\Delta_n$, uniformly over $u\in \mathcal{U}, j\in[p]$: ${\mathbb{E}_n}[r_u^2v_{uj}^2\vert\varpi] + {\mathbb{E}_n}[r_u^2\vert\varpi]\lesssim n^{-1}s\log(p|V| n)$, ${\mathbb{E}_n}[K_\varpi(W)\{|r_u|+r_u^2\}(Z^a\delta)^2] \leqslant \delta_n {\mathbb{E}_n}[K_\varpi(W)f_u(Z^a\delta)^2]$;

(iv) For a fixed $q \geqslant 4 \lor (1+d_W)$, $\mathrm{diam}(\mathcal{W}) \leqslant n^{1/2q}$, ${\mathrm{E}}[ \max_{i\leqslant n}\|X_{iV}\|_\infty^q\vee \max_{a\in V}\|Z^a_i\|_\infty^q]^{1/q}/\mu_\mathcal{W}\leqslant M_n$, ${\mathrm{E}}[\max_{i\leqslant n} \sup_{u\in\mathcal{U}, j\in[p]} |v_{iuj}|^q ]^{1/q}\leqslant L_n $, $ (L_f+L_K)^2M_n^2\log^2(p|V|n)\leqslant \delta_n n\mu_\mathcal{W}^3{\underline{f}}_\mathcal{U}^6 $, $M_n^{4}\log(p|V|n) \log n\leqslant \delta_n^2 n \mu_\mathcal{W}^2{\underline{f}}_\mathcal{U}^2$, $s^2\log^2(p|V|n)\leqslant \delta_n^2 n {\underline{f}}_\mathcal{U}^4\mu_\mathcal{W}^6$, $s^3\log^3(p|V|n)\leqslant \delta_n^4 n \underline{f}_{\mathcal{U}}^2\mu_\mathcal{W}^3$, $L_n^2s\log^{3/2} (p|V|n) \leqslant \delta_n \underline{f}_{\mathcal{U}}(n\mu_\mathcal{W})^{1/2}$, $M_n s \sqrt{\log( p|V|n)} \leqslant \delta_n n^{1/2}\mu_\mathcal{W}{\underline{f}}_\mathcal{U}$.

Condition CI assumes various conditional moment conditions to allow for the estimation to be conditional on $\varpi \in \mathcal{W}$. Those are analogous to the (unconditional) conditions in the high-dimensional literature in quantile regression models, BCK2013robustQR. In particular, condition CI(i) assumes smoothness of the density function, and of coefficients. Condition CI(ii) assumes conditions on the (conditional) population design matrices such as the ratio between eigenvalues. Condition CI(iii) pertains to the approximations errors and assumes mild moment conditions. Finally Condition CI(iv) provides sufficient conditions on the allowed growth of the model via $p$ and $|V|$ relative to the available sample size $n$. Note, Condition CI(iii) also assume $d_W$ is bounded by fixed $q$, and the proof can easily be extended to other cases.

Condition CI is a high level condition intended to allow approximate sparse models, approximation errors, tail events in $\mathcal{W}$, and to require only $q$ moments (going beyond sub-Gaussian variables). When applied to the special case of sub-Gaussian, exactly sparse, and singleton $\mathcal{W}$, it becomes a relatively standard assumption. For example, without approximation error, e.g. the multivariate Gaussian case, Condition CI(iii) can be removed entirely. By allowing for a large number of variables (and transformations) the approximation errors can be controlled when the quantile functions belong to some smooth function class (e.g. Sobolev space). Although it is outside the scope of the current work, the ideas and results can be generalized to dependent data, using results from chernozhukov2014clt.

Based on Condition CI, we derive our main results regarding the proposed estimator. Moreover, we also establish new results for $\ell_1$-penalized quantile regression methods that hold uniformly over the indices $u \in {\mathcal{U}}$. The following theorems summarize these results.

Theorem (ref) builds upon ideas in BC-SparseQR however the proof strategy is designed to derive rates that are adaptive to each $u\in \mathcal{U}$. Indeed the rates of convergence are $u$-dependent and they show a slower rate for rare events $\varpi \in \mathcal{W}$.

The following result establishes a uniform Bahadur representation for the final estimators.

Theorem (ref) plays a key role. However, it is important to note that the marginal distribution of $\mathbb{U}_n(u,j)$ is pivotal. Nonetheless, there is a non-trivial correlation structure between $U(a,\varpi)$ and $U(\tilde a,\tilde \varpi)$. In order to construct confidence regions with non-conservative guarantees, we rely on a multiplier bootstrap method. We will approximate the process $\mathcal{N}=(\mathcal{N}_{uj})_{u\in\mathcal{U}, j\in[p]}$ by the Gaussian multiplier bootstrap based on estimates $\widehat\psi_{uj}:= \{\tau(1-\tau){\mathbb{E}_n}[K_{\varpi}(W)\widehat{v}_{uj}^{2}]\}^{-1/2}(\tau-1\{ X_a \leqslant Z^a\widehat\beta_u\})K_\varpi(W_i)\widehat v_{uj}$ of $\bar\psi_{uj}(U,W)=\{\tau(1-\tau){\mathbb{E}_n}[K_{\varpi}(W)v_{uj}^{2}]\}^{-1/2}(\tau - 1\{ U(a,\varpi) \leqslant \tau\})K_\varpi(W)v_{uj}$, namely $$ \widehat{\mathcal{G}} = (\widehat{\mathcal{G}}_{uj})_{u\in\mathcal{U},j\in[p]} = \left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^ng_i\widehat\psi_{uj}(X_i,W_i)\right\}_{u\in\mathcal{U},j\in[p]}$$ where $(g_i)_{i=1}^n$ are independent standard normal random variables which are independent from the data $(W_i)_{i=1}^n$. Based on Theorem 5.2 of chernozhukov2013gaussian, the following result shows that the multiplier bootstrap provides a valid approximation to the large sample probability law of $\sqrt{n}(\check \beta_{uj}- \beta_{uj})_{u \in \mathcal{U},j\in[p]}$ which is suitable for the construction of uniform confidence bands over the set of indices associated with $I_a(b)$ for all $a,b \in V$. We let $\mathcal{P}_n$ denote the collection of distributions $P$ for the data such that Condition CI is satisfied for given $n$. This is the collection of all approximately sparse models where the above sparsity conditions, moment conditions, and growth conditions are satisfied.

Corollary (ref) allows the construction of simultaneous confidence regions for the coefficients that are uniformly valid over the set of data generating processes induced by Condition CI. Based on the coefficients whose intervals do not overlap zero, we can construct a conditional independence graph process $\widehat E^{I}(\tau,\varpi), \tau \in \mathcal{T}, \varpi\in\mathcal{W}$ that contains the true conditional independence quantile graph with a specified probability.

$\mathcal{W}$-Conditional PQGM

In this section, we derive theoretical guarantees for the $\mathcal{W}$-conditional predictive quantile estimators uniformly over $u=(a,\tau,\varpi)\in \mathcal{U}$. For each $u\in \mathcal{U}$ the estimand of interest is $\beta_u \in {\mathbb{R}}^p $ that corresponds to the best linear predictor under asymmetric loss function, namely

where the event $\varpi \in \mathcal{W}$ is used for further conditioning. In the analysis below, the conditioning is implemented through the function $K_\varpi(W) = 1\{ W \in \varpi\}$.

In the analysis of this case, the main issue is to handle the inherent misspecification of the linear form $X_{-a}'\beta_u$ with respect to the true conditional quantile. The first consequence is to handle the identification condition. Given $X_{-a}$ and $\varpi\in\mathcal{W}$, we let $f_{u}:=f_{X_a\vert X_{-a}, \varpi } (X_{-a}'\beta_{u} \vert X_{-a}, \varpi )$ denote the value of the conditional density function evaluated at $X_{-a}'\beta_{u}$. In our analysis, we will consider

We remark that $\underline f_{u}$ defined in ((ref)) differs from ((ref)) which is the standard conditional density at the true quantile value. It turns out that Knight's identity can be used by exploiting the first order condition associated with the optimization problem ((ref)) which yields zero mean condition similar to the conditional quantile condition.

A second consequence of the misspecification is the lack of pivotality of the score. Such pivotal property was convenient in the previous section to define penalty parameters and to conduct inference. We will exploit bounds on the VC-dimension of the relevant classes of sets formally stated below.

{\bf Condition P.} Let $\mathcal{U} = V \times \mathcal{T}\times \mathcal{W}$ and $(X_i,W_i)_{i=1}^n$ denote a sequence of independent and identically distributed random vectors generated accordingly to models ((ref)):

(i) Suppose that $\sup_{u\in\mathcal{U}}\|\beta_u\|\leqslant C$ and $\mathcal{T}$ is a fixed compact set: (a) there exists $s=s_n$ and $\bar \beta_u$ such that $\sup_{u\in\mathcal{U}}\|\bar \beta_u\|_0 \leqslant s$, $\sup_{u\in\mathcal{U}}\|\bar \beta_u-\beta_u\|+s^{-1/2}\|\bar \beta_u-\beta_u\|_1 \leqslant \sqrt{s/n}$; (b) the conditional distribution function of $X_a$ given $X_{-a}$ and $\varpi$ is absolutely continuous with continuously differentiable density $f_{X_a\mid X_{ -a},\varpi}(t\mid X_{-a},\varpi)$ such that its values are bounded by $\bar f$ and its derivative is bounded by $\bar f'$ uniformly over $u\in\mathcal{U}$; (c) $|f_u-f_{u'}|\leqslant L_f\|u-u'\|$, $\|\beta_u-\beta_{u'}\|\leqslant L_\beta \|u-u'\|^\kappa$ with $\kappa \in [1/2,1]$, and ${\mathrm{E}}[ |K_\varpi(W)-K_{\varpi'}(W)|] \leqslant L_K\|\varpi-\varpi'\|$; (d) the VC dimension $d_W$ of the set $\mathcal{W}$ is fixed, $\{1\{ X_a \leqslant X_{-a}'\beta_u\} : (\tau,\varpi)\in \mathcal{T}\times \mathcal{W}\}$ is a VC-class with VC-dimension $1+d_W$ for every $a\in V$;

(ii) The following moment conditions hold uniformly over $u\in\mathcal{U}$: $\min_{a\in V}\inf_{\|\delta\|=1}{\mathrm{E}}[\{X_{-a}'\delta\}^2 \vert \varpi]\geqslant c$, $\max_{a\in V}\sup_{\|\delta\|=1}{\mathrm{E}}[ \{X_{-a}'\delta\}^4 \vert \varpi ]\leqslant C$;

((iii) With probability $1-\Delta_n$, uniformly over $u\in \mathcal{U}$ and $a\in V$: ${\mathbb{E}_n}[K_\varpi(W)\{|X_{-a}'(\bar\beta_u-\beta_u)|+|X_{-a}'(\bar\beta_u-\beta_u)|^2\}(Z^a\delta)^2] \leqslant \delta_n {\mathbb{E}_n}[K_\varpi(W)f_u(X_{-a}'\delta)^2]$;

(iv) For a fixed $q \geqslant 4 \lor (1+d_W)$, we have that: $\mathrm{diam}(\mathcal{W}) \leqslant n^{1/2q}$, $ {\mathrm{E}}[\max_{i\leqslant n}\|X_{iV}\|_\infty^q]^{1/q}/\mu_\mathcal{W} \leqslant M_n$, $M_n^2 \log^7(n|V|) \leqslant \delta_n n {\underline{f}}_\mathcal{U}^2\mu_\mathcal{W}^2$, $M_n^{4}\log(n|V|) \log n\leqslant \delta_n n \mu_\mathcal{W}$, $(L_f+L_K)^2M_n^2\log^2(|V|n)\leqslant \delta_n n \mu_\mathcal{W}^3{\underline{f}}_\mathcal{U}^6$, $M_n^2s\log^{3/2} (n|V|) \leqslant \delta_n \underline{f}_{\mathcal{U}}(n\mu_\mathcal{W})^{1/2}$, $M_n s \sqrt{\log(n |V|)} \leqslant \delta_n (n \mu_\mathcal{W})^{1/2}$, and $s^3\log^5(n|V|)\leqslant \delta_n n \underline{f}_{\mathcal{U}}^2\mu_\mathcal{W}^2$.

Condition P is a high-level condition. It allows to cover conditioning events $\varpi \in \mathcal{W}$ whose probability can decrease to zero (although slower than $n^{-1/4}$).

Next we derive our main results regarding the proposed estimator for the best linear predictor. These results are also new $\ell_1$-penalized quantile regression methods as it holds under possible misspecification of the conditional quantile function and hold uniformly over the indices $u\in \mathcal{U}$. The following theorem summarizes the result.

The data-driven choice of penalty parameter helps diminish the regularization bias and also allow to obtain sparse estimators with provably rates of convergence (through thresholding). Moreover, the $u$ specific penalty parameter combined with the new analysis yields an adaptive rate of convergence to each $u\in \mathcal{U}$ unlike previous works.

Application: International Financial Contagion and Systemic Risk

There is widespread disagreement about what finacial contagion entails, e.g. Forbes2002, diebold2015trans. The existing measures of contagion are mainly based on linear correlation and can only account for certain types of risk network structure or do not provide inference procedures for the large scale networks estimated. This paper defines contagion occurs whenever the quantile partial correlation from one country to another country is nonzero, i.e. the presence of edges in PQGM. This definition takes into account network spillover effects when identifying contagion and measuring systemic risk. Here, the weight of an edge indicates the strength of contagion effects. The estimated contagion network taking into account global interconnectedness is important for Eurozone financial regulators to identify globally systemically important EU countries, or for global financial portofolio diversification.

We revisit the analysis of of international financial contagion, Claessens2001. We provide an alternative approach to the literature by visualizing tail interdependence via PQGM. As shown in Section (ref), our framework naturally extends the systemic risk measure CoVaR taking into account tail network spillover effects, hence after learning PQGM from data, we identify systemically important countries using our new systemic risk measures. We can also provide inference on networks estimated. To simplify the visualization, we provide graphical visualization for the confidence intervals of $\Delta CoVaR$s in Figure (ref) of Section (ref).

We focus on examining financial contagion through the volatility spillovers perspective, i.e. recovering volatility interconnectedness. Engle1993 reported that international stock markets are related through their volatilities instead of returns. DieboldYilmaz2009 studied the return and volatility spillovers of 19 countries and found differences in return and volatility spillovers. \footnote{Modelling the time dependence in volatility is an important issue, although not the focus of this work, there is no uniform agreement on whether there is dependence once taking into account the heteroskedasticity of market returns or shifts in standard errors, see starica2005nonstationarities. }

We use average two-day equity index returns, \footnote{This is to control for the fact that markets in different countries are not open during the same hours. Results are robust to whether using two-day returns or using daily returns as in older versions of our work. Daily returns are also adjusted for weekends and holidays. } September 2009 to September 2013, from Morgan Stanley Capital International (MSCI). The returns are all translated into dollar-equivalents as of September 6th 2013. \footnote{We calculate returns based on U.S. dollars since these were most frequently used in past work on contagion. } We use absolute returns as a proxy for volatility. \footnote{This has historically been used in the literature. While we do recognize that there are many different methods calculating volatility measures, volatility measuring itself is a large research study area and is outside the scope of the current work.} We have a total of 45 countries in our sample, there are 21 developed markets (Australia, Austria, Belgium, Canada, Denmark, France, Germany, Hong Kong, Ireland, Italy, Japan, Netherlands, New Zealand, Norway, Portugal, Singapore, Spain, Sweden, Switzerland, the United Kingdom, the United States), 21 emerging markets (Brazil, Chile, Mexico, Greece, Israel, China, Colombia, Czech Republic, Egypt, Hungary, India, Indonesia, Korea, Malaysia, Peru, Philippines, Poland, Russia, Taiwan, Thailand, Turkey), and 3 frontier markets (Argentina, Morocco, Jordan).

Contagion Networks.

Figure (ref) provides a full-sample analysis of global volatility network spillovers at different tails. The networks are estimated via Algorithm (ref).

We denote 10% quantile as Low Tail, 50% quantile as Median, 90% quantile as Up Tail. Results learnt from both PQGMs and GGM are presented. GGM or "Gaussian Graph" in Figure (ref) means the graph is estimated via graphical lasso (e.g., Friedman2008), and the final graph is chosen by Extended Bayesian Information Criterion (ebic), see Foygel2010. Our purpose is to show the usefulness of PQGM in representing nonlinear tail interdependence allowing for heteroscedasticity and to show that PQGM can measure correlation asymmetry through looking at the tails of the distribution (not specific to any model).

There are significant differences in the network structure in terms of volatility spillovers when using PQGM and GGM. PQGM permits asymmetries in correlation dynamics, suited to investigate the presence of asymmetric responses. We find significant increase interdependence at the up tail between the volatility series, that is we find downside correlations (high volatility) are much larger than upside correlations (low volatility). This confirms findings in the finance literature that financial markets become more interdependent during high volatility periods.

We also find if two countries locate in the same geographic region, with many similarities in terms of market structure and history, they tend to be more closely connected (homophily effect as stated in network terminology), while two economies locate in separate geographic regions are less likely directly connected. In addition, we find among European Union member countries, Germany appears to play a major role in the transmission of shocks to others; while in Asia, Hong Kong, Thailand, and Singapore appear to play major roles; and among all the north and south American countries, Canada and US play major roles.

Systemic Risk.

With the estimated network, we can use different network statistics to measure the systemic risk contributions. Below we focus on the modified $\Delta CoVaR$ measure mentioned in Section (ref).

Figure (ref) provides German's $\Delta CoVaR$s with $\tau = 0.9$ and their $90\%$ uniform confidence intervals obtained via Corollary (ref). It reconfirms that France, Italy and UK contribute the most to German's $\Delta CoVaR$, means conditional on those countries being under distress relative to their median states, German would be affected the most. It is also interesting to find that other countries such as Netherlands can also have effects on German's $\Delta CoVaR$ although less statistically significant in terms of the magnitude of the effect.

In addition, we present net-$\Delta CoVaR$ discussed in Section (ref) with $\tau = 0.9$, i.e. the Up Tail, in Figure (ref) which shows that: globally, total volatility spillovers from Germany and France to the others are much larger than total volatility spillovers from the others to them, and their net-$\Delta CoVaR$ are positive. Both Greece and Spain have negative net-$\Delta CoVaR$.