Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
52,983 characters · 17 sections · 60 citation commands
Expert Aggregation for Financial Forecasting
Over the last decade, data science techniques have been regularly tested in finance to improve traditional forecasting techniques. Machine learning algorithms promise, among other things, to address the challenges of high dimensional data and to consider a broader class of functions, exploiting non-linearities or interactions in the data to improve prediction. These algorithms have been successfully applied for credit risk khandani2010consumer,butaru2016risk and mortgage risk sadhwani2021deep. On the portfolio construction side, MoritzZimmermann2016 use tree-based techniques to classify stock returns and build portfolios accordingly, while heaton2017deep use deep learning hierarchical models for financial prediction and classification. Additional work focuses on time series forecasting. For instance, rapach2013international explore lead-lag relationships among country stock returns and take advantage of LASSO models to forecast stock returns in the US. freyberger2020dissecting use adaptive group LASSO to determine which firm characteristics provide incremental information for the cross section of expected stock returns. hutchinson1994nonparametric,yao2000option consider a non-parametric approach with neural networks to forecast derivatives prices. rasekhschaffe2019machine,kozak2020shrinking explore how machine learning models can improve stock return forecasts while avoiding over-fitting. Finally, GuKellyXiu2018 compare the performance of thirteen machine learning techniques including neural networks, random forests and linear models to forecast stock returns and build portfolios from the predictions.
In practice however, choosing a model and its hyper-parameters is not straightforward. Initiated by bates1969combination and based on game theory concepts (blackwell1956analog and hannan1957approximation), the idea of combining predictions can be very effective for predictive learning tasks. Averaging models may lead to a reduction in variance and induces smaller generalisation errors breiman2001random. A key point is the diversity of the models considered in the ensemble brown2005diversity,brown2005managing. Bagging breiman1996bagging and Boosting freund1996experiments,schapire1990strength are for example two popular methods for generating ensembles. Combining different models has also been used to improve time series forecasts, such as exponential smoothing with ARIMA bai2010forecasting, AdaBoost with recurrent neural networks sun2018adaboost or to forecast stock market trading patterns lin2021learning. weng2018predicting predict stock prices from a neural network ensemble, a support vector ensemble, a boosted tree and a random forest. yang2020deep integrate different reinforcement learning algorithms to learn a stock trading strategy. nti2020comprehensive,albuquer2022quemaking provide a comprehensive review of ensemble techniques used in finance. Nevertheless, an algorithm may outperform others during specific time periods only and such ensemble methods are not robust to data distribution changes.
To tackle unstable accuracy over time, littlestone1994weighted and vovk1990aggregating independently introduced one successful approach for time series forecasting: the online aggregation of experts. This method allows to combine in a single approach the forecasts of a set of models, called experts cesa2006prediction. A new forecast is obtained with the help of sequential decision techniques and is guaranteed by the theory to be on average almost as accurate as the forecast of the best expert freund1997using,vovk1997competitive,vovk1998game. The resulting mixture is continuously updated as soon as the expert forecasts become available. This is a desirable feature in non-stationary environments as it allows to reconsider regularly the best models. This approach is all the more appealing that it makes no assumption about the data generation process. The framework is also a way to meet the challenge of tuning hyper-parameters, by considering every possible parameter combinations with the same algorithm. In addition, aggregation with expert advice reduces the average excess risk of the estimator while benefiting from theoretically sound results on the optimal regret bound, i.e. aggregation guarantees to recover online the best possible combination of experts.
These attractive properties partly explain why sequential aggregation procedures have been intensively studied in recent years azoury2001relative,vovk2006line,atiya2020does,petropoulos2020forecasting. The book cesa2006prediction provides an in-depth introduction to this approach. Aggregation methods have been used successfully for time series forecasting applications, such as energy consumption or electricity prices GaillardGoude2014, nowotarski2018recent, weather taillardat2016calibrated,thorey2017online, pollution debry2014ensemble,auder2016sequential or exchange rates amat2018fundamentals.
In this paper, online expert aggregation is used to address the difficulty of having to choose between several investment strategies, and to ensure robustness to changing market conditions (i.e. guarantee satisfactory performance over time). Thirteen different portfolios are constructed based on various machine learning algorithms (linear, tree-based, neural networks) forecasting one-month-ahead stock returns from firms' financial characteristics. The dataset includes 94 characteristics (size, momentum, etc.) for a large collection of 30,000 stocks over the 1957-2016 period. Zero-net-investment portfolios are constituted based on model's forecasts, buying stocks in the the highest expected return decile and selling stocks in the lowest. Eventually, the state-of-the-art Bernstein Online Aggregation from wintenberger2017optimal provides a convex combination of the long and short strategies based on individual experts' forecasts to build a robust portfolio. The aggregation assigns every month a weight to each expert according to its current performance.
Results of this paper show that robust online aggregation leads to attractive portfolio performances even in adversarial environments characterised by strong non-stationarity of the data distribution. The aggregated portfolio not only outperforms the experts, but also makes the approach more robust by dynamically adapting to market changes online, which greatly reduces the shortfall risk. The aggregation allows to build an investment strategy with an annual Sharpe Ratio of 2.82, slightly higher than the best expert (a neural network) with 2.74, while having a maximum monthly loss of 7%, more than twice as low as that same best expert (16%). Eventually, the aggregated portfolio turnover stays close to the one of each individual strategy around 120%.
To our knowledge, this paper provides the first application of online expert aggregation for financial strategies. This work adds on the growing literature testing machine learning techniques for portfolio management, using an adaptive mixture of long-short portfolios based on data-driven individual stock price predictions. The contribution is plural. First, state-of-the-art Bernstein Online Aggregation (BOA) is applied on portfolio construction. The aggregation ponders directly the stock weights of each individual expert portfolio, allowing to consider any algorithm, even black-box models. Second, the tests provide a comparison of the performance of the aggregated strategy with thirteen machine learning experts, studied by GuKellyXiu2018, on a large dataset (more than 30,000 US stocks). Eleven of the thirteen experts used in the paper are ensemble-based by construction for stability purposes, but also to compare the performance of online aggregation with static ensemble methods. Finally, expert and aggregation specialisations are proposed to improve the global mixture. An expert outperforming the aggregation gives the opportunity to increase the initial set of experts with additional models derived from this best expert. In the same spirit, aggregation specialisation is introduced and explores the possibility to refine the aggregation depending on the context.
Instead of relying only on one model forecast, a more robust approach considers ensemble forecasts. The aggregation framework considered in this paper tackles unstable accuracy of forecasting models in non-stationary environments in an online manner, without hypothesis on models and the data distribution.
A set of forecasting algorithms, called experts, estimate independently the next value of a given sequence. A set of observations $\mathcal{D}_t=\{(x_1, y_1), . . . ,(x_t, y_t)\}$ is given at each time $t>0$ where the target $y_t$ is a bounded value on $\mathbb{R}$ and $x_t\in\mathbb{R}^d$ is a feature vector Each forecasting algorithm $k$ at $t$ is a function $f^k_t: \mathbb{R}^d \mapsto \mathbb{R}$ providing a forecast $f^k_t(x_{t+1})$ that has to be as close as possible to $y_{t+1}$. The forecasts are obtained element by element by learning the (assumed) relationship between the input space $\mathbb{R}^d$ and a bounded subset of $\mathbb{R}$. An online expert $f^k = (f^k_0, f^k_1, f^k_2, \ldots)$ is a sequential algorithm that produces at each time $t$ an expert $f^k_t$.
The relevance of the expert's forecast is measured at each time step by a convex loss function $\ell: \mathbb{R}\times\mathbb{R} \mapsto \mathbb{R}_+$. In an online setting, the goal of the experts is to minimise their cumulative empirical error $\sum_{t\geq0} \ell(y_{t+1}, f^k_t(x_{t+1}))$ between the true value $y_{t+1}$ and the expert's forecast $f^k_t(x_{t+1})$.
Experts aggregation is a sequential forecasting framework allowing to mix several forecasting models in a robust approach cesa2006prediction. The algorithm provides as forecast a convex combination of the outcomes from a finite set of experts, where the weights are computed according to a chosen deterministic policy\footnote{Note that instead of considering convex combination of expert, some policies allow model selection aggregation problem (see cesa2006prediction,wintenberger2017optimal).}. Let $f^1,...,f^K$ be $K$ online experts providing bounded estimations (so that losses are bounded as well). Aggregation aims at finding the optimal online convex combination $$f_w=\sum_{k=1}^Kw_k f^k=\left(\sum_{k=1}^Kw_{k,t}f^k_t\right)_{t\geq0}$$ with weights $w_{k,t}\in S$ where $S$ is a closed and bounded subset of $\mathbb{R}^K$. In the following $S=\{w_k\in\mathbb{R}^K_+, \sum_{k=1}^Kw_k=1\}$. The performance of the online aggregation procedure is measured by the cumulative error $\sum_{t\geq0}\ell(y_{t+1}, f_{w,t}(x_{t+1}))$ between the target $y_{t+1}$ to be predicted and the mixture's forecast $f_{w,t}(x_{t+1})$.
However, if the experts' accuracy is low, so will be the accuracy of the mix, it is thus impossible to ensure a low cumulative loss for the aggregation in absolute terms. Therefore, the aggregation framework seeks to ensure a low cumulative error compared to the cumulative errors of the experts. To do so, the mixture is compared to the best possible fixed combination of experts, called oracle\footnote{Specific aggregations or settings include non-stationary oracles, see for instance herbster1998tracking.}. The goal of the aggregation is thus to retrieve online the oracle.
The regret freund1997using compares a given online aggregation procedure with the oracle in terms of cumulative errors. The regret is defined at time $T$ by
where the first term is the cumulative error of the mixture and the second term is the approximation error, i.e. the cumulative error of the oracle compared to the target $y_{t+1}$ . By minimising regret, one seeks to avoid sub-optimal mixtures and thus reduce the number of actions taken where, in hindsight, a better choice would have been possible. These choices are directed by a given policy, called rule.
The rule of an aggregation determines how the weights are assigned to each expert. The rules are deterministic, need all expert forecasts at each time\footnote{devaine2013forecasting, gaillard2014second propose theoretical guarantees about regret convergences when some expert predictions are missing. Called sleeping experts, the missing estimates can be replaced by those from the aggregation. mourtada2017efficient explore aggregations of a set of experts that is no longer fixed but can increase over time, which is particularly useful for dealing with non-stationarity.} and depend on a learning rate parameter $\eta>0$. The learning rate, which is preferably tuned online, guides the aggregation rule adaptability to the environment. Having a high $\eta$ leads to follow the best expert, while a lower rate leads the mixture to a more uniform and conservative distribution. Many rules exist in the literature and differ according to the application. littlestone1994weighted and vovk1990aggregating propose to use an online convex aggregation rule called Exponentially Weighted Average (EWA) allowing rough changes in the weights allocation. Multiple Learning rate (ML Poly, cesa2003potential,gaillard2014second) has its own learning parameter calibration rule which is faster than the empirical tuning described by devaine2013forecasting. Fixed Share forecaster (FS, herbster1998tracking) competes not only with the best fixed expert but also with the best sequence of experts and Ridge allows non-positive weights and non-convex combinations azoury2001relative,vovk2006line. Most of the aggregation rules, in particular the one used in this paper, ensure that the regret converges to zero when $T$ goes to infinity. So, this study focuses on reducing the approximation error by increasing the heterogeneity of the expert set (see GaillardGoude2014 and Stoltz2005 for further details).
This paper considers the Bernstein Online Aggregation (BOA, wintenberger2017optimal). At time $t$, BOA assigns a new weight to each expert according to its accuracy compared to the other experts, by minimising the loss $\ell_{k,t+1} = \ell(y_{t+1} , f_{k,t}(x_{t+1})) - \ell(y_{t+1} , f_{w,t}(x_{t+1}))$. Given the losses $\ell_t=(\ell_{1,t},\dots,\ell_{K,t})$ suffered by the experts at each instance $t$, BOA assigns to expert $k$ the weight
$$ w_{k,t} = \frac{\exp(-\eta \ell_{k,t}(1+\eta \ell_{k,t}))}{\exp(-\eta \ell_{w,t}(1+\eta \ell_{w,t}))}w_{k,t-1}, $$
where $\ell_{w,t}=\sum_{k=1}^K w_{k,t-1} \ell_{k,t}$ is the loss suffered at time $t$ by the aggregation at time $t-1$ (with weights $w_{t-1}$). The second order refinement of the loss is designed to penalise large errors and stabilises the weight allocation. BOA benefits from a faster rate of convergence than other rules. The learning rate $\eta$ is optimally tuned in the BOA process and ensures minimising regret with the fast rate of convergence $\log(K)/T$ in deviation. BOA procedure is reported in Algorithm (ref). For further details, see wintenberger2017optimal.
\FloatBarrier
The online aggregation procedure is applied to portfolio returns, by minimising the cumulative loss between each expert’s portfolio and the best possible portfolio, the target.\footnote{Note that an alternative approach would have been to aggregate stock return forecasts rather than portfolios' weights. But, because the final goal is to improve directly portfolio performance from any (potentially black-box) strategy and not return forecast accuracy, the online mixture is applied on portfolio weights.} The target for the long (resp. short) portfolio is obtained by buying at each rebalancing date the 10% best (resp. worst) performing stocks. In practice, the aggregation assigns at the end of each month a vector of weights to the expert portfolios to minimise the difference between the mixture returns and this optimal portfolio returns. Note that the target should not be confused with the oracle discussed in Section (ref), which corresponds to the best possible mixture of experts (and could be very different from the target depending on the quality of the experts). Two aggregations are applied, one for the long strategies and another one for the short strategies. The best long experts are not necessarily the same as the short ones at each instant. Using two distinct aggregations allows to take advantage of different experts at different times. The returns of the long-short aggregation are the difference between the long aggregation portfolio and the returns of the short one.
Data comes from Wharton Research Data Services (WRDSdatabase), including CRSP and Compustat database, and covers more than 30,000 US stocks over 1957-2017 period. The 94 standard firm characteristics used by GuKellyXiu2018 are considered as features to feed the stock return forecasting algorithms.\footnote{In the original paper, 920 stock characteristics are used. For simplicity, the eight macroeconomic predictors as well as the interactions between firm-level characteristics and macroeconomic state variables are omitted. However, the benchmark results obtained in this paper are essentially the same and detailed in Table (ref) in Appendix.} Twenty among these features are updated monthly, thirteen updated quarterly and sixty-one updated annually. Following GuKellyXiu2018 and freyberger2020dissecting, a cross-section rank transformation is performed each month on all firm characteristics that maps these ranks into the $[-1,1]$ interval. Missing data are replaced by their cross-sectional median. In order to avoid forward-looking bias (information at month $t$ is only known at month $t+1$ for monthly characteristics, $t+4$ for quarterly and $t+6$ for annual ones), monthly variables are delayed by one month, quarterly data by four months, and annual data by six months.
\FloatBarrier
The online aggregation is compared to the strategies based on single forecasting algorithms and follow the same methodology as GuKellyXiu2018 regarding data construction, forecasting windows and models construction. This section describes the datasets and the thirteen forecasting models. Then, the performance of long-short strategies based on individual experts is presented.
Stocks are characterised by a set of features, such as firm size or stock return momentum. For each month and each stock, a forecasting model is fed with these features to predict next month return. Due to computationally intensive forecasting procedures, the models are re-calibrated only each year. The training set size starts in 1957 with 18 years and increases with time. Models are re-fitted by increasing the training sample by one year. The validation set size of 12 years is maintained constant by rolling it forward to include the most recent year. The unobserved one-year testing set is picked within a 30-year period from 1987 to 2016. So, the first training (over 30 ones) is done on the 1957-1974 period, the validation on 1975-1986 and the out-of-sample test on year 1987. The second training is then done on 1957-1975, validation on 1976-1987, test on 1988, and so on until the testing year reaches 2016.
A unique model per method is trained for all stocks, and the model stays the same over the training period as done in GuKellyXiu2018. This avoids intensive computational costs and tends to stabilise return estimates of individual stocks. Note that the forecasting models are not strictly online, but their associated portfolios are, consistent with the online aggregation framework. The thirteen models are reported in Table (ref) and their hyper-parameters in Table (ref) in Appendix.
Models are trained by minimising the squared error between the observed and the estimated stock returns, except for models followed by “+H” indicating the use of Huber loss defined in Appendix (ref). Huber loss minimises the squared loss when residuals are below an (optimised) threshold and the absolute loss above. Huber is thus robust to outliers while not ignoring their effects.
Except for Partial Least Square (PLS) and Principal Component Regressor (PCR), all the models are ensemble-based by construction. A given model is trained several times on the dataset, then averaged to produce the forecast. This allows to stabilise the estimators, reducing over-fitting for the linear models with Huber loss and the neural networks trained using gradient descent, but also to compare aggregation with static ensemble methods.
The performances of the forecasting models are reported in Table (ref) in Appendix. Following lewellen2014cross, the accuracy of three basic forecasting benchmarks is shown in Table (ref) in Appendix, based on three ordinary least squared regressions with respectively 3, 7 or 15 variables.
Stocks are sorted according to each model's predictions. A long (resp. short) portfolio is built by buying 10% of the stocks having the highest (resp. lowest) estimated returns. Both equally and value weighted portfolios are considered. Value weighted results are presented in the appendix. Value weighted portfolios are less sensitive to illiquidity of small cap stocks, but as the objective functions of both forecasting models and aggregation minimize equally weighted forecast errors, the main paper focuses on equally weighted portfolios. When comparing portfolio performance with the estimation accuracy in Table (ref), improved forecasts do not necessarily lead to better portfolios. For instance, the portfolio OLS+H outperforms the experts based on linear and tree algorithms, while having one of the lowest %R2 among the forecasting models. Average monthly returns generally increase monotonically decile by decile for each algorithm, as reported in Appendix Table (ref) for equally weighted portfolios and Table (ref) for value weighted.
Table (ref) reports the performances of each expert portfolio on the out-of-sample test period. The performance metrics definitions are precised in Appendix (ref). In general, expert performance improves with the model complexity. An exception occurs with the linear model OLS+H which provides similar performance to the neural networks\footnote{This is a notable distinction from the results of GuKellyXiu2018 where tree-based models and neural networks significantly outperformed OLS+H.}. Portfolio performances are in line with forecast accuracy results. Neural network offer the best portfolio performance, outperforming the other strategies with an annual Sharpe Ratio (SR) always greater than 2.2. The expert NN2 dominates the other algorithms with an annual return of 0.5 for the equally weighted portfolio and 0.37 for value weighted, leading to a SR of 2.74 and 2.67 respectively. Linear models ENet+H and GLM+H are also valuable strategies, with a SR of 1.77 and 1.81 respectively, very close to experts based on dimension reduction (PLS 1.85 and PCR 1.78). Tree-based models GBRT+H and RF have a SR of 1.71 and 1.96 respectively, thanks to their low volatility (0.15 and 0.14 respectively). Surprisingly, OLS+H proposes a comparable SR (2.28) as neural networks. While the model is restricted to linear functions and not particularly well fitted for high-dimensional data, when considering long-short stock strategies, OLS+H competes with more sophisticated models able to deal with non-linear relationships between variables. OLS3+H (which has a limited number of stock characteristics) has the lowest SR at 1.1.
At first sight, selecting a best strategy is not obvious. NN2 offers better returns and SR, but the low turnover of RF (92% against an average of 120%) or the low maximum monthly loss of OLS+H (13%) makes these strategies appealing in practice. NN5, although appealing in terms of SR, is characterised by higher extreme risks (highest kurtosis among all experts and a maximum drawdown substantially greater than that of NN2 (47% against 17%). The qualitative conclusions remain when considering value weighted portfolios (reported in Table (ref) in Appendix).
This section shows how the aggregation of individual strategies based on machine learning models can enhance portfolio performance and adapt to changing market conditions. Variant aggregations that improve the mixtures in specific contexts are presented. Finally, the importance of each expert in the aggregation is examined and expert specialisation is discussed.
BOA rule is applied on the expert portfolios, minimising the square loss between the best possible portfolio returns and the returns of the $K=13$ expert strategies listed in Table (ref). The resulting portfolio is called PtfBOA. By dynamically weighting strategies, one can expect to retrieve (at least) the best expert's portfolio returns and reduce the risk of betting on only one expert. A uniform mixture of the $K$ portfolios, called PtfUNI, is used as a benchmark and assigns a constant weight of $1/K$ to each expert throughout the test period.
Table (ref) shows the best expert in terms of annual Sharpe Ratio (NN2), PtfUNI, PtfBOA and the oracle, i.e. the best possible convex combination over the test period, unachievable in practice. Two additional mixtures are presented to compare BOA with simple ensemble approaches. The best convex combination calibrated over the last year of the validation set provides a fixed weighting that is less naive than the uniform mixture. To adapt to changing market conditions, the best one-year rolling convex mixture assigns the best fixed convex combination estimated the previous year to the next year. The latter does not benefit from the same theoretical guarantees than BOA and can induce rough variations in portfolio weights through time.
The aggregation PtfBOA brings a significant improvement to classical machine learning techniques and the different mixtures. BOA portfolio provides the highest SR at 2.77, followed by NN2 (2.74), while decreasing substantially the portfolio maximum monthly loss (8%) compared to NN2 and all portfolios. PtfBOA has the highest skewness (3.11) and the lowest max DD (8%), with a similar turnover (123%) as NN2 and the mixtures. These results are all the more encouraging that the expert aggregation does not directly optimise these indicators, but only considers the error between the monthly returns of the expert portfolios and the target. The uniform aggregation PtfUNI offers the third best SR at 2.56, ex-aequo with NN4, despite lower annalised returns (0.36) compared to NN2 and PtfBOA. Value weighted strategies give similar results (see Table (ref) in Appendix), PtfBOA being able to outperform the experts and mixtures in multiple metrics. As expected, the turnover of all value weighted portfolios is lower than in the equally weighted case.
The oracle's SR (2.92) indicates that there is only marginal room for improvement, by designing better objective functions or better rules for the online aggregation. The two simple ensemble-based portfolios (the fixed and one-year rolling best convex combination), with a SR of 2.28 and 2.60 respectively, do not manage to beat the best expert. Both mixtures also underperform PtfBOA in terms of returns, SR, max DD and max Loss. To put emphasis on the relationship between experts and the online aggregation, the following analysis focuses on PtfBOA and PtfUNI.
\paragraph*{Weights analysis}
Figure (ref) displays the dynamic weights of the thirteen experts within BOA as well as the cumulative log returns of both long and short strategies. The bottom graph highlights how aggregation proceeds: starting with uniform weights, a transition phase over the first few months gives way to the best experts until the mixture converges.
The mixture favors neural networks and OLS+H. NN2 is the best expert all over the test period for the long strategy in terms of cumulative returns. The linear model OLS+H is the best expert for the short strategy before 2001, and therefore share a large part of the weighting in the mixture (aroung 40%). In 2001, a regime break (coinciding with the dot.com bubble burst) reduces OLS+H importance in the mixture at the benefit of neural networks. Aggregation adapts quickly its weights during the 2001 regime shifts, which partly explains the attractive performance and robustness of the strategy. PtfBOA is the second best profitable strategy, for both the long and short portfolios. It is worth noting that the difference in cumulative performance between PtfBOA and PtfUNI is large, and encourages the adoption of an online mixture.
Over the entire test period, expert weights are relatively stable from 1992 to 2000 and then from 2001 until 2016. These two regimes of stable outperformance of the best experts encourage the mix to “follow the leader” instead of considering a more heterogeneous mixture. This is an interesting result, especially in a non-stationary environment where the most profitable strategy may vary from one instant to the next. Here, aggregation is based mainly on the best portfolio and benefits little from the opinion of the other experts.
Interestingly, during the 2008 Subprime crisis, all experts and the aggregation experience a large drop in performances. However, the crisis has only a small impact on individual experts' weights in the aggregation. Neural networks manage to get higher average returns compared to the other experts on this specific period, and thus retain their dominant position in the mix (Figure (ref) in Appendix).
Average weights of the experts on the test period are given in the Table (ref) in Appendix. In particular, NN2 and OLS+H share 67% of the weight allocated to the thirteen strategies on average over the test period. PtfBOA has the closest weight combination to the oracle.
\paragraph*{Experts ranking} Figure (ref) illustrates the distribution of each Sharpe Ratio's rank for the thirteen individual experts, the uniform mixture and the two aggregations (PtfBOA and PtfUNI). Each column indicates the number of time that the strategy has been ranked 1, 2, 3, $\dots$, 15 in terms of SR over the test period (1987-2016). PtfBOA reaches rank 1 close to 17% of the time over the test period, followed by OLS+H (13%). However, OLS+H is often one of the least profitable experts, as illustrated by its significant proportion among low ranks (from 9 to 15). The five neural networks (over thirteen experts) and PtfBOA represent more than half of the overall area on the top 3 ranks. In particular, PtfBOA appears in 14% of the top 3 SR, when OLS+H reaches 11% and NN2 9%, close to the uniform mixture PtfUNI 8%. Besides, the ranks of NN2 or OLS+H are more disparate compared to the ranks of the aggregation.
In this section, the aggregation is performed on two sub-samples of US stocks, specifically the top and bottom 1000 stocks in terms of market capitalisation. The aim is to test if the aggregation performance hold on the sub-sample of large stocks, because small and micro cap stocks may be more difficult to trade frequently due to liquidity issues.
Table (ref) reports the Sharpe Ratios of experts and aggregation-based strategies on the two different market capitalisation universes. It is striking to see that the best experts are very different on the two universes of stocks. OLS+H provides the highest SR (0.94) among the largest stocks (top 1000), while NN1 is the best expert on small stocks (bottom 1000) with a SR at 2.57, very close to NN2 at 2.56. However, PtfBOA outperforms all individual experts for the universe of large stocks, with a SR of 0.95, and an attractive SR (2.59) on the smallest ones. The naive constant weighting PtfUNI provides the highest SR for small stocks (3.07). Depending on the stock size, best experts differ and the online aggregation allows to adjust experts' weights accordingly.
A strong advantage of the aggregation lies also in its easy adaptation to alternative objectives. Two improvements of the aggregation are proposed, either by adding some prior in the global mixture (Section (ref)) or by specialising the best experts (Section (ref)).
At initialisation, expert's weights are uniformly set (at 1/K with $K$=13 the number of experts) and are then updated according to their losses and BOA rule. In order to speed up the convergence, one can pre-train the online mixture on the last year of the validation set, namely 1986. Thus, over the year 1987, the aggregation benefits from a prior.
Figure (ref) presents the portfolio returns and corresponding weights of the pre-trained BOA. The top left and top right graphs display respectively the average monthly returns and the cumulative log returns of all the strategies. Like the original PtfBOA, the pre-trained PtfBOA offers on average a slightly lower annual return than the best expert NN2 (49.0% vs 50.4%), but a slightly higher return than the original PtfBOA (49.3%). The two BOA aggregations have a significantly higher annual return than the naive uniform mixture (36.0%) and non-neural network experts, all below 48.0%. Looking at cumulative log returns, PtfBOA also appears more resilient in crisis periods such as 2001. The bottom left and right graphs display the boxplot of the weights of the individual experts within PtfBOA (left) and their dynamic evolution (right). The PtfBOA starts by giving more importance to OLS+H and NN2 and converges faster to the first stationary regime. Similar to the standard aggregation presented in the previous section, OLS+H and neural networks dominate the mixture. Table (ref) reports performance metrics of the best expert, the uniform mixture and the two PtfBOA aggregations. Small improvements come from the validation set prior. The SR of the pre-trained aggregation reaches 2.78, slightly higher than the original aggregation at 2.77. Its volatility is slightly lower (0.17 vs 0.18) and its positive skewness reaches 3.15 (3.11 for the original PtfBOA).
All in all, pre-training enables to retrieve sooner stable weights that converge to the regimes observed in the previous section. Adding some prior information is beneficial for the mixture, which (slightly) improves its portfolio performances compared to the standard aggregation.
\FloatBarrier
\paragraph*{Expert importance} To analyse the sensitivity of the results to the set of experts considered in the aggregation, the variation of several portfolio performance metrics are calculated by individually dropping each expert from the mixture. The study focuses on three performance metrics: annualised average return, volatility and annual Sharpe Ratio over the test period. The expert importance is defined as the difference between the performance of the mixture considering all experts and the performance obtained by excluding a given expert. The experts' importance indicator is then derived by normalising the differences of all the individual experts to sum to one. The expert importance for each performance indicator is given in Figure (ref).
The aggregation has a lower performance when dropping OLS+H or NN2 and this loss cannot be compensated by other experts or any convex combination of them. Dropping OLS+H from the aggregation leads to a much larger volatility and a smaller SR. OLS+H seems to be a stable expert on which the mixture should rely on average, a somewhat unexpected result given that this linear model can be sensitive to over-fitting in high dimension. On the opposite, the importance of NN2 relies on its high return, at the cost of higher volatility. These results lead to search how variations of OLS+H or NN2 predictions could affect the mixture.
\paragraph*{Expert specialisation} An individual expert can provide higher average returns on the test period than the aggregated Portfolio. This is an opportunity to improve the overall mixture by adding a slight variation of this best expert to the initial set of experts. To do so, if an expert obtains a lower loss than the aggregation, the expert is split into several new experts by re-calibrating it several times with new parameters or less inputs, as done in devaine2013forecasting for time series forecasting. This new set of experts is added to the initial set and a new aggregation is performed. Several methods to create new expert have been explored in the literature, such as Bagging, Specialisation, Temp Double Scale or Boosting GaillardGoude2014.
Based on the analysis of experts' importance, the neural network NN2 and OLS+H are split in several additional experts with the Bagging method, which gives better empirical results and is more computationally efficient. The method consists in training a bunch of identical models in parallel, where each model is trained by a random subset of the data. $K^{'}=10$ new Ordinary Least Squared with Huber loss are trained as described in Section (ref), and differ from the original OLS+H in the way they are fed during the training process (containing 80% of the original data). Models optimised with the Huber loss (noted “+H”) are trained by gradient descent which could be sensitive to local minima. Bagging allows to make the estimation more robust. Then the new strategies are added to the initial set of experts. Bagging with NN2 is done in a similar way.
Figure (ref) shows annual Sharpe Ratios of the specialised aggregation with $K+2K^{'}$ (here 33) experts. The extended PtfBOA outperforms all experts and reaches an annual Sharpe Ratio of 2.82, followed by the extended uniform mixture (2.79) and the original PtfBOA (2.77). More statistics can be found in Table (ref) in Appendix, in particular the extended PtfBOA keeps a low maximum monthly loss (7%). Expert specialisation brings Sharpe Ratio improvement while keeping the attractive properties (especially in terms of risk) of the original mixture.
\FloatBarrier
A portfolio construction methodology based on a sequential aggregation of experts is presented. The strategies, called experts, lie on several forecasting algorithms such as linear models, tree-based models and neural networks. The aggregation performs online a convex combination of experts and adapts their weights dynamically according to their performance. The originality of the approach is to apply online aggregation directly on strategies, which is an easy way to improve portfolio allocation by combining heterogeneous strategies in a single algorithm. Online aggregation is particularly promising in finance where market conditions are known to be non-stationary. The aggregation is not computationally costly and considers directly the forecasts of the experts without any assumption on the data distribution and the expert models, allowing to consider any (potentially black-box) strategy. Moreover, the aggregation rules can easily be interpreted and are theoretically grounded. By building long-short strategy based on US stocks, numerical tests show that BOA aggregation offers higher performance than individual experts and simple mixtures. Betting on a single expert could be more attractive when one focuses on cumulative returns, but the aggregation oappears to be more robust over time and reduces the risk by deceasing significantly the maximum monthly loss and the maximum draw down. Further works could examine the design of specific loss functions for portfolio construction, leveraging the easy adaptation of the aggregation framework to alternative objectives.
This work is supported by FiME, Laboratoire de Finance des March\'es de l'\'Energie, and EDF Lab.
\medbreak \noindentConflict of interest The authors declare that they have no conflict of interest