The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
82,363 characters
Probabilistic forecasting of German electricity imbalance prices
\def\spacingset#1{\renewcommand{\baselinestretch}
{#1}\small\normalsize} \spacingset{1}
\if00
{
\title{\bf Probabilistic forecasting of German electricity imbalance prices}
\author{Michał Narajewski\\
University of Duisburg-Essen\\
}
\maketitle
} \fi
\if10
{
\bigskip
\begin{center}
{\LARGE\bf Probabilistic forecasting of German electricity imbalance prices}
\end{center}
} \fi
\begin{abstract}
The exponential growth of renewable energy capacity has brought much uncertainty to electricity prices and to electricity generation. To address this challenge, the energy exchanges have been developing further trading possibilities, especially the intraday and balancing markets. For an energy trader participating in both markets, the forecasting of imbalance prices is of particular interest. Therefore, in this manuscript we conduct a very short-term probabilistic forecasting of imbalance prices, contributing to the scarce literature in this novel subject. The forecasting is performed 30 minutes before the delivery, so that the trader might still choose the trading place. The distribution of the imbalance prices is modelled and forecasted using methods well-known in the electricity price forecasting literature: lasso with bootstrap, gamlss, and probabilistic neural networks. The methods are compared with a naive benchmark in a meaningful rolling window study. The results provide evidence of the efficiency between the intraday and balancing markets as the sophisticated methods do not substantially overperform the intraday continuous price index. On the other hand, they significantly improve the empirical coverage. The analysis was conducted on the German market, however it could be easily applied to any other market of similar structure.
\end{abstract}
\noindent
{\it Keywords: imbalance price, balancing market, probabilistic forecasting, neural networks, lasso, gamlss}
\vfill
\newpage
\spacingset{1.45}
\section{Introduction and motivation}
Since the liberalization of electricity markets the market design has undergone a constant development. Currently, it consists of 3 parts: futures, spot and balancing market. The futures market allows the market participants to trade the electricity in a longer horizon. The spot market consists of the day-ahead and intraday parts, and it is the main electricity market. Here, the market players can trade one day to a few minutes prior the physical delivery. The balancing market, however, is of no less importance as it preserves the system stability. The futures and spot markets are often being run by big energy exchanges, as e.g. the European Energy Exchange (EEX) or Nord Pool, whereas the balancing market is still run locally by the Transmission System Operators (TSOs). Thus, the design of the former ones is rather unified, while the design of the latter one could deviate depending on the control zone. We can particularly distinguish the single and two price imbalance settlement methods. In the study, we consider the German market data, and therefore we focus ourselves on the single price design.
Large deviations from nominal electric grid frequency may lead to disconnections or even blackouts. Thus, the need for electricity balancing is undebatable, and it only gains on importance with the growth of renewable energy capacity, even though the introduction of intraday continuous trading and quarter-hourly products has reduced the need for short-term balancing reserves \cite{ocker2017german, koch2019short}. The German balancing market comprises the capacity and energy markets \cite{Viehmann2017}. The capacity market takes place on the day before the physical delivery period and the traders declare there their balancing capacity for a given price. Then, the balancing service providers (BSPs) that offer the cheapest capacity are accepted and may participate in the balancing energy market. A detailed description of the market is presented in Section~\ref{sec:market}.
This paper raises the novel issue of very short-term probabilistic forecasting of German electricity imbalance prices. We apply the methods well-known in electricity price forecasting (EPF) in order to model and predict the distribution of imbalance prices 30 minutes before the delivery. The motivation for such setting is the possibility to trade the energy in the intraday continuous market in the respective control zones, after gate closure, until 5 minutes before the delivery or in the balancing market. Having precise imbalance price probabilistic forecasts and access to intraday continuous limit order book, the market participant may choose between these two to maximize their profit.
The utilized modelling methods are: lasso with bootstrapped in-sample errors, gamlss with lasso-based variable selection and probabilistic neural networks.
For gamlss and neural networks we assume two distributions: normal and Student's t.
The models are compared against a naive benchmark -- EPEX ID$_1$ Price in a rolling window study, what is inline with the existing EPF literature. The models are presented in detail in Section~\ref{sec:models} and the application study in Section~\ref{sec:study}.
The electricity balancing markets have already drawn the researchers' attention. The balancing market design was studied by \citet{van2012agent, van2016electricity, poplavskaya2020effect}. The authors additionally analyse the impact of the imbalance pricing mechanism on market behaviour, and they conclude that although the system imbalance is similar for different mechanisms, the mechanism that minimizes the imbalance costs for the market is the single price settlement. The literature on modelling and forecasting in electricity balancing markets can be split to imbalance forecasting \cite{toubeau2021interpretable, bunn2018trading, bottieau2019very, bunn2021statistical}, imbalance price forecasting \cite{klaeboe2015benchmarking, lucas2020price, dumas2019probabilistic} and the application in trading \cite{browell2018risk, bunn2021statistical, bunn2018trading, bottieau2019very, kumbartzky2017optimal}. The scarce electricity imbalance price forecasting literature focuses on point forecasting \cite{klaeboe2015benchmarking, lucas2020price}, interval forecasting \cite{klaeboe2015benchmarking} and probabilistic forecasting \cite{dumas2019probabilistic}. The work of \citet{dumas2019probabilistic} is naturally the closest one to our study. The authors utilize a two-step approach, namely they first calculate the probabilities for the net imbalance and then based on that make predictions regarding the imbalance prices. On the other hand, we forecast the imbalance prices directly and do not make any prior assumptions.
The research on EPF is much wider than the one particularly focused on balancing markets. \citet{weron2014electricity} provides a review of point forecasting methods and \citet{nowotarski2018recent} present an overview of probabilistic forecasting methods in electricity markets.
The big majority of the EPF literature considers the day-ahead market
\cite{
ziel2016forecasting, ziel2018day, marcjasz2018selection, lago2018forecasting, serafin2019averaging,
lago2021forecasting}, however the intraday market gains on importance both in practice and in literature \cite{uniejewski2019understanding, narajewski2020econometric, oksuz2019neural,janke2019forecasting, narajewski2020ensemble, marcjasz2020beating}.
Similarly, much more research has been done on point forecasting than on probabilistic forecasting \cite{nowotarski2018recent}. The most popular and effective methods in recent point EPF literature are lasso \cite{ziel2016forecasting, ziel2018day, lago2021forecasting, uniejewski2019understanding, narajewski2020econometric, marcjasz2020beating} and deep neural networks
\cite{lago2021forecasting, oksuz2019neural}, whereas for probabilistic EPF we can name quantile regression \cite{maciejowska2016probabilistic, maciejowska2020assessing, uniejewski2021regularized} and gamlss \cite{gianfreda2018stochastic, narajewski2020ensemble}.
A relatively big attention is also being paid to the forecasting combination of electricity prices
\cite{nowotarski2015computing, marcjasz2018selection, serafin2019averaging}.
Now, let us summarize the major contributions of the manuscript:
\begin{enumerate}
\itemsep0em
\item It is the first work on direct probabilistic forecasting of electricity imbalance prices.
\item The imbalance market is inevitable for any market player, and thus this paper may contribute also to electricity trading literature.
\item Various probabilistic models are compared in an exhaustive forecasting study.
\item We contribute to the scarce electricity balancing literature by drawing researchers' attention to the German electricity balancing market.
\item The paper provides evidence of the efficiency between the intraday and balancing markets.
\end{enumerate}
Let us additionally note that the importance of this research is emphasized by the need of including the imbalance market in the electricity trading strategies \cite{narajewski2021optimal}.
The remainder of this manuscript has the following structure. Section~\ref{sec:market} describes the electricity balancing market in Germany, the calculation of the imbalance price and the data utilized in the study. The models and estimation methods are discussed in Section~\ref{sec:models}. Section~\ref{sec:study} presents the application study, including the description of the setting, and the empirical results. Finally, Section~\ref{sec:conclusion} closes the paper with conclusion.
\section{Electricity balancing market}\label{sec:market}
This section familiarizes the reader with the German balancing market and provides a description of calculation of the imbalance price. Additionally, we present the data used in the purpose of this study.
\subsection{Balancing market in Germany}\label{sec:balancing}
The balancing market is a crucial part of every electricity market. In Germany, it was adjusted many times in recent years, unlike the spot market, which is already well-developed and the appearing changes are rather minor. The current timeline of electricity spot and balancing market in Germany can be seen in Figure~\ref{fig:market}.
\begin{figure*}[!t]
\begin{tikzpicture}[scale=1]
\draw [->] [ultra thick] (0,0) -- (14.9,0);
\draw [line width = 1] (0.5,0) -- (0.5, -1);
\node [align = center, above, font = \scriptsize] at (0.5, 0.3) {$d-1$,};
\node [align = center, above, font = \scriptsize] at (0.5, 0) {08:00};
\node [align = center, below, font = \scriptsize] at (0.5, -1) {FCR};
\draw [line width = 1] (1.5,0) -- (1.5, -1);
\node [align = center, above, font = \scriptsize] at (1.5, 0.3) {$d-1$,};
\node [align = center, above, font = \scriptsize] at (1.5, 0) {09:00};
\node [align = center, below, font = \scriptsize] at (1.5, -1) {aFRR};
\draw [line width = 1] (2.5,0) -- (2.5, -1);
\node [align = center, above, font = \scriptsize] at (2.5, 0.3) {$d-1$,};
\node [align = center, above, font = \scriptsize] at (2.5, 0) {10:00};
\node [align = center, below, font = \scriptsize] at (2.5, -1) {mFRR};
\node [align = center, below, font = \scriptsize] at (1.5, -1.3) {capacity auctions};
\draw [line width = 1] (3.3,1) -- (3.3, 0);
\node [align = center, below, font = \scriptsize] at (3.3, 0) {$d-1$,};
\node [align = center, below, font = \scriptsize] at (3.3, -0.3) {12:00};
\node [align = center, above, font = \scriptsize] at (3.3,1.3) {Day-Ahead};
\node [align = center, above, font = \scriptsize] at (3.3,1) {Auction};
\draw [line width = 1] (4.7,1) -- (4.7, 0);
\node [align = center, below, font = \scriptsize] at (4.7, 0) {$d-1$,};
\node [align = center, below, font = \scriptsize] at (4.7, -0.3) {15:00};
\node [align = center, above, font = \scriptsize] at (4.7,1.3) {Intraday};
\node [align = center, above, font = \scriptsize] at (4.7,1) {Auction};
\node [align = center, above right, font = \scriptsize] at (4.7,0) {Hourly Intraday Continuous};
\draw [line width = 1] (5.75,1) -- (5.75, 0);
\node [align = center, below, font = \scriptsize] at (5.75, 0) {$d-1$,};
\node [align = center, below, font = \scriptsize] at (5.75, -0.3) {16:00};
\node [align = center, above right, font = \scriptsize] at (5.75,0.3) {Quarter-Hourly};
\draw [line width = 1] (9.6,0) -- (9.6, -1);
\node [align = center, below, font = \scriptsize] at (9.6,-1) {aFRR/mFRR};
\node [align = center, below, font = \scriptsize] at (9.6,-1.3) {energy market};
\node [align = center, above, font = \scriptsize] at (9.6, 0.3) {$d$,};
\node [align = center, above, font = \scriptsize] at (9.6, 0) {$h - 60$ min};
\draw [line width = 1] (10.4,1) -- (10.4, 0);
\node [align = center, above, font = \scriptsize] at (10.4,1.3) {IC market};
\node [align = center, above, font = \scriptsize] at (10.4,1) {closes};
\node [align = center, below, font = \scriptsize] at (10.4, 0) {$d$,};
\node [align = center, below, font = \scriptsize] at (10.4, -0.3) {$h - 30$ min};
\draw [line width = 1] (11.2,0) -- (11.2, -1);
\node [align = center, below, font = \scriptsize] at (11.2,-1) {mFRR};
\node [align = center, above, font = \scriptsize] at (11.2, 0.3) {$d$,};
\node [align = center, above, font = \scriptsize] at (11.2, 0) {$h - 15$ min};
\draw [line width = 1] (12.2,1) -- (12.2, -1);
\node [align = center, above, font = \scriptsize] at (12.2,1.3) {IC control};
\node [align = center, above, font = \scriptsize] at (12.2,1) {zones close};
\node [align = center, below, font = \scriptsize] at (12.3, 0) {$d$, $h - 5$ min};
\node [align = center, below, font = \scriptsize] at (12.2,-1) {aFRR};
\node [align = center, below, font = \scriptsize] at (12.2,-1.3) {call for activation};
\draw [line width = 1] (13.2,0) -- (13.2, -1);
\node [align = center, below, font = \scriptsize] at (13.2,-1) {FCR};
\node [align = center, above, font = \scriptsize] at (13.2, 0.3) {$d$,};
\node [align = center, above, font = \scriptsize] at (13.2, 0) {$h - 30$ sec};
\draw [line width = 1] (14.3,1) -- (14.3, -1);
\node [align = center, above, font = \scriptsize] at (14.3,1) {Delivery};
\node [align = center, below, font = \scriptsize] at (14.3, 0) {$d$, $h$};
\node [align = center, below, font = \scriptsize] at (14.3, -1) {Balancing};
\node [align = center, below, font = \scriptsize] at (14.3, -1.3) {market};
\end{tikzpicture}
\caption{The daily routine of the German electricity spot (top) and balancing (bottom) markets. $d, h$~correspond to the day and hour of the delivery, respectively.}
\label{fig:market}
\end{figure*}
The spot market is presented in the top part and it consists of the Day-Ahead Auction (DA), Intraday Auction (IA) and Intraday Continuous (IC). The DA takes place on the day before the delivery at 12:00 and it is the main part of the market, where the majority of power volume is traded. The IA takes place 3 hours after the DA, at 15:00 and here the market participants can trade quarter-hourly contracts, whereas in the DA one may trade only hourly contracts. This part of the market serves mainly the purpose of balancing the ramping effects of demand and power generation
\cite{kremer2020intraday, kremer2021econometric}, however \citet{narajewski2021optimal} show that a trader could make significant gains by incorporating this market in their trading strategy. The IC is the last part of the spot market, and it starts on the day before the delivery at 15:00 for hourly products and at 16:00 for quarter-hourly products\footnote{Strictly speaking, one can trade also half-hourly products starting at 15:30, however they are not very popular in the market.}. Here, the market players can trade power continuously until 30 minutes before the delivery in whole Germany and until 5 minutes before the delivery in respective TSO control zones\footnote{The German market is divided to 4 control zones.}. Also, starting at 22:00 the previous day until 1 hour before the delivery the market participants can trade cross-border using the XBID system \cite{kath2019modeling}. The purpose of the IC market is to enable the traders to react to changing generation or consumption forecasts and adjust their positions. Even though the trading window is very long, the most of the power volume traded in the IC is traded in the last couple of hours before the delivery
\cite{narajewski2019estimation, kramer2021exogenous}. Therefore, the most important IC price indicators are the volume-weighted average prices ID$_1$ and ID$_3$ \cite{uniejewski2019understanding, narajewski2020econometric, narajewski2020ensemble} which measure the price level in the last 1 and 3 hours before the delivery, respectively.
For the spot market participants of particular interest is the balancing market and especially the imbalance price. As many of the producers and consumers face high uncertainty due to the stochastic nature of weather conditions and people's behaviour, it is basically impossible for them to balance their generation or consumption perfectly. Thus, any deviations from the scheduled generation or consumption are then handled by the TSOs during the delivery. The costs of balancing the energy are then divided between the market players, often called balance responsible parties (BRPs), who contributed to the imbalance. On the other hand, the BRPs that deviated from their schedule, but their deviation reduced the overall system imbalance, are rewarded for this imbalance reduction. Let us note that even though we name the final energy balancing a market which is inevitable for any market participant, it is not really a market in which the BRPs can make bids. Instead, they need to accept the imbalance price that is a derivative of total balancing costs and total system imbalance.
The bottom part of Figure~\ref{fig:market} presents the balancing market routine. To avoid big deviations from nominal frequency in the electricity grid, the TSOs have three types of BSPs at their disposal: FCR, aFRR and mFRR. The Frequency Containment Reserve (FCR), also referred as primary reserve, is fully activated after 30 seconds and is a first response to any occurring imbalance. If the imbalance persists, the Automatic Frequency Restoration Reserve (aFRR), also referred as secondary reserve, is activated and in case of longer and deeper imbalances, the Manual Frequency Restoration Reserve (mFRR), also referred as tertiary reserve, is activated. The full activation time of aFRR and mFRR is 5 and 15 minutes, respectively. The balancing market is divided to capacity and energy markets. In the capacity market, the BSPs offer their readiness to deliver or receive the unscheduled electricity and in the energy market, they define the costs for given amount of balancing energy. Let us note that the balancing services are offered in 4-hours positive or negative blocks and the FCR does not participate in the energy market due to negligible volumes.
The capacity auctions take place on the day before the delivery at 08:00 (FCR), 09:00 (aFRR) and 10:00 (mFRR)\footnote{In the past, they were taking place in the week before the delivery, and later also two days before the delivery \cite{Viehmann2017}.}. Based on the demand from TSOs, the cheapest offers are accepted. The winning BSPs are remunerated with pay-as-cleared (FCR) and pay-as-bid (aFRR and mFRR) mechanisms\footnote{In future it is planned to incorporate the pay-as-cleared mechanism also for aFRR and mFRR.}. Then, until one hour before the 4-hour delivery block the BSPs can make bids in the energy market\footnote{In the past, the capacity and energy markets were taking place simultaneously.}. The offers are sorted creating a merit order list and in case of imbalance they are activated with pay-as-bid remuneration mechanism. The costs of balancing energy are carried over to BRPs, whereas the costs of balancing capacity are carried over to end consumers.
\subsection{Imbalance price}
As mentioned, in the German electricity market (but also in many other European markets) the imbalance price is settled using a single price mechanism. The German TSOs have established a Grid Control Cooperation (GCC) and thus the price is unified for all German control zones. The basic formula is as follows
\begin{equation}
\text{IP}^{d,qh}_{\text{basic}} = \frac{\sum \text{Costs}^{d,qh}_{GCC} - \sum \text{Revenues}^{d,qh}_{GCC}}{\text{net balance position}^{d,qh}_{GCC}}.
\label{eq:IP1}
\end{equation}
Let us note that the price is in EUR/MWh, and it is calculated separately for each quarter-hour. The balancing costs and revenues of the GCC are derived based on the activated energy from aFRR and mFRR suppliers. Since the numerator and denominator of equation~\eqref{eq:IP1} can be both negative and positive, the same applies to the imbalance price. The BRPs that contribute to the imbalance, i.e. are short/long in case of system under/over-supply, pay the price to the TSOs. However, the BRPs that reduce the system imbalance by being short/long in case of system over/under-supply are being paid the price by the TSOs.
The price given in equation~\eqref{eq:IP1} is not the final imbalance price. Before it reaches its ultimate value, it undergoes multiple modifications. In the following, we list the modifications, however we do not go deep into details as the formulas are cumbersome and not much explanatory.
\begin{enumerate}
\itemsep0em
\item Price cap in the case of a small GCC balance.
\item Additional price cap in the case of a small GCC balance.
\item Price comparison with the intraday market and setting a minimum price distance to it in such direction that it is less profitable to contribute to the imbalance.
\item Surcharge/discount on the imbalance price in the event of GCC reaching 80\% of the positive/negative balancing capacity.
\end{enumerate}
The details of the current and past imbalance price calculation method are available on the regelleistung.net website \cite{regelleistung}. The first two modifications are meant to avoid extreme imbalance prices in the case of small net GCC balance. The third one compares the imbalance price with the Intraday Price Index and sets a minimum distance of 25\%, but at least 10 EUR/MWh between them. This modification pushes the price in such direction that the BRPs contributing to the imbalance get worse price as they would have got in the intraday market. The Intraday Price Index is a volume-weighted average price that uses for calculation all the transactions in the intraday continuous on the hourly and quarter-hourly product on the particular day. The fourth modification is an additional penalty on the BRPs that contribute to the system imbalance in the case it reaches very high values. All the measures make it very unprofitable to contribute to the imbalance, but on the other hand very lucrative for the BRPs to reduce it. We denote the adjusted imbalance price as $\text{IP}^{d,qh}$ and refer to it as the imbalance price.
Figure~\ref{fig:price_tsplot} presents the time series of three electricity prices: the DA price, the quarterly ID$_1$ price, and the imbalance price.
\begin{figure}[b!]
\centering
\includegraphics[width=\linewidth]{price_tsplot.pdf}
\caption{Time series plots of various electricity price data in EUR/MWh.}
\label{fig:price_tsplot}
\end{figure}
The plots show clearly that the imbalance price is much more volatile than the prices in the quarterly IC market or in the DA market. Moreover, in the considered time-frame the imbalance price exhibited many positive and negative extreme spikes, with a minimum of around $-6500$ EUR/MWh and a maximum of around $24500$ EUR/MWh (for better clarity of Figure~\ref{fig:price_tsplot} we do not show the extremes in the plot). Such values are impossible to reach in the DA (a min of $-500$ and a max of $3000$) and IC (a min of $-9999$ and a max of $9999$) markets. Therefore, the participation in the balancing market comes with a high risk for a BRP. This confirms Figure~\ref{fig:price_histogram} which shows histograms of imbalance prices for selected hours (the range of prices was limited for better clarity of the histograms).
\begin{figure}[b!]
\centering
\includegraphics[width=\linewidth]{prices_hist.pdf}
\caption{Histograms of imbalance prices with fitted densities for selected hours.}
\label{fig:price_histogram}
\end{figure}
The fitted densities prove that the data is heavy-tailed as the three parametric student's t distribution $t(\mu, \sigma, \tau)$ seems to fit the data much better than the normal distribution $\mathcal{N}(\mu, \sigma^2)$. The two mentioned distributions will be later utilized in the application study. All the distributions belong to the location-scale family with $\mu$, $\sigma$, $\tau$ and being the location, scale, and tail-weight (degrees of freedom) parameters, respectively.
\subsection{Data}
The data utilized in the study are collected from 4 different sources. The spot market data (DA, IA and IC transactions and prices) from the EEX transparency, the day-ahead forecast data (load and renewable generation) from the ENTSO-E transparency, the balancing market data (imbalance price, imbalance volume, aFRR and mFRR capacity and energy market data) from the regelleistung.net and the fuels and emission allowance prices from the ICE. The complete dataset contains observations between 12.07.2018 and 31.12.2021 as the aFRR and mFRR data is not available for the preceding time. We cleaned the data from missing values using the R package tsrobprep \cite{narajewski2021tsrobprep}.
Figure~\ref{fig:data_tsplot} presents time series plots of selected external regressors and is a complement to Figure~\ref{fig:price_tsplot}.
\begin{figure}[t!]
\centering
\includegraphics[width=\linewidth]{data_tsplot.pdf}
\caption{Time series plots of selected external regressors. POS and NEG stand for positive and negative, respectively.}
\label{fig:data_tsplot}
\end{figure}
In both figures we marked the initial in-sample, the hyperparameter tuning, and the out-of-sample periods. Let us note the structural break in the aFRR positive and negative average energy prices between October 2018 and July 2019. During this period the mixed-pricing method was used in the tendering of aFRR and mFRR, i.e. both capacity and energy prices were used to select the cheapest BSPs. However, this method was abolished following a decision by the Düsseldorf Higher Regional Court due to an appeal and the capacity pricing (as described in Section~\ref{sec:balancing}) was immediately re-introduced.
\section{Models and estimation}\label{sec:models}
This section describes the input features and the models that use them to forecast the imbalance price $\text{IP}^{d,qh}$. In the EPF literature, it is typical to use autoregressive effects of the modelled prices, here however we cannot do it as the German imbalance prices are published once a month. For the price calculation in the IC market, we use the ${}_x\text{ID}_y$ definition of \citet{narajewski2020econometric}. Let us recall that the ${}_x\text{ID}_y$ is a volume-weighted average price of all transactions in the IC market that take place in the $[x+y, x)$ time interval prior the delivery.
\subsection{Input features}\label{sec:input_features}
The following features are considered in the exercise of modelling the imbalance price $\text{IP}^{d,qh}$ for day $d$ and quarter-hour $qh$ with $qh = 1, \dots, 96$. Whenever mentioning the corresponding product, we mean the same delivery hour, e.g. for $qh=6$ the corresponding hourly delivery time is 01:00 and quarter-hourly is 01:15. Note that we utilize only the information available until 30 minutes before the delivery.
\begin{itemize}
\itemsep0em
\item Corresponding EPEX price indices: DA$^{d,h}$, IA$^{d,qh}$, ID$_1^{d,i}$, ID$_3^{d,i}$, and ID-Index$^{d,i}$ for $i = h,qh$ (8 regressors).\footnote{The ID-Index is a volume-weighted average price of all corresponding ID transactions.}
\item Most recent 15-minute intraday prices ${}_x\text{ID}_{15\text{min}}^{d,h}$ for $h =1, \dots, 24$ and ${}_x\text{ID}_{15\text{min}}^{d,qh}$ for $qh = 1, \dots, 96$ ($24+96=120$ regressors).
\item Corresponding intraday price differences $\Delta {}_{x}^{}\text{ID}_{5\text{min}}^{d,qh}$ with $x = 30, 35, \dots, 55$ min (6~regressors).
\item DA forecasts of load, wind onshore, wind offshore and solar generation: $\text{Load}^{d,qh}$, $\text{WiOn}^{d,qh}$, $\text{WiOff}^{d,qh}$, $\text{Solar}^{d,qh}$ for $qh = 1, \dots, 96$ ($96 \cdot 4 = 384$ regressors).
\item DA forecasts mentioned above for the previous day: $\text{Load}^{d-1,qh}$, $\text{WiOn}^{d-1,qh}$, $\text{WiOff}^{d-1,qh}$, $\text{Solar}^{d-1,qh}$ for $qh = 1, \dots, 96$ ($96 \cdot 4 = 384$ regressors).
\item Most recent available imbalance values $\text{Imb}^{d,qh-i}$ for $i = 4, \dots, 7$ (4 regressors).
\item aFRR prices: $\text{aFRR}_{i,j,k}^{d,qh}$ for $i = \text{POS}, \text{NEG}$ indicating the positive or negative balancing side, $j = \text{CAP}, \text{EN}$ indicating the capacity or energy price, and $k = \text{min}, \text{avg}, \text{max}$ indicating the minimum, average or maximum price ($6\cdot 2$ regressors).
\item mFRR prices: $\text{mFRR}_{i,j,k}^{d,qh}$ with $i,j,k$ as above ($6\cdot 2$ regressors).
\item Previous day coal, gas, oil and EUA prices: $\text{Coal}^{d-1}$, $\text{Gas}^{d-1}$, $\text{Oil}^{d-1}$, $\text{EUA}^{d-1}$ (4~regressors).
\item Weekday dummies $\text{DoW}^d_i$ for $i = 1, \dots, 7$ (7~regressors).
\item Cubic periodic B-splines $S^{d}_i$ for $i=1,\dots, 6$ constructed as in \citet{ziel2016forecasting2} (6~regressors).
\end{itemize}
In total, we consider 948 regressors for the modelling exercise. Let us shortly motivate the choice of these particular variables. Previous studies \cite{narajewski2020econometric, narajewski2020ensemble} have shown that the past prices can bring a lot of information regarding the future intraday price level and distribution. We expect similar behaviour in the imbalance price development, and thus we consider the price data, especially the most recent intraday prices and price differences. Similarly, the DA forecasts of fundamental variables might help in explaining the expected volatility. Naturally, the most recent intraday forecasts would be much more informative, but unfortunately this data is not publicly available and very expensive to obtain. The most recent observed imbalance values might indicate the expected imbalance in the considered quarter-hour. The aFRR and mFRR prices are natural regressors for the imbalance prices, as they directly contribute to their values. The fuel and EUA prices should explain the general price trend, and finally the weekday dummies and cubic B-splines account for weekly and annual seasonality, respectively.
\subsection{Naive}
Following the research on intraday markets \cite{narajewski2020econometric,narajewski2020ensemble, marcjasz2020beating,janke2019forecasting} where the authors find the most recent intraday price to be a very good and simple model, we construct the naive model in similar manner. That is to say, we assume the expected imbalance price to be equal the observed quarter-hourly ID$_1$ price
\begin{equation}
\mathbb{E}\left(\text{IP}^{d,qh}\right) = \text{ID}_1^{d,qh}.
\end{equation}
To obtain a distribution of imbalance prices, we use additionally the bootstrap method~\cite{efron1979bootstrap} which was successfully applied in previous EPF research studies \cite{narajewski2020ensemble, narajewski2021optimal, nowotarski2018recent, marcjasz2022distributional}. The in-sample bootstrapped errors are added to the forecasted expected price to derive the distribution forecast
\begin{equation}
\widehat{\text{IP}}^{D+1,qh}_{m} = \widehat{\mathbb{E}\left(\text{IP}^{D+1,qh}\right)} + \widehat{\varepsilon}_m^{D+1, qh} \, \, \text{for} \, \, m = 1, \dots, M
\label{eq:bootstrap}
\end{equation}
where $\widehat{\varepsilon}_m^{D+1, qh}$ are drawn with replacement in-sample residuals for day $D+1$, i.e. we sample from the set of $\widehat{\varepsilon}^{d, qh} = \text{IP}^{d,qh} - \widehat{\text{IP}}^{d,qh}$ for $d = 1, \dots, D$.
\subsection{Lasso with bootstrap}
The lasso regression of \citet{tibshirani1996regression} is a very simple and powerful tool for linear model estimation, and thus gained high popularity and reputation in the EPF literature \cite{ziel2016forecasting, ziel2018day, lago2021forecasting, uniejewski2019understanding, narajewski2020econometric, marcjasz2020beating}. It serves both model estimation and variable selection, and therefore for the model we use all the regressors described in Section~\ref{sec:input_features}, and we denote such vector as $\bm{X}^{d,qh}$. The formula for the model is
\begin{equation}
\text{IP}^{d,qh} = \bm{X}^{d,qh}\bm{\beta}^{qh} + \varepsilon^{d,qh}
\end{equation}
and the lasso estimator is given by
\begin{equation}
\widehat{\bm{\beta}}^{qh} = \arg\min_{\bm{\beta}}\left\{\left|\left|\text{IP}^{d, qh} - \bm{X}^{d,qh}\bm{\beta}^{qh}\right|\right|_2^2 + \lambda ||\bm{\beta}||_1\right\}
\label{eq:lasso_reg}
\end{equation}
where $\lambda$ is a tuning parameter. The lasso estimator expects scaled inputs, and in addition to that we apply on the inputs the variance stabilizing asinh transformation as suggested by \citet{uniejewski2017variance} with the inverse proposed by \citet{narajewski2020econometric}. The $\lambda$ parameter is tuned based on Bayesian information criterion (BIC) for $\lambda \in \Lambda = \{\lambda_i = 2^i | i \in \mathcal{G}\}$, where $\mathcal{G}$ is an equidistant grid from $-15$ to $1$ of length 50, similarly as in the paper of \citet{narajewski2020econometric}.
Let us note that similarly as for the naive, the lasso model estimates the expected imbalance price and to obtain a distribution forecast we need the bootstrap procedure described in equation~\eqref{eq:bootstrap}.
\subsection{Gamlss with lasso}
The gamlss framework of \citet{rigby2005generalized} is an extension of the generalized additive models by allowing to build explicit additive models not only for the location, but also scale and shape parameters of a given distribution. Its potential was already noticed in the EPF literature \cite{gianfreda2018stochastic, narajewski2020ensemble}, however it has not gained yet such popularity as the lasso estimation. For the input vector $\bm{X}^{d,qh}$ we have the following model
\begin{equation}
g_i(\theta_i^{d,qh}) = \bm{X}^{d,qh}\bm{\beta}^{qh}_i
\label{eq:gamlss}
\end{equation}
with $g_i$ being the link function, $\theta_i^{d,qh} \in \Theta^{d,qh}$ and distribution given by the cumulative distribution function $F(x;\Theta^{d,qh})$.
In the study, we consider the normal and t distributions.
The link function for the location parameter is the identity function $g_1(x) = x$, and for the scale and tail-weight the softplus function $g_2(x) = \log(\exp(x)+1)$. The link functions are shown in Figure~\ref{fig:links}.
\begin{figure}[t!]
\centering
\includegraphics[width=\linewidth]{links.pdf}
\caption{Link functions used in the estimation of distribution parameters}
\label{fig:links}
\end{figure}
The model in equation~\eqref{eq:gamlss} is actually a glmlss one as we consider only linear effects of the inputs. Moreover, the size of $\bm{X}^{d,qh}$ could make the optimizing algorithm converge very slowly, especially for the 3-parametric distribution. Therefore, we additionally use the lasso regularization~\eqref{eq:lasso_reg} as described e.g. by \citet{ziel2021m5}, however we do not directly use the gamlss \cite{stasinopoulos2008generalized} and gamlss.lasso \cite{ziel2021extra} R packages as their deterministic algorithm has issues with convergence due to the very heavy tails of our data. Instead, we utilize the TensorFlow \cite{tensorflow2015-whitepaper} and Keras \cite{chollet2015keras} framework by building a simple neural network with a single linear hidden layer and given probability distribution as output. For each of the distribution parameters we use different regularization parameter $\lambda_i \in \left(10^{-5}, 10\right)$. We also allow for no regularization of each of the distribution parameters. The model is estimated by maximizing the log-likelihood using the Adam algorithm. The learning rate is assumed to be in the interval $(10^{-5}, 10^{-1})$, and we tune all the parameters using the Optuna \cite{akiba2019optuna} package in Python with the number of iterations arbitrarily set to 500. Depending on distribution, we have 5 or 7 hyperparameters to tune. Let us note that the input vector $\bm{X}^{d,qh}$ is standardized prior the modelling.
\subsection{Probabilistic neural networks}
The probabilistic neural network model is simply a multilayer perceptron (MLP) that models distribution parameters instead of price values, as shown in Figure~\ref{fig:ann}. Let us note that if we remove the hidden layers, we get the gamlss model described in the previous section.
\begin{figure}[t!]
\tikzset{
input neuron/.style={
circle,
draw,
minimum size=.5cm
},
every neuron/.style={
circle,
draw,
minimum size=.6cm
},
neuron missing/.style={
draw=none,
scale=1.2,
text height=.25cm,
execute at begin node=\color{black}$\vdots$
},
sigmoid/.style={path picture= {
\begin{scope}[x=.7pt,y=7pt]
\draw plot[domain=-6:6] (\x,{1/(1 + exp(-\x))-0.5});
\end{scope}
}
},
linear/.style={path picture= {
\begin{scope}[x=5pt,y=5pt]
\draw plot[domain=-1:1] (\x,\x);
\end{scope}
}
},
mu/.style={path picture= {
\begin{scope}[x=5pt,y=5pt]
\draw node {${\mu}$};
\end{scope}
}
},
sigma/.style={path picture= {
\begin{scope}[x=5pt,y=5pt]
\draw node {${\sigma}$};
\end{scope}
}
},
tau/.style={path picture= {
\begin{scope}[x=5pt,y=5pt]
\draw node {${\tau}$};
\end{scope}
}
},
nu/.style={path picture= {
\begin{scope}[x=5pt,y=5pt]
\draw node {${\nu}$};
\end{scope}
}
},
gaussian/.style={path picture= {
\begin{scope}[x=2pt,y=4pt]
\draw plot[domain=-3:4] (\x,{1/sqrt(2*3.14)*(.5*exp(-\x*\x) +.5*exp(-(\x-.5)*(\x-1)/2))*5.5-.8});
\end{scope}
}
},
}
\centering
\begin{tikzpicture}[x=1cm, y=.65cm, >=stealth]
\node [input neuron/.try, neuron 1/.try] (input-1) at (-1,2.5-1) {\makebox[8.5pt]{}};
\node [input neuron/.try, neuron 2/.try] (input-2) at (-1,2.5-2) {\makebox[8.5pt]{}};
\node [input neuron/.try, neuron 3/.try] (input-3) at (-1,2.5-3) {\makebox[8.5pt]{}};
\node [input neuron/.try, neuron 4/.try] (input-4) at (-1,2.5-4) {\makebox[8.5pt]{}};
\node [input neuron/.try, neuron 5/.try] (input-5) at (-1,2.5-5) {\makebox[8.5pt]{}};
\node [input neuron/.try, neuron 6/.try] (input-6) at (-1,2.5-6) {\makebox[8.5pt]{}};
\foreach \m [count=\y] in {1,2,3,4}
\node [every neuron/.try, neuron \m/.try, fill= black!20, sigmoid ] (hidden-\m) at (2,7/3-\y*4/3) {};
\foreach \m [count=\y] in {1,2,3,4}
\node [every neuron/.try, neuron \m/.try, fill= black!20, sigmoid ] (hidden2-\m) at (4,7/3-\y*4/3) {};
\foreach \m [count=\y] in {1}
\node [every neuron/.try, neuron \m/.try, fill= blue!20, mu ] (par-1) at (6,2-1*1.5) {};
\foreach \m [count=\y] in {1}
\node [every neuron/.try, neuron \m/.try, fill= blue!20, sigma ] (par-2) at (6,2-2*1.5) {};
\foreach \m [count=\y] in {1}
\node [every neuron/.try, neuron \m/.try, fill= blue!20, tau ] (par-3) at (6,2-3*1.5) {};
\foreach \m [count=\y] in {1}
\node [every neuron/.try, neuron \m/.try, fill= blue!20, gaussian ] (output-\y) at (8,-1) {};
\foreach \l [count=\i] in {1,2,3,4}
\node [above] at (hidden-\i.north) {};
\foreach \i in {1,...,6}
\foreach \j in {1,...,4}
\draw [->] (input-\i) -- (hidden-\j);
\foreach \i in {1,...,4}
\foreach \j in {1,...,4}
\draw [->] (hidden-\i) -- (hidden2-\j);
\foreach \i in {1,...,4}
\draw [->] (hidden2-\i) -- (par-1);
\foreach \i in {1,...,4}
\draw [->] (hidden2-\i) -- (par-2);
\foreach \i in {1,...,4}
\draw [->] (hidden2-\i) -- (par-3);
\foreach \i in {1,...,3}
\foreach \j in {1}
\draw [->] (par-\i) -- (output-\j);
\foreach \l [count=\x from 1] in {Hidden, Hidden, Distribution, Output}
\node [align=center, above] at (\x*2,2) {\footnotesize\l \\[-3pt] \footnotesize{layer}};
\node [align=center, above] at (-1,2) {\footnotesize Input \\[-3pt] \footnotesize{layer}};
\end{tikzpicture}
\caption{Exemplary network structure of the probabilistic MLP.}
\label{fig:ann}
\end{figure}
The approach of probabilistic MLP in EPF was first introduced by \citet{marcjasz2022distributional} for the day-ahead prices. For mathematical details see the aforementioned manuscript. The considered model assumes 2 or 3 hidden layers and outputs normal or t distribution. For the distribution parameters, we use the same link functions as in Figure~\ref{fig:links}. We regularize the model through input feature selection, $L_1$ regularization of the hidden layers and their weights, and a dropout layer.
We tune them together with the number of hidden layers, their activations functions, number of neurons and the learning rate. In the following, we present a list of all hyperparameters considered in the tuning.
\begin{itemize}
\itemsep0em
\item Input feature selection as described in Section~\ref{sec:input_features} (20~hyperparameters).
\item Dropout layer -- whether to use the dropout layer after the input layer, and if yes at what rate. The rate parameter is drawn from $(0,1)$ interval (up to 2~hyperparameters).
\item Size of the network -- either 2 or 3 hidden layers (1~hyperparameter).
\item Activation functions in the hidden layers. The possible functions are: elu, relu, sigmoid, softmax, softplus, and tanh (1~hyperparameter per layer).
\item Number of neurons in the hidden layers. The values are drawn from $[24,1024]$ interval (1~hyperparameter per layer).
\item $L_1$ regularization -- whether to use the $L_1$ regularization on the hidden layers and their weights and if yes at what rate. The rate is drawn from $(10^{-5}, 10)$ interval (up to 4~hyperparameters per layer).
\item Learning rate for the Adam algorithm drawn from $(10^{-5}, 10^{-1})$ interval (1~hyperparameter).
\end{itemize}
In total, we have up to 42 hyperparameters to tune. The selected input features are normalized prior the model estimation. Similarly as for the gamlss model, we use the Tensorflow \cite{tensorflow2015-whitepaper} and Keras \cite{chollet2015keras} framework for model estimation, and the Optuna \cite{akiba2019optuna} for hyperparameter tuning with the number of iterations arbitrarily set to 1000. The model contains additionally some elements which are not subject of the tuning exercise. These are size of the learning and validation sets, the optimizing algorithm, the number of epochs fixed to 1500, and batch size fixed to 32. We estimate the model by maximizing the log-likelihood, and we use the early stopping callback with patience of 50 epochs.
\section{Application study}\label{sec:study}
\subsection{Setting}
Due to the high complexity of the models and the need for comprehensive and computational heavy hyperparameter tuning, we consider in the study only selected quarter-hours. That is to say, we use all quarter-hours of representative hours 0, 6, 12 and 18, i.e. $qh\in QH= \{1,2,3,4,25,26,27,28,49,50,51,52,73,74, 75, 76\}$. Such approach was already used in the literature. As described in Section~\ref{sec:models}, for each $qh$ we build separate models, including a separate hyperparameter tuning. Thus, we reduce the number of them from 96 to 16 without loss of generality.
The forecasting study utilizes a rolling window scheme with $D=730$ days in-sample and $N=539$ days out-of-sample. In case of gamlss and probabilistic MLP models, the in-sample period is split to $547$ days used for training and $183$ for validation. The hyperparameter tuning is performed once, using the initial in-sample data, as shown in Figures~\ref{fig:price_tsplot} and~\ref{fig:data_tsplot}. We aim for a very short-term forecasting utilizing the information available up to 30 minutes before the delivery. The naive and lasso models forecast the imbalance price distribution through $M=10000$ bootstrap samples, whereas the gamlss and probabilistic ANN models forecast directly the assumed distribution.
\subsection{Evaluation}
Following the conclusions of \citet{gneiting2007strictly}, our main evaluation measure is the continuous ranked probability score (CRPS) as it is a strictly proper scoring rule for marginal distribution forecasts. Additionally, we calculate the values of the RMSE, MAE and empirical coverage as supplementary measures. For statistically significant conclusions, we conduct the \citet{diebold1995comparing} test using the respective CRPS losses. In this subsection, we provide details regarding the calculation of the mentioned measures.
The CRPS is approximated using the pinball loss
\begin{equation}
\text{CRPS}^{d,qh} = \frac{1}{R} \sum_{\tau \in r} \text{PB}^{d,qh}_{\tau}
\end{equation}
for a dense equidistant grid of probabilities $r$ between 0 and 1 of size $R$, see e.g.~\cite{nowotarski2018recent}. In our study, we consider $r = \{0.01, 0.02,\dots, 0.99\}$ of size $R = 99$. $\text{PB}^{d,qh}_{\tau}$ is the pinball loss with respect to probability $\tau$. Its formula is given by
\begin{equation}
\text{PB}^{d,qh}_{\tau} = \left(\tau - \mathds{1}_{\left\{ {\text{IP}}^{d,qh} < \widehat{Q}_{\tau}^{d,qh}\right\}} \right) \left({\text{IP}}^{d,qh} - \widehat{Q}_{\tau}^{d,qh} \right)
\end{equation}
where $\widehat{Q}_{\tau}^{d,qh}$ is a forecast of $\tau$-th quantile of $\text{IP}^{d,qh}$ price. To calculate the overall CRPS value we use a simple average
\begin{equation}
\text{CRPS} = \frac{1}{16N} \sum_{qh \in QH} \sum_{d = 1}^{N} \text{CRPS}^{d,qh}.
\end{equation}
The formulas for the supplementary measures are given by
\begin{equation}
\tau\%\text{-cov} = \frac{1}{16N } \sum_{qh \in QH} \sum_{d = 1}^{N} \mathds{1}_{\left\{ \widehat{Q}^{d,qh}_{(1-\tau)/2} < \text{IP}^{d,qh} < \widehat{Q}^{d,qh}_{(1+\tau)/2} \right\} },
\end{equation}
\begin{equation}
\text{RMSE} = \sqrt{\frac{1}{16N } \sum_{qh \in QH} \sum_{d = 1}^{N}\left({\text{IP}}^{d,qh} - \widehat{\mu}^{d,qh}\right)^2}
\label{eq:rmse}
\end{equation}
and
\begin{equation}
\text{MAE} = \frac{1}{16N } \sum_{qh \in QH} \sum_{d = 1}^{N} \left|{\text{IP}}^{d,qh} - \widehat{Q}_{0.5}^{d,qh} \right|
\end{equation}
where $\tau \in \{0.5, 0.9, 0.98\}$ and $\widehat{\mu}^{d,qh}$ is a forecast of expected $\text{IP}^{d,qh}$ price.
The DM test measures the statistical significance of the difference between the accuracy of the forecasts of model $A$ and model $B$, and it is commonly used in the EPF literature \cite{narajewski2020econometric, narajewski2020ensemble, uniejewski2019understanding, ziel2018day}. Denote $L_Z^{d} = (L_Z^{d,qh})'_{qh\in QH}$ the vector of out-of-sample losses for day $d$ of model $Z$. Formally, we choose $L_Z^{d,qh} = \text{CRPS}^{d,qh}$.
The multivariate loss differential series
\begin{equation}
\Delta_{A,B}^d = ||L_A^d||_1 - ||L_B^d||_1
\end{equation}
defines the difference of losses in $||\cdot||_1$ norm. For each pair of models, we compute the p-value of two one-sided DM tests. The first one is with the null hypothesis $\mathcal{H}_0: \mathbb{E}(\Delta_{A,B}^d) \leq 0$, that is to say the outperformance of the forecasts of model $B$ by the ones of model $A$. The second test is with the reverse null hypothesis $\mathcal{H}_0: \mathbb{E}(\Delta_{A,B}^d) \geq 0$ and it complements the former one.
\subsection{Results}
\begin{table}[b!]
\centering
\begin{tabular}{rrrrrrr}
\hline
& CRPS & MAE & RMSE & 50\%-cov & 90\%-cov & 98\%-cov \\
\hline
Naive & \cellcolor[rgb]{0.531,0.91,0.5} {23.04} & \cellcolor[rgb]{0.5,0.9,0.5} {\textbf{61.22}} & \cellcolor[rgb]{0.5,0.9,0.5} {\textbf{115.2}} & \cellcolor[rgb]{1,0.5,0.55} {0.2847} & \cellcolor[rgb]{1,0.5,0.542} {0.8011} & \cellcolor[rgb]{1,0.75,0.5} {0.9525} \\
Lasso & \cellcolor[rgb]{1,1,0.5} {24.91} & \cellcolor[rgb]{0.963,1,0.5} {65.06} & \cellcolor[rgb]{1,0.948,0.5} {124.5} & \cellcolor[rgb]{1,0.5,0.55} {0.2784} & \cellcolor[rgb]{1,0.5,0.55} {0.7825} & \cellcolor[rgb]{1,0.606,0.5} {0.9417} \\
gamlss.N & \cellcolor[rgb]{1,0.5,0.55} {36.09} & \cellcolor[rgb]{1,0.5,0.55} {88.70} & \cellcolor[rgb]{1,0.5,0.55} {154.5} & \cellcolor[rgb]{1,0.546,0.5} {0.3294} & \cellcolor[rgb]{1,0.5,0.55} {0.6158} & \cellcolor[rgb]{1,0.5,0.55} {0.7067} \\
gamlss.t & \cellcolor[rgb]{0.866,1,0.5} {24.24} & \cellcolor[rgb]{1,0.994,0.5} {65.70} & \cellcolor[rgb]{1,0.952,0.5} {124.2} & \cellcolor[rgb]{1,0.799,0.5} {0.4074} & \cellcolor[rgb]{0.846,1,0.5} {\textbf{0.8851}} & \cellcolor[rgb]{0.854,1,0.5} {0.9739} \\
probNN.N & \cellcolor[rgb]{1,0.5,0.55} {35.67} & \cellcolor[rgb]{1,0.5,0.55} {92.84} & \cellcolor[rgb]{1,0.5,0.55} {368.5} & \cellcolor[rgb]{0.572,0.924,0.5} {\textbf{0.5037}} & \cellcolor[rgb]{1,0.768,0.5} {0.8460} & \cellcolor[rgb]{1,0.5,0.55} {0.9249} \\
probNN.t & \cellcolor[rgb]{1,0.951,0.5} {25.85} & \cellcolor[rgb]{1,0.922,0.5} {68.74} & \cellcolor[rgb]{1,0.869,0.5} {129.2} & \cellcolor[rgb]{1,0.747,0.5} {0.3912} & \cellcolor[rgb]{1,0.957,0.5} {0.8733} & \cellcolor[rgb]{1,0.911,0.5} {0.9645} \\
Combination & \cellcolor[rgb]{0.5,0.9,0.5} {\textbf{22.94}} & \cellcolor[rgb]{0.731,0.977,0.5} {62.84} & \cellcolor[rgb]{0.755,0.985,0.5} {117.8} & \cellcolor[rgb]{1,0.606,0.5} {0.3480} & \cellcolor[rgb]{1,0.968,0.5} {0.8749} & \cellcolor[rgb]{0.5,0.9,0.5} {\textbf{0.9787}} \\
\hline
\end{tabular}
\caption{Error measures of the considered models. Colour indicates the performance columnwise (the greener, the better). With bold, we depicted the best values
in each column.}
\label{tab:results}
\end{table}
Table~\ref{tab:results} presents the results of the forecasting study. We see that the lowest error values are obtained for the naive model which forecasts the imbalance price simply with the quarterly ID$_1$ price. However, its empirical coverage is very bad.
The second-lowest errors are produced by the gamlss model that assumes the t-distribution, and this model provides the best values in terms of 90\% and 98\% empirical coverage. The generalization from gamlss to probabilistic neural network model does not bring any improvement for the t-distribution.
Based on the performance of the two mentioned models, we decided to try a simple forecast combination by averaging the forecasts of the two models. This brings a small improvement in the CRPS and in the coverage, compared to the naive model.
Let us also mention very high errors of the models that assume the normal distribution. This is inline with the previous studies \cite{narajewski2020ensemble} on intraday price development, and it was expected based on Figures~\ref{fig:price_tsplot} and~\ref{fig:price_histogram}. Interestingly, the probNN.N model provides a very accurate 50\% coverage, but not as good 90\% or 98\% coverages. Finally, the lasso model performs slightly worse than the naive in all terms what indicates that one cannot gain any improvement only with linear terms.
\begin{figure}[b!]
\centering
\includegraphics[width=\linewidth]{PS_over_quantiles.pdf}
\caption{Pinball score (left) and its ratio to the naive (right) over quantiles $\tau \in r$. The right graph shows selected models for better clarity.}
\label{fig:PS_over_quantiles}
\end{figure}
\begin{figure}[b!]
\centering
\includegraphics[width=\linewidth]{PS_over_hours.pdf}
\caption{CRPS (left) and its ratio to the naive (right) over quarter-hours $qh \in QH$. The right graph shows selected models for better clarity.}
\label{fig:PS_over_hours}
\end{figure}
Figure~\ref{fig:PS_over_quantiles} shows the pinball score values over quantiles $\tau \in r$ and the ratio to the naive model. For better clarity, we removed from the right plot the models assuming normal distribution. We see that the models have generally more issues with forecasting the right tail of the distribution. Interestingly, the lasso model forecasts the quantiles up to around 0.3 slightly better than the naive, however it loses very much in the higher quantiles. Also, the combination of naive and gamlss.t is slightly better than the naive in both tails, however not that good in the central part of the distribution. This shows that a forecast combination, as e.g. in \citet{berrisch2021crps}, could likely improve the overall score. Figure~\ref{fig:PS_over_hours} presents the CRPS values over considered quarter-hours. The naive model seems to be the best across all quarter-hours except for two at hour 6. There, the gamlss.t is slightly better than the naive, however the difference is not large and probably not significant. Again, some additional improvement comes as a result of combining the naive with the gamlss.t model.
\begin{figure}[t!]
\centering
\begin{tikzpicture}[x=1pt,y=1pt]
\definecolor{fillColor}{RGB}{255,255,255}
\path[use as bounding box,fill=fillColor,fill opacity=0.00] (0,0) rectangle (397.48,198.74);
\begin{scope}
\path[clip] ( 0.12, 0.12) rectangle (397.36,198.62);
\definecolor{drawColor}{RGB}{255,255,255}
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (121.53, 62.82) circle ( 2.25);
\end{scope}
\begin{scope}
\path[clip] ( 0.12, 0.12) rectangle (397.36,198.62);
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (135.57,176.40) --
(107.49,176.40) --
(107.49,197.05) --
(135.57,197.05) --
cycle;
\definecolor{drawColor}{RGB}{0,0,0}
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (135.57,176.40) --
(107.49,197.05);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (135.57,197.05) --
(107.49,176.40);
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (163.65,176.40) --
(135.57,176.40) --
(135.57,197.05) --
(163.65,197.05) --
cycle;
\path[fill=fillColor] (191.72,176.40) --
(163.65,176.40) --
(163.65,197.05) --
(191.72,197.05) --
cycle;
\path[fill=fillColor] (219.80,176.40) --
(191.72,176.40) --
(191.72,197.05) --
(219.80,197.05) --
cycle;
\path[fill=fillColor] (247.88,176.40) --
(219.80,176.40) --
(219.80,197.05) --
(247.88,197.05) --
cycle;
\path[fill=fillColor] (275.96,176.40) --
(247.88,176.40) --
(247.88,197.05) --
(275.96,197.05) --
cycle;
\path[fill=fillColor] (304.03,176.40) --
(275.96,176.40) --
(275.96,197.05) --
(304.03,197.05) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (135.57,155.75) --
(107.49,155.75) --
(107.49,176.40) --
(135.57,176.40) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (163.65,155.75) --
(135.57,155.75) --
(135.57,176.40) --
(163.65,176.40) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (163.65,155.75) --
(135.57,176.40);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (163.65,176.40) --
(135.57,155.75);
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (191.72,155.75) --
(163.65,155.75) --
(163.65,176.40) --
(191.72,176.40) --
cycle;
\definecolor{fillColor}{RGB}{94,255,0}
\path[fill=fillColor] (219.80,155.75) --
(191.72,155.75) --
(191.72,176.40) --
(219.80,176.40) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (247.88,155.75) --
(219.80,155.75) --
(219.80,176.40) --
(247.88,176.40) --
cycle;
\path[fill=fillColor] (275.96,155.75) --
(247.88,155.75) --
(247.88,176.40) --
(275.96,176.40) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (304.03,155.75) --
(275.96,155.75) --
(275.96,176.40) --
(304.03,176.40) --
cycle;
\path[fill=fillColor] (135.57,135.10) --
(107.49,135.10) --
(107.49,155.75) --
(135.57,155.75) --
cycle;
\path[fill=fillColor] (163.65,135.10) --
(135.57,135.10) --
(135.57,155.75) --
(163.65,155.75) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (191.72,135.10) --
(163.65,135.10) --
(163.65,155.75) --
(191.72,155.75) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (191.72,135.10) --
(163.65,155.75);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (191.72,155.75) --
(163.65,135.10);
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (219.80,135.10) --
(191.72,135.10) --
(191.72,155.75) --
(219.80,155.75) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (247.88,135.10) --
(219.80,135.10) --
(219.80,155.75) --
(247.88,155.75) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (275.96,135.10) --
(247.88,135.10) --
(247.88,155.75) --
(275.96,155.75) --
cycle;
\path[fill=fillColor] (304.03,135.10) --
(275.96,135.10) --
(275.96,155.75) --
(304.03,155.75) --
cycle;
\path[fill=fillColor] (135.57,114.45) --
(107.49,114.45) --
(107.49,135.10) --
(135.57,135.10) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (163.65,114.45) --
(135.57,114.45) --
(135.57,135.10) --
(163.65,135.10) --
cycle;
\path[fill=fillColor] (191.72,114.45) --
(163.65,114.45) --
(163.65,135.10) --
(191.72,135.10) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (219.80,114.45) --
(191.72,114.45) --
(191.72,135.10) --
(219.80,135.10) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (219.80,114.45) --
(191.72,135.10);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (219.80,135.10) --
(191.72,114.45);
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (247.88,114.45) --
(219.80,114.45) --
(219.80,135.10) --
(247.88,135.10) --
cycle;
\path[fill=fillColor] (275.96,114.45) --
(247.88,114.45) --
(247.88,135.10) --
(275.96,135.10) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (304.03,114.45) --
(275.96,114.45) --
(275.96,135.10) --
(304.03,135.10) --
cycle;
\path[fill=fillColor] (135.57, 93.80) --
(107.49, 93.80) --
(107.49,114.45) --
(135.57,114.45) --
cycle;
\path[fill=fillColor] (163.65, 93.80) --
(135.57, 93.80) --
(135.57,114.45) --
(163.65,114.45) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (191.72, 93.80) --
(163.65, 93.80) --
(163.65,114.45) --
(191.72,114.45) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (219.80, 93.80) --
(191.72, 93.80) --
(191.72,114.45) --
(219.80,114.45) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (247.88, 93.80) --
(219.80, 93.80) --
(219.80,114.45) --
(247.88,114.45) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (247.88, 93.80) --
(219.80,114.45);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (247.88,114.45) --
(219.80, 93.80);
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (275.96, 93.80) --
(247.88, 93.80) --
(247.88,114.45) --
(275.96,114.45) --
cycle;
\path[fill=fillColor] (304.03, 93.80) --
(275.96, 93.80) --
(275.96,114.45) --
(304.03,114.45) --
cycle;
\path[fill=fillColor] (135.57, 73.14) --
(107.49, 73.14) --
(107.49, 93.80) --
(135.57, 93.80) --
cycle;
\path[fill=fillColor] (163.65, 73.14) --
(135.57, 73.14) --
(135.57, 93.80) --
(163.65, 93.80) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (191.72, 73.14) --
(163.65, 73.14) --
(163.65, 93.80) --
(191.72, 93.80) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (219.80, 73.14) --
(191.72, 73.14) --
(191.72, 93.80) --
(219.80, 93.80) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (247.88, 73.14) --
(219.80, 73.14) --
(219.80, 93.80) --
(247.88, 93.80) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (275.96, 73.14) --
(247.88, 73.14) --
(247.88, 93.80) --
(275.96, 93.80) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (275.96, 73.14) --
(247.88, 93.80);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (275.96, 93.80) --
(247.88, 73.14);
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (304.03, 73.14) --
(275.96, 73.14) --
(275.96, 93.80) --
(304.03, 93.80) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (135.57, 52.49) --
(107.49, 52.49) --
(107.49, 73.14) --
(135.57, 73.14) --
cycle;
\path[fill=fillColor] (163.65, 52.49) --
(135.57, 52.49) --
(135.57, 73.14) --
(163.65, 73.14) --
cycle;
\path[fill=fillColor] (191.72, 52.49) --
(163.65, 52.49) --
(163.65, 73.14) --
(191.72, 73.14) --
cycle;
\path[fill=fillColor] (219.80, 52.49) --
(191.72, 52.49) --
(191.72, 73.14) --
(219.80, 73.14) --
cycle;
\path[fill=fillColor] (247.88, 52.49) --
(219.80, 52.49) --
(219.80, 73.14) --
(247.88, 73.14) --
cycle;
\path[fill=fillColor] (275.96, 52.49) --
(247.88, 52.49) --
(247.88, 73.14) --
(275.96, 73.14) --
cycle;
\definecolor{fillColor}{RGB}{128,128,128}
\path[fill=fillColor] (304.03, 52.49) --
(275.96, 52.49) --
(275.96, 73.14) --
(304.03, 73.14) --
cycle;
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (304.03, 52.49) --
(275.96, 73.14);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (304.03, 73.14) --
(275.96, 52.49);
\path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round] (107.49, 52.49) --
(107.49,197.05) --
(304.03,197.05) --
(304.03, 52.49) --
cycle;
\definecolor{fillColor}{RGB}{0,128,0}
\path[fill=fillColor] (318.07, 53.22) --
(318.07, 51.77) --
(351.77, 51.77) --
(351.77, 53.22) --
cycle;
\definecolor{fillColor}{RGB}{0,140,0}
\path[fill=fillColor] (318.07, 54.66) --
(318.07, 53.22) --
(351.77, 53.22) --
(351.77, 54.66) --
cycle;
\definecolor{fillColor}{RGB}{0,153,0}
\path[fill=fillColor] (318.07, 56.11) --
(318.07, 54.66) --
(351.77, 54.66) --
(351.77, 56.11) --
cycle;
\definecolor{fillColor}{RGB}{0,166,0}
\path[fill=fillColor] (318.07, 57.55) --
(318.07, 56.11) --
(351.77, 56.11) --
(351.77, 57.55) --
cycle;
\definecolor{fillColor}{RGB}{0,179,0}
\path[fill=fillColor] (318.07, 59.00) --
(318.07, 57.55) --
(351.77, 57.55) --
(351.77, 59.00) --
cycle;
\definecolor{fillColor}{RGB}{0,191,0}
\path[fill=fillColor] (318.07, 60.44) --
(318.07, 59.00) --
(351.77, 59.00) --
(351.77, 60.44) --
cycle;
\definecolor{fillColor}{RGB}{0,204,0}
\path[fill=fillColor] (318.07, 61.89) --
(318.07, 60.44) --
(351.77, 60.44) --
(351.77, 61.89) --
cycle;
\definecolor{fillColor}{RGB}{0,217,0}
\path[fill=fillColor] (318.07, 63.33) --
(318.07, 61.89) --
(351.77, 61.89) --
(351.77, 63.33) --
cycle;
\definecolor{fillColor}{RGB}{0,230,0}
\path[fill=fillColor] (318.07, 64.78) --
(318.07, 63.33) --
(351.77, 63.33) --
(351.77, 64.78) --
cycle;
\definecolor{fillColor}{RGB}{0,242,0}
\path[fill=fillColor] (318.07, 66.23) --
(318.07, 64.78) --
(351.77, 64.78) --
(351.77, 66.23) --
cycle;
\definecolor{fillColor}{RGB}{0,255,0}
\path[fill=fillColor] (318.07, 67.67) --
(318.07, 66.23) --
(351.77, 66.23) --
(351.77, 67.67) --
cycle;
\definecolor{fillColor}{RGB}{6,255,0}
\path[fill=fillColor] (318.07, 69.12) --
(318.07, 67.67) --
(351.77, 67.67) --
(351.77, 69.12) --
cycle;
\definecolor{fillColor}{RGB}{13,255,0}
\path[fill=fillColor] (318.07, 70.56) --
(318.07, 69.12) --
(351.77, 69.12) --
(351.77, 70.56) --
cycle;
\definecolor{fillColor}{RGB}{19,255,0}
\path[fill=fillColor] (318.07, 72.01) --
(318.07, 70.56) --
(351.77, 70.56) --
(351.77, 72.01) --
cycle;
\definecolor{fillColor}{RGB}{26,255,0}
\path[fill=fillColor] (318.07, 73.45) --
(318.07, 72.01) --
(351.77, 72.01) --
(351.77, 73.45) --
cycle;
\definecolor{fillColor}{RGB}{32,255,0}
\path[fill=fillColor] (318.07, 74.90) --
(318.07, 73.45) --
(351.77, 73.45) --
(351.77, 74.90) --
cycle;
\definecolor{fillColor}{RGB}{38,255,0}
\path[fill=fillColor] (318.07, 76.34) --
(318.07, 74.90) --
(351.77, 74.90) --
(351.77, 76.34) --
cycle;
\definecolor{fillColor}{RGB}{45,255,0}
\path[fill=fillColor] (318.07, 77.79) --
(318.07, 76.34) --
(351.77, 76.34) --
(351.77, 77.79) --
cycle;
\definecolor{fillColor}{RGB}{51,255,0}
\path[fill=fillColor] (318.07, 79.24) --
(318.07, 77.79) --
(351.77, 77.79) --
(351.77, 79.24) --
cycle;
\definecolor{fillColor}{RGB}{57,255,0}
\path[fill=fillColor] (318.07, 80.68) --
(318.07, 79.24) --
(351.77, 79.24) --
(351.77, 80.68) --
cycle;
\definecolor{fillColor}{RGB}{64,255,0}
\path[fill=fillColor] (318.07, 82.13) --
(318.07, 80.68) --
(351.77, 80.68) --
(351.77, 82.13) --
cycle;
\definecolor{fillColor}{RGB}{70,255,0}
\path[fill=fillColor] (318.07, 83.57) --
(318.07, 82.13) --
(351.77, 82.13) --
(351.77, 83.57) --
cycle;
\definecolor{fillColor}{RGB}{76,255,0}
\path[fill=fillColor] (318.07, 85.02) --
(318.07, 83.57) --
(351.77, 83.57) --
(351.77, 85.02) --
cycle;
\definecolor{fillColor}{RGB}{83,255,0}
\path[fill=fillColor] (318.07, 86.46) --
(318.07, 85.02) --
(351.77, 85.02) --
(351.77, 86.46) --
cycle;
\definecolor{fillColor}{RGB}{89,255,0}
\path[fill=fillColor] (318.07, 87.91) --
(318.07, 86.46) --
(351.77, 86.46) --
(351.77, 87.91) --
cycle;
\definecolor{fillColor}{RGB}{96,255,0}
\path[fill=fillColor] (318.07, 89.36) --
(318.07, 87.91) --
(351.77, 87.91) --
(351.77, 89.36) --
cycle;
\definecolor{fillColor}{RGB}{102,255,0}
\path[fill=fillColor] (318.07, 90.80) --
(318.07, 89.36) --
(351.77, 89.36) --
(351.77, 90.80) --
cycle;
\definecolor{fillColor}{RGB}{108,255,0}
\path[fill=fillColor] (318.07, 92.25) --
(318.07, 90.80) --
(351.77, 90.80) --
(351.77, 92.25) --
cycle;
\definecolor{fillColor}{RGB}{115,255,0}
\path[fill=fillColor] (318.07, 93.69) --
(318.07, 92.25) --
(351.77, 92.25) --
(351.77, 93.69) --
cycle;
\definecolor{fillColor}{RGB}{121,255,0}
\path[fill=fillColor] (318.07, 95.14) --
(318.07, 93.69) --
(351.77, 93.69) --
(351.77, 95.14) --
cycle;
\definecolor{fillColor}{RGB}{127,255,0}
\path[fill=fillColor] (318.07, 96.58) --
(318.07, 95.14) --
(351.77, 95.14) --
(351.77, 96.58) --
cycle;
\definecolor{fillColor}{RGB}{134,255,0}
\path[fill=fillColor] (318.07, 98.03) --
(318.07, 96.58) --
(351.77, 96.58) --
(351.77, 98.03) --
cycle;
\definecolor{fillColor}{RGB}{140,255,0}
\path[fill=fillColor] (318.07, 99.47) --
(318.07, 98.03) --
(351.77, 98.03) --
(351.77, 99.47) --
cycle;
\definecolor{fillColor}{RGB}{147,255,0}
\path[fill=fillColor] (318.07,100.92) --
(318.07, 99.47) --
(351.77, 99.47) --
(351.77,100.92) --
cycle;
\definecolor{fillColor}{RGB}{153,255,0}
\path[fill=fillColor] (318.07,102.37) --
(318.07,100.92) --
(351.77,100.92) --
(351.77,102.37) --
cycle;
\definecolor{fillColor}{RGB}{159,255,0}
\path[fill=fillColor] (318.07,103.81) --
(318.07,102.37) --
(351.77,102.37) --
(351.77,103.81) --
cycle;
\definecolor{fillColor}{RGB}{166,255,0}
\path[fill=fillColor] (318.07,105.26) --
(318.07,103.81) --
(351.77,103.81) --
(351.77,105.26) --
cycle;
\definecolor{fillColor}{RGB}{172,255,0}
\path[fill=fillColor] (318.07,106.70) --
(318.07,105.26) --
(351.77,105.26) --
(351.77,106.70) --
cycle;
\definecolor{fillColor}{RGB}{179,255,0}
\path[fill=fillColor] (318.07,108.15) --
(318.07,106.70) --
(351.77,106.70) --
(351.77,108.15) --
cycle;
\definecolor{fillColor}{RGB}{185,255,0}
\path[fill=fillColor] (318.07,109.59) --
(318.07,108.15) --
(351.77,108.15) --
(351.77,109.59) --
cycle;
\definecolor{fillColor}{RGB}{191,255,0}
\path[fill=fillColor] (318.07,111.04) --
(318.07,109.59) --
(351.77,109.59) --
(351.77,111.04) --
cycle;
\definecolor{fillColor}{RGB}{198,255,0}
\path[fill=fillColor] (318.07,112.48) --
(318.07,111.04) --
(351.77,111.04) --
(351.77,112.48) --
cycle;
\definecolor{fillColor}{RGB}{204,255,0}
\path[fill=fillColor] (318.07,113.93) --
(318.07,112.48) --
(351.77,112.48) --
(351.77,113.93) --
cycle;
\definecolor{fillColor}{RGB}{210,255,0}
\path[fill=fillColor] (318.07,115.38) --
(318.07,113.93) --
(351.77,113.93) --
(351.77,115.38) --
cycle;
\definecolor{fillColor}{RGB}{217,255,0}
\path[fill=fillColor] (318.07,116.82) --
(318.07,115.38) --
(351.77,115.38) --
(351.77,116.82) --
cycle;
\definecolor{fillColor}{RGB}{223,255,0}
\path[fill=fillColor] (318.07,118.27) --
(318.07,116.82) --
(351.77,116.82) --
(351.77,118.27) --
cycle;
\definecolor{fillColor}{RGB}{229,255,0}
\path[fill=fillColor] (318.07,119.71) --
(318.07,118.27) --
(351.77,118.27) --
(351.77,119.71) --
cycle;
\definecolor{fillColor}{RGB}{236,255,0}
\path[fill=fillColor] (318.07,121.16) --
(318.07,119.71) --
(351.77,119.71) --
(351.77,121.16) --
cycle;
\definecolor{fillColor}{RGB}{242,255,0}
\path[fill=fillColor] (318.07,122.60) --
(318.07,121.16) --
(351.77,121.16) --
(351.77,122.60) --
cycle;
\definecolor{fillColor}{RGB}{249,255,0}
\path[fill=fillColor] (318.07,124.05) --
(318.07,122.60) --
(351.77,122.60) --
(351.77,124.05) --
cycle;
\definecolor{fillColor}{RGB}{255,255,0}
\path[fill=fillColor] (318.07,125.50) --
(318.07,124.05) --
(351.77,124.05) --
(351.77,125.50) --
cycle;
\definecolor{fillColor}{RGB}{250,25,0}
\path[fill=fillColor] (318.07,126.94) --
(318.07,125.50) --
(351.77,125.50) --
(351.77,126.94) --
cycle;
\definecolor{fillColor}{RGB}{245,25,0}
\path[fill=fillColor] (318.07,128.39) --
(318.07,126.94) --
(351.77,126.94) --
(351.77,128.39) --
cycle;
\definecolor{fillColor}{RGB}{240,24,0}
\path[fill=fillColor] (318.07,129.83) --
(318.07,128.39) --
(351.77,128.39) --
(351.77,129.83) --
cycle;
\definecolor{fillColor}{RGB}{235,24,0}
\path[fill=fillColor] (318.07,131.28) --
(318.07,129.83) --
(351.77,129.83) --
(351.77,131.28) --
cycle;
\definecolor{fillColor}{RGB}{230,23,0}
\path[fill=fillColor] (318.07,132.72) --
(318.07,131.28) --
(351.77,131.28) --
(351.77,132.72) --
cycle;
\definecolor{fillColor}{RGB}{225,22,0}
\path[fill=fillColor] (318.07,134.17) --
(318.07,132.72) --
(351.77,132.72) --
(351.77,134.17) --
cycle;
\definecolor{fillColor}{RGB}{220,22,0}
\path[fill=fillColor] (318.07,135.61) --
(318.07,134.17) --
(351.77,134.17) --
(351.77,135.61) --
cycle;
\definecolor{fillColor}{RGB}{215,21,0}
\path[fill=fillColor] (318.07,137.06) --
(318.07,135.61) --
(351.77,135.61) --
(351.77,137.06) --
cycle;
\definecolor{fillColor}{RGB}{210,21,0}
\path[fill=fillColor] (318.07,138.51) --
(318.07,137.06) --
(351.77,137.06) --
(351.77,138.51) --
cycle;
\definecolor{fillColor}{RGB}{204,20,0}
\path[fill=fillColor] (318.07,139.95) --
(318.07,138.51) --
(351.77,138.51) --
(351.77,139.95) --
cycle;
\definecolor{fillColor}{RGB}{199,20,0}
\path[fill=fillColor] (318.07,141.40) --
(318.07,139.95) --
(351.77,139.95) --
(351.77,141.40) --
cycle;
\definecolor{fillColor}{RGB}{194,19,0}
\path[fill=fillColor] (318.07,142.84) --
(318.07,141.40) --
(351.77,141.40) --
(351.77,142.84) --
cycle;
\definecolor{fillColor}{RGB}{189,19,0}
\path[fill=fillColor] (318.07,144.29) --
(318.07,142.84) --
(351.77,142.84) --
(351.77,144.29) --
cycle;
\definecolor{fillColor}{RGB}{184,18,0}
\path[fill=fillColor] (318.07,145.73) --
(318.07,144.29) --
(351.77,144.29) --
(351.77,145.73) --
cycle;
\definecolor{fillColor}{RGB}{179,18,0}
\path[fill=fillColor] (318.07,147.18) --
(318.07,145.73) --
(351.77,145.73) --
(351.77,147.18) --
cycle;
\definecolor{fillColor}{RGB}{174,17,0}
\path[fill=fillColor] (318.07,148.63) --
(318.07,147.18) --
(351.77,147.18) --
(351.77,148.63) --
cycle;
\definecolor{fillColor}{RGB}{169,17,0}
\path[fill=fillColor] (318.07,150.07) --
(318.07,148.63) --
(351.77,148.63) --
(351.77,150.07) --
cycle;
\definecolor{fillColor}{RGB}{164,16,0}
\path[fill=fillColor] (318.07,151.52) --
(318.07,150.07) --
(351.77,150.07) --
(351.77,151.52) --
cycle;
\definecolor{fillColor}{RGB}{158,16,0}
\path[fill=fillColor] (318.07,152.96) --
(318.07,151.52) --
(351.77,151.52) --
(351.77,152.96) --
cycle;
\definecolor{fillColor}{RGB}{153,15,0}
\path[fill=fillColor] (318.07,154.41) --
(318.07,152.96) --
(351.77,152.96) --
(351.77,154.41) --
cycle;
\definecolor{fillColor}{RGB}{148,15,0}
\path[fill=fillColor] (318.07,155.85) --
(318.07,154.41) --
(351.77,154.41) --
(351.77,155.85) --
cycle;
\definecolor{fillColor}{RGB}{143,14,0}
\path[fill=fillColor] (318.07,157.30) --
(318.07,155.85) --
(351.77,155.85) --
(351.77,157.30) --
cycle;
\definecolor{fillColor}{RGB}{138,14,0}
\path[fill=fillColor] (318.07,158.74) --
(318.07,157.30) --
(351.77,157.30) --
(351.77,158.74) --
cycle;
\definecolor{fillColor}{RGB}{133,13,0}
\path[fill=fillColor] (318.07,160.19) --
(318.07,158.74) --
(351.77,158.74) --
(351.77,160.19) --
cycle;
\definecolor{fillColor}{RGB}{128,13,0}
\path[fill=fillColor] (318.07,161.64) --
(318.07,160.19) --
(351.77,160.19) --
(351.77,161.64) --
cycle;
\definecolor{fillColor}{RGB}{123,12,0}
\path[fill=fillColor] (318.07,163.08) --
(318.07,161.64) --
(351.77,161.64) --
(351.77,163.08) --
cycle;
\definecolor{fillColor}{RGB}{118,12,0}
\path[fill=fillColor] (318.07,164.53) --
(318.07,163.08) --
(351.77,163.08) --
(351.77,164.53) --
cycle;
\definecolor{fillColor}{RGB}{112,11,0}
\path[fill=fillColor] (318.07,165.97) --
(318.07,164.53) --
(351.77,164.53) --
(351.77,165.97) --
cycle;
\definecolor{fillColor}{RGB}{107,11,0}
\path[fill=fillColor] (318.07,167.42) --
(318.07,165.97) --
(351.77,165.97) --
(351.77,167.42) --
cycle;
\definecolor{fillColor}{RGB}{102,10,0}
\path[fill=fillColor] (318.07,168.86) --
(318.07,167.42) --
(351.77,167.42) --
(351.77,168.86) --
cycle;
\definecolor{fillColor}{RGB}{97,10,0}
\path[fill=fillColor] (318.07,170.31) --
(318.07,168.86) --
(351.77,168.86) --
(351.77,170.31) --
cycle;
\definecolor{fillColor}{RGB}{92,9,0}
\path[fill=fillColor] (318.07,171.75) --
(318.07,170.31) --
(351.77,170.31) --
(351.77,171.75) --
cycle;
\definecolor{fillColor}{RGB}{87,9,0}
\path[fill=fillColor] (318.07,173.20) --
(318.07,171.75) --
(351.77,171.75) --
(351.77,173.20) --
cycle;
\definecolor{fillColor}{RGB}{82,8,0}
\path[fill=fillColor] (318.07,174.65) --
(318.07,173.20) --
(351.77,173.20) --
(351.77,174.65) --
cycle;
\definecolor{fillColor}{RGB}{77,8,0}
\path[fill=fillColor] (318.07,176.09) --
(318.07,174.65) --
(351.77,174.65) --
(351.77,176.09) --
cycle;
\definecolor{fillColor}{RGB}{72,7,0}
\path[fill=fillColor] (318.07,177.54) --
(318.07,176.09) --
(351.77,176.09) --
(351.77,177.54) --
cycle;
\definecolor{fillColor}{RGB}{66,7,0}
\path[fill=fillColor] (318.07,178.98) --
(318.07,177.54) --
(351.77,177.54) --
(351.77,178.98) --
cycle;
\definecolor{fillColor}{RGB}{61,6,0}
\path[fill=fillColor] (318.07,180.43) --
(318.07,178.98) --
(351.77,178.98) --
(351.77,180.43) --
cycle;
\definecolor{fillColor}{RGB}{56,6,0}
\path[fill=fillColor] (318.07,181.87) --
(318.07,180.43) --
(351.77,180.43) --
(351.77,181.87) --
cycle;
\definecolor{fillColor}{RGB}{51,5,0}
\path[fill=fillColor] (318.07,183.32) --
(318.07,181.87) --
(351.77,181.87) --
(351.77,183.32) --
cycle;
\definecolor{fillColor}{RGB}{46,5,0}
\path[fill=fillColor] (318.07,184.77) --
(318.07,183.32) --
(351.77,183.32) --
(351.77,184.77) --
cycle;
\definecolor{fillColor}{RGB}{41,4,0}
\path[fill=fillColor] (318.07,186.21) --
(318.07,184.77) --
(351.77,184.77) --
(351.77,186.21) --
cycle;
\definecolor{fillColor}{RGB}{36,4,0}
\path[fill=fillColor] (318.07,187.66) --
(318.07,186.21) --
(351.77,186.21) --
(351.77,187.66) --
cycle;
\definecolor{fillColor}{RGB}{31,3,0}
\path[fill=fillColor] (318.07,189.10) --
(318.07,187.66) --
(351.77,187.66) --
(351.77,189.10) --
cycle;
\definecolor{fillColor}{RGB}{26,3,0}
\path[fill=fillColor] (318.07,190.55) --
(318.07,189.10) --
(351.77,189.10) --
(351.77,190.55) --
cycle;
\definecolor{fillColor}{RGB}{20,2,0}
\path[fill=fillColor] (318.07,191.99) --
(318.07,190.55) --
(351.77,190.55) --
(351.77,191.99) --
cycle;
\definecolor{fillColor}{RGB}{15,2,0}
\path[fill=fillColor] (318.07,193.44) --
(318.07,191.99) --
(351.77,191.99) --
(351.77,193.44) --
cycle;
\definecolor{fillColor}{RGB}{10,1,0}
\path[fill=fillColor] (318.07,194.88) --
(318.07,193.44) --
(351.77,193.44) --
(351.77,194.88) --
cycle;
\definecolor{fillColor}{RGB}{5,1,0}
\path[fill=fillColor] (318.07,196.33) --
(318.07,194.88) --
(351.77,194.88) --
(351.77,196.33) --
cycle;
\definecolor{fillColor}{RGB}{0,0,0}
\path[fill=fillColor] (318.07,197.78) --
(318.07,196.33) --
(351.77,196.33) --
(351.77,197.78) --
cycle;
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77, 46.30) {0\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77, 60.75) {1\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77, 75.21) {2\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77, 89.66) {3\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,104.12) {4\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,118.58) {5\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,133.03) {6\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,147.49) {7\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,161.94) {8\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,176.40) {9\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 0.90] at (357.77,190.86) {10\%};
\node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale= 1.00] at (318.46, 35.74) {p-value};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49,184.43) {Naive};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49,163.78) {Lasso};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49,143.13) {gamlss.N};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49,122.48) {gamlss.t};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49,101.83) {probNN.N};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49, 81.17) {probNN.t};
\node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (101.49, 60.52) {Combination};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (128.38, 46.74) {Naive};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (156.46, 46.74) {Lasso};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (184.54, 46.74) {gamlss.N};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (212.62, 46.74) {gamlss.t};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (240.69, 46.74) {probNN.N};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (268.77, 46.74) {probNN.t};
\node[text=drawColor,rotate= 45.00,anchor=base east,inner sep=0pt, outer sep=0pt, scale= 1.00] at (296.85, 46.74) {Combination};
\end{scope}
\end{tikzpicture}
\caption{Results of the Diebold-Mariano test. The plots present p-values for the CRPS$^{d,qh}$ loss --- the closer they are to zero ($\to$ dark green), the more significant the difference is between forecasts of X-axis model (better) and forecasts of the Y-axis model (worse).}
\label{fig:dm_test}
\end{figure}
Finally, Figure~\ref{fig:dm_test} provides p-values of the DM test obtained using CRPS loss. This figure only confirms the conclusions that we made based on Table~\ref{tab:results}. Namely, the forecasts of the naive model are significantly the best among considered models and the ones of gamlss.t model the second-best. Moreover, the combination of naive and gamlss.t is not significantly different to the naive itself.
\section{Conclusion}\label{sec:conclusion}
The paper raised the novel issue of probabilistic imbalance electricity price forecasting in the German market. The participation in the balancing is mandatory for every market player, and therefore this subject is crucial for them. The analysis assumed a setting of a very short-term forecasting, 30 minutes before the physical delivery. We considered various state-of-art methods for probabilistic EPF, however none of them could provide better forecasts in terms of CRPS, MAE and RMSE than the naive ID$_1$ price. On the other hand, the gamlss and probabilistic neural networks models provide forecasts with far higher empirical coverage than the naive. This is an evidence that the results might be improved, e.g. using intraday power generation forecasts or forecasting combination methods, e.g. \cite{berrisch2021crps}.
The obtained results are an argument towards the market efficiency between the intraday and balancing markets. This extends the conclusions of intraday market being close to market efficiency \cite{narajewski2020econometric, marcjasz2020beating}. Therefore, given the difficulty in forecasting the imbalance prices and the potential size of forecasting errors, the BRPs should minimize their imbalance rather than seeking opportunities in the balancing market.
\section*{Acknowledgments}
This research article was partially supported by the German Research Foundation (DFG, Germany) and the National Science Center (NCN, Poland) through BEETHOVEN grant no. 2016/23/G/HS4/01005.
\vspace{-5mm}
\bibliographystyle{unsrtnat1}
\bibliography{bibliography}