Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
37,009 characters · 12 sections · 43 citation commands
Amortized Neural Networks for Agent-Based Model Forecasting
In this paper, we propose a new procedure for unconditional and conditional forecasting in agent-based models. The proposed algorithm is based on the application of amortized neural networks and consists of two steps. The first step simulates artificial datasets from the model. In the second step, a neural network is trained to predict the future values of the variables using the history of observations. The main advantage of the proposed algorithm is its speed. This is due to the fact that, after the training procedure, it can be used to yield predictions for almost any data without additional simulations or the re-estimation of the neural network.
JEL-classification: C11, C15, C32, C45, C53, C63.
Keywords: agent-based models, amortized simulation-based inference, Bayesian models, forecasting, neural networks.
Agent-based models (hereinafter, ABM) are gaining more and more popularity among economists both in the academic community and at economic institutions. As they are simulation based, they do not require theoretical solution and they allow the extension of various assumptions which are often incorporated into classical economic models, such as the homogeneity/representativeness of agents, their rationality, and the availability of full information (see 12, 17, and 2). However, their flexibility is also a disadvantage. Realistic ABMs often include hundreds (see 8) to millions of agents (see 30) and are represented as nonlinear state-space models that contain a number of latent variables which is proportional to the number of agents. Parameter estimation and forecasting in models of this size are computationally complex tasks and have not been fully studied in the literature.
Many articles have been devoted to ABM parameter estimation (see the review in 10), and although this work is still in progress, the use of simulation-based inference (hereinafter, SBI, see 7) has allowed the achievement of some progress in this area. ABM forecasting remains mostly uncharted territory, however. In discussing future opportunities and challenges in ABM, 2 describe the state of the field as follows:
'Conventional macro-models are constructed in terms of aggregate variables that coincide with acquired data about the economy. This is convenient because the resulting data can be directly used to initialize these models. But ABMs are dynamical systems that model the world at the level of individual agents, such as households and firms. In order to run the model it is necessary to initialize the states of all of the individual agents in a way that is also compatible with aggregate measurements. Doing this properly requires complete micro-data on individuals, which is typically not available.
Absent such data it becomes necessary to invent plausible states for each individual agent so that the aggregate states of the model match the measured aggregate data. However the individual states in the model also need to compatible with each other and with the inherent dynamics of the model. If the states are not compatible in this sense, the model will generate transient behaviors that will result in poor forecasts. Given that model forecasts are never perfect, this is a recurrent problem - as time passes the forecasts of the model inevitably deviate from the measured aggregate data, and the initialization process must be repeated again and again. Finding good methods for doing this is an open problem that must be solved if we are to use ABMs for time series forecasting.'
In current practice, forecasting in ABMs is based either on initializing all states using microdata or their distributions and then simulating the data (see 20), as described by 2, or by using surrogate procedures (which do not require the initialization of the states of all agents) based on approximating joint distributions of the last observed data and the future values of the variables (see 34).\footnote{Although there are a number of articles on estimating parameters and hidden states based on particle filters (see 25), we have found none that use particle filters for prediction in ABM that are comparable in size to those used in practice.} The first method, as 2 note, requires knowledge of a large amount of data that is not always available to researchers, while the second is computationally demanding for problems with more than two variables and has only approximate properties in terms of convergence to the posterior distributions of forecasts.
In this paper, we propose a new method for unconditional and conditional forecasting in an ABM that allows predictions to be made without the initialization of all hidden states. The method is based on the idea of meta-learning (see 13) on simulated data\footnote{ For examples of training predictive models on simulated data in the context of meta-learning, see 18 and 14, which motivated the algorithm described in this paper.} and consists of two steps. In the first step, multiple artificial datasets are simulated, as in SBI. In the second step, an algorithm which predicts future values for a particular dataset from the history of observations is trained using a set of similar problems, as in meta-learning. In contrast to the previously proposed ones, our algorithm has the property of amortization, that is, once it is trained, it can be used for almost any data. Moreover, unlike forecasting on the basis of surrogate algorithms, such as by 34, the algorithm described below has good theoretical properties and should converge exactly to the posterior distribution of the forecast when the neural network is flexible enough. Despite the fact that the main interest for us is the application of the algorithm in the context of ABM, the procedure used is general enough and can be applied to a wide range of problems, so in Section 2, we describe the conditional and unconditional forecasting procedures in a general form, almost without relying on the specifics of ABM. Section 3 discusses the metrics that will be further used to evaluate the quality of the proposed algorithm. Section 4 is devoted to a description of the experimental results. In Section 5, we discuss several important issues concerning quality estimation and potential extensions of the algorithm. Section 6 presents the conclusion.
We assume that the model is specified as a Bayesian state-space model with a prior distribution of model parameters $p(\theta)$, state equations $s_t \sim p(s|s_{t-1}, \theta)$, and observation equations $y_t \sim p(y|s_t, \theta)$,\footnote{In an ABM, this equation can have a stochastic form, as in the case of measurement errors, which are introduced into the model to mitigate small differences between the modeled and observed variables, or a deterministic form, as in the case where the model variables are strongly related to the observed ones.} where $\theta$ is the vector of model parameters, $s_t$ is the vector of the models hidden states at time $t$, and $y_t$ is the vector of observed variables at time $t$. It is additionally assumed that it is possible to simulate datasets consisting of the observed variables.
Later in this section, we describe the unconditional forecasting procedure as well as its modification for the case of conditional forecasting.
The main idea is that the prediction from the Bayesian state-space model at time $t$ can be represented as an estimation of states $y_{t+1},...,y_{t+h}$, which are not observable at that moment. To solve this problem, we modify the algorithm for estimating several characteristics of the marginal state distributions described by 21. The algorithm can be represented as a two-step procedure. In the first stage, a set of artificial datasets of different lengths $t$, each of which consists of the history of observed variables $x=\{y_1,...,y_t \}$ and the future values of these variables $y=\{y_{t+1},...,y_{t+h}\}$, is generated. In the second stage, a neural network is trained to predict $y$ from $x$. The formal description of the algorithm is presented below:
Algorithm 1. Amortized neural network algorithm for unconditional forecasting ($N$ is the number of simulations, $T_{min}$ is the minimum length of the time series, $T_{max}$ is the maximum length of the time series, $h$ is the maximum forecasting horizon, and $L$ is the loss function)
1. Generation of artificial data \[X=\{\},Y=\{\} \]
For $n=1,...,N$:
1.a. Sample parameters from the prior distribution
\[\theta^n \sim p(\theta)\]
1.b. Sample states conditionally on the parameters \[s_0^n \sim p(s_0|\theta^n )\] \[s_t^n \sim p(s|s_{t-1}^n, \theta^n ),\quad t=1,..., T_{max}\]
1.c. Sample observable variables conditionally on the parameters and states
\[y_t^n \sim p(y|s_t^n, \theta^n), \quad t=1,..., T_{max}\]
1.d. For $\tau=T_{min},..., T_{max}-1$, create data \[x^{n,\tau}=\{y_1^n,...,y_\tau^n \}\] \[y^{n,\tau}=\{y_{\tau+1}^n,...,y_{\tau+min(h,T_{max}-\tau)}^n\}\footnote{A subset of the variables can be chosen as $y$ instead of the full set of observed variables. This may be especially useful when the dimension of the vector of observed variables is large.}\]
and add them to the dataset \[X=X+\{x^{n,\tau} \} \] \[Y=Y+\{y^{n,\tau}\} \]
2. Neural network training \[\varphi^* = argmin_\varphi \sum_{x^i \in X, y^i \in Y} L(y^i, f_\varphi(x^i))\]
In this paper, we take the cross-entropy with a diagonal normal distribution as a loss function, which allows us to correctly estimate the mean and standard deviation of the forecasts, however, as noted by 21, any M-estimator (see Chapter 5 in 33) can be used. The architecture of the neural network is similar to the architecture for the DSGE model of 21 and consists of convolutional, recurrent, and fully connected layers.\footnote{Four types of one-dimensional convolutional layers with 16 filters each and lengths of 3, 5, 7, and 9, respectively, a two-layer GRU block with a dimension of 64, and three fully connected layers with an intermediate dimension of 100 and ReLU activation.} The three key differences are 1) the use of one-directional recurrent blocks to avoid 'looking ahead', 2) the dimensionality of the final outputs, which in our case is equal to the dimensionality of $y$, and 3) the addition of skip connection between the inputs and outputs (see 19).
The proposed algorithm has the property of amortization, i.e., after training, the neural network can be used for any datasets without running computationally complex procedures such as particle filter (see 15). Instead, the data are fed into the neural network as inputs, and the desired characteristics of the forecast distribution (the mean and standard deviation in our case) are obtained as outputs within hundredths or tenths of a second.
The main difference between conditional and unconditional forecasting is the presence of a scenario. To account for the presence of a scenario in Algorithm 1, we modify feature matrix $x$ by adding scenario variables $z$ to it, assuming that they can be expressed in terms of the observed variables. In this case, the prediction algorithm is written as:
Algorithm 2. Amortized neural network algorithm for conditional forecasting ($N$ is the number of simulations, $T_{min}$ is the minimum length of the time series, $T_{max}$ is the maximum length of the time series, $h$ is the maximum forecasting horizon, and $L$ is the loss function)
1. Generation of artificial data \[X=\{\},Y=\{\} \]
For $n=1,..., N:$
1.a. Sample parameters from the prior distribution \[\theta^n \sim p(\theta) \]
1.b. Sample states conditionally on the parameters \[s_0^n \sim p(s_0|\theta^n )\] \[s_t^n \sim p(s|s_{t-1}^n, \theta^n ),\quad t=1,..., T_{max}\]
1.c. Sample observable variables conditionally on the parameters and states \[y_t^n \sim p(y|s_t^n, \theta^n), \quad t=1,..., T_{max}\]
1.d. For $\tau=T_{min},..., T_{max}-1$, create data \[x^{n,\tau}=\{[y_1^n,z_1^n(y_1^n, y_2^n,...,...,y_{1+min(h, T_{max}-1)}^n)],...,[y_\tau^n,z_\tau^n(y_1^n,..., y_\tau^n, y_{\tau+ 1}^n,...,y_{\tau+min(h, T_{max}-\tau)}^n)]\}\] \[y^{n,\tau}=\{y_{\tau+1}^n,...,y_{\tau+min(h,T_{max}-\tau)}^n\} \]
and add them to the dataset \[X=X+\{x^{n,\tau} \} \] \[Y=Y+\{y^{n,\tau}\} \]
2. Neural network training \[\varphi^* = argmin_\varphi \sum_{x^i \in X, y^i \in Y} L(y^i, f_\varphi(x^i))\]
The architecture of the neural network for Algorithm 2 is similar to the architecture of the neural network for Algorithm 1, except that scenario variables are added as inputs at the level of the recurrent layer. Note that the presence of scenarios formulated in terms of the observed variables allows the input of data at the level of the recurrent or convolutional layers, because it is necessary to know the scenarios in the past periods for the correct training of a neural network with such an architecture. In order to incorporate scenarios that potentially include unobserved variables (for example, to calculate impulse responses that depend on the history of observations) in the neural network architecture, they must be fed into the input of the neural network in layers after which knowledge of the scenarios in the other time periods is not required. For the architecture proposed in Section 2.1, this can be done in the fully connected layers.
In spite of the fact that this neural network architecture is more general in terms of the number of tasks to be solved, through additional experiments (see Appendix C), we find that such networks are often much more difficult to train and require a more flexible architecture in the fully connected layers. Therefore, we recommend using Algorithm 2 for models in which the scenario is based on the observable variables only.
The algorithms proposed in the last section should converge exactly to the mean and standard deviation of the posterior distribution of forecasts with the number of simulations tending to infinity and with a sufficiently flexible neural network. In practice, however, the number of observations and the size of the neural network are large, but nevertheless finite, and the loss function contains many local optima. All this can lead to the trained neural networks yielding results different from the posterior of the forecasts,\footnote {See the discussion of the quality of various simulation algorithms, including amortized neural networks, in the context of finding posterior distributions of parameters presented by 23} so checking the quality of the resulting approximations is an important step. In this section, we describe two metrics that we use to check the quality of neural network training.
Of course, the best way to answer the question about the quality of the approximation of the characteristics of the posterior forecast distribution is to compare it with the characteristics of the posterior forecast distribution itself. Unfortunately, for most simulation models, and in particular for ABM, which is the focus of this paper, constructing an exact posterior distribution of forecasts is impossible, and approximations of it based on MCMC algorithms (see 1) or sequential Monte Carlo algorithms (SMC; see 4) require computationally complex algorithms with a particle filter (see 15) or variations of it.
Forecast error standardization. In research on probabilistic time series forecasting,\footnote{See 5} the forecasts are often tested by interval calibration or probability integral transforms. These methods cannot be applied directly to a case in which only the mean and the standard deviation of the forecast are estimated, since the probability distribution of the forecasts is not fully specified. However, it can be noted that, similarly to the probability integral transforms, which must match the normal distribution and have zero autocorrelation, the standardized forecast errors (with the mean removed and divided by the standard deviation) should have the same properties (zero mean and unit standard deviation), other than the form of distribution,\footnote{The distribution need not be normal.} for a well-trained model. Thus, we look at the mean and the standard deviation of the standardized forecast errors and the mean and the standard deviation of the product of standardized errors separated by $k$ periods to test the quality. The latter, in fact, is equivalent to testing for autocorrelation, but it does not require the adjustment of the asymptotic distribution due to the finite length of the time series. Thus, there is no worry that the distribution of the correlation estimates will have a mean and standard deviation different from zero and one.
There are two points to note about this quality metric. First, the results of the test described above should be considered one stage of the verification of the quality of the model. Not passing it should serve as a signal of problems with the quality of the neural network. The reverse situation, in which the test is passed, is not a guarantee that the model works well. For example, if the conditional forecasting model is poorly trained and does not take the presence of a scenario into account in any way, but produces only an unconditional forecast, it will pass the test. Second, we cannot apply the standard formal hypothesis testing about the mean and standard deviation, since the drift of the neural network coefficients at non-zero learning rates (see 26) makes a comparable contribution to the distribution of the mean and standard deviation estimates.\footnote{It is probable that the estimates could be improved by applying ensembles of models based on several runs of the training procedure, or by using averaging at different iterations within the same training procedure, but this issue is beyond the scope of this paper.} Therefore, below, we look at these quantities without formal hypothesis testing.
Comparison with the benchmark model. As mentioned above, passing the test on standardized forecast errors is only an indirect confirmation of the quality of the neural network, since, among other things, it can be passed by models that do not take all relevant information into account. To see how well the neural network takes historical information into account, we estimate the lower bound of forecast quality on a test dataset.
Such an estimate can be made with a benchmark model. Note that after optimization, the neural network should have the smallest mean square forecast error (hereinafter, MSFE) for each variable and for each horizon by construction of the loss function. Moreover, the mean log predictive scores (hereinafter, LPS) should be the largest in the class of normal distributions. These two facts allow us to conclude that, no matter which forecasting model we build on the test data, it should not outperform the neural network in terms of MSFE and LPS.
The benchmark model should be chosen based on a balance of flexibility and training time. On the one hand, the more flexible the model, the tighter the lower bound will be. On the other hand, estimating the lower bound should take adequate time. A representative test dataset often contains thousands or tens of thousands of time series sets, so it is necessary to train the model hundreds of thousands of times (the number of test sets multiplied by the number of periods in which forecasts are made) to test one forecast horizon for one variable.
In this paper, vector autoregression (VAR) with ridge regularization is chosen as a benchmark model. It is computationally easy enough to estimate and allows the calculation of both conditional and unconditional predictions in adequate time. The VAR model in the experiments is put in as comfortable a setting as possible, making it more difficult to pass the test. We start predicting only from the 101st period to give the VAR model more information to train the coefficients (re-estimated recurrently on an expanding window) and also optimize the number of lags and the regularization parameter on the test data over the grid.
In this section, we show how the proposed algorithm works when making forecasts in the ABM. However, before moving on to the ABM, where the comparison with MCMC is time consuming, we show in Section 4.1 how our procedure and quality metrics behave using the toy example of a Bayesian AR(1) regression. Then, in Sections 4.2 and 4.3, we show the results for conditional and unconditional forecasts in the ABM.
We use the Bayesian AR(1) model (see Appendix B.1 for details) to test the proposed algorithm. We generate 1,010,000 time series (10,000 are taken as the test dataset) with a length of 200 to estimate the model. We choose $T_{min}=50$ because we find a slight degradation in quality compared to MCMC for models with $T_{min}$ and $T_{max}$ differing by an order of magnitude. At the same time, time-series lengths from 50 to 200 correspond to quarterly data from 12.5 to 50 years, which is in line with the series lengths used by macroeconomists in practice. The final training dataset for each of the forecasting horizons is approximately 150,000,000 examples (${\sim}N(T_{max}-T_{min} )$) and 1,500,000 for the test. The model is trained with the ADAM algorithm (see 22) 500,000 iterations with a batch size of 100 and learning rate $\varepsilon_n$:
The training takes about 5 hours on an NVIDIA GeForce RTX 2070 GPU for a neural network implemented in PyTorch (see 29).
Figure 1 (see Appendix A) presents examples of the forecasts of the neural network and predictions using the MCMC algorithm on randomly generated data (not used in training) with sample lengths of 50, 100, 150, and 200 on horizons from 1 to 12. It can be seen from the figure that the forecasts are well approximated by the neural network in all cases: autocorrelation is close to zero (upper graph), intermediate autocorrelation (two middle graphs) and high persistence (lower graph).
Figures 2 and 3 show how the errors and the products of the forecast errors are distributed for different forecast horizons on the test data. The means and standard deviations are also shown in the titles of the figures. The neural network approximating the posterior distribution of the forecasts of the Bayesian AR(1) model passes the standardized error test, since for all distributions, the mean and standard deviation are close to 0 and 1. Note that the error distributions are close to normal, but as mentioned above, this is not a necessary condition for our verification\footnote{As we have found, this is not the case for unconditional prediction in the ABM (graphs are not presented for the sake of space).} in contrast to the probability integral transforms.
Table 1 compares the MSFE and LPS neural network errors with respect to the AR(1) model, which is a data generation process for each time series. It can be observed that the neural network statistically significantly outperforms the AR(1) model on all forecast horizons. This may seem counterintuitive, but there is nothing strange in this, since the Bayesian model knows a little more information. In particular, it knows information about how the parameters are distributed over the test datasets in addition to the data generation process.
To summarize, we can say that the neural network copes well with our toy example. The characteristics of the posterior distributions of the forecasts almost coincide with the characteristics of the forecasts based on the MCMC algorithm, which is the main measure of the quality of training. Moreover, this neural network easily passes the indirect quality tests, which are the main tests in our work with the ABM, as expected.
To demonstrate the properties of the proposed algorithm for the ABM, we build a simplified version of the model of 8, which contains 50 C-firms, 500 consumers, the public sector, and a bank (see Appendix B.2). In all examples for unconditional and conditional forecasts, we generate 510,000 datasets (10,000 of which are the test data) of length 300, 100 of which are removed as a burn-in period. This is the minimum number of simulations to prevent overfitting in a model with four observable variables (see below) without regularization as we have seen from the experiments. Of course, regularization procedures such as early stopping, L1/L2-regularization, or dropout\footnote{See 31} can be used to reduce the number of simulations, but the selection of the correct regularization and its hyperparameters lies outside the scope of this paper.
In the case of the ABM, contrast to AR(1), the data simulations take most of the time compared to the training of the neural network. We vectorize our model to speed up the simulation procedure. This allows us to generate 10,000 simulations simultaneously in a single notebook in Python. We run 5 Python notebooks in parallel, which allows us to generate 50,000 simulations in about 3 hours on a CPU (Intel(R) Core (TM) i7-8750H CPU @ 2.20GHz, 16GB RAM). As a result, the final dataset is generated in 30 hours. The neural network training for each of the three sets of variables described below takes about 5 hours and is run with similar hyperparameters to the Bayesian AR(1) model.
We estimate the model with two, three, and four observed variables: \[X_2=\{10 log(1+P), 10U\}\] \[X_3=\{10 log(1+P), 10U, 10 log(1+C) \}\] \[X_4=\{10 log(1+P), 10U, 10 log(1+C),log(1+L) \}\] where $X_2$, $X_3$, and $X_4$ are datasets of two, three, and four variables, respectively, $P$ is the average price level of the C-firms, $U$ is the unemployment rate, $C$ is the consumption level, and $L$ is the volume of loans. The price, consumption, and credit variables are transformed using a transformation that is close to a log transformation,\footnote{Log transformation is usually applied to these variables in macroeconomic forecasting.} with a modification that allows us to train the model without additional filtering for degenerate cases (when the values in the data are zero). The data are also scaled so that the standard deviations of the variables in the individual datasets are of the order of 1 on average.
Table 2 presents the means and standard deviations of the forecast error distributions for models trained on datasets of two, three, and four variables. It can be seen that, in all cases, the means are close to 0 and that the standard deviations are close to 1, which means that the trained neural networks pass the first part of the test proposed in Section 3. Deviations of a few hundredths are caused by a small oscillation of the coefficients at learning rates that do not tend to zero, as is mentioned in the description of the test. When calculating the standard deviations of the product of the standardized forecast errors, we truncate the sample by removing points that are greater than 30 in absolute value (about 100 of the 1,500,000 test points). This is done because the standard deviations of the product of the errors are the fourth moments of the error distribution, and the presence of outliers can significantly affect the estimates. These outliers are associated with very extreme single simulations for unemployment and loans. Examples of such simulations are presented in Figure 4. Excepting such cases, Table 3 shows that the characteristics are close to ideal and differ by no more than a few hundredths.
The neural network approximating the ABM demonstrates good predictive properties compared to the benchmark VAR model. Table 4 presents a comparison in terms of MSFE and LPS. It can be seen that, for all models and forecast horizons, the neural network is not (statistically significantly) worse than the VAR model. Moreover, it almost always (in 212 of 216 cases) outperforms the benchmark at the 1% significance level. Together with the results of the previous test, this allows us to hope for a reasonably good approximation of the posterior of the ABM forecasts. Examples of forecasts on randomly generated data for the ABM with four variables are presented in Figure 5.
The model of Section 4.2 is used to illustrate the properties of neural networks in an amortizing conditional forecast task in the ABM. We train a neural network to predict consumption for 12 periods, assuming that unemployment is known over the forecast horizon.\footnote{The variables are transformed in the same way as in Section 4.2.} These variables are strongly correlated in our model (the average absolute correlation is greater than 0.5), so unemployment is quite informative with respect to consumption, and the forecast should be very different from the unconditional forecast. This allows us to clearly show how Algorithm 2 works.
Figure 6 presents an example of running a trained neural network on the same random data for consumption and unemployment as in Figure 5. It demonstrates that the results of the conditional forecast are much more accurate and narrower than those of the unconditional forecasts from the model with four variables. This serves as indirect evidence that the neural network is well-trained. This is also signalled by the quality metrics proposed in Section 3. Figures 7 and 8 and the values of the mean and standard deviation of the distributions (which are close to 0 and 1) for the forecast errors and their products show that the neural network passes the test on standardized forecast errors.
As before, we estimate a VAR as the benchmark model for the second test from Section 3. The conditional forecasts from the VAR are constructed using Kalman smoothing (see 9). Table 5 shows that the neural network outperforms the VAR model on all forecast horizons. Like the results described in the previous paragraph, this is also indirect evidence of the quality of the neural network.
In this section, we discuss issues that we consider important in the context of the future application and development of the algorithms described. In particular, these are questions that are related to 1) the estimation of the performance of the algorithms based on indirect metrics, 2) scenario forecasting tasks which take into account scenarios built on unobserved variables, 3) the use of an algorithm on microdata, and 4) working with a fixed computational budget on simulations.
Use of indirect quality metrics. The results of the previous section allow us to conclude that the proposed amortized forecasting procedure based on neural networks demonstrates adequate results. Nevertheless, the quality estimates for both the conditional and unconditional forecasts are based on indirect metrics. Although this cuts off inadequate results, it cannot serve as a 100% guarantee of a good approximation of the posterior distribution of forecasts. The benchmark model serves only as a lower bound on the quality of the forecast and may not be tight enough. Verification based on standardized forecast errors has problems similar to the validation of Bayesian model parameters (see 6; 32), in the sense that this test can be passed even by models whose distribution characteristics are far from those of the posterior distribution.\footnote{For example, the prior distribution can pass this test.} Moreover, in the case of non-zero learning rates, the latter procedure is not formally strong due to the fact that it is not based on statistical tests. As mentioned in Section 3, the errors introduced by the oscillation of the coefficients are comparable to the statistical error or even exceed it. The use of indirect metrics creates the risk that the predictions of underfitted models (which differ from the posterior ones) will be accepted as true predictions from the ABM, which may affect decisions made based on these predictions.\footnote{Note that such risks are associated not only with our algorithm, but are also applicable to many other approximate forecasting methods in structural models.} We therefore consider the improvement of forecast quality testing procedures as one of the most important directions for the further development of forecasting with ABMs. Among other things, we see great potential for testing amortized algorithms in formalizing tests on standardized forecast errors and building tests for the dependence of forecast errors on previous data.
Scenario forecasting based on unobservable variables. We also believe, despite the fact that we fail to achieve convergence to the optimum of the algorithm for scenario analysis, which depends not only on the observable variables, but also on unobservable ones (see Appendix C), that future research will find neural network architectures and learning algorithms that are more successful and help solve a wider range of problems than only unconditional and conditional forecasting.
Application of the algorithm to microdata. In addition to estimation on aggregate data, when working with ABMs in practice, it is often necessary to work with models that are fitted to historical microdata or their distributions (see 20). Algorithms 1 and 2 described in Section 3 can also be used for such problems by adding appropriate variables to the set of observables. These are the microdata themselves for the case of microdata,\footnote{Modifications to the structure of the neural network that perform aggregation operations in the first layers or that take the structure of the data into account more specifically, such as the relationship graph structure (see 11), may be needed to reduce the time and improve the quality.} and sufficient statistics (if any) or a small set of characteristics describing the properties of the distribution in the case of distributions.\footnote{See, for example, 3 for a representation of the dynamics of distributions through a set of characteristics in the context of functional VAR models.}
Fixed budget for simulations. Simulation time in AMBs can be large, and a single simulation can take several minutes. This can render the time required to generate the necessary amount of artificial data prohibitive. There are many multi-round methods in the SBI literature that work under such constraints (see 28, 24, 16). We suggest using the idea of the rectangular truncation of the prior distribution of parameters from 27. It can be used to extend our approach by focusing on simulations in regions in which the parameters are more or less consistent with the observed data. Despite the fact that, like other algorithms, this approach does not have the property of amortization, it is nevertheless locally amortized within the truncated regions. Explicit amortization boundaries make it possible to estimate local properties using the quality metrics described above. Moreover, as for other algorithms, local amortization in the parameter space allows us to hope that when new data (new points in the time dimension) arrive, the old simulations can be reused, since the posterior distribution of the parameters usually does not change much when several new points are added.
In this paper, we describe a forecasting algorithm for ABMs based on amortized neural networks, which allows for the almost instantaneous estimation of the characteristics of the posterior distribution of forecasts after pre-training. The experimental results for Bayesian AR(1) and verification based on indirect quality metrics for ABM show that the algorithm demonstrates good properties. We hope that our work will serve as a starting point for the development of forecasting procedures in ABMs based on neural networks, and also help to make forecasting in ABMs a routine procedure in the future.
\nocite{*}