Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
44,481 characters · 20 sections · 7 citation commands
tempdisagg: A Python Framework for Temporal Disaggregation of Time Series Data
{Keywords: temporal disaggregation; econometric modeling; statistical interpolation; ensemble learning; open source software; machine learning}
tempdisagg is a modern, production-ready Python package for temporal disaggregation of time series—designed to transform low-frequency data (e.g., yearly or quarterly statistics) into consistent and statistically sound high-frequency estimates (e.g., monthly or weekly). It bridges the persistent gap between the availability of official aggregate statistics and the growing demand for granular, timely insights in economic monitoring, policy design, and operational forecasting.
Inspired by the well-known R package tempdisagg by sax2013tempdisagg, this Python implementation reimagines the methodology within a clean, extensible, and modular framework. It supports a wide range of classical approaches, including regression-based methods such as chow1971 and litterman1983, smoothing-based techniques such as Denton denton1971 and fernandez1981, as well as uniform interpolation. In addition, it introduces enhanced variants that automatically estimate key model parameters - most notably the autocorrelation coefficient \(\rho\) - using maximum likelihood or residual minimization strategies quilis2018, improving both robustness and predictive accuracy.
tempdisagg goes beyond traditional implementations by introducing a set of powerful enhancements. These include automatic handling of missing subperiods, optional imputation of missing target values via a dedicated Retropolarizer module, post-estimation adjustments to correct for negative predictions, and a flexible ensemble prediction engine that optimally combines multiple models into a single disaggregated estimate. Each component is designed with practical use in mind, offering flexibility without sacrificing methodological rigor.
From an architectural perspective, the library is organized around modular components that encapsulate distinct stages of the disaggregation process: input validation and interpolation, aggregation matrix construction, model fitting, post-processing, and visualization. Its API is deliberately modeled after scikit-learn, providing familiar methods such as fit(), predict(), adjust_output(), summary(), and plot(), ensuring smooth integration into existing data science pipelines.
The library is aimed at a broad audience, including economists, statisticians, data scientists, and public institutions, and can be used for tasks ranging from national accounts reconciliation and macroeconomic forecasting to teaching and applied research. Extensive documentation, illustrative notebooks, and real-world examples are provided to accelerate adoption and ensure reproducibility.
In summary, tempdisagg offers a comprehensive and extensible solution for temporal disaggregation in Python. By combining classical econometric techniques with modern features like automatic parameter tuning, ensemble learning, post-estimation correction, and regression-based imputation, it enables high-quality granular estimation from coarse data while remaining fully transparent, reproducible, and open source under the MIT license.
Key Contributions. This paper introduces tempdisagg, a comprehensive Python temporal disaggregation library, with the following key contributions:
The tempdisagg framework follows a structured modular methodology for temporal disaggregation, systematically converting low-frequency time series into high-frequency estimates while maintaining statistical coherence and consistency. The process is organized into a sequence of well-defined steps, each addressing a key component of the modeling pipeline.
The process begins with a rigorous validation of the input data to ensure structural integrity. The data set must contain four essential columns: Index, which identifies the low-frequency grouping (e.g., year or quarter); Grain, which identifies the high-frequency subperiod within each group (e.g., month or quarter number); y, the low-frequency target variable; and X, the high-frequency indicator variable. Any structural inconsistencies are flagged early to prevent downstream issues. To be compatible with TempDisModel, the input must be formatted as a long format DataFrame, with one row per high-frequency observation. The required columns are summarized in Table (ref).
An example of a correctly formatted input DataFrame is shown in Table (ref).
After validation, the framework fills in any missing high-frequency sub-periods through robust interpolation. This ensures that all combinations of Index and Grain are present, which is essential to construct a complete and coherent time series. Interpolation methods such as linear, nearest-neighbor, or spline can be used depending on the characteristics of the data. The goal is to produce a structurally complete dataset suitable for modeling.
Once the data are complete, an aggregation matrix \( C \) is constructed to represent the mapping from high-frequency estimates to low-frequency aggregates. This matrix plays a central role in ensuring internal consistency throughout the modeling process. The user can specify the aggregation rule and the matrix is dynamically built to reflect the relationship between observations.
The supported aggregation rules and corresponding matrix structures are as follows:
This aggregation matrix is then used to constrain model predictions, ensuring that disaggregated estimates remain consistent with the original low-frequency data.
The tempdisagg framework supports a comprehensive suite of temporal disaggregation methods, which include both classical econometric approaches and modern computational enhancements. Each method takes as input a univariate low-frequency series \( y_l \) and a high-frequency indicator series \( X \), producing a high-frequency estimate \( y \) that respects the aggregation constraint defined by a matrix \( C \), such that \( C y \approx y_l \).
Currently, all models are univariate, meaning that only one explanatory variable is used per disaggregation task. Despite this limitation, the library offers a wide range of disaggregation strategies, including regression-based, smoothing-based, and distribution-based methods. The choice of method depends on the availability and quality of the indicators, the trade-off between smoothness and adherence to \( X \), and assumptions about the underlying data-generating process, such as autocorrelation or prior structures. Below is a summary of the main models implemented:
All models are implemented in a modular fashion within the ModelsHandler class. When applicable, they return not only the high-frequency prediction \( \hat{y} \), but also the estimated parameters \( \hat{\beta} \), autocorrelation \( \hat{\rho} \), residuals, and associated covariance structures (\( Q \), vcov). Table (ref) summarizes the key characteristics and computational principles of each method.
In regression-based temporal disaggregation models such as Chow-Lin and Litterman, the autocorrelation parameter \( \rho \) plays a central role in capturing the serial dependence structure of model residuals. An inappropriate or arbitrary choice of \( \rho \) can lead to biased estimates or poor aggregation consistency. To address this, tempdisagg implements a dedicated RhoOptimizer class that automatically estimates the optimal value of \( \rho \) from the data. The optimizer supports two methods for estimating \( \rho \):
Optimization is performed using scalar bounded minimization over a user-defined interval, typically \([-0.9, 0.99]\). Internally, the algorithm:
Once the optimal \( \rho \) is found, the method returns not only the point estimates for \( \rho \), \( \beta \), and the residuals, but also the full inference results, including:
This module is fully integrated into the regression-based methods of the framework (e.g., chow-lin-opt, litterman-opt), providing a robust and transparent approach to autocorrelation estimation that improves both the statistical validity and the practical usefulness of the resulting disaggregated series.
In scenarios where no single disaggregation method performs consistently well in all datasets or applications, ensemble modeling offers a powerful alternative. The tempdisagg framework integrates an ensemble prediction module that combines the outputs of multiple disaggregation models into a single unified estimate. This ensemble is designed to capitalize on the complementary strengths of individual models while mitigating their weaknesses.
The ensemble is constructed through a constrained nonnegative least squares (NNLS) optimization, which assigns a weight \( w_i \) to each model's high-frequency prediction \( \hat{y}_i \), so that the aggregated prediction best approximates the observed low-frequency totals \( y_l \). Specifically, the optimization problem is defined as follows:
\[ \min_{w \geq 0} \left\| C \left( \sum_{i=1}^{M} w_i \hat{y}_i \right) - y_l \right\|^2 \quad \text{subject to} \quad \sum_{i=1}^{M} w_i = 1 \]
where \( C \) is the aggregation matrix and \( M \) is the number of models being combined. The non-negativity constraint ensures interpretability (no negative contribution from any model), while the sum-to-one constraint ensures that the ensemble remains a convex combination of valid predictions.
This process is managed by the EnsemblePredictor class, which orchestrates the following steps:
The ensemble model is compatible with the larger tempdisagg architecture and can be treated as a standard model object. It includes attributes for accessing the ensemble prediction, weights, constituent models, and inference results. Moreover, it supports exporting to a compatible model interface via to_model(), enabling seamless use in validation and reporting. In practice, the ensemble approach improves the robustness and reliability of temporal disaggregation, especially in heterogeneous or noisy environments. The ensemble prediction strategy uses nonnegative least squares (NNLS) to compute optimal weights for combining individual model outputs, under the constraint that weights are non-negative and sum to one. This approach provides a more robust alternative to simple averaging, avoiding negative contributions and improving interpretability by highlighting the relative importance of each method.
A critical step in practical temporal disaggregation is ensuring that the high-frequency predictions remain non-negative, particularly relevant for variables such as population, GDP, or sales, where negative values are not interpretable. Although most of the disaggregation methods implemented in tempdisagg are statistically rigorous, some may still yield negative values, especially under noisy or sparse indicators. To address this, the framework includes a dedicated post-estimation utility class, PostEstimation, which adjusts predicted values to enforce non-negativity while preserving internal consistency with the selected aggregation rule.
PostEstimation operates after model predictions and supports all major aggregation types: sum, average, first, and last. For each low-frequency group with negative predictions, the method applies one of the following correction strategies:
Each correction is applied group-wise based on the low-frequency identifier (e.g., year or quarter), and the adjusted values replace the original predictions in a copy of the DataFrame. Importantly, these corrections are designed to avoid large distortions in the overall distribution while ensuring that the aggregation constraint (e.g. \(\sum y = y_l\)) is respected. This post-estimation step adds robustness to the modeling pipeline and ensures that the outputs of tempdisagg are not only statistically sound, but also practical and interpretable in applied settings. The summary of these procedures can be seen in Table (ref).
An additional feature of tempdisagg is the ability to estimate missing values in the low-frequency target variable \( y_l \) before disaggregation, through a module known as the Retropolarizer. This component is designed to address situations where recent or historical values of \( y_l \) are not available but are required for temporal disaggregation, particularly in real-time forecasting, incomplete administrative records, or data reporting delays.
The Retropolarizer operates by inferring missing values in \( y_l \) using the available high-frequency indicator \( X \) (or an auxiliary predictor if specified). It supports a range of inference techniques tailored to the characteristics of the data:
The choice of method can be explicitly specified by the user or automatically selected on the basis of the completeness or variability of the data. Once completed, the augmented low-frequency series \( y_l \) is seamlessly passed to the disaggregation workflow. If activated, the system checks for missing values in \( y_l \), applies the selected reconstruction method, and logs the operation. This ensures that disaggregation remains feasible even when partial data is supplied, increasing the usability of the framework in forecasting and nowcasting applications.
To illustrate the basic functionality of tempdisagg, we used the macrodata dataset available in the statsmodels library. This data set contains quarterly macroeconomic indicators for the United States from 1959Q1 to 2009Q3. Specifically, we simulated a temporal disaggregation scenario in which the real Gross Domestic Product (GDP) was artificially aggregated to an annual frequency and then disaggregated back to quarterly estimates using real consumption as the high-frequency indicator. The annual GDP values were computed as the mean of the quarterly GDP observations for each year. These aggregated totals served as the low-frequency target variable (y), while the original quarterly consumption series (realcons) was used as the high-frequency indicator (X). The resulting data set was then passed to \texttt{tempdisagg}, demonstrating the model's ability to recover coherent quarterly estimates that align with the original annual aggregates. As shown in Figure (ref), the disaggregated estimates closely follow the quarterly dynamics implied by the high-frequency indicator.
To test the performance of tempdisagg on real high-frequency data, we used the Industrial Production Index (INDPRO) of the US Federal Reserve Economic Data (FRED) as the monthly indicator from 1947M1 to 2024M12. The INDPRO series reflects real output from the manufacturing, mining, and utilities sectors and is widely used as a proxy for economic activity at high temporal resolutions.
In this case, we simulated a low-frequency annual series by aggregating the monthly INDPRO values using a summation rule, which served as the target variable for disaggregation. The original monthly INDPRO values were retained as the high-frequency indicator. This setting allowed us to evaluate how well tempdisagg could recover monthly-level signals that are consistent with the aggregated annual totals.
The model was applied using the chow-lin-opt method with the sum conversion rule, and the resulting estimates showed strong alignment with the underlying monthly patterns of the indicator. As shown in Figure (ref), the temporal disaggregation of annual GDP growth into monthly estimates generally follows the underlying trend implied by the indicator. However, the model has reduced the accuracy in capturing abrupt or atypical fluctuations in economic growth. These deviations highlight a known limitation of classical disaggregation methods such as Chow-Lin, which tend to smooth out extreme values and may underrepresent sharp inflections or economic shocks at higher frequencies.
To evaluate the performance of the proposed framework using population data, we applied tempdisagg to the official projections of Colombia's population. The disaggregation task consisted of transforming annual national totals into monthly estimates per department. Specifically, we used the chow-lin-opt method with the average aggregation rule, using official national monthly totals as a high-frequency indicator. The model generated monthly estimates for each department (dep_population) which were then aggregated back to the annual level. These reconstructed values were compared with the official annual figures (population) to assess the precision of the disaggregation technique.
We computed standard error metrics for each department, namely, mean absolute error (MAE), root mean squared error (RMSE), and mean squared error (MSE), and averaged the results across all departments. The average MAE was 107.61, suggesting that, on average, departmental aggregates deviated by approximately 108 individuals per year from the official national totals. The average RMSE reached 537.76, highlighting the presence of more substantial deviations in certain departments. These results indicate that tempdisagg is capable of producing consistent and accurate subnational estimates from national-level population data, even under real-world conditions. However, caution is advised when interpreting extreme cases, as traditional disaggregation methods may struggle with highly volatile or irregular series. A detailed summary of these results in terms of the three metrics is provided in Table (ref).
To further assess the behavior of temporal disaggregation methods in different software ecosystems, we compared the results obtained with tempdisagg against those produced by the R package tempdisagg sax2013tempdisagg. Both frameworks were configured to use the chow-lin-opt method with an average aggregation rule, using the same input dataset and high-frequency indicator. The disaggregated estimates generated by each library were highly similar throughout most of the series, with both approaches producing nearly identical monthly population values when reaggregated to the annual level.
However, a key divergence emerged at the boundaries of the time series, particularly in the last year, because of differences in how each library handles incomplete periods and automatic padding. While the R implementation of tempdisagg trims or excludes periods that do not form complete low-frequency groups, the Python framework explicitly allows for padding and interpolation of partial subperiods via the TimeSeriesCompleter component. As a result, Python estimates preserve continuity by filling in missing months at the end of the series, potentially improving applicability in real-time or near-term forecasting. However, this flexibility introduces slight discrepancies in the final period compared to the R output, as the interpolation step affects the structure of the aggregation matrix and thus the final disaggregated estimates. Overall, both implementations are consistent in their core estimation logic and provide statistically robust disaggregation results. However, users should be aware of the default assumptions that each library makes regarding series completeness, padding, and structural interpolation, as these choices can subtly affect the resulting high-frequency predictions, especially at the edges of the data.
Currently, tempdisagg supports only univariate temporal disaggregation methods, which rely on a single high-frequency indicator series. Although this design ensures simplicity and interpretability, it limits the capacity to capture complex dynamics that may require multiple explanatory variables. Support for multivariate disaggregation—incorporating several high-frequency indicators simultaneously—is planned as a key future enhancement.
Furthermore, classical disaggregation models, such as Chow-Lin, Denton, and Litterman, assume relatively smooth behavior in the underlying high-frequency structure. When the target series exhibits abrupt changes, discontinuities, or extreme volatility - common in projections for certain regions or administrative units - these models may struggle to produce stable and realistic estimates. This sensitivity to irregularity can result in extreme values or local inconsistencies, particularly in sparse or noisy settings.
Future work will explore the integration of robust statistical techniques, regularization, and machine learning models to better handle such challenging cases. Extensions to support hierarchical reconciliation, uncertainty quantification, and scenario-based projections are also being considered to improve the framework's applicability across diverse domains and data environments.
The tempdisagg package provides a comprehensive and versatile solution for temporal disaggregation in Python, delivering an accessible and production-ready toolkit for researchers, statisticians, economists, and data analysts. Its modular design, rigorous validation framework, and user-friendly API ensure high usability, transparency, and reproducibility, effectively bridging the gap between traditional econometric practices and modern data science workflows.
Beyond replicating classical methods like Chow-Lin, Denton, Fernández, and Litterman, the package introduces ensemble modeling capabilities and post-estimation adjustments, allowing users to improve robustness and respect real-world constraints such as non-negativity and aggregation consistency. These features make tempdisagg particularly valuable for applications in national statistics, policy evaluation, and economic forecasting.
By combining theoretical rigor with software engineering best practices, tempdisagg serves as both a practical tool and an extensible research platform. Future developments may include support for multivariate disaggregation, integration with state-space models, and Bayesian estimation approaches, further extending its applicability in time-series analysis.
Thanks to the R package tempdisagg for their methodological foundation. The appreciation also goes to the open source community for tools such as NumPy, SciPy, and Pandas.
This project did not receive external funding. The author has no competing interests to declare.